Optimal reconstruction of dynamical systems: a noise amplification approach

A robust time-delay selection criterion applied to convergent cross mapping

This work presents a heuristic for the selection of a time delay based on optimizing the global maximum of mutual information in orthonormal coordinates for embedding a dynamical system. This criterion is demonstrated to be more robust compared to methods that utilize a local minimum, as the global maximum is guaranteed to exist in the proposed coordinate system for any dynamical system. By contrast, methods using local minima can be ill-posed as a local minimum can be difficult to identify in the presence of noise or may simply not exist. The performance of the global maximum and local minimum methods are compared in the context of causality detection using convergent cross mapping using both a noisy Lorenz system and experimental data from an oscillating plasma source. The proposed heuristic for time lag selection is shown to be more consistent in the presence of noise and closer to an optimal uniform time lag selection.

Estimation of Carleman operator from a univariate time series

Reconstructing a nonlinear dynamical system from empirical time series is a fundamental task in data-driven analysis. One of the main challenges is the existence of hidden variables; we only have records for some variables, and those for hidden variables are unavailable. In this work, the techniques for Carleman linearization, phase-space embedding, and dynamic mode decomposition are integrated to rebuild an optimal dynamical system from time series for one specific variable. Using the Takens theorem, the embedding dimension is determined, which is adopted as the dynamical system's dimension. The Carleman linearization is then used to transform this finite nonlinear system into an infinite linear system, which is further truncated into a finite linear system using the dynamic mode decomposition technique. We illustrate the performance of this integrated technique using data generated by the well-known Lorenz model, the Duffing oscillator, and empirical records of electrocardiogram, electroencephalogram, and measles outbreaks. The results show that this solution accurately estimates the operators of the nonlinear dynamical systems. This work provides a new data-driven method to estimate the Carleman operator of nonlinear dynamical systems.

Model adaptive phase space reconstruction

Phase space reconstruction (PSR) methods allow for the analysis of low-dimensional data with methods from dynamical systems theory, but their application to prediction models, such as those from machine learning (ML), is limited. Therefore, we here present a model adaptive phase space reconstruction (MAPSR) method that unifies the process of PSR with the modeling of the dynamical system. MAPSR is a differentiable PSR based on time-delay embedding and enables ML methods for modeling. The quality of the reconstruction is evaluated by the prediction loss. The discrete-time signal is converted into a continuous-time signal to achieve a loss function, which is differentiable with respect to the embedding delays. The delay vector, which stores all potential embedding delays, is updated along with the trainable parameters of the model to minimize prediction loss. Thus, MAPSR does not rely on any threshold or statistical criterion for determining the dimension and the set of delay values for the embedding process. We apply the MAPSR method to uni- and multivariate time series stemming from chaotic dynamical systems and a turbulent combustor. We find that for the Lorenz system, the model trained with the MAPSR method is able to predict chaotic time series for nearly seven to eight Lyapunov time scales, which is found to be much better compared to other PSR methods [AMI-FNN (average mutual information-false nearest neighbor) and PECUZAL (Pecora-Uzal) methods]. For the univariate time series from the turbulent combustor, the long-term cumulative prediction error of the MAPSR method for the regime of chaos stays between other methods, and for the regime of intermittency, MAPSR outperforms other PSR methods.

Multilayer Perceptron Network Optimization for Chaotic Time Series Modeling

Chaotic time series are widely present in practice, but due to their characteristics-such as internal randomness, nonlinearity, and long-term unpredictability-it is difficult to achieve high-precision intermediate or long-term predictions. Multi-layer perceptron (MLP) networks are an effective tool for chaotic time series modeling. Focusing on chaotic time series modeling, this paper presents a generalized degree of freedom approximation method of MLP. We then obtain its Akachi information criterion, which is designed as the loss function for training, hence developing an overall framework for chaotic time series analysis, including phase space reconstruction, model training, and model selection. To verify the effectiveness of the proposed method, it is applied to two artificial chaotic time series and two real-world chaotic time series. The numerical results show that the proposed optimized method is effective to obtain the best model from a group of candidates. Moreover, the optimized models perform very well in multi-step prediction tasks.

Selecting embedding delays: An overview of embedding techniques and a new method using persistent homology

Delay embedding methods are a staple tool in the field of time series analysis and prediction. However, the selection of embedding parameters can have a big impact on the resulting analysis. This has led to the creation of a large number of methods to optimize the selection of parameters such as embedding lag. This paper aims to provide a comprehensive overview of the fundamentals of embedding theory for readers who are new to the subject. We outline a collection of existing methods for selecting embedding lag in both uniform and non-uniform delay embedding cases. Highlighting the poor dynamical explainability of existing methods of selecting non-uniform lags, we provide an alternative method of selecting embedding lags that includes a mixture of both dynamical and topological arguments. The proposed method, Significant Times on Persistent Strands (SToPS), uses persistent homology to construct a characteristic time spectrum that quantifies the relative dynamical significance of each time lag. We test our method on periodic, chaotic, and fast-slow time series and find that our method performs similar to existing automated non-uniform embedding methods. Additionally, n-step predictors trained on embeddings constructed with SToPS were found to outperform other embedding methods when predicting fast-slow time series.

Stabilizing embedology: Geometry-preserving delay-coordinate maps

Delay-coordinate mapping is an effective and widely used technique for reconstructing and analyzing the dynamics of a nonlinear system based on time-series outputs. The efficacy of delay-coordinate mapping has long been supported by Takens' embedding theorem, which guarantees that delay-coordinate maps use the time-series output to provide a reconstruction of the hidden state space that is a one-to-one embedding of the system's attractor. While this topological guarantee ensures that distinct points in the reconstruction correspond to distinct points in the original state space, it does not characterize the quality of this embedding or illuminate how the specific parameters affect the reconstruction. In this paper, we extend Takens' result by establishing conditions under which delay-coordinate mapping is guaranteed to provide a stable embedding of a system's attractor. Beyond only preserving the attractor topology, a stable embedding preserves the attractor geometry by ensuring that distances between points in the state space are approximately preserved. In particular, we find that delay-coordinate mapping stably embeds an attractor of a dynamical system if the stable rank of the system is large enough to be proportional to the dimension of the attractor. The stable rank reflects the relation between the sampling interval and the number of delays in delay-coordinate mapping. Our theoretical findings give guidance to choosing system parameters, echoing the tradeoff between irrelevancy and redundancy that has been heuristically investigated in the literature. Our initial result is stated for attractors that are smooth submanifolds of Euclidean space, with extensions provided for the case of strange attractors.

Limits to Causal Inference with State-Space Reconstruction for Infectious Disease

Infectious diseases are notorious for their complex dynamics, which make it difficult to fit models to test hypotheses. Methods based on state-space reconstruction have been proposed to infer causal interactions in noisy, nonlinear dynamical systems. These “model-free” methods are collectively known as convergent cross-mapping (CCM). Although CCM has theoretical support, natural systems routinely violate its assumptions. To identify the practical limits of causal inference under CCM, we simulated the dynamics of two pathogen strains with varying interaction strengths. The original method of CCM is extremely sensitive to periodic fluctuations, inferring interactions between independent strains that oscillate with similar frequencies. This sensitivity vanishes with alternative criteria for inferring causality. However, CCM remains sensitive to high levels of process noise and changes to the deterministic attractor. This sensitivity is problematic because it remains challenging to gauge noise and dynamical changes in natural systems, including the quality of reconstructed attractors that underlie cross-mapping. We illustrate these challenges by analyzing time series of reportable childhood infections in New York City and Chicago during the pre-vaccine era. We comment on the statistical and conceptual challenges that currently limit the use of state-space reconstruction in causal inference.

Possibility of short-term probabilistic forecasts for large earthquakes making good use of the limitations of existing catalogs

Earthquakes are quite hard to predict. One of the possible reasons can be the fact that the existing catalogs of past earthquakes are limited at most to the order of 100 years, while their characteristic time scale is sometimes greater than that time span. Here we rather use these limitations positively and characterize some large earthquake events as abnormal events that are not included there. When we constructed probabilistic forecasts for large earthquakes in Japan based on similarity and difference to their past patterns-which we call known and unknown abnormalities, respectively-our forecast achieved probabilistic gains of 5.7 and 2.4 against a time-independent model for main shocks with the magnitudes of 7 or above. Moreover, the two abnormal conditions covered 70% of days whose maximum magnitude was 7 or above.

Optimal state-space reconstruction using derivatives on projected manifold

A paradigm for optimal state-space reconstruction with nonuniform time delays is proposed. A comparison based on a diffeomorphic measure and a smoothness cost function shows that the proposed methodology achieves a better reconstruction compared to a reconstruction based on time delays that are multiples of the first minimum of mutual information. It is also shown how the proposed methodology is a more reliable approach to determining the embedding dimension.