1
|
Hart JD. Attractor reconstruction with reservoir computers: The effect of the reservoir's conditional Lyapunov exponents on faithful attractor reconstruction. CHAOS (WOODBURY, N.Y.) 2024; 34:043123. [PMID: 38579149 DOI: 10.1063/5.0196257] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/05/2024] [Accepted: 03/22/2024] [Indexed: 04/07/2024]
Abstract
Reservoir computing is a machine learning framework that has been shown to be able to replicate the chaotic attractor, including the fractal dimension and the entire Lyapunov spectrum, of the dynamical system on which it is trained. We quantitatively relate the generalized synchronization dynamics of a driven reservoir during the training stage to the performance of the trained reservoir computer at the attractor reconstruction task. We show that, in order to obtain successful attractor reconstruction and Lyapunov spectrum estimation, the maximal conditional Lyapunov exponent of the driven reservoir must be significantly more negative than the most negative Lyapunov exponent of the target system. We also find that the maximal conditional Lyapunov exponent of the reservoir depends strongly on the spectral radius of the reservoir adjacency matrix; therefore, for attractor reconstruction and Lyapunov spectrum estimation, small spectral radius reservoir computers perform better in general. Our arguments are supported by numerical examples on well-known chaotic systems.
Collapse
Affiliation(s)
- Joseph D Hart
- U.S. Naval Research Laboratory, Code 5675, Washington, DC 20375, USA
| |
Collapse
|
2
|
Harding S, Leishman Q, Lunceford W, Passey DJ, Pool T, Webb B. Global forecasts in reservoir computers. CHAOS (WOODBURY, N.Y.) 2024; 34:023136. [PMID: 38407397 DOI: 10.1063/5.0181694] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/17/2023] [Accepted: 01/24/2024] [Indexed: 02/27/2024]
Abstract
A reservoir computer is a machine learning model that can be used to predict the future state(s) of time-dependent processes, e.g., dynamical systems. In practice, data in the form of an input-signal are fed into the reservoir. The trained reservoir is then used to predict the future state of this signal. We develop a new method for not only predicting the future dynamics of the input-signal but also the future dynamics starting at an arbitrary initial condition of a system. The systems we consider are the Lorenz, Rossler, and Thomas systems restricted to their attractors. This method, which creates a global forecast, still uses only a single input-signal to train the reservoir but breaks the signal into many smaller windowed signals. We examine how well this windowed method is able to forecast the dynamics of a system starting at an arbitrary point on a system's attractor and compare this to the standard method without windows. We find that the standard method has almost no ability to forecast anything but the original input-signal while the windowed method can capture the dynamics starting at most points on an attractor with significant accuracy.
Collapse
Affiliation(s)
- S Harding
- Mathematics Department, Brigham Young University, Provo, Utah 84602, USA
| | - Q Leishman
- Mathematics Department, Brigham Young University, Provo, Utah 84602, USA
| | - W Lunceford
- Mathematics Department, Brigham Young University, Provo, Utah 84602, USA
| | - D J Passey
- Mathematics Department, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina 27599, USA
| | - T Pool
- The Robotics Institute, Carnegie Mellon University, Pittsburg, Pennsylvania 15289, USA
| | - B Webb
- Mathematics Department, Brigham Young University, Provo, Utah 84602, USA
| |
Collapse
|
3
|
Li X, Small M, Lei Y. Reservoir computing with higher-order interactive coupled pendulums. Phys Rev E 2023; 108:064304. [PMID: 38243442 DOI: 10.1103/physreve.108.064304] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/28/2023] [Accepted: 11/28/2023] [Indexed: 01/21/2024]
Abstract
The reservoir computing approach utilizes a time series of measurements as input to a high-dimensional dynamical system known as a reservoir. However, the approach relies on sampling a random matrix to define its underlying reservoir layer, which leads to numerous hyperparameters that need to be optimized. Here, we propose a nonlocally coupled pendulum model with higher-order interactions as a novel reservoir, which requires no random underlying matrices and fewer hyperparameters. We use Bayesian optimization to explore the hyperparameter space within a minimal number of iterations and train the coupled pendulums model to reproduce the chaotic attractors, which simplifies complicated hyperparameter optimization. We illustrate the effectiveness of our technique with the Lorenz system and the Hindmarsh-Rose neuronal model, and we calculate the Pearson correlation coefficients between time series and the Hausdorff metrics in the phase space. We demonstrate the contribution of higher-order interactions by analyzing the interaction between different reservoir configurations and prediction performance, as well as computations of the largest Lyapunov exponents. The chimera state is found as the most effective dynamical regime for prediction. The findings, where we present a new reservoir structure, offer potential applications in the design of high-performance modeling of dynamics in physical systems.
Collapse
Affiliation(s)
- Xueqi Li
- School of Mathematics and Statistics, Northwestern Polytechnical University, Xi'an 710072, China
| | - Michael Small
- Complex Systems Group, Department of Mathematics and Statistics, The University of Western Australia, Crawley WA 6009, Australia
- Mineral Resources, CSIRO, Kensington WA 6151, Australia
| | - Youming Lei
- School of Mathematics and Statistics, Northwestern Polytechnical University, Xi'an 710072, China
- Ministry of Industry and Information Technology Key Laboratory of Dynamics and Control of Complex Systems, Northwestern Polytechnical University, Xi'an 710072, China
| |
Collapse
|
4
|
Platt JA, Penny SG, Smith TA, Chen TC, Abarbanel HDI. Constraining chaos: Enforcing dynamical invariants in the training of reservoir computers. CHAOS (WOODBURY, N.Y.) 2023; 33:103107. [PMID: 37788385 DOI: 10.1063/5.0156999] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/04/2023] [Accepted: 08/14/2023] [Indexed: 10/05/2023]
Abstract
Drawing on ergodic theory, we introduce a novel training method for machine learning based forecasting methods for chaotic dynamical systems. The training enforces dynamical invariants-such as the Lyapunov exponent spectrum and the fractal dimension-in the systems of interest, enabling longer and more stable forecasts when operating with limited data. The technique is demonstrated in detail using reservoir computing, a specific kind of recurrent neural network. Results are given for the Lorenz 1996 chaotic dynamical system and a spectral quasi-geostrophic model of the atmosphere, both typical test cases for numerical weather prediction.
Collapse
Affiliation(s)
- Jason A Platt
- Department of Physics, University of California San Diego, San Diego, California 92093, USA
| | - Stephen G Penny
- Sofar Ocean, 28 Pier Annex, San Francisco, California 94105, USA
- Cooperative Institute for Research in Environmental Sciences, University of Colorado Boulder, Boulder, Colorado 80309, USA
| | - Timothy A Smith
- Cooperative Institute for Research in Environmental Sciences, University of Colorado Boulder, Boulder, Colorado 80309, USA
- Physical Sciences Laboratory, National Oceanic and Atmospheric Administration, Boulder, Colorado 80305, USA
| | - Tse-Chun Chen
- Pacific Northwest National Laboratory, 902 Battelle Blvd, Richland, Washington 99354, USA
| | - Henry D I Abarbanel
- Department of Physics, University of California San Diego, San Diego, California 92093, USA
- Marine Physical Laboratory, Scripps Institution of Oceanography, 9500 Gilman Drive, La Jolla, California 92093, USA
| |
Collapse
|
5
|
Remarks on Fractal-Fractional Malkus Waterwheel Model with Computational Analysis. Symmetry (Basel) 2022. [DOI: 10.3390/sym14102220] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022] Open
Abstract
In this paper, we investigate the fractal-fractional Malkus Waterwheel model in detail. We discuss the existence and uniqueness of a solution of the fractal-fractional model using the fixed point technique. We apply a very effective method to obtain the solutions of the model. We prove with numerical simulations the accuracy of the proposed method. We put in evidence the effects of the fractional order and the fractal dimension for a symmetric Malkus Waterwheel model.
Collapse
|
6
|
Barbosa WAS, Gauthier DJ. Learning spatiotemporal chaos using next-generation reservoir computing. CHAOS (WOODBURY, N.Y.) 2022; 32:093137. [PMID: 36182396 DOI: 10.1063/5.0098707] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/11/2022] [Accepted: 08/30/2022] [Indexed: 06/16/2023]
Abstract
Forecasting the behavior of high-dimensional dynamical systems using machine learning requires efficient methods to learn the underlying physical model. We demonstrate spatiotemporal chaos prediction using a machine learning architecture that, when combined with a next-generation reservoir computer, displays state-of-the-art performance with a computational time 10- 10 times faster for training process and training data set ∼ 10 times smaller than other machine learning algorithms. We also take advantage of the translational symmetry of the model to further reduce the computational cost and training data, each by a factor of ∼10.
Collapse
Affiliation(s)
- Wendson A S Barbosa
- Department of Physics, The Ohio State University, 191 W. Woodruff Ave., Columbus, Ohio 43210, USA
| | - Daniel J Gauthier
- Department of Physics, The Ohio State University, 191 W. Woodruff Ave., Columbus, Ohio 43210, USA
| |
Collapse
|