Predicting chaotic dynamics from incomplete input via reservoir computing with (D+1)-dimension input and output

Chaotic time series classification by means of reservoir-based convolutional neural networks

We propose a novel Reservoir Computing (RC) based classification method that distinguishes between different chaotic time series. Our method is composed of two steps: (i) we use the reservoir as a feature extracting machine that captures the salient features of time series data; (ii) the readout layer of the reservoir is subsequently fed into a Convolutional Neural Network (CNN) to facilitate classification and recognition. One of the notable advantages is that the readout layer, as obtained by randomly generated empirical hyper-parameters within the RC module, provides sufficient information for the CNN to accomplish the classification tasks effectively. The quality of extracted features by RC is independently evaluated by the root mean square error, which measures how well the training signal may be reconstructed from the input time series. Furthermore, we propose two ways to implement the RC module, namely, a single shallow RC and parallel RC configurations, to further improve the classification accuracy. The important roles of RC in feature extraction are demonstrated by comparing the results when the CNN is provided with either ordinal pattern probability features or unprocessed raw time series directly, both of which perform worse than RC-based method. In addition to CNN, we show that the readout of RC is good for other classification tools as well. The successful classification of electroencephalogram recordings of different brain states suggests that our RC-based classification tools can be used for experimental studies.

Learning noise-induced transitions by multi-scaling reservoir computing

Noise is usually regarded as adversarial to extracting effective dynamics from time series, such that conventional approaches usually aim at learning dynamics by mitigating the noisy effect. However, noise can have a functional role in driving transitions between stable states underlying many stochastic dynamics. We find that leveraging a machine learning model, reservoir computing, can learn noise-induced transitions. We propose a concise training protocol with a focus on a pivotal hyperparameter controlling the time scale. The approach is widely applicable, including a bistable system with white noise or colored noise, where it generates accurate statistics of transition time for white noise and specific transition time for colored noise. Instead, the conventional approaches such as SINDy and the recurrent neural network do not faithfully capture stochastic transitions even for the case of white noise. The present approach is also aware of asymmetry of the bistable potential, rotational dynamics caused by non-detailed balance, and transitions in multi-stable systems. For the experimental data of protein folding, it learns statistics of transition time between folded states, enabling us to characterize transition dynamics from a small dataset. The results portend the exploration of extending the prevailing approaches in learning dynamics from noisy time series.

Predicting nonsmooth chaotic dynamics by reservoir computing

Reservoir computing (RC) has been widely applied to predict the chaotic dynamics in many systems. Yet much broader areas related to nonsmooth dynamics have seldom been touched by the RC community which have great theoretical and practical importance. The generalization of RC to this kind of system is reported in this paper. The numerical work shows that the conventional RC with a hyperbolic tangent activation function is not able to predict the dynamics of nonsmooth systems very well, especially when reconstructing attractors (long-term prediction). A nonsmooth activation function with a piecewise nature is proposed. A kind of physics-informed RC scheme is established based on this activation function. The feasibility of this scheme has been proven by its successful application to the predictions of the short- and long-term (reconstructing chaotic attractor) dynamics of four nonsmooth systems with different complexity, including the tent map, piecewise linear map with a gap, both noninvertible and discontinuous compound circle maps, and Lozi map. The results show that RC with the new activation function is efficient and easy to run. It can make perfectly both short- and long-term predictions. The precision of reconstructing attractors depends on their complexity. This work reveals that, to make efficient predictions, the activation function of an RC approach should match the smooth or nonsmooth nature of the dynamical systems.