Discrete-Time Signatures and Randomness in Reservoir Computing

Universality of reservoir systems with recurrent neural networks

Approximation capability of reservoir systems whose reservoir is a recurrent neural network (RNN) is discussed. We show what we call uniform strong universality of RNN reservoir systems for a certain class of dynamical systems. This means that, given an approximation error to be achieved, one can construct an RNN reservoir system that approximates each target dynamical system in the class just via adjusting its linear readout. To show the universality, we construct an RNN reservoir system via parallel concatenation that has an upper bound of approximation error independent of each target in the class.

Complexities of feature-based learning systems, with application to reservoir computing

This paper studies complexity measures of reservoir systems. For this purpose, a more general model that we call a feature-based learning system, which is the composition of a feature map and of a final estimator, is studied. We study complexity measures such as growth function, VC-dimension, pseudo-dimension and Rademacher complexity. On the basis of the results, we discuss how the unadjustability of reservoirs and the linearity of readouts can affect complexity measures of the reservoir systems. Furthermore, some of the results generalize or improve the existing results.

Infinite-dimensional reservoir computing

Reservoir computing approximation and generalization bounds are proved for a new concept class of input/output systems that extends the so-called generalized Barron functionals to a dynamic context. This new class is characterized by the readouts with a certain integral representation built on infinite-dimensional state-space systems. It is shown that this class is very rich and possesses useful features and universal approximation properties. The reservoir architectures used for the approximation and estimation of elements in the new class are randomly generated echo state networks with either linear or ReLU activation functions. Their readouts are built using randomly generated neural networks in which only the output layer is trained (extreme learning machines or random feature neural networks). The results in the paper yield a recurrent neural network-based learning algorithm with provable convergence guarantees that do not suffer from the curse of dimensionality when learning input/output systems in the class of generalized Barron functionals and measuring the error in a mean-squared sense.

Quantum reservoir computing in finite dimensions

Most existing results in the analysis of quantum reservoir computing (QRC) systems with classical inputs have been obtained using the density matrix formalism. This paper shows that alternative representations can provide better insight when dealing with design and assessment questions. More explicitly, system isomorphisms are established that unify the density matrix approach to QRC with the representation in the space of observables using Bloch vectors associated with Gell-Mann bases. It is shown that these vector representations yield state-affine systems previously introduced in the classical reservoir computing literature and for which numerous theoretical results have been established. This connection is used to show that various statements in relation to the fading memory property (FMP) and the echo state property (ESP) are independent of the representation and also to shed some light on fundamental questions in QRC theory in finite dimensions. In particular, a necessary and sufficient condition for the ESP and FMP to hold is formulated using standard hypotheses, and contractive quantum channels that have exclusively trivial semi-infinite solutions are characterized in terms of the existence of input-independent fixed points.