Robust Recurrent Kernel Online Learning

Online Kernel Learning With Adaptive Bandwidth by Optimal Control Approach

Online learning methods are designed to establish timely predictive models for machine learning problems. The methods for online learning of nonlinear systems are usually developed in the reproducing kernel Hilbert space (RKHS) associated with Gaussian kernel in which the kernel bandwidth is manually selected and remains steady during the entire modeling process in most cases. This setting may make the learning model rigid and inappropriate for complex data streams. Since the bandwidth appears in a nonlinear term of the kernel model, it raises substantial challenges in the development of learning methods with an adaptive bandwidth. In this article, we propose a novel approach to address this important open issue. By a carefully casted linearization scheme, the nonlinear learning problem is reasonably transformed into a state feedback control problem for a series of controllable systems. Then, by employing optimal control techniques, an effective algorithm is developed, and the parameters in the learning model including kernel bandwidth can be efficiently updated in a real-time manner. By taking advantage of the particular structure of the Gaussian kernel model, a theoretical analysis on the convergence and rationality of the proposed method is also provided. Compared with the kernel algorithms with a fixed bandwidth, our novel learning framework can not only achieve adaptive learning results with a better prediction accuracy but also show performance that is more robust with a faster convergence speed. Encouraging numerical results are provided to demonstrate the advantages of our new method.

Spatio-Temporal Interpolated Echo State Network for Meteorological Series Prediction

Spatio-temporal series prediction has attracted increasing attention in the field of meteorology in recent years. The spatial and temporal joint effect makes predictions challenging. Most of the existing spatio-temporal prediction models are computationally complicated. To develop an accurate but easy-to-implement spatio-temporal prediction model, this paper designs a novel spatio-temporal prediction model based on echo state networks. For real-world observed meteorological data with randomness and large changes, we use a cubic spline method to bridge the gaps between the neighboring points, which results in a pleasingly smooth series. The interpolated series is later input into the spatio-temporal echo state networks, in which the spatial coefficients are computed by the elastic-net algorithm. This approach offers automatic selection and continuous shrinkage of the spatial variables. The proposed model provides an intuitive but effective approach to address the interaction of spatial and temporal effects. To demonstrate the practicality of the proposed model, we apply it to predict two real-world datasets: monthly precipitation series and daily air quality index series. Experimental results demonstrate that the proposed model achieves a normalized root-mean-square error of approximately 0.250 on both datasets. Similar results are achieved on the long short-term memory model, but the computation time of our proposed model is considerably shorter. It can be inferred that our proposed neural network model has advantages on predicting meteorological series over other models.

Multivariate Chaotic Time Series Online Prediction Based on Improved Kernel Recursive Least Squares Algorithm

Kernel recursive least squares (KRLS) is a kind of kernel methods, which has attracted wide attention in the research of time series online prediction. It has low computational complexity and updates in a recursive form. However, as data size increases, computational complexity of calculating kernel inverse matrix will raise. And it has some difficulties in accommodating time-varying environments. Therefore, we have presented an improved KRLS algorithm for multivariate chaotic time series online prediction. Approximate linear dependency, dynamic adjustment, and coherence criterion are combined with quantization to form our improved KRLS algorithm. In the process of online prediction, it can bring computational efficiency up and adjust weights adaptively in time-varying environments. Moreover, Lorenz chaotic time series, El Nino-Southern Oscillation indexes chaotic time series, yearly sunspots and runoff of the Yellow River chaotic time series online prediction are presented to prove the effectiveness of our proposed algorithm.