Prediction intervals for a noisy nonlinear time series based on a bootstrapping reservoir computing network ensemble

Hierarchical Granular Computing-Based Model and Its Reinforcement Structural Learning for Construction of Long-Term Prediction Intervals

As one of the most essential sources of energy, byproduct gas plays a pivotal role in the steel industry, for which the flow tendency is generally regarded as the guidance for planning and scheduling in real production. In order to obtain the numeric estimation along with its reliability, the construction of prediction intervals (PIs) is highly demanded by any practical applications as well as being long term for providing more information on future trends. Bearing this in mind, in this article, a hierarchical granular computing (HGrC)-based model is established for constructing long-term PIs, in which probabilistic modeling gives rise to a long horizon of numeric prediction, and the deployment of information granularities hierarchically extends the result to be interval-valued format. Considering that the structure of this model has a direct impact on its performance, Monte-Carlo search with a policy gradient technique is then applied for reinforcement structure learning. Compared with the existing methods, the size (length) of the granules in the proposed approach is unequal so that it becomes effective for not only periodic but also nonperiodic data. Furthermore, with the use of parallel strategy, the efficiency can be also guaranteed for real-world applications. The experimental results demonstrate that the proposed method is superior to other commonly encountered techniques, and the stability of the structure learning process behaves better when compared with other reinforcement learning approaches.

Granular Prediction and Dynamic Scheduling Based on Adaptive Dynamic Programming for the Blast Furnace Gas System

A timely and effective scheduling of the byproduct gas system plays a pivotal role in realizing intelligent manufacturing and energy conservation in the steel industry. In order to realize real-time dynamic scheduling of the blast furnace gas (BFG) system, a granular prediction and dynamic scheduling process based on adaptive dynamic programming is proposed in this paper. To reflect the specificity of production reflected in the fluctuation of data, a series of information granules is constructed and described. In the dynamic scheduling phase, based on the granular feature description, a scheduling action network is established and further updates of information granules are realized. Considering a slow adjustment process and delay characteristics of the BFG system, the cumulative reward of the critic network is calculated on the basis of the data partition to construct a tendency attenuation-based cost function. In order to determine the future trends of the gas tank level that targets real-time determination of the scheduling moment, a reinforcement learning-based granulation and prediction process is also proposed. To demonstrate the performance of the proposed method, a number of comparative experiments are presented by using the practical industrial data. The results indicate that the proposed method exhibits high accuracy and can deliver an effective solution to justified scheduling of the BFG system.

Ensemble Stochastic Configuration Networks for Estimating Prediction Intervals: A Simultaneous Robust Training Algorithm and Its Application

Obtaining accurate point prediction of industrial processes' key variables is challenging due to the outliers and noise that are common in industrial data. Hence the prediction intervals (PIs) have been widely adopted to quantify the uncertainty related to the point prediction. In order to improve the prediction accuracy and quantify the level of uncertainty associated with the point prediction, this article estimates the PIs by using ensemble stochastic configuration networks (SCNs) and bootstrap method. The estimated PIs can guarantee both the modeling stability and computational efficiency. To encourage the cooperation among the base SCNs and improve the robustness of the ensemble SCNs when the training data are contaminated with noise and outliers, a simultaneous robust training method of the ensemble SCNs is developed based on the Bayesian ridge regression and M-estimate. Moreover, the hyperparameters of the assumed distributions over noise and output weights of the ensemble SCNs are estimated by the expectation-maximization (EM) algorithm, which can result in the optimal PIs and better prediction accuracy. Finally, the performance of the proposed approach is evaluated on three benchmark data sets and a real-world data set collected from a refinery. The experimental results demonstrate that the proposed approach exhibits better performance in terms of the quality of PIs, prediction accuracy, and robustness.

Solar Power Interval Prediction via Lower and Upper Bound Estimation with a New Model Initialization Approach

This paper proposes a new model initialization approach for solar power prediction interval based on the lower and upper bound estimation (LUBE) structure. The linear regression interval estimation (LRIE) was first used to initialize the prediction interval and the extreme learning machine auto encoder (ELM-AE) is then employed to initialize the input weight matrix of the LUBE. Based on the initialized prediction interval and input weight matrix, the output weight matrix of the LUBE could be obtained, which was close to optimal values. The heuristic algorithm was employed to train the LUBE prediction model due to the invalidation of the traditional training approach. The proposed model initialization approach was compared with the point prediction initialization and random initialization approaches. To validate its performance, four heuristic algorithms, including particle swarm optimization (PSO), simulated annealing (SA), harmony search (HS), and differential evolution (DE), were introduced. Based on the experiment results, the proposed model initialization approach with different heuristic algorithms was better than the point prediction initialization and random initialization approaches. The PSO can obtain the best efficiency and effectiveness of the optimal solution searching in four heuristic algorithms. Besides, the ELM-AE can weaken the over-fitting phenomenon of the training model, which is brought in by the heuristic algorithm, and guarantee the model stable output.

Robust manifold broad learning system for large-scale noisy chaotic time series prediction: A perturbation perspective

Noises and outliers commonly exist in dynamical systems because of sensor disturbations or extreme dynamics. Thus, the robustness and generalization capacity are of vital importance for system modeling. In this paper, the robust manifold broad learning system(RM-BLS) is proposed for system modeling and large-scale noisy chaotic time series prediction. Manifold embedding is utilized for chaotic system evolution discovery. The manifold representation is randomly corrupted by perturbations while the features not related to low-dimensional manifold embedding are discarded by feature selection. It leads to a robust learning paradigm and achieves better generalization performance. We also develop an efficient solution for Stiefel manifold optimization, in which the orthogonal constraints are maintained by Cayley transformation and curvilinear search algorithm. Furthermore, we discuss the common thoughts between random perturbation approximation and other mainstream regularization methods. We also prove the equivalence between perturbations to manifold embedding and Tikhonov regularization. Simulation results on large-scale noisy chaotic time series prediction illustrates the robustness and generalization performance of our method.

Landslide Displacement Prediction With Uncertainty Based on Neural Networks With Random Hidden Weights

In this paper, we propose a new approach to establish a landslide displacement forecasting model based on artificial neural networks (ANNs) with random hidden weights. To quantify the uncertainty associated with the predictions, a framework for probabilistic forecasting of landslide displacement is developed. The aim of this paper is to construct prediction intervals (PIs) instead of deterministic forecasting. A lower-upper bound estimation (LUBE) method is adopted to construct ANN-based PIs, while a new single hidden layer feedforward ANN with random hidden weights for LUBE is proposed. Unlike the original implementation of LUBE, the input weights and hidden biases of the ANN are randomly chosen, and only the output weights need to be adjusted. Combining particle swarm optimization (PSO) and gravitational search algorithm (GSA), a hybrid evolutionary algorithm, PSOGSA, is utilized to optimize the output weights. Furthermore, a new ANN objective function, which combines a modified combinational coverage width-based criterion with one-norm regularization, is proposed. Two benchmark data sets and two real-world landslide data sets are presented to illustrate the capability and merit of our method. Experimental results reveal that the proposed method can construct high-quality PIs.

Two Efficient Twin ELM Methods With Prediction Interval

In the operational optimization and scheduling problems of actual industrial processes, such as iron and steel, and microelectronics, the operational indices and process parameters usually need to be predicted. However, for some input and output variables of these prediction models, there may exist a lot of uncertainties coming from themselves, the measurement error, the rough representation, and so on. In such cases, constructing a prediction interval (PI) for the output of the corresponding prediction model is very necessary. In this paper, two twin extreme learning machine (TELM) models for constructing PIs are proposed. First, we propose a regularized asymmetric least squares extreme learning machine (RALS-ELM) method, in which different weights of its squared error loss function are set according to whether the error of the model output is positive or negative in order that the above error can be differentiated in the parameter learning process, and Tikhonov regularization is introduced to reduce overfitting. Then, we propose an asymmetric Bayesian extreme learning machine (AB-ELM) method based on the Bayesian framework with the asymmetric Gaussian distribution (AB-ELM), in which the weights of its likelihood function are determined as the same method in RALS-ELM, and the type II maximum likelihood algorithm is derived to learn the parameters of AB-ELM. Based on RALS-ELM and AB-ELM, we use a pair of weights following the reciprocal relationship to obtain two nonparallel regressors, including a lower-bound regressor and an upper-bound regressor, respectively, which can be used for calculating the PIs. Finally, some discussions are given, about how to adjust the weights adaptively to meet the desired PI, how to use the proposed TELMs for nonlinear quantile regression, and so on. Results of numerical comparison on data from one synthetic regression problem, three University of California Irvine benchmark regression problems, and two actual industrial regression problems show the effectiveness of the proposed models.

Short-term load and wind power forecasting using neural network-based prediction intervals

Electrical power systems are evolving from today's centralized bulk systems to more decentralized systems. Penetrations of renewable energies, such as wind and solar power, significantly increase the level of uncertainty in power systems. Accurate load forecasting becomes more complex, yet more important for management of power systems. Traditional methods for generating point forecasts of load demands cannot properly handle uncertainties in system operations. To quantify potential uncertainties associated with forecasts, this paper implements a neural network (NN)-based method for the construction of prediction intervals (PIs). A newly introduced method, called lower upper bound estimation (LUBE), is applied and extended to develop PIs using NN models. A new problem formulation is proposed, which translates the primary multiobjective problem into a constrained single-objective problem. Compared with the cost function, this new formulation is closer to the primary problem and has fewer parameters. Particle swarm optimization (PSO) integrated with the mutation operator is used to solve the problem. Electrical demands from Singapore and New South Wales (Australia), as well as wind power generation from Capital Wind Farm, are used to validate the PSO-based LUBE method. Comparative results show that the proposed method can construct higher quality PIs for load and wind power generation forecasts in a short time.