Robust support vector regression for uncertain input and output data

A state of health estimation method for electric vehicle Li-ion batteries using GA-PSO-SVR

State of health (SOH) is the ratio of the currently available maximum capacity of the battery to the rated capacity. It is an important index to describe the degradation state of a pure electric vehicle battery and has an important reference value in evaluating the health level of the retired battery and estimating the driving range. In this study, the random forest algorithm is first used to find the most important health factors to lithium-ion batteries based on the dataset released by National Aeronautics and Space Administration (NASA). Then the support vector regression (SVR) algorithm is developed to predict the SOH of a lithium-ion battery. The genetic algorithm-particle swarm optimization (GA-PSO) algorithm is brought forward to optimize the parameter values of the SVR, which could improve the estimation accuracy and convergence speed. The proposed SOH estimation method is applied to four batteries and gets a root mean square error (RMSE) of 0.40% and an average absolute percentage error (MAPE) of 0.56%. In addition, the method is also compared with genetic algorithm-support vector regression (GA-SVR) and particle swarm optimization-support vector regression (PSO-SVR), respectively. The results show that (i) compared with the PSO-SVR method, the proposed method can decrease the average RMSE by 0.10%, and the average MAPE by 0.17%; (ii) compared with the GA-PSO method, number of iterations under the proposed method can be reduced by 7 generations.

Scalable Kernel Ordinal Regression via Doubly Stochastic Gradients

Ordinal regression (OR) is one of the most important machine learning tasks. The kernel method is a major technique to achieve nonlinear OR. However, traditional kernel OR solvers are inefficient due to increased complexity introduced by multiple ordinal thresholds as well as the cost of kernel computation. Doubly stochastic gradient (DSG) is a very efficient and scalable kernel learning algorithm that combines random feature approximation with stochastic functional optimization. However, the theory and algorithm of DSG can only support optimization tasks within the unique reproducing kernel Hilbert space (RKHS), which is not suitable for OR problems where the multiple ordinal thresholds usually lead to multiple RKHSs. To address this problem, we construct a kernel whose RKHS can contain the decision function with multiple thresholds. Based on this new kernel, we further propose a novel DSG-like algorithm, DSGOR. In each iteration of DSGOR, we update the decision functional as well as the function bias with appropriately set learning rates for each. Our theoretic analysis shows that DSGOR can achieve O(1/t) convergence rate, which is as good as DSG, even though dealing with a much harder problem. Extensive experimental results demonstrate that our algorithm is much more efficient than traditional kernel OR solvers, especially on large-scale problems.

Robust Extreme Learning Machine With its Application to Indoor Positioning

The increasing demands of location-based services have spurred the rapid development of indoor positioning system and indoor localization system interchangeably (IPSs). However, the performance of IPSs suffers from noisy measurements. In this paper, two kinds of robust extreme learning machines (RELMs), corresponding to the close-to-mean constraint, and the small-residual constraint, have been proposed to address the issue of noisy measurements in IPSs. Based on whether the feature mapping in extreme learning machine is explicit, we respectively provide random-hidden-nodes and kernelized formulations of RELMs by second order cone programming. Furthermore, the computation of the covariance in feature space is discussed. Simulations and real-world indoor localization experiments are extensively carried out and the results demonstrate that the proposed algorithms can not only improve the accuracy and repeatability, but also reduce the deviation and worst case error of IPSs compared with other baseline algorithms.

Global nonlinear kernel prediction for large data set with a particle swarm-optimized interval support vector regression

A new global nonlinear predictor with a particle swarm-optimized interval support vector regression (PSO-ISVR) is proposed to address three issues (viz., kernel selection, model optimization, kernel method speed) encountered when applying SVR in the presence of large data sets. The novel prediction model can reduce the SVR computing overhead by dividing input space and adaptively selecting the optimized kernel functions to obtain optimal SVR parameter by PSO. To quantify the quality of the predictor, its generalization performance and execution speed are investigated based on statistical learning theory. In addition, experiments using synthetic data as well as the stock volume weighted average price are reported to demonstrate the effectiveness of the developed models. The experimental results show that the proposed PSO-ISVR predictor can improve the computational efficiency and the overall prediction accuracy compared with the results produced by the SVR and other regression methods. The proposed PSO-ISVR provides an important tool for nonlinear regression analysis of big data.

Robust Novelty Detection via Worst Case CVaR Minimization

Novelty detection models aim to find the minimum volume set covering a given probability mass. This paper proposes a robust single-class support vector machine (SSVM) for novelty detection, which is mainly based on the worst case conditional value-at-risk minimization. By assuming that every input is subject to an uncertainty with a specified symmetric support, this robust formulation results in a maximization term that is similar to the regularization term in the classical SSVM. When the uncertainty set is l1 -norm, l∞ -norm or box, its training can be reformulated to a linear program; while the uncertainty set is l2 -norm or ellipsoidal, its training is a tractable second-order cone program. The proposed method has a nice consistent statistical property. As the training size goes to infinity, the estimated normal region converges to the true provided that the magnitude of the uncertainty set decreases in a systematic way. The experimental results on three data sets clearly demonstrate its superiority over three benchmark models.

Incremental Support Vector Learning for Ordinal Regression

Support vector ordinal regression (SVOR) is a popular method to tackle ordinal regression problems. However, until now there were no effective algorithms proposed to address incremental SVOR learning due to the complicated formulations of SVOR. Recently, an interesting accurate on-line algorithm was proposed for training ν -support vector classification (ν-SVC), which can handle a quadratic formulation with a pair of equality constraints. In this paper, we first present a modified SVOR formulation based on a sum-of-margins strategy. The formulation has multiple constraints, and each constraint includes a mixture of an equality and an inequality. Then, we extend the accurate on-line ν-SVC algorithm to the modified formulation, and propose an effective incremental SVOR algorithm. The algorithm can handle a quadratic formulation with multiple constraints, where each constraint is constituted of an equality and an inequality. More importantly, it tackles the conflicts between the equality and inequality constraints. We also provide the finite convergence analysis for the algorithm. Numerical experiments on the several benchmark and real-world data sets show that the incremental algorithm can converge to the optimal solution in a finite number of steps, and is faster than the existing batch and incremental SVOR algorithms. Meanwhile, the modified formulation has better accuracy than the existing incremental SVOR algorithm, and is as accurate as the sum-of-margins based formulation of Shashua and Levin.

Semi-supervised and unsupervised extreme learning machines

Extreme learning machines (ELMs) have proven to be efficient and effective learning mechanisms for pattern classification and regression. However, ELMs are primarily applied to supervised learning problems. Only a few existing research papers have used ELMs to explore unlabeled data. In this paper, we extend ELMs for both semi-supervised and unsupervised tasks based on the manifold regularization, thus greatly expanding the applicability of ELMs. The key advantages of the proposed algorithms are as follows: 1) both the semi-supervised ELM (SS-ELM) and the unsupervised ELM (US-ELM) exhibit learning capability and computational efficiency of ELMs; 2) both algorithms naturally handle multiclass classification or multicluster clustering; and 3) both algorithms are inductive and can handle unseen data at test time directly. Moreover, it is shown in this paper that all the supervised, semi-supervised, and unsupervised ELMs can actually be put into a unified framework. This provides new perspectives for understanding the mechanism of random feature mapping, which is the key concept in ELM theory. Empirical study on a wide range of data sets demonstrates that the proposed algorithms are competitive with the state-of-the-art semi-supervised or unsupervised learning algorithms in terms of accuracy and efficiency.

SVR learning-based spatiotemporal fuzzy logic controller for nonlinear spatially distributed dynamic systems

A data-driven 3-D fuzzy-logic controller (3-D FLC) design methodology based on support vector regression (SVR) learning is developed for nonlinear spatially distributed dynamic systems. Initially, the spatial information expression and processing as well as the fuzzy linguistic expression and rule inference of a 3-D FLC are integrated into spatial fuzzy basis functions (SFBFs), and then the 3-D FLC can be depicted by a three-layer network structure. By relating SFBFs of the 3-D FLC directly to spatial kernel functions of an SVR, an equivalence relationship of the 3-D FLC and the SVR is established, which means that the 3-D FLC can be designed with the help of the SVR learning. Subsequently, for an easy implementation, a systematic SVR learning-based 3-D FLC design scheme is formulated. In addition, the universal approximation capability of the proposed 3-D FLC is presented. Finally, the control of a nonlinear catalytic packed-bed reactor is considered as an application to demonstrate the effectiveness of the proposed 3-D FLC.