ELITE: ensemble of optimal input-pruned neural networks using TRUST-TECH

Trust-Tech Source-Point Method for Systematically Computing Multiple Local Optimal Solutions: Theory and Method

In this article, a Trust-Tech source-point method is proposed to systematically compute multiple local optimal solutions (LOSs) for continuous unconstrained nonlinear optimization problems. This proposed method consists of four stages. Stage I finds one LOS (in which existing effective optimizers can be applied), stage II is the stage of escaping an LOS while stage III is the stage for entering the stability region (SR) of another stable equilibrium point (SEP) (i.e., another LOS). Stage IV computes other SEPs (i.e., LOSs) in corresponding SRs. A theoretical foundation for both stages II and III is developed, and these theoretical results are quite general on their own. The proposed method is numerically evaluated to compute multiple LOSs. For instance, a total of 5085 LOSs have been computed by the proposed Trust-Tech source point method on a 50-D test function. In addition, the proposed method can find the global optimal solutions of several test functions with 50 dimensions and 100 dimensions.

A Novel Consensus-Based Particle Swarm Optimization-Assisted Trust-Tech Methodology for Large-Scale Global Optimization

A novel three-stage methodology, termed the "consensus-based particle swarm optimization (PSO)-assisted Trust-Tech methodology," to find global optimal solutions for nonlinear optimization problems is presented. It is composed of Trust-Tech methods, consensus-based PSO, and local optimization methods that are integrated to compute a set of high-quality local optimal solutions that can contain the global optimal solution. The proposed methodology compares very favorably with several recently developed PSO algorithms based on a set of small-dimension benchmark optimization problems and 20 large-dimension test functions from the CEC 2010 competition. The analytical basis for the proposed methodology is also provided. Experimental results demonstrate that the proposed methodology can rapidly obtain high-quality optimal solutions that can contain the global optimal solution. The scalability of the proposed methodology is promising.

Multimodal Degradation Prognostics Based on Switching Kalman Filter Ensemble

For accurate prognostics, users have to determine the current health of the system and predict future degradation pattern of the system. An increasingly popular approach toward tackling prognostic problems involves the use of switching models to represent various degradation phases, which the system undergoes. Such approaches have the advantage of determining the exact degradation phase of the system and being able to handle nonlinear degradation models through piecewise linear approximation. However, limitations of such existing methods include, limited applicability due to the discretization of predicted remaining useful life, insufficient robustness due to the use of single models and others. This paper circumvents these limitations by proposing a hybrid of ensemble methods with switching methods. The proposed method first implements a switching Kalman filter (SKF) to classify between various linear degradation phases, then predict the future propagation of fault dimension using appropriate Kalman filters for each phase. This proposed method achieves both continuous and discrete prediction values representing the remaining life and degradation phase of the system, respectively. The proposed framework is shown via a case study on benchmark simulated aeroengine data sets. The evaluation of the proposed framework shows that the proposed method achieves better accuracy and robustness against noise compared with other methods reported in the literature. The results also indicate the effectiveness of the SKF in detecting the switching point between various degradation modes.

FREL: A Stable Feature Selection Algorithm

Two factors characterize a good feature selection algorithm: its accuracy and stability. This paper aims at introducing a new approach to stable feature selection algorithms. The innovation of this paper centers on a class of stable feature selection algorithms called feature weighting as regularized energy-based learning (FREL). Stability properties of FREL using L1 or L2 regularization are investigated. In addition, as a commonly adopted implementation strategy for enhanced stability, an ensemble FREL is proposed. A stability bound for the ensemble FREL is also presented. Our experiments using open source real microarray data, which are challenging high dimensionality small sample size problems demonstrate that our proposed ensemble FREL is not only stable but also achieves better or comparable accuracy than some other popular stable feature weighting methods.

Global sensitivity analysis approach for input selection and system identification purposes--a new framework for feedforward neural networks

A new algorithm for the selection of input variables of neural network is proposed. This new method, applied after the training stage, ranks the inputs according to their importance in the variance of the model output. The use of a global sensitivity analysis technique, extended Fourier amplitude sensitivity test, gives the total sensitivity index for each variable, which allows for the ranking and the removal of the less relevant inputs. Applied to some benchmarking problems in the field of features selection, the proposed approach shows good agreement in keeping the relevant variables. This new method is a useful tool for removing superfluous inputs and for system identification.

SSC: a classifier combination method based on signal strength

We propose a new classifier combination method, the signal strength-based combining (SSC) approach, to combine the outputs of multiple classifiers to support the decision-making process in classification tasks. As ensemble learning methods have attracted growing attention from both academia and industry recently, it is critical to understand the fundamental issues of the combining rule. Motivated by the signal strength concept, our proposed SSC algorithm can effectively integrate the individual vote from different classifiers in an ensemble learning system. Comparative studies of our method with nine major existing combining rules, namely, geometric average rule, arithmetic average rule, median value rule, majority voting rule, Borda count, max and min rule, weighted average, and weighted majority voting rules, is presented. Furthermore, we also discuss the relationship of the proposed method with respect to margin-based classifiers, including the boosting method (AdaBoost.M1 and AdaBoost.M2) and support vector machines by margin analysis. Detailed analyses of margin distribution graphs are presented to discuss the characteristics of the proposed method. Simulation results for various real-world datasets illustrate the effectiveness of the proposed method.