Boosted network classifiers for local feature selection

Adaptive Unsupervised Feature Selection With Structure Regularization

Feature selection is one of the most important dimension reduction techniques for its efficiency and interpretation. Since practical data in large scale are usually collected without labels, and labeling these data are dramatically expensive and time-consuming, unsupervised feature selection has become a ubiquitous and challenging problem. Without label information, the fundamental problem of unsupervised feature selection lies in how to characterize the geometry structure of original feature space and produce a faithful feature subset, which preserves the intrinsic structure accurately. In this paper, we characterize the intrinsic local structure by an adaptive reconstruction graph and simultaneously consider its multiconnected-components (multicluster) structure by imposing a rank constraint on the corresponding Laplacian matrix. To achieve a desirable feature subset, we learn the optimal reconstruction graph and selective matrix simultaneously, instead of using a predetermined graph. We exploit an efficient alternative optimization algorithm to solve the proposed challenging problem, together with the theoretical analyses on its convergence and computational complexity. Finally, extensive experiments on clustering task are conducted over several benchmark data sets to verify the effectiveness and superiority of the proposed unsupervised feature selection algorithm.

RandomBoost: simplified multiclass boosting through randomization

We propose a novel boosting approach to multiclass classification problems, in which multiple classes are distinguished by a set of random projection matrices in essence. The approach uses random projections to alleviate the proliferation of binary classifiers typically required to perform multiclass classification. The result is a multiclass classifier with a single vector-valued parameter, irrespective of the number of classes involved. Two variants of this approach are proposed. The first method randomly projects the original data into new spaces, while the second method randomly projects the outputs of learned weak classifiers. These methods are not only conceptually simple but also effective and easy to implement. A series of experiments on synthetic, machine learning, and visual recognition data sets demonstrate that our proposed methods could be compared favorably with existing multiclass boosting algorithms in terms of both the convergence rate and classification accuracy.

Effective neural network ensemble approach for improving generalization performance

This paper, with an aim at improving neural networks' generalization performance, proposes an effective neural network ensemble approach with two novel ideas. One is to apply neural networks' output sensitivity as a measure to evaluate neural networks' output diversity at the inputs near training samples so as to be able to select diverse individuals from a pool of well-trained neural networks; the other is to employ a learning mechanism to assign complementary weights for the combination of the selected individuals. Experimental results show that the proposed approach could construct a neural network ensemble with better generalization performance than that of each individual in the ensemble combining with all the other individuals, and than that of the ensembles with simply averaged weights.