Scaling KNN multi-class twin support vector machine via safe instance reduction

MTKSVCR: A novel multi-task multi-class support vector machine with safe acceleration rule

Regularized multi-task learning (RMTL) has shown good performance in tackling multi-task binary problems. Although RMTL can be used to handle multi-class problems based on "one-versus-one" and "one-versus-rest" techniques, the information of the samples is not fully utilized and the class imbalance problem occurs. Motivated by the regularization technique in RMTL, we propose an original multi-task multi-class model termed MTKSVCR based on "one-versus-one-versus-rest" strategy to achieve better testing accuracy. Due to the utilization of the idea of RMTL, the related information included in multiple tasks is mined by setting different penalty parameters before task-common and task-specific regularization terms. However, the proposed MTKSVCR is time-consuming since it employs all samples in each optimization problem. Therefore, a multi-parameter safe acceleration rule termed SA is further presented to reduce the time consumption. It identifies and deletes most of the superfluous samples corresponding to 0 elements in the dual optimal solution before solving. Then, only a reduced dual problem is to be solved and the computational efficiency is improved accordingly. The biggest advantage of the proposed SA lies in safety. Namely, it derives an identical optimal solution to the primal problem without SA. In addition, our method remains effective when multiple parameters change simultaneously. Experiments on different artificial datasets and benchmark datasets verify the validity of the proposed methods.

Neighborhood Granule Classifiers

Classifiers are divided into linear and nonlinear classifiers. The linear classifiers are built on a basis of some hyper planes. The nonlinear classifiers are mainly neural networks. In this paper, we propose a novel neighborhood granule classifier based on a concept of granular structure and neighborhood granules of datasets. By introducing a neighborhood rough set model, the condition features and decision features of classification systems are respectively granulated to form some condition neighborhood granules and decision neighborhood granules. These neighborhood granules are sets; thus, their calculations are intersection and union operations of sets. A condition neighborhood granule and a decision neighborhood granule form a granular rule, and the collection of granular rules constitutes a granular rule library. Furthermore, we propose two kinds of distance and similarity metrics to measure granules, which are used for the searching and matching of granules. Thus, we design a granule classifier by the similarity metric. Finally, we use the granule classifier proposed in this paper for a classification test with UCI datasets. The theoretical analysis and experiments show that the proposed granule classifier achieves a better classification performance under an appropriate neighborhood granulation parameter.