A Kernel-Based Two-Class Classifier for Imbalanced Data Sets

INTUITIONISTIC FUZZY REASONING FOR MULTIPLE TWO-CLASS CLASSIFIERS FUSION

Combining outputs of a pool of individual classifiers appropriately, as a hot research topic of pattern classification, can generate statistically significant increase in classification performances. During the last decades, several fusion algorithms were presented, but few of those focus on two-class classification which possesses wide application area such as sentiment classification, cancer differentiation and so on. Thus the main purpose of this paper is to develop a highly effective fusion algorithm, i.e. intuitionistic fuzzy reasoning fusion algorithm, to increase the performance of a multiple two-class classifiers system. The outputs of component classifiers are represented by a set of intuitionistic fuzzy values at first and the fusion process is interpreted as aggregation of intuitionistic fuzzy information. The proposed algorithm includes three versions using the intuitionistic fuzzy arithmetic average operator, intuitionistic fuzzy weighted average operator and induced intuitionistic fuzzy ordered weighted average aggregation operator, respectively. The proposed fusion algorithm can combine both evidences of the hypothesis that a test pattern belongs to a class and evidences of the hypothesis that the test pattern does not belong to the class at the same time. We compare the versions of our algorithm with four other fusion techniques on five open-access datasets. The proposed algorithm exhibits good predictive abilities, compared to the best individual classifiers and other comparable fusion techniques. Further, the experiments show some interesting results about measuring and weighting component classifiers.

RAMOBoost: Ranked Minority Oversampling in Boosting

In recent years, learning from imbalanced data has attracted growing attention from both academia and industry due to the explosive growth of applications that use and produce imbalanced data. However, because of the complex characteristics of imbalanced data, many real-world solutions struggle to provide robust efficiency in learning-based applications. In an effort to address this problem, this paper presents Ranked Minority Oversampling in Boosting (RAMOBoost), which is a RAMO technique based on the idea of adaptive synthetic data generation in an ensemble learning system. Briefly, RAMOBoost adaptively ranks minority class instances at each learning iteration according to a sampling probability distribution that is based on the underlying data distribution, and can adaptively shift the decision boundary toward difficult-to-learn minority and majority class instances by using a hypothesis assessment procedure. Simulation analysis on 19 real-world datasets assessed over various metrics-including overall accuracy, precision, recall, F-measure, G-mean, and receiver operation characteristic analysis-is used to illustrate the effectiveness of this method.

Kernel-matching pursuits with arbitrary loss functions

The purpose of this research is to develop a classifier capable of state-of-the-art performance in both computational efficiency and generalization ability while allowing the algorithm designer to choose arbitrary loss functions as appropriate for a give problem domain. This is critical in applications involving heavily imbalanced, noisy, or non-Gaussian distributed data. To achieve this goal, a kernel-matching pursuit (KMP) framework is formulated where the objective is margin maximization rather than the standard error minimization. This approach enables excellent performance and computational savings in the presence of large, imbalanced training data sets and facilitates the development of two general algorithms. These algorithms support the use of arbitrary loss functions allowing the algorithm designer to control the degree to which outliers are penalized and the manner in which non-Gaussian distributed data is handled. Example loss functions are provided and algorithm performance is illustrated in two groups of experimental results. The first group demonstrates that the proposed algorithms perform equivalent to several state-of-the-art machine learning algorithms on well-published, balanced data. The second group of results illustrates superior performance by the proposed algorithms on imbalanced, non-Gaussian data achieved by employing loss functions appropriate for the data characteristics and problem domain.