1
|
Fang Q, Xiang C, Duan J, Soufiyan B, Shao C, Yang X, Xu S, Yu H. OMAL: A Multi-Label Active Learning Approach from Data Streams. ENTROPY (BASEL, SWITZERLAND) 2025; 27:363. [PMID: 40282598 PMCID: PMC12026165 DOI: 10.3390/e27040363] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/16/2025] [Revised: 03/26/2025] [Accepted: 03/26/2025] [Indexed: 04/29/2025]
Abstract
With the rapid growth of digital computing, communication, and storage devices applied in various real-world scenarios, more and more data have been collected and stored to drive the development of machine learning techniques. It is also noted that the data that emerge in real-world applications tend to become more complex. In this study, we regard a complex data type, i.e., multi-label data, acquired with a time constraint in a dynamic online scenario. Under such conditions, constructing a learning model has to face two challenges: it requires dynamically adapting the variances in label correlations and imbalanced data distributions and it requires more labeling consumptions. To solve these two issues, we propose a novel online multi-label active learning (OMAL) algorithm that considers simultaneously adopting uncertainty (using the average entropy of prediction probabilities) and diversity (using the average cosine distance between feature vectors) as an active query strategy. Specifically, to focus on label correlations, we use a classifier chain (CC) as the multi-label learning model and design a label co-occurrence ranking strategy to arrange label sequence in CC. To adapt the naturally imbalanced distribution of the multi-label data, we select weight extreme learning machine (WELM) as the basic binary-class classifier in CC. The experimental results on ten benchmark multi-label datasets that were transformed into streams show that our proposed method is superior to several popular static multi-label active learning algorithms in terms of both the Macro-F1 and Micro-F1 metrics, indicating its specifical adaptions in the dynamic data stream environment.
Collapse
Affiliation(s)
- Qiao Fang
- School of Computer, Jiangsu University of Science and Technology, Zhenjiang 212100, China; (Q.F.); (C.X.); (J.D.); (B.S.); (C.S.); (X.Y.)
| | - Chen Xiang
- School of Computer, Jiangsu University of Science and Technology, Zhenjiang 212100, China; (Q.F.); (C.X.); (J.D.); (B.S.); (C.S.); (X.Y.)
| | - Jicong Duan
- School of Computer, Jiangsu University of Science and Technology, Zhenjiang 212100, China; (Q.F.); (C.X.); (J.D.); (B.S.); (C.S.); (X.Y.)
| | - Benallal Soufiyan
- School of Computer, Jiangsu University of Science and Technology, Zhenjiang 212100, China; (Q.F.); (C.X.); (J.D.); (B.S.); (C.S.); (X.Y.)
| | - Changbin Shao
- School of Computer, Jiangsu University of Science and Technology, Zhenjiang 212100, China; (Q.F.); (C.X.); (J.D.); (B.S.); (C.S.); (X.Y.)
| | - Xibei Yang
- School of Computer, Jiangsu University of Science and Technology, Zhenjiang 212100, China; (Q.F.); (C.X.); (J.D.); (B.S.); (C.S.); (X.Y.)
| | - Sen Xu
- School of Information Technology, Yancheng Institute of Technology, Yancheng 224051, China;
| | - Hualong Yu
- School of Computer, Jiangsu University of Science and Technology, Zhenjiang 212100, China; (Q.F.); (C.X.); (J.D.); (B.S.); (C.S.); (X.Y.)
| |
Collapse
|
2
|
Zhou S, Gu Y, Yu H, Yang X, Gao S. RUE: A Robust Personalized Cost Assignment Strategy for Class Imbalance Cost-sensitive Learning. JOURNAL OF KING SAUD UNIVERSITY - COMPUTER AND INFORMATION SCIENCES 2023. [DOI: 10.1016/j.jksuci.2023.03.001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 03/08/2023]
|
3
|
Ding H, Sun Y, Huang N, Shen Z, Wang Z, Iftekhar A, Cui X. RVGAN-TL: A generative adversarial networks and transfer learning-based hybrid approach for imbalanced data classification. Inf Sci (N Y) 2023. [DOI: 10.1016/j.ins.2023.01.147] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/05/2023]
|
4
|
Zhang A, Yu H, Zhou S, Huan Z, Yang X. Instance weighted SMOTE by indirectly exploring the data distribution. Knowl Based Syst 2022. [DOI: 10.1016/j.knosys.2022.108919] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
|
5
|
RLMD-PA: A Reinforcement Learning-Based Myocarditis Diagnosis Combined with a Population-Based Algorithm for Pretraining Weights. CONTRAST MEDIA & MOLECULAR IMAGING 2022; 2022:8733632. [PMID: 35833074 PMCID: PMC9262570 DOI: 10.1155/2022/8733632] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/16/2022] [Revised: 04/07/2022] [Accepted: 04/13/2022] [Indexed: 11/17/2022]
Abstract
Myocarditis is heart muscle inflammation that is becoming more prevalent these days, especially with the prevalence of COVID-19. Noninvasive imaging cardiac magnetic resonance (CMR) can be used to diagnose myocarditis, but the interpretation is time-consuming and requires expert physicians. Computer-aided diagnostic systems can facilitate the automatic screening of CMR images for triage. This paper presents an automatic model for myocarditis classification based on a deep reinforcement learning approach called as reinforcement learning-based myocarditis diagnosis combined with population-based algorithm (RLMD-PA) that we evaluated using the Z-Alizadeh Sani myocarditis dataset of CMR images prospectively acquired at Omid Hospital, Tehran. This model addresses the imbalanced classification problem inherent to the CMR dataset and formulates the classification problem as a sequential decision-making process. The policy of architecture is based on convolutional neural network (CNN). To implement this model, we first apply the artificial bee colony (ABC) algorithm to obtain initial values for RLMD-PA weights. Next, the agent receives a sample at each step and classifies it. For each classification act, the agent gets a reward from the environment in which the reward of the minority class is greater than the reward of the majority class. Eventually, the agent finds an optimal policy under the guidance of a particular reward function and a helpful learning environment. Experimental results based on standard performance metrics show that RLMD-PA has achieved high accuracy for myocarditis classification, indicating that the proposed model is suitable for myocarditis diagnosis.
Collapse
|
6
|
DE-PNN: Differential Evolution-Based Feature Optimization with Probabilistic Neural Network for Imbalanced Arrhythmia Classification. SENSORS 2022; 22:s22124450. [PMID: 35746232 PMCID: PMC9227752 DOI: 10.3390/s22124450] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 04/26/2022] [Revised: 05/31/2022] [Accepted: 06/06/2022] [Indexed: 11/17/2022]
Abstract
In this research, a heartbeat classification method is presented based on evolutionary feature optimization using differential evolution (DE) and classification using a probabilistic neural network (PNN) to discriminate between normal and arrhythmic heartbeats. The proposed method follows four steps: (1) preprocessing, (2) heartbeat segmentation, (3) DE feature optimization, and (4) PNN classification. In this method, we have employed direct signal amplitude points constituting the heartbeat acquired from the ECG holter device with no secondary feature extraction step usually used in case of hand-crafted, frequency transformation or other features. The heartbeat types include normal, left bundle branch block, right bundle branch block, premature ventricular contraction, atrial premature, ventricular escape, ventricular flutter and paced beat. Using ECG records from the MIT-BIH, heartbeats are identified to start at 250 ms before and end at 450 ms after the respective R-peak positions. In the next step, the DE method is applied to reduce and optimize the direct heartbeat features. Although complex and highly computational ECG heartbeat classification algorithms have been proposed in the literature, they failed to achieve high performance in detecting some minority heartbeat categories, especially for imbalanced datasets. To overcome this challenge, we propose an optimization step for the deep CNN model using a novel classification metric called the Matthews correlation coefficient (MCC). This function focuses on arrhythmia (minority) heartbeat classes by increasing their importance. Maximum MCC is used as a fitness function to identify the optimum combination of features for the uncorrelated and non-uniformly distributed eight beat class samples. The proposed DE-PNN scheme can provide better classification accuracy considering 8 classes with only 36 features optimized from a 253 element feature set implying an 85.77% reduction in direct amplitude features. Our proposed method achieved overall 99.33% accuracy, 94.56% F1, 93.84% sensitivity, and 99.21% specificity.
Collapse
|
7
|
Few-Shot Learning with Collateral Location Coding and Single-Key Global Spatial Attention for Medical Image Classification. ELECTRONICS 2022. [DOI: 10.3390/electronics11091510] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/04/2023]
Abstract
Humans are born with the ability to learn quickly by discerning objects from a few samples, to acquire new skills in a short period of time, and to make decisions based on limited prior experience and knowledge. The existing deep learning models for medical image classification often rely on a large number of labeled training samples, whereas the fast learning ability of deep neural networks has failed to develop. In addition, it requires a large amount of time and computing resource to retrain the model when the deep model encounters classes it has never seen before. However, for healthcare applications, enabling a model to generalize new clinical scenarios is of great importance. The existing image classification methods cannot explicitly use the location information of the pixel, making them insensitive to cues related only to the location. Besides, they also rely on local convolution and cannot properly utilize global information, which is essential for image classification. To alleviate these problems, we propose a collateral location coding to help the network explicitly exploit the location information of each pixel to make it easier for the network to recognize cues related to location only, and a single-key global spatial attention is designed to make the pixels at each location perceive the global spatial information in a low-cost way. Experimental results on three medical image benchmark datasets demonstrate that our proposed algorithm outperforms the state-of-the-art approaches in both effectiveness and generalization ability.
Collapse
|
8
|
Huynh T, Nibali A, He Z. Semi-supervised learning for medical image classification using imbalanced training data. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2022; 216:106628. [PMID: 35101700 DOI: 10.1016/j.cmpb.2022.106628] [Citation(s) in RCA: 20] [Impact Index Per Article: 6.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/01/2021] [Revised: 12/20/2021] [Accepted: 01/07/2022] [Indexed: 06/14/2023]
Abstract
BACKGROUND AND OBJECTIVE Medical image classification is often challenging for two reasons: a lack of labelled examples due to expensive and time-consuming annotation protocols, and imbalanced class labels due to the relative scarcity of disease-positive individuals in the wider population. Semi-supervised learning methods exist for dealing with a lack of labels, but they generally do not address the problem of class imbalance. Hence, the purpose of this study is to explore a new approach to perturbation-based semi-supervised learning which tackles the problem of applying semi-supervised learning to medical image classification with imbalanced training data. METHODS In this study we propose Adaptive Blended Consistency Loss (ABCL), a simple yet effective drop-in replacement for consistency loss in perturbation-based semi-supervised learning methods. ABCL counteracts data skew by adaptively mixing the target class distribution of the consistency loss in accordance with class frequency. Our proposed method is evaluated and compared with existing methods on two different imbalanced medical image classification datasets. An ablation study is also provided to analyse the properties and effectiveness of our proposed method. RESULTS Our experiments with ABCL reveal improvements to unweighted average recall (UAR) when compared with existing consistency losses that are not designed to counteract class imbalance and other existing methods. Our proposed ABCL method is able to improve the performance of the baseline consistency loss approach from 0.59 to 0.67 UAR and outperforms methods that address the class imbalance problem for labelled data (between 0.51 and 0.59 UAR) and for unlabelled data (0.61 UAR) on the imbalanced skin cancer dataset. On the imbalanced retinal fundus glaucoma dataset, ABCL (combined with Weighted Cross Entropy loss) achieves 0.67 UAR, which is an improvement over the best existing approach (0.57 UAR). CONCLUSIONS Overall the results show the effectiveness of ABCL to alleviate the class imbalance problem for semi-supervised classification for medical images.
Collapse
Affiliation(s)
- Tri Huynh
- Department of Computer Science and Information Technology, La Trobe University, Melbourne, Australia.
| | - Aiden Nibali
- Department of Computer Science and Information Technology, La Trobe University, Melbourne, Australia
| | - Zhen He
- Department of Computer Science and Information Technology, La Trobe University, Melbourne, Australia
| |
Collapse
|
9
|
|
10
|
Cai Y, Wu S, Zhou M, Gao S, Yu H. Early Warning of Gas Concentration in Coal Mines Production Based on Probability Density Machine. SENSORS 2021; 21:s21175730. [PMID: 34502619 PMCID: PMC8433910 DOI: 10.3390/s21175730] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 07/06/2021] [Revised: 08/22/2021] [Accepted: 08/24/2021] [Indexed: 11/16/2022]
Abstract
Gas explosion has always been an important factor restricting coal mine production safety. The application of machine learning techniques in coal mine gas concentration prediction and early warning can effectively prevent gas explosion accidents. Nearly all traditional prediction models use a regression technique to predict gas concentration. Considering there exist very few instances of high gas concentration, the instance distribution of gas concentration would be extremely imbalanced. Therefore, such regression models generally perform poorly in predicting high gas concentration instances. In this study, we consider early warning of gas concentration as a binary-class problem, and divide gas concentration data into warning class and non-warning class according to the concentration threshold. We proposed the probability density machine (PDM) algorithm with excellent adaptability to imbalanced data distribution. In this study, we use the original gas concentration data collected from several monitoring points in a coal mine in Datong city, Shanxi Province, China, to train the PDM model and to compare the model with several class imbalance learning algorithms. The results show that the PDM algorithm is superior to the traditional and state-of-the-art class imbalance learning algorithms, and can produce more accurate early warning results for gas explosion.
Collapse
Affiliation(s)
| | | | | | | | - Hualong Yu
- Correspondence: ; Tel.: +86-159-5289-4360
| |
Collapse
|
11
|
Tang J, Li J, Xu W, Tian Y, Ju X, Zhang J. Robust cost-sensitive kernel method with Blinex loss and its applications in credit risk evaluation. Neural Netw 2021; 143:327-344. [PMID: 34182234 DOI: 10.1016/j.neunet.2021.06.016] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/06/2021] [Revised: 05/10/2021] [Accepted: 06/10/2021] [Indexed: 10/21/2022]
Abstract
Credit risk evaluation is a crucial yet challenging problem in financial analysis. It can not only help institutions reduce risk and ensure profitability, but also improve consumers' fair practices. The data-driven algorithms such as artificial intelligence techniques regard the evaluation as a classification problem and aim to classify transactions as default or non-default. Since non-default samples greatly outnumber default samples, it is a typical imbalanced learning problem and each class or each sample needs special treatment. Numerous data-level, algorithm-level and hybrid methods are presented, and cost-sensitive support vector machines (CSSVMs) are representative algorithm-level methods. Based on the minimization of symmetric and unbounded loss functions, CSSVMs impose higher penalties on the misclassification costs of minority instances using domain specific parameters. However, such loss functions as error measurement cannot have an obvious cost-sensitive generalization. In this paper, we propose a robust cost-sensitive kernel method with Blinex loss (CSKB), which can be applied in credit risk evaluation. By inheriting the elegant merits of Blinex loss function, i.e., asymmetry and boundedness, CSKB not only flexibly controls distinct costs for both classes, but also enjoys noise robustness. As a data-driven decision-making paradigm of credit risk evaluation, CSKB can achieve the "win-win" situation for both the financial institutions and consumers. We solve linear and nonlinear CSKB by Nesterov accelerated gradient algorithm and Pegasos algorithm respectively. Moreover, the generalization capability of CSKB is theoretically analyzed. Comprehensive experiments on synthetic, UCI and credit risk evaluation datasets demonstrate that CSKB compares more favorably than other benchmark methods in terms of various measures.
Collapse
Affiliation(s)
- Jingjing Tang
- School of Business Administration, Faculty of Business Administration, Southwestern University of Finance and Economics, Chengdu 611130, China.
| | - Jiahui Li
- School of Business Administration, Faculty of Business Administration, Southwestern University of Finance and Economics, Chengdu 611130, China.
| | - Weiqi Xu
- School of Business Administration, Faculty of Business Administration, Southwestern University of Finance and Economics, Chengdu 611130, China.
| | - Yingjie Tian
- School of Economics and Management, University of Chinese Academy of Sciences, Beijing 100190, China; Research Center on Fictitious Economy and Data Science, Chinese Academy of Sciences, Beijing 100190, China; Key Laboratory of Big Data Mining and Knowledge Management, Chinese Academy of Sciences, Beijing 100190, China.
| | - Xuchan Ju
- College of Mathematics and Statistics, Shenzhen University, Shenzhen 518060, China.
| | - Jie Zhang
- Alibaba Group, Beijing 100102, China.
| |
Collapse
|
12
|
Yu H, Chen C, Yang H. Two-Stage Game Strategy for Multiclass Imbalanced Data Online Prediction. Neural Process Lett 2020. [DOI: 10.1007/s11063-020-10358-w] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2022]
|
13
|
Anh DN, Hung BD, Huy PQ, Tho DX. Feature Analysis for Imbalanced Learning. JOURNAL OF ADVANCED COMPUTATIONAL INTELLIGENCE AND INTELLIGENT INFORMATICS 2020. [DOI: 10.20965/jaciii.2020.p0648] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
Based on the results of artificial samples generated in the minority class and through the label regulation of the neighbor samples of the majority class, the precision of the classification prediction for imbalanced learning has clearly been enhanced. This article presents a unified solution combining learning factors to improve the learning performance. The proposed method solves this imbalance through a feature selection incorporating the generation of artificial samples and label regulation. A probabilistic representation is used for all aspects of learning: class, sample, and feature. A Bayesian inference is applied to the learning model to interpret the imbalance occurring in the training data and to describe solutions for recovering the balance. We show that the generation of artificial samples is sample based approach and label regulation is class based approach. We discovered that feature selection achieves surprisingly good results when combined with a sample- or class-based solution.
Collapse
|
14
|
Adaptive Decision Threshold-Based Extreme Learning Machine for Classifying Imbalanced Multi-label Data. Neural Process Lett 2020. [DOI: 10.1007/s11063-020-10343-3] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/25/2022]
|
15
|
Cheng K, Gao S, Dong W, Yang X, Wang Q, Yu H. Boosting label weighted extreme learning machine for classifying multi-label imbalanced data. Neurocomputing 2020. [DOI: 10.1016/j.neucom.2020.04.098] [Citation(s) in RCA: 13] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/27/2022]
|
16
|
Wang Z, Li Y, Li D, Zhu Z, Du W. Entropy and gravitation based dynamic radius nearest neighbor classification for imbalanced problem. Knowl Based Syst 2020. [DOI: 10.1016/j.knosys.2020.105474] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/25/2022]
|
17
|
Fu GH, Wu YJ, Zong MJ, Pan J. Hellinger distance-based stable sparse feature selection for high-dimensional class-imbalanced data. BMC Bioinformatics 2020; 21:121. [PMID: 32293252 PMCID: PMC7092448 DOI: 10.1186/s12859-020-3411-3] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/05/2019] [Accepted: 02/12/2020] [Indexed: 11/11/2022] Open
Abstract
Background Feature selection in class-imbalance learning has gained increasing attention in recent years due to the massive growth of high-dimensional class-imbalanced data across many scientific fields. In addition to reducing model complexity and discovering key biomarkers, feature selection is also an effective method of combating overlapping which may arise in such data and become a crucial aspect for determining classification performance. However, ordinary feature selection techniques for classification can not be simply used for addressing class-imbalanced data without any adjustment. Thus, more efficient feature selection technique must be developed for complicated class-imbalanced data, especially in the context of high-dimensionality. Results We proposed an algorithm called sssHD to achieve stable sparse feature selection applied it to complicated class-imbalanced data. sssHD is based on the Hellinger distance (HD) coupled with sparse regularization techniques. We stated that Hellinger distance is not only class-insensitive but also translation-invariant. Simulation result indicates that HD-based selection algorithm is effective in recognizing key features and control false discoveries for class-imbalance learning. Five gene expression datasets are also employed to test the performance of the sssHD algorithm, and a comparison with several existing selection procedures is performed. The result shows that sssHD is highly competitive in terms of five assessment metrics. In addition, sssHD presents limited differences between performing and not performing re-balance preprocessing. Conclusions sssHD is a practical feature selection method for high-dimensional class-imbalanced data, which is simple and can be an alternative for performing feature selection in class-imbalanced data. sssHD can be easily extended by connecting it with different re-balance preprocessing, different sparse regularization structures as well as different classifiers. As such, the algorithm is extremely general and has a wide range of applicability.
Collapse
Affiliation(s)
- Guang-Hui Fu
- School of Science, Kunming University of Science and Technology, Kunming, 650500, People's Republic of China.
| | - Yuan-Jiao Wu
- School of Science, Kunming University of Science and Technology, Kunming, 650500, People's Republic of China
| | - Min-Jie Zong
- School of Science, Kunming University of Science and Technology, Kunming, 650500, People's Republic of China
| | - Jianxin Pan
- School of Mathematics, The University of Manchester, Manchester, M13 9PL, UK
| |
Collapse
|
18
|
|
19
|
Extreme learning machine with hybrid cost function of G-mean and probability for imbalance learning. INT J MACH LEARN CYB 2020. [DOI: 10.1007/s13042-020-01090-x] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/24/2022]
|
20
|
Yu H, Sun X, Yan X. Sequential prediction for imbalanced data stream via weighted OS-ELM and dynamic GAN. INTELL DATA ANAL 2019. [DOI: 10.3233/ida-184377] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
|
21
|
Dong Q, Gong S, Zhu X. Imbalanced Deep Learning by Minority Class Incremental Rectification. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 2019; 41:1367-1381. [PMID: 29993438 DOI: 10.1109/tpami.2018.2832629] [Citation(s) in RCA: 85] [Impact Index Per Article: 14.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/08/2023]
Abstract
Model learning from class imbalanced training data is a long-standing and significant challenge for machine learning. In particular, existing deep learning methods consider mostly either class balanced data or moderately imbalanced data in model training, and ignore the challenge of learning from significantly imbalanced training data. To address this problem, we formulate a class imbalanced deep learning model based on batch-wise incremental minority (sparsely sampled) class rectification by hard sample mining in majority (frequently sampled) classes during model training. This model is designed to minimise the dominant effect of majority classes by discovering sparsely sampled boundaries of minority classes in an iterative batch-wise learning process. To that end, we introduce a Class Rectification Loss (CRL) function that can be deployed readily in deep network architectures. Extensive experimental evaluations are conducted on three imbalanced person attribute benchmark datasets (CelebA, X-Domain, DeepFashion) and one balanced object category benchmark dataset (CIFAR-100). These experimental results demonstrate the performance advantages and model scalability of the proposed batch-wise incremental minority class rectification model over the existing state-of-the-art models for addressing the problem of imbalanced data learning.
Collapse
|
22
|
Yu H, Yang X, Zheng S, Sun C. Active Learning From Imbalanced Data: A Solution of Online Weighted Extreme Learning Machine. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2019; 30:1088-1103. [PMID: 30137013 DOI: 10.1109/tnnls.2018.2855446] [Citation(s) in RCA: 35] [Impact Index Per Article: 5.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/08/2023]
Abstract
It is well known that active learning can simultaneously improve the quality of the classification model and decrease the complexity of training instances. However, several previous studies have indicated that the performance of active learning is easily disrupted by an imbalanced data distribution. Some existing imbalanced active learning approaches also suffer from either low performance or high time consumption. To address these problems, this paper describes an efficient solution based on the extreme learning machine (ELM) classification model, called active online-weighted ELM (AOW-ELM). The main contributions of this paper include: 1) the reasons why active learning can be disrupted by an imbalanced instance distribution and its influencing factors are discussed in detail; 2) the hierarchical clustering technique is adopted to select initially labeled instances in order to avoid the missed cluster effect and cold start phenomenon as much as possible; 3) the weighted ELM (WELM) is selected as the base classifier to guarantee the impartiality of instance selection in the procedure of active learning, and an efficient online updated mode of WELM is deduced in theory; and 4) an early stopping criterion that is similar to but more flexible than the margin exhaustion criterion is presented. The experimental results on 32 binary-class data sets with different imbalance ratios demonstrate that the proposed AOW-ELM algorithm is more effective and efficient than several state-of-the-art active learning algorithms that are specifically designed for the class imbalance scenario.
Collapse
|
23
|
|
24
|
Intelligent fault diagnosis of rotating machinery using improved multiscale dispersion entropy and mRMR feature selection. Knowl Based Syst 2019. [DOI: 10.1016/j.knosys.2018.09.004] [Citation(s) in RCA: 64] [Impact Index Per Article: 10.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/21/2022]
|
25
|
|
26
|
|
27
|
Zhang X, Zhuang Y, Wang W, Pedrycz W. Transfer Boosting With Synthetic Instances for Class Imbalanced Object Recognition. IEEE TRANSACTIONS ON CYBERNETICS 2018; 48:357-370. [PMID: 28026795 DOI: 10.1109/tcyb.2016.2636370] [Citation(s) in RCA: 29] [Impact Index Per Article: 4.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/06/2023]
Abstract
A challenging problem in object recognition is to train a robust classifier with small and imbalanced data set. In such cases, the learned classifier tends to overfit the training data and has low prediction accuracy on the minority class. In this paper, we address the problem of class imbalanced object recognition by combining synthetic minorities over-sampling technique (SMOTE) and instance-based transfer boosting to rebalance the skewed class distribution. We present ways of generating synthetic instances under the learning framework of transfer Adaboost. A novel weighted SMOTE technique (WSMOTE) is proposed to generate weighted synthetic instances with weighted source and target instances at each boosting round. Based on WSMOTE, we propose a novel class imbalanced transfer boosting algorithm called WSMOTE-TrAdaboost and experimentally demonstrate its effectiveness on four datasets (Office, Caltech256, SUN2012, and VOC2012) for object recognition application. Bag-of-words model with SURF features and histogram of oriented gradient features are separately used to represent an image. We experimentally demonstrated the effectiveness and robustness of our approaches by comparing it with several baseline algorithms in boosting family for class imbalanced learning.
Collapse
|
28
|
Jia C, Li X, Wang K, Ding D. Adaptive control of nonlinear system using online error minimum neural networks. ISA TRANSACTIONS 2016; 65:125-132. [PMID: 27522102 DOI: 10.1016/j.isatra.2016.07.012] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/26/2016] [Revised: 07/11/2016] [Accepted: 07/26/2016] [Indexed: 06/06/2023]
Abstract
In this paper, a new learning algorithm named OEM-ELM (Online Error Minimized-ELM) is proposed based on ELM (Extreme Learning Machine) neural network algorithm and the spreading of its main structure. The core idea of this OEM-ELM algorithm is: online learning, evaluation of network performance, and increasing of the number of hidden nodes. It combines the advantages of OS-ELM and EM-ELM, which can improve the capability of identification and avoid the redundancy of networks. The adaptive control based on the proposed algorithm OEM-ELM is set up which has stronger adaptive capability to the change of environment. The adaptive control of chemical process Continuous Stirred Tank Reactor (CSTR) is also given for application. The simulation results show that the proposed algorithm with respect to the traditional ELM algorithm can avoid network redundancy and improve the control performance greatly.
Collapse
Affiliation(s)
- Chao Jia
- School of Automation and Electrical Engineering, University of Science and Technology Beijing, Beijing 100083, P.R. China
| | - Xiaoli Li
- College of Electronic Information and Control Engineering, Beijing University of Technology, Beijing 100124, P.R. China.
| | - Kang Wang
- School of Automation and Electrical Engineering, University of Science and Technology Beijing, Beijing 100083, P.R. China
| | - Dawei Ding
- School of Automation and Electrical Engineering, University of Science and Technology Beijing, Beijing 100083, P.R. China
| |
Collapse
|
29
|
|