Oversampling the Minority Class in the Feature Space

Racial and Ethnic Disparities in Predictive Accuracy of Machine Learning Algorithms Developed Using a National Database for 30-Day Complications Following Total Joint Arthroplasty

BACKGROUND

While predictive capabilities of machine learning (ML) algorithms for hip and knee total joint arthroplasty (TJA) have been demonstrated in previous studies, their performance in racial and ethnic minority patients has not been investigated. This study aimed to assess the performance of ML algorithms in predicting 30-days complications following TJA in racial and ethnic minority patients.

METHODS

A total of 267,194 patients undergoing primary TJA between 2013 and 2020 were identified from a national outcomes database. The patient cohort was stratified according to race, with further substratification into Hispanic or non-Hispanic ethnicity. There were two ML algorithms, histogram-based gradient boosting (HGB), and random forest (RF), that were modeled to predict 30-days complications following primary TJA in the overall population. They were subsequently assessed in each racial and ethnic subcohort using discrimination, calibration, accuracy, and potential clinical usefulness.

RESULTS

Both models achieved excellent (Area under the curve (AUC) > 0.8) discrimination (AUCHGB = AUCRF = 0.86), calibration, and accuracy (HGB: slope = 1.00, intercept = -0.03, Brier score = 0.12; RF: slope = 0.97, intercept = 0.02, Brier score = 0.12) in the non-Hispanic White population (N = 224,073). Discrimination decreased in the White Hispanic (N = 10,429; AUC = 0.75 to 0.76), Black (N = 25,116; AUC = 0.77), Black Hispanic (N = 240; AUC = 0.78), Asian non-Hispanic (N = 4,809; AUC = 0.78 to 0.79), and overall (N = 267,194; AUC = 0.75 to 0.76) cohorts, but remained well-calibrated. We noted the poorest model discrimination (N = 1,870; AUC = 0.67 to 0.68) and calibration in the American-Indian cohort.

CONCLUSIONS

The ML algorithms demonstrate an inferior predictive ability for 30-days complications following primary TJA in racial and ethnic minorities when trained on existing healthcare big data. This may be attributed to the disproportionate underrepresentation of minority groups within these databases, as demonstrated by the smaller sample sizes available to train the ML models. The ML models developed using smaller datasets (e.g., in racial and ethnic minorities) may not be as accurate as larger datasets, highlighting the need for equity-conscious model development.

LEVEL OF EVIDENCE

III; retrospective cohort study.

Assessment of non-fatal injuries among university students in Hainan: a machine learning approach to exploring key factors

Background

Injuries constitute a significant global public health concern, particularly among individuals aged 0-34. These injuries are affected by various social, psychological, and physiological factors and are no longer viewed merely as accidental occurrences. Existing research has identified multiple risk factors for injuries; however, they often focus on the cases of children or the older adult, neglecting the university students. Machine learning (ML) can provide advanced analytics and is better suited to complex, nonlinear data compared to traditional methods. That said, ML has been underutilized in injury research despite its great potential. To fill this gap, this study applies ML to analyze injury data among university students in Hainan Province. The purpose is to provide insights into developing effective prevention strategies. To explore the relationship between scores on the self-rating anxiety scale and self-rating depression scale and the risk of non-fatal injuries within 1 year, we categorized these scores into two groups using restricted cubic splines.

Methods

Chi-square tests and LASSO regression analysis were employed to filter factors potentially associated with non-fatal injuries. The Synthetic Minority Over-Sampling Technique (SMOTE) was applied to balance the dataset. Subsequent analyses were conducted using random forest, logistic regression, decision tree, and XGBoost models. Each model underwent 10-fold cross-validation to mitigate overfitting, with hyperparameters being optimized to improve performance. SHAP was utilized to identify the primary factors influencing non-fatal injuries.

Results

The Random Forest model has proved effective in this study. It identified three primary risk factors for predicting non-fatal injuries: being male, favorable household financial situation, and stable relationship. Protective factors include reduced internet time and being an only child in the family.

Conclusion

The study highlighted five key factors influencing non-fatal injuries: sex, household financial situation, relationship stability, internet time, and sibling status. In identifying these factors, the Random Forest, Logistic Regression, Decision Tree, and XGBoost models demonstrated varying effectiveness, with the Random Forest model exhibiting superior performance.

Deep learning-based facial analysis for predicting difficult videolaryngoscopy: a feasibility study

While videolaryngoscopy has resulted in better overall success rates of tracheal intubation, airway assessment is still an important prerequisite for safe airway management. This study aimed to create an artificial intelligence model to identify difficult videolaryngoscopy using a neural network. Baseline characteristics, medical history, bedside examination and seven facial images were included as predictor variables. ResNet-18 was introduced to recognise images and extract features. Different machine learning algorithms were utilised to develop predictive models. A videolaryngoscopy view of Cormack-Lehane grade of 1 or 2 was classified as 'non-difficult', while grade 3 or 4 was classified as 'difficult'. A total of 5849 patients were included, of whom 5335 had non-difficult and 514 had difficult videolaryngoscopy. The facial model (only including facial images) using the Light Gradient Boosting Machine algorithm showed the highest area under the curve (95%CI) of 0.779 (0.733-0.825) with a sensitivity (95%CI) of 0.757 (0.650-0.845) and specificity (95%CI) of 0.721 (0.626-0.794) in the test set. Compared with bedside examination and multivariate scores (El-Ganzouri and Wilson), the facial model had significantly higher predictive performance (p < 0.001). Artificial intelligence-based facial analysis is a feasible technique for predicting difficulty during videolaryngoscopy, and the model developed using neural networks has higher predictive performance than traditional methods.

Deep learning and radiomic feature-based blending ensemble classifier for malignancy risk prediction in cystic renal lesions

BACKGROUND

The rising prevalence of cystic renal lesions (CRLs) detected by computed tomography necessitates better identification of the malignant cystic renal neoplasms since a significant majority of CRLs are benign renal cysts. Using arterial phase CT scans combined with pathology diagnosis results, a fusion feature-based blending ensemble machine learning model was created to identify malignant renal neoplasms from cystic renal lesions (CRLs). Histopathology results were adopted as diagnosis standard. Pretrained 3D-ResNet50 network was selected for non-handcrafted features extraction and pyradiomics toolbox was selected for handcrafted features extraction. Tenfold cross validated least absolute shrinkage and selection operator regression methods were selected to identify the most discriminative candidate features in the development cohort. Feature's reproducibility was evaluated by intra-class correlation coefficients and inter-class correlation coefficients. Pearson correlation coefficients for normal distribution and Spearman's rank correlation coefficients for non-normal distribution were utilized to remove redundant features. After that, a blending ensemble machine learning model were developed in training cohort. Area under the receiver operator characteristic curve (AUC), accuracy score (ACC), and decision curve analysis (DCA) were employed to evaluate the performance of the final model in testing cohort.

RESULTS

The fusion feature-based machine learning algorithm demonstrated excellent diagnostic performance in external validation dataset (AUC = 0.934, ACC = 0.905). Net benefits presented by DCA are higher than Bosniak-2019 version classification for stratifying patients with CRL to the appropriate surgery procedure.

CONCLUSIONS

Fusion feature-based classifier accurately distinguished malignant and benign CRLs which outperformed the Bosniak-2019 version classification and illustrated improved clinical decision-making utility.

HS-Gen: a hypersphere-constrained generation mechanism to improve synthetic minority oversampling for imbalanced classification

Mitigating the impact of class-imbalance data on classifiers is a challenging task in machine learning. SMOTE is a well-known method to tackle this task by modifying class distribution and generating synthetic instances. However, most of the SMOTE-based methods focus on the phase of data selection, while few consider the phase of data generation. This paper proposes a hypersphere-constrained generation mechanism (HS-Gen) to improve synthetic minority oversampling. Unlike linear interpolation commonly used in SMOTE-based methods, HS-Gen generates a minority instance in a hypersphere rather than on a straight line. This mechanism expands the distribution range of minority instances with significant randomness and diversity. Furthermore, HS-Gen is attached with a noise prevention strategy that adaptively shrinks the hypersphere by determining whether new instances fall into the majority class region. HS-Gen can be regarded as an oversampling optimization mechanism and flexibly embedded into the SMOTE-based methods. We conduct comparative experiments by embedding HS-Gen into the original SMOTE, Borderline-SMOTE, ADASYN, k-means SMOTE, and RSMOTE. Experimental results show that the embedded versions can generate higher quality synthetic instances than the original ones. Moreover, on these oversampled datasets, the conventional classifiers (C4.5 and Adaboost) obtain significant performance improvement in terms of F1 measure and G-mean.

Stratification of malignant renal neoplasms from cystic renal lesions using deep learning and radiomics features based on a stacking ensemble CT machine learning algorithm

Using nephrographic phase CT images combined with pathology diagnosis, we aim to develop and validate a fusion feature-based stacking ensemble machine learning model to distinguish malignant renal neoplasms from cystic renal lesions (CRLs). This retrospective research includes 166 individuals with CRLs for model training and 47 individuals with CRLs in another institution for model testing. Histopathology results are adopted as diagnosis criterion. Nephrographic phase CT scans are selected to build the fusion feature-based machine learning algorithms. The pretrained 3D-ResNet50 CNN model and radiomics methods are selected to extract deep features and radiomics features, respectively. Fivefold cross-validated least absolute shrinkage and selection operator (LASSO) regression methods are adopted to identify the most discriminative candidate features in the development cohort. Intraclass correlation coefficients and interclass correlation coefficients are employed to evaluate feature’s reproducibility. Pearson correlation coefficients for normal distribution features and Spearman’s rank correlation coefficients for non-normal distribution features are used to eliminate redundant features. After that, stacking ensemble machine learning models are developed in the training cohort. The area under the receiver operator characteristic curve (ROC), calibration curve, and decision curve analysis (DCA) are adopted in the testing cohort to evaluate the performance of each model. The stacking ensemble machine learning algorithm reached excellent diagnostic performance in the testing dataset. The calibration plot shows good stability when using the stacking ensemble model. Net benefits presented by DCA are higher than the Bosniak 2019 version classification when employing any machine learning algorithm. The fusion feature-based machine learning algorithm accurately distinguishes malignant renal neoplasms from CRLs, which outperformed the Bosniak 2019 version classification, and proves to be more applicable for clinical decision-making.

Globalized Multiple Balanced Subsets With Collaborative Learning for Imbalanced Data

The skewed distribution of data brings difficulties to classify minority and majority samples in the imbalanced problem. The balanced bagging randomly undersampes majority samples several times and combines the selected majority samples with minority samples to form several balanced subsets, in which the numbers of minority and majority samples are roughly equal. However, the balanced bagging is the lack of a unified learning framework. Moreover, it fails to concern the connection of all subsets and the global information of the entire data distribution. To this end, this article puts several balanced subsets into an effective learning framework with a criterion function. In the learning framework, one regularization term called R_S establishes the connection and realizes the collaborative learning of all subsets by requiring the consistent outputs of the minority samples in different subsets. Besides, another regularization term called R_W provides the global information to each basic classifier by reducing the difference between the direction of the solution vector in each subset and that in the entire dataset. The proposed learning framework is called globalized multiple balanced subsets with collaborative learning (GMBSCL). The experimental results validate the effectiveness of the proposed GMBSCL.

An Oversampling Method for Class Imbalance Problems on Large Datasets

Several oversampling methods have been proposed for solving the class imbalance problem. However, most of them require searching the k-nearest neighbors to generate synthetic objects. This requirement makes them time-consuming and therefore unsuitable for large datasets. In this paper, an oversampling method for large class imbalance problems that do not require the k-nearest neighbors’ search is proposed. According to our experiments on large datasets with different sizes of imbalance, the proposed method is at least twice as fast as 8 the fastest method reported in the literature while obtaining similar oversampling quality.

RB-CCR: Radial-Based Combined Cleaning and Resampling algorithm for imbalanced data classification

Real-world classification domains, such as medicine, health and safety, and finance, often exhibit imbalanced class priors and have asynchronous misclassification costs. In such cases, the classification model must achieve a high recall without significantly impacting precision. Resampling the training data is the standard approach to improving classification performance on imbalanced binary data. However, the state-of-the-art methods ignore the local joint distribution of the data or correct it as a post-processing step. This can causes sub-optimal shifts in the training distribution, particularly when the target data distribution is complex. In this paper, we propose Radial-Based Combined Cleaning and Resampling (RB-CCR). RB-CCR utilizes the concept of class potential to refine the energy-based resampling approach of CCR. In particular, RB-CCR exploits the class potential to accurately locate sub-regions of the data-space for synthetic oversampling. The category sub-region for oversampling can be specified as an input parameter to meet domain-specific needs or be automatically selected via cross-validation. Our $$5\times 2$$ 5 × 2 cross-validated results on 57 benchmark binary datasets with 9 classifiers show that RB-CCR achieves a better precision-recall trade-off than CCR and generally out-performs the state-of-the-art resampling methods in terms of AUC and G-mean.

A novel optimized repeatedly random undersampling for selecting negative samples: A case study in an SVM-based forest fire susceptibility assessment

The negative sample selection method is a key issue in studies of using machine learning approaches to spatially assess natural hazards. Recently, a Repeatedly Random Undersampling (RRU) was proposed to address the randomness problem faced in Single Random Sampling. However, the RRU cannot guarantee that the generated classifier has the best classification performance during the repeatedly random sampling process. To address this weakness, in this study we proposed an optimized RRU, which follows the idea of RRU, and then changing its rule to find a best classifier. Then, the selected classifier, the actual most accurate classifier (MAC), was employed to compute the probability of hazard occurrence. Support Vector Machine (SVM) was selected as the analysis method, and Genetic Algorithm was employed to compute the parameters of SVM. Forest fire susceptibility was assessed in Huichang County in China due to its forest values and frequent fire events. The results indicated that compared with the RRU, the optimized RRU can find out an actual MAC which has the best classification performance among possible MACs; also, the fire susceptibility map generated by the actual MAC comforts to objective facts. The generated fire susceptibility map can provide useful decision supports for local government to reduce forest fire risks. Moreover, the proposed sampling method, the optimized RRU, presented an enhanced approach for selecting negative samples, which makes the results of forest fire susceptibility assessment more reliable and accurate.

Radial-Based Oversampling for Multiclass Imbalanced Data Classification

Learning from imbalanced data is among the most popular topics in the contemporary machine learning. However, the vast majority of attention in this field is given to binary problems, while their much more difficult multiclass counterparts are relatively unexplored. Handling data sets with multiple skewed classes poses various challenges and calls for a better understanding of the relationship among classes. In this paper, we propose multiclass radial-based oversampling (MC-RBO), a novel data-sampling algorithm dedicated to multiclass problems. The main novelty of our method lies in using potential functions for generating artificial instances. We take into account information coming from all of the classes, contrary to existing multiclass oversampling approaches that use only minority class characteristics. The process of artificial instance generation is guided by exploring areas where the value of the mutual class distribution is very small. This way, we ensure a smart oversampling procedure that can cope with difficult data distributions and alleviate the shortcomings of existing methods. The usefulness of the MC-RBO algorithm is evaluated on the basis of extensive experimental study and backed-up with a thorough statistical analysis. Obtained results show that by taking into account information coming from all of the classes and conducting a smart oversampling, we can significantly improve the process of learning from multiclass imbalanced data.

Geometric Structural Ensemble Learning for Imbalanced Problems

The classification on imbalanced data sets is a great challenge in machine learning. In this paper, a geometric structural ensemble (GSE) learning framework is proposed to address the issue. It is known that the traditional ensemble methods train and combine a series of basic classifiers according to various weights, which might lack the geometric meaning. Oppositely, the GSE partitions and eliminates redundant majority samples by generating hyper-sphere through the Euclidean metric and learns basic classifiers to enclose the minority samples, which achieves higher efficiency in the training process and seems easier to understand. In detail, the current weak classifier builds boundaries between the majority and the minority samples and removes the former. Then, the remaining samples are used to train the next. When the training process is done, all of the majority samples could be cleaned and the combination of all basic classifiers is obtained. To further improve the generalization, two relaxation techniques are proposed. Theoretically, the computational complexity of GSE could approach O(ndlog(n_min)log(n _maj)) . The comprehensive experiments validate both the effectiveness and efficiency of GSE.

Ensemble Classification for Skewed Data Streams Based on Neural Network

Data stream learning in non-stationary environments and skewed class distributions has been receiving more attention in machine learning communities. This paper proposes a novel ensemble classification method (ECSDS) for classifying data streams with skewed class distributions. In the proposed ensemble method, back-propagation neural network is selected as the base classifier. In order to demonstrate the effectiveness of our proposed method, we choose three baseline methods based on ECSDS and evaluate their overall performance on ten datasets from UCI machine learning repository. Moreover, the performance of incremental learning is also evaluated by these datasets. The experimental results show our proposed method can effectively deal with classification problems on non-stationary data streams with class imbalance.

Classifying Cognitive Profiles Using Machine Learning with Privileged Information in Mild Cognitive Impairment

Early diagnosis of dementia is critical for assessing disease progression and potential treatment. State-or-the-art machine learning techniques have been increasingly employed to take on this diagnostic task. In this study, we employed Generalized Matrix Learning Vector Quantization (GMLVQ) classifiers to discriminate patients with Mild Cognitive Impairment (MCI) from healthy controls based on their cognitive skills. Further, we adopted a “Learning with privileged information” approach to combine cognitive and fMRI data for the classification task. The resulting classifier operates solely on the cognitive data while it incorporates the fMRI data as privileged information (PI) during training. This novel classifier is of practical use as the collection of brain imaging data is not always possible with patients and older participants. MCI patients and healthy age-matched controls were trained to extract structure from temporal sequences. We ask whether machine learning classifiers can be used to discriminate patients from controls and whether differences between these groups relate to individual cognitive profiles. To this end, we tested participants in four cognitive tasks: working memory, cognitive inhibition, divided attention, and selective attention. We also collected fMRI data before and after training on a probabilistic sequence learning task and extracted fMRI responses and connectivity as features for machine learning classifiers. Our results show that the PI guided GMLVQ classifiers outperform the baseline classifier that only used the cognitive data. In addition, we found that for the baseline classifier, divided attention is the only relevant cognitive feature. When PI was incorporated, divided attention remained the most relevant feature while cognitive inhibition became also relevant for the task. Interestingly, this analysis for the fMRI GMLVQ classifier suggests that (1) when overall fMRI signal is used as inputs to the classifier, the post-training session is most relevant; and (2) when the graph feature reflecting underlying spatiotemporal fMRI pattern is used, the pre-training session is most relevant. Taken together these results suggest that brain connectivity before training and overall fMRI signal after training are both diagnostic of cognitive skills in MCI.

Semi-supervised learning for ordinal Kernel Discriminant Analysis

Ordinal classification considers those classification problems where the labels of the variable to predict follow a given order. Naturally, labelled data is scarce or difficult to obtain in this type of problems because, in many cases, ordinal labels are given by a user or expert (e.g. in recommendation systems). Firstly, this paper develops a new strategy for ordinal classification where both labelled and unlabelled data are used in the model construction step (a scheme which is referred to as semi-supervised learning). More specifically, the ordinal version of kernel discriminant learning is extended for this setting considering the neighbourhood information of unlabelled data, which is proposed to be computed in the feature space induced by the kernel function. Secondly, a new method for semi-supervised kernel learning is devised in the context of ordinal classification, which is combined with our developed classification strategy to optimise the kernel parameters. The experiments conducted compare 6 different approaches for semi-supervised learning in the context of ordinal classification in a battery of 30 datasets, showing (1) the good synergy of the ordinal version of discriminant analysis and the use of unlabelled data and (2) the advantage of computing distances in the feature space induced by the kernel function.