Learning from Imbalanced Data

An explainable predictive machine learning model of gangrenous cholecystitis based on clinical data: a retrospective single center study

BACKGROUND

Gangrenous cholecystitis (GC) is a serious clinical condition associated with high morbidity and mortality rates. Machine learning (ML) has significant potential in addressing the diverse characteristics of real data. We aim to develop an explainable and cost-effective predictive model for GC utilizing ML and Shapley Additive explanation (SHAP) algorithm.

RESULTS

This study included a total of 1006 patients with 26 clinical features. Through 5-fold CV, the best performing integrated learning model, XGBoost, was identified. The model was interpreted using SHAP to derive the feature subsets WBC, NLR, D-dimer, Gallbladder width, Fibrinogen, Gallbladder wallness, Hypokalemia or hyponatremia, these subsets comprised the final diagnostic prediction model.

CONCLUSIONS

The study developed a explainable predictive tool for GC at an early stage. This could assist doctors to make quick surgical intervention decisions and perform surgery on patients with GC as soon as possible.

NutriSighT: Interpretable Transformer Model for Dynamic Prediction of Hypocaloric Enteral Nutrition in Mechanically Ventilated Patients

Achieving adequate enteral nutrition among mechanically ventilated patients is challenging, yet critical. We developed NutriSighT, a transformer model using learnable positional coding to predict which patients would achieve hypocaloric nutrition between days 3-7 of mechanical ventilation. Using retrospective data from two large ICU databases (3,284 patients from AmsterdamUMCdb - development set, and 6,456 from MIMIC-IV - external validation set), we included adult patients intubated for at least 72 hours. NutriSighT achieved AUROC of 0.81 (95% CI: 0.81 - 0.82) and an AUPRC of 0.70 (95% CI: 0.70 - 0.72) on internal test set. External validation with MIMIC-IV data yielded a AUROC of 0.76 (95% CI: 0.75 - 0.76) and an AUPRC of (95% CI: 0.69 - 0.70). At a threshold of 0.5, the model achieved a 75.16% sensitivity, 60.57% specificity, 58.30% positive predictive value, and 76.88% negative predictive value. This approach may help clinicians personalize nutritional therapy among critically ill patients, improving patient outcomes.

Performance evaluation and comparative analysis of different machine learning algorithms in predicting postnatal care utilization: Evidence from the ethiopian demographic and health survey 2016

Postnatal care refers to the support provided to mothers and their newborns immediately after childbirth and during the first six weeks of life, a period when most maternal and neonatal deaths occur. In the 30 countries studied, nearly 40 percent of women did not receive a postpartum care check-up. This research aims to evaluate and compare the effectiveness of machine learning algorithms in predicting postnatal care utilization in Ethiopia and to identify the key factors involved. The study employs machine learning techniques to analyse secondary data from the 2016 Ethiopian Demographic and Health Survey. It aims to predict postnatal care utilization and identify key predictors via Python software, applying fifteen machine-learning algorithms to a sample of 7,193 women. Feature importance techniques were used to select the top predictors. The models' effectiveness was evaluated using sensitivity, specificity, F1 score, precision, accuracy, and area under the curve. Among the four experiments, tenfold cross-validation with balancing using Synthetic Minority Over-sampling Technique was outperformed. From fifteen models, the MLP Classifier (f1 score = 0.9548, AUC = 0.99), Random Forest Classifier (f1 score = 0.9543, AUC = 0.98), and Bagging Classifier (f1 score = 0.9498, AUC = 0.98) performed excellently, with a strong ability to differentiate between classes. The Region, residence, maternal education, religion, wealth index, health insurance status, and place of delivery are identified as contributing factors that predict postnatal care utilization. This study assessed machine learning models for forecasting postnatal care usage. Ten-fold cross-validation with Synthetic Minority Oversampling Technique produced the best results, emphasizing the significance of addressing class imbalance in healthcare datasets. This approach enhances the accuracy and dependability of predictive models. Key findings reveal regional and socioeconomic factors influencing PNC utilization, which can guide targeted initiatives to improve postnatal care utilization and ultimately enhance maternal and child health.

Metrics for evaluation of automatic epileptogenic zone localization in intracranial electrophysiology

INTRODUCTION

Precise localization of the epileptogenic zone is critical for successful epilepsy surgery. However, imbalanced datasets in terms of epileptic vs. normal electrode contacts and a lack of standardized evaluation guidelines hinder the consistent evaluation of automatic machine learning localization models.

METHODS

This study addresses these challenges by analyzing class imbalance in clinical datasets and evaluating common assessment metrics. Data from 139 drug-resistant epilepsy patients across two Institutions were analyzed. Metric behaviors were examined using clinical and simulated data.

RESULTS

Complementary use of Area Under the Receiver Operating Characteristic (AUROC) and Area Under the Precision-Recall Curve (AUPRC) provides an optimal evaluation approach. This must be paired with an analysis of class imbalance and its impact due to significant variations found in clinical datasets.

CONCLUSIONS

The proposed framework offers a comprehensive and reliable method for evaluating machine learning models in epileptogenic zone localization, improving their precision and clinical relevance.

SIGNIFICANCE

Adopting this framework will improve the comparability and multicenter testing of machine learning models in epileptogenic zone localization, enhancing their reliability and ultimately leading to better surgical outcomes for epilepsy patients.

Machine and Deep Learning Methods for Predicting 3D Genome Organization

Three-dimensional (3D) chromatin interactions, such as enhancer-promoter interactions (EPIs), loops, topologically associating domains (TADs), and A/B compartments, play critical roles in a wide range of cellular processes by regulating gene expression. Recent development of chromatin conformation capture technologies has enabled genome-wide profiling of various 3D structures, even with single cells. However, current catalogs of 3D structures remain incomplete and unreliable due to differences in technology, tools, and low data resolution. Machine learning methods have emerged as an alternative to obtain missing 3D interactions and/or improve resolution. Such methods frequently use genome annotation data (ChIP-seq, DNAse-seq, etc.), DNA sequencing information (k-mers and transcription factor binding site (TFBS) motifs), and other genomic properties to learn the associations between genomic features and chromatin interactions. In this review, we discuss computational tools for predicting three types of 3D interactions (EPIs, chromatin interactions, and TAD boundaries) and analyze their pros and cons. We also point out obstacles to the computational prediction of 3D interactions and suggest future research directions.

InsectNet: Real-time identification of insects using an end-to-end machine learning pipeline

Insect pests significantly impact global agricultural productivity and crop quality. Effective integrated pest management strategies require the identification of insects, including beneficial and harmful insects. Automated identification of insects under real-world conditions presents several challenges, including the need to handle intraspecies dissimilarity and interspecies similarity, life-cycle stages, camouflage, diverse imaging conditions, and variability in insect orientation. An end-to-end approach for training deep-learning models, InsectNet, is proposed to address these challenges. Our approach has the following key features: (i) uses a large dataset of insect images collected through citizen science along with label-free self-supervised learning to train a global model, (ii) fine-tuning this global model using smaller, expert-verified regional datasets to create a local insect identification model, (iii) which provides high prediction accuracy even for species with small sample sizes, (iv) is designed to enhance model trustworthiness, and (v) democratizes access through streamlined machine learning operations. This global-to-local model strategy offers a more scalable and economically viable solution for implementing advanced insect identification systems across diverse agricultural ecosystems. We report accurate identification (>96% accuracy) of numerous agriculturally and ecologically relevant insect species, including pollinators, parasitoids, predators, and harmful insects. InsectNet provides fine-grained insect species identification, works effectively in challenging backgrounds, and avoids making predictions when uncertain, increasing its utility and trustworthiness. The model and associated workflows are available through a web-based portal accessible through a computer or mobile device. We envision InsectNet to complement existing approaches, and be part of a growing suite of AI technologies for addressing agricultural challenges.

Data-driven insights into pre-slaughter mortality: Machine learning for predicting high dead on arrival in meat-type ducks

Dead on arrival (DOA) refers to animals, particularly poultry, that die during the pre-slaughter phase. Elevated rates of DOA frequently signify substandard welfare conditions and might stem from multiple causes, resulting in diminished productivity and economic losses. This study included 18,643 truckload entries from 45 farms, encompassing a total of 23,191,809 meat-type ducks sent to a single slaughterhouse in Eastern Thailand between January 2019 and December 2023. The objective of this study was twofold: first, to classify high DOA rates (≥ 0.15%) using several predictors, including season, period of the day, number of ducks per truckload, distance, duration of transportation, age, average body weight, lairage time, and temperature at the lairage area. This classification was performed using machine learning (ML) algorithms such as Least Absolute Shrinkage and Selection Operator (LASSO), Support Vector Machine (SVM), Decision Tree (DT), Random Forests (RF), and Extreme Gradient Boosting (XGBoost). Additionally, several data-sampling techniques, including oversampling, undersampling, Random Over-Sampling Examples (ROSE), and Synthetic Minority Over-sampling Technique (SMOTE), were utilized to address the issue of imbalanced data. Second, to analyze variable importance contributing to the predictive outcomes. The descriptive analysis revealed a mean DOA percentage of 0.14% (range: 0 to 22.46%, SD = 0.49). The results of the high DOA classification indicated that among all models, XGBoost-Up, XGBoost-Down, and RF-Down were the top three models, achieving the highest overall scores in evaluation metrics including Area Under the ROC Curve (AUC), sensitivity, precision, and F1-score. The primary factors contributing to the high predictive performance of the models were the number of ducks per truckload, temperature at the lairage area, and average body weight. Additionally, the duration and distance of transportation, as well as the period of transportation, were secondary factors contributing to the outcome. These factors should be further investigated to minimize losses during the pre-slaughter phase in meat-type ducks. Additionally, considering these factors when managing transportation can help create conditions that reduce duck deaths.

Open-set long-tailed recognition via orthogonal prototype learning and false rejection correction

Learning from data with long-tailed and open-ended distributions is highly challenging. In this work, we propose OLPR, which is a new dual-stream Open-set Long-tailed recognition framework based on orthogonal Prototype learning and false Rejection correction. It consists of a Probabilistic Prediction Learning (PPL) branch and a Distance Metric Learning (DML) branch. The former is used to generate prediction probability for image classification. The latter learns orthogonal prototypes for each class by computing three distance losses, which are the orthogonal prototype loss among all the prototypes, the balanced Softmin distance based cross-entropy loss between each prototype and its corresponding input sample, and the adversarial loss for making the open-set space more compact. Furthermore, for open-set learning, instead of merely relying on binary decisions, we propose an Iterative Clustering Module (ICM) to categorize similar open-set samples and correct the false rejected closed-set samples simultaneously. If a sample is detected as a false rejection, i.e., a sample of the known classes is incorrectly identified as belonging to the unknown classes, we will re-classify the sample to the closest known/closed-set class. We conduct extensive experiments on ImageNet-LT, Places-LT, CIFAR-10/100-LT benchmark datasets, as well as a new long-tailed open-ended dataset that we build. Experimental results demonstrate that OLPR improves over the best competitors by up to 2.2% in terms of overall classification accuracy in closed-set settings, and up to 4% in terms of F-measure in open-set settings, which are very remarkable.

Advancing Laboratory Medicine Practice With Machine Learning: Swift yet Exact

Machine learning (ML) is currently being widely studied and applied in data analysis and prediction in various fields, including laboratory medicine. To comprehensively evaluate the application of ML in laboratory medicine, we reviewed the literature on ML applications in laboratory medicine published between February 2014 and March 2024. A PubMed search using a search string yielded 779 articles on the topic, among which 144 articles were selected for this review. These articles were analyzed to extract and categorize related fields within laboratory medicine, research objectives, specimen types, data types, ML models, evaluation metrics, and sample sizes. Sankey diagrams and pie charts were used to illustrate the relationships between categories and the proportions within each category. We found that most studies involving the application of ML in laboratory medicine were designed to improve efficiency through automation or expand the roles of clinical laboratories. The most common ML models used are convolutional neural networks, multilayer perceptrons, and tree-based models, which are primarily selected based on the type of input data. Our findings suggest that, as the technology evolves, ML will rise in prominence in laboratory medicine as a tool for expanding research activities. Nonetheless, expertise in ML applications should be improved to effectively utilize this technology.

Machine Learning-based Prediction of Blood Stream Infection in Pediatric Febrile Neutropenia

OBJECTIVES

This study aimed to develop machine learning (ML) prediction models for identifying bloodstream infection (BSI) and septic shock (SS) in pediatric patients with cancer who presenting febrile neutropenia (FN) at emergency department (ED) visit.

MATERIALS AND METHODS

A retrospective study was conducted on patients, younger than 18 years of age, who visited a tertiary university-affiliated hospital ED due to FN between January 2004 and August 2022. ML models, based on XGBoost, were developed for BSI and SS prediction.

RESULTS

After applying the exclusion criteria, we identified 4423 FN events during the study period. We identified 195 (4.4%) BSI and 107 (2.4%) SS events. The BSI and SS models demonstrated promising performance, with area under the receiver operating characteristic curve values of 0.87 and 0.88, respectively, which were superior to those of the logistic regression models. Clinical features, including body temperature, some laboratory results, vital signs, and diagnosis of acute myeloblastic leukemia were identified as significant predictors.

CONCLUSIONS

The ML-based prediction models, which use data obtainable at ED visits may be valuable tools for ED physicians to predict BSI or SS.

Does synthetic data augmentation improve the performances of machine learning classifiers for identifying health problems in patient-nurse verbal communications in home healthcare settings?

BACKGROUND

Identifying health problems in audio-recorded patient-nurse communication is important to improve outcomes in home healthcare patients who have complex conditions with increased risks of hospital utilization. Training machine learning classifiers for identifying problems requires resource-intensive human annotation.

OBJECTIVE

To generate synthetic patient-nurse communication and to automatically annotate for common health problems encountered in home healthcare settings using GPT-4. We also examined whether augmenting real-world patient-nurse communication with synthetic data can improve the performance of machine learning to identify health problems.

DESIGN

Secondary data analysis of patient-nurse verbal communication data in home healthcare settings.

METHODS

The data were collected from one of the largest home healthcare organizations in the United States. We used 23 audio recordings of patient-nurse communications from 15 patients. The audio recordings were transcribed verbatim and manually annotated for health problems (e.g., circulation, skin, pain) indicated in the Omaha System Classification scheme. Synthetic data of patient-nurse communication were generated using the in-context learning prompting method, enhanced by chain-of-thought prompting to improve the automatic annotation performance. Machine learning classifiers were applied to three training datasets: real-world communication, synthetic communication, and real-world communication augmented by synthetic communication.

RESULTS

Average F1 scores improved from 0.62 to 0.63 after training data were augmented with synthetic communication. The largest increase was observed using the XGBoost classifier where F1 scores improved from 0.61 to 0.64 (about 5% improvement). When trained solely on either real-world communication or synthetic communication, the classifiers showed comparable F1 scores of 0.62-0.61, respectively.

CONCLUSION

Integrating synthetic data improves machine learning classifiers' ability to identify health problems in home healthcare, with performance comparable to training on real-world data alone, highlighting the potential of synthetic data in healthcare analytics.

CLINICAL RELEVANCE

This study demonstrates the clinical relevance of leveraging synthetic patient-nurse communication data to enhance machine learning classifier performances to identify health problems in home healthcare settings, which will contribute to more accurate and efficient problem identification and detection of home healthcare patients with complex health conditions.

Predictive analytics technique based on hybrid sampling to manage unbalanced data in smart cities

A smart city is deemed smart enough because it has the capability to make decisions on its own. Artificial intelligence needs a lot of data from the physical world to make correct decisions. IoT sensor devices collect data from the surroundings, which is further used for predictive analytics. Collected data may be balanced or imbalanced. Unbalanced data used for decision-making without any pre-processing may lead to ravaging results. This paper proposes a novel predictive analytical technique to manage unbalanced data. A pipeline is designed using Principal Component Analysis (PCA), a hybrid sampling method, and a Machine Learning (ML) prediction method. SMOTE + ENN, a hybrid data balancing method, is used to specify imbalanced data to a balanced state. ML method is applied to form clusters and make predictions over the dataset. A large Smart City IoT dataset having 4,05,184 records has been used in this study. The proposed technique is used to predict the presence of a person in the vicinity of IoT devices. Evaluation parameters such as accuracy, precision, recall, F1-score, and Area Under Curve (AUC)/Receiver Operating Characteristic (ROC) curve are used to evaluate the proposed approach. Accuracy, Precision, Recall, F1-score, and AUC obtained using the proposed technique for cluster 0 are 0.79, 1.0, 0.79, 0.87, and 0.88 and for cluster 1 are 0.86 0.99, 0.86, 0.92, and 0.92, respectively. In view of the encouraging results, the proposed technique may prove to be a good choice to help in decision-making in different application domains in real life.

Normalizing Batch Normalization for Long-Tailed Recognition

In real-world scenarios, the number of training samples across classes usually subjects to a long-tailed distribution. The conventionally trained network may achieve unexpected inferior performance on the rare class compared to the frequent class. Most previous works attempt to rectify the network bias from the data-level or from the classifier-level. Differently, in this paper, we identify that the bias towards the frequent class may be encoded into features, i.e., the rare-specific features which play a key role in discriminating the rare class are much weaker than the frequent-specific features. Based on such an observation, we introduce a simple yet effective approach, normalizing the parameters of Batch Normalization (BN) layer to explicitly rectify the feature bias. To achieve this end, we represent theWeight/Bias parameters of a BN layer as a vector, normalize it into a unit one and multiply the unit vector by a scalar learnable parameter. Through decoupling the direction and magnitude of parameters in BN layer to learn, the Weight/Bias exhibits a more balanced distribution and thus the strength of features becomes more even. Extensive experiments on various long-tailed recognition benchmarks (i.e., CIFAR-10/100-LT, ImageNet-LT and iNaturalist 2018) show that our method outperforms previous state-of-the-arts remarkably.

Anomalous Node Detection in Blockchain Networks Based on Graph Neural Networks

With the rapid development of blockchain technology, fraudulent activities have significantly increased, posing a major threat to the personal assets of blockchain users. The blockchain transaction network formed during user transactions can be represented as a graph consisting of nodes and edges, making it suitable for a graph data structure. Fraudulent nodes in the transaction network are referred to as anomalous nodes. In recent years, the mainstream method for detecting anomalous nodes in graphs has been the use of graph data mining techniques. However, anomalous nodes typically constitute only a small portion of the transaction network, known as the minority class, while the majority of nodes are normal nodes, referred to as the majority class. This discrepancy in sample sizes results in class imbalance data, where models tend to overfit the features of the majority class and neglect those of the minority class. This issue presents significant challenges for traditional graph data mining techniques. In this paper, we propose a novel graph neural network method to overcome class imbalance issues by improving the Graph Attention Network (GAT) and incorporating ensemble learning concepts. Our method combines GAT with a subtree attention mechanism and two ensemble learning methods: Bootstrap Aggregating (Bagging) and Categorical Boosting (CAT), called SGAT-BC. We conducted experiments on four real-world blockchain transaction datasets, and the results demonstrate that SGAT-BC outperforms existing baseline models.

Predictive Modeling of Gene Expression and Localization of DNA Binding Site Using Deep Convolutional Neural Networks

Despite the sequencing revolution, large swaths of the genomes sequenced to date lack any information about the arrangement of transcription factor binding sites on regulatory DNA. Massively Parallel Reporter Assays (MPRAs) have the potential to dramatically accelerate our genomic annotations by making it possible to measure the gene expression levels driven by thousands of mutational variants of a regulatory region. However, the interpretation of such data often assumes that each base pair in a regulatory sequence contributes independently to gene expression. To enable the analysis of this data in a manner that accounts for possible correlations between distant bases along a regulatory sequence, we developed the Deep learning Adaptable Regulatory Sequence Identifier (DARSI). This convolutional neural network leverages MPRA data to predict gene expression levels directly from raw regulatory DNA sequences. By harnessing this predictive capacity, DARSI systematically identifies transcription factor binding sites within regulatory regions at single-base pair resolution. To validate its predictions, we benchmarked DARSI against curated databases, confirming its accuracy in predicting transcription factor binding sites. Additionally, DARSI predicted novel unmapped binding sites, paving the way for future experimental efforts to confirm the existence of these binding sites and to identify the transcription factors that target those sites. Thus, by automating and improving the annotation of regulatory regions, DARSI generates experimentally actionable predictions that can feed iterations of the theory-experiment cycle aimed at reaching a predictive understanding of transcriptional control.

Domain adaptation in small-scale and heterogeneous biological datasets

Machine-learning models are key to modern biology, yet models trained on one dataset are often not generalizable to other datasets from different cohorts or laboratories due to both technical and biological differences. Domain adaptation, a type of transfer learning, alleviates this problem by aligning different datasets so that models can be applied across them. However, most state-of-the-art domain adaptation methods were designed for large-scale data such as images, whereas biological datasets are smaller and have more features, and these are also complex and heterogeneous. This Review discusses domain adaptation methods in the context of such biological data to inform biologists and guide future domain adaptation research. We describe the benefits and challenges of domain adaptation in biological research and critically explore some of its objectives, strengths, and weaknesses. We argue for the incorporation of domain adaptation techniques to the computational biologist's toolkit, with further development of customized approaches.

Challenges in data-driven geospatial modeling for environmental research and practice

Machine learning-based geospatial applications offer unique opportunities for environmental monitoring due to domains and scales adaptability and computational efficiency. However, the specificity of environmental data introduces biases in straightforward implementations. We identify a streamlined pipeline to enhance model accuracy, addressing issues like imbalanced data, spatial autocorrelation, prediction errors, and the nuances of model generalization and uncertainty estimation. We examine tools and techniques for overcoming these obstacles and provide insights into future geospatial AI developments. A big picture of the field is completed from advances in data processing in general, including the demands of industry-related solutions relevant to outcomes of applied sciences.

Sample Size Requirements for Popular Classification Algorithms in Tabular Clinical Data: Empirical Study

BACKGROUND

The performance of a classification algorithm eventually reaches a point of diminishing returns, where the additional sample added does not improve the results. Thus, there is a need to determine an optimal sample size that maximizes performance while accounting for computational burden or budgetary concerns.

OBJECTIVE

This study aimed to determine optimal sample sizes and the relationships between sample size and dataset-level characteristics over a variety of binary classification algorithms.

METHODS

A total of 16 large open-source datasets were collected, each containing a binary clinical outcome. Furthermore, 4 machine learning algorithms were assessed: XGBoost (XGB), random forest (RF), logistic regression (LR), and neural networks (NNs). For each dataset, the cross-validated area under the curve (AUC) was calculated at increasing sample sizes, and learning curves were fit. Sample sizes needed to reach the observed full-dataset AUC minus 2 points (0.02) were calculated from the fitted learning curves and compared across the datasets and algorithms. Dataset-level characteristics, minority class proportion, full-dataset AUC, number of features, type of features, and degree of nonlinearity were examined. Negative binomial regression models were used to quantify relationships between these characteristics and expected sample sizes within each algorithm. A total of 4 multivariable models were constructed, which selected the best-fitting combination of dataset-level characteristics.

RESULTS

Among the 16 datasets (full-dataset sample sizes ranging from 70,000-1,000,000), median sample sizes were 9960 (XGB), 3404 (RF), 696 (LR), and 12,298 (NN) to reach AUC stability. For all 4 algorithms, more balanced classes (multiplier: 0.93-0.96 for a 1% increase in minority class proportion) were associated with decreased sample size. Other characteristics varied in importance across algorithms-in general, more features, weaker features, and more complex relationships between the predictors and the response increased expected sample sizes. In multivariable analysis, the top selected predictors were minority class proportion among all 4 algorithms assessed, full-dataset AUC (XGB, RF, and NN), and dataset nonlinearity (XGB, RF, and NN). For LR, the top predictors were minority class proportion, percentage of strong linear features, and number of features. Final multivariable sample size models had high goodness-of-fit, with dataset-level predictors explaining a majority (66.5%-84.5%) of the total deviance in the data among all 4 models.

CONCLUSIONS

The sample sizes needed to reach AUC stability among 4 popular classification algorithms vary by dataset and method and are associated with dataset-level characteristics that can be influenced or estimated before the start of a research study.

Hybrid bagging and boosting with SHAP based feature selection for enhanced predictive modeling in intrusion detection systems

The novelty and growing sophistication of cyber threats mean that high accuracy and interpretable machine learning models are needed more than ever before for Intrusion Detection and Prevention Systems. This study aims to solve this challenge by applying Explainable AI techniques, including Shapley Additive explanations feature selection, to improve model performance, robustness, and transparency. The method systematically employs different classifiers and proposes a new hybrid method called Hybrid Bagging-Boosting and Boosting on Residuals. Then, performance is taken in four steps: the multistep evaluation of hybrid ensemble learning methods for binary classification and fine-tuning of performance; feature selection using Shapley Additive explanations values retraining the hybrid model for better performance and reducing overfitting; the generalization of the proposed model for multiclass classification; and the evaluation using standard information metrics such as accuracy, precision, recall, and F1-score. Key results indicate that the proposed methods outperform state-of-the-art algorithms, achieving a peak accuracy of 98.47% and an F1 score of 96.19%. These improvements stem from advanced feature selection and resampling techniques, enhancing model accuracy and balancing precision and recall. Integrating Shapley Additive explanations-based feature selection with hybrid ensemble methods significantly boosts the predictive and explanatory power of Intrusion Detection and Prevention Systems, addressing common pitfalls in traditional cybersecurity models. This study paves the way for further research on statistical innovations to enhance Intrusion Detection and Prevention Systems performance.

Entropy-based dynamic ensemble classication algorithm for imbalanced data stream with concept drift

Online imbalanced learning is an emerging topic that combines the challenges of class imbalance and concept drift. However, current works account for issues of class imbalance and concept drift. And only few works have considered these issues simultaneously. To this end, this paper proposes an entropy-based dynamic ensemble classification algorithm (EDAC) to consider data streams with class imbalance and concept drift simultaneously. First, to address the problem of imbalanced learning in training data chunks arriving at different times, EDAC adopts an entropy-based balanced strategy. It divides the data chunks into multiple balanced sample pairs based on the differences in the information entropy between classes in the sample data chunk. Additionally, we propose a density-based sampling method to improve the accuracy of classifying minority class samples into high quality samples and common samples via the density of similar samples. In this manner high quality and common samples are randomly selected for training the classifier. Finally, to solve the issue of concept drift, EDAC designs and implements an ensemble classifier that uses a self-feedback strategy to determine the initial weight of the classifier by adjusting the weight of the sub-classifier according to the performance on the arrived data chunks. The experimental results demonstrate that EDAC outperforms five state-of-the-art algorithms considering four synthetic and one real-world data streams.

Wound imaging software and digital platform to assist review of surgical wounds using patient smartphones: The development and evaluation of artificial intelligence (WISDOM AI study)

INTRODUCTION

Surgical patients frequently experience post-operative complications at home. Digital remote monitoring of surgical wounds via image-based systems has emerged as a promising solution for early detection and intervention. However, the increased clinician workload from reviewing patient-submitted images presents a challenge. This study utilises artificial intelligence (AI) to prioritise surgical wound images for clinician review, aiming to efficiently manage workload.

METHODS AND ANALYSIS

Conducted from September 2023 to March 2024, the study phases included compiling a training dataset of 37,974 images, creating a testing set of 3,634 images, developing an AI algorithm using 'You Only Look Once' models, and conducting prospective tests compared against clinical nurse specialists' evaluations. The primary objective was to validate the AI's sensitivity in prioritising wound reviews, alongside assessing intra-rater reliability. Secondary objectives focused on specificity, positive predictive value (PPV), and negative predictive value (NPV) for various wound features.

RESULTS

The AI demonstrated a sensitivity of 89%, exceeding the target of 85% and proving effective in identifying cases requiring priority review. Intra-rater reliability was perfect, achieving 100% consistency in repeated assessments. Observations indicated variations in detecting wound characteristics across different skin tones; sensitivity was notably lower for incisional separation and discolouration in darker skin tones. Specificity remained high overall, with some results favouring darker skin tones. The NPV were similar for both light and dark skin tones. However, the NPV was slightly higher for dark skin tones at 95% (95% CI: 93%-97%) compared to 91% (95% CI: 87%-92%) for light skin tones. Both PPV and NPV varied, especially in identifying sutures or staples, indicating areas needing further refinement to ensure equitable accuracy.

CONCLUSION

The AI algorithm not only met but surpassed the expected sensitivity for identifying priority cases, showing high reliability. Nonetheless, the disparities in performance across skin tones, especially in recognising certain wound characteristics like discolouration or incisional separation, underline the need for ongoing training and adaptation of the AI to ensure fairness and effectiveness across diverse patient groups.

Deep learning analysis of fMRI data for predicting Alzheimer's Disease: A focus on convolutional neural networks and model interpretability

The early detection of Alzheimer's Disease (AD) is thought to be important for effective intervention and management. Here, we explore deep learning methods for the early detection of AD. We consider both genetic risk factors and functional magnetic resonance imaging (fMRI) data. However, we found that the genetic factors do not notably enhance the AD prediction by imaging. Thus, we focus on building an effective imaging-only model. In particular, we utilize data from the Alzheimer's Disease Neuroimaging Initiative (ADNI), employing a 3D Convolutional Neural Network (CNN) to analyze fMRI scans. Despite the limitations posed by our dataset (small size and imbalanced nature), our CNN model demonstrates accuracy levels reaching 92.8% and an ROC of 0.95. Our research highlights the complexities inherent in integrating multimodal medical datasets. It also demonstrates the potential of deep learning in medical imaging for AD prediction.

Tremor Severity and Operative Parameters Predict Imbalance in Patients Undergoing Focused Ultrasound Thalamotomy

BACKGROUND

Imbalance is the most commonly reported side effect following focused ultrasound (FUS) thalamotomy for essential tremor (ET). It remains unknown which patients are more likely to develop imbalance following FUS treatment.

OBJECTIVE

To identify preoperative and treatment-related sonication parameters that are predictive of imbalance following FUS treatment.

METHODS

We retrospectively collected demographic data, preoperative Fahn-Tolosa-Marin Clinical Rating Scale for Tremor (FTM) scores and FUS treatment parameters in patients undergoing FUS thalamotomy for treatment of ET. The presence of imbalance was evaluated at several discrete time-points with up to 4 years of follow-up. Multiple machine learning classifiers were built and evaluated, aiming to maximize accuracy while minimizing feature set.

RESULTS

Of the 297 patients identified, the presence of imbalance peaked at 1 week following operation at 79%. This declined rapidly with 29% reporting imbalance at 3 months, and only 15% at 4 years. At 1 week, total preoperative FTM scores and Maximum Energy delivered in FUS could predict the presence of imbalance at 92.8% accuracy. At 3 months, the total preoperative FTM scores and maximum power delivered could predict the presence of imbalance with 90.6% accuracy. Post-operative lesion size and extent into thalamic nuclei, internal capsule, and subthalamic regions were identified as likely key underlying drivers of these predictors.

CONCLUSIONS

A machine learning model based on preoperative tremor scores and maximum energy/power delivered predicted the development of short-term imbalance and long-term imbalance following FUS thalamotomy.

A multi-layer neural network approach for the stability analysis of the Hepatitis B model

In the present study, we explore the dynamics of Hepatitis B virus infection, a significant global health issue, through a newly developed dynamics system. This model is distinguished by its inclusion of asymptomatic carriers and the impact of vaccination and treatment strategies. Compared to Hepatitis A, Hepatitis B poses a more serious health risk, with some cases progressing from acute to chronic. To diagnose and predict disease recurrence, the basic reproduction number (R0) is calculated. We investigate the stability of the disease's dynamics under different conditions, using the Lyapunov function to confirm our model's global stability. Our findings highlight the relevance of vaccination and early treatment in reducing Hepatitis B virus spread, making them a useful tool for public health efforts aiming at eradicating Hepatitis B virus. In our research, we investigate the dynamics of a specific model that is characterized by a system of differential equations. This work uses deep neural networks (DNNs) technique to improve model accuracy, proving the use of DNNs in epidemiological modeling. Additionally, we want to find the curves that suit the target solutions with the minimum residual errors. The simulations we conducted demonstrate our methodology's capability to accurately predict the behavior of systems across various conditions. We rigorously test the solutions obtained via the DNNs by comparing them to benchmark solutions and undergoing stages of testing, validation, and training. To determine the accuracy and reliability of our approach, we perform a series of analyses, including convergence studies, error distribution evaluations, regression analyses, and detailed curve fitting for each equation.

Prediction of 90 day readmission in heart failure with preserved ejection fraction by interpretable machine learning

AIMS

Certain critical risk factors of heart failure with preserved ejection fraction (HFpEF) patients were significantly different from those of heart failure with reduced ejection fraction (HFrEF) patients, resulting in the limitations of existing predictive models in real-world situations. This study aimed to develop a machine learning model for predicting 90 day readmission for HFpEF patients.

METHODS AND RESULTS

Data were extracted from electronic health records from 1 August 2020 to 1 August 2021 and follow-up records of patients with HFpEF within 3 months after discharge. Feature extraction was performed by univariate analysis combined with the least absolute shrinkage and selection operator (LASSO) algorithms. Machine learning models like eXtreme Gradient Boosting (XGBoost), random forest, neural network and logistic regression were adopted to construct models. The discrimination and calibration of each model were compared, and the Shapley Additive exPlanations (SHAP) method was used to explore the interpretability of the model. The cohort included 746 patients, of whom 103 (13.8%) were readmitted within 90 days. XGBoost owned the best performance [area under the curve (AUC) = 0.896, precision-recall area under the curve (PR-AUC) = 0.868, sensitivity = 0.817, specificity = 0.837, balanced accuracy = 0.827]. The Kolmogorov-Smirnov (KS) statistic was 0.694 at 0.468 in the XGBoost model. SHAP identified the top 12 risk features, including activities of daily living (ADL), left atrial dimension (LAD), left ventricular end-diastolic diameter (LVDD), shortness, nitrates, length of stay, nutritional risk, fall risk, accompanied by other symptoms, educational level, anticoagulants and edema.

CONCLUSIONS

Our model could help medical agencies achieve the early identification of 90 day readmission risk in HFpEF patients and reveal risk factors that provide valuable insights for treatments.

Comparing AI/ML approaches and classical regression for predictive modeling using large population health databases: Applications to COVID-19 case prediction

Background

Research comparing artificial intelligence and machine learning (AI/ML) methods with classical statistical methods applied to large population health databases is limited.

Objectives

This retrospective cohort study aimed to compare the predictive performance of AI/ML algorithms against conventional multivariate logistic regression models using linked health administrative data.

Methods

Using Ontario's population health databases, we created a cohort of residents of the city of Ottawa, Ontario, who underwent a PCR test for COVID-19 between March 10, 2020, and May 13, 2021. Using demographic, socio-economic and health data (including COVID-19 PCR test results and available, symptom data), we developed predictive models for the purpose of COVID-19 case identification using the following approaches: classical multivariate logistic regression (LR); deep neural network (DNN); random forest (RF); and gradient boosting trees (GBT). Model performance comparisons were made using the area under the curve (AUC) swarm plot for 10-fold cross-validation.

Results

The cohort consisted of n = 351,248 Ottawa residents tested for COVID-19 during the study period. Among whom, a total of n = 883,879 unique COVID-19 tests were performed (2.6 % positive test results). Inclusion of COVID-19 symptoms data in the analysis improved model performance and variable predictive value across all tested models (p < 0.0001), with the 10-fold cross-validation AUC increasing to near or over 0.7 in all models when symptoms data were included. In various pairwise comparisons, the GBT method had the highest predictive ability (AUC = 0.796 ± 0.017), significantly outperforming multivariate logistic regression and the other AI/ML approaches.

Conclusions

Conventional multivariate regression-based models are better than some and worse than other machine learning algorithms to provide good predictive accuracy in a moderate dataset with a reasonable number of features. However, whenever possible, the AI/ML GBT approach should be considered.

A framework for categorizing sources of uncertainty in in silico toxicology methods: Considerations for chemical toxicity predictions

Improving regulatory confidence and acceptance of in silico toxicology methods for chemical risk assessment requires assessment of associated uncertainties. Therefore, there is a need to identify and systematically categorize sources of uncertainty relevant to the methods and their predictions. In the present study, we analyzed studies that have characterized sources of uncertainty across commonly applied in silico toxicology methods. Our study reveals variations in the kind and number of uncertainty sources these studies cover. Additionally, the studies use different terminologies to describe similar sources of uncertainty; consequently, a majority of the sources considerably overlap. Building on an existing framework, we developed a new uncertainty categorization framework that systematically consolidates and categorizes the different uncertainty sources described in the analyzed studies. We then illustrate the importance of the developed framework through a case study involving QSAR prediction of the toxicity of five compounds, as well as compare it with the QSAR Assessment Framework (QAF). The framework can provide a structured (and potentially more transparent) understanding of where the uncertainties reside within in silico toxicology models and model predictions, thus promoting critical reflection on appropriate strategies to address the uncertainties.

A knowledge-transfer-based approach for combining ordinal regression and medical scoring system in the early prediction of sepsis with electronic health records

OBJECTIVE

The prediction of sepsis, especially early diagnosis, has received a significant attention in biomedical research. In order to improve current medical scoring system and overcome the limitations of class imbalance and sample size of local EHR (electronic health records), we propose a novel knowledge-transfer-based approach, which combines a medical scoring system and an ordinal logistic regression model.

MATERIALS AND METHODS

Medical scoring systems (i.e. NEWS, SIRS and QSOFA) are generally robust and useful for sepsis diagnosis. With local EHR, machine-learning-based methods have been widely used for building prediction models/methods, but they are often impacted by class imbalance and sample size. Knowledge distillation and knowledge transfer have recently been proposed as a combination approach for improving the prediction performance and model generalization. In this study, we developed a novel knowledge-transfer-based method for combining a medical scoring system (after a proposed score transformation) and an ordinal logistic regression model. We mathematically confirmed that it was equivalent to a specific form of the weighted regression. Furthermore, we theoretically explored its effectiveness in the scenario of class imbalance.

RESULTS

For the local dataset and the MIMIC-IV dataset, the VUS (the volume under the multi-dimensional ROC surface, a generalization measure of AUC-ROC for ordinal categories) of the knowledge-transfer-based model (ORNEWS) based on the NEWS scoring system were 0.384 and 0.339, respectively, while the VUS of the traditional ordinal regression model (OR) were 0.352 and 0.322, respectively. Consistent analysis results were also observed for the knowledge-transfer-based models based on the SIRS/QSOFA scoring systems in the ordinal scenarios. Additionally, the predicted probabilities and the binary classification ROC curves of the knowledge-transfer-based models indicated that this approach enhanced the predicted probabilities for the minority classes while reducing the predicted probabilities for the majority classes, which improved AUCs/VUSs on imbalanced data.

DISCUSSION

Knowledge transfer, which combines a medical scoring system and a machine-learning-based model, improves the prediction performance for early diagnosis of sepsis, especially in the scenarios of class imbalance and limited sample size.

Prediction models for COVID-19 disease outcomes

SARS-CoV-2 has caused over 6.9 million deaths and continues to produce lasting health consequences. COVID-19 manifests broadly from no symptoms to death. In a retrospective cross-sectional study, we developed personalized risk assessment models that predict clinical outcomes for individuals with COVID-19 and inform targeted interventions. We sequenced viruses from SARS-CoV-2-positive nasopharyngeal swab samples between July 2020 and July 2022 from 4450 individuals in Missouri and retrieved associated disease courses, clinical history, and urban-rural classification. We integrated this data to develop machine learning-based predictive models to predict hospitalization, ICU admission, and long COVID.The mean age was 38.3 years (standard deviation = 21.4) with 55.2% (N = 2453) females and 44.8% (N = 1994) males (not reported, N = 4). Our analyses revealed a comprehensive set of predictors for each outcome, encompassing human, environment, and virus genome-wide genetic markers. Immunosuppression, cardiovascular disease, older age, cardiac, gastrointestinal, and constitutional symptoms, rural residence, and specific amino acid substitutions were associated with hospitalization. ICU admission was associated with acute respiratory distress syndrome, ventilation, bacterial co-infection, rural residence, and non-wild type SARS-CoV-2 variants. Finally, long COVID was associated with hospital admission, ventilation, and female sex.Overall, we developed risk assessment models that offer the capability to identify patients with COVID-19 necessitating enhanced monitoring or early interventions. Of importance, we demonstrate the value of including key elements of virus, host, and environmental factors to predict patient outcomes, serving as a valuable platform in the field of personalized medicine with the potential for adaptation to other infectious diseases.

Merge Loss Calculation Method for Highly Imbalanced Data Multiclass Classification

In real classification scenarios, the number distribution of modeling samples is usually out of proportion. Most of the existing classification methods still face challenges in comprehensive model performance for imbalanced data. In this article, a novel theoretical framework is proposed that establishes a proportion coefficient independent of the number distribution of modeling samples and a general merge loss calculation method independent of class distribution. The loss calculation method of the imbalanced problem focuses on both the global and batch sample levels. Specifically, the loss function calculation introduces the true-positive rate (TPR) and the false-positive rate (FPR) to ensure the independence and balance of loss calculation for each class. Based on this, global and local loss weight coefficients are generated from the entire dataset and batch dataset for the multiclass classification problem, and a merge weight loss function is calculated after unifying the weight coefficient scale. Furthermore, the designed loss function is applied to different neural network models and datasets. The method shows better performance on imbalanced datasets than state-of-the-art methods.

Discovery of dual-specificity tyrosine-phosphorylation-regulated kinase 1A (DYRK1A) inhibitors using an artificial intelligence model and their effects on tau and tubulin dynamics

The dual-specificity tyrosine-phosphorylation-regulated kinase 1 A (DYRK1A) presents a promising therapeutic target for neurological diseases. However, current inhibitors lack selectivity, which can lead to unexpected side effects and increase the difficulty of studying DYRK1A. Therefore, identifying selective inhibitors targeting DYRK1A is essential for reducing side effects and facilitating neurological disease research. This study aimed to discover DYRK1A inhibitors through a screening pipeline incorporating a deep neural network (DNN) model. Herein, we report an optimized model with an accuracy of 0.93 on a testing set. The pipeline was then performed to identify potential DYRK1A inhibitors from the National Cancer Institute (NCI) library. Four novel DYRK1A inhibitors were identified, and compounds NSC657702 and NSC31059 were noteworthy for their potent inhibition, with IC50 values of 50.9 and 39.5 nM, respectively. NSC31059 exhibited exceptional selectivity across 70 kinases. The compounds also significantly reduced DYRK1A-induced tau phosphorylation at key sites associated with the pathology of neurodegenerative diseases. Moreover, they promoted tubulin polymerization, suggesting a role in microtubule stabilization. Cytotoxicity assessments further confirmed the neuronal safety of the compounds. Together, the results demonstrated a promising screening pipeline and novel DYRK1A inhibitors as candidates for further optimization and development.

Empirical investigation of multi-source cross-validation in clinical ECG classification

Traditionally, machine learning-based clinical prediction models have been trained and evaluated on patient data from a single source, such as a hospital. Cross-validation methods can be used to estimate the accuracy of such models on new patients originating from the same source, by repeated random splitting of the data. However, such estimates tend to be highly overoptimistic when compared to accuracy obtained from deploying models to sources not represented in the dataset, such as a new hospital. The increasing availability of multi-source medical datasets provides new opportunities for obtaining more comprehensive and realistic evaluations of expected accuracy through source-level cross-validation designs. In this study, we present a systematic empirical evaluation of standard K-fold cross-validation and leave-source-out cross-validation methods in a multi-source setting. We consider the task of electrocardiogram based cardiovascular disease classification, combining and harmonizing the openly available PhysioNet/CinC Challenge 2021 and the Shandong Provincial Hospital datasets for our study. Our results show that K-fold cross-validation, both on single-source and multi-source data, systemically overestimates prediction performance when the end goal is to generalize to new sources. Leave-source-out cross-validation provides more reliable performance estimates, having close to zero bias though larger variability. The evaluation highlights the dangers of obtaining misleading cross-validation results on medical data and demonstrates how these issues can be mitigated when having access to multi-source data.

Detection of mussels contaminated with cadmium by near-infrared reflectance spectroscopy based on RELS-TSVM

Eating mussels contaminated with cadmium (Cd) can seriously harm health. In this study, a non-destructive and rapid detection method for Cd-contaminated mussels based on near-infrared reflectance spectroscopy was studied. The spectral data of Cd-contaminated and non-contaminated mussels were collected in the range of 950-1700 nm. The model based on a robust energy-based least squares twin support vector machine (RELS-TSVM) was established to detect Cd-contaminated mussels. The influence of parameters on the RELS-TSVM model was analyzed, and the most suitable parameters were determined. The average accuracy of the proposed RELS-TSVM model in detecting Cd-contaminated mussels reached 99.92%, which was better than other twin support vector machine-derived models. For test datasets with different kinds of spectral noises (Gaussian noise, baseline shift, stray light, and wavelength shift), the RELS-TSVM model had a high robustness for noise disturbance. The results show that near-infrared spectroscopy combined with the RELS-TSVM model can realize the detection of Cd-contaminated mussels, which can provide technical support for the monitoring of heavy metals in shellfish. PRACTICAL APPLICATION: The method of detecting Cd-contaminated mussels by the NIRS has important practical significance for ensuring the safety of consumers. It provides a new way for the quality assessment and safety detection of shellfish and provides a technical basis for the marine environment assessment and management.

Bootstrap each lead's latent: A novel method for self-supervised learning of multilead electrocardiograms

BACKGROUND AND OBJECTIVE

Electrocardiogram (ECG) is one of the most important diagnostic tools for cardiovascular diseases (CVDs). Recent studies show that deep learning models can be trained using labeled ECGs to achieve automatic detection of CVDs, assisting cardiologists in diagnosis. However, the deep learning models heavily rely on labels in training, while manual labeling is costly and time-consuming. This paper proposes a new self-supervised learning (SSL) method for multilead ECGs: bootstrap each lead's latent (BELL) to reduce the reliance and boost model performance in various tasks, especially when training data are insufficient.

METHOD

BELL is a variant of the well-known bootstrap your own latent (BYOL). The BELL aims to learn prior knowledge from unlabeled ECGs by pretraining, benefitting downstream tasks. It leverages the characteristics of multilead ECGs. First, BELL uses the multiple-branch skeleton, which is more effective in processing multilead ECGs. Moreover, it proposes intra-lead and inter-lead mean square error (MSE) to guide pretraining, and their fusion can result in better performances. Additionally, BELL inherits the main advantage of the BYOL: No negative pair is used in pretraining, making it more efficient.

RESULTS

In most cases, BELL surpasses previous works in the experiments. More importantly, the pretraining improves model performances by 0.69% ∼ 8.89% in downstream tasks when only 10% of training data are available. Furthermore, BELL shows excellent adaptability to uncurated ECG data from a real-world hospital. Only slight performance degradation occurs (<1% in most cases) when using these data.

CONCLUSION

The results suggest that the BELL can alleviate the reliance on manual ECG labels from cardiologists, a critical bottleneck of the current deep learning-based models. In this way, the BELL can also help deep learning extend its application on automatic ECG analysis, reducing the cardiologists' burden in real-world diagnosis.

Harmonization for Parkinson's Disease Multi-Dataset T1 MRI Morphometry Classification

Classification of disease and healthy volunteer cohorts provides a useful clinical alternative to traditional group statistics due to individualized, personalized predictions. Classifiers for neurodegenerative disease can be trained on structural MRI morphometry, but require large multi-scanner datasets, introducing confounding batch effects. We test ComBat, a common harmonization model, in an example application to classify subjects with Parkinson's disease from healthy volunteers and identify common pitfalls, including data leakage. We used a multi-dataset cohort of 372 subjects (216 with Parkinson's disease, 156 healthy volunteers) from 11 identified scanners. We extracted both FreeSurfer and the determinant of Jacobian morphometry to compare single-scanner and multi-scanner classification pipelines. We confirm the presence of batch effects by running single scanner classifiers which could achieve wildly divergent AUCs on scanner-specific datasets (mean:0.651 ± 0.144). Multi-scanner classifiers that considered neurobiological batch effects between sites could easily achieve a test AUC of 0.902, though pipelines that prevented data leakage could only achieve a test AUC of 0.550. We conclude that batch effects remain a major issue for classification problems, such that even impressive single-scanner classifiers are unlikely to generalize to multiple scanners, and that solving for batch effects in a classifier problem must avoid circularity and reporting overly optimistic results.

Discriminating Myalgic Encephalomyelitis/Chronic Fatigue Syndrome and comorbid conditions using metabolomics in UK Biobank

BACKGROUND

Diagnosing complex illnesses like Myalgic Encephalomyelitis/Chronic Fatigue Syndrome (ME/CFS) is complicated due to the diverse symptomology and presence of comorbid conditions. ME/CFS patients often present with multiple health issues, therefore, incorporating comorbidities into research can provide a more accurate understanding of the condition's symptomatology and severity, to better reflect real-life patient experiences.

METHODS

We performed association studies and machine learning on 1194 ME/CFS individuals with blood plasma nuclear magnetic resonance (NMR) metabolomics profiles, and seven exclusive comorbid cohorts: hypertension (n = 13,559), depression (n = 2522), asthma (n = 6406), irritable bowel syndrome (n = 859), hay fever (n = 3025), hypothyroidism (n = 1226), migraine (n = 1551) and a non-diseased control group (n = 53,009).

RESULTS

We present a lipoprotein perspective on ME/CFS pathophysiology, highlighting gender-specific differences and identifying overlapping associations with comorbid conditions, specifically surface lipids, and ketone bodies from 168 significant individual biomarker associations. Additionally, we searched for, trained, and optimised a machine learning algorithm, resulting in a predictive model using 19 baseline characteristics and nine NMR biomarkers which could identify ME/CFS with an AUC of 0.83 and recall of 0.70. A multi-variable score was subsequently derived from the same 28 features, which exhibited ~2.5 times greater association than the top individual biomarker.

CONCLUSIONS

This study provides an end-to-end analytical workflow that explores the potential clinical utility that association scores may have for ME/CFS and other difficult to diagnose conditions.

The Impact of the SMOTE Method on Machine Learning and Ensemble Learning Performance Results in Addressing Class Imbalance in Data Used for Predicting Total Testosterone Deficiency in Type 2 Diabetes Patients

BACKGROUND AND OBJECTIVE

Diabetes Mellitus is a long-term, multifaceted metabolic condition that necessitates ongoing medical management. Hypogonadism is a syndrome that is a clinical and/or biochemical indicator of testosterone deficiency. Cross-sectional studies have reported that 20-80.4% of all men with Type 2 diabetes have hypogonadism, and Type 2 diabetes is related to low testosterone. This study presents an analysis of the use of ML and EL classifiers in predicting testosterone deficiency. In our study, we compared optimized traditional ML classifiers and three EL classifiers using grid search and stratified k-fold cross-validation. We used the SMOTE method for the class imbalance problem.

METHODS

This database contains 3397 patients for the assessment of testosterone deficiency. Among these patients, 1886 patients with Type 2 diabetes were included in the study. In the data preprocessing stage, firstly, outlier/excessive observation analyses were performed with LOF and missing value analyses were performed with random forest. The SMOTE is a method for generating synthetic samples of the minority class. Four basic classifiers, namely MLP, RF, ELM and LR, were used as first-level classifiers. Tree ensemble classifiers, namely ADA, XGBoost and SGB, were used as second-level classifiers.

RESULTS

After the SMOTE, while the diagnostic accuracy decreased in all base classifiers except ELM, sensitivity values increased in all classifiers. Similarly, while the specificity values decreased in all classifiers, F1 score increased. The RF classifier gave more successful results on the base-training dataset. The most successful ensemble classifier in the training dataset was the ADA classifier in the original data and in the SMOTE data. In terms of the testing data, XGBoost is the most suitable model for your intended use in evaluating model performance. XGBoost, which exhibits a balanced performance especially when the SMOTE is used, can be preferred to correct class imbalance.

CONCLUSIONS

The SMOTE is used to correct the class imbalance in the original data. However, as seen in this study, when the SMOTE was applied, the diagnostic accuracy decreased in some models but the sensitivity increased significantly. This shows the positive effects of the SMOTE in terms of better predicting the minority class.

Benchmarking recent computational tools for DNA-binding protein identification

Identification of DNA-binding proteins (DBPs) is a crucial task in genome annotation, as it aids in understanding gene regulation, DNA replication, transcriptional control, and various cellular processes. In this paper, we conduct an unbiased benchmarking of 11 state-of-the-art computational tools as well as traditional tools such as ScanProsite, BLAST, and HMMER for identifying DBPs. We highlight the data leakage issue in conventional datasets leading to inflated performance. We introduce new evaluation datasets to support further development. Through a comprehensive evaluation pipeline, we identify potential limitations in models, feature extraction techniques, and training methods, and recommend solutions regarding these issues. We show that combining the predictions of the two best computational tools with BLAST-based prediction significantly enhances DBP identification capability. We provide this consensus method as user-friendly software. The datasets and software are available at https://github.com/Rafeed-bot/DNA_BP_Benchmarking.

Applications of Machine Learning on Electronic Health Record Data to Combat Antibiotic Resistance

There is growing excitement about the clinical use of artificial intelligence and machine learning (ML) technologies. Advancements in computing and the accessibility of ML frameworks enable researchers to easily train predictive models using electronic health record data. However, several practical factors must be considered when employing ML on electronic health record data. We provide a primer on ML and approaches commonly taken to address these challenges. To illustrate how these approaches have been applied to address antimicrobial resistance, we review the use of electronic health record data to construct ML models for predicting pathogen carriage or infection, optimizing empiric therapy, and aiding antimicrobial stewardship tasks. ML shows promise in promoting the appropriate use of antimicrobials, although clinical deployment is limited. We conclude by describing the potential dangers of, and barriers to, implementation of ML models in the clinic.

FLEX-SMOTE: Synthetic over-sampling technique that flexibly adjusts to different minority class distributions

Class imbalance is a challenge that affects the prediction rate on a minority class. To remedy this problem, various SMOTEs (synthetic minority over-sampling techniques) have been designed to populate synthetic minority instances. Some SMOTEs operate on the border of a minority class, while others concentrate on the class core. Unfortunately, it is difficult to put the right SMOTE to the right dataset because distributions of classes are varied and might not be obvious. This paper proposes a new technique, called FLEX-SMOTE, that is flexible enough to be used with all sorts of datasets. The key idea is that an over-sampled region is selected based on the characteristics of minority classes. This approach is based on a density function that is used to describe the distributions of minority classes. Herein, we have included experimental results showing that FLEX-SMOTE can significantly improve the predictive performance of a minority class.

Data-centric approach for instance segmentation in optical waste sorting

Computer vision systems have been integrated into facilities dealing with the sorting of household waste. This solution allows for the sorting efficiency improvement and cost reduction. However, challenges associated with the poor annotation quality of existing waste segmentation datasets, unsuitable environment for recognition on a conveyor belt, or limited data for creating an effective and cost-efficient sorting system using visible range cameras significantly limit the application efficiency of computer vision systems. In this article, we report on the data-centric pipeline for enhancing the precision of predictions in multiclass household waste segmentation on a conveyor belt. In particular, we have demonstrated that by employing a pseudo-annotation approach combined with an object-based data augmentation algorithm, it is possible to train a model on a set of 'simple' images and achieve satisfactory results when estimating the model on a set of 'complex' images. We collected and prepared the dataset consisting of 5 k manually labeled data and additionally 10 k pseudo-labeled data by object-based augmentation. The proposed pipeline incorporates data balancing, transfer learning, and pseudo-labeling to improve the mean Average Precision (mAP) of the YOLOV8 segmentation model from 67 % to 83 % for 'simple' use case scenarios and from 42 % to 59 % or 'complex' industrial solutions.

Using Atrial Fibrillation Burden Trends and Machine Learning to Predict Near-Term Risk of Cardiovascular Hospitalization

BACKGROUND

Atrial fibrillation is associated with an increased risk of cardiovascular hospitalization (CVH), which may be triggered by changes in daily burden. Machine learning of dynamic trends in atrial fibrillation burden, as measured by insertable cardiac monitors (ICMs), may be useful in predicting near-term CVH.

METHODS

Using Optum's deidentified Clinformatics Data Mart Database (2007-2019), linked with the Medtronic CareLink ICM database, we identified patients with >1 days of ICM-detected atrial fibrillation. ICM-detected diagnostic parameters were transformed into simple moving averages over different periods for daily follow-up. A diagnostic trend was defined as the comparison of 2 simple moving averages of different periods for each diagnostic parameter. CVH was defined as any hospital, emergency department, or ambulatory surgical center encounter with a cardiovascular diagnosis-related group or diagnosis code. Machine learning was used to determine which diagnostic trends could best predict patient risk 5 days before CVH.

RESULTS

A total of 2616 patients with ICMs met the inclusion criteria (71±11 years; 55% male). Among them, 1998 (76%) had a planned or unplanned CVH over 605 363 days. Machine learning revealed distinct groups: (A) sinus rhythm (reference), (B) below-average burden, (C) above-average burden, and (D) above-average burden with decreasing patient activity. The relative risk was increased in all groups versus the reference (B, 4.49 [95% CI, 3.74-5.40]; C, 8.41 [95% CI, 7.00-10.11]; D, 11.15 [95% CI, 9.10-13.65]), including a 21% increase in CVH detection over prespecified burden thresholds of duration (≥1 hour) and quantity (≥5%). The area under the receiver operating characteristic curve increased from 0.55 when using hourly burden amounts to 0.66 when using burden trends and decreasing patient activity (P<0.001), a 20% increase in predictive power.

CONCLUSIONS

Trends in atrial fibrillation were strongly associated with near-term CVH, especially above-average burden coupled with low patient activity. This approach could provide actionable information to guide treatment and reduce CVH.

Development of fully automated models for staging liver fibrosis using non-contrast MRI and artificial intelligence: a retrospective multicenter study

Background

Accurate staging of liver fibrosis (LF) is essential for clinical management in chronic liver disease. While non-contrast MRI (NC-MRI) yields valuable information for liver assessment, its effectiveness in predicting LF remains underexplored. This study aimed to develop and validate artificial intelligence (AI)-powered models utilizing NC-MRI for staging LF.

Methods

A total of 1726 patients from Shengjing Hospital of China Medical University, registered between October 2003 and October 2022, were retrospectively collected, and divided into development (n = 1208) and internal test (n = 518) cohorts. An external test cohort consisting of 337 individuals from six centers, registered between June 2015 and November 2022, were also included. All participants underwent NC-MRI (T1-weighted imaging, T1WI; and T2-fat-suppressed imaging, T2FS) and liver biopsies. Two classification models (CMs), named T1 and T2FS, were trained on respective image types using 3D contextual transformer networks and evaluated on both test cohorts. Additionally, three CMs-Clinic, Image, and Fusion-were developed using clinical features, T1 and T2FS scores, and their integration via logistic regression. Classification effectiveness of CMs was assessed using the area under the receiver operating characteristic curve (AUC). A comparison was conducted between the optimal models (OMs) with highest AUC and other methods (transient elastography, five serum biomarkers, and six radiologists).

Findings

Fusion models (i.e., OM) yielded the highest AUC among the CMs, achieving AUCs of 0.810 for significant fibrosis, 0.881 for advanced fibrosis, and 0.918 for cirrhosis in the internal test cohort, and 0.808, 0.868, and 0.925, respectively, in the external test cohort. The OMs demonstrated superior performance in AUC, significantly surpassing transient elastography (only for staging ≥ F2 and ≥ F3 grades), serum biomarkers, and three junior radiologists for staging LF. Radiologists, with the aid of the OMs, can achieve a higher AUC in LF assessment.

Interpretation

AI-powered models utilizing NC-MRI, including T1WI and T2FS, accurately stage LF.

Funding

National Natural Science Foundation of China (No. 82071885); General Program of the Liaoning Provincial Department of Education (LJKMZ20221160); Liaoning Province Science and Technology Joint Plan (2023JH2/101700127); the Leading Young Talent Program of Xingliao Yingcai in Liaoning Province (XLYC2203037).

Integrating multi-task and cost-sensitive learning for predicting mortality risk of chronic diseases in the elderly using real-world data

BACKGROUND AND OBJECTIVE

Real-world data encompass population diversity, enabling insights into chronic disease mortality risk among the elderly. Deep learning excels on large datasets, offering promise for real-world data. However, current models focus on single diseases, neglecting comorbidities prevalent in patients. Moreover, mortality is infrequent compared to illness, causing extreme class imbalance that impedes reliable prediction. We aim to develop a deep learning framework that accurately forecasts mortality risk from real-world data by addressing comorbidities and class imbalance.

METHODS

We integrated multi-task and cost-sensitive learning, developing an enhanced deep neural network architecture that extends multi-task learning to predict mortality risk across multiple chronic diseases. Each patient cohort with a chronic disease was assigned to a separate task, with shared lower-level parameters capturing inter-disease complexities through distinct top-level networks. Cost-sensitive functions were incorporated to ensure learning of positive class characteristics for each task and achieve accurate prediction of the risk of death from multiple chronic diseases.

RESULTS

Our study covers 15 prevalent chronic diseases and is experimented with real-world data from 482,145 patients (including 9,516 deaths) in Shenzhen, China. The proposed model is compared with six models including three machine learning models: logistic regression, XGBoost, and CatBoost, and three state-of-the-art deep learning models: 1D-CNN, TabNet, and Saint. The experimental results show that, compared with the other compared algorithms, MTL-CSDNN has better prediction results on the test set (ACC=0.99, REC=0.99, PRAUC=0.97, MCC=0.98, G-means = 0.98).

CONCLUSIONS

Our method provides valuable insights into leveraging real-world data for precise multi-disease mortality risk prediction, offering potential applications in optimizing chronic disease management, enhancing well-being, and reducing healthcare costs for the elderly population.

Performance of Atrial Fibrillation Burden Trends for Stroke Risk Stratification

BACKGROUND

Atrial fibrillation (AF) is associated with an increased risk of stroke, yet the limitations of conventional monitoring have restricted our understanding of AF burden risk thresholds. Predictive algorithms incorporating continuous AF burden measures may be useful for predicting stroke. This study evaluated the performance of temporal AF burden trends as predictors of stroke from a large cohort with insertable cardiac monitors.

METHODS

Using deidentified data from Optum Clinformatics Data Mart (2007-2019) linked with the Medtronic CareLink insertable cardiac monitor database, we identified patients with an insertable cardiac monitor for AF management (n=1197), suspected AF (n=1611), and cryptogenic stroke (n=2205). Daily AF burden was transformed into simple moving averages, and temporal AF burden trends were defined as the comparison of unique simple moving average pairs. Classification trees were used to predict ischemic stroke, and AF burden significance was quantified using bootstrapped mean variable importance.

RESULTS

Of 5013 patients (age, 69.2±11.7 years; 50% male; CHA2DS2-VASc, 3.7±1.9) who met inclusion criteria, 869 had an ischemic stroke over 2 409 437 days total follow-up. Prior stroke or transient ischemic attack (variable importance, 13.13) was the number 1 predictor of future stroke followed by no prior diagnosis of AF (7.35) and AF burden trends in follow-up (2.59). Temporal proximity of AF and risk of stroke differed by device indication (simple moving averages: AF management, <8 days and suspected AF and cryptogenic stroke, 8-21 days). Together, baseline characteristics and AF burden trends performed optimally for the area under the receiver operating characteristic curve (0.73), specificity (0.70), and relative risk (5.00).

CONCLUSIONS

AF burden trends may provide incremental prognostic value as leading indicators of stroke risk compared with conventional schemes.

Comparison of machine learning and conventional statistical modeling for predicting readmission following acute heart failure hospitalization

INTRODUCTION

Developing accurate models for predicting the risk of 30-day readmission is a major healthcare interest. Evidence suggests that models developed using machine learning (ML) may have better discrimination than conventional statistical models (CSM), but the calibration of such models is unclear.

OBJECTIVES

To compare models developed using ML with those developed using CSM to predict 30-day readmission for cardiovascular and noncardiovascular causes in HF patients.

METHODS

We retrospectively enrolled 10,919 patients with HF (> 18 years) discharged alive from a hospital or emergency department (2004-2007) in Ontario, Canada. The study sample was randomly divided into training and validation sets in a 2:1 ratio. CSMs to predict 30-day readmission were developed using Fine-Gray subdistribution hazards regression (treating death as a competing risk), and the ML algorithm employed random survival forests for competing risks (RSF-CR). Models were evaluated in the validation set using both discrimination and calibration metrics.

RESULTS

In the validation sample of 3602 patients, RSF-CR (c-statistic=0.620) showed similar discrimination to the Fine-Gray competing risk model (c-statistic=0.621) for 30-day cardiovascular readmission. In contrast, for 30-day noncardiovascular readmission, the Fine-Gray model (c-statistic=0.641) slightly outperformed the RSF-CR model (c-statistic=0.632). For both outcomes, The Fine-Gray model displayed better calibration than RSF-CR using calibration plots of observed vs predicted risks across the deciles of predicted risk.

CONCLUSIONS

Fine-Gray models had similar discrimination but superior calibration to the RSF-CR model, highlighting the importance of reporting calibration metrics for ML-based prediction models. The discrimination was modest in all readmission prediction models regardless of the methods used.

Latent disentanglement in mesh variational autoencoders improves the diagnosis of craniofacial syndromes and aids surgical planning

BACKGROUND AND OBJECTIVE

The use of deep learning to undertake shape analysis of the complexities of the human head holds great promise. However, there have traditionally been a number of barriers to accurate modelling, especially when operating on both a global and local level.

METHODS

In this work, we will discuss the application of the Swap Disentangled Variational Autoencoder (SD-VAE) with relevance to Crouzon, Apert and Muenke syndromes. The model is trained on a dataset of 3D meshes of healthy and syndromic patients which was increased in size with a novel data augmentation technique based on spectral interpolation. Thanks to its semantically meaningful and disentangled latent representation, SD-VAE is used to analyse and generate head shapes while considering the influence of different anatomical sub-units.

RESULTS

Although syndrome classification is performed on the entire mesh, it is also possible, for the first time, to analyse the influence of each region of the head on the syndromic phenotype. By manipulating specific parameters of the generative model, and producing procedure-specific new shapes, it is also possible to approximate the outcome of a range of craniofacial surgical procedures.

CONCLUSION

This work opens new avenues to advance diagnosis, aids surgical planning and allows for the objective evaluation of surgical outcomes. Our code is available at github.com/simofoti/CraniofacialSD-VAE.

Advancing groundwater quality predictions: Machine learning challenges and solutions

Machine learning (ML) is revolutionizing groundwater quality research by enhancing predictive accuracy and management strategies for contamination. This comprehensive review explores the evolution of ML technologies and their integration into environmental science, assessing 230 papers to understand the advancements and challenges in groundwater quality research. It reveals that a substantial portion of the research neglects critical preprocessing steps, crucial for model accuracy, with 83 % of the studies overlooking this phase. Furthermore, while model optimization is more commonly addressed, being implemented in 65 % of the papers, there is a noticeable gap in model interpretability, with only 15 % of the research providing explanations for model outcomes. Comparative evaluation of ML algorithms and careful selection of evaluation metrics are deemed essential for determining model fitness and reliability. The review underscores the need for interdisciplinary collaboration, methodological rigor, and continuous innovation to advance ML in groundwater management. By addressing these challenges and implementing solutions, the full potential of ML can be harnessed to tackle complex environmental issues and ensure sustainable groundwater management. This comprehensive and critical review paper can serve as a guiding framework to establish minimum standards for developing ML in groundwater quality studies.

Using interpretable machine learning methods to identify the relative importance of lifestyle factors for overweight and obesity in adults: pooled evidence from CHNS and NHANES

BACKGROUND

Overweight and obesity pose a huge burden on individuals and society. While the relationship between lifestyle factors and overweight and obesity is well-established, the relative contribution of specific lifestyle factors remains unclear. To address this gap in the literature, this study utilizes interpretable machine learning methods to identify the relative importance of specific lifestyle factors as predictors of overweight and obesity in adults.

METHODS

Data were obtained from 46,057 adults in the China Health and Nutrition Survey (2004-2011) and the National Health and Nutrition Examination Survey (2007-2014). Basic demographic information, self-reported lifestyle factors, including physical activity, macronutrient intake, tobacco and alcohol consumption, and body weight status were collected. Three machine learning models, namely decision tree, random forest, and gradient-boosting decision tree, were employed to predict body weight status from lifestyle factors. The SHapley Additive exPlanation (SHAP) method was used to interpret the prediction results of the best-performing model by determining the contributions of specific lifestyle factors to the development of overweight and obesity in adults.

RESULTS

The performance of the gradient-boosting decision tree model outperformed the decision tree and random forest models. Analysis based on the SHAP method indicates that sedentary behavior, alcohol consumption, and protein intake were important lifestyle factors predicting the development of overweight and obesity in adults. The amount of alcohol consumption and time spent sedentary were the strongest predictors of overweight and obesity, respectively. Specifically, sedentary behavior exceeding 28-35 h/week, alcohol consumption of more than 7 cups/week, and protein intake exceeding 80 g/day increased the risk of being predicted as overweight and obese.

CONCLUSION

Pooled evidence from two nationally representative studies suggests that recognizing demographic differences and emphasizing the relative importance of sedentary behavior, alcohol consumption, and protein intake are beneficial for managing body weight status in adults. The specific risk thresholds for lifestyle factors observed in this study can help inform and guide future research and public health actions.

Ensemble feature selection and tabular data augmentation with generative adversarial networks to enhance cutaneous melanoma identification and interpretability

BACKGROUND

Cutaneous melanoma is the most aggressive form of skin cancer, responsible for most skin cancer-related deaths. Recent advances in artificial intelligence, jointly with the availability of public dermoscopy image datasets, have allowed to assist dermatologists in melanoma identification. While image feature extraction holds potential for melanoma detection, it often leads to high-dimensional data. Furthermore, most image datasets present the class imbalance problem, where a few classes have numerous samples, whereas others are under-represented.

METHODS

In this paper, we propose to combine ensemble feature selection (FS) methods and data augmentation with the conditional tabular generative adversarial networks (CTGAN) to enhance melanoma identification in imbalanced datasets. We employed dermoscopy images from two public datasets, PH2 and Derm7pt, which contain melanoma and not-melanoma lesions. To capture intrinsic information from skin lesions, we conduct two feature extraction (FE) approaches, including handcrafted and embedding features. For the former, color, geometric and first-, second-, and higher-order texture features were extracted, whereas for the latter, embeddings were obtained using ResNet-based models. To alleviate the high-dimensionality in the FE, ensemble FS with filter methods were used and evaluated. For data augmentation, we conducted a progressive analysis of the imbalance ratio (IR), related to the amount of synthetic samples created, and evaluated the impact on the predictive results. To gain interpretability on predictive models, we used SHAP, bootstrap resampling statistical tests and UMAP visualizations.

RESULTS

The combination of ensemble FS, CTGAN, and linear models achieved the best predictive results, achieving AUCROC values of 87% (with support vector machine and IR=0.9) and 76% (with LASSO and IR=1.0) for the PH2 and Derm7pt, respectively. We also identified that melanoma lesions were mainly characterized by features related to color, while not-melanoma lesions were characterized by texture features.

CONCLUSIONS

Our results demonstrate the effectiveness of ensemble FS and synthetic data in the development of models that accurately identify melanoma. This research advances skin lesion analysis, contributing to both melanoma detection and the interpretation of main features for its identification.