Exploring Active Learning Based on Representativeness and Uncertainty for Biomedical Data Classification

Active learning and margin strategies for arrhythmia classification in implantable devices

BACKGROUND AND OBJECTIVES

The massive storage of cardiac arrhythmic episodes from Implantable Cardioverter Defibrillators (ICD) and the advent of new artificial intelligence algorithms are opening up new opportunities for electrophysiological knowledge extraction. However, in this context, accurate and reliable episode labeling by expert cardiologists still remains a manual, costly, and time-consuming process.

METHODS

In this work, we propose using Active Learning (AL) to design classification models that streamline the manual labeling of cardiac arrhythmic episodes. When AL is used, relevant episodes for classification are selected and then presented to the human expert for labeling, thereby dramatically reducing the manual labeling burden.

RESULTS

We adapted four large-margin-based AL strategies to a previously proposed classification methodology. We benchmarked them on problems involving 3 and 8 arrhythmia types using 9908 episodes from a massive national ICD data repository. Specifically, the relevance of episode-patient diversity for classification was evaluated. Results showed that the gold standard performance, achieved using all episodes, was reached by using approximately 20% (50%) of episodes from 60% (85%) of patients in the 3-class (8-class) model design.

CONCLUSIONS

We can conclude that AL techniques are advantageous for designing classification models and can streamline the human labeling process of large ICD datasets.

Rethinking Domain-Specific Pretraining by Supervised or Self-Supervised Learning for Chest Radiograph Classification: A Comparative Study Against ImageNet Counterparts in Cold-Start Active Learning

Objective

Deep learning (DL) has become the prevailing method in chest radiograph analysis, yet its performance heavily depends on large quantities of annotated images. To mitigate the cost, cold-start active learning (AL), comprising an initialization followed by subsequent learning, selects a small subset of informative data points for labeling. Recent advancements in pretrained models by supervised or self-supervised learning tailored to chest radiograph have shown broad applicability to diverse downstream tasks. However, their potential in cold-start AL remains unexplored.

Methods

To validate the efficacy of domain-specific pretraining, we compared two foundation models: supervised TXRV and self-supervised REMEDIS with their general domain counterparts pretrained on ImageNet. Model performance was evaluated at both initialization and subsequent learning stages on two diagnostic tasks: psychiatric pneumonia and COVID-19. For initialization, we assessed their integration with three strategies: diversity, uncertainty, and hybrid sampling. For subsequent learning, we focused on uncertainty sampling powered by different pretrained models. We also conducted statistical tests to compare the foundation models with ImageNet counterparts, investigate the relationship between initialization and subsequent learning, examine the performance of one-shot initialization against the full AL process, and investigate the influence of class balance in initialization samples on initialization and subsequent learning.

Results

First, domain-specific foundation models failed to outperform ImageNet counterparts in six out of eight experiments on informative sample selection. Both domain-specific and general pretrained models were unable to generate representations that could substitute for the original images as model inputs in seven of the eight scenarios. However, pretrained model-based initialization surpassed random sampling, the default approach in cold-start AL. Second, initialization performance was positively correlated with subsequent learning performance, highlighting the importance of initialization strategies. Third, one-shot initialization performed comparably to the full AL process, demonstrating the potential of reducing experts' repeated waiting during AL iterations. Last, a U-shaped correlation was observed between the class balance of initialization samples and model performance, suggesting that the class balance is more strongly associated with performance at middle budget levels than at low or high budgets.

Conclusions

In this study, we highlighted the limitations of medical pretraining compared to general pretraining in the context of cold-start AL. We also identified promising outcomes related to cold-start AL, including initialization based on pretrained models, the positive influence of initialization on subsequent learning, the potential for one-shot initialization, and the influence of class balance on middle-budget AL. Researchers are encouraged to improve medical pretraining for versatile DL foundations and explore novel AL methods.

Deep active learning with high structural discriminability for molecular mutagenicity prediction

The assessment of mutagenicity is essential in drug discovery, as it may lead to cancer and germ cells damage. Although in silico methods have been proposed for mutagenicity prediction, their performance is hindered by the scarcity of labeled molecules. However, experimental mutagenicity testing can be time-consuming and costly. One solution to reduce the annotation cost is active learning, where the algorithm actively selects the most valuable molecules from a vast chemical space and presents them to the oracle (e.g., a human expert) for annotation, thereby rapidly improving the model's predictive performance with a smaller annotation cost. In this paper, we propose muTOX-AL, a deep active learning framework, which can actively explore the chemical space and identify the most valuable molecules, resulting in competitive performance with a small number of labeled samples. The experimental results show that, compared to the random sampling strategy, muTOX-AL can reduce the number of training molecules by about 57%. Additionally, muTOX-AL exhibits outstanding molecular structural discriminability, allowing it to pick molecules with high structural similarity but opposite properties.

Assessment of clustering techniques to support the analyses of soybean seed vigor

Soy is the main product of Brazilian agriculture and the fourth most cultivated bean globally. Since soy cultivation tends to increase and due to this large market, the guarantee of product quality is an indispensable factor for enterprises to stay competitive. Industries perform vigor tests to acquire information and evaluate the quality of soy planting. The tetrazolium test, for example, provides information about moisture damage, bedbugs, or mechanical damage. However, the verification of the damage reason and its severity are done by an analyst, one by one. Since this is massive and exhausting work, it is susceptible to mistakes. Proposals involving different supervised learning approaches, including active learning strategies, have already been used, and have brought significant results. Therefore, this paper analyzes the performance of non-supervised techniques for classifying soybeans. An extensive experimental evaluation was performed, considering (9) different clustering algorithms (partitional, hierarchical, and density-based) applied to 5 image datasets of soybean seeds submitted to the tetrazolium test, including different damages and/or their levels. To describe those images, we considered 18 extractors of traditional features. We also considered four metrics (accuracy, FOWLKES, DAVIES, and CALINSKI) and two-dimensionality reduction techniques (principal component analysis and t-distributed stochastic neighbor embedding) for validation. Results show that this paper presents essential contributions since it makes it possible to identify descriptors and clustering algorithms that shall be used as preprocessing in other learning processes, accelerating and improving the classification process of key agricultural problems.

Transformer-based multi-task learning for classification and segmentation of gastrointestinal tract endoscopic images

Despite being widely utilized to help endoscopists identify gastrointestinal (GI) tract diseases using classification and segmentation, models based on convolutional neural network (CNN) have difficulties in distinguishing the similarities among some ambiguous types of lesions presented in endoscopic images, and in the training when lacking labeled datasets. Those will prevent CNN from further improving the accuracy of diagnosis. To address these challenges, we first proposed a Multi-task Network (TransMT-Net) capable of simultaneously learning two tasks (classification and segmentation), which has the transformer designed to learn global features and can combine the advantages of CNN in learning local features so that to achieve a more accurate prediction in identifying the lesion types and regions in GI tract endoscopic images. We further adopted the active learning in TransMT-Net to tackle the labeled image-hungry problem. A dataset was created from the CVC-ClinicDB dataset, Macau Kiang Wu Hospital, and Zhongshan Hospital to evaluate the model performance. Then, the experimental results show that our model not only achieved 96.94% accuracy in the classification task and 77.76% Dice Similarity Coefficient in the segmentation task but also outperformed those of other models on our test set. Meanwhile, active learning also produced positive results for the performance of our model with a small-scale initial training set, and even its performance with 30% of the initial training set was comparable to that of most comparable models with the full training set. Consequently, the proposed TransMT-Net has demonstrated its potential performance in GI tract endoscopic images and it through active learning can alleviate the shortage of labeled images.

Integrated Random Negative Sampling and Uncertainty Sampling in Active Learning Improve Clinical Drug Safety Drug-Drug Interaction Information Retrieval

Clinical drug-drug interactions (DDIs) have been a major cause for not only medical error but also adverse drug events (ADEs). The published literature on DDI clinical toxicity continues to grow significantly, and high-performance DDI information retrieval (IR) text mining methods are in high demand. The effectiveness of IR and its machine learning (ML) algorithm depends on the availability of a large amount of training and validation data that have been manually reviewed and annotated. In this study, we investigated how active learning (AL) might improve ML performance in clinical safety DDI IR analysis. We recognized that a direct application of AL would not address several primary challenges in DDI IR from the literature. For instance, the vast majority of abstracts in PubMed will be negative, existing positive and negative labeled samples do not represent the general sample distributions, and potentially biased samples may arise during uncertainty sampling in an AL algorithm. Therefore, we developed several novel sampling and ML schemes to improve AL performance in DDI IR analysis. In particular, random negative sampling was added as a part of AL since it has no expanse in the manual data label. We also used two ML algorithms in an AL process to differentiate random negative samples from manually labeled negative samples, and updated both the training and validation samples during the AL process to avoid or reduce biased sampling. Two supervised ML algorithms, support vector machine (SVM) and logistic regression (LR), were used to investigate the consistency of our proposed AL algorithm. Because the ultimate goal of clinical safety DDI IR is to retrieve all DDI toxicity-relevant abstracts, a recall rate of 0.99 was set in developing the AL methods. When we used our newly proposed AL method with SVM, the precision in differentiating the positive samples from manually labeled negative samples improved from 0.45 in the first round to 0.83 in the second round, and the precision in differentiating the positive samples from random negative samples improved from 0.70 to 0.82 in the first and second rounds, respectively. When our proposed AL method was used with LR, the improvements in precision followed a similar trend. However, the other AL algorithms tested did not show improved precision largely because of biased samples caused by the uncertainty sampling or differences between training and validation data sets.

The Promise of Clinical Decision Support Systems Targetting Low-Resource Settings

Low-resource clinical settings are plagued by low physician-to-patient ratios and a shortage of high-quality medical expertise and infrastructure. Together, these phenomena lead to over-burdened healthcare systems that under-serve the needs of the community. Alleviating this burden can be undertaken by the introduction of clinical decision support systems (CDSSs); systems that support stakeholders (ranging from physicians to patients) within the clinical setting in their day-to-day activities. Such systems, which have proven to be effective in the developed world, remain to be under-explored in low-resource settings. This review attempts to summarize the research focused on clinical decision support systems that either target stakeholders within low-resource clinical settings or diseases commonly found in such environments. When categorizing our findings according to disease applications, we find that CDSSs are predominantly focused on dealing with bacterial infections and maternal care, do not leverage deep learning, and have not been evaluated prospectively. Together, these highlight the need for increased research in this domain in order to impact a diverse set of medical conditions and ultimately improve patient outcomes.

Photoplethysmography based atrial fibrillation detection: a review

Atrial fibrillation (AF) is a cardiac rhythm disorder associated with increased morbidity and mortality. It is the leading risk factor for cardioembolic stroke and its early detection is crucial in both primary and secondary stroke prevention. Continuous monitoring of cardiac rhythm is today possible thanks to consumer-grade wearable devices, enabling transformative diagnostic and patient management tools. Such monitoring is possible using low-cost easy-to-implement optical sensors that today equip the majority of wearables. These sensors record blood volume variations-a technology known as photoplethysmography (PPG)-from which the heart rate and other physiological parameters can be extracted to inform about user activity, fitness, sleep, and health. Recently, new wearable devices were introduced as being capable of AF detection, evidenced by large prospective trials in some cases. Such devices would allow for early screening of AF and initiation of therapy to prevent stroke. This review is a summary of a body of work on AF detection using PPG. A thorough account of the signal processing, machine learning, and deep learning approaches used in these studies is presented, followed by a discussion of their limitations and challenges towards clinical applications.