A scoping review of transfer learning research on medical image analysis using ImageNet

Data integrity of food and machine learning: Strategies, advances and prospective

Data integrity is an emerging concept aimed at recording real food properties in the form of data throughout the food lifecycle. However, due to the one-sided nature of current food control data, the comprehensive implementation of data integrity has not been fully achieved. Cause food data integrity realization is required to establish the connection of data-algorithm-application. Machine learning (ML) provides a possibility for the practical carrier of food data integrity. Despite ML is one of top-trend in food quality and safety, ML applications are floating on the surface. The current review does not reveal the relationships behind different algorithms and data patterns. Similarly, due to the rapid development of ML, the current advanced concepts and data explanation tools have not been systematically reviewed. This paper expounds the feasibility of machine learning to achieve data integrity and looks forward to the future vision brought about by artificial intelligence to data integrity.

Multimodal Deep Learning Fusing Clinical and Radiomics Scores for Prediction of Early-Stage Lung Adenocarcinoma Lymph Node Metastasis

RATIONALE AND OBJECTIVES

To develop and validate a multimodal deep learning (DL) model based on computed tomography (CT) images and clinical knowledge to predict lymph node metastasis (LNM) in early lung adenocarcinoma.

MATERIALS AND METHODS

A total of 724 pathologically confirmed early invasive lung adenocarcinoma patients were retrospectively included from two centers. Clinical and CT semantic features of the patients were collected, and 3D radiomics features were extracted from nonenhanced CT images. We proposed a multimodal feature fusion DL network based on the InceptionResNetV2 architecture, which can effectively extract and integrate image and clinical knowledge to predict LNM.

RESULTS

A total of 524 lung adenocarcinoma patients from Center 1 were randomly divided into training (n=418) and internal validation (n=106) sets in a 4:1 ratio, while 200 lung adenocarcinoma patients from Center 2 served as the independent test set. Among the 16 collected clinical and imaging features, 8 were selected: gender, serum carcinoembryonic antigen, cytokeratin 19 fragment antigen 21-1, neuron-specific enolase, tumor size, location, density, and centrality. From the 1595 extracted radiomics features, six key features were identified. The CS-RS-DL fusion model achieved the highest area under the receiver operating characteristic curve in both the internal validation set (0.877) and the independent test set (0.906) compared to other models. The Delong test results for the independent test set indicated that the CS-RS-DL model significantly outperformed the clinical model (0.844), radiomics model (0.850), CS-RS model (0.872), single DL model (0.848), and the CS-DL model (0.875) (all P<0.05). Additionally, the CS-RS-DL model exhibited the highest sensitivity (0.941) and average precision (0.642).

CONCLUSION

The knowledge derived from clinical, radiomics, and DL is complementary in predicting LNM in lung adenocarcinoma. The integration of clinical and radiomics scores through DL can significantly improve the accuracy of lymph node status assessment.

An interpretable framework for gastric cancer classification using multi-channel attention mechanisms and transfer learning approach on histopathology images

The importance of gastric cancer (GC) and the role of deep learning techniques in categorizing GC histopathology images have recently increased. Identifying the drawbacks of traditional deep learning models, including lack of interpretability, inability to capture complex patterns, lack of adaptability, and sensitivity to noise. A multi-channel attention mechanism-based framework is proposed that can overcome the limitations of conventional deep learning models by dynamically focusing on relevant features, enhancing extraction, and capturing complex relationships in medical data. The proposed framework uses three different attention mechanism channels and convolutional neural networks to extract multichannel features during the classification process. The proposed framework's strong performance is confirmed by competitive experiments conducted on a publicly available Gastric Histopathology Sub-size Image Database, which yielded remarkable classification accuracies of 99.07% and 98.48% on the validation and testing sets, respectively. Additionally, on the HCRF dataset, the framework achieved high classification accuracy of 99.84% and 99.65% on the validation and testing sets, respectively. The effectiveness and interchangeability of the three channels are further confirmed by ablation and interchangeability experiments, highlighting the remarkable performance of the framework in GC histopathological image classification tasks. This offers an advanced and pragmatic artificial intelligence solution that addresses challenges posed by unique medical image characteristics for intricate image analysis. The proposed approach in artificial intelligence medical engineering demonstrates significant potential for enhancing diagnostic precision by achieving high classification accuracy and treatment outcomes.

Rethinking Domain-Specific Pretraining by Supervised or Self-Supervised Learning for Chest Radiograph Classification: A Comparative Study Against ImageNet Counterparts in Cold-Start Active Learning

Objective

Deep learning (DL) has become the prevailing method in chest radiograph analysis, yet its performance heavily depends on large quantities of annotated images. To mitigate the cost, cold-start active learning (AL), comprising an initialization followed by subsequent learning, selects a small subset of informative data points for labeling. Recent advancements in pretrained models by supervised or self-supervised learning tailored to chest radiograph have shown broad applicability to diverse downstream tasks. However, their potential in cold-start AL remains unexplored.

Methods

To validate the efficacy of domain-specific pretraining, we compared two foundation models: supervised TXRV and self-supervised REMEDIS with their general domain counterparts pretrained on ImageNet. Model performance was evaluated at both initialization and subsequent learning stages on two diagnostic tasks: psychiatric pneumonia and COVID-19. For initialization, we assessed their integration with three strategies: diversity, uncertainty, and hybrid sampling. For subsequent learning, we focused on uncertainty sampling powered by different pretrained models. We also conducted statistical tests to compare the foundation models with ImageNet counterparts, investigate the relationship between initialization and subsequent learning, examine the performance of one-shot initialization against the full AL process, and investigate the influence of class balance in initialization samples on initialization and subsequent learning.

Results

First, domain-specific foundation models failed to outperform ImageNet counterparts in six out of eight experiments on informative sample selection. Both domain-specific and general pretrained models were unable to generate representations that could substitute for the original images as model inputs in seven of the eight scenarios. However, pretrained model-based initialization surpassed random sampling, the default approach in cold-start AL. Second, initialization performance was positively correlated with subsequent learning performance, highlighting the importance of initialization strategies. Third, one-shot initialization performed comparably to the full AL process, demonstrating the potential of reducing experts' repeated waiting during AL iterations. Last, a U-shaped correlation was observed between the class balance of initialization samples and model performance, suggesting that the class balance is more strongly associated with performance at middle budget levels than at low or high budgets.

Conclusions

In this study, we highlighted the limitations of medical pretraining compared to general pretraining in the context of cold-start AL. We also identified promising outcomes related to cold-start AL, including initialization based on pretrained models, the positive influence of initialization on subsequent learning, the potential for one-shot initialization, and the influence of class balance on middle-budget AL. Researchers are encouraged to improve medical pretraining for versatile DL foundations and explore novel AL methods.

MoMA: Momentum contrastive learning with multi-head attention-based knowledge distillation for histopathology image analysis

There is no doubt that advanced artificial intelligence models and high quality data are the keys to success in developing computational pathology tools. Although the overall volume of pathology data keeps increasing, a lack of quality data is a common issue when it comes to a specific task due to several reasons including privacy and ethical issues with patient data. In this work, we propose to exploit knowledge distillation, i.e., utilize the existing model to learn a new, target model, to overcome such issues in computational pathology. Specifically, we employ a student-teacher framework to learn a target model from a pre-trained, teacher model without direct access to source data and distill relevant knowledge via momentum contrastive learning with multi-head attention mechanism, which provides consistent and context-aware feature representations. This enables the target model to assimilate informative representations of the teacher model while seamlessly adapting to the unique nuances of the target data. The proposed method is rigorously evaluated across different scenarios where the teacher model was trained on the same, relevant, and irrelevant classification tasks with the target model. Experimental results demonstrate the accuracy and robustness of our approach in transferring knowledge to different domains and tasks, outperforming other related methods. Moreover, the results provide a guideline on the learning strategy for different types of tasks and scenarios in computational pathology.

Classification models for arthropathy grades of multiple joints based on hierarchical continual learning

PURPOSE

To develop a hierarchical continual arthropathy classification model for multiple joints that can be updated continuously for large-scale studies of various anatomical structures.

MATERIALS AND METHODS

This study included a total of 1371 radiographs of knee, elbow, ankle, shoulder, and hip joints from three tertiary hospitals. For model development, 934 radiographs of the knee, elbow, ankle, and shoulder were gathered from Sinchon Severance Hospital between July 1 and December 31, 2022. For external validation, 125 hip radiographs were collected from Yongin Severance Hospital between January 1 and December 31, 2022, and 312 knee cases were gathered from Gangnam Severance Hospital between January 1 and June 31, 2023. The Hierarchical Dynamically Expandable Representation (Hi-DER) model was trained stepwise on four joints using five-fold cross-validation. Arthropathy classification was evaluated at three hierarchical levels: abnormal classification (L1), low-grade or high-grade classification (L2), and specific grade classification (L3). The model's performance was compared with the grading predictions of two other AI models and three radiologists. For model explainability, gradient-weighted class activation mapping (Grad-CAM) and progressive erasing plus progressive restoration (PEPPR) were employed.

RESULTS

The model achieved a weighted average AUC of 0.994 (95% CI: 0.985, 0.999) for L1, 0.980 (95% CI: 0.958, 0.996) for L2, and 0.973 (95% CI: 0.943, 0.993) for L3. The model maintained an AUC above 0.800 with 70% of the input regions erased. During external validation on hip joints, the model demonstrated a weighted average AUC of 0.978 (95% CI: 0.952, 0.996) for L1, 0.977 (95% CI: 0.946, 0.996) for L2, and 0.971 (95% CI: 0.934, 0.996) for L3. For external knee data, the model yielded a weighted average AUC of 0.934 (95%: CI 0.904, 0.958), 0.929 (95% CI: 0.900, 0.954), and 0.857 (95% CI: 0.816, 0.894) for L1, L2, and L3, respectively.

CONCLUSION

The Hi-DER may enhance the efficiency of arthropathy diagnosis through accurate classification of arthropathy grades across multiple joints, potentially enabling early treatment.

Predicting inflammatory response of biomimetic nanofibre scaffolds for tissue regeneration using machine learning and graph theory

Tissue regeneration after a wound occurs through three main overlapping and interrelated stages namely inflammatory, proliferative, and remodelling phases, respectively. The inflammatory phase is key for successful tissue reconstruction and triggers the proliferative phase. The macrophages in the non-healing wounds remain in the inflammatory loop, but their phenotypes can be changed via interactions with nanofibre-based scaffolds mimicking the organisation of the native structural support of healthy tissues. However, the organisation of extracellular matrix (ECM) is highly complex, combining order and disorder, which makes it difficult to replicate. The possibility of predicting the desirable biomimetic geometry and chemistry of these nanofibre scaffolds would streamline the scaffold design process. Fifteen families of nanofibre scaffolds, electrospun from combinations of polyesters (polylactide, polyhydroxybutyrate), polysaccharides (polysucrose, carrageenan, cellulose), and polyester ether (polydioxanone) were investigated and analysed using machine learning (ML). The Random Forest model had the best performance (92.8%) in predicting inflammatory responses of macrophages on the nanoscaffolds using tumour necrosis factor-alpha as the output. CellProfiler proved to be an effective tool to process scanning electron microscopy (SEM) images of the macrophages on the scaffolds, successfully extracting various features and measurements related to cell phenotypes M0, M1, and M2. Deep learning modelling indicated that convolutional neural network models have the potential to be applied to SEM images to classify macrophage cells according to their phenotypes. The complex organisation of the nanofibre scaffolds can be analysed using graph theory (GT), revealing the underlying connectivity patterns of the nanofibres. Analysis of GT descriptors showed that the electrospun membranes closely mimic the connectivity patterns of the ECM. We conclude that ML-facilitated, GT-quantified engineering of cellular scaffolds has the potential to predict cell interactions, streamlining the pipeline for tissue engineering.

Breast cancer classification based on breast tissue structures using the Jigsaw puzzle task in self-supervised learning

Self-supervised learning (SSL) has gained attention in the medical field as a deep learning approach utilizing unlabeled data. The Jigsaw puzzle task in SSL enables models to learn both features of images and the positional relationships within images. In breast cancer diagnosis, radiologists evaluate not only lesion-specific features but also the surrounding breast structures. However, deep learning models that adopt a diagnostic approach similar to human radiologists are still limited. This study aims to evaluate the effectiveness of the Jigsaw puzzle task in characterizing breast tissue structures for breast cancer classification on mammographic images. Using the Chinese Mammography Database (CMMD), we compared four pre-training pipelines: (1) IN-Jig, pre-trained with both the ImageNet classification task and the Jigsaw puzzle task, (2) Scratch-Jig, pre-trained only with the Jigsaw puzzle task, (3) IN, pre-trained only with the ImageNet classification task, and (4) Scratch, that is trained from random initialization without any pre-training tasks. All pipelines were fine-tuned using binary classification to distinguish between the presence or absence of breast cancer. Performance was evaluated based on the area under the receiver operating characteristic curve (AUC), sensitivity, and specificity. Additionally, detailed analysis was conducted for performance across different radiological findings, breast density, and regions of interest were visualized using gradient-weighted class activation mapping (Grad-CAM). The AUC for the four models were 0.925, 0.921, 0.918, 0.909, respectively. Our results suggest the Jigsaw puzzle task is an effective pre-training method for breast cancer classification, with the potential to enhance diagnostic accuracy with limited data.

Development of a diagnostic classification model for lateral cephalograms based on multitask learning

OBJECTIVES

This study aimed to develop a cephalometric classification method based on multitask learning for eight diagnostic classifications.

METHODS

This study was retrospective. A total of 3,310 lateral cephalograms were collected to construct a dataset. Eight clinical classifications were employed, including sagittal and vertical skeletal facial patterns, maxillary and mandibular anteroposterior positions, inclinations of upper and lower incisors, as well as their anteroposterior positions. The images were manually annotated for initially classification, which was verified by senior orthodontists. The data were randomly divided into training, validation, and test sets at a ratio of approximately 8:1:1. The multitask learning classification model was constructed based on the ResNeXt50_32 × 4d network and consisted of shared layers and task-specific layers. The performance of the model was evaluated using classification accuracy, precision, sensitivity, specificity and area under the curve (AUC).

RESULTS

This model could perform eight clinical diagnostic classifications on cephalograms within an average of 0.0096 s. The accuracy of the six classifications was 0.8-0.9, and the accuracy of the two classifications was 0.75-0.8. The overall AUC values for each classification exceeded 0.9.

CONCLUSIONS

An automatic diagnostic classification model for lateral cephalograms was established based on multitask learning to achieve simultaneous classification of eight common clinical diagnostic items. The multitask learning model achieved better classification performance and reduced the computational costs, providing a novel perspective and reference for addressing such problems.

Associations of Postencephalitic Epilepsy Using Multi-Contrast Whole Brain MRI: A Large Self-Supervised Vision Foundation Model Strategy

BACKGROUND

Postencephalitic epilepsy (PEE) is a severe neurological complication following encephalitis. Early identification of individuals at high risk for PEE is important for timely intervention.

PURPOSE

To develop a large self-supervised vision foundation model using a big dataset of multi-contrast head MRI scans, followed by fine-tuning with MRI data and follow-up outcomes from patients with PEE to develop a PEE association model.

STUDY TYPE

Retrospective.

POPULATION

Fifty-seven thousand six hundred twenty-one contrast-enhanced head MRI scans from 34,871 patients for foundation model construction, and head MRI scans from 144 patients with encephalitis (64 PEE, 80 N-PEE) for the PEE association model.

FIELD STRENGTH/SEQUENCE

1.5-T, 3-T, T1-weighted imaging, T2-weighted imaging, fluid attenuated inversion recovery, T1-weighted contrast-enhanced imaging.

ASSESSMENT

The foundation model was developed using self-supervised learning and cross-contrast context recovery. Patients with encephalitis were monitored for a median of 3.7 years (range 0.7-7.5 years), with epilepsy diagnosed according to International League Against Epilepsy. Occlusion sensitivity mapping highlighted brain regions involved in PEE classifications. Model performance was compared with DenseNet without pre-trained weights.

STATISTICAL TESTS

Performance was assessed via confusion matrices, accuracy, sensitivity, specificity, precision, F1 score, and area under the receiver operating characteristic curve (AUC). The DeLong test evaluated AUC between the two models (P < 0.05 for statistical significance).

RESULTS

The PEE association model achieved accuracy, sensitivity, specificity, precision, F1 score, and AUC of 79.3% (95% CI: 0.71-0.92), 92.3% (95% CI: 0.80-1.00), 68.8% (95% CI: 0.55-0.87), 70.6% (95% CI: 0.61-0.90), 80.0% (95% CI: 0.71-0.93), and 81.0% (95% CI: 0.68-0.92), respectively. A significant AUC improvement was found compared to DenseNet (Delong test, P = 0.03). The association model focused on brain regions affected by encephalitis.

DATA CONCLUSION

Using extensive unlabeled data via self-supervised learning addressed the limitations of supervised tasks with limited data. The fine-tuned foundation model outperformed DenseNet, which was trained exclusively on task data.

PLAIN LANGUAGE SUMMARY

This research develops a model to assess the occurrence epilepsy after encephalitis, a severe brain inflammation condition. By using over 57,000 brain scans, the study trains a computer program to recognize patterns in brain images. The model analyzes whole-brain scans to identify areas commonly affected by the disease, such as the temporal and frontal lobes. It was tested on data from patients with encephalitis and showed better performance than older methods. The model can assess the risk of secondary epilepsy in patients with encephalitis, allowing doctors to intervene early and improve treatment outcomes for those affected by this condition.

EVIDENCE LEVEL

4 TECHNICAL EFFICACY: Stage 1.

Deep Learning and Habitat Radiomics for the Prediction of Glioma Pathology Using Multiparametric MRI: A Multicenter Study

RATIONALE AND OBJECTIVES

Recent radiomics studies on predicting pathological outcomes of glioma have shown immense potential. However, the predictive ability remains suboptimal due to the tumor intrinsic heterogeneity. We aimed to achieve better pathological prediction outcomes by combining habitat analysis with deep learning.

MATERIALS AND METHODS

387 cases of primary glioma from three hospitals were collected, along with their T1 contrast-enhanced and T2-weighted MR sequences, pathological reports and clinical histories. The training set consisted of 264 patients, 82 patients composed the test set, and 41 patients were used as the validation set for hyperparameter tuning and optimal model selection. All groups were sourced from different centers. Through radiomics, deep learning, habitat analysis and combined analysis, we extracted imaging features separately and jointly modeled them with clinical features. We identified the optimal models for predicting glioma grades, Ki67 expression levels, P53 mutation and IDH1 mutation.

RESULTS

Using a LightGBM model with DenseNet161 features based on habitat subregions, the best tumor grade prediction model was achieved. A LightGBM model with ResNet50 features based on habitat subregions yielded the best Ki67 expression level prediction model. An SVM model with Radiomics and Inception_v3 features provided the best prediction of P53 mutation. The best model for predicting IDH1 mutation was achieved by an MLP model with Radiomics features based on habitat subregions. Clinical features might be potentially helpful for the prediction with relatively weak evidence.

CONCLUSION

Habitat+Deep Learning feature extraction methods were optimal for predicting grades and Ki67 levels. Deep Learning is optimal for predicting P53 mutation, while the combination of Habitat+ Radiomics models yielded the best prediction for IDH1 mutation.

RadImageNet and ImageNet as Datasets for Transfer Learning in the Assessment of Dental Radiographs: A Comparative Study

Transfer learning (TL) is an alternative approach to the full training of deep learning (DL) models from scratch and can transfer knowledge gained from large-scale data to solve different problems. ImageNet, which is a publicly available large-scale dataset, is a commonly used dataset for TL-based image analysis; many studies have applied pre-trained models from ImageNet to clinical prediction tasks and have reported promising results. However, some have questioned the effectiveness of using ImageNet, which consists solely of natural images, for medical image analysis. The aim of this study was to evaluate whether pre-trained models using RadImageNet, which is a large-scale medical image dataset, could achieve superior performance in classification tasks in dental imaging modalities compared with ImageNet pre-trained models. To evaluate the classification performance of RadImageNet and ImageNet pre-trained models for TL, two dental imaging datasets were used. The tasks were (1) classifying the presence or absence of supernumerary teeth from a dataset of panoramic radiographs and (2) classifying sex from a dataset of lateral cephalometric radiographs. Performance was evaluated by comparing the area under the curve (AUC). On the panoramic radiograph dataset, the RadImageNet models gave average AUCs of 0.68 ± 0.15 (p < 0.01), and the ImageNet models had values of 0.74 ± 0.19. In contrast, on the lateral cephalometric dataset, the RadImageNet models demonstrated average AUCs of 0.76 ± 0.09, and the ImageNet models achieved values of 0.75 ± 0.17. The difference in performance between RadImageNet and ImageNet models in TL depends on the dental image dataset used.

Updating methods for artificial intelligence-based clinical prediction models: a scoping review

OBJECTIVES

To give an overview of methods for updating artificial intelligence (AI)-based clinical prediction models based on new data.

STUDY DESIGN AND SETTING

We comprehensively searched Scopus and Embase up to August 2022 for articles that addressed developments, descriptions, or evaluations of prediction model updating methods. We specifically focused on articles in the medical domain involving AI-based prediction models that were updated based on new data, excluding regression-based updating methods as these have been extensively discussed elsewhere. We categorized and described the identified methods used to update the AI-based prediction model as well as the use cases in which they were used.

RESULTS

We included 78 articles. The majority of the included articles discussed updating for neural network methods (93.6%) with medical images as input data (65.4%). In many articles (51.3%) existing, pretrained models for broad tasks were updated to perform specialized clinical tasks. Other common reasons for model updating were to address changes in the data over time and cross-center differences; however, more unique use cases were also identified, such as updating a model from a broad population to a specific individual. We categorized the identified model updating methods into four categories: neural network-specific methods (described in 92.3% of the articles), ensemble-specific methods (2.5%), model-agnostic methods (9.0%), and other (1.3%). Variations of neural network-specific methods are further categorized based on the following: (1) the part of the original neural network that is kept, (2) whether and how the original neural network is extended with new parameters, and (3) to what extent the original neural network parameters are adjusted to the new data. The most frequently occurring method (n = 30) involved selecting the first layer(s) of an existing neural network, appending new, randomly initialized layers, and then optimizing the entire neural network.

CONCLUSION

We identified many ways to adjust or update AI-based prediction models based on new data, within a large variety of use cases. Updating methods for AI-based prediction models other than neural networks (eg, random forest) appear to be underexplored in clinical prediction research.

PLAIN LANGUAGE SUMMARY

AI-based prediction models are increasingly used in health care, helping clinicians with diagnosing diseases, guiding treatment decisions, and informing patients. However, these prediction models do not always work well when applied to hospitals, patient populations, or times different from those used to develop the models. Developing new models for every situation is neither practical nor desired, as it wastes resources, time, and existing knowledge. A more efficient approach is to adjust existing models to new contexts ('updating'), but there is limited guidance on how to do this for AI-based clinical prediction models. To address this, we reviewed 78 studies in detail to understand how researchers are currently updating AI-based clinical prediction models, and the types of situations in which these updating methods are used. Our findings provide a comprehensive overview of the available methods to update existing models. This is intended to serve as guidance and inspiration for researchers. Ultimately, this can lead to better reuse of existing models and improve the quality and efficiency of AI-based prediction models in health care.

Investigating the key principles in two-step heterogeneous transfer learning for early laryngeal cancer identification

Data scarcity in medical images makes transfer learning a common approach in computer-aided diagnosis. Some disease classification tasks can rely on large homogeneous public datasets to train the transferred model, while others cannot, i.e., endoscopic laryngeal cancer image identification. Distinguished from most current works, this work pioneers exploring a two-step heterogeneous transfer learning (THTL) framework for laryngeal cancer identification and summarizing the fundamental principles for the intermediate domain selection. For heterogeneity and clear vascular representation, diabetic retinopathy images were chosen as THTL's intermediate domain. The experiment results reveal two vital principles in intermediate domain selection for future studies: 1) the size of the intermediate domain is not a sufficient condition to improve the transfer learning performance; 2) even distinct vascular features in the intermediate domain do not guarantee improved performance in the target domain. We observe that radial vascular patterns benefit benign classification, whereas twisted and tangled patterns align more with malignant classification. Additionally, to compensate for the absence of twisted patterns in the intermediate domains, we propose the Step-Wise Fine-Tuning (SWFT) technique, guided by the Layer Class Activate Map (LayerCAM) visualization result, getting 20.4% accuracy increases compared to accuracy from THTL's, even higher than fine-tune all layers.

Deep-Learning-based Automated Identification of Ventriculoperitoneal-Shunt Valve Models from Skull X-rays

INTRODUCTION

Ventriculoperitoneal shunts (VPS) are an essential part of the treatment of hydrocephalus, with numerous valve models available with different ways of indicating pressure levels. The model types often need to be identified on X‑rays to assess pressure levels using a matching template. Artificial intelligence (AI), in particular deep learning, is ideally suited to automate repetitive tasks such as identifying different VPS valve models. The aim of this work was to investigate whether AI, in particular deep learning, allows the identification of VPS models in cranial X‑rays.

METHODS

959 cranial X‑rays of patients with a VPS were included and reviewed for image quality and complete visualization of VPS valves. The images included four VPS model types: Codman Hakim (n = 774, 81%), Codman Certas Plus (n = 117, 12%), Sophysa Sophy Mini SM8 (n = 35, 4%) and proGAV 2.0 (n = 33, 3%). A Convolutional Neural Network (CNN) was trained using stratified five-fold cross-validation to classify the four VPS model types in the dataset. A finetuned CNN pretrained on the ImageNet dataset as well as a model trained from scratch were compared. The averaged performance and uncertainty metrics were evaluated across the cross-validation splits.

RESULTS

The fine-tuned model identified VPS valve models with a mean accuracy of 0.98 ± 0.01, macro-averaged F1 score of 0.93 ± 0.04, a recall of 0.94 ± 0.03 and a precision of 0.95 ± 0.08 across the five cross-validation splits.

CONCLUSION

Automatic classification of VPS valve models in skull X‑rays, using fully automatable preprocessing steps and a CNN, is feasible. This is an encouraging finding to further explore the possibility of automating VPS valve model identification and pressure level reading in skull X‑rays.

Multi-Class Brain Tumor Grades Classification Using a Deep Learning-Based Majority Voting Algorithm and Its Validation Using Explainable-AI

Biopsy is considered the gold standard for diagnosing brain tumors, but its invasive nature can pose risks to patients. Additionally, tissue analysis can be cumbersome and inconsistent among observers. This research aims to develop a cost-effective, non-invasive, MRI-based computer-aided diagnosis tool that can reliably, accurately and swiftly identify brain tumor grades. Our system employs ensemble deep learning (EDL) within an MRI multiclass framework that includes five datasets: two-class (C2), three-class (C3), four-class (C4), five-class (C5) and six-class (C6). The EDL utilizes a majority voting algorithm to classify brain tumors by combining seven renowned deep learning (DL) models-EfficientNet, VGG16, ResNet18, GoogleNet, ResNet50, Inception-V3 and DarkNet-and seven machine learning (ML) models, including support vector machine, K-nearest neighbour, Naïve Bayes, decision tree, linear discriminant analysis, artificial neural network and random forest. Additionally, local interpretable model-agnostic explanations (LIME) are employed as an explainable AI algorithm, providing a visual representation of the CNN's internal workings to enhance the credibility of the results. Through extensive five-fold cross-validation experiments, the DL-based majority voting algorithm outperformed the ML-based majority voting algorithm, achieving the highest average accuracies of 100 ± 0.00%, 98.55 ± 0.35%, 98.47 ± 0.63%, 95.34 ± 1.17% and 96.61 ± 0.85% for the C2, C3, C4, C5 and C6 datasets, respectively. Majority voting algorithms typically yield consistent results across different folds of the brain tumor data and enhance performance compared to any individual deep learning and machine learning models.

Explainable deep learning identifies patterns and drivers of freshwater harmful algal blooms

The escalating magnitude, frequency, and duration of harmful algal blooms (HABs) pose significant challenges to freshwater ecosystems worldwide. However, the mechanisms driving HABs remain poorly understood, in part due to the strong regional specificity of algal processes and the uneven data availability. These complexities make it difficult to generalize HAB dynamics and effectively predict their occurrence using traditional models. To address these challenges, we developed an explainable deep learning approach using long short-term memory (LSTM) models combined with explanation techniques that can capture complex patterns and provide explainable insights into key HAB drivers. We applied this approach for algal density modeling at 102 sites in China's lakes and reservoirs over three years. LSTMs effectively captured daily algal dynamics, achieving mean and maximum Nash-Sutcliffe efficiency coefficients of 0.48 and 0.95 during testing phase. Moreover, water temperature emerged as the primary driver of HABs both nationally and in over 30% of localities, with stronger water temperature sensitivity observed in mid-to low-latitudes. We also identified regional similarities that allow for the successful transferability in modeling algal dynamics. Specifically, using fine-tuned transfer learning, we improved the prediction accuracy in over 75% of poorly gauged areas. Overall, LSTM-based explainable deep learning approach effectively addresses key challenges in HAB modeling by tackling both regional specificity and data limitations. By accurately predicting algal dynamics and identifying critical drivers, this approach provides actionable insights into the mechanisms of HABs, ultimately aids in the implementation of effective mitigation measures for nationwide and regional freshwater ecosystems.

Transfer learning in spirometry: CNN models for automated flow-volume curve quality control in paediatric populations

PROBLEM

Current spirometers face challenges in evaluating acceptability criteria, often requiring manual visual inspection by trained specialists. Automating this process could improve diagnostic workflows and reduce variability in test assessments.

AIM

This study aimed to apply transfer learning to convolutional neural networks (CNNs) to automate the classification of spirometry flow-volume curves based on acceptability criteria.

METHODS

A total of 5287 spirometry flow-volume curves were divided into three categories: (A) all criteria met, (B) early termination, and (C) non-acceptable results. Six CNN models (VGG16, InceptionV3, Xception, ResNet152V2, InceptionResNetV2, DenseNet121) were trained using a balanced dataset after data augmentation. The models' performance was evaluated on part of the original unbalanced dataset with accuracy, precision, recall, and F1-score metrics.

RESULTS

VGG16 achieved the highest accuracy at 93.9 %, while ResNet152V2 had the lowest at 83.0 %. Non-acceptable curves (Group C) were the easiest to classify, with precision reaching at least 87.7 %. Early termination curves (Group B) were the most challenging, with precision ranging from 75.0 % to 90.3 %.

CONCLUSION

CNN models, particularly VGG16, show promise in automating spirometry quality control, potentially reducing the need for manual inspection by specialized technicians. This approach can streamline spirometry assessments, offering consistent, high-quality diagnostics even in non-specialized or low-resource environments.

Microfluidic Nanoparticle Separation for Precision Medicine

A deeper understanding of disease heterogeneity highlights the urgent need for precision medicine. Microfluidics, with its unique advantages, such as high adjustability, diverse material selection, low cost, high processing efficiency, and minimal sample requirements, presents an ideal platform for precision medicine applications. As nanoparticles, both of biological origin and for therapeutic purposes, become increasingly important in precision medicine, microfluidic nanoparticle separation proves particularly advantageous for handling valuable samples in personalized medicine. This technology not only enhances detection, diagnosis, monitoring, and treatment accuracy, but also reduces invasiveness in medical procedures. This review summarizes the fundamentals of microfluidic nanoparticle separation techniques for precision medicine, starting with an examination of nanoparticle properties essential for separation and the core principles that guide various microfluidic methods. It then explores passive, active, and hybrid separation techniques, detailing their principles, structures, and applications. Furthermore, the review highlights their contributions to advancements in liquid biopsy and nanomedicine. Finally, it addresses existing challenges and envisions future development spurred by emerging technologies such as advanced materials science, 3D printing, and artificial intelligence. These interdisciplinary collaborations are anticipated to propel the platformization of microfluidic separation techniques, significantly expanding their potential in precision medicine.

Evaluating Brain Tumor Detection with Deep Learning Convolutional Neural Networks Across Multiple MRI Modalities

Central Nervous System (CNS) tumors represent a significant public health concern due to their high morbidity and mortality rates. Magnetic Resonance Imaging (MRI) has emerged as a critical non-invasive modality for the detection, diagnosis, and management of brain tumors, offering high-resolution visualization of anatomical structures. Recent advancements in deep learning, particularly convolutional neural networks (CNNs), have shown potential in augmenting MRI-based diagnostic accuracy for brain tumor detection. In this study, we evaluate the diagnostic performance of six fundamental MRI sequences in detecting tumor-involved brain slices using four distinct CNN architectures enhanced with transfer learning techniques. Our dataset comprises 1646 MRI slices from the examinations of 62 patients, encompassing both tumor-bearing and normal findings. With our approach, we achieved a classification accuracy of 98.6%, underscoring the high potential of CNN-based models in this context. Additionally, we assessed the performance of each MRI sequence across the different CNN models, identifying optimal combinations of MRI modalities and neural networks to meet radiologists' screening requirements effectively. This study offers critical insights into the integration of deep learning with MRI for brain tumor detection, with implications for improving diagnostic workflows in clinical settings.

Developing an AI-based application for caries index detection on intraoral photographs

This study evaluates the effectiveness of an Artificial Intelligence (AI)-based smartphone application designed for decay detection on intraoral photographs, comparing its performance to that of junior dentists. Conducted at The Aga Khan University Hospital, Karachi, Pakistan, this study utilized a dataset comprising 7,465 intraoral images, including both primary and secondary dentitions. These images were meticulously annotated by two experienced dentists and further verified by senior dentists. A YOLOv5s model was trained on this dataset and integrated into a smartphone application, while a Detection Transformer was also fine-tuned for comparative purposes. Explainable AI techniques were employed to assess the AI's decision-making processes. A sample of 70 photographs was used to directly compare the application's performance with that of junior dentists. Results showed that the YOLOv5s-based smartphone application achieved a precision of 90.7%, sensitivity of 85.6%, and an F1 score of 88.0% in detecting dental decay. In contrast, junior dentists achieved 83.3% precision, 64.1% sensitivity, and an F1 score of 72.4%. The study concludes that the YOLOv5s algorithm effectively detects dental decay on intraoral photographs and performs comparably to junior dentists. This application holds potential for aiding in the evaluation of the caries index within populations, thus contributing to efforts aimed at reducing the disease burden at the community level.

Multi-task neural networks by learned contextual inputs

This paper explores learned-context neural networks. It is a multi-task learning architecture based on a fully shared neural network and an augmented input vector containing trainable task parameters. The architecture is interesting due to its powerful task adaption mechanism, which facilitates a low-dimensional task parameter space. Theoretically, we show that a scalar task parameter is sufficient for universal approximation of all tasks, which is not necessarily the case for more common architectures. Empirically it is shown that, for homogeneous tasks, the dimension of the task parameter may vary with the complexity of the tasks, but a small task parameter space is generally viable. The task parameter space is found to be well-behaved, which simplifies workflows related to updating models as new data arrives, and learning new tasks with the shared parameters are frozen. Additionally, the architecture displays robustness towards datasets where tasks have few data points. The architecture's performance is compared to similar neural network architectures on ten datasets, with competitive results.

Suitability of GPT-4o as an evaluator of cardiopulmonary resuscitation skills examinations

AIM

To assess the accuracy and reliability of GPT-4o for scoring examinees' performance on cardiopulmonary resuscitation (CPR) skills tests.

METHODS

This study included six experts certified to supervise the national medical licensing examination (three junior and three senior) who reviewed the CPR skills test videos across 103 examinees. All videos reviewed by the experts were subjected to automated assessment by GPT-4o. Both the experts and GPT-4o scored the videos across four sections: patient assessment, chest compressions, rescue breathing, and repeated operations. The experts subsequently rated GPT-4o's reliability on a 5-point Likert scale (1, completely unreliable; 5, completely reliable). GPT-4o's accuracy was evaluated using the intraclass correlation coefficient (for the first three sections) and Fleiss' Kappa (for the last section) to assess the agreement between its scores vs. those of the experts.

RESULTS

The mean accuracy scores for the patient assessment, chest compressions, rescue breathing, and repeated operation sections were 0.65, 0.58, 0.60, and 0.31, respectively, when comparing the GPT-4o's vs. junior experts' scores and 0.75, 0.65, 0.72, and 0.41, respectively, when comparing the GPT-4o's vs. senior experts' scores. For reliability, the median Likert scale scores were 4.00 (interquartile range [IQR] = 3.66-4.33, mean [standard deviation] = 3.95 [0.55]) and 4.33 (4.00-4.67, 4.29 [0.50]) for the junior and senior experts, respectively.

CONCLUSIONS

GPT-4o demonstrated a level of accuracy that was similar to that of senior experts in examining CPR skills examination videos. The results demonstrate the potential for deploying this large language model in medical examination settings.

Application of radiomics for diagnosis, subtyping, and prognostication of medulloblastomas: a systematic review

Applications of radiomics for clinical management of medulloblastomas are expanding. This systematic review aims to aggregate and overview the current role of radiomics in the diagnosis, subtyping, and prognostication of medulloblastomas. The present systematic review was performed in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. PubMed/MEDLINE were searched using a standardized search term. Articles found within the database from the inception until November 2022 were considered for screening. Retrieved records were screened independently by two authors based on their titles and abstracts. The full text of selected articles was reviewed to finalize the eligibility. Due to the heterogeneity of included studies, no formal data synthesis was conducted. Of the 249 screened citations, 21 studies were included and analyzed. Radiomics demonstrated promising performance for discriminating medulloblastomas from other posterior fossa tumors, particularly ependymomas and pilocytic astrocytomas. It was also efficacious in determining the subtype (i.e., WNT+, SHH+, group 3, and group 4) of medulloblastomas non-invasively. Regarding prognostication, radiomics exhibited some ability to predict overall survival and progression-free survival of patients with medulloblastomas. Our systematic review revealed that radiomics represents a promising tool for diagnosis and prognostication of medulloblastomas. Further prospective research measuring the clinical value of radiomics in this setting is warranted.

Utilizing Pseudo Color Image to Improve the Performance of Deep Transfer Learning-Based Computer-Aided Diagnosis Schemes in Breast Mass Classification

The purpose of this study is to investigate the impact of using morphological information in classifying suspicious breast lesions. The widespread use of deep transfer learning can significantly improve the performance of the mammogram based CADx schemes. However, digital mammograms are grayscale images, while deep learning models are typically optimized using the natural images containing three channels. Thus, it is needed to convert the grayscale mammograms into three channel images for the input of deep transfer models. This study aims to develop a novel pseudo color image generation method which utilizes the mass contour information to enhance the classification performance. Accordingly, a total of 830 breast cancer cases were retrospectively collected, which contains 310 benign and 520 malignant cases, respectively. For each case, a total of four regions of interest (ROI) are collected from the grayscale images captured for both the CC and MLO views of the two breasts. Meanwhile, a total of seven pseudo color image sets are generated as the input of the deep learning models, which are created through a combination of the original grayscale image, a histogram equalized image, a bilaterally filtered image, and a segmented mass. Accordingly, the output features from four identical pre-trained deep learning models are concatenated and then processed by a support vector machine-based classifier to generate the final benign/malignant labels. The performance of each image set was evaluated and compared. The results demonstrate that the pseudo color sets containing the manually segmented mass performed significantly better than all other pseudo color sets, which achieved an AUC (area under the ROC curve) up to 0.889 ± 0.012 and an overall accuracy up to 0.816 ± 0.020, respectively. At the same time, the performance improvement is also dependent on the accuracy of the mass segmentation. The results of this study support our hypothesis that adding accurately segmented mass contours can provide complementary information, thereby enhancing the performance of the deep transfer model in classifying suspicious breast lesions.

Artificial intelligence performance in testing microfluidics for point-of-care

Artificial intelligence (AI) is revolutionizing medicine by automating tasks like image segmentation and pattern recognition. These AI approaches support seamless integration with existing platforms, enhancing diagnostics, treatment, and patient care. While recent advancements have demonstrated AI superiority in advancing microfluidics for point of care (POC) diagnostics, a gap remains in comparative evaluations of AI algorithms in testing microfluidics. We conducted a comparative evaluation of AI models specifically for the two-class classification problem of identifying the presence or absence of bubbles in microfluidic channels under various imaging conditions. Using a model microfluidic system with a single channel loaded with 3D transparent objects (bubbles), we challenged each of the tested machine learning (ML) (n = 6) and deep learning (DL) (n = 9) models across different background settings. Evaluation revealed that the random forest ML model achieved 95.52% sensitivity, 82.57% specificity, and 97% AUC, outperforming other ML algorithms. Among DL models suitable for mobile integration, DenseNet169 demonstrated superior performance, achieving 92.63% sensitivity, 92.22% specificity, and 92% AUC. Remarkably, DenseNet169 integration into a mobile POC system demonstrated exceptional accuracy (>0.84) in testing microfluidics at under challenging imaging settings. Our study confirms the transformative potential of AI in healthcare, emphasizing its capacity to revolutionize precision medicine through accurate and accessible diagnostics. The integration of AI into healthcare systems holds promise for enhancing patient outcomes and streamlining healthcare delivery.

Advances in Medical Image Segmentation: A Comprehensive Review of Traditional, Deep Learning and Hybrid Approaches

Medical image segmentation plays a critical role in accurate diagnosis and treatment planning, enabling precise analysis across a wide range of clinical tasks. This review begins by offering a comprehensive overview of traditional segmentation techniques, including thresholding, edge-based methods, region-based approaches, clustering, and graph-based segmentation. While these methods are computationally efficient and interpretable, they often face significant challenges when applied to complex, noisy, or variable medical images. The central focus of this review is the transformative impact of deep learning on medical image segmentation. We delve into prominent deep learning architectures such as Convolutional Neural Networks (CNNs), Fully Convolutional Networks (FCNs), U-Net, Recurrent Neural Networks (RNNs), Adversarial Networks (GANs), and Autoencoders (AEs). Each architecture is analyzed in terms of its structural foundation and specific application to medical image segmentation, illustrating how these models have enhanced segmentation accuracy across various clinical contexts. Finally, the review examines the integration of deep learning with traditional segmentation methods, addressing the limitations of both approaches. These hybrid strategies offer improved segmentation performance, particularly in challenging scenarios involving weak edges, noise, or inconsistent intensities. By synthesizing recent advancements, this review provides a detailed resource for researchers and practitioners, offering valuable insights into the current landscape and future directions of medical image segmentation.

Survival prediction in diffuse large B-cell lymphoma patients: multimodal PET/CT deep features radiomic model utilizing automated machine learning

PURPOSE

We sought to develop an effective combined model for predicting the survival of patients with diffuse large B-cell lymphoma (DLBCL) based on the multimodal PET-CT deep features radiomics signature (DFR-signature).

METHODS

369 DLBCL patients from two medical centers were included in this study. Their PET and CT images were fused to construct the multimodal PET-CT images using a deep learning fusion network. Then the deep features were extracted from those fused PET-CT images, and the DFR-signature was constructed through an Automated machine learning (AutoML) model. Combined with clinical indexes from the Cox regression analysis, we constructed a combined model to predict the progression-free survival (PFS) and the overall survival (OS) of patients. In addition, the combined model was evaluated in the concordance index (C-index) and the time-dependent area under the ROC curve (tdAUC).

RESULTS

A total of 1000 deep features were extracted to build a DFR-signature. Besides the DFR-signature, the combined model integrating metabolic and clinical factors performed best in terms of PFS and OS. For PFS, the C-indices are 0.784 and 0.739 in the training cohort and internal validation cohort, respectively. For OS, the C-indices are 0.831 and 0.782 in the training cohort and internal validation cohort.

CONCLUSIONS

DFR-signature constructed from multimodal images improved the classification accuracy of prognosis for DLBCL patients. Moreover, the constructed DFR-signature combined with NCCN-IPI exhibited excellent potential for risk stratification of DLBCL patients.

Medical image foundation models in assisting diagnosis of brain tumors: a pilot study

OBJECTIVES

To build self-supervised foundation models for multicontrast MRI of the whole brain and evaluate their efficacy in assisting diagnosis of brain tumors.

METHODS

In this retrospective study, foundation models were developed using 57,621 enhanced head MRI scans through self-supervised learning with a pretext task of cross-contrast context restoration with two different content dropout schemes. Downstream classifiers were constructed based on the pretrained foundation models and fine-tuned for brain tumor detection, discrimination, and molecular status prediction. Metrics including accuracy, sensitivity, specificity, and area under the ROC curve (AUC) were used to evaluate the performance. Convolutional neural networks trained exclusively on downstream task data were employed for comparative analysis.

RESULTS

The pretrained foundation models demonstrated their ability to extract effective representations from multicontrast whole-brain volumes. The best classifiers, endowed with pretrained weights, showed remarkable performance with accuracies of 94.9, 92.3, and 80.4%, and corresponding AUC values of 0.981, 0.972, and 0.852 on independent test datasets in brain tumor detection, discrimination, and molecular status prediction, respectively. The classifiers with pretrained weights outperformed the convolutional classifiers trained from scratch by approximately 10% in terms of accuracy and AUC across all tasks. The saliency regions in the correctly predicted cases are mainly clustered around the tumors. Classifiers derived from the two dropout schemes differed significantly only in the detection of brain tumors.

CONCLUSIONS

Foundation models obtained from self-supervised learning have demonstrated encouraging potential for scalability and interpretability in downstream brain tumor-related tasks and hold promise for extension to neurological diseases with diffusely distributed lesions.

CLINICAL RELEVANCE STATEMENT

The application of our proposed method to the prediction of key molecular status in gliomas is expected to improve treatment planning and patient outcomes. Additionally, the foundation model we developed could serve as a cornerstone for advancing AI applications in the diagnosis of brain-related diseases.

Automated segmentation and classification of supraspinatus fatty infiltration in shoulder magnetic resonance image using a convolutional neural network

Background

Goutallier's fatty infiltration of the supraspinatus muscle is a critical condition in degenerative shoulder disorders. Deep learning research primarily uses manual segmentation and labeling to detect this condition. Employing unsupervised training with a hybrid framework of segmentation and classification could offer an efficient solution.

Aim

To develop and assess a two-step deep learning model for detecting the region of interest and categorizing the magnetic resonance image (MRI) supraspinatus muscle fatty infiltration according to Goutallier's scale.

Materials and methods

A retrospective study was performed from January 1, 2019 to September 20, 2020, using 900 MRI T2-weighted images with supraspinatus muscle fatty infiltration diagnoses. A model with two sequential neural networks was implemented and trained. The first sub-model automatically detects the region of interest using a U-Net model. The second sub-model performs a binary classification using the VGG-19 architecture. The model's performance was computed as the average of five-fold cross-validation processes. Loss, accuracy, Dice coefficient (CI. 95%), AU-ROC, sensitivity, and specificity (CI. 95%) were reported.

Results

Six hundred and six shoulders MRIs were analyzed. The Goutallier distribution was presented as follows: 0 (66.50%); 1 (18.81%); 2 (8.42%); 3 (3.96%); 4 (2.31%). Segmentation results demonstrate high levels of accuracy (0.9977 ± 0.0002) and Dice score (0.9441 ± 0.0031), while the classification model also results in high levels of accuracy (0.9731 ± 0.0230); sensitivity (0.9000 ± 0.0980); specificity (0.9788 ± 0.0257); and AUROC (0.9903 ± 0.0092).

Conclusion

The two-step training method proposed using a deep learning model demonstrated strong performance in segmentation and classification tasks.

RA-XTNet: A Novel CNN Model to Predict Rheumatoid Arthritis from Hand Radiographs and Thermal Images: A Comparison with CNN Transformer and Quantum Computing

The aim and objective of the research are to develop an automated diagnosis system for the prediction of rheumatoid arthritis (RA) based on artificial intelligence (AI) and quantum computing for hand radiographs and thermal images. The hand radiographs and thermal images were segmented using a UNet++ model and color-based k-means clustering technique, respectively. The attributes from the segmented regions were generated using the Speeded-Up Robust Features (SURF) feature extractor and classification was performed using k-star and Hoeffding classifiers. For the ground truth and the predicted test image, the study utilizing UNet++ segmentation achieved a pixel-wise accuracy of 98.75%, an intersection over union (IoU) of 0.87, and a dice coefficient of 0.86, indicating a high level of similarity. The custom RA-X-ray thermal imaging (XTNet) surpassed all the models for the detection of RA with a classification accuracy of 90% and 93% for X-ray and thermal imaging modalities, respectively. Furthermore, the study employed quantum support vector machine (QSVM) as a quantum computing approach which yielded an accuracy of 93.75% and 87.5% for the detection of RA from hand X-ray and thermal images. In addition, vision transformer (ViT) was employed to classify RA which obtained an accuracy of 80% for hand X-rays and 90% for thermal images. Thus, depending on the performance measures, the RA-XTNet model can be used as an effective automated diagnostic method to diagnose RA accurately and rapidly in hand radiographs and thermal images.

Novel Hybrid Quantum Architecture-Based Lung Cancer Detection Using Chest Radiograph and Computerized Tomography Images

Lung cancer, the second most common type of cancer worldwide, presents significant health challenges. Detecting this disease early is essential for improving patient outcomes and simplifying treatment. In this study, we propose a hybrid framework that combines deep learning (DL) with quantum computing to enhance the accuracy of lung cancer detection using chest radiographs (CXR) and computerized tomography (CT) images. Our system utilizes pre-trained models for feature extraction and quantum circuits for classification, achieving state-of-the-art performance in various metrics. Not only does our system achieve an overall accuracy of 92.12%, it also excels in other crucial performance measures, such as sensitivity (94%), specificity (90%), F1-score (93%), and precision (92%). These results demonstrate that our hybrid approach can more accurately identify lung cancer signatures compared to traditional methods. Moreover, the incorporation of quantum computing enhances processing speed and scalability, making our system a promising tool for early lung cancer screening and diagnosis. By leveraging the strengths of quantum computing, our approach surpasses traditional methods in terms of speed, accuracy, and efficiency. This study highlights the potential of hybrid computational technologies to transform early cancer detection, paving the way for wider clinical applications and improved patient care outcomes.

Application of artificial intelligence in cancer diagnosis and tumor nanomedicine

Cancer is a major health concern due to its high incidence and mortality rates. Advances in cancer research, particularly in artificial intelligence (AI) and deep learning, have shown significant progress. The swift evolution of AI in healthcare, especially in tools like computer-aided diagnosis, has the potential to revolutionize early cancer detection. This technology offers improved speed, accuracy, and sensitivity, bringing a transformative impact on cancer diagnosis, treatment, and management. This paper provides a concise overview of the application of artificial intelligence in the realms of medicine and nanomedicine, with a specific emphasis on the significance and challenges associated with cancer diagnosis. It explores the pivotal role of AI in cancer diagnosis, leveraging structured, unstructured, and multimodal fusion data. Additionally, the article delves into the applications of AI in nanomedicine sensors and nano-oncology drugs. The fundamentals of deep learning and convolutional neural networks are clarified, underscoring their relevance to AI-driven cancer diagnosis. A comparative analysis is presented, highlighting the accuracy and efficiency of traditional methods juxtaposed with AI-based approaches. The discussion not only assesses the current state of AI in cancer diagnosis but also delves into the challenges faced by AI in this context. Furthermore, the article envisions the future development direction and potential application of artificial intelligence in cancer diagnosis, offering a hopeful prospect for enhanced cancer detection and improved patient prognosis.

Advancing maxillofacial prosthodontics by using pre-trained convolutional neural networks: Image-based classification of the maxilla

PURPOSE

The study aimed to compare the performance of four pre-trained convolutional neural networks in recognizing seven distinct prosthodontic scenarios involving the maxilla, as a preliminary step in developing an artificial intelligence (AI)-powered prosthesis design system.

MATERIALS AND METHODS

Seven distinct classes, including cleft palate, dentulous maxillectomy, edentulous maxillectomy, reconstructed maxillectomy, completely dentulous, partially edentulous, and completely edentulous, were considered for recognition. Utilizing transfer learning and fine-tuned hyperparameters, four AI models (VGG16, Inception-ResNet-V2, DenseNet-201, and Xception) were employed. The dataset, consisting of 3541 preprocessed intraoral occlusal images, was divided into training, validation, and test sets. Model performance metrics encompassed accuracy, precision, recall, F1 score, area under the receiver operating characteristic curve (AUC), and confusion matrix.

RESULTS

VGG16, Inception-ResNet-V2, DenseNet-201, and Xception demonstrated comparable performance, with maximum test accuracies of 0.92, 0.90, 0.94, and 0.95, respectively. Xception and DenseNet-201 slightly outperformed the other models, particularly compared with InceptionResNet-V2. Precision, recall, and F1 scores exceeded 90% for most classes in Xception and DenseNet-201 and the average AUC values for all models ranged between 0.98 and 1.00.

CONCLUSIONS

While DenseNet-201 and Xception demonstrated superior performance, all models consistently achieved diagnostic accuracy exceeding 90%, highlighting their potential in dental image analysis. This AI application could help work assignments based on difficulty levels and enable the development of an automated diagnosis system at patient admission. It also facilitates prosthesis designing by integrating necessary prosthesis morphology, oral function, and treatment difficulty. Furthermore, it tackles dataset size challenges in model optimization, providing valuable insights for future research.

Selection of pre-trained weights for transfer learning in automated cytomegalovirus retinitis classification

Cytomegalovirus retinitis (CMVR) is a significant cause of vision loss. Regular screening is crucial but challenging in resource-limited settings. A convolutional neural network is a state-of-the-art deep learning technique to generate automatic diagnoses from retinal images. However, there are limited numbers of CMVR images to train the model properly. Transfer learning (TL) is a strategy to train a model with a scarce dataset. This study explores the efficacy of TL with different pre-trained weights for automated CMVR classification using retinal images. We utilised a dataset of 955 retinal images (524 CMVR and 431 normal) from Siriraj Hospital, Mahidol University, collected between 2005 and 2015. Images were processed using Kowa VX-10i or VX-20 fundus cameras and augmented for training. We employed DenseNet121 as a backbone model, comparing the performance of TL with weights pre-trained on ImageNet, APTOS2019, and CheXNet datasets. The models were evaluated based on accuracy, loss, and other performance metrics, with the depth of fine-tuning varied across different pre-trained weights. The study found that TL significantly enhances model performance in CMVR classification. The best results were achieved with weights sequentially transferred from ImageNet to APTOS2019 dataset before application to our CMVR dataset. This approach yielded the highest mean accuracy (0.99) and lowest mean loss (0.04), outperforming other methods. The class activation heatmaps provided insights into the model's decision-making process. The model with APTOS2019 pre-trained weights offered the best explanation and highlighted the pathologic lesions resembling human interpretation. Our findings demonstrate the potential of sequential TL in improving the accuracy and efficiency of CMVR diagnosis, particularly in settings with limited data availability. They highlight the importance of domain-specific pre-training in medical image classification. This approach streamlines the diagnostic process and paves the way for broader applications in automated medical image analysis, offering a scalable solution for early disease detection.

Comparison of deep learning architectures for predicting amyloid positivity in Alzheimer's disease, mild cognitive impairment, and healthy aging, from T1-weighted brain structural MRI

Abnormal β-amyloid (Aβ) accumulation in the brain is an early indicator of Alzheimer's disease (AD) and is typically assessed through invasive procedures such as PET (positron emission tomography) or CSF (cerebrospinal fluid) assays. As new anti-Alzheimer's treatments can now successfully target amyloid pathology, there is a growing interest in predicting Aβ positivity (Aβ+) from less invasive, more widely available types of brain scans, such as T1-weighted (T1w) MRI. Here we compare multiple approaches to infer Aβ + from standard anatomical MRI: (1) classical machine learning algorithms, including logistic regression, XGBoost, and shallow artificial neural networks, (2) deep learning models based on 2D and 3D convolutional neural networks (CNNs), (3) a hybrid ANN-CNN, combining the strengths of shallow and deep neural networks, (4) transfer learning models based on CNNs, and (5) 3D Vision Transformers. All models were trained on paired MRI/PET data from 1,847 elderly participants (mean age: 75.1 yrs. ± 7.6SD; 863 females/984 males; 661 healthy controls, 889 with mild cognitive impairment (MCI), and 297 with Dementia), scanned as part of the Alzheimer's Disease Neuroimaging Initiative. We evaluated each model's balanced accuracy and F1 scores. While further tests on more diverse data are warranted, deep learning models trained on standard MRI showed promise for estimating Aβ + status, at least in people with MCI. This may offer a potential screening option before resorting to more invasive procedures.

Image Quality Assessment Using Convolutional Neural Network in Clinical Skin Images

The image quality received for clinical evaluation is often suboptimal. The goal is to develop an image quality analysis tool to assess patient- and primary care physician-derived images using deep learning model. Dataset included patient- and primary care physician-derived images from August 21, 2018 to June 30, 2022 with 4 unique quality labels. VGG16 model was fine tuned with input data, and optimal threshold was determined by Youden's index. Ordinal labels were transformed to binary labels using a majority vote because model distinguishes between 2 categories (good vs bad). At a threshold of 0.587, area under the curve for the test set was 0.885 (95% confidence interval = 0.838-0.933); sensitivity, specificity, positive predictive value, and negative predictive value were 0.829, 0.784, 0.906, and 0.645, respectively. Independent validation of 300 additional images (from patients and primary care physicians) demonstrated area under the curve of 0.864 (95% confidence interval = 0.818-0.909) and area under the curve of 0.902 (95% confidence interval = 0.85-0.95), respectively. The sensitivity, specificity, positive predictive value, and negative predictive value for the 300 images were 0.827, 0.800, 0.959, and 0.450, respectively. We demonstrate a practical approach improving the image quality for clinical workflow. Although users may have to capture additional images, this is offset by the improved workload and efficiency for clinical teams.

Augmenting Radiological Diagnostics with AI for Tuberculosis and COVID-19 Disease Detection: Deep Learning Detection of Chest Radiographs

In this research, we introduce a network that can identify pneumonia, COVID-19, and tuberculosis using X-ray images of patients' chests. The study emphasizes tuberculosis, COVID-19, and healthy lung conditions, discussing how advanced neural networks, like VGG16 and ResNet50, can improve the detection of lung issues from images. To prepare the images for the model's input requirements, we enhanced them through data augmentation techniques for training purposes. We evaluated the model's performance by analyzing the precision, recall, and F1 scores across training, validation, and testing datasets. The results show that the ResNet50 model outperformed VGG16 with accuracy and resilience. It displayed superior ROC AUC values in both validation and test scenarios. Particularly impressive were ResNet50's precision and recall rates, nearing 0.99 for all conditions in the test set. On the hand, VGG16 also performed well during testing-detecting tuberculosis with a precision of 0.99 and a recall of 0.93. Our study highlights the performance of our deep learning method by showcasing the effectiveness of ResNet50 over traditional approaches like VGG16. This progress utilizes methods to enhance classification accuracy by augmenting data and balancing them. This positions our approach as an advancement in using state-of-the-art deep learning applications in imaging. By enhancing the accuracy and reliability of diagnosing ailments such as COVID-19 and tuberculosis, our models have the potential to transform care and treatment strategies, highlighting their role in clinical diagnostics.

ImSpect: Image-driven self-supervised learning for surgical margin evaluation with mass spectrometry

PURPOSE

Real-time assessment of surgical margins is critical for favorable outcomes in cancer patients. The iKnife is a mass spectrometry device that has demonstrated potential for margin detection in cancer surgery. Previous studies have shown that using deep learning on iKnife data can facilitate real-time tissue characterization. However, none of the existing literature on the iKnife facilitate the use of publicly available, state-of-the-art pretrained networks or datasets that have been used in computer vision and other domains.

METHODS

In a new framework we call ImSpect, we convert 1D iKnife data, captured during basal cell carcinoma (BCC) surgery, into 2D images in order to capitalize on state-of-the-art image classification networks. We also use self-supervision to leverage large amounts of unlabeled, intraoperative data to accommodate the data requirements of these networks.

RESULTS

Through extensive ablation studies, we show that we can surpass previous benchmarks of margin evaluation in BCC surgery using iKnife data, achieving an area under the receiver operating characteristic curve (AUC) of 81%. We also depict the attention maps of the developed DL models to evaluate the biological relevance of the embedding space CONCLUSIONS: We propose a new method for characterizing tissue at the surgical margins, using mass spectrometry data from cancer surgery.

Sex estimation from maxillofacial radiographs using a deep learning approach

The purpose of this study was to construct deep learning models for more efficient and reliable sex estimation. Two deep learning models, VGG16 and DenseNet-121, were used in this retrospective study. In total, 600 lateral cephalograms were analyzed. A saliency map was generated by gradient-weighted class activation mapping for each output. The two deep learning models achieved high values in each performance metric according to accuracy, sensitivity (recall), precision, F1 score, and areas under the receiver operating characteristic curve. Both models showed substantial differences in the positions indicated in saliency maps for male and female images. The positions in saliency maps also differed between VGG16 and DenseNet-121, regardless of sex. This analysis of our proposed system suggested that sex estimation from lateral cephalograms can be achieved with high accuracy using deep learning.

Semi-supervised model based on implicit neural representation and mutual learning (SIMN) for multi-center nasopharyngeal carcinoma segmentation on MRI

BACKGROUND

The issue of using deep learning to obtain accurate gross tumor volume (GTV) and metastatic lymph nodes (MLN) segmentation for nasopharyngeal carcinoma (NPC) on heterogeneous magnetic resonance imaging (MRI) images with limited labeling remains unsolved.

METHOD

We collected 918 patients with MRI images from three hospitals to develop and validate models and proposed a semi-supervised framework for the fine delineation of multi-center NPC boundaries by integrating uncertainty-based implicit neural representations named SIMN. The framework utilizes the deep mutual learning approach with CNN and Transformer, incorporating dynamic thresholds. Additionally, domain adaptive algorithms are employed to enhance the performance.

RESULTS

SIMN predictions have a high overlap ratio with the ground truth. Under the 20 % labeled cases, for the internal test cohorts, the average DSC in GTV and MLN are 0.7981 and 0.7804, respectively; for external test cohort Wu Zhou Red Cross Hospital, the average DSC in GTV and MLN are 0.7217 and 0.7581, respectively; for external test cohorts First People Hospital of Foshan, the average DSC in GTV and MLN are 0.7004 and 0.7692, respectively. No significant differences are found in DSC, HD95, ASD, and Recall for patients with different clinical categories. Moreover, SIMN outperformed existing classical semi-supervised methods.

CONCLUSIONS

SIMN showed a highly accurate GTV and MLN segmentation for NPC on multi-center MRI images under Semi-Supervised Learning (SSL), which can easily transfer to other centers without fine-tuning. It suggests that it has the potential to act as a generalized delineation solution for heterogeneous MRI images with limited labels in clinical deployment.

BarlowTwins-CXR: enhancing chest X-ray abnormality localization in heterogeneous data with cross-domain self-supervised learning

BACKGROUND

Chest X-ray imaging based abnormality localization, essential in diagnosing various diseases, faces significant clinical challenges due to complex interpretations and the growing workload of radiologists. While recent advances in deep learning offer promising solutions, there is still a critical issue of domain inconsistency in cross-domain transfer learning, which hampers the efficiency and accuracy of diagnostic processes. This study aims to address the domain inconsistency problem and improve autonomic abnormality localization performance of heterogeneous chest X-ray image analysis, particularly in detecting abnormalities, by developing a self-supervised learning strategy called "BarlwoTwins-CXR".

METHODS

We utilized two publicly available datasets: the NIH Chest X-ray Dataset and the VinDr-CXR. The BarlowTwins-CXR approach was conducted in a two-stage training process. Initially, self-supervised pre-training was performed using an adjusted Barlow Twins algorithm on the NIH dataset with a Resnet50 backbone pre-trained on ImageNet. This was followed by supervised fine-tuning on the VinDr-CXR dataset using Faster R-CNN with Feature Pyramid Network (FPN). The study employed mean Average Precision (mAP) at an Intersection over Union (IoU) of 50% and Area Under the Curve (AUC) for performance evaluation.

RESULTS

Our experiments showed a significant improvement in model performance with BarlowTwins-CXR. The approach achieved a 3% increase in mAP50 accuracy compared to traditional ImageNet pre-trained models. In addition, the Ablation CAM method revealed enhanced precision in localizing chest abnormalities. The study involved 112,120 images from the NIH dataset and 18,000 images from the VinDr-CXR dataset, indicating robust training and testing samples.

CONCLUSION

BarlowTwins-CXR significantly enhances the efficiency and accuracy of chest X-ray image-based abnormality localization, outperforming traditional transfer learning methods and effectively overcoming domain inconsistency in cross-domain scenarios. Our experiment results demonstrate the potential of using self-supervised learning to improve the generalizability of models in medical settings with limited amounts of heterogeneous data. This approach can be instrumental in aiding radiologists, particularly in high-workload environments, offering a promising direction for future AI-driven healthcare solutions.

Convolutional Neural Network to Classify Infrared Thermal Images of Fractured Wrists in Pediatrics

Convolutional neural network (CNN) models were devised and evaluated to classify infrared thermal (IRT) images of pediatric wrist fractures. The images were recorded from 19 participants with a wrist fracture and 21 without a fracture (sprain). The injury diagnosis was by X-ray radiography. For each participant, 299 IRT images of their wrists were recorded. These generated 11,960 images (40 participants × 299 images). For each image, the wrist region of interest (ROI) was selected and fast Fourier transformed (FFT) to obtain a magnitude frequency spectrum. The spectrum was resized to 100 × 100 pixels from its center as this region represented the main frequency components. Image augmentations of rotation, translation and shearing were applied to the 11,960 magnitude frequency spectra to assist with the CNN generalization during training. The CNN had 34 layers associated with convolution, batch normalization, rectified linear unit, maximum pooling and SoftMax and classification. The ratio of images for the training and test was 70:30, respectively. The effects of augmentation and dropout on CNN performance were explored. Wrist fracture identification sensitivity and accuracy of 88% and 76%, respectively, were achieved. The CNN model was able to identify wrist fractures; however, a larger sample size would improve accuracy.

OPG-based dental age estimation using a data-technical exploration of deep learning techniques

Dental age estimation, a cornerstone in forensic age assessment, has been extensively tried and tested, yet manual methods are impeded by tedium and interobserver variability. Automated approaches using deep transfer learning encounter challenges like data scarcity, suboptimal training, and fine-tuning complexities, necessitating robust training methods. This study explores the impact of convolutional neural network hyperparameters, model complexity, training batch size, and sample quantity on age estimation. EfficientNet-B4, DenseNet-201, and MobileNet V3 models underwent cross-validation on a dataset of 3896 orthopantomograms (OPGs) with batch sizes escalating from 10 to 160 in a doubling progression, as well as random subsets of this training dataset. Results demonstrate the EfficientNet-B4 model, trained on the complete dataset with a batch size of 160, as the top performer with a mean absolute error of 0.562 years on the test set, notably surpassing the MAE of 1.01 at a batch size of 10. Increasing batch size consistently improved performance for EfficientNet-B4 and DenseNet-201, whereas MobileNet V3 performance peaked at batch size 40. Similar trends emerged in training with reduced sample sizes, though they were outperformed by the complete models. This underscores the critical role of hyperparameter optimization in adopting deep learning for age estimation from complete OPGs. The findings not only highlight the nuanced interplay of hyperparameters and performance but also underscore the potential for accurate age estimation models through optimization. This study contributes to advancing the application of deep learning in forensic age estimation, emphasizing the significance of tailored training methodologies for optimal outcomes.

Detecting dental caries on oral photographs using artificial intelligence: A systematic review

OBJECTIVES

This systematic review aimed at evaluating the performance of artificial intelligence (AI) models in detecting dental caries on oral photographs.

METHODS

Methodological characteristics and performance metrics of clinical studies reporting on deep learning and other machine learning algorithms were assessed. The risk of bias was evaluated using the quality assessment of diagnostic accuracy studies 2 (QUADAS-2) tool. A systematic search was conducted in EMBASE, Medline, and Scopus.

RESULTS

Out of 3410 identified records, 19 studies were included with six and seven studies having low risk of biases and applicability concerns for all the domains, respectively. Metrics varied widely and were assessed on multiple levels. F1-scores for classification and detection tasks were 68.3%-94.3% and 42.8%-95.4%, respectively. Irrespective of the task, F1-scores were 68.3%-95.4% for professional cameras, 78.8%-87.6%, for intraoral cameras, and 42.8%-80% for smartphone cameras. Limited studies allowed assessing AI performance for lesions of different severity.

CONCLUSION

Automatic detection of dental caries using AI may provide objective verification of clinicians' diagnoses and facilitate patient-clinician communication and teledentistry. Future studies should consider more robust study designs, employ comparable and standardized metrics, and focus on the severity of caries lesions.

Automated detection of otosclerosis with interpretable deep learning using temporal bone computed tomography images

Objective

This study aimed to develop an automated detection schema for otosclerosis with interpretable deep learning using temporal bone computed tomography images.

Methods

With approval from the institutional review board, we retrospectively analyzed high-resolution computed tomography scans of the temporal bone of 182 participants with otosclerosis (67 male subjects and 115 female subjects; average age, 36.42 years) and 157 participants without otosclerosis (52 male subjects and 102 female subjects; average age, 30.61 years) using deep learning. Transfer learning with the pretrained VGG19, Mask RCNN, and EfficientNet models was used. In addition, 3 clinical experts compared the system's performance by reading the same computed tomography images for a subset of 35 unseen subjects. An area under the receiver operating characteristic curve and a saliency map were used to further evaluate the diagnostic performance.

Results

In prospective unseen test data, the diagnostic performance of the automatically interpretable otosclerosis detection system at the optimal threshold was 0.97 and 0.98 for sensitivity and specificity, respectively. In comparison with the clinical acumen of otolaryngologists at P < 0.05, the proposed system was not significantly different. Moreover, the area under the receiver operating characteristic curve for the proposed system was 0.99, indicating satisfactory diagnostic accuracy.

Conclusion

Our research develops and evaluates a deep learning system that detects otosclerosis at a level comparable with clinical otolaryngologists. Our system is an effective schema for the differential diagnosis of otosclerosis in computed tomography examinations.

Rule-based deep learning method for prognosis of neonatal hypoxic-ischemic encephalopathy by using susceptibility weighted image analysis

OBJECTIVE

Susceptibility weighted imaging (SWI) of neonatal hypoxic-ischemic brain injury can provide assistance in the prognosis of neonatal hypoxic-ischemic encephalopathy (HIE). We propose a convolutional neural network model to classify SWI images with HIE.

MATERIALS AND METHODS

Due to the lack of a large dataset, transfer learning method with fine-tuning a pre-trained ResNet 50 is introduced. We randomly select 11 datasets from patients with normal neurology outcomes (n = 31) and patients with abnormal neurology outcomes (n = 11) at 24 months of age to avoid bias in classification due to any imbalance in the data.

RESULTS

We develop a rule-based system to improve the classification performance, with an accuracy of 0.93 ± 0.09. We also compute heatmaps produced by the Grad-CAM technique to analyze which areas of SWI images contributed more to the classification patients with abnormal neurology outcome.

CONCLUSION

Such regions that are important in the classification accuracy can interpret the relationship between the brain regions affected by hypoxic-ischemic and neurodevelopmental outcomes of infants with HIE at the age of 2 years.

Transfer learning for the generalization of artificial intelligence in breast cancer detection: a case-control study

BACKGROUND

Some researchers have questioned whether artificial intelligence (AI) systems maintain their performance when used for women from populations not considered during the development of the system.

PURPOSE

To evaluate the impact of transfer learning as a way of improving the generalization of AI systems in the detection of breast cancer.

MATERIAL AND METHODS

This retrospective case-control Finnish study involved 191 women diagnosed with breast cancer and 191 matched healthy controls. We selected a state-of-the-art AI system for breast cancer detection trained using a large US dataset. The selected baseline system was evaluated in two experimental settings. First, we examined our private Finnish sample as an independent test set that had not been considered in the development of the system (unseen population). Second, the baseline system was retrained to attempt to improve its performance in the unseen population by means of transfer learning. To analyze performance, we used areas under the receiver operating characteristic curve (AUCs) with DeLong's test.

RESULTS

Two versions of the baseline system were considered: ImageOnly and Heatmaps. The ImageOnly and Heatmaps versions yielded mean AUC values of 0.82±0.008 and 0.88±0.003 in the US dataset and 0.56 (95% CI=0.50-0.62) and 0.72 (95% CI=0.67-0.77) when evaluated in the unseen population, respectively. The retrained systems achieved AUC values of 0.61 (95% CI=0.55-0.66) and 0.69 (95% CI=0.64-0.75), respectively. There was no statistical difference between the baseline system and the retrained system.

CONCLUSION

Transfer learning with a small study sample did not yield a significant improvement in the generalization of the system.

Risk prediction of pulse wave for hypertensive target organ damage based on frequency-domain feature map

The application of deep learning to the classification of pulse waves in Traditional Chinese Medicine (TCM) related to hypertensive target organ damage (TOD) is hindered by challenges such as low classification accuracy and inadequate generalization performance. To address these challenges, we introduce a lightweight transfer learning model named MobileNetV2SCP. This model transforms time-domain pulse waves into 36-dimensional frequency-domain waveform feature maps and establishes a dedicated pre-training network based on these maps to enhance the learning capability for small samples. To improve global feature correlation, we incorporate a novel fusion attention mechanism (SAS) into the inverted residual structure, along with the utilization of 3 × 3 convolutional layers and BatchNorm layers to mitigate model overfitting. The proposed model is evaluated using cross-validation results from 805 cases of pulse waves associated with hypertensive TOD. The assessment metrics, including Accuracy (92.74 %), F1-score (91.47 %), and Area Under Curve (AUC) (97.12 %), demonstrate superior classification accuracy and generalization performance compared to various state-of-the-art models. Furthermore, this study investigates the correlations between time-domain and frequency-domain features in pulse waves and their classification in hypertensive TOD. It analyzes key factors influencing pulse wave classification, providing valuable insights for the clinical diagnosis of TOD.

Feature extraction of particle morphologies of pharmaceutical excipients from scanning electron microscope images using convolutional neural networks

Scanning electron microscopy (SEM) images are the most widely used tool for evaluating particle morphology; however, quantitative evaluation using SEM images is time-consuming and often neglected. In this study, we aimed to extract features related to particle morphology of pharmaceutical excipients from SEM images using a convolutional neural network (CNN). SEM images of 67 excipients were acquired and used as models. A classification CNN model of the excipients was constructed based on the SEM images. Further, features were extracted from the middle layer of this CNN model, and the data was compressed to two dimensions using uniform manifold approximation and projection. Lastly, hierarchical clustering analysis (HCA) was performed to categorize the excipients into several clusters and identify similarities among the samples. The classification CNN model showed high accuracy, allowing each excipient to be identified with a high degree of accuracy. HCA revealed that the 67 excipients were classified into seven clusters. Additionally, the particle morphologies of excipients belonging to the same cluster were found to be very similar. These results suggest that CNN models are useful tools for extracting information and identifying similarities among the particle morphologies of excipients.