Diagnosis of Esophageal Lesions by Multi-Classification and Segmentation Using an Improved Multi-Task Deep Learning Model

Machine learning applications for early detection of esophageal cancer: a systematic review

INTRODUCTION

Esophageal cancer (EC) is a significant global health problem, with an estimated 7th highest incidence and 6th highest mortality rate. Timely diagnosis and treatment are critical for improving patients' outcomes, as over 40% of patients with EC are diagnosed after metastasis. Recent advances in machine learning (ML) techniques, particularly in computer vision, have demonstrated promising applications in medical image processing, assisting clinicians in making more accurate and faster diagnostic decisions. Given the significance of early detection of EC, this systematic review aims to summarize and discuss the current state of research on ML-based methods for the early detection of EC.

METHODS

We conducted a comprehensive systematic search of five databases (PubMed, Scopus, Web of Science, Wiley, and IEEE) using search terms such as "ML", "Deep Learning (DL (", "Neural Networks (NN)", "Esophagus", "EC" and "Early Detection". After applying inclusion and exclusion criteria, 31 articles were retained for full review.

RESULTS

The results of this review highlight the potential of ML-based methods in the early detection of EC. The average accuracy of the reviewed methods in the analysis of endoscopic and computed tomography (CT (images of the esophagus was over 89%, indicating a high impact on early detection of EC. Additionally, the highest percentage of clinical images used in the early detection of EC with the use of ML was related to white light imaging (WLI) images. Among all ML techniques, methods based on convolutional neural networks (CNN) achieved higher accuracy and sensitivity in the early detection of EC compared to other methods.

CONCLUSION

Our findings suggest that ML methods may improve accuracy in the early detection of EC, potentially supporting radiologists, endoscopists, and pathologists in diagnosis and treatment planning. However, the current literature is limited, and more studies are needed to investigate the clinical applications of these methods in early detection of EC. Furthermore, many studies suffer from class imbalance and biases, highlighting the need for validation of detection algorithms across organizations in longitudinal studies.

CS-based multi-task learning network for arrhythmia reconstruction and classification using ECG signals

Objective. Although deep learning-based current methods have achieved impressive results in electrocardiograph (ECG) arrhythmia classification issues, they rely on using the original data to identify arrhythmia categories. However, a large amount of data generated by long-term ECG monitoring pose a significant challenge to the limited-bandwidth and real-time systems, which limits the application of deep learning in ECG monitoring.Approach. This paper, therefore, proposed a novel multi-task network that combined compressed sensing and convolutional neural networks, namely CSML-Net. According to the proposed model, the ECG signals were compressed by utilizing a learning measurement matrix and then recovered and classified simultaneously via shared layers and two task branches. Among them, the multi-scale feature module was designed to improve model performance.Main results. Experimental results on the MIT-BIH arrhythmia dataset demonstrate that our proposed method is superior to all the approaches that have been compared in terms of reconstruction quality and classification performance.Significance. Consequently, the proposed model achieving the reconstruction and classification in the compressed domain can be an improvement and become a promising approach for ECG arrhythmia reconstruction and classification.

Artificial intelligence assisted detection of superficial esophageal squamous cell carcinoma in white-light endoscopic images by using a generalized system

BACKGROUND

The use of artificial intelligence (AI) assisted white light imaging (WLI) detection systems for superficial esophageal squamous cell carcinoma (SESCC) is limited by training with images from one specific endoscopy platform.

METHODS

In this study, we developed an AI system with a convolutional neural network (CNN) model using WLI images from Olympus and Fujifilm endoscopy platforms. The training dataset consisted of 5892 WLI images from 1283 patients, and the validation dataset included 4529 images from 1224 patients. We assessed the diagnostic performance of the AI system and compared it with that of endoscopists. We analyzed the system's ability to identify cancerous imaging characteristics and investigated the efficacy of the AI system as an assistant in diagnosis.

RESULTS

In the internal validation set, the AI system's per-image analysis had a sensitivity, specificity, accuracy, positive predictive value (PPV), and negative predictive value (NPV) of 96.64%, 95.35%, 91.75%, 90.91%, and 98.33%, respectively. In patient-based analysis, these values were 90.17%, 94.34%, 88.38%, 89.50%, and 94.72%, respectively. The diagnostic results in the external validation set were also favorable. The CNN model's diagnostic performance in recognizing cancerous imaging characteristics was comparable to that of expert endoscopists and significantly higher than that of mid-level and junior endoscopists. This model was competent in localizing SESCC lesions. Manual diagnostic performances were significantly improved with the assistance by AI system, especially in terms of accuracy (75.12% vs. 84.95%, p = 0.008), specificity (63.29% vs. 76.59%, p = 0.017) and PPV (64.95% vs. 75.23%, p = 0.006).

CONCLUSIONS

The results of this study demonstrate that the developed AI system is highly effective in automatically recognizing SESCC, displaying impressive diagnostic performance, and exhibiting strong generalizability. Furthermore, when used as an assistant in the diagnosis process, the system improved manual diagnostic performance.

Transformer-based multi-task learning for classification and segmentation of gastrointestinal tract endoscopic images

Despite being widely utilized to help endoscopists identify gastrointestinal (GI) tract diseases using classification and segmentation, models based on convolutional neural network (CNN) have difficulties in distinguishing the similarities among some ambiguous types of lesions presented in endoscopic images, and in the training when lacking labeled datasets. Those will prevent CNN from further improving the accuracy of diagnosis. To address these challenges, we first proposed a Multi-task Network (TransMT-Net) capable of simultaneously learning two tasks (classification and segmentation), which has the transformer designed to learn global features and can combine the advantages of CNN in learning local features so that to achieve a more accurate prediction in identifying the lesion types and regions in GI tract endoscopic images. We further adopted the active learning in TransMT-Net to tackle the labeled image-hungry problem. A dataset was created from the CVC-ClinicDB dataset, Macau Kiang Wu Hospital, and Zhongshan Hospital to evaluate the model performance. Then, the experimental results show that our model not only achieved 96.94% accuracy in the classification task and 77.76% Dice Similarity Coefficient in the segmentation task but also outperformed those of other models on our test set. Meanwhile, active learning also produced positive results for the performance of our model with a small-scale initial training set, and even its performance with 30% of the initial training set was comparable to that of most comparable models with the full training set. Consequently, the proposed TransMT-Net has demonstrated its potential performance in GI tract endoscopic images and it through active learning can alleviate the shortage of labeled images.

CIABNet: Category imbalance attention block network for the classification of multi-differentiated types of esophageal cancer

BACKGROUND

Esophageal cancer has become one of the important cancers that seriously threaten human life and health, and its incidence and mortality rate are still among the top malignant tumors. Histopathological image analysis is the gold standard for diagnosing different differentiation types of esophageal cancer.

PURPOSE

The grading accuracy and interpretability of the auxiliary diagnostic model for esophageal cancer are seriously affected by small interclass differences, imbalanced data distribution, and poor model interpretability. Therefore, we focused on developing the category imbalance attention block network (CIABNet) model to try to solve the previous problems.

METHODS

First, the quantitative metrics and model visualization results are integrated to transfer knowledge from the source domain images to better identify the regions of interest (ROI) in the target domain of esophageal cancer. Second, in order to pay attention to the subtle interclass differences, we propose the concatenate fusion attention block, which can focus on the contextual local feature relationships and the changes of channel attention weights among different regions simultaneously. Third, we proposed a category imbalance attention module, which treats each esophageal cancer differentiation class fairly based on aggregating different intensity information at multiple scales and explores more representative regional features for each class, which effectively mitigates the negative impact of category imbalance. Finally, we use feature map visualization to focus on interpreting whether the ROIs are the same or similar between the model and pathologists, thus better improving the interpretability of the model.

RESULTS

The experimental results show that the CIABNet model outperforms other state-of-the-art models, which achieves the most advanced results in classifying the differentiation types of esophageal cancer with an average classification accuracy of 92.24%, an average precision of 93.52%, an average recall of 90.31%, an average F1 value of 91.73%, and an average AUC value of 97.43%. In addition, the CIABNet model has essentially similar or identical to the ROI of pathologists in identifying histopathological images of esophageal cancer.

CONCLUSIONS

Our experimental results prove that our proposed computer-aided diagnostic algorithm shows great potential in histopathological images of multi-differentiated types of esophageal cancer.

Development and Validation of Deep Learning Models for the Multiclassification of Reflux Esophagitis Based on the Los Angeles Classification

This study is to evaluate the feasibility of deep learning (DL) models in the multiclassification of reflux esophagitis (RE) endoscopic images, according to the Los Angeles (LA) classification for the first time. The images were divided into three groups, namely, normal, LA classification A + B, and LA C + D. The images from the HyperKvasir dataset and Suzhou hospital were divided into the training and validation datasets as a ratio of 4 : 1, while the images from Jintan hospital were the independent test set. The CNNs- or Transformer-architectures models (MobileNet, ResNet, Xception, EfficientNet, ViT, and ConvMixer) were transfer learning via Keras. The visualization of the models was proposed using Gradient-weighted Class Activation Mapping (Grad-CAM). Both in the validation set and the test set, the EfficientNet model showed the best performance as follows: accuracy (0.962 and 0.957), recall for LA A + B (0.970 and 0.925) and LA C + D (0.922 and 0.930), Marco-recall (0.946 and 0.928), Matthew's correlation coefficient (0.936 and 0.884), and Cohen's kappa (0.910 and 0.850), which was better than the other models and the endoscopists. According to the EfficientNet model, the Grad-CAM was plotted and highlighted the target lesions on the original images. This study developed a series of DL-based computer vision models with the interpretable Grad-CAM to evaluate the feasibility in the multiclassification of RE endoscopic images. It firstly suggests that DL-based classifiers show promise in the endoscopic diagnosis of esophagitis.

Global research trends of artificial intelligence applied in esophageal carcinoma: A bibliometric analysis (2000-2022) via CiteSpace and VOSviewer

Objective

Using visual bibliometric analysis, the application and development of artificial intelligence in clinical esophageal cancer are summarized, and the research progress, hotspots, and emerging trends of artificial intelligence are elucidated.

Methods

On April 7th, 2022, articles and reviews regarding the application of AI in esophageal cancer, published between 2000 and 2022 were chosen from the Web of Science Core Collection. To conduct co-authorship, co-citation, and co-occurrence analysis of countries, institutions, authors, references, and keywords in this field, VOSviewer (version 1.6.18), CiteSpace (version 5.8.R3), Microsoft Excel 2019, R 4.2, an online bibliometric platform (http://bibliometric.com/) and an online browser plugin (https://www.altmetric.com/) were used.

Results

A total of 918 papers were included, with 23,490 citations. 5,979 authors, 39,962 co-cited authors, and 42,992 co-cited papers were identified in the study. Most publications were from China (317). In terms of the H-index (45) and citations (9925), the United States topped the list. The journal “New England Journal of Medicine” of Medicine, General & Internal (IF = 91.25) published the most studies on this topic. The University of Amsterdam had the largest number of publications among all institutions. The past 22 years of research can be broadly divided into two periods. The 2000 to 2016 research period focused on the classification, identification and comparison of esophageal cancer. Recently (2017-2022), the application of artificial intelligence lies in endoscopy, diagnosis, and precision therapy, which have become the frontiers of this field. It is expected that closely esophageal cancer clinical measures based on big data analysis and related to precision will become the research hotspot in the future.

Conclusions

An increasing number of scholars are devoted to artificial intelligence-related esophageal cancer research. The research field of artificial intelligence in esophageal cancer has entered a new stage. In the future, there is a need to continue to strengthen cooperation between countries and institutions. Improving the diagnostic accuracy of esophageal imaging, big data-based treatment and prognosis prediction through deep learning technology will be the continuing focus of research. The application of AI in esophageal cancer still has many challenges to overcome before it can be utilized.