Gastrointestinal Disease Classification in Endoscopic Images Using Attention-Guided Convolutional Neural Networks

A two-phase transfer learning framework for gastrointestinal diseases classification

Gastrointestinal (GI) disorders are common and often debilitating health issues that affect a significant portion of the population. Recent advancements in artificial intelligence, particularly computer vision algorithms, have shown great potential in detecting and classifying medical images. These algorithms utilize deep convolutional neural network architectures to learn complex spatial features in images and make predictions for similar unseen images. The proposed study aims to assist gastroenterologists in making more efficient and accurate diagnoses of GI patients by utilizing its two-phase transfer learning framework to identify GI diseases from endoscopic images. Three pre-trained image classification models, namely Xception, InceptionResNetV2, and VGG16, are fine-tuned on publicly available datasets of annotated endoscopic images of the GI tract. Additionally, two custom convolutional neural networks are constructed and fully trained for comparative analysis of their performance. Four different classification tasks are examined based on the endoscopic image categories. The proposed architecture employing InceptionResNetV2 achieves the most consistent and generalized performance across most classification tasks, yielding accuracy scores of 85.7% for general classification of GI tract (eight-category classification), 97.6% for three-diseases classification, 99.5% for polyp identification (binary classification), and 74.2% for binary classification of esophagitis severity on unseen endoscopic images. The results indicate the effectiveness of the two-phase transfer learning framework for clinical use to enhance the identification of GI diseases, aiding in their early diagnosis and treatment.

Automated Detection of Gastrointestinal Diseases Using Resnet50*-Based Explainable Deep Feature Engineering Model with Endoscopy Images

This work aims to develop a novel convolutional neural network (CNN) named ResNet50* to detect various gastrointestinal diseases using a new ResNet50*-based deep feature engineering model with endoscopy images. The novelty of this work is the development of ResNet50*, a new variant of the ResNet model, featuring convolution-based residual blocks and a pooling-based attention mechanism similar to PoolFormer. Using ResNet50*, a gastrointestinal image dataset was trained, and an explainable deep feature engineering (DFE) model was developed. This DFE model comprises four primary stages: (i) feature extraction, (ii) iterative feature selection, (iii) classification using shallow classifiers, and (iv) information fusion. The DFE model is self-organizing, producing 14 different outcomes (8 classifier-specific and 6 voted) and selecting the most effective result as the final decision. During feature extraction, heatmaps are identified using gradient-weighted class activation mapping (Grad-CAM) with features derived from these regions via the final global average pooling layer of the pretrained ResNet50*. Four iterative feature selectors are employed in the feature selection stage to obtain distinct feature vectors. The classifiers k-nearest neighbors (kNN) and support vector machine (SVM) are used to produce specific outcomes. Iterative majority voting is employed in the final stage to obtain voted outcomes using the top result determined by the greedy algorithm based on classification accuracy. The presented ResNet50* was trained on an augmented version of the Kvasir dataset, and its performance was tested using Kvasir, Kvasir version 2, and wireless capsule endoscopy (WCE) curated colon disease image datasets. Our proposed ResNet50* model demonstrated a classification accuracy of more than 92% for all three datasets and a remarkable 99.13% accuracy for the WCE dataset. These findings affirm the superior classification ability of the ResNet50* model and confirm the generalizability of the developed architecture, showing consistent performance across all three distinct datasets.

Spatial-attention ConvMixer architecture for classification and detection of gastrointestinal diseases using the Kvasir dataset

Gastrointestinal (GI) disorders, encompassing conditions like cancer and Crohn's disease, pose a significant threat to public health. Endoscopic examinations have become crucial for diagnosing and treating these disorders efficiently. However, the subjective nature of manual evaluations by gastroenterologists can lead to potential errors in disease classification. In addition, the difficulty of diagnosing diseased tissues in GI and the high similarity between classes made the subject a difficult area. Automated classification systems that use artificial intelligence to solve these problems have gained traction. Automatic detection of diseases in medical images greatly benefits in the diagnosis of diseases and reduces the time of disease detection. In this study, we suggested a new architecture to enable research on computer-assisted diagnosis and automated disease detection in GI diseases. This architecture, called Spatial-Attention ConvMixer (SAC), further developed the patch extraction technique used as the basis of the ConvMixer architecture with a spatial attention mechanism (SAM). The SAM enables the network to concentrate selectively on the most informative areas, assigning importance to each spatial location within the feature maps. We employ the Kvasir dataset to assess the accuracy of classifying GI illnesses using the SAC architecture. We compare our architecture's results with Vanilla ViT, Swin Transformer, ConvMixer, MLPMixer, ResNet50, and SqueezeNet models. Our SAC method gets 93.37% accuracy, while the other architectures get respectively 79.52%, 74.52%, 92.48%, 63.04%, 87.44%, and 85.59%. The proposed spatial attention block improves the accuracy of the ConvMixer architecture on the Kvasir, outperforming the state-of-the-art methods with an accuracy rate of 93.37%.

Multi-feature fusion and dandelion optimizer based model for automatically diagnosing the gastrointestinal diseases

It is a known fact that gastrointestinal diseases are extremely common among the public. The most common of these diseases are gastritis, reflux, and dyspepsia. Since the symptoms of these diseases are similar, diagnosis can often be confused. Therefore, it is of great importance to make these diagnoses faster and more accurate by using computer-aided systems. Therefore, in this article, a new artificial intelligence-based hybrid method was developed to classify images with high accuracy of anatomical landmarks that cause gastrointestinal diseases, pathological findings and polyps removed during endoscopy, which usually cause cancer. In the proposed method, firstly trained InceptionV3 and MobileNetV2 architectures are used and feature extraction is performed with these two architectures. Then, the features obtained from InceptionV3 and MobileNetV2 architectures are merged. Thanks to this merging process, different features belonging to the same images were brought together. However, these features contain irrelevant and redundant features that may have a negative impact on classification performance. Therefore, Dandelion Optimizer (DO), one of the most recent metaheuristic optimization algorithms, was used as a feature selector to select the appropriate features to improve the classification performance and support vector machine (SVM) was used as a classifier. In the experimental study, the proposed method was also compared with different convolutional neural network (CNN) models and it was found that the proposed method achieved better results. The accuracy value obtained in the proposed model is 93.88%.

GIT-Net: An Ensemble Deep Learning-Based GI Tract Classification of Endoscopic Images

This paper presents an ensemble of pre-trained models for the accurate classification of endoscopic images associated with Gastrointestinal (GI) diseases and illnesses. In this paper, we propose a weighted average ensemble model called GIT-NET to classify GI-tract diseases. We evaluated the model on a KVASIR v2 dataset with eight classes. When individual models are used for classification, they are often prone to misclassification since they may not be able to learn the characteristics of all the classes adequately. This is due to the fact that each model may learn the characteristics of specific classes more efficiently than the other classes. We propose an ensemble model that leverages the predictions of three pre-trained models, DenseNet201, InceptionV3, and ResNet50 with accuracies of 94.54%, 88.38%, and 90.58%, respectively. The predictions of the base learners are combined using two methods: model averaging and weighted averaging. The performances of the models are evaluated, and the model averaging ensemble has an accuracy of 92.96% whereas the weighted average ensemble has an accuracy of 95.00%. The weighted average ensemble outperforms the model average ensemble and all individual models. The results from the evaluation demonstrate that utilizing an ensemble of base learners can successfully classify features that were incorrectly learned by individual base learners.

Hybrid Techniques for Diagnosing Endoscopy Images for Early Detection of Gastrointestinal Disease Based on Fusion Features

Gastrointestinal (GI) diseases, particularly tumours, are considered one of the most widespread and dangerous diseases and thus need timely health care for early detection to reduce deaths. Endoscopy technology is an effective technique for diagnosing GI diseases, thus producing a video containing thousands of frames. However, it is difficult to analyse all the images by a gastroenterologist, and it takes a long time to keep track of all the frames. Thus, artificial intelligence systems provide solutions to this challenge by analysing thousands of images with high speed and effective accuracy. Hence, systems with different methodologies are developed in this work. The first methodology for diagnosing endoscopy images of GI diseases is by using VGG-16 + SVM and DenseNet-121 + SVM. The second methodology for diagnosing endoscopy images of gastrointestinal diseases by artificial neural network (ANN) is based on fused features between VGG-16 and DenseNet-121 before and after high-dimensionality reduction by the principal component analysis (PCA). The third methodology is by ANN and is based on the fused features between VGG-16 and handcrafted features and features fused between DenseNet-121 and the handcrafted features. Herein, handcrafted features combine the features of gray level cooccurrence matrix (GLCM), discrete wavelet transform (DWT), fuzzy colour histogram (FCH), and local binary pattern (LBP) methods. All systems achieved promising results for diagnosing endoscopy images of the gastroenterology data set. The ANN network reached an accuracy, sensitivity, precision, specificity, and an AUC of 98.9%, 98.70%, 98.94%, 99.69%, and 99.51%, respectively, based on fused features of the VGG-16 and the handcrafted.

Semi-Supervised Segmentation Framework for Gastrointestinal Lesion Diagnosis in Endoscopic Images

BACKGROUND

Accurate gastrointestinal (GI) lesion segmentation is crucial in diagnosing digestive tract diseases. An automatic lesion segmentation in endoscopic images is vital to relieving physicians' burden and improving the survival rate of patients. However, pixel-wise annotations are highly intensive, especially in clinical settings, while numerous unlabeled image datasets could be available, although the significant results of deep learning approaches in several tasks heavily depend on large labeled datasets. Limited labeled data also hinder trained models' generalizability under fully supervised learning for computer-aided diagnosis (CAD) systems.

METHODS

This work proposes a generative adversarial learning-based semi-supervised segmentation framework for GI lesion diagnosis in endoscopic images to tackle the challenge of limited annotations. The proposed approach leverages limited annotated and large unlabeled datasets in the training networks. We extensively tested the proposed method on 4880 endoscopic images.

RESULTS

Compared with current related works, the proposed method validates better results (Dice similarity coefficient = 89.42 ± 3.92, Intersection over union = 80.04 ± 5.75, Precision = 91.72 ± 4.05, Recall = 90.11 ± 5.64, and Hausdorff distance = 23.28 ± 14.36) on the challenging multi-sited datasets, confirming the effectiveness of the proposed framework.

CONCLUSION

We explore a semi-supervised lesion segmentation method to employ the full use of multiple unlabeled endoscopic images to improve lesion segmentation accuracy. Experimental results confirmed the potential of our method and outperformed promising results compared with the current related works. The proposed CAD system can minimize diagnostic errors.

Comparative study of convolutional neural network architectures for gastrointestinal lesions classification

The gastrointestinal (GI) tract can be affected by different diseases or lesions such as esophagitis, ulcers, hemorrhoids, and polyps, among others. Some of them can be precursors of cancer such as polyps. Endoscopy is the standard procedure for the detection of these lesions. The main drawback of this procedure is that the diagnosis depends on the expertise of the doctor. This means that some important findings may be missed. In recent years, this problem has been addressed by deep learning (DL) techniques. Endoscopic studies use digital images. The most widely used DL technique for image processing is the convolutional neural network (CNN) due to its high accuracy for modeling complex phenomena. There are different CNNs that are characterized by their architecture. In this article, four architectures are compared: AlexNet, DenseNet-201, Inception-v3, and ResNet-101. To determine which architecture best classifies GI tract lesions, a set of metrics; accuracy, precision, sensitivity, specificity, F1-score, and area under the curve (AUC) were used. These architectures were trained and tested on the HyperKvasir dataset. From this dataset, a total of 6,792 images corresponding to 10 findings were used. A transfer learning approach and a data augmentation technique were applied. The best performing architecture was DenseNet-201, whose results were: 97.11% of accuracy, 96.3% sensitivity, 99.67% specificity, and 95% AUC.

Time-Series Clustering of Single-Cell Trajectories in Collective Cell Migration

Simple Summary

In this study, we normalized trajectories containing both mesenchymal and epithelial cells to remove the effect of cell location on clustering, and performed a dimensionality reduction on the time series data before clustering. When the clustering results were superimposed on the trajectories prior to normalization, the results still showed similarities in location, indicating that this method can find cells with similar migration patterns. These data highlight the reliability of this method in identifying consistent migration patterns in collective cell migration.

Abstract

Collective invasion drives multicellular cancer cells to spread to surrounding normal tissues. To fully comprehend metastasis, the methodology of analysis of individual cell migration in tissue should be well developed. Extracting and classifying cells with similar migratory characteristics in a colony would facilitate an understanding of complex cell migration patterns. Here, we used electrospun fibers as the extracellular matrix for the in vitro modeling of collective cell migration, clustering of mesenchymal and epithelial cells based on trajectories, and analysis of collective migration patterns based on trajectory similarity. We normalized the trajectories to eliminate the effect of cell location on clustering and used uniform manifold approximation and projection to perform dimensionality reduction on the time-series data before clustering. When the clustering results were superimposed on the trajectories before normalization, the results still exhibited positional similarity, thereby demonstrating that this method can identify cells with similar migration patterns. The same cluster contained both mesenchymal and epithelial cells, and this result was related to cell location and cell division. These data highlight the reliability of this method in identifying consistent migration patterns during collective cell migration. This provides new insights into the epithelial–mesenchymal interactions that affect migration patterns.

Deep Feature Fusion and Optimization-Based Approach for Stomach Disease Classification

Cancer is the deadliest disease among all the diseases and the main cause of human mortality. Several types of cancer sicken the human body and affect organs. Among all the types of cancer, stomach cancer is the most dangerous disease that spreads rapidly and needs to be diagnosed at an early stage. The early diagnosis of stomach cancer is essential to reduce the mortality rate. The manual diagnosis process is time-consuming, requires many tests, and the availability of an expert doctor. Therefore, automated techniques are required to diagnose stomach infections from endoscopic images. Many computerized techniques have been introduced in the literature but due to a few challenges (i.e., high similarity among the healthy and infected regions, irrelevant features extraction, and so on), there is much room to improve the accuracy and reduce the computational time. In this paper, a deep-learning-based stomach disease classification method employing deep feature extraction, fusion, and optimization using WCE images is proposed. The proposed method comprises several phases: data augmentation performed to increase the dataset images, deep transfer learning adopted for deep features extraction, feature fusion performed on deep extracted features, fused feature matrix optimized with a modified dragonfly optimization method, and final classification of the stomach disease was performed. The features extraction phase employed two pre-trained deep CNN models (Inception v3 and DenseNet-201) performing activation on feature derivation layers. Later, the parallel concatenation was performed on deep-derived features and optimized using the meta-heuristic method named the dragonfly algorithm. The optimized feature matrix was classified by employing machine-learning algorithms and achieved an accuracy of 99.8% on the combined stomach disease dataset. A comparison has been conducted with state-of-the-art techniques and shows improved accuracy.