Development of a multi-fusion convolutional neural network (MF-CNN) for enhanced gastrointestinal disease diagnosis in endoscopy image analysis

Gastrointestinal (GI) diseases are prevalent medical conditions that require accurate and timely diagnosis for effective treatment. To address this, we developed the Multi-Fusion Convolutional Neural Network (MF-CNN), a deep learning framework that strategically integrates and adapts elements from six deep learning models, enhancing feature extraction and classification of GI diseases from endoscopic images. The MF-CNN architecture leverages truncated and partially frozen layers from existing models, augmented with novel components such as Auxiliary Fusing Layers (AuxFL), Fusion Residual Block (FuRB), and Alpha Dropouts (αDO) to improve precision and robustness. This design facilitates the precise identification of conditions such as ulcerative colitis, polyps, esophagitis, and healthy colons. Our methodology involved preprocessing endoscopic images sourced from open databases, including KVASIR and ETIS-Larib Polyp DB, using adaptive histogram equalization (AHE) to enhance their quality. The MF-CNN framework supports detailed feature mapping for improved interpretability of the model's internal workings. An ablation study was conducted to validate the contribution of each component, demonstrating that the integration of AuxFL, αDO, and FuRB played a crucial part in reducing overfitting and efficiency saturation and enhancing overall model performance. The MF-CNN demonstrated outstanding performance in terms of efficacy, achieving an accuracy rate of 99.25%. It also excelled in other key performance metrics with a precision of 99.27%, a recall of 99.25%, and an F1-score of 99.25%. These metrics confirmed the model's proficiency in accurate classification and its capability to minimize false positives and negatives across all tested GI disease categories. Furthermore, the AUC values were exceptional, averaging 1.00 for both test and validation sets, indicating perfect discriminative ability. The findings of the P-R curve analysis and confusion matrix further confirmed the robust classification performance of the MF-CNN. This research introduces a technique for medical imaging that can potentially transform diagnostics in gastrointestinal healthcare facilities worldwide.

Advancements in Barrett's esophagus detection: The role of artificial intelligence and its implications

Artificial intelligence (AI) is making significant strides in revolutionizing the detection of Barrett's esophagus (BE), a precursor to esophageal adenocarcinoma. In the research article by Tsai et al, researchers utilized endoscopic images to train an AI model, challenging the traditional distinction between endoscopic and histological BE. This approach yielded remarkable results, with the AI system achieving an accuracy of 94.37%, sensitivity of 94.29%, and specificity of 94.44%. The study's extensive dataset enhances the AI model's practicality, offering valuable support to endoscopists by minimizing unnecessary biopsies. However, questions about the applicability to different endoscopic systems remain. The study underscores the potential of AI in BE detection while highlighting the need for further research to assess its adaptability to diverse clinical settings.

Multi-feature fusion and dandelion optimizer based model for automatically diagnosing the gastrointestinal diseases

It is a known fact that gastrointestinal diseases are extremely common among the public. The most common of these diseases are gastritis, reflux, and dyspepsia. Since the symptoms of these diseases are similar, diagnosis can often be confused. Therefore, it is of great importance to make these diagnoses faster and more accurate by using computer-aided systems. Therefore, in this article, a new artificial intelligence-based hybrid method was developed to classify images with high accuracy of anatomical landmarks that cause gastrointestinal diseases, pathological findings and polyps removed during endoscopy, which usually cause cancer. In the proposed method, firstly trained InceptionV3 and MobileNetV2 architectures are used and feature extraction is performed with these two architectures. Then, the features obtained from InceptionV3 and MobileNetV2 architectures are merged. Thanks to this merging process, different features belonging to the same images were brought together. However, these features contain irrelevant and redundant features that may have a negative impact on classification performance. Therefore, Dandelion Optimizer (DO), one of the most recent metaheuristic optimization algorithms, was used as a feature selector to select the appropriate features to improve the classification performance and support vector machine (SVM) was used as a classifier. In the experimental study, the proposed method was also compared with different convolutional neural network (CNN) models and it was found that the proposed method achieved better results. The accuracy value obtained in the proposed model is 93.88%.

Identification of gastric signet ring cell carcinoma based on endoscopic images using few-shot learning

BACKGROUND

Deep learning (DL) models perform poorly when there are limited gastric signet ring cell carcinoma (SRCC) samples. Few-shot learning (FSL) can address the small sample problem.

METHODS

EfficientNetV2-S was first pretrained on ImageNet and then pretrained on esophageal endoscopic images (i.e., base classes: normal vs. early cancer vs. advanced cancer) using transfer learning. Second, images of gastric benign ulcers, adenocarcinoma and SRCC, i.e., novel classes (n = 50 per class), were included. Image features were extracted as vectors using the dual pretrained EfficientNetV2-S. Finally, a k-nearest neighbor classifier was used to identify SRCC. The above proposed 3-way 3-shot FSL framework was conducted in three rounds.

RESULTS

Dual pretrained FSL performed better than single pretrained FSL, endoscopists and traditional EfficientNetV2-S models. Dual pretrained FSL obtained the highest accuracy (79.4%), sensitivity (68.8%), recall (68.8%), precision (69.3%) and F1-score (0.691), and the senior endoscopist achieved the highest specificity of 93.6% when identifying SRCC. The macro-AUC and F1-score for dual pretraining (0.763 and 0.703, respectively) were higher than those for single pretraining (0.656 and 0.537, respectively), along with endoscopists and traditional EfficientNetV2-S models. The 2-way 3-shot FSL also performed better.

CONCLUSION

The proposed FSL framework showed practical performance in the differentiation of SRCC on endoscopic images, suggesting the potential of FSL in the computer-aided diagnosis for rare diseases.

DLGNet: A dual-branch lesion-aware network with the supervised Gaussian Mixture model for colon lesions classification in colonoscopy images

Colorectal cancer is one of the malignant tumors with the highest mortality due to the lack of obvious early symptoms. It is usually in the advanced stage when it is discovered. Thus the automatic and accurate classification of early colon lesions is of great significance for clinically estimating the status of colon lesions and formulating appropriate diagnostic programs. However, it is challenging to classify full-stage colon lesions due to the large inter-class similarities and intra-class differences of the images. In this work, we propose a novel dual-branch lesion-aware neural network (DLGNet) to classify intestinal lesions by exploring the intrinsic relationship between diseases, composed of four modules: lesion location module, dual-branch classification module, attention guidance module, and inter-class Gaussian loss function. Specifically, the elaborate dual-branch module integrates the original image and the lesion patch obtained by the lesion localization module to explore and interact with lesion-specific features from a global and local perspective. Also, the feature-guided module guides the model to pay attention to the disease-specific features by learning remote dependencies through spatial and channel attention after network feature learning. Finally, the inter-class Gaussian loss function is proposed, which assumes that each feature extracted by the network is an independent Gaussian distribution, and the inter-class clustering is more compact, thereby improving the discriminative ability of the network. The extensive experiments on the collected 2568 colonoscopy images have an average accuracy of 91.50%, and the proposed method surpasses the state-of-the-art methods. This study is the first time that colon lesions are classified at each stage and achieves promising colon disease classification performance. To motivate the community, we have made our code publicly available via https://github.com/soleilssss/DLGNet.

Endoscopic image feature matching via motion consensus and global bilateral regression

BACKGROUND AND OBJECTIVE

Feature matching of endoscopic images is of crucial importance in many clinical applications, such as object tracking and surface reconstruction. However, with the presence of low texture, specular reflections and deformations, the feature matching methods of natural scene are facing great challenges in minimally invasive surgery (MIS) scenarios. We propose a novel motion consensus-based method for endoscopic image feature matching to address these problems.

METHODS

Our method starts by correcting the radial distortion with the spherical projection model and removing the specular reflection regions with an adaptive detection method, which helps to eliminate the image distortion and to reduce the quantity of outliers. We solve the matching problem with a two-stage strategy that progressively estimates a consensus of inliers; the result is a precisely smoothed motion field. First, we construct a spatial motion field from candidate feature matches and estimate its maximum posterior with expectation maximization algorithm, which is computationally efficient and able to obtain smoothed motion field quickly. Second, we extend the smoothed motion field to the affine domain and refine it with bilateral regression to preserve locally subtle motions. The true matches can be identified by checking the difference of feature motion against the estimated field.

RESULTS

Evaluations are implemented on two simulation datasets of deformation (218 images) and four different types of endoscopic datasets (1032 images). Our method is compared with three other state-of-the-art methods and achieves the best performance on affine transformation and nonrigid deformation simulations, with inlier ratio of 86.7% and 94.3%, sensitivity of 90.0% and 96.2%, precision of 88.2% and 93.9%, and F1-Score of 89.1% and 95.0%, respectively. On clinical datasets evaluations, the proposed method achieves an average reprojection error of 3.7 pixels and a consistent performance in multi-image correspondence of sequential images. Furthermore, we also present a surface reconstruction result from rhinoscopic images to validate the reliability of our method, which shows high-quality feature matching results.

CONCLUSIONS

The proposed motion consensus-based feature matching method is proved effective and robust for endoscopic images correspondence. This demonstrates its capability to generate reliable feature matches for surface reconstruction and other meaningful applications in MIS scenarios.

ReTouchImg: Fusioning from-local-to-global context detection and graph data structures for fully-automatic specular reflection removal for endoscopic images

Minimally invasive surgical and diagnostic systems are commonly used in clinical practices. However, the accuracy and robustness of these systems depend heavily on computer based processes such as tracking, detecting or segmenting clinically meaningful regions of interest, which are significantly affected by the inherent specular reflections that appear on the organs' surfaces. Restoration of the acquired data for clinical purposes still presents challenges because of the high texture and color variations across the image. In this work, we propose a novel fully-automated solution for endoscopic image restoration, which we call ReTouchImg. Our approach is designed as a two-step scheme. The first is a detection step that is based on the synergy of a set of color variations and gradient information conditions. For the second step, we introduce an inpainting process which is based on graph data structures for recovering the missing information. We exhaustively evaluate our approach on real endoscopic datasets and compare it against some works from the body of literature. We also demonstrate that our solution deals with complex cases such as strong illumination variation and large affected areas through a careful quantitative evaluation of a range of numerical results.

Color reproduction and processing algorithm based on real-time mapping for endoscopic images

In this paper, we present a real-time preprocessing algorithm for image enhancement for endoscopic images. A novel dictionary based color mapping algorithm is used for reproducing the color information from a theme image. The theme image is selected from a nearby anatomical location. A database of color endoscopy image for different location is prepared for this purpose. The color map is dynamic as its contents change with the change of the theme image. This method is used on low contrast grayscale white light images and raw narrow band images to highlight the vascular and mucosa structures and to colorize the images. It can also be applied to enhance the tone of color images. The statistic visual representation and universal image quality measures show that the proposed method can highlight the mucosa structure compared to other methods. The color similarity has been verified using Delta E color difference, structure similarity index, mean structure similarity index and structure and hue similarity. The color enhancement was measured using color enhancement factor that shows considerable improvements. The proposed algorithm has low and linear time complexity, which results in higher execution speed than other related works.