Detecting and Locating Gastrointestinal Anomalies Using Deep Learning and Iterative Cluster Unification

A novel lightweight deep learning based approaches for the automatic diagnosis of gastrointestinal disease using image processing and knowledge distillation techniques

BACKGROUND

Gastrointestinal (GI) diseases pose significant challenges for healthcare systems, largely due to the complexities involved in their detection and treatment. Despite the advancements in deep neural networks, their high computational demands hinder their practical use in clinical environments.

OBJECTIVE

This study aims to address the computational inefficiencies of deep neural networks by proposing a lightweight model that integrates model compression techniques, ConvLSTM layers, and ConvNext Blocks, all optimized through Knowledge Distillation (KD).

METHODS

A dataset of 6000 endoscopic images of various GI diseases was utilized. Advanced image preprocessing techniques, including adaptive noise reduction and image detail enhancement, were employed to improve accuracy and interpretability. The model's performance was assessed in terms of accuracy, computational cost, and disk space usage.

RESULTS

The proposed lightweight model achieved an exceptional overall accuracy of 99.38 %. It operates efficiently with a computational cost of 0.61 GFLOPs and occupies only 3.09 MB of disk space. Additionally, Grad-CAM visualizations demonstrated enhanced model saliency and interpretability, offering insights into the decision-making process of the model post-KD.

CONCLUSION

The proposed model represents a significant advancement in the diagnosis of GI diseases. It provides a cost-effective and efficient alternative to traditional deep neural network methods, overcoming their computational limitations and contributing valuable insights for improved clinical application.

Reducing reading time and assessing disease in capsule endoscopy videos: A deep learning approach

BACKGROUND

The wireless capsule endoscope (CE) is a valuable diagnostic tool in gastroenterology, offering a safe and minimally invasive visualization of the gastrointestinal tract. One of the few drawbacks identified by the gastroenterology community is the time-consuming task of analyzing CE videos.

OBJECTIVES

This article investigates the feasibility of a computer-aided diagnostic method to speed up CE video analysis. We aim to generate a significantly smaller CE video with all the anomalies (i.e., diseases) identified by the medical doctors in the original video.

METHODS

The summarized video consists of the original video frames classified as anomalous by a pre-trained convolutional neural network (CNN). We evaluate our approach on a testing dataset with eight CE videos captured with five CE types and displaying multiple anomalies.

RESULTS

On average, the summarized videos contain 93.33% of the anomalies identified in the original videos. The average playback time of the summarized videos is just 10 min, compared to 58 min for the original videos.

CONCLUSION

Our findings demonstrate the potential of deep learning-aided diagnostic methods to accelerate CE video analysis.

Interpretable deep learning architecture for gastrointestinal disease detection: A Tri-stage approach with PCA and XAI

GI abnormalities significantly increase mortality rates and impose considerable strain on healthcare systems, underscoring the essential requirement for rapid detection, precise diagnosis, and efficient strategic treatment. To develop a CAD system, this study aims to automatically classify GI disorders utilizing various deep learning methodologies. The proposed system features a three-stage lightweight architecture, consisting of a feature extractor using PSE-CNN, a feature selector employing PCA, and a classifier based on DELM. The framework, designed with only 24 layers and 1.25 million parameters, is employed on the largest dataset, GastroVision, containing 8000 images of 27 GI disorders. To improve visual clarity, a sequential preprocessing strategy is implemented. The model's robustness is evaluated through 5-fold cross-validation. Additionally, several XAI methods, namely Grad-CAM, heatmaps, saliency maps, SHAP, and activation feature maps, are used to explore the model's interpretability. Statistical significance is ensured by calculating the p-value, demonstrating the framework's reliability. The proposed model PSE-CNN-PCA-DELM has achieved outstanding results in the first stage, categorizing the diseases' positions into three primary classes, with average accuracy (97.24 %), precision (97.33 ± 0.01 %), recall (97.24 ± 0.01 %), F1-score (97.33 ± 0.01 %), ROC-AUC (99.38 %), and AUC-PR (98.94 %). In the second stage, the dataset is further divided into nine separate classes, considering the overall disease characteristics, and achieves excellent outcomes with average performance rates of 90.00 %, 89.71 ± 0.11 %, 89.59 ± 0.14 %, 89.51 ± 0.12 %, 98.49 %, and 94.63 %, respectively. The third stage involves a more detailed classification into twenty-seven classes, maintaining strong performance with scores of 93.00 %, 82.69 ± 0.37 %, 83.00 ± 0.38 %, 81.54 ± 0.35 %, 97.38 %, and 88.03 %, respectively. The framework's compact size of 14.88 megabytes and average testing time of 59.17 milliseconds make it highly efficient. Its effectiveness is further validated through comparisons with several TL approaches. Practically, the framework is extremely resilient for clinical implementation.

Computational pathology: A survey review and the way forward

Computational Pathology (CPath) is an interdisciplinary science that augments developments of computational approaches to analyze and model medical histopathology images. The main objective for CPath is to develop infrastructure and workflows of digital diagnostics as an assistive CAD system for clinical pathology, facilitating transformational changes in the diagnosis and treatment of cancer that are mainly address by CPath tools. With evergrowing developments in deep learning and computer vision algorithms, and the ease of the data flow from digital pathology, currently CPath is witnessing a paradigm shift. Despite the sheer volume of engineering and scientific works being introduced for cancer image analysis, there is still a considerable gap of adopting and integrating these algorithms in clinical practice. This raises a significant question regarding the direction and trends that are undertaken in CPath. In this article we provide a comprehensive review of more than 800 papers to address the challenges faced in problem design all-the-way to the application and implementation viewpoints. We have catalogued each paper into a model-card by examining the key works and challenges faced to layout the current landscape in CPath. We hope this helps the community to locate relevant works and facilitate understanding of the field's future directions. In a nutshell, we oversee the CPath developments in cycle of stages which are required to be cohesively linked together to address the challenges associated with such multidisciplinary science. We overview this cycle from different perspectives of data-centric, model-centric, and application-centric problems. We finally sketch remaining challenges and provide directions for future technical developments and clinical integration of CPath. For updated information on this survey review paper and accessing to the original model cards repository, please refer to GitHub. Updated version of this draft can also be found from arXiv.

A Comprehensive Review of Artificial Intelligence and Colon Capsule Endoscopy: Opportunities and Challenges

Colon capsule endoscopy (CCE) enables a comprehensive, non-invasive, and painless evaluation of the colon, although it still has limited indications. The lengthy reading times hinder its wider implementation, a drawback that could potentially be overcome through the integration of artificial intelligence (AI) models. Studies employing AI, particularly convolutional neural networks (CNNs), demonstrate great promise in using CCE as a viable option for detecting certain diseases and alterations in the colon, compared to other methods like colonoscopy. Additionally, employing AI models in CCE could pave the way for a minimally invasive panenteric or even panendoscopic solution. This review aims to provide a comprehensive summary of the current state-of-the-art of AI in CCE while also addressing the challenges, both technical and ethical, associated with broadening indications for AI-powered CCE. Additionally, it also gives a brief reflection of the potential environmental advantages of using this method compared to alternative ones.

Morphological Rule-Constrained Object Detection of Key Structures in Infant Fundus Image

The detection of optic disc and macula is an essential step for ROP (Retinopathy of prematurity) zone segmentation and disease diagnosis. This paper aims to enhance deep learning-based object detection with domain-specific morphological rules. Based on the fundus morphology, we define five morphological rules, i.e., number restriction (maximum number of optic disc and macula is one), size restriction (e.g., optic disc width: 1.05 +/- 0.13 mm), distance restriction (distance between the optic disc and macula/fovea: 4.4 +/- 0.4 mm), angle/slope restriction (optic disc and macula should roughly be positioned in the same horizontal line), position restriction (In OD, the macula is on the left side of the optic disc; vice versa for OS). A case study on 2953 infant fundus images (with 2935 optic disc instances and 2892 macula instances) proves the effectiveness of the proposed method. Without the morphological rules, naïve object detection accuracies of optic disc and macula are 0.955 and 0.719, respectively. With the proposed method, false-positive ROIs (region of interest) are further ruled out, and the accuracy of the macula is raised to 0.811. The IoU (intersection over union) and RCE (relative center error) metrics are also improved .

Advancements in microneedle fabrication techniques: artificial intelligence assisted 3D-printing technology

Microneedles (MNs) are micron-scale needles that are a painless alternative to injections for delivering drugs through the skin. MNs find applications as biosensing devices and could serve as real-time diagnosis tools. There have been numerous fabrication techniques employed for producing quality MN-based systems, prominent among them is the three-dimensional (3D) printing. 3D printing enables the production of quality MNs of tuneable characteristics using a variety of materials. Further, the possible integration of artificial intelligence (AI) tools such as machine learning (ML) and deep learning (DL) with 3D printing makes it an indispensable tool for fabricating microneedles. Provided that these AI tools can be trained and act with minimal human intervention to control the quality of products produced, there is also a possibility of mass production of MNs using these tools in the future. This work reviews the specific role of AI in the 3D printing of MN-based devices discussing the use of AI in predicting drug release patterns, its role as a quality control tool, and in predicting the biomarker levels. Additionally, the autonomous 3D printing of microneedles using an integrated system of the internet of things (IoT) and machine learning (ML) is discussed in brief. Different categories of machine learning including supervised learning, semi-supervised learning, unsupervised learning, and reinforced learning have been discussed in brief. Lastly, a brief section is dedicated to the biosensing applications of MN-based devices.

Explainable AI-driven model for gastrointestinal cancer classification

Although the detection procedure has been shown to be highly effective, there are several obstacles to overcome in the usage of AI-assisted cancer cell detection in clinical settings. These issues stem mostly from the failure to identify the underlying processes. Because AI-assisted diagnosis does not offer a clear decision-making process, doctors are dubious about it. In this instance, the advent of Explainable Artificial Intelligence (XAI), which offers explanations for prediction models, solves the AI black box issue. The SHapley Additive exPlanations (SHAP) approach, which results in the interpretation of model predictions, is the main emphasis of this work. The intermediate layer in this study was a hybrid model made up of three Convolutional Neural Networks (CNNs) (InceptionV3, InceptionResNetV2, and VGG16) that combined their predictions. The KvasirV2 dataset, which comprises pathological symptoms associated to cancer, was used to train the model. Our combined model yielded an accuracy of 93.17% and an F1 score of 97%. After training the combined model, we use SHAP to analyze images from these three groups to provide an explanation of the decision that affects the model prediction.

Development and validation of a deep learning system for detection of small bowel pathologies in capsule endoscopy: a pilot study in a Singapore institution

INTRODUCTION

Deep learning models can assess the quality of images and discriminate among abnormalities in small bowel capsule endoscopy (CE), reducing fatigue and the time needed for diagnosis. They serve as a decision support system, partially automating the diagnosis process by providing probability predictions for abnormalities.

METHODS

We demonstrated the use of deep learning models in CE image analysis, specifically by piloting a bowel preparation model (BPM) and an abnormality detection model (ADM) to determine frame-level view quality and the presence of abnormal findings, respectively. We used convolutional neural network-based models pretrained on large-scale open-domain data to extract spatial features of CE images that were then used in a dense feed-forward neural network classifier. We then combined the open-source Kvasir-Capsule dataset (n = 43) and locally collected CE data (n = 29).

RESULTS

Model performance was compared using averaged five-fold and two-fold cross-validation for BPMs and ADMs, respectively. The best BPM model based on a pre-trained ResNet50 architecture had an area under the receiver operating characteristic and precision-recall curves of 0.969±0.008 and 0.843±0.041, respectively. The best ADM model, also based on ResNet50, had top-1 and top-2 accuracies of 84.03±0.051 and 94.78±0.028, respectively. The models could process approximately 200-250 images per second and showed good discrimination on time-critical abnormalities such as bleeding.

CONCLUSION

Our pilot models showed the potential to improve time to diagnosis in CE workflows. To our knowledge, our approach is unique to the Singapore context. The value of our work can be further evaluated in a pragmatic manner that is sensitive to existing clinician workflow and resource constraints.

Comparison of clinical utility of deep learning-based systems for small-bowel capsule endoscopy reading

BACKGROUND AND AIM

Convolutional neural network (CNN) systems that automatically detect abnormalities from small-bowel capsule endoscopy (SBCE) images are still experimental, and no studies have directly compared the clinical usefulness of different systems. We compared endoscopist readings using an existing and a novel CNN system in a real-world SBCE setting.

METHODS

Thirty-six complete SBCE videos, including 43 abnormal lesions (18 mucosal breaks, 8 angioectasia, and 17 protruding lesions), were retrospectively prepared. Three reading processes were compared: (A) endoscopist readings without CNN screening, (B) endoscopist readings after an existing CNN screening, and (C) endoscopist readings after a novel CNN screening.

RESULTS

The mean number of small-bowel images was 14 747 per patient. Among these images, existing and novel CNN systems automatically captured 24.3% and 9.4% of the images, respectively. In this process, both systems extracted all 43 abnormal lesions. Next, we focused on the clinical usefulness. The detection rates of abnormalities by trainee endoscopists were not significantly different across the three processes: A, 77%; B, 67%; and C, 79%. The mean reading time of the trainees was the shortest during process C (10.1 min per patient), followed by processes B (23.1 min per patient) and A (33.6 min per patient). The mean psychological stress score while reading videos (scale, 1-5) was the lowest in process C (1.8) but was not significantly different between processes B (2.8) and A (3.2).

CONCLUSIONS

Our novel CNN system significantly reduced endoscopist reading time and psychological stress while maintaining the detectability of abnormalities. CNN performance directly affects clinical utility and should be carefully assessed.

Wireless capsule endoscopy multiclass classification using three-dimensional deep convolutional neural network model

BACKGROUND

Wireless capsule endoscopy (WCE) is a patient-friendly and non-invasive technology that scans the whole of the gastrointestinal tract, including difficult-to-access regions like the small bowel. Major drawback of this technology is that the visual inspection of a large number of video frames produced during each examination makes the physician diagnosis process tedious and prone to error. Several computer-aided diagnosis (CAD) systems, such as deep network models, have been developed for the automatic recognition of abnormalities in WCE frames. Nevertheless, most of these studies have only focused on spatial information within individual WCE frames, missing the crucial temporal data within consecutive frames.

METHODS

In this article, an automatic multiclass classification system based on a three-dimensional deep convolutional neural network (3D-CNN) is proposed, which utilizes the spatiotemporal information to facilitate the WCE diagnosis process. The 3D-CNN model fed with a series of sequential WCE frames in contrast to the two-dimensional (2D) model, which exploits frames as independent ones. Moreover, the proposed 3D deep model is compared with some pre-trained networks. The proposed models are trained and evaluated with 29 subject WCE videos (14,691 frames before augmentation). The performance advantages of 3D-CNN over 2D-CNN and pre-trained networks are verified in terms of sensitivity, specificity, and accuracy.

RESULTS

3D-CNN outperforms the 2D technique in all evaluation metrics (sensitivity: 98.92 vs. 98.05, specificity: 99.50 vs. 86.94, accuracy: 99.20 vs. 92.60). In conclusion, a novel 3D-CNN model for lesion detection in WCE frames is proposed in this study.

CONCLUSION

The results indicate the performance of 3D-CNN over 2D-CNN and some well-known pre-trained classifier networks. The proposed 3D-CNN model uses the rich temporal information in adjacent frames as well as spatial data to develop an accurate and efficient model.

CoInNet: A Convolution-Involution Network With a Novel Statistical Attention for Automatic Polyp Segmentation

Polyps are very common abnormalities in human gastrointestinal regions. Their early diagnosis may help in reducing the risk of colorectal cancer. Vision-based computer-aided diagnostic systems automatically identify polyp regions to assist surgeons in their removal. Due to their varying shape, color, size, texture, and unclear boundaries, polyp segmentation in images is a challenging problem. Existing deep learning segmentation models mostly rely on convolutional neural networks that have certain limitations in learning the diversity in visual patterns at different spatial locations. Further, they fail to capture inter-feature dependencies. Vision transformer models have also been deployed for polyp segmentation due to their powerful global feature extraction capabilities. But they too are supplemented by convolution layers for learning contextual local information. In the present paper, a polyp segmentation model CoInNet is proposed with a novel feature extraction mechanism that leverages the strengths of convolution and involution operations and learns to highlight polyp regions in images by considering the relationship between different feature maps through a statistical feature attention unit. To further aid the network in learning polyp boundaries, an anomaly boundary approximation module is introduced that uses recursively fed feature fusion to refine segmentation results. It is indeed remarkable that even tiny-sized polyps with only 0.01% of an image area can be precisely segmented by CoInNet. It is crucial for clinical applications, as small polyps can be easily overlooked even in the manual examination due to the voluminous size of wireless capsule endoscopy videos. CoInNet outperforms thirteen state-of-the-art methods on five benchmark polyp segmentation datasets.

A method of water resources accounting based on deep clustering and attention mechanism under the background of integration of public health data and environmental economy

Water resource accounting constitutes a fundamental approach for implementing sophisticated management of basin water resources. The quality of water plays a pivotal role in determining the liabilities associated with these resources. Evaluating the quality of water facilitates the computation of water resource liabilities during the accounting process. Traditional accounting methods rely on manual sorting and data analysis, which necessitate significant human effort. In order to address this issue, we leverage the remarkable feature extraction capabilities of convolutional operations to construct neural networks. Moreover, we introduce the self-attention mechanism module to propose an unsupervised deep clustering method. This method offers assistance in accounting tasks by automatically classifying the debt levels of water resources in distinct regions, thereby facilitating comprehensive water resource accounting. The methodology presented in this article underwent verification using three datasets: the United States Postal Service (USPS), Heterogeneity Human Activity Recognition (HHAR), and Association for Computing Machinery (ACM). The evaluation of Accuracy rate (ACC), Normalized Mutual Information (NMI), and Adjusted Rand Index (ARI) metrics yielded favorable results, surpassing those of K-means clustering, hierarchical clustering, and Density-based constraint extension (DCE). Specifically, the mean values of the evaluation metrics across the three datasets were 0.8474, 0.7582, and 0.7295, respectively.

E pluribus unum interpretable convolutional neural networks

The adoption of convolutional neural network (CNN) models in high-stake domains is hindered by their inability to meet society's demand for transparency in decision-making. So far, a growing number of methodologies have emerged for developing CNN models that are interpretable by design. However, such models are not capable of providing interpretations in accordance with human perception, while maintaining competent performance. In this paper, we tackle these challenges with a novel, general framework for instantiating inherently interpretable CNN models, named E pluribus unum interpretable CNN (EPU-CNN). An EPU-CNN model consists of CNN sub-networks, each of which receives a different representation of an input image expressing a perceptual feature, such as color or texture. The output of an EPU-CNN model consists of the classification prediction and its interpretation, in terms of relative contributions of perceptual features in different regions of the input image. EPU-CNN models have been extensively evaluated on various publicly available datasets, as well as a contributed benchmark dataset. Medical datasets are used to demonstrate the applicability of EPU-CNN for risk-sensitive decisions in medicine. The experimental results indicate that EPU-CNN models can achieve a comparable or better classification performance than other CNN architectures while providing humanly perceivable interpretations.

A novel framework of multiclass skin lesion recognition from dermoscopic images using deep learning and explainable AI

Skin cancer is a serious disease that affects people all over the world. Melanoma is an aggressive form of skin cancer, and early detection can significantly reduce human mortality. In the United States, approximately 97,610 new cases of melanoma will be diagnosed in 2023. However, challenges such as lesion irregularities, low-contrast lesions, intraclass color similarity, redundant features, and imbalanced datasets make improved recognition accuracy using computerized techniques extremely difficult. This work presented a new framework for skin lesion recognition using data augmentation, deep learning, and explainable artificial intelligence. In the proposed framework, data augmentation is performed at the initial step to increase the dataset size, and then two pretrained deep learning models are employed. Both models have been fine-tuned and trained using deep transfer learning. Both models (Xception and ShuffleNet) utilize the global average pooling layer for deep feature extraction. The analysis of this step shows that some important information is missing; therefore, we performed the fusion. After the fusion process, the computational time was increased; therefore, we developed an improved Butterfly Optimization Algorithm. Using this algorithm, only the best features are selected and classified using machine learning classifiers. In addition, a GradCAM-based visualization is performed to analyze the important region in the image. Two publicly available datasets-ISIC2018 and HAM10000-have been utilized and obtained improved accuracy of 99.3% and 91.5%, respectively. Comparing the proposed framework accuracy with state-of-the-art methods reveals improved and less computational time.

Gastric Section Correlation Network for Gastric Precancerous Lesion Diagnosis

Goal: Diagnosing the corpus-predominant gastritis index (CGI) which is an early precancerous lesion in the stomach has been shown its effectiveness in identifying high gastric cancer risk patients for preventive healthcare. However, invasive biopsies and time-consuming pathological analysis are required for the CGI diagnosis. Methods: We propose a novel gastric section correlation network (GSCNet) for the CGI diagnosis from endoscopic images of three dominant gastric sections, the antrum, body and cardia. The proposed network consists of two dominant modules including the scaling feature fusion module and section correlation module. The front one aims to extract scaling fusion features which can effectively represent the mucosa under variant viewing angles and scale changes for each gastric section. The latter one aims to apply the medical prior knowledge with three section correlation losses to model the correlations of different gastric sections for the CGI diagnosis. Results: The proposed method outperforms competing deep learning methods and achieves high testing accuracy, sensitivity, and specificity of 0.957, 0.938 and 0.962, respectively. Conclusions: The proposed method is the first method to identify high gastric cancer risk patients with CGI from endoscopic images without invasive biopsies and time-consuming pathological analysis.

Computer vision-based solutions to overcome the limitations of wireless capsule endoscopy

Endoscopic investigation plays a critical role in the diagnosis of gastrointestinal (GI) diseases. Since 2001, Wireless Capsule Endoscopy (WCE) has been available for small bowel exploration and is in continuous development. Over the last decade, WCE has achieved impressive improvements in areas such as miniaturisation, image quality and battery life. As a result, WCE is currently a very useful alternative to wired enteroscopy in the investigation of various small bowel abnormalities and has the potential to become the leading screening technique for the entire gastrointestinal tract. However, commercial solutions still have several limitations, namely incomplete examination and limited diagnostic capacity. These deficiencies are related to technical issues, such as image quality, motion estimation and power consumption management. Computational methods, based on image processing and analysis, can help to overcome these challenges and reduce both the time required by reviewers and human interpretation errors. Research groups have proposed a series of methods including algorithms for locating the capsule or lesion, assessing intestinal motility and improving image quality.In this work, we provide a critical review of computational vision-based methods for WCE image analysis aimed at overcoming the technological challenges of capsules. This article also reviews several representative public datasets used to evaluate the performance of WCE techniques and methods. Finally, some promising solutions of computational methods based on the analysis of multiple-camera endoscopic images are presented.

A Novel Multi-Feature Fusion Method for Classification of Gastrointestinal Diseases Using Endoscopy Images

The first step in the diagnosis of gastric abnormalities is the detection of various abnormalities in the human gastrointestinal tract. Manual examination of endoscopy images relies on a medical practitioner's expertise to identify inflammatory regions on the inner surface of the gastrointestinal tract. The length of the alimentary canal and the large volume of images obtained from endoscopic procedures make traditional detection methods time consuming and laborious. Recently, deep learning architectures have achieved better results in the classification of endoscopy images. However, visual similarities between different portions of the gastrointestinal tract pose a challenge for effective disease detection. This work proposes a novel system for the classification of endoscopy images by focusing on feature mining through convolutional neural networks (CNN). The model presented is built by combining a state-of-the-art architecture (i.e., EfficientNet B0) with a custom-built CNN architecture named Effimix. The proposed Effimix model employs a combination of squeeze and excitation layers and self-normalising activation layers for precise classification of gastrointestinal diseases. Experimental observations on the HyperKvasir dataset confirm the effectiveness of the proposed architecture for the classification of endoscopy images. The proposed model yields an accuracy of 97.99%, with an F1 score, precision, and recall of 97%, 97%, and 98%, respectively, which is significantly higher compared to the existing works.

A Robust Deep Model for Classification of Peptic Ulcer and Other Digestive Tract Disorders Using Endoscopic Images

Accurate patient disease classification and detection through deep-learning (DL) models are increasingly contributing to the area of biomedical imaging. The most frequent gastrointestinal (GI) tract ailments are peptic ulcers and stomach cancer. Conventional endoscopy is a painful and hectic procedure for the patient while Wireless Capsule Endoscopy (WCE) is a useful technology for diagnosing GI problems and doing painless gut imaging. However, there is still a challenge to investigate thousands of images captured during the WCE procedure accurately and efficiently because existing deep models are not scored with significant accuracy on WCE image analysis. So, to prevent emergency conditions among patients, we need an efficient and accurate DL model for real-time analysis. In this study, we propose a reliable and efficient approach for classifying GI tract abnormalities using WCE images by applying a deep Convolutional Neural Network (CNN). For this purpose, we propose a custom CNN architecture named GI Disease-Detection Network (GIDD-Net) that is designed from scratch with relatively few parameters to detect GI tract disorders more accurately and efficiently at a low computational cost. Moreover, our model successfully distinguishes GI disorders by visualizing class activation patterns in the stomach bowls as a heat map. The Kvasir-Capsule image dataset has a significant class imbalance problem, we exploited a synthetic oversampling technique BORDERLINE SMOTE (BL-SMOTE) to evenly distribute the image among the classes to prevent the problem of class imbalance. The proposed model is evaluated against various metrics and achieved the following values for evaluation metrics: 98.9%, 99.8%, 98.9%, 98.9%, 98.8%, and 0.0474 for accuracy, AUC, F1-score, precision, recall, and loss, respectively. From the simulation results, it is noted that the proposed model outperforms other state-of-the-art models in all the evaluation metrics.

Machine learning techniques to detect bleeding frame and area in wireless capsule endoscopy video

Wireless Capsule Endoscopy (WCE) allows direct visual inspecting of the full digestive system of the patient without invasion and pain, at the price of a long examination by physicians of a large number of photographs. This research presents a new approach to color extraction to differentiate bleeding frames from normal ones and locate more bleeding areas. We have a dual-system suggestion. We use entire color information on the WCE pictures and the pixel-represented clustering approach to get the clustered centers that characterize WCE pictures as words. Then we evaluate the status of a WCE framework using the nearby SVM and K methods (KNN). The classification performance is 95.75% accurate for the AUC 0.9771% and validates the exciting performance for bleeding classification provided by the suggested approach. Second, we present a two-step approach for extracting saliency maps to emphasize bleeding locations with a distinct color channel mixer to build a first-stage salience map. The second stage salience map was taken with optical contrast.We locate bleeding spots following a suitable fusion approach and threshold. Quantitative and qualitative studies demonstrate that our approaches can correctly distinguish bleeding sites from neighborhoods.

A Source-Free Domain Adaptive Polyp Detection Framework With Style Diversification Flow

The automatic detection of polyps across colonoscopy and Wireless Capsule Endoscopy (WCE) datasets is crucial for early diagnosis and curation of colorectal cancer. Existing deep learning approaches either require mass training data collected from multiple sites or use unsupervised domain adaptation (UDA) technique with labeled source data. However, these methods are not applicable when the data is not accessible due to privacy concerns or data storage limitations. Aiming to achieve source-free domain adaptive polyp detection, we propose a consistency based model that utilizes Source Model as Proxy Teacher (SMPT) with only a transferable pretrained model and unlabeled target data. SMPT first transfers the stored domain-invariant knowledge in the pretrained source model to the target model via Source Knowledge Distillation (SKD), then uses Proxy Teacher Rectification (PTR) to rectify the source model with temporal ensemble of the target model. Moreover, to alleviate the biased knowledge caused by domain gaps, we propose Uncertainty-Guided Online Bootstrapping (UGOB) to adaptively assign weights for each target image regarding their uncertainty. In addition, we design Source Style Diversification Flow (SSDF) that gradually generates diverse style images and relaxes style-sensitive channels based on source and target information to enhance the robustness of the model towards style variation. The capacities of SMPT and SSDF are further boosted with iterative optimization, constructing a stronger framework SMPT++ for cross-domain polyp detection. Extensive experiments are conducted on five distinct polyp datasets under two types of cross-domain settings. Our proposed method shows the state-of-the-art performance and even outperforms previous UDA approaches that require the source data by a large margin. The source code is available at github.com/CityU-AIM-Group/SFPolypDA.

Characterization of motion patterns by a spatio-temporal saliency descriptor in cardiac cine MRI

BACKGROUND AND OBJECTIVE

Abnormalities of the heart motion reveal the presence of a disease. However, a quantitative interpretation of the motion is still a challenge due to the complex dynamics of the heart. This work proposes a quantitative characterization of regional cardiac motion patterns in cine magnetic resonance imaging (MRI) by a novel spatio-temporal saliency descriptor.

METHOD

The strategy starts by dividing the cardiac sequence into a progression of scales which are in due turn mapped to a feature space of regional orientation changes, mimicking the multi-resolution decomposition of oriented primitive changes of visual systems. These changes are estimated as the difference between a particular time and the rest of the sequence. This decomposition is then temporarily and regionally integrated for a particular orientation and then for the set of different orientations. A final spatio-temporal 4D saliency map is obtained as the summation of the previously integrated information for the available scales. The saliency dispersion of this map was computed in standard cardiac locations as a measure of the regional motion pattern and was applied to discriminate control and hypertrophic cardiomyopathy (HCM) subjects during the diastolic phase.

RESULTS

Salient motion patterns were estimated from an experimental set, which consisted of 3D sequences acquired by MRI from 108 subjects (33 control, 35 HCM, 20 dilated cardiomyopathy (DCM), and 20 myocardial infarction (MINF) from heterogeneous datasets). HCM and control subjects were classified by an SVM that learned the salient motion patterns estimated from the presented strategy, by achieving a 94% AUC. In addition, statistical differences (test t-student, p<0.05) were found among groups of disease in the septal and anterior ventricular segments at both the ED and ES, with salient motion characteristics aligned with existing knowledge on the diseases.

CONCLUSIONS

Regional wall motion abnormality in the apical, anterior, basal, and inferior segments was associated with the saliency dispersion in HCM, DCM, and MINF compared to healthy controls during the systolic and diastolic phases. This saliency analysis may be used to detect subtle changes in heart function.

DSI-Net: Deep Synergistic Interaction Network for Joint Classification and Segmentation With Endoscope Images

Automatic classification and segmentation of wireless capsule endoscope (WCE) images are two clinically significant and relevant tasks in a computer-aided diagnosis system for gastrointestinal diseases. Most of existing approaches, however, considered these two tasks individually and ignored their complementary information, leading to limited performance. To overcome this bottleneck, we propose a deep synergistic interaction network (DSI-Net) for joint classification and segmentation with WCE images, which mainly consists of the classification branch (C-Branch), the coarse segmentation (CS-Branch) and the fine segmentation branches (FS-Branch). In order to facilitate the classification task with the segmentation knowledge, a lesion location mining (LLM) module is devised in C-Branch to accurately highlight lesion regions through mining neglected lesion areas and erasing misclassified background areas. To assist the segmentation task with the classification prior, we propose a category-guided feature generation (CFG) module in FS-Branch to improve pixel representation by leveraging the category prototypes of C-Branch to obtain the category-aware features. In such way, these modules enable the deep synergistic interaction between these two tasks. In addition, we introduce a task interaction loss to enhance the mutual supervision between the classification and segmentation tasks and guarantee the consistency of their predictions. Relying on the proposed deep synergistic interaction mechanism, DSI-Net achieves superior classification and segmentation performance on public dataset in comparison with state-of-the-art methods. The source code is available at https://github.com/CityU-AIM-Group/DSI-Net.

Small-Bowel Capsule Endoscopy-Optimizing Capsule Endoscopy in Clinical Practice

The small bowel is the longest organ within the gastrointestinal tract. The emergence of small bowel capsule endoscopy (SBCE) over the last 20 years has revolutionized the investigation and diagnosis of small bowel pathology. Its utility as a non-invasive and well-tolerated procedure, which can be performed in an outpatient setting, has made it a valuable diagnostic tool. The indications for SBCE include obscure gastrointestinal bleeding, small bowel Crohn's disease, and, less frequently for screening in polyposis syndromes, celiac disease, or other small bowel pathology. Currently, there are several small bowel capsules on the market from different manufacturers; however, they share many technological features. The European Society of Gastrointestinal Endoscopy (ESGE) only recently developed a set of key quality indicators to guide quality standards in this area. Many of the technical aspects of capsule endoscopy still feature a degree of uncertainty in terms of optimal performance. Incomplete studies due to slow transit through the bowel, poor imaging secondary to poor preparation, and the risk of capsule retention remain frustrations in its clinical utility. Capsule review is a time-consuming process; however, artificial intelligence and machine learning offer opportunities to improve this. This narrative review examines our current standing in a number of these aspects and the potential to further the application of SBCE in order to maximize its diagnostic utility.

A systematic review on application of deep learning in digestive system image processing

With the advent of the big data era, the application of artificial intelligence represented by deep learning in medicine has become a hot topic. In gastroenterology, deep learning has accomplished remarkable accomplishments in endoscopy, imageology, and pathology. Artificial intelligence has been applied to benign gastrointestinal tract lesions, early cancer, tumors, inflammatory bowel diseases, livers, pancreas, and other diseases. Computer-aided diagnosis significantly improve diagnostic accuracy and reduce physicians' workload and provide a shred of evidence for clinical diagnosis and treatment. In the near future, artificial intelligence will have high application value in the field of medicine. This paper mainly summarizes the latest research on artificial intelligence in diagnosing and treating digestive system diseases and discussing artificial intelligence's future in digestive system diseases. We sincerely hope that our work can become a stepping stone for gastroenterologists and computer experts in artificial intelligence research and facilitate the application and development of computer-aided image processing technology in gastroenterology.

Quantification of anomalies in rats' spinal cords using autoencoders

Computed tomography (CT) scans and magnetic resonance imaging (MRI) of spines are state-of-the-art for the evaluation of spinal cord lesions. This paper analyses micro-CT scans of rat spinal cords with the aim of generating lesion progression through the aggregation of anomaly-based scores. Since reliable labelling in spinal cords is only reasonable for the healthy class in the form of untreated spines, semi-supervised deviation-based anomaly detection algorithms are identified as powerful approaches. The main contribution of this paper is a large evaluation of different autoencoders and variational autoencoders for aggregated lesion quantification and a resulting spinal cord lesion quantification method that generates highly correlating quantifications. The conducted experiments showed that several models were able to generate 3D lesion quantifications of the data. These quantifications correlated with the weakly labelled true data with one model, reaching an average correlation of 0.83. We also introduced an area-based model, which correlated with a mean of 0.84. The possibility of the complementary use of the autoencoder-based method and the area feature were also discussed. Additionally to improving medical diagnostics, we anticipate features built on these quantifications to be useful for further applications like clustering into different lesions.

Deep learning and capsule endoscopy: automatic identification and differentiation of small bowel lesions with distinct haemorrhagic potential using a convolutional neural network

OBJECTIVE

Capsule endoscopy (CE) is pivotal for evaluation of small bowel disease. Obscure gastrointestinal bleeding most often originates from the small bowel. CE frequently identifies a wide range of lesions with different bleeding potentials in these patients. However, reading CE examinations is a time-consuming task. Convolutional neural networks (CNNs) are highly efficient artificial intelligence tools for image analysis. This study aims to develop a CNN-based model for identification and differentiation of multiple small bowel lesions with distinct haemorrhagic potential using CE images.

DESIGN

We developed, trained, and validated a denary CNN based on CE images. Each frame was labelled according to the type of lesion (lymphangiectasia, xanthomas, ulcers, erosions, vascular lesions, protruding lesions, and blood). The haemorrhagic potential was assessed by Saurin's classification. The entire dataset was divided into training and validation sets. The performance of the CNN was measured by the area under the receiving operating characteristic curve, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV).

RESULTS

A total of 53 555 CE images were included. The model had an overall accuracy of 99%, a sensitivity of 88%, a specificity of 99%, a PPV of 87%, and an NPV of 99% for detection of multiple small bowel abnormalities and respective classification of bleeding potential.

CONCLUSION

We developed and tested a CNN-based model for automatic detection of multiple types of small bowel lesions and classification of the respective bleeding potential. This system may improve the diagnostic yield of CE for these lesions and overall CE efficiency.

Multi-pathology detection and lesion localization in WCE videos by using the instance segmentation approach

The majority of current systems for automatic diagnosis considers the detection of a unique and previously known pathology. Considering specifically the diagnosis of lesions in the small bowel using endoscopic capsule images, very few consider the possible existence of more than one pathology and when they do, they are mainly detection based systems therefore unable to localize the suspected lesions. Such systems do not fully satisfy the medical community, that in fact needs a system that detects any pathology and eventually more than one, when they coexist. In addition, besides the diagnostic capability of these systems, localizing the lesions in the image has been of great interest to the medical community, mainly for training medical personnel purposes. So, nowadays, the inclusion of the lesion location in automatic diagnostic systems is practically mandatory. Multi-pathology detection can be seen as a multi-object detection task and as each frame can contain different instances of the same lesion, instance segmentation seems to be appropriate for the purpose. Consequently, we argue that a multi-pathology system benefits from using the instance segmentation approach, since classification and segmentation modules are both required complementing each other in lesion detection and localization. According to our best knowledge such a system does not yet exist for the detection of WCE pathologies. This paper proposes a multi-pathology system that can be applied to WCE images, which uses the Mask Improved RCNN (MI-RCNN), a new mask subnet scheme which has shown to significantly improve mask predictions of the high performing state-of-the-art Mask-RCNN and PANet systems. A novel training strategy based on the second momentum is also proposed for the first time for training Mask-RCNN and PANet based systems. These approaches were tested using the public database KID, and the included pathologies were bleeding, angioectasias, polyps and inflammatory lesions. Experimental results show significant improvements for the proposed versions, reaching increases of almost 7% over the PANet model when the new proposed training approach was employed.

Efficacy of a comprehensive binary classification model using a deep convolutional neural network for wireless capsule endoscopy

The manual reading of capsule endoscopy (CE) videos in small bowel disease diagnosis is time-intensive. Algorithms introduced to automate this process are premature for real clinical applications, and multi-diagnosis using these methods has not been sufficiently validated. Therefore, we developed a practical binary classification model, which selectively identifies clinically meaningful images including inflamed mucosa, atypical vascularity or bleeding, and tested it with unseen cases. Four hundred thousand CE images were randomly selected from 84 cases in which 240,000 images were used to train the algorithm to categorize images binarily. The remaining images were utilized for validation and internal testing. The algorithm was externally tested with 256,591 unseen images. The diagnostic accuracy of the trained model applied to the validation set was 98.067%. In contrast, the accuracy of the model when applied to a dataset provided by an independent hospital that did not participate during training was 85.470%. The area under the curve (AUC) was 0.922. Our model showed excellent internal test results, and the misreadings were slightly increased when the model was tested in unseen external cases while the classified 'insignificant' images contain ambiguous substances. Once this limitation is solved, the proposed CNN-based binary classification will be a promising candidate for developing clinically-ready computer-aided reading methods.

A deep CNN model for anomaly detection and localization in wireless capsule endoscopy images

Wireless capsule endoscopy (WCE) is one of the most efficient methods for the examination of gastrointestinal tracts. Computer-aided intelligent diagnostic tools alleviate the challenges faced during manual inspection of long WCE videos. Several approaches have been proposed in the literature for the automatic detection and localization of anomalies in WCE images. Some of them focus on specific anomalies such as bleeding, polyp, lesion, etc. However, relatively fewer generic methods have been proposed to detect all those common anomalies simultaneously. In this paper, a deep convolutional neural network (CNN) based model 'WCENet' is proposed for anomaly detection and localization in WCE images. The model works in two phases. In the first phase, a simple and efficient attention-based CNN classifies an image into one of the four categories: polyp, vascular, inflammatory, or normal. If the image is classified in one of the abnormal categories, it is processed in the second phase for the anomaly localization. Fusion of Grad-CAM++ and a custom SegNet is used for anomalous region segmentation in the abnormal image. WCENet classifier attains accuracy and area under receiver operating characteristic of 98% and 99%. The WCENet segmentation model obtains a frequency weighted intersection over union of 81%, and an average dice score of 56% on the KID dataset. WCENet outperforms nine different state-of-the-art conventional machine learning and deep learning models on the KID dataset. The proposed model demonstrates potential for clinical applications.

Magnetically Controlled Capsule Endoscopy Versus Conventional Gastroscopy: A Systematic Review and Meta-Analysis

BACKGROUND

The introduction of magnetically controlled capsule endoscopy overcame the restriction of passive capsule endoscopy movement, thus allowing an improved visualization of the gastrointestinal lumen, where other imaging studies seem to be unhelpful. The aim of this study is to systematically review the performance of magnetically controlled capsule endoscopy and evaluate its potential as a less invasive diagnostic method in the detection of gastric lesions.

METHODS

A systematic search was performed in PubMed (Medline), EMBASE, Google Scholar, Scopus, Who Global Health Library (GHL), Virtual Health Library (VHL), Clinicaltrials.gov, Cochrane Library, and ISI Web of Science databases. Proportion meta-analyses were performed to estimate the pooled sensitivity of magnetically controlled capsuled endoscopy in the detection of gastrointestinal lesions.

RESULTS

Among the 3026 studies that were initially assessed, 7 studies were finally included, with a total of 916 patients and 745 gastric lesions. The mean capsule endoscopy examination time was 21.92±8.87 minutes. The pooled overall sensitivity of magnetically controlled capsule endoscopy was 87% [95% confidence interval (CI), 84%-89%]. Subgroup analysis showed that the sensitivity of identifying gastric ulcers was 82% (95% CI: 71%-89%), gastric polyps was 82% (95% CI: 76%-87%), and gastric erosions was 95% (95% CI: 86%-98%). In general, magnetically controlled capsule endoscopy was well tolerated by the participants with minimal adverse events.

CONCLUSION

The magnetically controlled capsule endoscopy demonstrated an acceptable sensitivity of identifying gastric lesions. Further prospective comparative studies are needed to identify the risks and benefits of this new technique, as well as to determine its role as a replacement for conventional gastroscopy.

Deep Learning Application for Analyzing of Constituents and Their Correlations in the Interpretations of Medical Images

The need for time and attention, given by the doctor to the patient, due to the increased volume of medical data to be interpreted and filtered for diagnostic and therapeutic purposes has encouraged the development of the option to support, constructively and effectively, deep learning models. Deep learning (DL) has experienced an exponential development in recent years, with a major impact on interpretations of the medical image. This has influenced the development, diversification and increase of the quality of scientific data, the development of knowledge construction methods and the improvement of DL models used in medical applications. All research papers focus on description, highlighting, classification of one of the constituent elements of deep learning models (DL), used in the interpretation of medical images and do not provide a unified picture of the importance and impact of each constituent in the performance of DL models. The novelty in our paper consists primarily in the unitary approach, of the constituent elements of DL models, namely, data, tools used by DL architectures or specifically constructed DL architecture combinations and highlighting their "key" features, for completion of tasks in current applications in the interpretation of medical images. The use of "key" characteristics specific to each constituent of DL models and the correct determination of their correlations, may be the subject of future research, with the aim of increasing the performance of DL models in the interpretation of medical images.

Role of Artificial Intelligence in Video Capsule Endoscopy

Capsule endoscopy (CE) has been increasingly utilised in recent years as a minimally invasive tool to investigate the whole gastrointestinal (GI) tract and a range of capsules are currently available for evaluation of upper GI, small bowel, and lower GI pathology. Although CE is undoubtedly an invaluable test for the investigation of small bowel pathology, it presents considerable challenges and limitations, such as long and laborious reading times, risk of missing lesions, lack of bowel cleansing score and lack of locomotion. Artificial intelligence (AI) seems to be a promising tool that may help improve the performance metrics of CE, and consequently translate to better patient care. In the last decade, significant progress has been made to apply AI in the field of endoscopy, including CE. Although it is certain that AI will find soon its place in day-to-day endoscopy clinical practice, there are still some open questions and barriers limiting its widespread application. In this review, we provide some general information about AI, and outline recent advances in AI and CE, issues around implementation of AI in medical practice and potential future applications of AI-aided CE.

A comprehensive analysis of classification methods in gastrointestinal endoscopy imaging

Gastrointestinal (GI) endoscopy has been an active field of research motivated by the large number of highly lethal GI cancers. Early GI cancer precursors are often missed during the endoscopic surveillance. The high missed rate of such abnormalities during endoscopy is thus a critical bottleneck. Lack of attentiveness due to tiring procedures, and requirement of training are few contributing factors. An automatic GI disease classification system can help reduce such risks by flagging suspicious frames and lesions. GI endoscopy consists of several multi-organ surveillance, therefore, there is need to develop methods that can generalize to various endoscopic findings. In this realm, we present a comprehensive analysis of the Medico GI challenges: Medical Multimedia Task at MediaEval 2017, Medico Multimedia Task at MediaEval 2018, and BioMedia ACM MM Grand Challenge 2019. These challenges are initiative to set-up a benchmark for different computer vision methods applied to the multi-class endoscopic images and promote to build new approaches that could reliably be used in clinics. We report the performance of 21 participating teams over a period of three consecutive years and provide a detailed analysis of the methods used by the participants, highlighting the challenges and shortcomings of the current approaches and dissect their credibility for the use in clinical settings. Our analysis revealed that the participants achieved an improvement on maximum Mathew correlation coefficient (MCC) from 82.68% in 2017 to 93.98% in 2018 and 95.20% in 2019 challenges, and a significant increase in computational speed over consecutive years.

Medical needs related to the endoscopic technology and colonoscopy for colorectal cancer diagnosis

Background

The high incidence and mortality rate of colorectal cancer require new technologies to improve its early diagnosis. This study aims at extracting the medical needs related to the endoscopic technology and the colonoscopy procedure currently used for colorectal cancer diagnosis, essential for designing these demanded technologies.

Methods

Semi-structured interviews and an online survey were used.

Results

Six endoscopists were interviewed and 103 were surveyed, obtaining the demanded needs that can be divided into: a) clinical needs, for better polyp detection and classification (especially flat polyps), location, size, margins and penetration depth; b) computer-aided diagnosis (CAD) system needs, for additional visual information supporting polyp characterization and diagnosis; and c) operational/physical needs, related to limitations of image quality, colon lighting, flexibility of the endoscope tip, and even poor bowel preparation.

Conclusions

This study shows some undertaken initiatives to meet the detected medical needs and challenges to be solved. The great potential of advanced optical technologies suggests their use for a better polyp detection and classification since they provide additional functional and structural information than the currently used image enhancement technologies. The inspection of remaining tissue of diminutive polyps (< 5 mm) should be addressed to reduce recurrence rates. Few progresses have been made in estimating the infiltration depth. Detection and classification methods should be combined into one CAD system, providing visual aids over polyps for detection and displaying a Kudo-based diagnosis suggestion to assist the endoscopist on real-time decision making. Estimated size and location of polyps should also be provided. Endoscopes with 360° vision are still a challenge not met by the mechanical and optical systems developed to improve the colon inspection. Patients and healthcare providers should be trained to improve the patient’s bowel preparation.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12885-021-08190-z.

State of the Art: The Impact of Artificial Intelligence in Endoscopy 2020

PURPOSE OF REVIEW

Recently numerous researchers have shown remarkable progress using convolutional neural network-based artificial intelligence (AI) for endoscopy. In this manuscript we aim to summarize recent AI impact on endoscopy.

RECENT FINDINGS

AI for detecting colon polyps has been the most promising area for application of AI in endoscopy. Recent prospective randomized studies showed that AI assisted colonoscopy increased adenoma detection rate and the mean number of adenomas per patient compared to standard colonoscopy alone. AI for optical biopsy of colon polyp showed a negative predictive value of ≥90%. For capsule endoscopy, applying AI to pre-read the video images decreased physician reading time significantly. Recently, researchers are broadening the area of AI to quality assessment of endoscopy such as bowel preparation and automated report generation. AI systems have shown great potential to increase physician performance by enhancing detection, reducing procedure time, and providing real-time feedback of endoscopy quality. To build a generally applicable AI, we need further investigations in real world settings and also integration of AI tools into pragmatic platforms.