Artificial intelligence that determines the clinical significance of capsule endoscopy images can increase the efficiency of reading

Impact of Artificial Intelligence on Gastroenterology Trainee Education

Artificial intelligence (AI) is transforming gastroenterology, particularly in endoscopy, which has a direct impact on trainees and their education. AI can serve as a valuable resource, providing real-time feedback and aiding in tasks like polyp detection and lesion differentiation, which are challenging for trainees. However, its implementation raises concerns about cognitive overload, overreliance, and even access disparities, which could affect training outcomes. Beyond endoscopy, AI shows promise in clinical management and interpreting diagnostic studies such as motility testing. Thoughtful adoption of AI can optimize training and prepare future trainees for the modern healthcare landscape.

Reducing the workload of medical diagnosis through artificial intelligence: A narrative review

Artificial intelligence (AI) has revolutionized medical diagnostics by enhancing efficiency, improving accuracy, and reducing variability. By alleviating the workload of medical staff, AI addresses challenges such as increasing diagnostic demands, workforce shortages, and reliance on subjective interpretation. This review examines the role of AI in reducing diagnostic workload and enhancing efficiency across medical fields from January 2019 to February 2024, identifying limitations and areas for improvement. A comprehensive PubMed search using the keywords "artificial intelligence" or "AI," "efficiency" or "workload," and "patient" or "clinical" identified 2587 articles, of which 51 were reviewed. These studies analyzed the impact of AI on radiology, pathology, and other specialties, focusing on efficiency, accuracy, and workload reduction. The final 51 articles were categorized into 4 groups based on diagnostic efficiency, where category A included studies with supporting material provided, category B consisted of those with reduced data volume, category C focused on independent AI diagnosis, and category D included studies that reported data reduction without changes in diagnostic time. In radiology and pathology, which require skilled techniques and large-scale data processing, AI improved accuracy and reduced diagnostic time by approximately 90% or more. Radiology, in particular, showed a high proportion of category C studies, as digitized data and standardized protocols facilitated independent AI diagnoses. AI has significant potential to optimize workload management, improve diagnostic efficiency, and enhance accuracy. However, challenges remain in standardizing applications and addressing ethical concerns. Integrating AI into healthcare workforce planning is essential for fostering collaboration between technology and clinicians, ultimately improving patient care.

Impact of human and artificial intelligence collaboration on workload reduction in medical image interpretation

Clinicians face increasing workloads in medical imaging interpretation, and artificial intelligence (AI) offers potential relief. This meta-analysis evaluates the impact of human-AI collaboration on image interpretation workload. Four databases were searched for studies comparing reading time or quantity for image-based disease detection before and after AI integration. The Quality Assessment of Studies of Diagnostic Accuracy was modified to assess risk of bias. Workload reduction and relative diagnostic performance were pooled using random-effects model. Thirty-six studies were included. AI concurrent assistance reduced reading time by 27.20% (95% confidence interval, 18.22%-36.18%). The reading quantity decreased by 44.47% (40.68%-48.26%) and 61.72% (47.92%-75.52%) when AI served as the second reader and pre-screening, respectively. Overall relative sensitivity and specificity are 1.12 (1.09, 1.14) and 1.00 (1.00, 1.01), respectively. Despite these promising results, caution is warranted due to significant heterogeneity and uneven study quality.

Artificial intelligence in capsule endoscopy: development status and future expectations

In this review, we aim to illustrate the state-of-the-art artificial intelligence (AI) applications in the field of capsule endoscopy. AI has made significant strides in gastrointestinal imaging, particularly in capsule endoscopy - a non-invasive procedure for capturing gastrointestinal tract images. However, manual analysis of capsule endoscopy videos is labour-intensive and error-prone, prompting the development of automated computational algorithms and AI models. While currently serving as a supplementary observer, AI has the capacity to evolve into an autonomous, integrated reading system, potentially significantly reducing capsule reading time while surpassing human accuracy. We searched Embase, Pubmed, Medline, and Cochrane databases from inception to 06 Jul 2023 for studies investigating the use of AI for capsule endoscopy and screened retrieved records for eligibility. Quantitative and qualitative data were extracted and synthesised to identify current themes. In the search, 824 articles were collected, and 291 duplicates and 31 abstracts were deleted. After a double-screening process and full-text review, 106 publications were included in the review. Themes pertaining to AI for capsule endoscopy included active gastrointestinal bleeding, erosions and ulcers, vascular lesions and angiodysplasias, polyps and tumours, inflammatory bowel disease, coeliac disease, hookworms, bowel prep assessment, and multiple lesion detection. This review provides current insights into the impact of AI on capsule endoscopy as of 2023. AI holds the potential for faster and precise readings and the prospect of autonomous image analysis. However, careful consideration of diagnostic requirements and potential challenges is crucial. The untapped potential within vision transformer technology hints at further evolution and even greater patient benefit.

Development and validation of a deep learning system for detection of small bowel pathologies in capsule endoscopy: a pilot study in a Singapore institution

INTRODUCTION

Deep learning models can assess the quality of images and discriminate among abnormalities in small bowel capsule endoscopy (CE), reducing fatigue and the time needed for diagnosis. They serve as a decision support system, partially automating the diagnosis process by providing probability predictions for abnormalities.

METHODS

We demonstrated the use of deep learning models in CE image analysis, specifically by piloting a bowel preparation model (BPM) and an abnormality detection model (ADM) to determine frame-level view quality and the presence of abnormal findings, respectively. We used convolutional neural network-based models pretrained on large-scale open-domain data to extract spatial features of CE images that were then used in a dense feed-forward neural network classifier. We then combined the open-source Kvasir-Capsule dataset (n = 43) and locally collected CE data (n = 29).

RESULTS

Model performance was compared using averaged five-fold and two-fold cross-validation for BPMs and ADMs, respectively. The best BPM model based on a pre-trained ResNet50 architecture had an area under the receiver operating characteristic and precision-recall curves of 0.969±0.008 and 0.843±0.041, respectively. The best ADM model, also based on ResNet50, had top-1 and top-2 accuracies of 84.03±0.051 and 94.78±0.028, respectively. The models could process approximately 200-250 images per second and showed good discrimination on time-critical abnormalities such as bleeding.

CONCLUSION

Our pilot models showed the potential to improve time to diagnosis in CE workflows. To our knowledge, our approach is unique to the Singapore context. The value of our work can be further evaluated in a pragmatic manner that is sensitive to existing clinician workflow and resource constraints.

Reading of small bowel capsule endoscopy after frame reduction using an artificial intelligence algorithm

OBJECTIVES

Poorly visualized images that appear during small bowel capsule endoscopy (SBCE) can confuse the interpretation of small bowel lesions and increase the physician's workload. Using a validated artificial intelligence (AI) algorithm that can evaluate the mucosal visualization, we aimed to assess whether SBCE reading after the removal of poorly visualized images could affect the diagnosis of SBCE.

METHODS

A study was conducted to analyze 90 SBCE cases in which a small bowel examination was completed. Two experienced endoscopists alternately performed two types of readings. They used the AI algorithm to remove poorly visualized images for the frame reduction reading (AI user group) and conducted whole frame reading without AI (AI non-user group) for the same patient. A poorly visualized image was defined as an image with < 50% mucosal visualization. The study outcomes were diagnostic concordance and reading time between the two groups. The SBCE diagnosis was classified as Crohn's disease, bleeding, polyp, angiodysplasia, and nonspecific finding.

RESULTS

The final SBCE diagnoses between the two groups showed statistically significant diagnostic concordance (k = 0.954, p < 0.001). The mean number of lesion images was 3008.5 ± 9964.9 in the AI non-user group and 1401.7 ± 4811.3 in the AI user group. There were no cases in which lesions were completely removed. Compared with the AI non-user group (120.9 min), the reading time was reduced by 35.6% in the AI user group (77.9 min).

CONCLUSIONS

SBCE reading after reducing poorly visualized frames using the AI algorithm did not have a negative effect on the final diagnosis. SBCE reading method integrated with frame reduction and mucosal visualization evaluation will help improve AI-assisted SBCE interpretation.

Wireless capsule endoscopy multiclass classification using three-dimensional deep convolutional neural network model

BACKGROUND

Wireless capsule endoscopy (WCE) is a patient-friendly and non-invasive technology that scans the whole of the gastrointestinal tract, including difficult-to-access regions like the small bowel. Major drawback of this technology is that the visual inspection of a large number of video frames produced during each examination makes the physician diagnosis process tedious and prone to error. Several computer-aided diagnosis (CAD) systems, such as deep network models, have been developed for the automatic recognition of abnormalities in WCE frames. Nevertheless, most of these studies have only focused on spatial information within individual WCE frames, missing the crucial temporal data within consecutive frames.

METHODS

In this article, an automatic multiclass classification system based on a three-dimensional deep convolutional neural network (3D-CNN) is proposed, which utilizes the spatiotemporal information to facilitate the WCE diagnosis process. The 3D-CNN model fed with a series of sequential WCE frames in contrast to the two-dimensional (2D) model, which exploits frames as independent ones. Moreover, the proposed 3D deep model is compared with some pre-trained networks. The proposed models are trained and evaluated with 29 subject WCE videos (14,691 frames before augmentation). The performance advantages of 3D-CNN over 2D-CNN and pre-trained networks are verified in terms of sensitivity, specificity, and accuracy.

RESULTS

3D-CNN outperforms the 2D technique in all evaluation metrics (sensitivity: 98.92 vs. 98.05, specificity: 99.50 vs. 86.94, accuracy: 99.20 vs. 92.60). In conclusion, a novel 3D-CNN model for lesion detection in WCE frames is proposed in this study.

CONCLUSION

The results indicate the performance of 3D-CNN over 2D-CNN and some well-known pre-trained classifier networks. The proposed 3D-CNN model uses the rich temporal information in adjacent frames as well as spatial data to develop an accurate and efficient model.

Automatic Classification of GI Organs in Wireless Capsule Endoscopy Using a No-Code Platform-Based Deep Learning Model

The first step in reading a capsule endoscopy (CE) is determining the gastrointestinal (GI) organ. Because CE produces too many inappropriate and repetitive images, automatic organ classification cannot be directly applied to CE videos. In this study, we developed a deep learning algorithm to classify GI organs (the esophagus, stomach, small bowel, and colon) using a no-code platform, applied it to CE videos, and proposed a novel method to visualize the transitional area of each GI organ. We used training data (37,307 images from 24 CE videos) and test data (39,781 images from 30 CE videos) for model development. This model was validated using 100 CE videos that included "normal", "blood", "inflamed", "vascular", and "polypoid" lesions. Our model achieved an overall accuracy of 0.98, precision of 0.89, recall of 0.97, and F1 score of 0.92. When we validated this model relative to the 100 CE videos, it produced average accuracies for the esophagus, stomach, small bowel, and colon of 0.98, 0.96, 0.87, and 0.87, respectively. Increasing the AI score's cut-off improved most performance metrics in each organ (p < 0.05). To locate a transitional area, we visualized the predicted results over time, and setting the cut-off of the AI score to 99.9% resulted in a better intuitive presentation than the baseline. In conclusion, the GI organ classification AI model demonstrated high accuracy on CE videos. The transitional area could be more easily located by adjusting the cut-off of the AI score and visualization of its result over time.

Small Bowel Detection for Wireless Capsule Endoscopy Using Convolutional Neural Networks with Temporal Filtering

By automatically classifying the stomach, small bowel, and colon, the reading time of the wireless capsule endoscopy (WCE) can be reduced. In addition, it is an essential first preprocessing step to localize the small bowel in order to apply automated small bowel lesion detection algorithms based on deep learning. The purpose of the study was to develop an automated small bowel detection method from long untrimmed videos captured from WCE. Through this, the stomach and colon can also be distinguished. The proposed method is based on a convolutional neural network (CNN) with a temporal filtering on the predicted probabilities from the CNN. For CNN, we use a ResNet50 model to classify three organs including stomach, small bowel, and colon. The hybrid temporal filter consisting of a Savitzky–Golay filter and a median filter is applied to the temporal probabilities for the “small bowel” class. After filtering, the small bowel and the other two organs are differentiated with thresholding. The study was conducted on dataset of 200 patients (100 normal and 100 abnormal WCE cases), which was divided into a training set of 140 cases, a validation set of 20 cases, and a test set of 40 cases. For the test set of 40 patients (20 normal and 20 abnormal WCE cases), the proposed method showed accuracy of 99.8% in binary classification for the small bowel. Transition time errors for gastrointestinal tracts were only 38.8 ± 25.8 seconds for the transition between stomach and small bowel and 32.0 ± 19.1 seconds for the transition between small bowel and colon, compared to the ground truth organ transition points marked by two experienced gastroenterologists.

Development of a Deep-Learning Algorithm for Small Bowel-Lesion Detection and a Study of the Improvement in the False-Positive Rate

Deep learning has recently been gaining attention as a promising technology to improve the identification of lesions, and deep-learning algorithms for lesion detection have been actively developed in small-bowel capsule endoscopy (SBCE). We developed a detection algorithm for abnormal findings by deep learning (convolutional neural network) the SBCE imaging data of 30 cases with abnormal findings. To enable the detection of a wide variety of abnormal findings, the training data were balanced to include all major findings identified in SBCE (bleeding, angiodysplasia, ulceration, and neoplastic lesions). To reduce the false-positive rate, "findings that may be responsible for hemorrhage" and "findings that may require therapeutic intervention" were extracted from the images of abnormal findings and added to the training dataset. For the performance evaluation, the sensitivity and the specificity were calculated using 271 detectable findings in 35 cases. The sensitivity was calculated using 68,494 images of non-abnormal findings. The sensitivity and specificity were 93.4% and 97.8%, respectively. The average number of images detected by the algorithm as having abnormal findings was 7514. We developed an image-reading support system using deep learning for SBCE and obtained a good detection performance.

Automated evaluation of colon capsule endoscopic severity of ulcerative colitis using ResNet50

Capsule endoscopy has been widely used as a non-invasive diagnostic tool for small or large intestinal lesions. In recent years, automated lesion detection systems using machine learning have been devised. This study aimed to develop an automated system for capsule endoscopic severity in patients with ulcerative colitis along the entire length of the colon using ResNet50. Capsule endoscopy videos from patients with ulcerative colitis were collected prospectively. Each single examination video file was partitioned into four segments: the cecum and ascending colon, transverse colon, descending and sigmoid colon, and rectum. Fifty still pictures (576 × 576 pixels) were extracted from each partitioned video. A patch (128 × 128 pixels) was trimmed from the still picture at every 32-pixel-strides. A total of 739,021 patch images were manually classified into six categories: 0) Mayo endoscopic subscore (MES) 0, 1) MES1, 2) MES2, 3) MES3, 4) inadequate quality for evaluation, and 5) ileal mucosa. ResNet50, a deep learning framework, was trained using 483,644 datasets and validated using 255,377 independent datasets. In total, 31 capsule endoscopy videos from 22 patients were collected. The accuracy rates of the training and validation datasets were 0.992 and 0.973, respectively. An automated evaluation system for the capsule endoscopic severity of ulcerative colitis was developed. This could be a useful tool for assessing topographic disease activity, thus decreasing the burden of image interpretation on endoscopists.

Challenges and opportunities in the application of artificial intelligence in gastroenterology and hepatology

Artificial intelligence (AI) is an umbrella term used to describe a cluster of interrelated fields. Machine learning (ML) refers to a model that learns from past data to predict future data. Medicine and particularly gastroenterology and hepatology, are data-rich fields with extensive data repositories, and therefore fruitful ground for AI/ML-based software applications. In this study, we comprehensively review the current applications of AI/ML-based models in these fields and the opportunities that arise from their application. Specifically, we refer to the applications of AI/ML-based models in prevention, diagnosis, management, and prognosis of gastrointestinal bleeding, inflammatory bowel diseases, gastrointestinal premalignant and malignant lesions, other nonmalignant gastrointestinal lesions and diseases, hepatitis B and C infection, chronic liver diseases, hepatocellular carcinoma, cholangiocarcinoma, and primary sclerosing cholangitis. At the same time, we identify the major challenges that restrain the widespread use of these models in healthcare in an effort to explore ways to overcome them. Notably, we elaborate on the concerns regarding intrinsic biases, data protection, cybersecurity, intellectual property, liability, ethical challenges, and transparency. Even at a slower pace than anticipated, AI is infiltrating the healthcare industry. AI in healthcare will become a reality, and every physician will have to engage with it by necessity.

Artificial Intelligence in Capsule Endoscopy: A Practical Guide to Its Past and Future Challenges

Artificial intelligence (AI) has revolutionized the medical diagnostic process of various diseases. Since the manual reading of capsule endoscopy videos is a time-intensive, error-prone process, computerized algorithms have been introduced to automate this process. Over the past decade, the evolution of convolutional neural network (CNN) enabled AI to detect multiple lesions simultaneously with increasing accuracy and sensitivity. Difficulty in validating CNN performance and unique characteristics of capsule endoscopy images make computer-aided reading systems in capsule endoscopy still on a preclinical level. Although AI technology can be used as an auxiliary second observer in capsule endoscopy, it is expected that in the near future, it will effectively reduce the reading time and ultimately become an independent, integrated reading system.

A Current and Newly Proposed Artificial Intelligence Algorithm for Reading Small Bowel Capsule Endoscopy

Small bowel capsule endoscopy (SBCE) is one of the most useful methods for diagnosing small bowel mucosal lesions. However, it takes a long time to interpret the capsule images. To solve this problem, artificial intelligence (AI) algorithms for SBCE readings are being actively studied. In this article, we analyzed several studies that applied AI algorithms to SBCE readings, such as automatic lesion detection, automatic classification of bowel cleanliness, and automatic compartmentalization of small bowels. In addition to automatic lesion detection using AI algorithms, a new direction of AI algorithms related to shorter reading times and improved lesion detection accuracy should be considered. Therefore, it is necessary to develop an integrated AI algorithm composed of algorithms with various functions in order to be used in clinical practice.

Current situation and prospect of artificial intelligence application in endoscopic diagnosis of Helicobacter pylori infection

With the appearance and prevalence of deep learning, artificial intelligence (AI) has been broadly studied and made great progress in various fields of medicine, including gastroenterology. Helicobacter pylori (H. pylori), closely associated with various digestive and extradigestive diseases, has a high infection rate worldwide. Endoscopic surveillance can evaluate H. pylori infection situations and predict the risk of gastric cancer, but there is no objective diagnostic criteria to eliminate the differences between operators. The computer-aided diagnosis system based on AI technology has demonstrated excellent performance for the diagnosis of H. pylori infection, which is superior to novice endoscopists and similar to skilled. Compared with the visual diagnosis of H. pylori infection by endoscopists, AI possesses voluminous advantages: High accuracy, high efficiency, high quality control, high objectivity, and high-effect teaching. This review summarizes the previous and recent studies on AI-assisted diagnosis of H. pylori infection, points out the limitations, and puts forward prospect for future research.

Development and Verification of a Deep Learning Algorithm to Evaluate Small-Bowel Preparation Quality

Capsule endoscopy (CE) quality control requires an objective scoring system to evaluate the preparation of the small bowel (SB). We propose a deep learning algorithm to calculate SB cleansing scores and verify the algorithm’s performance. A 5-point scoring system based on clarity of mucosal visualization was used to develop the deep learning algorithm (400,000 frames; 280,000 for training and 120,000 for testing). External validation was performed using additional CE cases (n = 50), and average cleansing scores (1.0 to 5.0) calculated using the algorithm were compared to clinical grades (A to C) assigned by clinicians. Test results obtained using 120,000 frames exhibited 93% accuracy. The separate CE case exhibited substantial agreement between the deep learning algorithm scores and clinicians’ assessments (Cohen’s kappa: 0.672). In the external validation, the cleansing score decreased with worsening clinical grade (scores of 3.9, 3.2, and 2.5 for grades A, B, and C, respectively, p < 0.001). Receiver operating characteristic curve analysis revealed that a cleansing score cut-off of 2.95 indicated clinically adequate preparation. This algorithm provides an objective and automated cleansing score for evaluating SB preparation for CE. The results of this study will serve as clinical evidence supporting the practical use of deep learning algorithms for evaluating SB preparation quality.

Automatic lumen detection and magnetic alignment control for magnetic-assisted capsule colonoscope system optimization

We developed a magnetic-assisted capsule colonoscope system with integration of computer vision-based object detection and an alignment control scheme. Two convolutional neural network models A and B for lumen identification were trained on an endoscopic dataset of 9080 images. In the lumen alignment experiment, models C and D used a simulated dataset of 8414 images. The models were evaluated using validation indexes for recall (R), precision (P), mean average precision (mAP), and F1 score. Predictive performance was evaluated with the area under the P-R curve. Adjustments of pitch and yaw angles and alignment control time were analyzed in the alignment experiment. Model D had the best predictive performance. Its R, P, mAP, and F1 score were 0.964, 0.961, 0.961, and 0.963, respectively, when the area of overlap/area of union was at 0.3. In the lumen alignment experiment, the mean degrees of adjustment for yaw and pitch in 160 trials were 21.70° and 13.78°, respectively. Mean alignment control time was 0.902 s. Finally, we compared the cecal intubation time between semi-automated and manual navigation in 20 trials. The average cecal intubation time of manual navigation and semi-automated navigation were 9 min 28.41 s and 7 min 23.61 s, respectively. The automatic lumen detection model, which was trained using a deep learning algorithm, demonstrated high performance in each validation index.