Computer-aided diagnosis system using only white-light endoscopy for the prediction of invasion depth in colorectal cancer

The Role of Artificial Intelligence for Advanced Endoscopy

Artificial intelligence (AI) application in gastroenterology has grown in the last decade and continues to evolve very rapidly. Early promising results have opened the door to explore its potential application to advanced endoscopy (AE). The aim of this review is to discuss the current state of the art and future directions of AI in AE. Current evidence suggests that AI-assisted endoscopic ultrasound models can be used in clinical practice to distinguish between benign and malignant pancreatic diseases with excellent results. AI-assisted endoscopic retrograde cholangiopancreatography models could also be useful in identifying the papilla, predicting difficult cannulation, and differentiating between benign and malignant strictures.

Colon Cancer Screening, Surveillance, and Treatment: Novel Artificial Intelligence Driving Strategies in the Management of Colon Lesions

Colonoscopy, a crucial procedure for detecting and removing colorectal polyps, has seen transformative advancements through the integration of artificial intelligence, specifically in computer-aided detection (CADe) and diagnosis (CADx). These tools enhance real-time detection and characterization of lesions, potentially reducing human error, and standardizing the quality of colonoscopy across endoscopists. CADe has proven effective in increasing adenoma detection rate, potentially reducing long-term colorectal cancer incidence. However, CADe's benefits are accompanied by challenges, such as potentially longer procedure times, increased non-neoplastic polyp resections, and a higher surveillance burden. CADx, although promising in differentiating neoplastic and non-neoplastic diminutive polyps, encounters limitations in accuracy, particularly in the proximal colon. Real-world data also revealed gaps between trial efficacy and practical outcomes, emphasizing the need for further research in uncontrolled settings. Moreover, CADx limited specificity and binary output underscore the necessity for explainable artificial intelligence to gain endoscopists' trust. This review aimed to explore the benefits, harms, and limitations of artificial intelligence for colon cancer screening, surveillance, and treatment focusing on CADe and CADx systems for lesion detection and characterization, respectively, while addressing challenges in integrating these technologies into clinical practice.

Current Status of Artificial Intelligence Use in Colonoscopy

BACKGROUND

Artificial intelligence (AI) has significantly impacted medical imaging, particularly in gastrointestinal endoscopy. Computer-aided detection and diagnosis systems (CADe and CADx) are thought to enhance the quality of colonoscopy procedures.

SUMMARY

Colonoscopy is essential for colorectal cancer screening but often misses a significant percentage of adenomas. AI-assisted systems employing deep learning offer improved detection and differentiation of colorectal polyps, potentially increasing adenoma detection rates by 8%-10%. The main benefit of CADe is in detecting small adenomas, whereas it has a limited impact on advanced neoplasm detection. Recent advancements include real-time CADe systems and CADx for histopathological predictions, aiding in the differentiation of neoplastic and nonneoplastic lesions. Biases such as the Hawthorne effect and potential overdiagnosis necessitate large-scale clinical trials to validate the long-term benefits of AI. Additionally, novel concepts such as computer-aided quality improvement systems are emerging to address limitations facing current CADe systems.

KEY MESSAGES

Despite the potential of AI for enhancing colonoscopy outcomes, its effectiveness in reducing colorectal cancer incidence and mortality remains unproven. Further prospective studies are essential to establish the overall utility and clinical benefits of AI in colonoscopy.

Artificial Intelligence in Gastrointestinal Endoscopy

Recent advancements in artificial intelligence (AI) have significantly impacted the field of gastrointestinal (GI) endoscopy, with applications spanning a wide range of clinical indications. The central goals for AI in GI endoscopy are to improve endoscopic procedural performance and quality assessment, optimize patient outcomes, and reduce administrative burden. Despite early progress, such as Food and Drug Administration approval of the first computer-aided polyp detection system in 2021, there are numerous important challenges to be faced on the path toward broader adoption of AI algorithms in clinical endoscopic practice.

In vivo evaluation of complex polyps with endoscopic optical coherence tomography and deep learning during routine colonoscopy: a feasibility study

Standard-of-care (SoC) imaging for assessing colorectal polyps during colonoscopy, based on white-light colonoscopy (WLC) and narrow-band imaging (NBI), does not have sufficient accuracy to assess the invasion depth of complex polyps non-invasively during colonoscopy. We aimed to evaluate the feasibility of a custom endoscopic optical coherence tomography (OCT) probe for assessing colorectal polyps during routine colonoscopy. Patients referred for endoscopic treatment of large colorectal polyps were enrolled in this pilot clinical study, which used a side-viewing OCT catheter developed for use with an adult colonoscope. OCT images of polyps were captured during colonoscopy immediately before SoC treatment. A deep learning model was trained to differentiate benign from deeply invasive lesions for real-time diagnosis. 35 polyps from 32 patients were included. OCT imaging added on average 3:40 min (range 1:54-8:20) to the total procedure time. No complications due to OCT were observed. OCT revealed distinct subsurface tissue structures that correlated with histological findings, including tubular adenoma (n = 20), tubulovillous adenoma (n = 10), sessile serrated polyps (n = 3), and invasive cancer (n = 2). The deep learning model achieved an area under the receiver operating characteristic curve (AUROC) of 0.984 (95%CI 0.972-0.996) and Cohen's kappa of 0.845 (95%CI 0.774-0.915) when compared to gold standard histopathology. OCT is feasible and safe for polyp assessment during routine colonoscopy. When combined with deep learning, OCT offers clinicians increase confidence in identifying deeply invasive cancers, potentially improving clinical decision-making. Compared to previous studies, ours offers a nuanced comparison between not just benign and malignant lesions, but across multiple histological subtypes of polyps.

Differences in regions of interest to identify deeply invasive colorectal cancers: Computer-aided diagnosis vs expert endoscopists

Background and study aims Diagnostic performance of a computer-aided diagnosis (CAD) system for deep submucosally invasive (T1b) colorectal cancer was excellent, but the "regions of interest" (ROI) within images are not obvious. Class activation mapping (CAM) enables identification of the ROI that CAD utilizes for diagnosis. The purpose of this study was a quantitative investigation of the difference between CAD and endoscopists. Patients and methods Endoscopic images collected for validation of a previous study were used, including histologically proven T1b colorectal cancers (n = 82; morphology: flat 36, polypoid 46; median maximum diameter 20 mm, interquartile range 15-25 mm; histological subtype: papillary 5, well 51, moderate 24, poor 2; location: proximal colon 26, distal colon 27, rectum 29). Application of CAM was limited to one white light endoscopic image (per lesion) to demonstrate findings of T1b cancers. The CAM images were generated from the weights of the previously fine-tuned ResNet50. Two expert endoscopists depicted the ROI in identical images. Concordance of the ROI was rated by intersection over union (IoU) analysis. Results Pixel counts of ROIs were significantly lower using 165K[x103] [108K-227K] than by endoscopists (300K [208K-440K]; P < 0.0001) and median [interquartile] of the IoU was 0.198 [0.024-0.349]. IoU was significantly higher in correctly identified lesions (n = 54, 0.213 [0.116-0.364]) than incorrect ones (n=28, 0.070 [0.000-0.2750, P = 0.033). Concusions IoU was larger in correctly diagnosed T1b colorectal cancers. Optimal annotation of the ROI may be the key to improving diagnostic sensitivity of CAD for T1b colorectal cancers.

Improving the endoscopic recognition of early colorectal carcinoma using artificial intelligence: current evidence and future directions

Background and study aims Artificial intelligence (AI) has great potential to improve endoscopic recognition of early stage colorectal carcinoma (CRC). This scoping review aimed to summarize current evidence on this topic, provide an overview of the methodologies currently used, and guide future research. Methods A systematic search was performed following the PRISMA-Scr guideline. PubMed (including Medline), Scopus, Embase, IEEE Xplore, and ACM Digital Library were searched up to January 2024. Studies were eligible for inclusion when using AI for distinguishing CRC from colorectal polyps on endoscopic imaging, using histopathology as gold standard, reporting sensitivity, specificity, or accuracy as outcomes. Results Of 5024 screened articles, 26 were included. Computer-aided diagnosis (CADx) system classification categories ranged from two categories, such as lesions suitable or unsuitable for endoscopic resection, to five categories, such as hyperplastic polyp, sessile serrated lesion, adenoma, cancer, and other. The number of images used in testing databases varied from 69 to 84,585. Diagnostic performances were divergent, with sensitivities varying from 55.0% to 99.2%, specificities from 67.5% to 100% and accuracies from 74.4% to 94.4%. Conclusions This review highlights that using AI to improve endoscopic recognition of early stage CRC is an upcoming research field. We introduced a suggestions list of essential subjects to report in research regarding the development of endoscopy CADx systems, aiming to facilitate more complete reporting and better comparability between studies. There is a knowledge gap regarding real-time CADx system performance during multicenter external validation. Future research should focus on development of CADx systems that can differentiate CRC from premalignant lesions, while providing an indication of invasion depth.

Role of the artificial intelligence in the management of T1 colorectal cancer

Approximately 10% of submucosal invasive (T1) colorectal cancers demonstrate extraintestinal lymph node metastasis, necessitating surgical intervention with lymph node dissection. The ability to identify T1b (submucosal invasion depth ≥ 1000 µm) as a risk factor for lymph node metastasis via pre-treatment endoscopy is crucial in guiding treatment strategies. Accurately distinguishing T1b from T1a (submucosal invasion depth < 1000 µm) or dysplasia remains a significant challenge for artificial intelligence (AI) systems, which require high and consistent diagnostic capabilities. Moreover, as endoscopic therapies like endoscopic full-thickness resection and endoscopic intermuscular dissection evolve, and the focus on reducing unnecessary surgeries intensifies, the initial management of T1 colorectal cancers via endoscopic treatment is anticipated to increase. Consequently, the development of highly accurate and reliable AI systems is essential, not only for pre-treatment depth assessment but also for post-treatment risk stratification of lymph node metastasis. While such AI diagnostic systems are still under development, significant advancements are expected in the near future to improve decision-making in T1 colorectal cancer management.

Precision endoscopy in colorectal polyps' characterization and planning of endoscopic therapy

Precision endoscopy in the management of colorectal polyps and early colorectal cancer has emerged as the standard of care. It includes optical characterization of polyps and estimation of submucosal invasion depth of large nonpedunculated colorectal polyps to select the appropriate endoscopic resection modality. Over time, several imaging modalities have been implemented in endoscopic practice to improve optical performance. Among these, image-enhanced endoscopy systems and magnification endoscopy represent now well-established tools. New advanced technologies, such as endocytoscopy and confocal laser endomicroscopy, have recently shown promising results in predicting the histology of colorectal polyps. In recent years, artificial intelligence has continued to enhance endoscopic performance in the characterization of colorectal polyps, overcoming the limitations of other imaging modes. In this review we retrace the path of precision endoscopy, analyzing the yield of various endoscopic imaging techniques in personalizing management of colorectal polyps and early colorectal cancer.

Use of artificial intelligence in the management of T1 colorectal cancer: a new tool in the arsenal or is deep learning out of its depth?

The field of artificial intelligence is rapidly evolving, and there has been an interest in its use to predict the risk of lymph node metastasis in T1 colorectal cancer. Accurately predicting lymph node invasion may result in fewer patients undergoing unnecessary surgeries; conversely, inadequate assessments will result in suboptimal oncological outcomes. This narrative review aims to summarize the current literature on deep learning for predicting the probability of lymph node metastasis in T1 colorectal cancer, highlighting areas of potential application and barriers that may limit its generalizability and clinical utility.

Time to change? Present and prospects of hemorrhoidal classification

As a common benign anal condition, the high incidence and recurrence of hemorrhoids pose challenges for both patients and doctors. The classification of hemorrhoids plays a crucial role in assessing, diagnosing, and treating the condition. By using appropriate classification and corresponding treatment strategies, we can achieve higher cure rates and lower recurrence rates of hemorrhoids. Since the introduction of the Miles classification in 1919, various classifications have been developed, which include objective classifications based on anatomical or instrumental assessment and subjective classifications based on symptoms and patient sensations. These classifications aim to accurately evaluate the condition. In this study, we discuss the evaluation values of each classification in terms of their advantages, disadvantages, treatment relevance, reproducibility, practicality, and assessment value. We also analyze the significant and essential factors, principles of use, and components of assessment indicators of hemorrhoidal classification. This study proposes several strategies to address the limitations of current hemorrhoidal assessment methods. All these will provide a reference for the development regarding the assessment and classification of hemorrhoids and clinical diagnosis and management of hemorrhoids.

pT1 colorectal cancer: A treatment dilemma

The implementation of population screening programs for colorectal cancer (CRC) has led to a considerable increase in the prevalence pT1-CRC originating on polyps amenable by local treatments. However, a high proportion of patients are referred for unnecessary oncological surgeries without a clear benefit in terms of survival. Selecting the appropriate endoscopic resection technique in the moment of diagnosis becomes crucial to provide the best treatment alternative to each individual polyp and patient. For this, it is imperative to increase the optical diagnostic skill for differentiating pT1-CRCs and decide the appropriate initial therapy. En bloc resection is crucial to obtain an adequate histological specimen that might allow organ preserving therapeutic management. In this review, we address key challenges in T1 CRC management, explore the efficacy and safety of the available diagnostic and therapeutic approaches, and shed light on upcoming advances in the field.

Computer-aided diagnosis in predicting the invasion depth of early colorectal cancer: a systematic review and meta-analysis of diagnostic test accuracy

BACKGROUND

Endoscopic resection (ER) is widely applied to treat early colorectal cancer (CRC). Predicting the invasion depth of early CRC is critical in determining treatment strategies. The use of computer-aided diagnosis (CAD) algorithms could theoretically make accurate and objective predictions regarding the suitability of lesions for ER indication based on invasion depth. This study aimed to assess diagnostic test accuracy of CAD algorithms in predicting the invasion depth of early CRC and to compare the performance between the CAD algorithms and endoscopists.

METHODS

Multiple databases were searched until June 30, 2022 for studies that evaluated the diagnostic performance of CAD algorithms for invasion depth of CRC. Meta-analysis of diagnostic test accuracy using a bivariate mixed-effects model was performed.

RESULTS

Ten studies consisting of 13 arms (13,918 images from 1472 lesions) were included. Due to significant heterogeneity, studies were stratified into Japan/Korea-based or China-based studies. For the former, the area under the curve (AUC), sensitivity, and specificity of the CAD algorithms were 0.89 (95% CI 0.86-0.91), 62% (95% CI 50-72%), and 96% (95% CI 93-98%), respectively. For the latter, AUC, sensitivity, and specificity were 0.94 (95% CI 0.92-0.96), 88% (95% CI 78-94%), and 88% (95% CI 80-93%), respectively. The performance of the CAD algorithms in Japan/Korea-based studies was not significantly different from that of all endoscopists (0.88 vs. 0.91, P = 0.10) but was inferior to that of expert endoscopists (0.88 vs. 0.92, P = 0.03). The performance of the CAD algorithms in China-based studies was better than that of all endoscopists (0.94 vs. 0.90, P = 0.01).

CONCLUSION

The CAD algorithms showed comparable accuracy for prediction of invasion depth of early CRC compared to all endoscopists, which was still lower than expert endoscopists in diagnostic accuracy; more improvements should be achieved before it can be extensively applied to clinical practice.

Computer-aided diagnosis of early-stage colorectal cancer using nonmagnified endoscopic white-light images (with videos)

BACKGROUND AND AIMS

Differentiation of colorectal cancers (CRCs) with deep submucosal invasion (T1b) from CRCs with superficial invasion (T1a) or no invasion (Tis) is not straightforward. This study aimed to develop a computer-aided diagnosis (CADx) system to establish the diagnosis of early-stage cancers using nonmagnified endoscopic white-light images alone.

METHODS

From 5108 images, 1513 lesions (Tis, 1074; T1a, 145; T1b, 294) were collected from 1470 patients at 10 academic hospitals and assigned to training and testing datasets (3:1). The ResNet-50 network was used as the backbone to extract features from images. Oversampling and focal loss were used to compensate class imbalance of the invasive stage. Diagnostic performance was assessed using the testing dataset including 403 CRCs with 1392 images. Two experts and 2 trainees read the identical testing dataset.

RESULTS

At a 90% cutoff for the per-lesion score, CADx showed the highest specificity of 94.4% (95% confidence interval [CI], 91.3-96.6), with 59.8% (95% CI, 48.3-70.4) sensitivity and 87.3% (95% CI, 83.7-90.4) accuracy. The area under the characteristic curve was 85.1% (95% CI, 79.9-90.4) for CADx, 88.2% (95% CI, 83.7-92.8) for expert 1, 85.9% (95% CI, 80.9-90.9) for expert 2, 77.0% (95% CI, 71.5-82.4) for trainee 1 (vs CADx; P = .0076), and 66.2% (95% CI, 60.6-71.9) for trainee 2 (P < .0001). The function was also confirmed on 9 short videos.

CONCLUSIONS

A CADx system developed with endoscopic white-light images showed excellent per-lesion specificity and accuracy for T1b lesion diagnosis, equivalent to experts and superior to trainees. (Clinical trial registration number: UMIN000037053.).

Artificial Intelligence Applied to Colonoscopy: Is It Time to Take a Step Forward?

Growing evidence indicates that artificial intelligence (AI) applied to medicine is here to stay. In gastroenterology, AI computer vision applications have been stated as a research priority. The two main AI system categories are computer-aided polyp detection (CADe) and computer-assisted diagnosis (CADx). However, other fields of expansion are those related to colonoscopy quality, such as methods to objectively assess colon cleansing during the colonoscopy, as well as devices to automatically predict and improve bowel cleansing before the examination, predict deep submucosal invasion, obtain a reliable measurement of colorectal polyps and accurately locate colorectal lesions in the colon. Although growing evidence indicates that AI systems could improve some of these quality metrics, there are concerns regarding cost-effectiveness, and large and multicentric randomized studies with strong outcomes, such as post-colonoscopy colorectal cancer incidence and mortality, are lacking. The integration of all these tasks into one quality-improvement device could facilitate the incorporation of AI systems in clinical practice. In this manuscript, the current status of the role of AI in colonoscopy is reviewed, as well as its current applications, drawbacks and areas for improvement.

Benefits and challenges in implementation of artificial intelligence in colonoscopy: World Endoscopy Organization position statement

The number of artificial intelligence (AI) tools for colonoscopy on the market is increasing with supporting clinical evidence. Nevertheless, their implementation is not going smoothly for a variety of reasons, including lack of data on clinical benefits and cost-effectiveness, lack of trustworthy guidelines, uncertain indications, and cost for implementation. To address this issue and better guide practitioners, the World Endoscopy Organization (WEO) has provided its perspective about the status of AI in colonoscopy as the position statement. WEO Position Statement: Statement 1.1: Computer-aided detection (CADe) for colorectal polyps is likely to improve colonoscopy effectiveness by reducing adenoma miss rates and thus increase adenoma detection; Statement 1.2: In the short term, use of CADe is likely to increase health-care costs by detecting more adenomas; Statement 1.3: In the long term, the increased cost by CADe could be balanced by savings in costs related to cancer treatment (surgery, chemotherapy, palliative care) due to CADe-related cancer prevention; Statement 1.4: Health-care delivery systems and authorities should evaluate the cost-effectiveness of CADe to support its use in clinical practice; Statement 2.1: Computer-aided diagnosis (CADx) for diminutive polyps (≤5 mm), when it has sufficient accuracy, is expected to reduce health-care costs by reducing polypectomies, pathological examinations, or both; Statement 2.2: Health-care delivery systems and authorities should evaluate the cost-effectiveness of CADx to support its use in clinical practice; Statement 3: We recommend that a broad range of high-quality cost-effectiveness research should be undertaken to understand whether AI implementation benefits populations and societies in different health-care systems.

Real-time automated diagnosis of colorectal cancer invasion depth using a deep learning model with multimodal data (with video)

BACKGROUND AND AIMS

The optical diagnosis of colorectal cancer (CRC) invasion depth with white light (WL) and image-enhanced endoscopy (IEE) remains challenging. We aimed to construct and validate a 2-modal deep learning-based system, incorporated with both WL and IEE images (named Endo-CRC) in estimating the invasion depth of CRC.

METHODS

Samples were retrospectively obtained from 3 hospitals in China. We combined WL and IEE images into image pairs. Altogether, 337,278 image pairs from 268 noninvasive and superficial CRC and 181,934 image pairs from 82 deep CRC were used for training. A total of 296,644 and 4528 image pairs were used for internal and external tests and for comparison with endoscopists. Thirty-five videos were used for evaluating the real-time performance of the Endo-CRC system. Two deep learning models, solely using either WL (model W) or IEE images (model I), were constructed to compare with Endo-CRC.

RESULTS

The accuracies of Endo-CRC in internal image tests with and without advanced CRC were 91.61% and 93.78%, respectively, and 88.65% in the external test, which did not include advanced CRC. In an endoscopist-machine competition, Endo-CRC achieved an expert comparable accuracy of 88.11% and the highest sensitivity compared with all endoscopists. In a video test, Endo-CRC achieved an accuracy of 100.00%. Compared with model W and model I, Endo-CRC had a higher accuracy (per image pair: 91.61% vs 88.27% compared with model I and 91.61% vs 81.32% compared with model W).

CONCLUSIONS

The Endo-CRC system has great potential for assisting in CRC invasion depth diagnosis and may be well applied in clinical practice.

Artificial Intelligence in Endoscopy

Artificial intelligence (AI) is rapidly developing in various medical fields, and there is an increase in research performed in the field of gastrointestinal (GI) endoscopy. In particular, the advent of convolutional neural network, which is a class of deep learning method, has the potential to revolutionize the field of GI endoscopy, including esophagogastroduodenoscopy (EGD), capsule endoscopy (CE), and colonoscopy. A total of 149 original articles pertaining to AI (27 articles in esophagus, 30 articles in stomach, 29 articles in CE, and 63 articles in colon) were identified in this review. The main focuses of AI in EGD are cancer detection, identifying the depth of cancer invasion, prediction of pathological diagnosis, and prediction of Helicobacter pylori infection. In the field of CE, automated detection of bleeding sites, ulcers, tumors, and various small bowel diseases is being investigated. AI in colonoscopy has advanced with several patient-based prospective studies being conducted on the automated detection and classification of colon polyps. Furthermore, research on inflammatory bowel disease has also been recently reported. Most studies of AI in the field of GI endoscopy are still in the preclinical stages because of the retrospective design using still images. Video-based prospective studies are needed to advance the field. However, AI will continue to develop and be used in daily clinical practice in the near future. In this review, we have highlighted the published literature along with providing current status and insights into the future of AI in GI endoscopy.

Artificial intelligence-enhanced white-light colonoscopy with attention guidance predicts colorectal cancer invasion depth

BACKGROUND AND AIMS

Endoscopic submucosal dissection (ESD) and EMR are applied in treating superficial colorectal neoplasms but are contraindicated by deeply invasive colorectal cancer (CRC). The invasion depth of neoplasms can be examined by an automated artificial intelligence (AI) system to determine the applicability of ESD and EMR.

METHODS

A deep convolutional neural network with a tumor localization branch to guide invasion depth classification was constructed on the GoogLeNet architecture. The model was trained using 7734 nonmagnified white-light colonoscopy (WLC) images supplemented by image augmentation from 657 lesions labeled with histopathologic analysis of invasion depth. An independent testing dataset consisting of 1634 WLC images from 156 lesions was used to validate the model.

RESULTS

For predicting noninvasive and superficially invasive neoplasms, the model achieved an overall accuracy of 91.1% (95% confidence interval [CI], 89.6%-92.4%), with 91.2% sensitivity (95% CI, 88.8%-93.3%) and 91.0% specificity (95% CI, 89.0%-92.7%) at an optimal cutoff of .41 and the area under the receiver operating characteristic (AUROC) curve of .970 (95% CI, .962-.978). Inclusion of the advanced CRC data significantly increased the sensitivity in differentiating superficial neoplasms from deeply invasive early CRC to 65.3% (95% CI, 61.9%-68.8%) with an AUROC curve of .729 (95% CI, .699-.759), similar to experienced endoscopists (.691; 95% CI, .624-.758).

CONCLUSIONS

We have developed an AI-enhanced attention-guided WLC system that differentiates noninvasive or superficially submucosal invasive neoplasms from deeply invasive CRC with high accuracy, sensitivity, and specificity.