Application of Deep Learning Models to Improve Ulcerative Colitis Endoscopic Disease Activity Scoring Under Multiple Scoring Systems

Clinical usefulness of hypoxia imaging colonoscopy for the objective measurement of ulcerative colitis disease activity

BACKGROUND AND AIMS

Colonic mucosal hypoxia is associated with mucosal inflammation in ulcerative colitis (UC). We aimed to assess the clinical usefulness of hypoxia imaging colonoscopy for the evaluation of clinical, endoscopic, and histologic disease activities of UC.

METHODS

This retrospective cohort study comprised 100 consecutive patients with UC who underwent hypoxia imaging colonoscopy between September 2022 and September 2023 at the University of Tsukuba Hospital. Colonic tissue oxygen saturation (StO2) was measured at the biopsy sites, and StO2 values between different disease activities were compared. Receiver-operating characteristic (ROC) analysis was used to calculate the area under the ROC curve (AUROC).

RESULTS

A significant correlation was identified between rectal StO2 and the Simple Clinical Colitis Activity Index, with moderate accuracy to predict bowel urgency at a 40.5% cutoff (AUROC, .74; 95% confidence interval [CI], .62-.87). Our analysis of 490 images showed median StO2 values for Mayo endoscopic subscores 0, 1, 2, and 3 as 52% (interquartile range [IQR], 48%-56%), 47% (IQR, 43%-52%), 42% (IQR, 38.8%-47%), and 39.5% (IQR, 37.3%-41.8%), respectively. Differences for all pairs were significant. Median StO2 was 49% (IQR, 44%-54%) for Geboes scores 0 to 2, significantly higher than histologically active disease (Geboes score ≥3). At a colonic StO2 cutoff of 45.5%, AUROCs for endoscopically and histologically active diseases were .79 (95% CI, .74-.84) and .72 (95% CI, .66-.77).

CONCLUSIONS

StO2 obtained by hypoxia imaging colonoscopy is useful for assessing clinical, endoscopic, and histologic activities of UC, suggesting that StO2 may be a novel and objective endoscopic measurement.

Artificial intelligence and endo-histo-omics: new dimensions of precision endoscopy and histology in inflammatory bowel disease

Integrating artificial intelligence into inflammatory bowel disease (IBD) has the potential to revolutionise clinical practice and research. Artificial intelligence harnesses advanced algorithms to deliver accurate assessments of IBD endoscopy and histology, offering precise evaluations of disease activity, standardised scoring, and outcome prediction. Furthermore, artificial intelligence offers the potential for a holistic endo-histo-omics approach by interlacing and harmonising endoscopy, histology, and omics data towards precision medicine. The emerging applications of artificial intelligence could pave the way for personalised medicine in IBD, offering patient stratification for the most beneficial therapy with minimal risk. Although artificial intelligence holds promise, challenges remain, including data quality, standardisation, reproducibility, scarcity of randomised controlled trials, clinical implementation, ethical concerns, legal liability, and regulatory issues. The development of standardised guidelines and interdisciplinary collaboration, including policy makers and regulatory agencies, is crucial for addressing these challenges and advancing artificial intelligence in IBD clinical practice and trials.

Deep learning-based lesion detection and severity grading of small-bowel Crohn's disease ulcers on double-balloon endoscopy images

BACKGROUND AND AIMS

Double-balloon endoscopy (DBE) is widely used in diagnosing small-bowel Crohn's disease (CD). However, CD misdiagnosis frequently occurs if inexperienced endoscopists cannot accurately detect the lesions. The CD evaluation may also be inaccurate owing to the subjectivity of endoscopists. This study aimed to use artificial intelligence (AI) to accurately detect and objectively assess small-bowel CD for more refined disease management.

METHODS

We collected 28,155 small-bowel DBE images from 628 patients from January 2018 to December 2022. Four expert gastroenterologists labeled the images, and at least 2 endoscopists made the final decision with agreement. A state-of-the-art deep learning model, EfficientNet-b5, was trained to detect CD lesions and evaluate CD ulcers. The detection included lesions of ulcer, noninflammatory stenosis, and inflammatory stenosis. Ulcer grading included ulcerated surface, ulcer size, and ulcer depth. A comparison of AI model performance with endoscopists was performed.

RESULTS

The EfficientNet-b5 achieved high accuracies of 96.3% (95% confidence interval [CI], 95.7%-96.7%), 95.7% (95% CI, 95.1%-96.2%), and 96.7% (95% CI, 96.2%-97.2%) for the detection of ulcers, noninflammatory stenosis, and inflammatory stenosis, respectively. In ulcer grading, the EfficientNet-b5 exhibited average accuracies of 87.3% (95% CI, 84.6%-89.6%) for grading the ulcerated surface, 87.8% (95% CI, 85.0%-90.2%) for grading the size of ulcers, and 85.2% (95% CI, 83.2%-87.0%) for ulcer depth assessment.

CONCLUSIONS

The EfficientNet-b5 achieved high accuracy in detecting CD lesions and grading CD ulcers. The AI model can provide expert-level accuracy and objective evaluation of small-bowel CD to optimize the clinical treatment plans.

Present and future of endoscopy precision for inflammatory bowel disease

Several advanced imaging techniques are now available for endoscopists managing inflammatory bowel disease (IBD) patients. These tools, including dye-based and virtual chromoendoscopy, probe-based confocal laser endomicroscopy and endocytoscopy, are increasingly innovative applications in clinical practice. They allow for a more in-depth and refined evaluation of the mucosal and vascular bowel surface, getting closer to histology. They have demonstrated a remarkable ability in assessing intestinal inflammation, histologic remission, and predicting relapse and favorable long-term outcomes. In addition, the future application of molecular endoscopy to predict biological drug responses has yielded preliminary but encouraging results. Furthermore, these techniques are crucial in detecting and characterizing IBD-related dysplasia, assisting endoscopic mucosal resection and submucosal dissection towards a surgery-sparing approach. Artificial intelligence (AI) holds great potential in this promising landscape, as it can provide an objective and reproducible assessment of inflammation and dysplasia. Moreover, it can improve the prediction of outcomes and aid in subsequent therapeutic decision-making. This review aims to summarize the promising role of state-of-the-art advanced endoscopic techniques and related AI-enabled models for managing IBD, paving the way for precision medicine.

Using Computer Vision to Improve Endoscopic Disease Quantification in Therapeutic Clinical Trials of Ulcerative Colitis

BACKGROUND & AIMS

Endoscopic assessment of ulcerative colitis (UC) typically reports only the maximum severity observed. Computer vision methods may better quantify mucosal injury detail, which varies among patients.

METHODS

Endoscopic video from the UNIFI clinical trial (A Study to Evaluate the Safety and Efficacy of Ustekinumab Induction and Maintenance Therapy in Participants With Moderately to Severely Active Ulcerative Colitis) comparing ustekinumab and placebo for UC were processed in a computer vision analysis that spatially mapped Mayo Endoscopic Score (MES) to generate the Cumulative Disease Score (CDS). CDS was compared with the MES for differentiating ustekinumab vs placebo treatment response and agreement with symptomatic remission at week 44. Statistical power, effect, and estimated sample sizes for detecting endoscopic differences between treatments were calculated using both CDS and MES measures. Endoscopic video from a separate phase 2 clinical trial replication cohort was performed for validation of CDS performance.

RESULTS

Among 748 induction and 348 maintenance patients, CDS was lower in ustekinumab vs placebo users at week 8 (141.9 vs 184.3; P < .0001) and week 44 (78.2 vs 151.5; P < .0001). CDS was correlated with the MES (P < .0001) and all clinical components of the partial Mayo score (P < .0001). Stratification by pretreatment CDS revealed ustekinumab was more effective than placebo (P < .0001) with increasing effect in severe vs mild disease (-85.0 vs -55.4; P < .0001). Compared with the MES, CDS was more sensitive to change, requiring 50% fewer participants to demonstrate endoscopic differences between ustekinumab and placebo (Hedges' g = 0.743 vs 0.460). CDS performance in the JAK-UC replication cohort was similar to UNIFI.

CONCLUSIONS

As an automated and quantitative measure of global endoscopic disease severity, the CDS offers artificial intelligence enhancement of traditional MES capability to better evaluate UC in clinical trials and potentially practice.

Histologic assessments in ulcerative colitis: the evidence behind a new endpoint in clinical trials

INTRODUCTION

Treatment goals for ulcerative colitis (UC) are evolving from the achievement of clinical remission to more rigorous goals defined by endoscopic and histologic healing. Achievement of deeper remission targets aims to reduce the risk of colectomy, hospitalizations, and colorectal cancer.

AREAS COVERED

This review covers histologic assessments, histologic remission as a clinical trial endpoint, and the association between histologic disease activity and clinical outcomes. Future directions are also discussed, including the use of advanced imaging and artificial intelligence technologies, as well as potential future treatment targets beyond histologic remission.

EXPERT OPINION

Histologic assessments are used for their sensitivity in measuring mucosal inflammatory changes in UC. Due to correlation with disease activity, histologic assessments may support clinical decision-making regarding treatment decisions as such assessments can be associated with rates of clinical relapse, hospitalization, colectomy, and neoplasia. While histologic remission is limited by varying definitions and multiple histologic indices, work is ongoing to create a consensus on the use of histologic assessments in clinical trials. As research advances, aspirational targets beyond histologic remission, such as molecular healing and disease clearance, are being explored.

Application of deep learning in the diagnosis and evaluation of ulcerative colitis disease severity

Background

Achieving endoscopic and histological remission is a critical treatment objective in ulcerative colitis (UC). Nevertheless, interobserver variability can significantly impact overall assessment performance.

Objectives

We aimed to develop a deep learning algorithm for the real-time and objective evaluation of endoscopic disease activity and prediction of histological remission in UC.

Design

This is a retrospective diagnostic study.

Methods

Two convolutional neural network (CNN) models were constructed and trained using 12,257 endoscopic images and biopsy results sourced from 1124 UC patients who underwent colonoscopy at a single center from January 2018 to December 2022. Mayo Endoscopy Subscore (MES) and UC Endoscopic Index of Severity Score (UCEIS) assessments were conducted by two experienced and independent reviewers. Model performance was evaluated in terms of accuracy, sensitivity, and positive predictive value. The output of the CNN models was also compared with the corresponding histological results to assess histological remission prediction performance.

Results

The MES-CNN model achieved 97.04% accuracy in diagnosing endoscopic remission of UC, while the MES-CNN and UCEIS-CNN models achieved 90.15% and 85.29% accuracy, respectively, in evaluating endoscopic severity of UC. For predicting histological remission, the CNN models achieved accuracy and kappa values of 91.28% and 0.826, respectively, attaining higher accuracy than human endoscopists (87.69%).

Conclusion

The proposed artificial intelligence model, based on MES and UCEIS evaluations from expert gastroenterologists, offered precise assessment of inflammation in UC endoscopic images and reliably predicted histological remission.

Precision medicine in monogenic inflammatory bowel disease: proposed mIBD REPORT standards

Owing to advances in genomics that enable differentiation of molecular aetiologies, patients with monogenic inflammatory bowel disease (mIBD) potentially have access to genotype-guided precision medicine. In this Expert Recommendation, we review the therapeutic research landscape of mIBD, the reported response to therapies, the medication-related risks and systematic bias in reporting. The mIBD field is characterized by the absence of randomized controlled trials and is dominated by retrospective observational data based on case series and case reports. More than 25 off-label therapeutics (including small-molecule inhibitors and biologics) as well as cellular therapies (including haematopoietic stem cell transplantation and gene therapy) have been reported. Heterogeneous reporting of outcomes impedes the generation of robust therapeutic evidence as the basis for clinical decision making in mIBD. We discuss therapeutic goals in mIBD and recommend standardized reporting (mIBD REPORT (monogenic Inflammatory Bowel Disease Report Extended Phenotype and Outcome of Treatments) standards) to stratify patients according to a genetic diagnosis and phenotype, to assess treatment effects and to record safety signals. Implementation of these pragmatic standards should help clinicians to assess the therapy responses of individual patients in clinical practice and improve comparability between observational retrospective studies and controlled prospective trials, supporting future meta-analysis.

A systematic review and meta-analysis of artificial intelligence-diagnosed endoscopic remission in ulcerative colitis

Endoscopic remission is an important therapeutic goal in ulcerative colitis (UC). The Ulcerative Colitis Endoscopic Index of Severity (UCEIS) and Mayo Endoscopic Score (MES) are the commonly used endoscopic scoring criteria. This systematic review and meta-analysis aimed to evaluate the accuracy of artificial intelligence (AI) in diagnosing endoscopic remission in UC. We also performed a meta-analysis of each of the four endoscopic remission criteria (UCEIS = 0, MES = 0, UCEIS = <1, MES = <1). Eighteen studies involving 13,687 patients were included. The combined sensitivity and specificity of AI for diagnosing endoscopic remission in UC was 87% (95% confidence interval [CI]:81-92%) and 92% (95% CI: 89-94%), respectively. The area under the curve (AUC) was 0.96 (95% CI: 0.94-0.97). The results showed that the AI model performed well regardless of which criteria were used to define endoscopic remission of UC.

Different levels of healing in inflammatory bowel diseases: mucosal, histological, transmural, barrier and complete healing

Mucosal healing on endoscopy has emerged as a key prognostic parameter in the management of patients with IBD (Crohn's disease, ulcerative colitis/UC) and can predict sustained clinical remission and resection-free survival. The structural basis for this type of mucosal healing is a progressive resolution of intestinal inflammation with associated healing of ulcers and improved epithelial barrier function. However, in some cases with mucosal healing on endoscopy, evidence of histological activity in mucosal biopsies has been observed. Subsequently, in UC, a second, deeper type of mucosal healing, denoted histological healing, was defined which requires the absence of active inflammation in mucosal biopsies. Both levels of mucosal healing should be considered as initial events in the resolution of gut inflammation in IBD rather than as indicators of complete transmural healing. In this review, the effects of anti-inflammatory, biological or immunosuppressive agents as well as small molecules on mucosal healing in clinical studies are highlighted. In addition, we focus on the implications of mucosal healing for clinical management of patients with IBD. Moreover, emerging techniques for the analysis of mucosal healing as well as potentially deeper levels of mucosal healing such as transmural healing and functional barrier healing of the mucosa are discussed. Although none of these new levels of healing indicate a definitive cure of the diseases, they make an important contribution to the assessment of patients' prognosis. The ultimate level of healing in IBD would be a resolution of all aspects of intestinal and extraintestinal inflammation (complete healing).

Artificial intelligence in endoscopy: Overview, applications, and future directions

Since the emergence of artificial intelligence (AI) in medicine, endoscopy applications in gastroenterology have been at the forefront of innovations. The ever-increasing number of studies necessitates the need to organize and classify applications in a useful way. Separating AI capabilities by computer aided detection (CADe), diagnosis (CADx), and quality assessment (CADq) allows for a systematic evaluation of each application. CADe studies have shown promise in accurate detection of esophageal, gastric and colonic neoplasia as well as identifying sources of bleeding and Crohn's disease in the small bowel. While more advanced CADx applications employ optical biopsies to give further information to characterize neoplasia and grade inflammatory disease, diverse CADq applications ensure quality and increase the efficiency of procedures. Future applications show promise in advanced therapeutic modalities and integrated systems that provide multimodal capabilities. AI is set to revolutionize clinical decision making and performance of endoscopy.

New Technologies in Digestive Endoscopy for Ulcerative Colitis Patients

Ulcerative colitis (UC) is a chronic inflammatory bowel disease primarily affecting the colon and rectum. Endoscopy plays a crucial role in the diagnosis and management of UC. Recent advancements in endoscopic technology, including chromoendoscopy, confocal laser endomicroscopy, endocytoscopy and the use of artificial intelligence, have revolutionized the assessment and treatment of UC patients. These innovative techniques enable early detection of dysplasia and cancer, more precise characterization of disease extent and severity and more targeted biopsies, leading to improved diagnosis and disease monitoring. Furthermore, these advancements have significant implications for therapeutic decision making, empowering clinicians to carefully consider a range of treatment options, including pharmacological therapies, endoscopic interventions and surgical approaches. In this review, we provide an overview of the latest endoscopic technologies and their applications for diagnosing and monitoring UC. We also discuss their impact on treatment decision making, highlighting the potential benefits and limitations of each technique.