Automatic, computer-aided determination of endoscopic and histological inflammation in patients with mild to moderate ulcerative colitis based on red density

Evaluation in real-time use of artificial intelligence during colonoscopy to predict relapse of ulcerative colitis: a prospective study

BACKGROUND AND AIMS

The use of artificial intelligence (AI) during colonoscopy is attracting attention as an endoscopist-independent tool to predict histologic disease activity of ulcerative colitis (UC). However, no study has evaluated the real-time use of AI to directly predict clinical relapse of UC. Hence, it is unclear whether the real-time use of AI during colonoscopy helps clinicians make real-time decisions regarding treatment interventions for patients with UC. This study aimed to establish the role of real-time AI in stratifying the relapse risk of patients with UC in clinical remission.

METHODS

This open-label, prospective, cohort study was conducted in a referral center. The cohort comprised 145 consecutive patients with UC in clinical remission who underwent AI-assisted colonoscopy with a contact-microscopy function. We classified patients into either the Healing group or Active group based on the AI outputs during colonoscopy. The primary outcome measure was clinical relapse of UC (defined as a partial Mayo score >2) during 12 months of follow-up after colonoscopy.

RESULTS

Overall, 135 patients completed the 12-month follow-up after AI-assisted colonoscopy. AI-assisted colonoscopy classified 61 patients as the Healing group and 74 as the Active group. The relapse rate was significantly higher in the AI-Active group (28.4% [21/74]; 95% confidence interval, 18.5%-40.1%) than in the AI-Healing group (4.9% [3/61]; 95% confidence interval, 1.0%-13.7%; P < .001).

CONCLUSIONS

Real-time use of AI predicts the risk of clinical relapse in patients with UC in clinical remission, which helps clinicians make real-time decisions regarding treatment interventions. (Clinical trial registration number: UMIN000036650.).

Artificial intelligence enabled automated diagnosis and grading of ulcerative colitis endoscopy images

Endoscopic evaluation to reliably grade disease activity, detect complications including cancer and verification of mucosal healing are paramount in the care of patients with ulcerative colitis (UC); but this evaluation is hampered by substantial intra- and interobserver variability. Recently, artificial intelligence methodologies have been proposed to facilitate more objective, reproducible endoscopic assessment. In a first step, we compared how well several deep learning convolutional neural network architectures (CNNs) applied to a diverse subset of 8000 labeled endoscopic still images derived from HyperKvasir, the largest multi-class image and video dataset from the gastrointestinal tract available today. The HyperKvasir dataset includes 110,079 images and 374 videos and could (1) accurately distinguish UC from non-UC pathologies, and (2) inform the Mayo score of endoscopic disease severity. We grouped 851 UC images labeled with a Mayo score of 0-3, into an inactive/mild (236) and moderate/severe (604) dichotomy. Weights were initialized with ImageNet, and Grid Search was used to identify the best hyperparameters using fivefold cross-validation. The best accuracy (87.50%) and Area Under the Curve (AUC) (0.90) was achieved using the DenseNet121 architecture, compared to 72.02% and 0.50 by predicting the majority class ('no skill' model). Finally, we used Gradient-weighted Class Activation Maps (Grad-CAM) to improve visual interpretation of the model and take an explainable artificial intelligence approach (XAI).

Evolution and New Horizons of Endoscopy in Inflammatory Bowel Diseases

Endoscopy is the mainstay of inflammatory bowel disease (IBD) evaluation and the pillar of colorectal cancer surveillance. Endoscopic equipment, both hardware and software, are advancing at an incredible pace. Virtual chromoendoscopy is now widely available, allowing the detection of subtle inflammatory changes, thus reducing the gap between endoscopic and histologic assessment. The progress in the field of artificial intelligence (AI) has been remarkable, and numerous applications are now in an advanced stage of development. Computer-aided diagnosis (CAD) systems are likely to reshape most of the evaluations that are now prerogative of human endoscopists. Furthermore, sophisticated tools such as endocytoscopy and probe-based confocal laser endomicroscopy (pCLE) are enhancing our assessment of inflammation and dysplasia. Finally, pCLE combined with molecular labeling could pave the way to a new paradigm of personalized medicine. This review aims to summarize the main changes that occurred in the field of IBD endoscopy and to explore the most promising novelties.

Artificial Endoscopy and Inflammatory Bowel Disease: Welcome to the Future

Artificial intelligence (AI) is assuming an increasingly important and central role in several medical fields. Its application in endoscopy provides a powerful tool supporting human experiences in the detection, characterization, and classification of gastrointestinal lesions. Lately, the potential of AI technology has been emerging in the field of inflammatory bowel disease (IBD), where the current cornerstone is the treat-to-target strategy. A sensible and specific tool able to overcome human limitations, such as AI, could represent a great ally and guide precision medicine decisions. Here we reviewed the available literature on the endoscopic applications of AI in order to properly describe the current state-of-the-art and identify the research gaps in IBD at the dawn of 2022.

Charting a path forward for clinical research in artificial intelligence and gastroenterology

Gastroenterology has been an early leader in bridging the gap between artificial intelligence (AI) model development and clinical trial validation, and in recent years we have seen the publication of several randomized clinical trials examining the role of AI in gastroenterology. As AI applications for clinical medicine advance rapidly, there is a clear need for guidance surrounding AI-specific study design, evaluation, comparison, analysis and reporting of results. Several initiatives are in the publication or pre-publication phase including AI-specific amendments to minimum reporting guidelines for clinical trials, society task force initiatives aimed at priority use cases and research priorities, and minimum reporting guidelines that guide the reporting of clinical prediction models. In this paper, we examine applications of AI in clinical trials and discuss elements of newly published AI-specific extensions to the Consolidated Standards of Reporting Trials and Standard Protocol Items: Recommendations for Interventional Trials statements that guide clinical trial reporting and development. We then review AI applications at the pre-trial level in both endoscopy and other subfields of gastroenterology and explore areas where further guidance is needed to supplement the current guidance available at the pre-trial level.

Artificial intelligence and inflammatory bowel disease: practicalities and future prospects

Artificial intelligence (AI) is an emerging technology predicted to have significant applications in healthcare. This review highlights AI applications that impact the patient journey in inflammatory bowel disease (IBD), from genomics to endoscopic applications in disease classification, stratification and self-monitoring to risk stratification for personalised management. We discuss the practical AI applications currently in use while giving a balanced view of concerns and pitfalls and look to the future with the potential of where AI can provide significant value to the care of the patient with IBD.

Deep neural network for video colonoscopy of ulcerative colitis: a cross-sectional study

BACKGROUND

A combination of endoscopic and histological evaluation is important in the management of patients with ulcerative colitis. We aimed to adapt our previous deep neural network system (deep neural ulcerative colitis [DNUC]) to full video colonoscopy and evaluate its validity in the real-time detection of histological mucosal inflammation.

METHODS

In this multicentre, cross-sectional study, we prospectively enrolled consecutive patients (≥15 years) with ulcerative colitis who had an indication for colonoscopy at five hospitals in Japan. Patients in clinical remission were randomly assigned (1:2) to study 1 and study 2. Those with clinically active disease were assigned to study 2 only. Study 1 assessed the validity of real-time histological assessment using DNUC and study 2 validated the consistency of endoscopic scoring between DNUC and experts. The primary endpoint for study 1 was comparison of the results judged by DNUC (healing or active) with biopsy specimens evaluated by pathologists. In study 2, the primary endpoint was the ability of DNUC to determine the Ulcerative Colitis Endoscopic Index of Severity score compared with centrally evaluated scoring by inflammatory bowel disease endoscopy experts.

FINDINGS

From April 1, 2020, to March 31, 2021, 770 patients (180 in study 1 and 590 in study 2) were enrolled. Using real-time histological evaluation, DNUC was able to evaluate the presence or absence of histological inflammation in 729 (81%) of 900 biopsy specimens. For predicting histological remission, the DNUC had a sensitivity of 97·9% (95% CI 97·0-98·5) and a specificity of 94·6% (91·1-96·9). Moreover, its positive predictive value was 98·6% (97·7-99·2) and negative predictive value was 92·1% (88·7-94·3). The intraclass correlation coefficient between DNUC and experts for endoscopic scoring was 0·927 (95% CI 0·915-0·938).

INTERPRETATION

DNUC provided consistently accurate endoscopic scoring and showed potential for reducing the number of biopsies required. This system is an objective and consistent application for video colonoscopy that has potential for use in various medical situations.

FUNDING

Tokyo Medical and Dental University and Sony.

The endoscopic diagnosis of mucosal healing and deep remission in inflammatory bowel disease

The therapeutic goal in inflammatory bowel disease (IBD) patients has shifted from controlling the clinical activity alone to managing other associated problems. The concept of mucosal healing (MH) and deep remission (DR) are advocated and regarded as new therapeutic goals in IBD. However, the definition of MH still remains controversial. It is unclear whether or not the histological structures or functional factors should be included in the definition of DR in addition to clinical remission and MH. The classifications of white-light imaging (e.g. Mayo endoscopic subscore, UCEIS, CD Endoscopic Index of Severity, simple Endoscopic Score-CD) have been proposed and are now widely used to assess the severity as well as the MH of inflammation in IBD. In ulcerative colitis, magnifying chromoendoscopy has been shown to be useful to assess the MH of inflammation while other types of image-enhanced endoscopy, such as narrow-band imaging, have not. Endocytoscopy and confocal laser endomicroscopy (CLE) are also applied to assess the activity in IBD. These endoscopic procedures can estimate MH with more precision through observing the details of superficial structures, such as crypt openings. In addition, CLE can partially assess the mucosal function by detecting fluorescence leakage. Molecular imaging can possibly detect the molecules associated with inflammation, intestinal regeneration and differentiation, and various functions including the intestinal barrier and mucus secretion. These novel procedures may improve the diagnosis strategy of DR through the assessment of DR-associated factors such as the histological structures and functional factors in the near future.

A New Dawn for the Use of Artificial Intelligence in Gastroenterology, Hepatology and Pancreatology

Artificial intelligence (AI) is rapidly becoming an essential tool in the medical field as well as in daily life. Recent developments in deep learning, a subfield of AI, have brought remarkable advances in image recognition, which facilitates improvement in the early detection of cancer by endoscopy, ultrasonography, and computed tomography. In addition, AI-assisted big data analysis represents a great step forward for precision medicine. This review provides an overview of AI technology, particularly for gastroenterology, hepatology, and pancreatology, to help clinicians utilize AI in the near future.

Artificial intelligence-assisted endoscopy changes the definition of mucosal healing in ulcerative colitis

The relevance of endoscopic monitoring of ulcerative colitis (UC) has been translated into the new concept of "mucosal healing (MH)" as the therapeutic goal to achieve because a large amount of scientific data have revealed the favorable prognostic value of a healed mucosa in determining the clinical outcome of UC. Recent interest in MH has skewed toward not only endoscopic remission but also histological improvement (so called histological MH). However, we should recognize that there have been no prospectively validated endoscopic scoring systems of UC activity in previous clinical trials. Artificial intelligence (AI)-assisted endoscopy has been developed for gastrointestinal cancer surveillance. Recently, several AI-assisted endoscopic systems have been developed for assessment of MH in UC. In the future, the development of a new endoscopic scoring system based on AI might standardize the definition of MH. Therefore, "The road to an exact definition of MH in the treatment of UC has begun only now".

Application of artificial intelligence in gastrointestinal disease: a narrative review

Objective

We collected evidence on the application of artificial intelligence (AI) in gastroenterology field. The review was carried out from two aspects of endoscopic types and gastrointestinal diseases, and briefly summarized the challenges and future directions in this field.

Background

Due to the advancement of computational power and a surge of available data, a solid foundation has been laid for the growth of AI. Specifically, varied machine learning (ML) techniques have been emerging in endoscopic image analysis. To improve the accuracy and efficiency of clinicians, AI has been widely applied to gastrointestinal endoscopy.

Methods

PubMed electronic database was searched using the keywords containing “AI”, “ML”, “deep learning (DL)”, “convolution neural network”, “endoscopy (such as white light endoscopy (WLE), narrow band imaging (NBI) endoscopy, magnifying endoscopy with narrow band imaging (ME-NBI), chromoendoscopy, endocytoscopy (EC), and capsule endoscopy (CE))”. Search results were assessed for relevance and then used for detailed discussion.

Conclusions

This review described the basic knowledge of AI, ML, and DL, and summarizes the application of AI in various endoscopes and gastrointestinal diseases. Finally, the challenges and directions of AI in clinical application were discussed. At present, the application of AI has solved some clinical problems, but more still needs to be done.

Machine Learning Predictive Outcomes Modeling in Inflammatory Bowel Diseases

There is a rising interest in use of big data approaches to personalize treatment of inflammatory bowel diseases (IBDs) and to predict and prevent outcomes such as disease flares and therapeutic nonresponse. Machine learning (ML) provides an avenue to identify and quantify features across vast quantities of data to produce novel insights in disease management. In this review, we cover current approaches in ML-driven predictive outcomes modeling for IBD and relate how advances in other fields of medicine may be applied to improve future IBD predictive models. Numerous studies have incorporated clinical, laboratory, or omics data to predict significant outcomes in IBD, including hospitalizations, outpatient corticosteroid use, biologic response, and refractory disease after colectomy, among others, with considerable health care dollars saved as a result. Encouraging results in other fields of medicine support efforts to use ML image analysis-including analysis of histopathology, endoscopy, and radiology-to further advance outcome predictions in IBD. Though obstacles to clinical implementation include technical barriers, bias within data sets, and incongruence between limited data sets preventing model validation in larger cohorts, ML-predictive analytics have the potential to transform the clinical management of IBD. Future directions include the development of models that synthesize all aforementioned approaches to produce more robust predictive metrics.

Role of artificial intelligence in multidisciplinary imaging diagnosis of gastrointestinal diseases

The use of artificial intelligence-based tools is regarded as a promising approach to increase clinical efficiency in diagnostic imaging, improve the interpretability of results, and support decision-making for the detection and prevention of diseases. Radiology, endoscopy and pathology images are suitable for deep-learning analysis, potentially changing the way care is delivered in gastroenterology. The aim of this review is to examine the key aspects of different neural network architectures used for the evaluation of gastrointestinal conditions, by discussing how different models behave in critical tasks, such as lesion detection or characterization (i.e. the distinction between benign and malignant lesions of the esophagus, the stomach and the colon). To this end, we provide an overview on recent achievements and future prospects in deep learning methods applied to the analysis of radiology, endoscopy and histologic whole-slide images of the gastrointestinal tract.

Artificial Intelligence Enhances Studies on Inflammatory Bowel Disease

Inflammatory bowel disease (IBD), which includes ulcerative colitis (UC) and Crohn’s disease (CD), is an idiopathic condition related to a dysregulated immune response to commensal intestinal microflora in a genetically susceptible host. As a global disease, the morbidity of IBD reached a rate of 84.3 per 100,000 persons and reflected a continued gradual upward trajectory. The medical cost of IBD is also notably extremely high. For example, in Europe, it has €3,500 in CD and €2,000 in UC per patient per year, respectively. In addition, taking into account the work productivity loss and the reduced quality of life, the indirect costs are incalculable. In modern times, the diagnosis of IBD is still a subjective judgment based on laboratory tests and medical images. Its early diagnosis and intervention is therefore a challenging goal and also the key to control its progression. Artificial intelligence (AI)-assisted diagnosis and prognosis prediction has proven effective in many fields including gastroenterology. In this study, support vector machines were utilized to distinguish the significant features in IBD. As a result, the reliability of IBD diagnosis due to its impressive performance in classifying and addressing region problems was improved. Convolutional neural networks are advanced image processing algorithms that are currently in existence. Digestive endoscopic images can therefore be better understood by automatically detecting and classifying lesions. This study aims to summarize AI application in the area of IBD, objectively evaluate the performance of these methods, and ultimately understand the algorithm–dataset combination in the studies.

Utility of artificial intelligence in colonoscopy

Colorectal cancer is one of the major causes of death worldwide. Colonoscopy is the most important tool that can identify neoplastic lesion in early stages and resect it in a timely manner which helps in reducing mortality related to colorectal cancer. However, the quality of colonoscopy findings depends on the expertise of the endoscopist and thus the rate of missed adenoma or polyp cannot be controlled. It is desirable to standardize the quality of colonoscopy by reducing the number of missed adenoma/polyps. Introduction of artificial intelligence (AI) in the field of medicine has become popular among physicians nowadays. The application of AI in colonoscopy can help in reducing miss rate and increasing colorectal cancer detection rate as per recent studies. Moreover, AI assistance during colonoscopy has also been utilized in patients with inflammatory bowel disease to improve diagnostic accuracy, assessing disease severity and predicting clinical outcomes. We conducted a literature review on the available evidence on use of AI in colonoscopy. In this review article, we discuss about the principles, application, limitations, and future aspects of AI in colonoscopy.

Artificial intelligence in gastrointestinal endoscopy for inflammatory bowel disease: a systematic review and new horizons

INTRODUCTION

Since the advent of artificial intelligence (AI) in clinical studies, luminal gastrointestinal endoscopy has made great progress, especially in the detection and characterization of neoplastic and preneoplastic lesions. Several studies have recently shown the potential of AI-driven endoscopy for the investigation of inflammatory bowel disease (IBD). This systematic review provides an overview of the current position and future potential of AI in IBD endoscopy.

METHODS

A systematic search was carried out in PubMed and Scopus up to 2 December 2020 using the following search terms: artificial intelligence, machine learning, computer-aided, inflammatory bowel disease, ulcerative colitis (UC), Crohn's disease (CD). All studies on human digestive endoscopy were included. A qualitative analysis and a narrative description were performed for each selected record according to the Joanna Briggs Institute methodologies and the PRISMA statement.

RESULTS

Of 398 identified records, 18 were ultimately included. Two-thirds of these (12/18) were published in 2020 and most were cross-sectional studies (15/18). No relevant bias at the study level was reported, although the risk of publication bias across studies cannot be ruled out at this early stage. Eleven records dealt with UC, five with CD and two with both. Most of the AI systems involved convolutional neural network, random forest and deep neural network architecture. Most studies focused on capsule endoscopy readings in CD (n = 5) and on the AI-assisted assessment of mucosal activity in UC (n = 10) for automated endoscopic scoring or real-time prediction of histological disease.

DISCUSSION

AI-assisted endoscopy in IBD is a rapidly evolving research field with promising technical results and additional benefits when tested in an experimental clinical scenario. External validation studies being conducted in large and prospective cohorts in real-life clinical scenarios will help confirm the added value of AI in assessing UC mucosal activity and in CD capsule reading.

PLAIN LANGUAGE SUMMARY

Artificial intelligence for inflammatory bowel disease endoscopy Artificial intelligence (AI) is a promising technology in many areas of medicine. In recent years, AI-assisted endoscopy has been introduced into several research fields, including inflammatory bowel disease (IBD) endoscopy, with promising applications that have the potential to revolutionize clinical practice and gastrointestinal endoscopy.We have performed the first systematic review of AI and its application in the field of IBD and endoscopy.A formal process of paper selection and analysis resulted in the assessment of 18 records. Most of these (12/18) were published in 2020 and were cross-sectional studies (15/18). No relevant biases were reported. All studies showed positive results concerning the novel technology evaluated, so the risk of publication bias cannot be ruled out at this early stage.Eleven records dealt with UC, five with CD and two with both. Most studies focused on capsule endoscopy reading in CD patients (n = 5) and on AI-assisted assessment of mucosal activity in UC patients (n = 10) for automated endoscopic scoring and real-time prediction of histological disease.We found that AI-assisted endoscopy in IBD is a rapidly growing research field. All studies indicated promising technical results. When tested in an experimental clinical scenario, AI-assisted endoscopy showed it could potentially improve the management of patients with IBD.Confirmatory evidence from real-life clinical scenarios should be obtained to verify the added value of AI-assisted IBD endoscopy in assessing UC mucosal activity and in CD capsule reading.

Breaking the therapeutic ceiling in drug development in ulcerative colitis

Increased knowledge of the intricate pathogenesis of ulcerative colitis has triggered an advance in drug development during the past two decades, resulting in the advent of several biological agents and small-molecule therapies. Although the increase in therapeutic options is positive, remission rates of patients with ulcerative colitis given new therapeutic agents in induction trials remain at a modest 20-30%, seemingly facing a so-called therapeutic ceiling. This therapeutic ceiling requires a critical appraisal and highlights the need for alternative strategies for drug development. In this Review, we objectively itemise the boundaries of therapeutic efficacy in ulcerative colitis, provide possible explanations for the shortcomings of current strategies, and propose solutions to achieve better therapeutic outcomes in ulcerative colitis.

Artificial intelligence applications in inflammatory bowel disease: Emerging technologies and future directions

Inflammatory bowel disease (IBD) is a complex and multifaceted disorder of the gastrointestinal tract that is increasing in incidence worldwide and associated with significant morbidity. The rapid accumulation of large datasets from electronic health records, high-definition multi-omics (including genomics, proteomics, transcriptomics, and metagenomics), and imaging modalities (endoscopy and endomicroscopy) have provided powerful tools to unravel novel mechanistic insights and help address unmet clinical needs in IBD. Although the application of artificial intelligence (AI) methods has facilitated the analysis, integration, and interpretation of large datasets in IBD, significant heterogeneity in AI methods, datasets, and clinical outcomes and the need for unbiased prospective validations studies are current barriers to incorporation of AI into clinical practice. The purpose of this review is to summarize the most recent advances in the application of AI and machine learning technologies in the diagnosis and risk prediction, assessment of disease severity, and prediction of clinical outcomes in patients with IBD.

Rediscovering histology: what is new in endoscopy for inflammatory bowel disease?

The potential of endoscopic evaluation in the management of inflammatory bowel diseases (IBD) has undoubtedly grown over the last few years. When dealing with IBD patients, histological remission (HR) is now considered a desirable target along with symptomatic and endoscopic remission, due to its association with better long-term outcomes. Consequently, the ability of endoscopic techniques to reflect microscopic findings in vivo without having to collect biopsies has become of upmost importance. In this context, a more accurate evaluation of inflammatory disease activity and the detection of dysplasia represent two mainstay targets for IBD endoscopists. New diagnostic technologies have been developed, such as dye-less chromoendoscopy, endomicroscopy, and molecular imaging, but their real incorporation in daily practice is not yet well defined. Although dye-chromoendoscopy is still recommended as the gold standard approach in dysplasia surveillance, recent research questioned the superiority of this technique over new advanced dye-less modalities [narrow band imaging (NBI), Fuji intelligent color enhancement (FICE), i-scan, blue light imaging (BLI) and linked color imaging (LCI)]. The endoscopic armamentarium might also be enriched by new video capsule endoscopy for monitoring disease activity, and high expectations are placed on the application of artificial intelligence (AI) systems to reduce operator-subjectivity and inter-observer variability. The goal of this review is to provide an updated insight on contemporary knowledge regarding new endoscopic techniques and devices, with special focus on their role in the assessment of disease activity and colorectal cancer surveillance.

Variability in the Distribution of Histological Disease Activity in the Colon of Patients with Ulcerative Colitis

BACKGROUND AND AIMS

Histological activity scores have been developed and validated. However, data on the distribution of histological inflammation within one segment in patients with ulcerative colitis [UC] are lacking. This impacts on the reliability of histological activity scores. The aim of this study was to assess the variability in histological activity within one endoscopic segment in patients with UC.

METHODS

Biopsies were taken in sequential patients with UC in three adjacent contiguous regions within a macroscopically homogeneous colonic segment. Biopsies were scored for Geboes score [GS], Robarts histological index [RHI] and Nancy histological index [NHI]. Variability was assessed by Kappa statistics for categorical outcomes and intraclass correlation coefficient [ICC] for continuous outcomes.

RESULTS

A total of 161 biopsy sets from 55 endoscopic segments of 21 patients were analysed. Endoscopically active disease was present in 45% of segments. The continuous histological scores showed excellent agreement between the different regions. The ICC for RHI in all segments was 0.974 (95% confidence interval [CI] 0.958-0.984; p < 0.0001) and 0.98 [95% CI: 0.968-0.988; p < 0.0001] for the numerically converted GS. The categorical NHI showed higher variability: κ = 0.574 [95% CI: 0.571-0.577; p < 0.0001]. In all segments the highest variability was seen in samples with NHI = 2. When dichotomizing based on histological remission, substantial agreement was seen for all scores, with κ > 0.734 for all cut-offs. The homogeneity in the distribution of histological disease activity was comparable between colonic segments.

CONCLUSION

The distribution of histological disease activity in UC follows a homogeneous pattern in different locations of one segment.

Endoscopic imaging in inflammatory bowel disease: current developments and emerging strategies

INTRODUCTION

Developments in enhanced and magnified endoscopy have signified major advances in endoscopic imaging of ileocolonic pathology in inflammatory bowel disease (IBD). Artificial intelligence is increasingly being used to augment the benefits of these advanced techniques. Nevertheless, treatment of IBD patients is frustrated by high rates of non-response to therapy, while delayed detection and failures to detect neoplastic lesions impede successful surveillance. A possible solution is offered by molecular imaging, which adds functional imaging data to mucosal morphology assessment through visualizing biological parameters. Other label-free modalities enable visualization beyond the mucosal surface without the need of tracers.

AREAS COVERED

A literature search up to May 2020 was conducted in PubMed/MEDLINE in order to find relevant articles that involve the (pre-)clinical application of high-definition white light endoscopy, chromoendoscopy, artificial intelligence, confocal laser endomicroscopy, endocytoscopy, molecular imaging, optical coherence tomography, and Raman spectroscopy in IBD.

EXPERT OPINION

Enhanced and magnified endoscopy have enabled an improved assessment of the ileocolonic mucosa. Implementing molecular imaging in endoscopy could overcome the remaining clinical challenges by giving practitioners a real-time in vivo view of targeted biomarkers. Label-free modalities could help optimize the endoscopic assessment of mucosal healing and dysplasia detection in IBD patients.

Artificial intelligence in endoscopy: Present and future perspectives

Artificial intelligence (AI) has been attracting considerable attention as an important scientific topic in the field of medicine. Deep-leaning (DL) technologies have been applied more dominantly than other traditional machine-learning methods. They have demonstrated excellent capability to retract visual features of objectives, even unnoticeable ones for humans, and analyze huge amounts of information within short periods. The amount of research applying DL-based models to real-time computer-aided diagnosis (CAD) systems has been increasing steadily in the GI endoscopy field. An array of published data has already demonstrated the advantages of DL-based CAD models in the detection and characterization of various neoplastic lesions, regardless of the level of the GI tract. Although the diagnostic performances and study designs vary widely, owing to a lack of academic standards to assess the capability of AI for GI endoscopic diagnosis fairly, the superiority of CAD models has been demonstrated for almost all applications studied so far. Most of the challenges associated with AI in the endoscopy field are general problems for AI models used in the real world outside of medical fields. Solutions have been explored seriously and some solutions have been tested in the endoscopy field. Given that AI has become the basic technology to make machines react to the environment, AI would be a major technological paradigm shift, for not only diagnosis but also treatment. In the near future, autonomous endoscopic diagnosis might no longer be just a dream, as we are witnessing with the advent of autonomously driven electric vehicles.

Current status and future perspective on artificial intelligence for lower endoscopy

The global incidence and mortality rate of colorectal cancer remains high. Colonoscopy is regarded as the gold standard examination for detecting and eradicating neoplastic lesions. However, there are some uncertainties in colonoscopy practice that are related to limitations in human performance. First, approximately one-fourth of colorectal neoplasms are missed on a single colonoscopy. Second, it is still difficult for non-experts to perform adequately regarding optical biopsy. Third, recording of some quality indicators (e.g. cecal intubation, bowel preparation, and withdrawal speed) which are related to adenoma detection rate, is sometimes incomplete. With recent improvements in machine learning techniques and advances in computer performance, artificial intelligence-assisted computer-aided diagnosis is being increasingly utilized by endoscopists. In particular, the emergence of deep-learning, data-driven machine learning techniques have made the development of computer-aided systems easier than that of conventional machine learning techniques, the former currently being considered the standard artificial intelligence engine of computer-aided diagnosis by colonoscopy. To date, computer-aided detection systems seem to have improved the rate of detection of neoplasms. Additionally, computer-aided characterization systems may have the potential to improve diagnostic accuracy in real-time clinical practice. Furthermore, some artificial intelligence-assisted systems that aim to improve the quality of colonoscopy have been reported. The implementation of computer-aided system clinical practice may provide additional benefits such as helping in educational poorly performing endoscopists and supporting real-time clinical decision-making. In this review, we have focused on computer-aided diagnosis during colonoscopy reported by gastroenterologists and discussed its status, limitations, and future prospects.

Artificial intelligence in inflammatory bowel disease endoscopy: current landscape and the road ahead

Inflammatory bowel disease is a complex chronic inflammatory disorder with challenges in diagnosis, choosing appropriate therapy, determining individual responsiveness, and prediction of future disease course to guide appropriate management. Artificial intelligence has been examined in the field of inflammatory bowel disease endoscopy with promising data in different domains of inflammatory bowel disease, including diagnosis, assessment of mucosal activity, and prediction of recurrence and complications. Artificial intelligence use during endoscopy could be a step toward precision medicine in inflammatory bowel disease care pathways. We reviewed available data on use of artificial intelligence for diagnosis of inflammatory bowel disease, grading of severity, prediction of recurrence, and dysplasia detection. We examined the potential role of artificial intelligence enhanced endoscopy in various aspects of inflammatory bowel disease care and future perspectives in this review.

Artificial intelligence and its impact on quality improvement in upper and lower gastrointestinal endoscopy

Artificial intelligence (AI) and its application in medicine has grown large interest. Within gastrointestinal (GI) endoscopy, the field of colonoscopy and polyp detection is the most investigated, however, upper GI follows the lead. Since endoscopy is performed by humans, it is inherently an imperfect procedure. Computer-aided diagnosis may improve its quality by helping prevent missing lesions and supporting optical diagnosis for those detected. An entire evolution in AI systems has been established in the last decades, resulting in optimization of the diagnostic performance with lower variability and matching or even outperformance of expert endoscopists. This shows a great potential for future quality improvement of endoscopy, given the outstanding diagnostic features of AI. With this narrative review, we highlight the potential benefit of AI to improve overall quality in daily endoscopy and describe the most recent developments for characterization and diagnosis as well as the recent conditions for regulatory approval.

Striving for quality improvement: can artificial intelligence help?

Artificial intelligence (AI) is of keen interest for global health development as potential support for current human shortcomings. Gastrointestinal (GI) endoscopy is an excellent substrate for AI, since it holds the genuine potential to improve quality in GI endoscopy and overall patient care by improving detection and diagnosis guiding the endoscopists in performing endoscopy to the highest quality standards. The possibility of large data acquisitioning to refine algorithms makes implementation of AI into daily practice a potential reality. With the start of a new era adopting deep learning, large amounts of data can easily be processed, resulting in better diagnostic performances. In the upper gastrointestinal tract, research currently focusses on the detection and characterization of neoplasia, including Barrett's, squamous cell and gastric carcinoma, with an increasing amount of AI studies demonstrating the potential and benefit of AI-augmented endoscopy. Deep learning applied to small bowel video capsule endoscopy also appears to enhance pathology detection and reduce capsule reading time. In the colon, multiple prospective trials including five randomized trials, showed a consistent improvement in polyp and adenoma detection rates, one of the main quality indicators in endoscopy. There are however potential additional roles for AI to assist in quality improvement of endoscopic procedures, training and therapeutic decision making. Further large-scale, multicenter validation trials are required before AI-augmented diagnostic gastrointestinal endoscopy can be integrated into our routine clinical practice.

Artificial intelligence: Thinking outside the box

Artificial intelligence (AI) for luminal gastrointestinal endoscopy is rapidly evolving. To date, most applications have focused on colon polyp detection and characterization. However, the potential of AI to revolutionize our current practice in endoscopy is much more broadly positioned. In this review article, the Authors provide new ideas on how AI might help endoscopists in the future to rediscover endoscopy practice.

Big data in IBD: big progress for clinical practice

IBD is a complex multifactorial inflammatory disease of the gut driven by extrinsic and intrinsic factors, including host genetics, the immune system, environmental factors and the gut microbiome. Technological advancements such as next-generation sequencing, high-throughput omics data generation and molecular networks have catalysed IBD research. The advent of artificial intelligence, in particular, machine learning, and systems biology has opened the avenue for the efficient integration and interpretation of big datasets for discovering clinically translatable knowledge. In this narrative review, we discuss how big data integration and machine learning have been applied to translational IBD research. Approaches such as machine learning may enable patient stratification, prediction of disease progression and therapy responses for fine-tuning treatment options with positive impacts on cost, health and safety. We also outline the challenges and opportunities presented by machine learning and big data in clinical IBD research.