Development of artificial intelligence system for quality control of photo documentation in esophagogastroduodenoscopy

Artificial intelligence for gastric cancer in endoscopy: From diagnostic reasoning to market

Recognition of gastric conditions during endoscopy exams, including gastric cancer, usually requires specialized training and a long learning curve. Besides that, the interobserver variability is frequently high due to the different morphological characteristics of the lesions and grades of mucosal inflammation. In this sense, artificial intelligence tools based on deep learning models have been developed to support physicians to detect, classify, and predict gastric lesions more efficiently. Even though a growing number of studies exists in the literature, there are multiple challenges to bring a model to practice in this field, such as the need for more robust validation studies and regulatory hurdles. Therefore, the aim of this review is to provide a comprehensive assessment of the current use of artificial intelligence applied to endoscopic imaging to evaluate gastric precancerous and cancerous lesions and the barriers to widespread implementation of this technology in clinical routine.

Applications of artificial intelligence in gastroscopy: a narrative review

Gastroscopy, a critical tool for the diagnosis of upper gastrointestinal diseases, has recently incorporated artificial intelligence (AI) technology to alleviate the challenges involved in endoscopic diagnosis of some lesions, thereby enhancing diagnostic accuracy. This narrative review covers the current status of research concerning various applications of AI technology to gastroscopy, then discusses future research directions. By providing this review, we hope to promote the integration of gastroscopy and AI technology, with long-term clinical applications that can assist patients.

Endoscopic Imaging for the Diagnosis of Neoplastic and Pre-Neoplastic Conditions of the Stomach

Gastric cancer is an aggressive disease with low long-term survival rates. An early diagnosis is essential to offer a better prognosis and curative treatment. Upper gastrointestinal endoscopy is the main tool for the screening and diagnosis of patients with gastric pre-neoplastic conditions and early lesions. Image-enhanced techniques such as conventional chromoendoscopy, virtual chromoendoscopy, magnifying imaging, and artificial intelligence improve the diagnosis and the characterization of early neoplastic lesions. In this review, we provide a summary of the currently available recommendations for the screening, surveillance, and diagnosis of gastric cancer, focusing on novel endoscopy imaging technologies.

Anatomical classification of pharyngeal and laryngeal endoscopic images using artificial intelligence

BACKGROUND

The entire pharynx should be observed endoscopically to avoid missing pharyngeal lesions. An artificial intelligence (AI) model recognizing anatomical locations can help identify blind spots. We developed and evaluated an AI model classifying pharyngeal and laryngeal endoscopic locations.

METHODS

The AI model was trained using 5382 endoscopic images, categorized into 15 anatomical locations, and evaluated using an independent dataset of 1110 images. The main outcomes were model accuracy, precision, recall, and F1-score. Moreover, we investigated focused regions in the input images contributing to the model predictions using gradient-weighted class activation mapping (Grad-CAM) and Guided Grad-CAM.

RESULTS

Our AI model correctly classified pharyngeal and laryngeal images into 15 anatomical locations, with an accuracy of 93.3%. The weighted averages of precision, recall, and F1-score were 0.934, 0.933, and 0.933, respectively.

CONCLUSION

Our AI model has an excellent performance determining pharyngeal and laryngeal anatomical locations, helping endoscopists notify of blind spots.

An Optimal Artificial Intelligence System for Real-Time Endoscopic Prediction of Invasion Depth in Early Gastric Cancer

We previously constructed a VGG-16 based artificial intelligence (AI) model (image classifier [IC]) to predict the invasion depth in early gastric cancer (EGC) using endoscopic static images. However, images cannot capture the spatio-temporal information available during real-time endoscopy-the AI trained on static images could not estimate invasion depth accurately and reliably. Thus, we constructed a video classifier [VC] using videos for real-time depth prediction in EGC. We built a VC by attaching sequential layers to the last convolutional layer of IC v2, using video clips. We computed the standard deviation (SD) of output probabilities for a video clip and the sensitivities in the manner of frame units to observe consistency. The sensitivity, specificity, and accuracy of IC v2 for static images were 82.5%, 82.9%, and 82.7%, respectively. However, for video clips, the sensitivity, specificity, and accuracy of IC v2 were 33.6%, 85.5%, and 56.6%, respectively. The VC performed better analysis of the videos, with a sensitivity of 82.3%, a specificity of 85.8%, and an accuracy of 83.7%. Furthermore, the mean SD was lower for the VC than IC v2 (0.096 vs. 0.289). The AI model developed utilizing videos can predict invasion depth in EGC more precisely and consistently than image-trained models, and is more appropriate for real-world situations.

Expected value of artificial intelligence in gastrointestinal endoscopy: European Society of Gastrointestinal Endoscopy (ESGE) Position Statement

This ESGE Position Statement defines the expected value of artificial intelligence (AI) for the diagnosis and management of gastrointestinal neoplasia within the framework of the performance measures already defined by ESGE. This is based on the clinical relevance of the expected task and the preliminary evidence regarding artificial intelligence in artificial or clinical settings. MAIN RECOMMENDATIONS:: (1) For acceptance of AI in assessment of completeness of upper GI endoscopy, the adequate level of mucosal inspection with AI should be comparable to that assessed by experienced endoscopists. (2) For acceptance of AI in assessment of completeness of upper GI endoscopy, automated recognition and photodocumentation of relevant anatomical landmarks should be obtained in ≥90% of the procedures. (3) For acceptance of AI in the detection of Barrett's high grade intraepithelial neoplasia or cancer, the AI-assisted detection rate for suspicious lesions for targeted biopsies should be comparable to that of experienced endoscopists with or without advanced imaging techniques. (4) For acceptance of AI in the management of Barrett's neoplasia, AI-assisted selection of lesions amenable to endoscopic resection should be comparable to that of experienced endoscopists. (5) For acceptance of AI in the diagnosis of gastric precancerous conditions, AI-assisted diagnosis of atrophy and intestinal metaplasia should be comparable to that provided by the established biopsy protocol, including the estimation of extent, and consequent allocation to the correct endoscopic surveillance interval. (6) For acceptance of artificial intelligence for automated lesion detection in small-bowel capsule endoscopy (SBCE), the performance of AI-assisted reading should be comparable to that of experienced endoscopists for lesion detection, without increasing but possibly reducing the reading time of the operator. (7) For acceptance of AI in the detection of colorectal polyps, the AI-assisted adenoma detection rate should be comparable to that of experienced endoscopists. (8) For acceptance of AI optical diagnosis (computer-aided diagnosis [CADx]) of diminutive polyps (≤5 mm), AI-assisted characterization should match performance standards for implementing resect-and-discard and diagnose-and-leave strategies. (9) For acceptance of AI in the management of polyps ≥ 6 mm, AI-assisted characterization should be comparable to that of experienced endoscopists in selecting lesions amenable to endoscopic resection.

Development and validation of a deep learning-based algorithm for colonoscopy quality assessment

BACKGROUND

Quality indicators should be assessed and monitored to improve colonoscopy quality in clinical practice. Endoscopists must enter relevant information in the endoscopy reporting system to facilitate data collection, which may be inaccurate. The current study aimed to develop a full deep learning-based algorithm to identify and analyze intra-procedural colonoscopy quality indicators based on endoscopy images obtained during the procedure.

METHODS

A deep learning system for classifying colonoscopy images for quality assurance purposes was developed and its performance was assessed with an independent dataset. The system was utilized to analyze captured images and results were compared with those of real-world reports.

RESULTS

In total, 10,417 images from the hospital endoscopy database and 3157 from Hyper-Kvasir open dataset were utilized to develop the quality assurance algorithm. The overall accuracy of the algorithm was 96.72% and that of the independent test dataset was 94.71%. Moreover, 761 real-world reports and colonoscopy images were analyzed. The accuracy of electronic reports about cecal intubation rate was 99.34% and that of the algorithm was 98.95%. The agreement rate for the assessment of polypectomy rates using the electronic reports and the algorithm was 0.87 (95% confidence interval 0.83-0.90). A good correlation was found between the withdrawal time calculated using the algorithm and that entered by the physician (correlation coefficient r = 0.959, p < 0.0001).

CONCLUSION

We proposed a novel deep learning-based algorithm that used colonoscopy images for quality assurance purposes. This model can be used to automatically assess intra-procedural colonoscopy quality indicators in clinical practice.

Upper endoscopy photodocumentation quality evaluation with novel deep learning system

OBJECTIVES

Visualization and photodocumentation during endoscopy procedures are suggested to be one indicator for endoscopy performance quality. However, this indicator is difficult to measure and audit manually in clinical practice. Artificial intelligence (AI) is an emerging technology that may solve this problem.

METHODS

A deep learning model with an accuracy of 96.64% was developed from 15,305 images for upper endoscopy anatomy classification in the unit. Endoscopy images for asymptomatic patients receiving screening endoscopy were evaluated with this model to assess the completeness of photodocumentation rate.

RESULTS

A total of 15,723 images from 472 upper endoscopies performed by 12 endoscopists were enrolled. The complete photodocumentation rate from the pharynx to the duodenum was 53.8% and from the esophagus to the duodenum was 78.0% in this study. Endoscopists with a higher adenoma detection rate had a higher complete examination rate from the pharynx to duodenum (60.0% vs. 38.7%, P < 0.0001) and from esophagus to duodenum (83.0% vs. 65.7%, P < 0.0001) compared with endoscopists with lower adenoma detection rate. The pharynx, gastric angle, gastric retroflex view, gastric antrum, and the first portion of duodenum are likely to be missed by endoscopists with lower adenoma detection rates.

CONCLUSIONS

We report the use of a deep learning model to audit endoscopy photodocumentation quality in our unit. Endoscopists with better performance in colonoscopy had a better performance for this quality indicator. The use of such an AI system may help the endoscopy unit audit endoscopy performance.

Correlation of the detection rate of upper GI cancer with artificial intelligence score: results from a multicenter trial (with video)

BACKGROUND AND AIMS

The quality of EGD is a prerequisite for a high detection rate of upper GI lesions, especially early gastric cancer. Our previous study showed that an artificial intelligence system, named intelligent detection endoscopic assistant (IDEA), could help to monitor blind spots and provide an operation score during EGD. Here, we verified the effectiveness of IDEA to help evaluate the quality of EGD in a large-scale multicenter trial.

METHODS

Patients undergoing EGD in 12 hospitals were consecutively enrolled. All hospitals were equipped with IDEA developed using deep convolutional neural networks and long short-term memory. Patients were examined by EGD, and the results were recorded by IDEA. The primary outcome was the detection rate of upper GI cancer. Secondary outcomes were part scores, total scores, and endoscopic procedure time, which were analyzed by IDEA.

RESULTS

A total of 17,787 patients were recruited. The total detection rate of cancer-positive cases was 1.50%, ranging from .60% to 3.94% in each hospital. The total detection rate of early cancer-positive cases was .36%, ranging from .00% to 1.58% in each hospital. The average total score analyzed by IDEA ranged from 64.87 ± 16.87 to 83.50 ± 9.57 in each hospital. The cancer detection rate in each hospital was positively correlated with total score (r = .775, P = .003). Similarly, the early cancer detection rate was positively correlated with total score (r = .756, P = .004).

CONCLUSIONS

This multicenter trial confirmed that the quality of the EGD result is positively correlated with the detection rate of cancer, which can be monitored by IDEA. (Clinical trial registration number: ChiCTR2000029001.).

Deep learning-based endoscopic anatomy classification: an accelerated approach for data preparation and model validation

BACKGROUND

Photodocumentation during endoscopy procedures is one of the indicators for endoscopy performance quality; however, this indicator is difficult to measure and audit in the endoscopy unit. Emerging artificial intelligence technology may solve this problem, which requires a large amount of material for model development. We developed a deep learning-based endoscopic anatomy classification system through convolutional neural networks with an accelerated data preparation approach.

PATIENTS AND METHODS

We retrospectively collected 8,041 images from esophagogastroduodenoscopy (EGD) procedures and labeled them using two experts for nine anatomical locations of the upper gastrointestinal tract. A base model for EGD image multiclass classification was first developed, and an additional 6,091 images were enrolled and classified by the base model. A total of 5,963 images were manually confirmed and added to develop the subsequent enhanced model. Additional internal and external endoscopy image datasets were used to test the model performance.

RESULTS

The base model achieved total accuracy of 96.29%. For the enhanced model, the total accuracy was 96.64%. The overall accuracy improved with the enhanced model compared with the base model for the internal test dataset without narrowband images (93.05% vs. 91.25%, p < 0.01) or with narrowband images (92.74% vs. 90.46%, p < 0.01). The total accuracy was 92.56% of the enhanced model on the external test dataset.

CONCLUSIONS

We constructed a deep learning-based model with an accelerated approach that can be used for quality control in endoscopy units. The model was also validated with both internal and external datasets with high accuracy.

Artificial Intelligence for Upper Gastrointestinal Endoscopy: A Roadmap from Technology Development to Clinical Practice

Stomach cancer is the third deadliest type of cancer in the world (0.86 million deaths in 2017). In 2035, a 20% increase will be observed both in incidence and mortality due to demographic effects if no interventions are foreseen. Upper GI endoscopy (UGIE) plays a paramount role in early diagnosis and, therefore, improved survival rates. On the other hand, human and technical factors can contribute to misdiagnosis while performing UGIE. In this scenario, artificial intelligence (AI) has recently shown its potential in compensating for the pitfalls of UGIE, by leveraging deep learning architectures able to efficiently recognize endoscopic patterns from UGIE video data. This work presents a review of the current state-of-the-art algorithms in the application of AI to gastroscopy. It focuses specifically on the threefold tasks of assuring exam completeness (i.e., detecting the presence of blind spots) and assisting in the detection and characterization of clinical findings, both gastric precancerous conditions and neoplastic lesion changes. Early and promising results have already been obtained using well-known deep learning architectures for computer vision, but many algorithmic challenges remain in achieving the vision of AI-assisted UGIE. Future challenges in the roadmap for the effective integration of AI tools within the UGIE clinical practice are discussed, namely the adoption of more robust deep learning architectures and methods able to embed domain knowledge into image/video classifiers as well as the availability of large, annotated datasets.

Quality indicators in esophagogastroduodenoscopy

Esophagogastroduodenoscopy (EGD) has been used to diagnose a wide variety of upper gastrointestinal diseases. In particular, EGD is used to screen high-risk subjects of gastric cancer. Quality control of EGD is important because the diagnostic rate is examiner-dependent. However, there is still no representative quality indicator that can be uniformly applied in EGD. There has been growing awareness of the importance of quality control in improving EGD performance. Therefore, we aimed to review the available and emerging quality indicators for diagnostic EGD.

Artificial Intelligence in the Analysis of Upper Gastrointestinal Disorders

In the past, conventional machine learning was applied to analyze tabulated medical data while deep learning was applied to study afflictions such as gastrointestinal disorders. Neural networks were used to detect, classify, and delineate various images of lesions because the local feature selection and optimization of the deep learning model enabled accurate image analysis. With the accumulation of medical records, the evolution of computational power and graphics processing units, and the widespread use of open-source libraries in large-scale machine learning processes, medical artificial intelligence (AI) is overcoming its limitations. While early studies prioritized the automatic diagnosis of cancer or pre-cancerous lesions, the current expanded scope of AI includes benign lesions, quality control, and machine learning analysis of big data. However, the limited commercialization of medical AI and the need to justify its application in each field of research are restricting factors. Modeling assumes that observations follow certain statistical rules, and external validation checks whether assumption is correct or generalizable. Therefore, unused data are essential in the training or internal testing process to validate the performance of the established AI models. This article summarizes the studies on the application of AI models in upper gastrointestinal disorders. The current limitations and the perspectives on future development have also been discussed.

Use of Artificial Intelligence to Improve the Quality Control of Gastrointestinal Endoscopy

With the rapid development of science and technology, artificial intelligence (AI) systems are becoming ubiquitous, and their utility in gastroenteroscopy is beginning to be recognized. Digestive endoscopy is a conventional and reliable method of examining and diagnosing digestive tract diseases. However, with the increase in the number and types of endoscopy, problems such as a lack of skilled endoscopists and difference in the professional skill of doctors with different degrees of experience have become increasingly apparent. Most studies thus far have focused on using computers to detect and diagnose lesions, but improving the quality of endoscopic examination process itself is the basis for improving the detection rate and correctly diagnosing diseases. In the present study, we mainly reviewed the role of AI in monitoring systems, mainly through the endoscopic examination time, reducing the blind spot rate, improving the success rate for detecting high-risk lesions, evaluating intestinal preparation, increasing the detection rate of polyps, automatically collecting maps and writing reports. AI can even perform quality control evaluations for endoscopists, improve the detection rate of endoscopic lesions and reduce the burden on endoscopists.