Foundation models in gastrointestinal endoscopic AI: Impact of architecture, pre-training approach and data efficiency

The Association Between Heatmap Position and the Diagnostic Accuracy of Artificial Intelligence for Colorectal Polyp Diagnosis

BACKGROUND/OBJECTIVES

Artificial intelligence (AI) algorithms for diagnosing colorectal polyps are emerging but not yet widely used. Trust in AI is lacking and could be improved by visually explainable AI, such as heatmaps. This study aims to investigate the association between heatmap position and AI accuracy for the endoscopic characterization of colorectal polyps.

METHODS

Four AI algorithms diagnosed 2133 prospectively collected images of 376 colorectal polyps from two hospitals, using histopathology as the gold standard. Heatmap position was compared to the human-annotated polyp position. Generalized estimating equations were used to assess the association between heatmap position and a correct AI diagnosis.

RESULTS

Higher percentages of heatmap covering the colorectal polyp were associated with correct diagnoses in all four algorithms (OR 1.013 [95% CI 1.006-1.019], OR 1.025 [95% CI 1.011-1.039], OR 1.038 [95% CI 1.024-1.053], and OR 1.039 [95% CI 1.020-1.058]-all p < 0.001). A higher percentage of polyp not covered by heatmap was associated with a correct diagnosis of Algorithm 1 (OR 1.006 [95% CI 1.003-1.010], p < 0.001), while in Algorithm 2, a lower percentage was associated with a correct diagnosis (OR 0.992 [95% CI 0.985-1.000], p 0.044). Algorithms 3 and 4 showed negative, but not statistically significant, associations.

CONCLUSIONS

Higher percentages of heatmap covering the polyp were associated with correct diagnoses of four AI algorithms. This indicates that it is clinically relevant to strive for AI predictions with heatmaps covering as much colorectal polyp tissue as possible. Knowing how to interpret heatmaps could increase trust in AI and, with that, benefit the implementation of AI in clinical practice.

The development and ex vivo evaluation of a computer-aided quality control system for Barrett's esophagus endoscopy

BACKGROUND : Timely detection of neoplasia in Barrett's esophagus (BE) remains challenging. While computer-aided detection (CADe) systems have been developed to assist endoscopists, their effectiveness depends heavily on the quality of the endoscopic procedure. This study introduces a novel computer-aided quality (CAQ) system for BE, evaluating its stand-alone performance and integration with a CADe system. METHOD : The CAQ system was developed using 7,463 images from 359 BE patients. It assesses objective quality parameters (e. g., blurriness, illumination) and subjective parameters (mucosal cleanliness, esophageal expansion) and can exclude low-quality images when integrated with a CADe system.To evaluate CAQ stand-alone performance, the Endoscopic Image Quality test set, consisting of 647 images from 51 BE patients across 8 hospitals, was labeled for objective and subjective quality. To assess the benefit of the CAQ system as a preprocessing filter of a CADe system, the Barrett CADe test set was developed. It consisted of 956 video frames from 62 neoplastic patients and 557 frames from 35 non-dysplastic patients, in 12 Barrett referral centers. RESULTS : As stand-alone tool, the CAQ system achieved Cohen's Kappa scores of 0.73, 0.91, and 0.89 for objective quality, mucosal cleanliness, and esophageal expansion, comparable to inter-annotator scores of 0.73, 0.93, and 0.83. As preprocessing filter, the CAQ system improved CADe sensitivity from 82 % to 90 % and AUC from 87 % to 91 %, while maintaining specificity at 75 %. CONCLUSION : This study presents the first CAQ system for automated quality control in BE. The system effectively distinguishes poorly from well-visualized mucosa and enhances neoplasia detection when integrated with CADe.