Specular highlight removal for endoscopic images using partial attention network

Objective.Endoscopic imaging is a visualization method widely used in minimally invasive surgery. However, owing to the strong reflection of the mucus layer on the organs, specular highlights often appear to degrade the imaging performance. Thus, it is necessary to develop an effective highlight removal method for endoscopic imaging.Approach.A specular highlight removal method using a partial attention network (PatNet) for endoscopic imaging is proposed to reduce the interference of bright light in endoscopic surgery. The method is designed as two procedures: highlight segmentation and endoscopic image inpainting. Image segmentation uses brightness threshold based on illumination compensation to divide the endoscopic image into the highlighted mask and the non-highlighted area. The image inpainting algorithm uses a partial convolution network that integrates an attention mechanism. A mask dataset with random hopping points is designed to simulate specular highlight in endoscopic imaging for network training. Through the filtering of masks, the method can focus on recovering defective pixels and preserving valid pixels as much as possible.Main results.The PatNet is compared with 3 highlight segmentation methods, 3 imaging inpainting methods and 5 highlight removal methods for effective analysis. Experimental results show that the proposed method provides better performance in terms of both perception and quantification. In addition, surgeons are invited to score the processing results for different highlight removal methods under realistic reflection conditions. The PatNet received the highest score of 4.18. Correspondingly, the kendall's W is 0.757 and the asymptotic significancep= 0.000 < 0.01, revealing that the subjective scores have good consistency and confidence.Significance.Generally, the method can realize irregular shape highlight reflection removal and image restoration close to the ground truth of endoscopic images. This method can improve the quality of endoscopic imaging for accurate image analysis.

On the rocks: can urologists identify stone composition based on endoscopic images alone? A worldwide survey of urologists

PURPOSE

As part of the management of nephrolithiasis, determination of chemical composition of stones is important. Our objective in this study is to assess urologists' accuracy in making visual, intraoperative determinations of stone composition.

MATERIALS AND METHODS

We conducted a REDCap survey asking urologists to predict stone composition based on intraoperative images of 10 different pure-composition kidney stones of 7 different types: calcium oxalate monohydrate (COM), calcium oxalate dihydrate (COD), calcium phosphate (CP) apatite, CP brushite, uric acid (UA), struvite (ST) and cystine (CY). To evaluate experience, we examined specific endourologic training, years of experience, and number of ureteroscopy (URS) cases/week. A self-assessment of ability to identify stone composition was also required.

RESULTS

With a response rate of 26% (366 completed surveys out of 1,370 deliveries), the overall accuracy of our cohort was 44%. COM, ST, and COD obtained the most successful identification rates (65.9%, 55.7%, and 52.0%, respectively). The most frequent misidentified stones were CP apatite (10.7%) and CY (14.2%). Predictors of increased overall accuracy included self-perceived ability to determine composition and number of ureteroscopies per week, while years of experience did not show a positive correlation.

CONCLUSIONS

Although endoscopic stone recognition can be an important tool for surgeons, it is not reliable enough to be utilized as a single method for stone identification, suggesting that urologists need to refine their ability to successfully recognize specific stone compositions intraoperatively.

Endoscopic diagnosis and treatment of gastric dysplasia and early cancer: Current evidence and what the future may hold

Gastric cancer accounts for a significant proportion of worldwide cancer-related morbidity and mortality. The well documented precancerous cascade provides an opportunity for clinicians to detect and treat gastric cancers at an endoscopically curable stage. In high prevalence regions such as Japan and Korea, this has led to the implementation of population screening programs. However, guidelines remain ambiguous in lower prevalence regions. In recent years, there have been many advances in the endoscopic diagnosis and treatment of early gastric cancer and precancerous lesions. More advanced endoscopic imaging has led to improved detection and characterization of gastric lesions as well as superior accuracy for delineation of margins prior to resection. In addition, promising early data on artificial intelligence in gastroscopy suggests a future role for this technology in maximizing the yield of advanced endoscopic imaging. Data on endoscopic resection (ER) are particularly robust in Japan and Korea, with high rates of curative ER and markedly reduced procedural morbidity. However, there is a shortage of data in other regions to support the applicability of protocols from these high prevalence countries. Future advances in endoscopic therapeutics will likely lead to further expansion of the current indications for ER, as both technology and proceduralist expertise continue to grow.

iHELMET: A 3D-printing solution for safe endoscopic Ca2+ recording in social neuroscience

BACKGROUND

In vivo calcium imaging using a microendoscope is a state-of-the-art technique to study the cellular activity inside the brain of freely moving animals such as mice or rats. A problem that can arise in social behaviour tests in rats, or similar size rodents, is that one animal interferes with or may even damage the miniature endoscopic camera attached to the second animal.

NEW METHOD

We outline an inexpensive, lightweight, 3D-printed protector (iHELMET) that surrounds but is not in physical contact with the camera, together with details of its design and construction.

RESULTS

Using a simple design, we demonstrate successful protection of the endoscope and recording in a social situation such as the social dominance tube test.

COMPARISON WITH EXISTING METHODS

The helmet's 3D-printed dimensions can be readily adjusted to work with various micro-endoscopes, which may be more difficult for the only other system of which we are aware.

CONCLUSIONS

In addition to camera protection, features of the design aid camera stability, helping to secure more optimal imaging of calcium transients in specific regions of interest during long recording sessions.

Artificial intelligence assisted endocytoscopy: A novel eye in endoscopy

Over the past few years, emerging new approaches in endoscopic imaging technologies facilitate a high-quality assessment of lesions found in the gastrointestinal (GI) tract. Endocytoscopy (EC), as a novel tool in endoscopy, aids the more accurate evaluation of superficial mucosal surface. This review article aims to represent the most relevant information related to the latest EC technology and its clinical application in the lower GI tract diagnostic. We discuss EC-computer-aided diagnosis capability to differentiate between non-neoplastic and neoplastic lesion that offers a closer look to in-vivo assessment and diagnosis of cancerous tissue. Nevertheless, artificial-assisted EC diagnostics could also be employed with benefits in patients with inflammatory bowel disease (IBD) by accurately highlighting the presence of mucosal injury. In our review we included those studies comprising data about colonoscopy with narrow banding imaging and computer-aided diagnosis, as well as EC. Last but not least, artificial-assisted EC facilitates in-vivo diagnosis of the lower GI tract and may, in the future, remodel the field of in-vivo endoscopic diagnosis of colorectal lesions, representing another step towards the so-called optical biopsy.

Comparison in the Diagnostic Yield between "Pillcam SB3" Capsule Endoscopy and "OMOM Smart Capsule 2" in Small Bowel Bleeding: A Randomized Head-to-Head Study

INTRODUCTION

Capsule endoscopy (CE) is the first-line tool for diagnosis of small bowel bleeding. There are some studies that have compared different types of CE. OMOM CE is one of the newest in the market and has not been compared with other types of CE. The objective of this study was to compare the diagnostic yield of the Pillcam SB3 and OMOM CE in small bowel bleeding.

MATERIALS AND METHODS

This is a prospective, comparative, randomized, and blinded study. Patients with suspected small bowel bleeding were included. All the patients were given both types of CE in random order. Diagnostic yield and functionality between the 2 types of CE were analyzed.

RESULTS

We included 44 patients, 54.5% were female with a median age of 63.5 years. Battery time was significantly longer with SB3 (816.5 vs. 700.5 min, p < 0.001), and the download time was shorter with OMOM (33 vs. 132 min, p < 0.001). Both CEs presented 1 failure. The cause of the bleeding was identified in 39 SB3 (88.6%) and in 34 OMOM CE (77.3%) (p = 0.256). P2 lesions were observed in 32 SB3 (72.7%) and in 29 OMOM CE (65.9%) (p = 0.784). The agreement between both CEs for P2 lesions was moderate (κ = 0.628).

CONCLUSIONS

Pillcam SB3 and OMOM devices are safe procedures and have a similar diagnostic yield. Significant differences were observed in the battery life and download time with both CEs.

Convolutional Neural Network Technology in Endoscopic Imaging: Artificial Intelligence for Endoscopy

Recently, significant improvements have been made in artificial intelligence. The artificial neural network was introduced in the 1950s. However, because of the low computing power and insufficient datasets available at that time, artificial neural networks suffered from overfitting and vanishing gradient problems for training deep networks. This concept has become more promising owing to the enhanced big data processing capability, improvement in computing power with parallel processing units, and new algorithms for deep neural networks, which are becoming increasingly successful and attracting interest in many domains, including computer vision, speech recognition, and natural language processing. Recent studies in this technology augur well for medical and healthcare applications, especially in endoscopic imaging. This paper provides perspectives on the history, development, applications, and challenges of deep-learning technology.

PMN-PT/Epoxy 1-3 composite based ultrasonic transducer for dual-modality photoacoustic and ultrasound endoscopy

Endoscopic dual-modality photoacoustic (PA) and ultrasound (US) imaging has the capability of providing morphology and molecular information simultaneously. An ultrasonic transducer was applied for detecting PA signals and performing US imaging which determines the sensitivity and performance of a dual-modality PA/US system. In our study, a miniature single element 32-MHz lead magnesium niobate-lead titanate (PMN-PT) epoxy 1-3 composite based ultrasonic transducer was developed. A miniature endoscopic probe based on this transducer has been fabricated. Using the dual modality PA/US system with a PMN-PT/epoxy 1-3 composite based ultrasonic transducer, phantom and in vivo animal studies have been conducted to evaluate the performance. The preliminary results show enhanced bandwidths of the new ultrasonic transducer and improved signal-to-noise ratio of PA and US images of rat colorectal wall compared with PMN-PT and lead zirconate titanate (PZT) composite based ultrasonic transducers.

Optical Diagnosis of Colorectal Polyps: Recent Developments

PURPOSE OF REVIEW

Optical diagnosis of diminutive colorectal polyps has been recently proposed as an alternative to histopathologic diagnosis. Recent developments in imaging techniques, new classification systems, and the use of artificial intelligence have allowed for increased viability of optical diagnosis. This review provides an up-to-date overview of optical diagnosis recommendations, classifications, outcomes, and recent developments.

RECENT FINDINGS

There are currently seven major classification systems and three major society recommendations for quality benchmarks for optical diagnosis of diminutive polyps. The NICE classification has been extensively studied and meets quality benchmarks for most imaging techniques but does not allow for the diagnosis of sessile serrated polyps (SSPs). The SIMPLE classification has met quality benchmarks for NBI and i-Scan and allows for the diagnosis of SSPs. Other classification systems need to be further studied to validate effectiveness. Computer-assisted diagnosis of colorectal polyps is a very promising recent development with first studies showing that society-recommended quality benchmarks for real-time colonoscopies on patients are being met. Limitations include a non-negligible percentage of failure to diagnose, low specificity, and low number of real-time diagnostic studies. More research needs to be performed to further understand the value of artificial intelligence for optical polyp diagnosis. Optical diagnosis of diminutive colorectal polyps is currently a viable strategy for experienced endoscopists using validated classifications and imaging-enhanced endoscopy. Artificial intelligence-based diagnosis could make optical diagnosis widely applicable but is currently in its early developmental stage.

Enhancing the Detection of Barrett Esophagus

In Western countries, the incidence of esophageal adenocarcinoma has increased rapidly in parallel with its premalignant condition, Barrett esophagus (BE). Unlike colonoscopy, endoscopic screening for BE is not currently recommended for all patients; however, surveillance endoscopy is advocated for patients with established BE. Novel imaging and sampling techniques have been developed and investigated for the purpose of improving the detection of Barrett esophagus, dysplasia, and neoplasia. This article discusses several screening and surveillance techniques, including Seattle protocol, chromoendoscopy, electronic chromoendoscopy, wide area transepithelial sampling with 3-dimensional analysis, nonendoscopic sampling devices, and transnasal endoscopy.

Advanced Imaging for Barrett's Esophagus and Early Neoplasia: Surface and Subsurface Imaging for Diagnosis and Management

PURPOSE OF REVIEW

Esophageal adenocarcinoma bears one of the fastest rising incidence of any cancers and generally arises in the setting of gastroesophageal reflux and Barrett's esophagus. However, early detection of neoplasia can be challenging since most patients are asymptomatic until they progress to more advanced and less curable stages, and early dysplastic lesions can be small, multifocal, and difficult to detect. Clearly, new imaging tools are needed in light of sampling error associated with random biopsies, the current standard of practice.

RECENT FINDINGS

Advances in endoscopic imaging including virtual chromoendoscopy, confocal laser endomicroscopy, and subsurface imaging with optical coherence tomography have ushered in a new era for detecting subtle neoplastic lesions. Moreover, in light of esophagus-sparing treatments for neoplastic lesions, such tools are likely to guide ablation and follow-up management. While there is no ideal single imaging modality to facilitate improved detection, staging, ablation, and follow-up of patients with dysplastic Barrett's esophagus, new advances in available technology, the potential for multimodal imaging, and the use of computer-aided diagnosis and biomarkers all hold great promise for improving detection and treatment.

Randomised controlled trial comparing modified Sano’s and narrow band imaging international colorectal endoscopic classifications for colorectal lesions

AIM

To assess the utility of modified Sano′s (MS) vs the narrow band imaging international colorectal endoscopic (NICE) classification in differentiating colorectal polyps.

METHODS

Patients undergoing colonoscopy between 2013 and 2015 were enrolled in this trial. Based on the MS or the NICE classifications, patients were randomised for real-time endoscopic diagnosis. This was followed by biopsies, endoscopic or surgical resection. The endoscopic diagnosis was then compared to the final (blinded) histopathology. The primary endpoint was the sensitivity (Sn), specificity (Sp), positive predictive value (PPV) and negative predictive value (NPV) of differentiating neoplastic and non-neoplastic polyps (MS II/IIo / IIIa / IIIb vs I or NICE 1 vs 2/3). The secondary endpoints were “endoscopic resectability” (MS II/IIo/IIIa vs I/IIIb or NICE 2 vs 1/3), NPV for diminutive distal adenomas and prediction of post-polypectomy surveillance intervals.

RESULTS

A total of 348 patients were evaluated. The Sn, Sp, PPV and NPV in differentiating neoplastic polyps from non-neoplastic polyps were, 98.9%, 85.7%, 98.2% and 90.9% for MS; and 99.1%, 57.7%, 95.4% and 88.2% for NICE, respectively. The area under the receiver operating characteristic curve (AUC) for MS was 0.92 (95%CI: 0.86-0.98); and AUC for NICE was 0.78 (95%CI: 0.69, 0.88). The Sn, Sp, PPV and NPV in predicting “endoscopic resectability” were 98.9%, 86.1%, 97.8% and 92.5% for MS; and 98.6%, 66.7%, 94.7% and 88.9% for NICE, respectively. The AUC for MS was 0.92 (95%CI: 0.87-0.98); and the AUC for NICE was 0.83 (95%CI: 0.75-0.90). The AUC values were statistically different for both comparisons (P = 0.0165 and P = 0.0420, respectively). The accuracy for diagnosis of sessile serrated adenoma/polyp (SSA/P) with high confidence utilizing MS classification was 93.2%. The differentiation of SSA/P from other lesions achieved Sp, Sn, PPV and NPV of 87.2%, 91.5%, 89.6% and 98.6%, respectively. The NPV for predicting adenomas in diminutive rectosigmoid polyps (n = 150) was 96.6% and 95% with MS and NICE respectively. The calculated accuracy of post-polypectomy surveillance for MS group was 98.2% (167 out of 170) and for NICE group was 92.1% (139 out of 151).

CONCLUSION

The MS classification outperformed the NICE classification in differentiating neoplastic polyps and predicting endoscopic resectability. Both classifications met ASGE PIVI thresholds.

All-optical dual photoacoustic and optical coherence tomography intravascular probe

Intravascular imaging in percutaneous coronary interventions can be an invaluable tool in the treatment of coronary artery disease. It is of significant interest to provide molecular imaging contrast that is complementary to structural contrast provided by optical coherence tomography (OCT) and intravascular ultrasound imaging (IVUS). In this study, we developed a dual-modality intravascular imaging probe comprising a commercial OCT catheter and a high sensitivity fiber optic ultrasound sensor, to provide both photoacoustic (PA) and OCT imaging. With PA imaging, the lateral resolution varied from 18 μm to 40 μm; the axial resolution was consistently in the vicinity of 45 μm. We demonstrated the clinical potential of the probe with 2-D circumferential PA and OCT imaging, and with multispectral PA imaging.

Biomarkers of Barrett's Esophagus: From the Laboratory to Clinical Practice

The currently recommended approach to managing cancer risk for patients with Barrett's esophagus is endoscopic surveillance including a biopsy protocol to sample the esophageal tissue randomly to detect dysplasia. However, there are numerous limitations in this practice that rely on the histopathological grading of dysplasia alone to make clinical decisions. The availability of in silico models demonstrating the potential cost-effectiveness of biomarker-based stratification has increased interest in finding a clinically relevant "Barrett's biomarker." The success of endoscopic eradication therapy in preventing neoplastic progression of dysplastic Barrett's esophagus has promoted the desire to stratify non-dysplastic Barrett's esophagus to those with "high risk" that may benefit from endotherapy. Furthermore, on the other end of the spectrum, there is interest in searching for a "low risk" marker that may identify those that would not likely benefit from endoscopy screening or surveillance. This review highlights recent data from the genomics (r)evolution revealing new genetic biomarkers of susceptibility to the development of Barrett's esophagus and novel pathways for its neoplastic progression, addresses the development of new modes of tissue sampling and imaging to detect early neoplasia in Barrett's esophagus, and discusses current progress in moving biomarkers from the laboratory into clinical practice in the era of precision medicine.

Sessile Serrated Adenomas: How to Detect, Characterize and Resect

Serrated polyps are important contributors to the burden of colorectal cancers (CRC). These lesions were once considered to have no malignant potential, but currently up to 30% of all CRC are recognized to arise from the serrated neoplasia pathway. The primary premalignant lesions are sessile serrated adenomas/polyps (SSA/Ps), although traditional serrated adenomas are relatively uncommon. Compared to conventional adenomas, SSA/Ps are morphologically subtle with indistinct borders, may be difficult to detect endoscopically, are more prevalent than previously thought, are associated with synchronous and metachronous advanced neoplasia, and have a higher risk of incomplete resection. Although many lesions remain “dormant,” progressive disease is associated with the development of dysplasia and more rapid progression to CRC. As a result, SSA/Ps are strongly implicated in the development of interval cancers. These factors represent unique challenges that require a meticulous approach to their management. In this review, we summarize the contemporary literature on the characterization, detection and resection of SSA/Ps.

Gastroscopic features and serum gastrin in schizophrenics taking long-term medication

AIM: To investigate the gastroscopic features and serum gastrin in schizophrenics taking long-term medication.

METHODS: Sixty schizophrenics with digestive diseases treated at our hospital were divided randomly and equally into two groups according to medication course (group A: < 0.5 year; group C (> 1 year). Diabetes or high blood pressure patients taking long-term medication were included as controls (group B). All patients underwent painless gastroscopy and examination of fasting serum gastrin.

RESULTS: The rates of bile reflux gastritis and gastric stasis were significantly higher in group C (34.37% and 30.21%, respectively) than in groups A and B (P < 0.05). The level of serum gastrin was significantly lower in group C than in groups A and B (78.43 pg/mL ± 10.68 pg/mL vs 88.72 pg/mL ± 11.35 pg/mL, 90.14 pg/mL ± 9.57 pg/mL; P < 0.05).

CONCLUSION: Schizophrenics taking long-term medication tend to develop bile reflux gastritis, gastric stasis and lowered serum gastrin.

[Narrow band imaging for early diagnosis of epithelial dysplasias and microinvasive tumors in the upper aerodigestive tract]

The various stages of tumor growth are characterized by typical epithelial, vascular, and secondary connective tissue changes. Narrow band imaging (NBI) endoscopy is a minimally invasive imaging technique that presents vascular structures in particular at a higher contrast than white light endoscopy alone. In combination with high-resolution image recording and reproduction (high-definition television, HDTV; ultra-high definition, 4K), progress has been made in otolaryngological differential diagnostics, both pre- and intraoperatively. This progress represents an important step towards a so-called optical biopsy. Flexible endoscopy in combination with NBI allows detailed assessment of areas of the upper aerodigestive tract which are difficult to assess by rigid endoscopy. Papillomas, precancerous, and cancerous lesions are characterized by epithelial and connective tissue changes, as well as by typical perpendicular vascular changes. Systematic use of NBI is recommended in the differential diagnosis of malignant lesions of the upper aerodigestive tract. NBI also convinces by a significant improvement in pre- and intraoperative assessment of superficial resection margins. In particular, the combination of NBI and contact endoscopy (compact endoscopy) permits excellent therapeutic decisions during tumor surgery. Intraoperative determination of resection margins at unprecedented precision is possible. In addition, assessment of the form and extent of the perpendicular vessel loops stimulated by epithelial signaling enables differential diagnostic decisions to be made, approximating our goal of an optical biopsy.

Molecular markers and imaging tools to identify malignant potential in Barrett's esophagus

Due to its rapidly rising incidence and high mortality, esophageal adenocarcinoma is a major public health concern, particularly in Western countries. The steps involved in the progression from its predisposing condition, gastroesophageal reflux disease, to its premalignant disorder, Barrett’s esophagus, and to cancer, are incompletely understood. Current screening and surveillance methods are limited by the lack of population-wide utility, incomplete sampling of standard biopsies, and subjectivity of evaluation. Advances in endoscopic ablation have raised the hope of effective therapy for eradication of high-risk Barrett’s lesions, but improvements are needed in determining when to apply this treatment and how to follow patients clinically. Researchers have evaluated numerous potential molecular biomarkers with the goal of detecting dysplasia, with varying degrees of success. The combination of biomarker panels with epidemiologic risk factors to yield clinical risk scoring systems is promising. New approaches to sample tissue may also be combined with these biomarkers for less invasive screening and surveillance. The development of novel endoscopic imaging tools in recent years has the potential to markedly improve detection of small foci of dysplasia in vivo. Current and future efforts will aim to determine the combination of markers and imaging modalities that will most effectively improve the rate of early detection of high-risk lesions in Barrett’s esophagus.

A micro-optical system for endoscopy based on mechanical compensation paradigm using miniature piezo-actuation

The goal of the study was to investigate the feasibility of a novel miniaturized optical system for endoscopy. Fostering the mechanical compensation paradigm, the modeled optical system, composed by 14 lenses, separated in 4 different sets, had a total length of 15.55mm, an effective focal length ranging from 1.5 to 4.5mm with a zoom factor of about 2.8×, and an angular field of view up to 56°. Predicted maximum lens travel was less than 3.5mm. The consistency of the image plane height across the magnification range testified the zoom capability. The maximum predicted achromatic astigmatism, transverse spherical aberration, longitudinal spherical aberration and relative distortion were less than or equal to 25μm, 15μm, 35μm and 12%, respectively. Tests on tolerances showed that the manufacturing and opto-mechanics mounting are critical as little deviations from design dramatically decrease the optical performances. However, recent micro-fabrication technology can guarantee tolerances close to nominal design. A closed-loop actuation unit, devoted to move the zoom and the focus lens sets, was implemented adopting miniaturized squiggle piezo-motors and magnetic position encoders based on Hall effect. Performance results, using a prototypical test board, showed a positioning accuracy of less than 5μm along a lens travel path of 4.0mm, which was in agreement with the lens set motion features predicted by the analysis. In conclusion, this study demonstrated the feasibility of the optical design and the viability of the actuation approach while tolerances must be carefully taken into account.

New Endoscopic Techniques: Challenges and Opportunities for Surgical Pathologists

In recent years, significant clinical and technological advances have been made in endoscopic methods for diagnosis and treatment of early gastrointestinal neoplasms. However, essential information related to these novel techniques and their implications for practicing surgical pathologists have largely been missing in the general pathology literature. This article provides a general introduction to these novel therapeutic and diagnostic methods, and discusses their indications, contraindications, and potential limitations. The article aims to enable surgical pathologists to interact more efficiently with basic scientists and clinical colleagues to help implement and improve the existing clinical methods and to advance the new technologies.