Automatic optical biopsy for colorectal cancer using hyperspectral imaging and artificial neural networks

Hyperspectral imaging combined with blood oxygen saturation for in vivo analysis of small intestinal necrosis tissue

Acute mesenteric ischemia (AMI) is a clinically significant vascular and gastrointestinal condition, which is closely related to the blood supply of the small intestine. Unfortunately, it is still challenging to properly discriminate small intestinal tissues with different degrees of ischemia. In this study, hyperspectral imaging (HSI) was used to construct pseudo-color images of oxygen saturation about small intestinal tissues and to discriminate different degrees of ischemia. First, several small intestine tissue models of New Zealand white rabbits were prepared and collected their hyperspectral data. Then, a set of isosbestic points were used to linearly transform the measurement data twice to match the reference spectra of oxyhemoglobin and deoxyhemoglobin, respectively. The oxygen saturation was measured at the characteristic peak band of oxyhemoglobin (560 nm). Ultimately, using the oxygenated hemoglobin reflectance spectrum as the benchmark, we obtained the relative amount of median oxygen saturation in normal tissues was 70.0 %, the IQR was 10.1 %, the relative amount of median oxygen saturation in ischemic tissues was 49.6 %, and the IQR was 14.6 %. The results demonstrate that HSI combined with the oxygen saturation computation method can efficiently differentiate between normal and ischemic regions of the small intestinal tissues. This technique provides a powerful support for internist to discriminate small bowel tissues with different degrees of ischemia, and also provides a new way of thinking for the diagnosis of AMI.

Surgical optomics: hyperspectral imaging and deep learning towards precision intraoperative automatic tissue recognition-results from the EX-MACHYNA trial

BACKGROUND

Hyperspectral imaging (HSI), combined with machine learning, can help to identify characteristic tissue signatures enabling automatic tissue recognition during surgery. This study aims to develop the first HSI-based automatic abdominal tissue recognition with human data in a prospective bi-center setting.

METHODS

Data were collected from patients undergoing elective open abdominal surgery at two international tertiary referral hospitals from September 2020 to June 2021. HS images were captured at various time points throughout the surgical procedure. Resulting RGB images were annotated with 13 distinct organ labels. Convolutional Neural Networks (CNNs) were employed for the analysis, with both external and internal validation settings utilized.

RESULTS

A total of 169 patients were included, 73 (43.2%) from Strasbourg and 96 (56.8%) from Verona. The internal validation within centers combined patients from both centers into a single cohort, randomly allocated to the training (127 patients, 75.1%, 585 images) and test sets (42 patients, 24.9%, 181 images). This validation setting showed the best performance. The highest true positive rate was achieved for the skin (100%) and the liver (97%). Misclassifications included tissues with a similar embryological origin (omentum and mesentery: 32%) or with overlaying boundaries (liver and hepatic ligament: 22%). The median DICE score for ten tissue classes exceeded 80%.

CONCLUSION

To improve automatic surgical scene segmentation and to drive clinical translation, multicenter accurate HSI datasets are essential, but further work is needed to quantify the clinical value of HSI. HSI might be included in a new omics science, namely surgical optomics, which uses light to extract quantifiable tissue features during surgery.

Fluorescence-guided surgery: comprehensive review

BACKGROUND

Despite significant improvements in preoperative workup and surgical planning, surgeons often rely on their eyes and hands during surgery. Although this can be sufficient in some patients, intraoperative guidance is highly desirable. Near-infrared fluorescence has been advocated as a potential technique to guide surgeons during surgery.

METHODS

A literature search was conducted to identify relevant articles for fluorescence-guided surgery. The literature search was performed using Medical Subject Headings on PubMed for articles in English until November 2022 and a narrative review undertaken.

RESULTS

The use of invisible light, enabling real-time imaging, superior penetration depth, and the possibility to use targeted imaging agents, makes this optical imaging technique increasingly popular. Four main indications are described in this review: tissue perfusion, lymph node assessment, anatomy of vital structures, and tumour tissue imaging. Furthermore, this review provides an overview of future opportunities in the field of fluorescence-guided surgery.

CONCLUSION

Fluorescence-guided surgery has proven to be a widely innovative technique applicable in many fields of surgery. The potential indications for its use are diverse and can be combined. The big challenge for the future will be in bringing experimental fluorophores and conjugates through trials and into clinical practice, as well as validation of computer visualization with large data sets. This will require collaborative surgical groups focusing on utility, efficacy, and outcomes for these techniques.

Artificial intelligence in improving the outcome of surgical treatment in colorectal cancer

Background

A considerable number of recent research have used artificial intelligence (AI) in the area of colorectal cancer (CRC). Surgical treatment of CRC still remains the most important curative component. Artificial intelligence in CRC surgery is not nearly as advanced as it is in screening (colonoscopy), diagnosis and prognosis, especially due to the increased complexity and variability of structures and elements in all fields of view, as well as a general shortage of annotated video banks for utilization.

Methods

A literature search was made and relevant studies were included in the minireview.

Results

The intraoperative steps which, at this moment, can benefit from AI in CRC are: phase and action recognition, excision plane navigation, endoscopy control, real-time circulation analysis, knot tying, automatic optical biopsy and hyperspectral imaging. This minireview also analyses the current advances in robotic treatment of CRC as well as the present possibility of automated CRC robotic surgery.

Conclusions

The use of AI in CRC surgery is still at its beginnings. The development of AI models capable of reproducing a colorectal expert surgeon's skill, the creation of large and complex datasets and the standardization of surgical colorectal procedures will contribute to the widespread use of AI in CRC surgical treatment.

Computer-Assisted Differentiation between Colon-Mesocolon and Retroperitoneum Using Hyperspectral Imaging (HSI) Technology

Complete mesocolic excision (CME), which involves the adequate resection of the tumor-bearing colonic segment with “en bloc” removal of its mesocolon along embryological fascial planes is associated with superior oncological outcomes. However, CME presents a higher complication rate compared to non-CME resections due to a higher risk of vascular injury. Hyperspectral imaging (HSI) is a contrast-free optical imaging technology, which facilitates the quantitative imaging of physiological tissue parameters and the visualization of anatomical structures. This study evaluates the accuracy of HSI combined with deep learning (DL) to differentiate the colon and its mesenteric tissue from retroperitoneal tissue. In an animal study including 20 pig models, intraoperative hyperspectral images of the sigmoid colon, sigmoid mesentery, and retroperitoneum were recorded. A convolutional neural network (CNN) was trained to distinguish the two tissue classes using HSI data, validated with a leave-one-out cross-validation process. The overall recognition sensitivity of the tissues to be preserved (retroperitoneum) and the tissues to be resected (colon and mesentery) was 79.0 ± 21.0% and 86.0 ± 16.0%, respectively. Automatic classification based on HSI and CNNs is a promising tool to automatically, non-invasively, and objectively differentiate the colon and its mesentery from retroperitoneal tissue.