In vivo evaluation of a hyperspectral imaging system for minimally invasive surgery (HSI-MIS)

Background Light Suppression for Multispectral Imaging in Surgical Settings

Multispectral imaging (MSI) enables non-invasive tissue differentiation based on spectral characteristics and has shown great potential as a tool for surgical guidance. However, adapting MSI to open surgeries is challenging. Systems that rely on light sources present in the operating room experience limitations due to frequent lighting changes, which distort the spectral data and require countermeasures such as disruptive recalibrations. On the other hand, MSI systems that rely on dedicated lighting require external light sources, such as surgical lights, to be turned off during open surgery settings. This disrupts the surgical workflow and extends operation times. To this end, we present an approach that addresses these issues by combining active illumination with smart background suppression. By alternately capturing images with and without a modulated light source at a desired wavelength, we isolate the target signal, enabling artifact-free spectral scanning. We demonstrate the performance of our approach using a smart pixel camera, emphasizing its signal-to-noise ratio (SNR) advantage over a conventional high-speed camera. Our results show that accurate reflectance measurements can be achieved in clinical settings with high background illumination. Medical application is demonstrated through the estimation of blood oxygenation, and its suitability for open surgeries is discussed.

Advancing hyperspectral imaging and machine learning tools toward clinical adoption in tissue diagnostics: A comprehensive review

Hyperspectral imaging (HSI) has become an evident transformative apparatus in medical diagnostics. The review aims to appraise the present advancement and challenges in HSI for medical applications. It features a variety of medical applications namely diagnosing diabetic retinopathy, neurodegenerative diseases like Parkinson's and Alzheimer's, which illustrates its effectiveness in early diagnosis, early caries detection in periodontal disease, and dermatology by detecting skin cancer. Regardless of these advances, the challenges exist within every aspect that limits its broader clinical adoption. It has various constraints including difficulties with technology related to the complexity of the HSI system and needing specialist training, which may act as a drawback to its clinical settings. This article pertains to potential challenges expressed in medical applications and probable solutions to overcome these constraints. Successful companies that perform advanced solutions with HSI in terms of medical applications are being emphasized in this study to signal the high level of interest in medical diagnosis for systems to incorporate machine learning ML and artificial intelligence AI to foster precision diagnosis and standardized clinical workflow. This advancement signifies progressive possibilities of HSI in real-time clinical assessments. In conclusion despite HSI has been presented as a significant advanced medical imaging tool, addressing its limitations and probable solutions is for broader clinical adoption.

Advancing hyperspectral imaging and machine learning tools toward clinical adoption in tissue diagnostics: A comprehensive review

Hyperspectral imaging (HSI) has become an evident transformative apparatus in medical diagnostics. The review aims to appraise the present advancement and challenges in HSI for medical applications. It features a variety of medical applications namely diagnosing diabetic retinopathy, neurodegenerative diseases like Parkinson's and Alzheimer's, which illustrates its effectiveness in early diagnosis, early caries detection in periodontal disease, and dermatology by detecting skin cancer. Regardless of these advances, the challenges exist within every aspect that limits its broader clinical adoption. It has various constraints including difficulties with technology related to the complexity of the HSI system and needing specialist training, which may act as a drawback to its clinical settings. This article pertains to potential challenges expressed in medical applications and probable solutions to overcome these constraints. Successful companies that perform advanced solutions with HSI in terms of medical applications are being emphasized in this study to signal the high level of interest in medical diagnosis for systems to incorporate machine learning ML and artificial intelligence AI to foster precision diagnosis and standardized clinical workflow. This advancement signifies progressive possibilities of HSI in real-time clinical assessments. In conclusion despite HSI has been presented as a significant advanced medical imaging tool, addressing its limitations and probable solutions is for broader clinical adoption.

Endoscopic In Vivo Hyperspectral Imaging for Head and Neck Tumor Surgeries Using a Medically Approved CE-Certified Camera with Rapid Visualization During Surgery

Background: In vivo visualization of malignant tumors remains the main challenge during head and neck cancer surgery. This can result in inadequate tumor margin assessment and incomplete tumor resection, adversely affecting patient outcomes. Hyperspectral imaging (HSI) is a promising approach to address this issue. However, its application in surgery has been limited by the lack of medically approved HSI devices compliant with MDR regulations, as well as challenges regarding the integration into the surgical workflow. Methods: In this feasibility study, we employed endoscopic HSI during surgery to visualize the tumor sites of 12 head and neck cancer patients. We optimized the HSI workflow to minimize time required during surgery and to reduce the adaptation period needed for surgeons to adjust to the new workflow. Additionally, we implemented data processing to enable real-time classification and visualization of HSI within the intraoperative setting. HSI evaluation was conducted using principal component analysis and k-means clustering, with this clustering validated through comparison with expert annotations. Results: Our complete HSI workflow requires two to three minutes, with each HSI measurement-including evaluation and visualization-taking less than 10 s, achieving an accuracy of 79%, sensitivity of 72%, and specificity of 84%. Medical personnel became proficient with the HSI system after two surgeries. Conclusions: This study presents an HSI workflow for in vivo tissue differentiation during head and neck cancer surgery, providing accurate and visually accessible results within minimal time. This approach enhances the in vivo evaluation of tumor margins, leading to more clear margins and, consequently, improved patient outcomes.

Surgical Hyperspectral imaging and Indocyanine green Near-infrared Examination (SHINE) for brain arteriovenous malformation resection: a case report on how to visualize perfusion

Background and importance

Arteriovenous malformations (AVMs) are complex vascular anomalies that pose significant risks, including intracranial hemorrhage and neurological deficits. Surgical resection is the preferred treatment, requiring precise intraoperative imaging to ensure complete removal while preserving critical structures. This case report presents the first combined use of hyperspectral imaging (HSI) and indocyanine green video angiography (ICG VA) to visualize perfusion during brain AVM surgery, highlighting the potential benefits of these advanced imaging techniques.

Case description

A 66-year-old male presented with chronic headaches but no neurological deficits. MRI revealed a superficial AVM in the left frontal lobe within the superior frontal sulcus, measuring approximately 2.4 cm. The AVM was fed by feeders from the pericallosal artery, callosomarginal artery, and middle cerebral artery (MCA) branches, with drainage through a dilated cortical vein into the superior sagittal sinus. Preoperative embolization of two MCA feeding branches was performed, followed by microsurgical resection with ICG VA and HSI.

Conclusions

This case report demonstrates the successful application of HSI and ICG VA in brain AVM surgery. The combined use of these technologies provided comprehensive intraoperative assessment, enhancing surgical precision and safety. The integration of HSI offers non-invasive, contrast-agent-free imaging, potentially improving outcomes by enabling detailed perfusion mapping. Future studies should explore the broader applications of these imaging modalities in neurovascular practice.

Post-Surgical Non-Invasive Wound Healing Monitoring in Oropharyngeal Mucosa

Postoperative bleeding is the most significant complication of tonsillectomy. Regular monitoring of post-surgical wound healing in the pharynx is required. For this purpose, we propose endoscope-based non-invasive perfusion mapping and quantification. The combination of imaging photoplethysmography and image processing provides automated wound area selection and microcirculation characterization. In this feasibility study, we demonstrate the first results of the proposed approach to wound monitoring in clinical trial on eight patients after tonsillectomy. Combination of probe-based optical system and image processing algorithms can provide the valuable and consistent data on perfusion distribution. The quantitative microcirculation data obtained 1, 4, and 7 days after surgery are in good agreement with existing monitoring protocols.

Hyperspectral imaging of 4T1 mammary carcinomas grown in dorsal skinfold window chambers: spectral renormalization and fluorescence modeling

Significance

Hyperspectral imaging (HSI) of murine tumor models grown in dorsal skinfold window chambers (DSWCs) offers invaluable insight into the tumor microenvironment. However, light loss in a glass coverslip is often overlooked, and particular tissue characteristics are improperly modeled, leading to errors in tissue properties extracted from hyperspectral images.

Aim

We highlight the significance of spectral renormalization in HSI of DSWC models and demonstrate the benefit of incorporating enhanced green fluorescent protein (EGFP) excitation and emission in the skin tissue model for tumors expressing genes to produce EGFP.

Approach

We employed an HSI system for intravital imaging of mice with 4T1 mammary carcinoma in a DSWC over 14 days. We performed spectral renormalization of hyperspectral images based on the measured reflectance spectra of glass coverslips and utilized an inverse adding-doubling (IAD) algorithm with a two-layer murine skin model, to extract tissue parameters, such as total hemoglobin concentration and tissue oxygenation (StO 2 ). The model was upgraded to consider EGFP fluorescence excitation and emission. Moreover, we conducted additional experiments involving tissue phantoms, human forearm skin imaging, and numerical simulations.

Results

Hyperspectral image renormalization and the addition of EGFP fluorescence in the murine skin model reduced the mean absolute percentage errors (MAPEs) of fitted and measured spectra by up to 10% in tissue phantoms, 0.55% to 1.5% in the human forearm experiment and numerical simulations, and up to 0.7% in 4T1 tumors. Similarly, the MAPEs for tissue parameters extracted by IAD were reduced by up to 3% in human forearms and numerical simulations. For some parameters, statistically significant differences ( p < 0.05 ) were observed in 4T1 tumors. Ultimately, we have shown that fluorescence emission could be helpful for 4T1 tumor segmentation.

Conclusions

The results contribute to improving intravital monitoring of DWSC models using HSI and pave the way for more accurate and precise quantitative imaging.

Endoscopic Hyperspectral Imaging System to Discriminate Tissue Characteristics in Tissue Phantom and Orthotopic Mouse Pancreatic Tumor Model

In this study, we developed an endoscopic hyperspectral imaging (eHSI) system and evaluated its performance in analyzing tissues within tissue phantoms and orthotopic mouse pancreatic tumor models. Our custom-built eHSI system incorporated a liquid crystal tunable filter. To assess its tissue discrimination capabilities, we acquired images of tissue phantoms, distinguishing between fat and muscle regions. The system underwent supervised training using labeled samples, and this classification model was then applied to other tissue phantom images for evaluation. In the tissue phantom experiment, the eHSI effectively differentiated muscle from fat and background tissues. The precision scores regarding fat tissue classification were 98.3% for the support vector machine, 97.7% for the neural network, and 96.0% with a light gradient-boosting machine algorithm, respectively. Furthermore, we applied the eHSI system to identify tumors within an orthotopic mouse pancreatic tumor model. The F-score of each pancreatic tumor-bearing model reached 73.1% for the KPC tumor model and 63.1% for the Pan02 tumor models. The refined imaging conditions and optimization of the fine-tuning of classification algorithms enhance the versatility and diagnostic efficacy of eHSI in biomedical applications.

Intraoperative Laparoscopic Hyperspectral Imaging during Esophagectomy-A Pilot Study Evaluating Esophagogastric Perfusion at the Anastomotic Sites

Hyperspectral imaging (HSI) is a non-invasive and contactless technique that enables the real-time acquisition of comprehensive information on tissue within the surgical field. In this pilot study, we investigated whether a new HSI system for minimally-invasive surgery, TIVITA® Mini (HSI-MIS), provides reliable insights into tissue perfusion of the proximal and distal esophagogastric anastomotic sites during 21 laparoscopic/thoracoscopic or robotic Ivor Lewis esophagectomies of patients with cancer to minimize the risk of dreaded anastomotic insufficiency. In this pioneering investigation, physiological tissue parameters were derived from HSI measurements of the proximal site of the anastomosis (esophageal stump) and the distal site of the anastomosis (tip of the gastric conduit) during the thoracic phase of the procedure. Tissue oxygenation (StO2), Near Infrared Perfusion Index (NIR-PI), and Tissue Water Index (TWI) showed similar median values at both anastomotic sites. Significant differences were observed only for NIR-PI (median: 76.5 vs. 63.9; p = 0.012) at the distal site (gastric conduit) compared to our previous study using an HSI system for open surgery. For all 21 patients, reliable and informative measurements were attainable, confirming the feasibility of HSI-MIS to assess anastomotic viability. Further studies on the added benefit of this new technique aiming to reduce anastomotic insufficiency are warranted.

A dual-camera hyperspectral laparoscopic imaging system

Minimally invasive surgery (MIS) has expanded broadly in the field of abdominal and pelvic surgery. However, there are still prevalent issues surrounding intracorporeal surgery, such as iatrogenic injury, anastomotic leakage, or the presence of positive tumor margins after resection. Current approaches to address these issues and advance laparoscopic imaging techniques often involve fluorescence imaging agents, such as indocyanine green (ICG), to improve visualization, but these have drawbacks. Hyperspectral imaging (HSI) is an emerging optical imaging modality that takes advantage of spectral characteristics of different tissues. Various applications include tissue classification and digital pathology. In this study, we developed a dual-camera system for high-speed hyperspectral imaging. This includes the development of a custom application interface and corresponding hardware setup. Characterization of the system was performed, including spectral accuracy and spatial resolution, showing little sacrifice in speed for the approximate doubling of the covered spectral range, with our system acquiring 29 spectral images from 460-850 nm. Reference color tiles with various reflectance profiles were imaged and a RMSE of 3.56 ± 1.36% was achieved. Sub-millimeter resolution was shown at 7 cm working distance for both hyperspectral cameras. Finally, we image ex vivo tissues, including porcine stomach, liver, intestine, and kidney with our system and use a high-resolution, radiometrically calibrated spectrometer for comparison and evaluation of spectral fidelity. The dual-camera hyperspectral laparoscopic imaging system can have immediate applications in various surgeries.

Switchable LED-based laparoscopic multispectral system for rapid high-resolution perfusion imaging

Significance

Multispectral imaging (MSI) is an approach for real-time, quantitative, and non-invasive tissue perfusion measurements. Current laparoscopic systems based on mosaic sensors or filter wheels lack high spatial resolution or acceptable frame rates.

Aim

To develop a laparoscopic system for MSI-based color video and tissue perfusion imaging during gastrointestinal surgery without compromising spatial or temporal resolution.

Approach

The system was built with 14 switchable light-emitting diodes in the visible and near-infrared spectral range, a 4K image sensor, and a 10 mm laparoscope. Illumination patterns were created for tissue oxygenation and hemoglobin content monitoring. The system was calibrated to a clinically approved laparoscopic hyperspectral system using linear regression models and evaluated in an occlusion study with 36 volunteers.

Results

The root mean squared errors between the MSI and reference system were 0.073 for hemoglobin content, 0.039 for oxygenation in deeper tissue layers, and 0.093 for superficial oxygenation. The spatial resolution at a working distance of 45 mm was 156    μ m . The effective frame rate was 20 fps.

Conclusions

High-resolution perfusion monitoring was successfully achieved. Hardware optimizations will increase the frame rate. Parameter optimizations through alternative illumination patterns, regression, or assumed tissue models are planned. Intraoperative measurements must confirm the suitability during surgery.

Intra-abdominal laparoscopic assessment of organs perfusion using imaging photoplethysmography

BACKGROUND

An objective evaluation of the functional state and viability of biological tissues during minimally invasive surgery remains unsolved task. Various non-contact methods for evaluating perfusion during laparoscopic surgery are discussed in the literature, but so far there have been no reports of their use in clinical settings.

METHODS AND PATIENTS

Imaging photoplethysmography (iPPG) is a new method for quantitative assessment of perfusion distribution along the tissue. This is the first study in which we demonstrate successful use of iPPG to assess perfusion of organs during laparoscopic surgery in an operation theater. We used a standard rigid laparoscope connected to a standard digital monochrome camera, and abdominal organs were illuminated by green light. A distinctive feature is the synchronous recording of video frames and electrocardiogram with subsequent correlation data processing. During the laparoscopically assisted surgeries in nine cancer patients, the gradient of perfusion of the affected organs was evaluated. In particular, measurements were carried out before preparing a part of the intestine or stomach for resection, after anastomosis, or during physiological tests.

RESULTS

The spatial distribution of perfusion and its changes over time were successfully measured in all surgical cases. In particular, perfusion gradient of an intestine before resection was visualized and quantified by our iPPG laparoscope in all respective cases. It was also demonstrated that systemic administration of norepinephrine leads to a sharper gradient between well and poorly perfused areas of the colon. In four surgical cases, we have shown capability of the laparoscopic iPPG system for intra-abdominal assessment of perfusion in the anastomosed organs. Moreover, good repeatability of continuous long-term measurements of tissue perfusion inside the abdominal cavity was experimentally demonstrated.

CONCLUSION

Our study carried out in real clinical settings has shown that iPPG laparoscope is feasible for intra-abdominal visualization and quantitative assessment of perfusion distribution.

The Safe Values of Quantitative Perfusion Parameters of ICG Angiography Based on Tissue Oxygenation of Hyperspectral Imaging for Laparoscopic Colorectal Surgery: A Prospective Observational Study

BACKGROUND

Safe values for quantitative perfusion parameters of indocyanine green (ICG) angiography have not been fully defined, and interpretation remains at the surgeon's discretion. This prospective observational study aimed to establish the safe values for the quantitative perfusion parameters by comparing tissue oxygenation levels from HSI images in laparoscopic colorectal surgery.

METHODS

ICG angiography was performed using a laparoscopic near-infrared (NIR) camera system with ICG diluted in 10 mL of distilled water. For quantitative perfusion parameters, the changes in fluorescence intensity with perfusion times were analyzed to plot a time-fluorescence intensity graph. To assess real-time tissue oxygen saturation (StO2) in the colon, the TIVITA® Tissue System was utilized for hyperspectral imaging (HSI) acquisition. The StO2 levels were compared with the quantitative perfusion parameters derived from ICG angiography at corresponding points to define the safe range of ICG parameters reflecting good tissue oxygenation.

RESULTS

In the regression analysis, T1/2MAX, TMAX, slope, and NIR perfusion index were correlated with tissue oxygen saturation. Using this regression model, the cutoff values of quantitative perfusion parameters were calculated as T1/2MAX ≤ 10 s, TMAX ≤ 30 s, slope ≥ 5, and NIR perfusion index ≥50, which best reflected colon StO2 higher than 60%. Diagnostic values were analyzed to predict colon StO2 of 60% or more, and the ICG perfusion parameters T1/2MAX, TMAX, and perfusion TR showed high sensitivity values of 97% or more, indicating their ability to correctly identify cases with acceptable StO2.

CONCLUSION

The safe values for quantitative perfusion parameters derived from ICG angiography were T1/2MAX ≤ 10 s and TMAX ≤ 30 s, which were associated with colon tissue oxygenation levels higher than 60% in the laparoscopic colorectal surgery.