Rapid multispectral endoscopic imaging system for near real-time mapping of the mucosa blood supply in the lung

Integrating multi-spectral imaging and Raman spectroscopy for in vivo endoscopic assessment of rat intestinal tract

An integrated system for in vivo multi-spectral imaging (MSI) and Raman spectroscopy was developed to understand the external morphology and internal molecular information of biological tissues. The achieved MSI images were reconstructed by eighteen separated images from 400 nm to 760 nm, whose illumination bands were selected with six tri-channel band filters. Based on the spectral analysis algorithms, the spatial distribution patterns of blood volume, blood oxygen content and tissue scatterer volume fraction were visualized. In vivo Raman spectral measurements were executed by inserting specially designed optical probe into instrumental channel of endoscope. By this way, the molecular composition at selected sampling points could be identified with its fingerprint spectral information under the guidance of molecular imaging modality. Therefore, both structural and compositional features of intestinal membrane could be addressed without labeling and continuously. The achieved results testified that our presented methodology reveals insights not easily extracted from either MSI or Raman spectroscopy individually, which brings the enrichment of biological and chemical meanings for future in vivo studies.

Studying on the in vivo pathological evolution of spinal cord injury with the rat model by the method of integrated multispectral imaging and Raman spectroscopy

Spinal cord injury (SCI) is a debilitating neurological and pathological condition that results in significant impairments in motor, sensory, and autonomic functions. By integrating multispectral imaging (MSI) with Raman spectroscopy, a label-free optical methodology was developed for achieving a non-invasive in vivo understanding on the pathological features of SCI evolution. Under the guidance of captured the spectral imaging data cube with a rigid endoscope based MSI system, a special designed fiber probe passed through the instrumental channel for acquiring the finger-print spectral information from compression rat SCI models. After identifying the main visual features of injured spinal cord tissue in all Sham, 0-, 3- and 7-days post injury (0 DPI, 3 DPI, and 7 DPI) groups, the blood volume and oxygen content were visualized to describe hemorrhage, hypoxia and inflammatory state after acute injury. The averaged reflectance spectra, which were deduced from MSI data cubes, were utilized for describing oxygen saturation and hemoglobin concentration in living tissue. The results of Raman spectroscopy addressed complex compositional and conformational phenomena during SCI progression, correlated with the well-known event like neuronal apoptosis, hemorrhage, demyelination, and even the upregulation of chondroitin sulfate proteoglycans (CSPGs). A principal component analysis and linear discriminate algorithm (PCA-LDA) based discriminate model was introduced for categorizing spectral features in different injury stages, which was applicable for intraoperative interpretations on the complex pathological courses of SCI and therapeutic outcomes.

Post-Surgical Non-Invasive Wound Healing Monitoring in Oropharyngeal Mucosa

Postoperative bleeding is the most significant complication of tonsillectomy. Regular monitoring of post-surgical wound healing in the pharynx is required. For this purpose, we propose endoscope-based non-invasive perfusion mapping and quantification. The combination of imaging photoplethysmography and image processing provides automated wound area selection and microcirculation characterization. In this feasibility study, we demonstrate the first results of the proposed approach to wound monitoring in clinical trial on eight patients after tonsillectomy. Combination of probe-based optical system and image processing algorithms can provide the valuable and consistent data on perfusion distribution. The quantitative microcirculation data obtained 1, 4, and 7 days after surgery are in good agreement with existing monitoring protocols.

Targeted multispectral filter array design for the optimization of endoscopic cancer detection in the gastrointestinal tract

Significance

Color differences between healthy and diseased tissue in the gastrointestinal (GI) tract are detected visually by clinicians during white light endoscopy; however, the earliest signs of cancer are often just a slightly different shade of pink compared to healthy tissue making it hard to detect. Improving contrast in endoscopy is important for early detection of disease in the GI tract during routine screening and surveillance.

Aim

We aim to target alternative colors for imaging to improve contrast using custom multispectral filter arrays (MSFAs) that could be deployed in an endoscopic "chip-on-tip" configuration.

Approach

Using an open-source toolbox, Opti-MSFA, we examined the optimal design of MSFAs for early cancer detection in the GI tract. The toolbox was first extended to use additional classification models (k -nearest neighbor, support vector machine, and spectral angle mapper). Using input spectral data from published clinical trials examining the esophagus and colon, we optimized the design of MSFAs with three to nine different bands.

Results

We examined the variation of the spectral and spatial classification accuracies as a function of the number of bands. The MSFA configurations tested showed good classification accuracies when compared to the full hyperspectral data available from the clinical spectra used in these studies.

Conclusion

The ability to retain good classification accuracies with a reduced number of spectral bands could enable the future deployment of multispectral imaging in an endoscopic chip-on-tip configuration using simplified MSFA hardware. Further studies using an expanded clinical dataset are needed to confirm these findings.

A rapid multispectral endoscopic imaging system for in vivo assessment of the morphological and physiological characteristics of mouse intestines

Accurate assessment of blood content in biological tissues is critical for the diagnosis and monitoring of various diseases, including cardiovascular disease, tumors, trauma, and the success rate of organ transplants. In this study, a multispectral endoscopic imaging system was built for capturing tissue reflection optical images in 18 bands across the wavelength range from 400 nm to 760 nm, non-invasively. The system was characterized by six tri-channel narrowband filters installed in front of the light source to achieve spectral separation and was equipped with a specially designed color CCD for achieving a speed of 24 multispectral imaging cubes per second. A method based on linear matrix inversion was proposed to calibrate the CCD spectral response overlaps, while a spectral analysis algorithm was developed for evaluating blood content and detecting tissue composition. The developed system was implemented in an in vivo mouse model for illustrating the blood volume, blood oxygen saturation index, and scattering particle size of the intestinal wall mucosa. The observations not only helped us to understand the blood supply situation in the intestinal mucosa, but also further testified the feasibility of our presented system. Meanwhile, the developed system could provide critical non-invasive optical information for intracavitary cancer diagnosis, surgery guidance, and treatment assessment.

First-in-human pilot study of snapshot multispectral endoscopy for early detection of Barrett's-related neoplasia

SIGNIFICANCE

The early detection of dysplasia in patients with Barrett's esophagus could improve outcomes by enabling curative intervention; however, dysplasia is often inconspicuous using conventional white-light endoscopy.

AIM

We sought to determine whether multispectral imaging (MSI) could be applied in endoscopy to improve detection of dysplasia in the upper gastrointestinal (GI) tract.

APPROACH

We used a commercial fiberscope to relay imaging data from within the upper GI tract to a snapshot MSI camera capable of collecting data from nine spectral bands. The system was deployed in a pilot clinical study of 20 patients (ClinicalTrials.gov NCT03388047) to capture 727 in vivo image cubes matched with gold-standard diagnosis from histopathology. We compared the performance of seven learning-based methods for data classification, including linear discriminant analysis, k-nearest neighbor classification, and a neural network.

RESULTS

Validation of our approach using a Macbeth color chart achieved an image-based classification accuracy of 96.5%. Although our patient cohort showed significant intra- and interpatient variance, we were able to resolve disease-specific contributions to the recorded MSI data. In classification, a combined principal component analysis and k-nearest-neighbor approach performed best, achieving accuracies of 95.8%, 90.7%, and 76.1%, respectively, for squamous, non-dysplastic Barrett's esophagus and neoplasia based on majority decisions per-image.

CONCLUSIONS

MSI shows promise for disease classification in Barrett's esophagus and merits further investigation as a tool in high-definition "chip-on-tip" endoscopes.

Laparoscopic system for simultaneous high-resolution video and rapid hyperspectral imaging in the visible and near-infrared spectral range

SIGNIFICANCE

Hyperspectral imaging (HSI) can support intraoperative perfusion assessment, the identification of tissue structures, and the detection of cancerous lesions. The practical use of HSI for minimal-invasive surgery is currently limited, for example, due to long acquisition times, missing video, or large set-ups.

AIM

An HSI laparoscope is described and evaluated to address the requirements for clinical use and high-resolution spectral imaging.

APPROACH

Reflectance measurements with reference objects and resected human tissue from 500 to 1000 nm are performed to show the consistency with an approved medical HSI device for open surgery. Varying object distances are investigated, and the signal-to-noise ratio (SNR) is determined for different light sources.

RESULTS

The handheld design enables real-time processing and visualization of HSI data during acquisition within 4.6 s. A color video is provided simultaneously and can be augmented with spectral information from push-broom imaging. The reflectance data from the HSI system for open surgery at 50 cm and the HSI laparoscope are consistent for object distances up to 10 cm. A standard rigid laparoscope in combination with a customized LED light source resulted in a mean SNR of 30 to 43 dB (500 to 950 nm).

CONCLUSIONS

Compact and rapid HSI with a high spatial- and spectral-resolution is feasible in clinical practice. Our work may support future studies on minimally invasive HSI to reduce intra- and postoperative complications.

Surgical spectral imaging

Recent technological developments have resulted in the availability of miniaturised spectral imaging sensors capable of operating in the multi- (MSI) and hyperspectral imaging (HSI) regimes. Simultaneous advances in image-processing techniques and artificial intelligence (AI), especially in machine learning and deep learning, have made these data-rich modalities highly attractive as a means of extracting biological information non-destructively. Surgery in particular is poised to benefit from this, as spectrally-resolved tissue optical properties can offer enhanced contrast as well as diagnostic and guidance information during interventions. This is particularly relevant for procedures where inherent contrast is low under standard white light visualisation. This review summarises recent work in surgical spectral imaging (SSI) techniques, taken from Pubmed, Google Scholar and arXiv searches spanning the period 2013-2019. New hardware, optimised for use in both open and minimally-invasive surgery (MIS), is described, and recent commercial activity is summarised. Computational approaches to extract spectral information from conventional colour images are reviewed, as tip-mounted cameras become more commonplace in MIS. Model-based and machine learning methods of data analysis are discussed in addition to simulation, phantom and clinical validation experiments. A wide variety of surgical pilot studies are reported but it is apparent that further work is needed to quantify the clinical value of MSI/HSI. The current trend toward data-driven analysis emphasises the importance of widely-available, standardised spectral imaging datasets, which will aid understanding of variability across organs and patients, and drive clinical translation.

A clinically translatable hyperspectral endoscopy (HySE) system for imaging the gastrointestinal tract

Hyperspectral imaging (HSI) enables visualisation of morphological and biochemical information, which could improve disease diagnostic accuracy. Unfortunately, the wide range of image distortions that arise during flexible endoscopy in the clinic have made integration of HSI challenging. To address this challenge, we demonstrate a hyperspectral endoscope (HySE) that simultaneously records intrinsically co-registered hyperspectral and standard-of-care white light images, which allows image distortions to be compensated computationally and an accurate hyperspectral data cube to be reconstructed as the endoscope moves in the lumen. Evaluation of HySE performance shows excellent spatial, spectral and temporal resolution and high colour fidelity. Application of HySE enables: quantification of blood oxygenation levels in tissue mimicking phantoms; differentiation of spectral profiles from normal and pathological ex vivo human tissues; and recording of hyperspectral data under freehand motion within an intact ex vivo pig oesophagus model. HySE therefore shows potential for enabling HSI in clinical endoscopy.

Quantitative evaluation of comb-structure correction methods for multispectral fibrescopic imaging

Removing the comb artifact introduced by imaging fibre bundles, or 'fibrescopes', for example in medical endoscopy, is essential to provide high quality images to the observer. Multispectral imaging (MSI) is an emerging method that combines morphological (spatial) and chemical (spectral) information in a single data 'cube'. When a fibrescope is coupled to a spectrally resolved detector array (SRDA) to perform MSI, comb removal is complicated by the demosaicking step required to reconstruct the multispectral data cube. To understand the potential for using SRDAs as multispectral imaging sensors in medical endoscopy, we assessed five comb correction methods with respect to five performance metrics relevant to biomedical imaging applications: processing time, resolution, smoothness, signal and the accuracy of spectral reconstruction. By assigning weights to each metric, which are determined by the particular imaging application, our results can be used to select the correction method to achieve best overall performance. In most cases, interpolation gave the best compromise between the different performance metrics when imaging using an SRDA.

Bayesian Estimation of Intrinsic Tissue Oxygenation and Perfusion From RGB Images

Multispectral imaging (MSI) can potentially assist the intra-operative assessment of tissue structure, function and viability, by providing information about oxygenation. In this paper, we present a novel technique for recovering intrinsic MSI measurements from endoscopic RGB images without custom hardware adaptations. The advantage of this approach is that it requires no modification to existing surgical and diagnostic endoscopic imaging systems. Our method uses a radiometric color calibration of the endoscopic camera's sensor in conjunction with a Bayesian framework to recover a per-pixel measurement of the total blood volume (THb) and oxygen saturation (SO₂) in the observed tissue. The sensor's pixel measurements are modeled as weighted sums over a mixture of Poisson distributions and we optimize the variables SO₂ and THb to maximize the likelihood of the observations. To validate our technique, we use synthetic images generated from Monte Carlo physics simulation of light transport through soft tissue containing sub-surface blood vessels. We also validate our method on in vivo data by comparing it to a MSI dataset acquired with a hardware system that sequentially images multiple spectral bands without overlap. Our results are promising and show that we are able to provide surgeons with additional relevant information by processing endoscopic images with our modeling and inference framework.