Blood vessel detection and artery-vein differentiation using hyperspectral imaging

Toward real-time margin assessment in breast-conserving surgery with hyperspectral imaging

Margin assessment in breast-conserving surgery (BSC) remains a critical challenge, with 20-25% of cases resulting in inadequate tumor resection, increasing the risk of local recurrence and the need for additional treatment. In this study, we evaluate the diagnostic performance of hyperspectral imaging (HSI) as a non-invasive technique for assessing resection margins in ex vivo lumpectomy specimens. A dataset of over 200 lumpectomy specimens was collected using two hyperspectral cameras, and a classification algorithm was developed to distinguish between healthy and tumor tissue within margins of 0 and 2 mm. The proposed approach achieved its highest diagnostic performance at a 0 mm margin, with a sensitivity of 92%, specificity of 78%, accuracy of 83%, Matthews correlation coefficient of 68%, and an area under the curve of 89%. The entire resection surface could be imaged and evaluated within 10 minutes, providing a rapid and non-invasive alternative to conventional margin assessment techniques. These findings represent a significant advancement toward real-time intraoperative margin assessment, highlighting the potential of HSI to enhance surgical precision and reduce re-excision rates in BCS.

Separating Surface Reflectance from Volume Reflectance in Medical Hyperspectral Imaging

Hyperspectral imaging has shown great promise for diagnostic applications, particularly in cancer surgery. However, non-bulk tissue-related spectral variations complicate the data analysis. Common techniques, such as standard normal variate normalization, often lead to a loss of amplitude and scattering information. This study investigates a novel approach to address these spectral variations in hyperspectral images of optical phantoms and excised human breast tissue. Our method separates surface and volume reflectance, hypothesizing that spectral variability arises from significant variations in surface reflectance across pixels. An illumination setup was developed to measure samples with a hyperspectral camera from different axial positions but with identical zenith angles. This configuration, combined with a novel data analysis approach, allows for the estimation and separation of surface reflectance for each direction and volume reflectance across all directions. Validated with optical phantoms, our method achieved an 83% reduction in spectral variability. Its functionality was further demonstrated in excised human breast tissue. Our method effectively addresses variations caused by surface reflectance or glare while conserving surface reflectance information, which may enhance sample analysis and evaluation. It benefits samples with unknown refractive index spectra and can be easily adapted and applied across a wide range of fields where hyperspectral imaging is used.

Study on an Automatic Classification Method for Determining the Malignancy Grade of Glioma Pathological Sections Based on Hyperspectral Multi-Scale Spatial-Spectral Fusion Features

This study describes a novel method for grading pathological sections of gliomas. Our own integrated hyperspectral imaging system was employed to characterize 270 bands of cancerous tissue samples from microarray slides of gliomas. These samples were then classified according to the guidelines developed by the World Health Organization, which define the subtypes and grades of diffuse gliomas. We explored a hyperspectral feature extraction model called SMLMER-ResNet using microscopic hyperspectral images of brain gliomas of different malignancy grades. The model combines the channel attention mechanism and multi-scale image features to automatically learn the pathological organization of gliomas and obtain hierarchical feature representations, effectively removing the interference of redundant information. It also completes multi-modal, multi-scale spatial-spectral feature extraction to improve the automatic classification of glioma subtypes. The proposed classification method demonstrated high average classification accuracy (>97.3%) and a Kappa coefficient (0.954), indicating its effectiveness in improving the automatic classification of hyperspectral gliomas. The method is readily applicable in a wide range of clinical settings, offering valuable assistance in alleviating the workload of clinical pathologists. Furthermore, the study contributes to the development of more personalized and refined treatment plans, as well as subsequent follow-up and treatment adjustment, by providing physicians with insights into the underlying pathological organization of gliomas.

HyperVein: A Hyperspectral Image Dataset for Human Vein Detection

HyperSpectral Imaging (HSI) plays a pivotal role in various fields, including medical diagnostics, where precise human vein detection is crucial. HyperSpectral (HS) image data are very large and can cause computational complexities. Dimensionality reduction techniques are often employed to streamline HS image data processing. This paper presents a HS image dataset encompassing left- and right-hand images captured from 100 subjects with varying skin tones. The dataset was annotated using anatomical data to represent vein and non-vein areas within the images. This dataset is utilised to explore the effectiveness of dimensionality reduction techniques, namely: Principal Component Analysis (PCA), Folded PCA (FPCA), and Ward's Linkage Strategy using Mutual Information (WaLuMI) for vein detection. To generate experimental results, the HS image dataset was divided into train and test datasets. Optimum performing parameters for each of the dimensionality reduction techniques in conjunction with the Support Vector Machine (SVM) binary classification were determined using the Training dataset. The performance of the three dimensionality reduction-based vein detection methods was then assessed and compared using the test image dataset. Results show that the FPCA-based method outperforms the other two methods in terms of accuracy. For visualization purposes, the classification prediction image for each technique is post-processed using morphological operators, and results show the significant potential of HS imaging in vein detection.

Visualization of Subcutaneous Blood Vessels Based on Hyperspectral Imaging and Three-Wavelength Index Images

Blood vessel visualization technology allows nursing staff to transition from traditional palpation or touch to locate the subcutaneous blood vessels to visualized localization by providing a clear visual aid for performing various medical procedures accurately and efficiently involving blood vessels; this can further improve the first-attempt puncture success rate for nursing staff and reduce the pain of patients. We propose a novel technique for hyperspectral visualization of blood vessels in human skin. An experiment with six participants with different skin types, race, and nationality backgrounds is described. A mere separation of spectral layers for different skin types is shown to be insufficient. The use of three-wavelength indices in imaging has shown a significant improvement in the quality of results compared to using only two-wavelength indices. This improvement can be attributed to an increase in the contrast ratio, which can be as high as 25%. We propose and implement a technique for finding new index formulae based on an exhaustive search and a binary blood-vessel image obtained through an expert assessment. As a result of the search, a novel index formula was deduced, allowing high-contrast blood vessel images to be generated for any skin type.

In vivo evaluation of a hyperspectral imaging system for minimally invasive surgery (HSI-MIS)

BACKGROUND

Hyperspectral Imaging (HSI) is a reliable and safe imaging method for taking intraoperative perfusion measurements. This is the first study translating intraoperative HSI to an in vivo laparoscopic setting using a CE-certified HSI-system for minimally invasive surgery (HSI-MIS). We aim to compare it to an established HSI-system for open surgery (HSI-Open).

METHODS

Intraoperative HSI was done using the HSI-MIS and HSI-Open at the Region of Interest (ROI). 19 patients undergoing gastrointestinal resections were analyzed in this study. The HSI-MIS-acquired images were aligned with those from the HSI-Open, and spectra and parameter images were compared pixel-wise. We calculated the Mean Absolute Error (MAE) for Tissue Oxygen Saturation (StO₂), Near-Infrared Perfusion Index (NIR-PI), Tissue Water Index (TWI), and Organ Hemoglobin Index (OHI), as well as the Root Mean Squared Error (RMSE) over the whole spectrum. Our analysis of parameters was optimized using partial least squares (PLS) regression. Two experienced surgeons carried out an additional color-change analysis, comparing the ROI images and deciding whether they provided the same (acceptable) or different visual information (rejected).

RESULTS

HSI and subsequent image registration was possible in 19 patients. MAE results for the original calculation were StO_{2 orig.} 17.2% (± 7.7%)_, NIR-PI_orig. 16.0 (± 9.5), TWI_orig. 18.1 (± 7.9), OHI_orig. 14.4 (± 4.5). For the PLS calculation, they were StO_{2 PLS} 12.6% (± 5.2%), NIR-PI_PLS 10.3 (± 6.0), TWI_PLS 10.6 (± 5.1), and OHI_PLS 11.6 (± 3.0). The RMSE between both systems was 0.14 (± 0.06). In the color-change analysis; both surgeons accepted more images generated using the PLS method.

CONCLUSION

Intraoperative HSI-MIS is a new technology and holds great potential for future applications in surgery. Parameter deviations are attributable to technical differences and can be reduced by applying improved calculation methods. This study is an important step toward the clinical implementation of HSI for minimally invasive surgery.

Video: Clinical evaluation of a laparoscopic hyperspectral imaging system

Background

Hyperspectral imaging (HSI) during surgical procedures is a new method for perfusion quantification and tissue discrimination. Its use has been limited to open surgery due to large camera sizes, missing color video, or long acquisition times. A hand-held, laparoscopic hyperspectral camera has been developed now to overcome those disadvantages and evaluated clinically for the first time.

Methods

In a clinical evaluation study, gastrointestinal resectates of ten cancer patients were investigated using the laparoscopic hyperspectral camera. Reference data from corresponding anatomical regions were acquired with a clinically approved HSI system. An image registration process was executed that allowed for pixel-wise comparisons of spectral data and parameter images (StO₂: oxygen saturation of tissue, NIR PI: near-infrared perfusion index, OHI: organ hemoglobin index, TWI: tissue water index) provided by both camera systems. The mean absolute error (MAE) and root mean square error (RMSE) served for the quantitative evaluations. Spearman’s rank correlation between factors related to the study design like the time of spectral white balancing and MAE, respectively RMSE, was calculated.

Results

The obtained mean MAEs between the TIVITA® Tissue and the laparoscopic hyperspectral system resulted in StO₂: 11% ± 7%, NIR PI: 14±3, OHI: 14± 5, and TWI: 10 ± 2. The mean RMSE between both systems was 0.1±0.03 from 500 to 750 nm and 0.15 ±0.06 from 750 to 1000 nm. Spearman’s rank correlation coefficients showed no significant correlation between MAE or RMSE and influencing factors related to the study design.

Conclusion

Qualitatively, parameter images of the laparoscopic system corresponded to those of the system for open surgery. Quantitative deviations were attributed to technical differences rather than the study design. Limitations of the presented study are addressed in current large-scale in vivo trials.

Supplementary Information

The online version contains supplementary material available at 10.1007/s00464-022-09282-y.

Differentiation of Livestock Internal Organs Using Visible and Short-Wave Infrared Hyperspectral Imaging Sensors

Automatic identification and sorting of livestock organs in the meat processing industry could reduce costs and improve efficiency. Two hyperspectral sensors encompassing the visible (400-900 nm) and short-wave infrared (900-1700 nm) spectra were used to identify the organs by type. A total of 104 parenchymatous organs of cattle and sheep (heart, kidney, liver, and lung) were scanned in a multi-sensory system that encompassed both sensors along a conveyor belt. Spectral data were obtained and averaged following manual markup of three to eight regions of interest of each organ. Two methods were evaluated to classify organs: partial least squares discriminant analysis (PLS-DA) and random forest (RF). In addition, classification models were obtained with the smoothed reflectance and absorbance and the first and second derivatives of the spectra to assess if one was superior to the rest. The in-sample accuracy for the visible, short-wave infrared, and combination of both sensors was higher for PLS-DA compared to RF. The accuracy of the classification models was not significantly different between data pre-processing methods or between visible and short-wave infrared sensors. Hyperspectral sensors, particularly those in the visible spectrum, seem promising to identify organs from slaughtered animals which could be useful for the automation of quality and process control in the food supply chain, such as in abattoirs.

Detection of Collaterals from Cone-Beam CT Images in Stroke

Collateral vessels play an important role in the restoration of blood flow to the ischemic tissues of stroke patients, and the quality of collateral flow has major impact on reducing treatment delay and increasing the success rate of reperfusion. Due to high spatial resolution and rapid scan time, advance imaging using the cone-beam computed tomography (CBCT) is gaining more attention over the conventional angiography in acute stroke diagnosis. Detecting collateral vessels from CBCT images is a challenging task due to the presence of noises and artifacts, small-size and non-uniform structure of vessels. This paper presents a technique to objectively identify collateral vessels from non-collateral vessels. In our technique, several filters are used on the CBCT images of stroke patients to remove noises and artifacts, then multiscale top-hat transformation method is implemented on the pre-processed images to further enhance the vessels. Next, we applied three types of feature extraction methods which are gray level co-occurrence matrix (GLCM), moment invariant, and shape to explore which feature is best to classify the collateral vessels. These features are then used by the support vector machine (SVM), random forest, decision tree, and K-nearest neighbors (KNN) classifiers to classify vessels. Finally, the performance of these classifiers is evaluated in terms of accuracy, sensitivity, precision, recall, F-Measure, and area under the receiver operating characteristics curve. Our results show that all classifiers achieve promising classification accuracy above 90% and able to detect the collateral and non-collateral vessels from images.

O’Shanahan A, Carrera D, Bisshopp S, Sosa C, Marquez M, Camacho R, Plaza MDLL, Morera J, M. Callico G. Most Relevant Spectral Bands Identification for Brain Cancer Detection Using Hyperspectral Imaging

Hyperspectral imaging (HSI) is a non-ionizing and non-contact imaging technique capable of obtaining more information than conventional RGB (red green blue) imaging. In the medical field, HSI has commonly been investigated due to its great potential for diagnostic and surgical guidance purposes. However, the large amount of information provided by HSI normally contains redundant or non-relevant information, and it is extremely important to identify the most relevant wavelengths for a certain application in order to improve the accuracy of the predictions and reduce the execution time of the classification algorithm. Additionally, some wavelengths can contain noise and removing such bands can improve the classification stage. The work presented in this paper aims to identify such relevant spectral ranges in the visual-and-near-infrared (VNIR) region for an accurate detection of brain cancer using in vivo hyperspectral images. A methodology based on optimization algorithms has been proposed for this task, identifying the relevant wavelengths to achieve the best accuracy in the classification results obtained by a supervised classifier (support vector machines), and employing the lowest possible number of spectral bands. The results demonstrate that the proposed methodology based on the genetic algorithm optimization slightly improves the accuracy of the tumor identification in ~5%, using only 48 bands, with respect to the reference results obtained with 128 bands, offering the possibility of developing customized acquisition sensors that could provide real-time HS imaging. The most relevant spectral ranges found comprise between 440.5-465.96 nm, 498.71-509.62 nm, 556.91-575.1 nm, 593.29-615.12 nm, 636.94-666.05 nm, 698.79-731.53 nm and 884.32-902.51 nm.

In-Vivo and Ex-Vivo Tissue Analysis through Hyperspectral Imaging Techniques: Revealing the Invisible Features of Cancer

In contrast to conventional optical imaging modalities, hyperspectral imaging (HSI) is able to capture much more information from a certain scene, both within and beyond the visual spectral range (from 400 to 700 nm). This imaging modality is based on the principle that each material provides different responses to light reflection, absorption, and scattering across the electromagnetic spectrum. Due to these properties, it is possible to differentiate and identify the different materials/substances presented in a certain scene by their spectral signature. Over the last two decades, HSI has demonstrated potential to become a powerful tool to study and identify several diseases in the medical field, being a non-contact, non-ionizing, and a label-free imaging modality. In this review, the use of HSI as an imaging tool for the analysis and detection of cancer is presented. The basic concepts related to this technology are detailed. The most relevant, state-of-the-art studies that can be found in the literature using HSI for cancer analysis are presented and summarized, both in-vivo and ex-vivo. Lastly, we discuss the current limitations of this technology in the field of cancer detection, together with some insights into possible future steps in the improvement of this technology.

Use of Hyperspectral/Multispectral Imaging in Gastroenterology. Shedding Some⁻Different⁻Light into the Dark

Hyperspectral/Multispectral imaging (HSI/MSI) technologies are able to sample from tens to hundreds of spectral channels within the electromagnetic spectrum, exceeding the capabilities of human vision. These spectral techniques are based on the principle that every material has a different response (reflection and absorption) to different wavelengths. Thereby, this technology facilitates the discrimination between different materials. HSI has demonstrated good discrimination capabilities for materials in fields, for instance, remote sensing, pollution monitoring, field surveillance, food quality, agriculture, astronomy, geological mapping, and currently, also in medicine. HSI technology allows tissue observation beyond the limitations of the human eye. Moreover, many researchers are using HSI as a new diagnosis tool to analyze optical properties of tissue. Recently, HSI has shown good performance in identifying human diseases in a non-invasive manner. In this paper, we show the potential use of these technologies in the medical domain, with emphasis in the current advances in gastroenterology. The main aim of this review is to provide an overview of contemporary concepts regarding HSI technology together with state-of-art systems and applications in gastroenterology. Finally, we discuss the current limitations and upcoming trends of HSI in gastroenterology.

Hyperspectral imaging as a possible tool for visualization of changes in hemoglobin oxygenation in patients with deficient hemodynamics – proof of concept

There is a lack of imaging tools for the evaluation of spatial alterations in microcirculation including blood oxygen saturation and hemoglobin distribution but recent innovative developments in hyperspectral technology may offer a solution. We examined different hemodynamic disorders in patients suffering from scleroderma, Dupuytren surgery, chronic foot ulcera and skin infections. Superficial and deeper blood oxygen saturation, hemoglobin distribution and water content were determined using hyperspectral imaging (HSI). In the patient with scleroderma, distinct cutaneous low perfused regions correlated with macroscopic skin aspects and seem to be potential therapy control marker. With HSI accurate clinical evaluation of a macroscopic conspicuous wound after Dupuytren surgery was possible and influenced further surveillance decisions. HSI clearly revealed the spatial geometry and also the clinically related perfusion parameters of abscess formation and chronic ulcer wounds. The hemodynamically relevant parameters like blood oxygen saturation (1 mm to approx. 6 mm subcutaneous), total hemoglobin distribution and tissue water content can be easily determined and visualized with HSI in near real time. Hence, this technique seems to be suitable for routine diagnostics of acute and chronic wounds as well as for the examination of systemic hemodynamic disturbances. Special indications may be transplant surveillance and monitoring of therapeutical interventions.

Hyperspectral imaging solutions for brain tissue metabolic and hemodynamic monitoring: past, current and future developments

Hyperspectral imaging (HSI) technologies have been used extensively in medical research, targeting various biological phenomena and multiple tissue types. Their high spectral resolution over a wide range of wavelengths enables acquisition of spatial information corresponding to different light-interacting biological compounds. This review focuses on the application of HSI to monitor brain tissue metabolism and hemodynamics in life sciences. Different approaches involving HSI have been investigated to assess and quantify cerebral activity, mainly focusing on: (1) mapping tissue oxygen delivery through measurement of changes in oxygenated (HbO2) and deoxygenated (HHb) hemoglobin; and (2) the assessment of the cerebral metabolic rate of oxygen (CMRO2) to estimate oxygen consumption by brain tissue. Finally, we introduce future perspectives of HSI of brain metabolism, including its potential use for imaging optical signals from molecules directly involved in cellular energy production. HSI solutions can provide remarkable insight in understanding cerebral tissue metabolism and oxygenation, aiding investigation on brain tissue physiological processes.

Detecting brain tumor in pathological slides using hyperspectral imaging

Hyperspectral imaging (HSI) is an emerging technology for medical diagnosis. This research work presents a proof-of-concept on the use of HSI data to automatically detect human brain tumor tissue in pathological slides. The samples, consisting of hyperspectral cubes collected from 400 nm to 1000 nm, were acquired from ten different patients diagnosed with high-grade glioma. Based on the diagnosis provided by pathologists, a spectral library of normal and tumor tissues was created and processed using three different supervised classification algorithms. Results prove that HSI is a suitable technique to automatically detect high-grade tumors from pathological slides.

An Intraoperative Visualization System Using Hyperspectral Imaging to Aid in Brain Tumor Delineation

Hyperspectral imaging (HSI) allows for the acquisition of large numbers of spectral bands throughout the electromagnetic spectrum (within and beyond the visual range) with respect to the surface of scenes captured by sensors. Using this information and a set of complex classification algorithms, it is possible to determine which material or substance is located in each pixel. The work presented in this paper aims to exploit the characteristics of HSI to develop a demonstrator capable of delineating tumor tissue from brain tissue during neurosurgical operations. Improved delineation of tumor boundaries is expected to improve the results of surgery. The developed demonstrator is composed of two hyperspectral cameras covering a spectral range of 400–1700 nm. Furthermore, a hardware accelerator connected to a control unit is used to speed up the hyperspectral brain cancer detection algorithm to achieve processing during the time of surgery. A labeled dataset comprised of more than 300,000 spectral signatures is used as the training dataset for the supervised stage of the classification algorithm. In this preliminary study, thematic maps obtained from a validation database of seven hyperspectral images of in vivo brain tissue captured and processed during neurosurgical operations demonstrate that the system is able to discriminate between normal and tumor tissue in the brain. The results can be provided during the surgical procedure (~1 min), making it a practical system for neurosurgeons to use in the near future to improve excision and potentially improve patient outcomes.

Preliminary evaluation of near-infrared vein visualization technology in the screening of palatal blood vessels

Background

Avoidance from palatal blood vessel rupture is a major concern during the palatal soft tissue graft surgery. There is no defined chair-side and case-specific palatal blood vessel detection approach to facilitate the harvesting process. The objective of this pilot study is to assess the feasibility of a near-infrared vein visualization system in the screening process of palatal blood vessels.

Material and Methods

An extraoral vein visualization device (AccuVein AV400) was applied to a total of 304 hemi-maxilla of 152 individuals by two blind examiners. The study groups were classified according to their maximum inter-incisal measurements. The distances between the coronal border of the vessel image and the mid-palatal gingival margins of the adjacent teeth were measured and in each group. The correlations among the measurements were evaluated within groups.

Results

The blood vessel to the adjacent teeth measurements exhibited no statistical difference between both examiners in all subjects (p<0.001). Correlations between the examiners gradually increased in all groups as the mouth opening rates of the subjects were increased (p<0.001).

Conclusions

In the current state, screening of the palatal blood vessels via near-infrared vein visualization technology seems to be not suitable for every individual due to the restrictive effect of mouth opening. However, the promising results of this preliminary study demonstrated increasing consistency between the measurements of the examiners as the inter-incisal distance increase which emphasized the need an intraoral version of the device. Considering the lack of local decision-making technology for the detection of palatal blood vessels, further studies are required for development and optimization of these systems.

Key words:Near-infrared vein visualization, palatal graft harvesting, surgical complications.

HYPERSPECTRAL AUTOFLUORESCENCE IMAGING OF DRUSEN AND RETINAL PIGMENT EPITHELIUM IN DONOR EYES WITH AGE-RELATED MACULAR DEGENERATION

PURPOSE

To elucidate the molecular pathogenesis of age-related macular degeneration (AMD) and interpretation of fundus autofluorescence imaging, the authors identified spectral autofluorescence characteristics of drusen and retinal pigment epithelium (RPE) in donor eyes with AMD.

METHODS

Macular RPE/Bruch membrane flat mounts were prepared from 5 donor eyes with AMD. In 12 locations (1-3 per eye), hyperspectral autofluorescence images in 10-nm-wavelength steps were acquired at 2 excitation wavelengths (λex 436, 480 nm). A nonnegative tensor factorization algorithm was used to recover 5 abundant emission spectra and their corresponding spatial localizations.

RESULTS

At λex 436 nm, the authors consistently localized a novel spectrum (SDr) with a peak emission near 510 nm in drusen and sub-RPE deposits. Abundant emission spectra seen previously (S0 in Bruch membrane and S1, S2, and S3 in RPE lipofuscin/melanolipofuscin, respectively) also appeared in AMD eyes, with the same shapes and peak wavelengths as in normal tissue. Lipofuscin/melanolipofuscin spectra localizations in AMD eyes varied widely in their overlap with drusen, ranging from none to complete.

CONCLUSION

An emission spectrum peaking at ∼510 nm (λex 436 nm) appears to be sensitive and specific for drusen and sub-RPE deposits. One or more abundant spectra from RPE organelles exhibit characteristic relationships with drusen.

Rapid prototyping of biomimetic vascular phantoms for hyperspectral reflectance imaging

The emerging technique of rapid prototyping with three-dimensional (3-D) printers provides a simple yet revolutionary method for fabricating objects with arbitrary geometry. The use of 3-D printing for generating morphologically biomimetic tissue phantoms based on medical images represents a potentially major advance over existing phantom approaches. Toward the goal of image-defined phantoms, we converted a segmented fundus image of the human retina into a matrix format and edited it to achieve a geometry suitable for printing. Phantoms with vessel-simulating channels were then printed using a photoreactive resin providing biologically relevant turbidity, as determined by spectrophotometry. The morphology of printed vessels was validated by x-ray microcomputed tomography. Channels were filled with hemoglobin (Hb) solutions undergoing desaturation, and phantoms were imaged with a near-infrared hyperspectral reflectance imaging system. Additionally, a phantom was printed incorporating two disjoint vascular networks at different depths, each filled with Hb solutions at different saturation levels. Light propagation effects noted during these measurements—including the influence of vessel density and depth on Hb concentration and saturation estimates, and the effect of wavelength on vessel visualization depth—were evaluated. Overall, our findings indicated that 3-D-printed biomimetic phantoms hold significant potential as realistic and practical tools for elucidating light–tissue interactions and characterizing biophotonic system performance.