Medical hyperspectral imaging to facilitate residual tumor identification during surgery

Visible-short wavelength near infrared hyperspectral imaging coupled with multivariate curve resolution-alternating least squares for diagnosis of breast cancer

This study investigates the application of visible-short wavelength near-infrared hyperspectral imaging (Vis-SWNIR HSI) in the wavelength range of 400-950 nm and advanced chemometric techniques for diagnosing breast cancer (BC). The research involved 56 ex-vivo samples encompassing both cancerous and non-cancerous breast tissue from females. First, HSI images were analyzed using multivariate curve resolution-alternating least squares (MCR-ALS) to exploit pure spatial and spectral profiles of active components. Then, the MCR-ALS resolved spatial profiles were arranged in a new data matrix for exploration and discrimination between benign and cancerous tissue samples using principal component analysis (PCA) and partial least squares-discriminant analysis (PLS-DA). The PLS-DA classification accuracy of 82.1 % showed the potential of HSI and chemometrics for non-invasive detection of BC. Additionally, the resolved spectral profiles by MCR-ALS can be used to track the changes in the breast tissue during cancer and treatment. It is concluded that the proposed strategy in this work can effectively differentiate between cancerous and non-cancerous breast tissue and pave the way for further studies and potential clinical implementation of this innovative approach, offering a promising avenue for improving early detection and treatment outcomes in BC patients.

Hardware-accelerated integrated optoelectronic platform towards real-time high-resolution hyperspectral video understanding

Recent advancements in artificial intelligence have significantly expanded capabilities in processing language and images. However, the challenge of comprehensively understanding video content still needs to be solved. The main problem is the requirement to process real-time multidimensional video information at data rates exceeding 1 Tb/s, a demand that current hardware technologies cannot meet. This work introduces a hardware-accelerated integrated optoelectronic platform specifically designed for the real-time analysis of multidimensional video. By leveraging optical information processing within artificial intelligence hardware and combining it with advanced machine vision networks, the platform achieves data processing speeds of 1.2 Tb/s. This capability supports the analysis of hundreds of frequency bands with megapixel spatial resolution at video frame rates, significantly outperforming existing technologies in speed by three to four orders of magnitude. The platform demonstrates effectiveness for AI-driven tasks, such as video semantic segmentation and object understanding, across indoor and aerial scenarios. By overcoming the current data processing speed limitations, the platform shows promise in real-time AI video understanding, with potential implications for enhancing human-machine interactions and advancing cognitive processing technologies.

Acne Detection Based on Reconstructed Hyperspectral Images

Acne Vulgaris is a common type of skin disease that affects more than 85% of teenagers and frequently continues even in adulthood. While it is not a dangerous skin disease, it can significantly impact the quality of life. Hyperspectral imaging (HSI), which captures a wide spectrum of light, has emerged as a tool for the detection and diagnosis of various skin conditions. However, due to the high cost of specialised HS cameras, it is limited in its use in clinical settings. In this research, a novel acne detection system that will utilise reconstructed hyperspectral (HS) images from RGB images is proposed. A dataset of reconstructed HS images is created using the best-performing HS reconstruction model from our previous research. A new acne detection algorithm that is based on reconstructed HS images and RetinaNet algorithm is introduced. The results indicate that the proposed algorithm surpasses other techniques based on RGB images. Additionally, reconstructed HS images offer a promising and cost-effective alternative to using expensive HSI equipment for detecting conditions like acne or other medical issues.

HyperColorization: propagating spatially sparse noisy spectral clues for reconstructing hyperspectral images

Hyperspectral cameras face challenging spatial-spectral resolution trade-offs and are more affected by shot noise than RGB photos taken over the same total exposure time. Here, we present a colorization algorithm to reconstruct hyperspectral images from a grayscale guide image and spatially sparse spectral clues. We demonstrate that our algorithm generalizes to varying spectral dimensions for hyperspectral images, and show that colorizing in a low-rank space reduces compute time and the impact of shot noise. To enhance robustness, we incorporate guided sampling, edge-aware filtering, and dimensionality estimation techniques. Our method surpasses previous algorithms in various performance metrics, including SSIM, PSNR, GFC, and EMD, which we analyze as metrics for characterizing hyperspectral image quality. Collectively, these findings provide a promising avenue for overcoming the time-space-wavelength resolution trade-off by reconstructing a dense hyperspectral image from samples obtained by whisk or push broom scanners, as well as hybrid spatial-spectral computational imaging systems.

Targeted multispectral filter array design for the optimization of endoscopic cancer detection in the gastrointestinal tract

Significance

Color differences between healthy and diseased tissue in the gastrointestinal (GI) tract are detected visually by clinicians during white light endoscopy; however, the earliest signs of cancer are often just a slightly different shade of pink compared to healthy tissue making it hard to detect. Improving contrast in endoscopy is important for early detection of disease in the GI tract during routine screening and surveillance.

Aim

We aim to target alternative colors for imaging to improve contrast using custom multispectral filter arrays (MSFAs) that could be deployed in an endoscopic "chip-on-tip" configuration.

Approach

Using an open-source toolbox, Opti-MSFA, we examined the optimal design of MSFAs for early cancer detection in the GI tract. The toolbox was first extended to use additional classification models (k -nearest neighbor, support vector machine, and spectral angle mapper). Using input spectral data from published clinical trials examining the esophagus and colon, we optimized the design of MSFAs with three to nine different bands.

Results

We examined the variation of the spectral and spatial classification accuracies as a function of the number of bands. The MSFA configurations tested showed good classification accuracies when compared to the full hyperspectral data available from the clinical spectra used in these studies.

Conclusion

The ability to retain good classification accuracies with a reduced number of spectral bands could enable the future deployment of multispectral imaging in an endoscopic chip-on-tip configuration using simplified MSFA hardware. Further studies using an expanded clinical dataset are needed to confirm these findings.

Handheld hyperspectral imaging as a tool for the post-mortem interval estimation of human skeletal remains

In forensic medicine, estimating human skeletal remains' post-mortem interval (PMI) can be challenging. Following death, bones undergo a series of chemical and physical transformations due to their interactions with the surrounding environment. Post-mortem changes have been assessed using various methods, but estimating the PMI of skeletal remains could still be improved. We propose a new methodology with handheld hyperspectral imaging (HSI) system based on the first results from 104 human skeletal remains with PMIs ranging between 1 day and 2000 years. To differentiate between forensic and archaeological bone material, the Convolutional Neural Network analyzed 65.000 distinct diagnostic spectra: the classification accuracy was 0.58, 0.62, 0.73, 0.81, and 0.98 for PMIs of 0 week-2 weeks, 2 weeks-6 months, 6 months-1 year, 1 year-10 years, and >100 years, respectively. In conclusion, HSI can be used in forensic medicine to distinguish bone materials >100 years old from those <10 years old with an accuracy of 98%. The model has adequate predictive performance, and handheld HSI could serve as a novel approach to objectively and accurately determine the PMI of human skeletal remains.

Development of a custom optical imaging system for non-invasive monitoring and delineation of lower limb varicose veins using hyperspectral imaging and quantitative phase analysis

BACKGROUND

Varicose veins (VV) are a prevalent chronic venous disorder, particularly affecting women of childbearing age. This condition is associated with significant complications, including pain, discomfort, leg cramps, ulceration, reduced quality of life, absenteeism, and even mortality. This study aims to develop a custom non-invasive, non-contact optical imaging system combined with magnitude and phase image calculation to monitor and visualize varicose veins and their tributaries using hyperspectral imaging and quantitative phase analysis with a k-means clustering algorithm.

RESULTS

Ten volunteers participated in the optical imaging system study. They were exposed to a polychromatic light source spanning the wavelength range of 400 nm-950 nm. The diffuse reflection spectra for varicose veins exhibited a peak at 530 nm, while leg veins showed a peak at 780 nm. Hyperspectral images obtained at these specific wavelengths were normalized in order to homogenize the spectral signatures of each pixel (converting the hyperspectral image to 8 bit RGB image) and filtered using a moving average filter. Subsequently, the varicose veins and leg veins were delineated and detected using quantitative phase analysis and a k-means clustering algorithm.

CONCLUSION

In conclusion, the custom optical imaging system, utilizing hyperspectral imaging and the associated clustering algorithm, provides detailed information regarding the spatial distribution of varicose veins. This information can assist vascular physicians in facilitating easier diagnosis and treatment planning.

Tissue Classification of Breast Cancer by Hyperspectral Unmixing

(1) Background: Assessing the resection margins during breast-conserving surgery is an important clinical need to minimize the risk of recurrent breast cancer. However, currently there is no technique that can provide real-time feedback to aid surgeons in the margin assessment. Hyperspectral imaging has the potential to overcome this problem. To classify resection margins with this technique, a tissue discrimination model should be developed, which requires a dataset with accurate ground-truth labels. However, establishing such a dataset for resection specimens is difficult. (2) Methods: In this study, we therefore propose a novel approach based on hyperspectral unmixing to determine which pixels within hyperspectral images should be assigned to the ground-truth labels from histopathology. Subsequently, we use this hyperspectral-unmixing-based approach to develop a tissue discrimination model on the presence of tumor tissue within the resection margins of ex vivo breast lumpectomy specimens. (3) Results: In total, 372 measured locations were included on the lumpectomy resection surface of 189 patients. We achieved a sensitivity of 0.94, specificity of 0.85, accuracy of 0.87, Matthew's correlation coefficient of 0.71, and area under the curve of 0.92. (4) Conclusion: Using this hyperspectral-unmixing-based approach, we demonstrated that the measured locations with hyperspectral imaging on the resection surface of lumpectomy specimens could be classified with excellent performance.

Feasibility of a Real-Time Embedded Hyperspectral Compressive Sensing Imaging System

Hyperspectral imaging has been attracting considerable interest as it provides spectrally rich acquisitions useful in several applications, such as remote sensing, agriculture, astronomy, geology and medicine. Hyperspectral devices based on compressive acquisitions have appeared recently as an alternative to conventional hyperspectral imaging systems and allow for data-sampling with fewer acquisitions than classical imaging techniques, even under the Nyquist rate. However, compressive hyperspectral imaging requires a reconstruction algorithm in order to recover all the data from the raw compressed acquisition. The reconstruction process is one of the limiting factors for the spread of these devices, as it is generally time-consuming and comes with a high computational burden. Algorithmic and material acceleration with embedded and parallel architectures (e.g., GPUs and FPGAs) can considerably speed up image reconstruction, making hyperspectral compressive systems suitable for real-time applications. This paper provides an in-depth analysis of the required performance in terms of computing power, data memory and bandwidth considering a compressive hyperspectral imaging system and a state-of-the-art reconstruction algorithm as an example. The results of the analysis show that real-time application is possible by combining several approaches, namely, exploitation of system matrix sparsity and bandwidth reduction by appropriately tuning data value encoding.

Evaluation of Preprocessing Methods on Independent Medical Hyperspectral Databases to Improve Analysis

Currently, one of the most common causes of death worldwide is cancer. The development of innovative methods to support the early and accurate detection of cancers is required to increase the recovery rate of patients. Several studies have shown that medical Hyperspectral Imaging (HSI) combined with artificial intelligence algorithms is a powerful tool for cancer detection. Various preprocessing methods are commonly applied to hyperspectral data to improve the performance of the algorithms. However, there is currently no standard for these methods, and no studies have compared them so far in the medical field. In this work, we evaluated different combinations of preprocessing steps, including spatial and spectral smoothing, Min-Max scaling, Standard Normal Variate normalization, and a median spatial smoothing technique, with the goal of improving tumor detection in three different HSI databases concerning colorectal, esophagogastric, and brain cancers. Two machine learning and deep learning models were used to perform the pixel-wise classification. The results showed that the choice of preprocessing method affects the performance of tumor identification. The method that showed slightly better results with respect to identifing colorectal tumors was Median Filter preprocessing (0.94 of area under the curve). On the other hand, esophagogastric and brain tumors were more accurately identified using Min-Max scaling preprocessing (0.93 and 0.92 of area under the curve, respectively). However, it is observed that the Median Filter method smooths sharp spectral features, resulting in high variability in the classification performance. Therefore, based on these results, obtained with different databases acquired by different HSI instrumentation, the most relevant preprocessing technique identified in this work is Min-Max scaling.

Label-free hyperspectral imaging and deep-learning prediction of retinal amyloid β-protein and phosphorylated tau

Alzheimer's disease (AD) is a major risk for the aging population. The pathological hallmarks of AD-an abnormal deposition of amyloid β-protein (Aβ) and phosphorylated tau (pTau)-have been demonstrated in the retinas of AD patients, including in prodromal patients with mild cognitive impairment (MCI). Aβ pathology, especially the accumulation of the amyloidogenic 42-residue long alloform (Aβ42), is considered an early and specific sign of AD, and together with tauopathy, confirms AD diagnosis. To visualize retinal Aβ and pTau, state-of-the-art methods use fluorescence. However, administering contrast agents complicates the imaging procedure. To address this problem from fundamentals, ex-vivo studies were performed to develop a label-free hyperspectral imaging method to detect the spectral signatures of Aβ42 and pS396-Tau, and predicted their abundance in retinal cross-sections. For the first time, we reported the spectral signature of pTau and demonstrated an accurate prediction of Aβ and pTau distribution powered by deep learning. We expect our finding will lay the groundwork for label-free detection of AD.

Detection improvement of gliomas in hyperspectral imaging of protoporphyrin IX fluorescence - in vitro comparison of visual identification and machine thresholds

BACKGROUND

5-aminolevulinic acid (5-ALA) - precursor of protoporphyrin IX (PpIX) - is utilized in fluorescence guided surgery (FGS) of high-grade gliomas. PpIX is used to identify traces of glioma during resection. Visual inspection of the fluorescence seems inaccurate in comparison to optic techniques such as hyperspectral imaging (HSI).

AIM

To characterize the limits of PpIX fluorescence detection of (i) visual evaluation and (ii) HSI analysis and to (iii) develop a classification system for visible and non-visible PpIX fluorescence.

METHODS

Samples with increasing concentrations (C) of PpIX and non-fluorescent controls were evaluated using a surgical microscope under blue light illumination. Similar samples were imaged with a HSI system tuned to PpIX fluorescence peak wavelength (635 nm) and control (RGB) channels. Samples' intensities were defined, leading to 96 analysed pixels after batching.

RESULTS

Three expert neurosurgeons assessed the PpIX samples (n = 16) and controls (n = 8) with unanimous decisions (ICC = 0.704), resulting in 63% recognition rate, 48% sensitivity, 92% specificity, 92% positive predictive value (PPV) and 47% negative predictive value (NPV). HSI image analysis, comparing mean relative values, resulted in 96%, 100%, 86%, 94%, 100%, respectively. Minimum PpIX concentration detection for experts was 0.6-1.8 μmol/l and HSI's 0.03-0.15 μmol/l.

CONCLUSIONS

PpIX concentrations of low-grade gliomas, and those reported on glioblastoma infiltration zones, are below experts' detection threshold. HSI analysis exceeds the performance of expert's visual inspection nearly by 20-fold. Hybrid FGS-HSI systems should be investigated in parallel to long-term outcomes. Described methods are applicable as a standard for calibration, testing and development of subvisual FGS techniques.

Discriminating healthy from tumor tissue in breast lumpectomy specimens using deep learning-based hyperspectral imaging

Achieving an adequate resection margin during breast-conserving surgery remains challenging due to the lack of intraoperative feedback. Here, we evaluated the use of hyperspectral imaging to discriminate healthy tissue from tumor tissue in lumpectomy specimens. We first used a dataset obtained on tissue slices to develop and evaluate three convolutional neural networks. Second, we fine-tuned the networks with lumpectomy data to predict the tissue percentages of the lumpectomy resection surface. A MCC of 0.92 was achieved on the tissue slices and an RMSE of 9% on the lumpectomy resection surface. This shows the potential of hyperspectral imaging to classify the resection margins of lumpectomy specimens.

Identification of DAPI-stained normal, inflammatory, and carcinoma hepatic cells based on hyperspectral microscopy

Gross chromatin imbalance and high DNA content are distinct features of various types of cancer cells. However, severe inflammation can also produce similar symptoms in cells. In this study, normal, inflammatory, and carcinoma hepatic cells were stained with 4',6-diamidino-2-phenylindole (DAPI) and investigated by hyperspectral microscopy. DAPI is a DNA-sensitive fluorochrome. Therefore, the differences in the cellular DNA of the samples can be revealed by the corresponding fluorescence. Our experimental results demonstrate that although chromosomal disorder and high DNA content both occur in severely inflammatory and carcinoma hepatic cells, there is still a slight difference in their DNA, making their fluorescent intensity and even their spectral shapes distinguishable. Based on these spectral features, we developed a method for the precise identification of normal, inflammatory, and carcinoma hepatic cells in the field of view. The identification accuracy for these three types of cells was 99.8%. We believe that examination that combines DAPI staining with hyperspectral microscopy is a potential method for the identification and investigation of various types of cancer tissues.

New Intraoperative Imaging Tools and Image-Guided Surgery in Gastric Cancer Surgery

Innovations and new advancements in intraoperative real-time imaging have gained significant importance in the field of gastric cancer surgery in the recent past. Currently, the most promising procedures include indocyanine green fluorescence imaging (ICG-FI) and hyperspectral imaging or multispectral imaging (HSI, MSI). ICG-FI is utilized in a broad range of clinical applications, e.g., assessment of perfusion or lymphatic drainage, and additional implementations are currently investigated. HSI is still in the experimental phase and its value and clinical relevance require further evaluation, but initial studies have shown a successful application in perfusion assessment, and prospects concerning non-invasive tissue and tumor classification are promising. The application of machine learning and artificial intelligence technologies might enable an automatic evaluation of the acquired image data in the future. Both methods facilitate the accurate visualization of tissue characteristics that are initially indistinguishable for the human eye. By aiding surgeons in optimizing the surgical procedure, image-guided surgery can contribute to the oncologic safety and reduction of complications in gastric cancer surgery and recent advances hold promise for the application of HSI in intraoperative tissue diagnostics.

Oxygen saturation measurements using novel diffused reflectance with hyperspectral imaging: Towards facile COVID-19 diagnosis

Oxygen saturation level plays a vital role in screening, diagnosis, and therapeutic assessment of disease's assortment. There is an urgent need to design and implement early detection devices and applications for the COVID-19 pandemic; this study reports on the development of customized, highly sensitive, non-invasive, non-contact diffused reflectance system coupled with hyperspectral imaging for mapping subcutaneous blood circulation depending on its oxygen saturation level. The forearm of 15 healthy adult male volunteers with age range of (20-38 years) were illuminated via a polychromatic light source of a spectrum range 400-980 nm. Each patient had been scanned five times to calculate the mean spectroscopic reflectance images using hyperspectral camera. The customized signal processing algorithm includes normalization and moving average filter for noise removal. Afterward, employing K-means clustering for image segmentation to assess the accuracy of blood oxygen saturation (SpO₂) levels. The reliability of the developed diffused reflectance system was verified with the ground truth technique, a standard pulse oximeter. Non-invasive, non-contact diffused reflectance spectrum demonstrated maximum signal variation at 610 nm according to SpO₂ level. Statistical analysis (mean, standard deviation) of diffused reflectance hyperspectral images at 610 nm offered precise calibrated measurements to the standard pulse oximeter. Diffused reflectance associated with hyperspectral imaging is a prospective technique to assist with phlebotomy and vascular approach. Additionally, it could permit future surgical or pharmacological intercessions that titrate or limit ischemic injury continuously. Furthermore, this technique could offer a fast reliable indication of SpO₂ levels for COVID-19 diagnosis.

Plasmonic color filter array based visible light spectroscopy

Compared with traditional Fabry-Perot optical filters, plasmonic color filters could greatly remedy the complexity and reduce the cost of manufacturing. In this paper we present end-to-end demonstration of visible light spectroscopy based on highly selective plasmonic color filter array based on resonant grating structure. The spectra of 6 assorted samples were measured using an array of 20 narrowband color filters and detected signals were used to reconstruct original spectra by using new unmixing algorithm and by solving least squares problem with smoothing regularization. The original spectra were reconstructed with less than 0.137 root mean squared error. This works shows promise towards fully integrating plasmonic color filter array in imagers used in hyperspectral cameras.

Evaluation of hyperspectral imaging to quantify perfusion changes during the modified Allen test

OBJECTIVES

To evaluate the capability of hyperspectral imaging (HSI), a contact-less and noninvasive technology, to monitor perfusion changes of the hand during a modified Allen test (MAT) and cuff occlusion test. Furthermore, the study aimed at obtaining objective perfusion parameters of the hand.

METHODS

HSI of the hand was performed on 20 healthy volunteers with a commercially available HSI system during a MAT and a cuff occlusion test. Besides gathering red-green-blue (RGB) images, the perfusion parameters tissue hemoglobin index (THI), (superficial tissue) hemoglobin oxygenation (StO2), near-infrared perfusion (NIR), and tissue water index (TWI) were calculated for four different regions of interest on the hand. For the MAT, occlusion (OI; the ratio between the condition during occlusion and before occlusion) and reperfusion (RI; the ratio between the non-occlusion state and the prior occlusion state) indices were calculated for each perfusion parameter. All data were correlated to the clinical findings.

RESULTS

False-color images showed visible differences between the various perfusion conditions during the MAT and cuff occlusion test. THI, StO2, and NIR behaved as expected from physiology, while TWI did not in the context of this study. During rest, mean THI, StO2, and NIR of the hand were 34 ± 2, 72 ± 9, and 61 ± 6, respectively. The RI for THI showed a roundabout threefold increase after reperfusion of both radial and ulnar artery and was thus, distinctly pronounced when compared with StO2 and NIR (~1.25). The OI was lowest for THI when compared with StO2 and NIR.

CONCLUSIONS

HSI with its parameters THI, StO2, and NIR proved to be suitable to evaluate perfusion of the hand. By this, it could complement visual inspection during the MAT for evaluating the functionality of the superficial palmary arch before radial or ulnar artery harvest. The presented RI might deliver useful comparative values to detect pathological perfusion disorders at an early stage. As microcirculation monitoring is crucial for many medical issues, HSI shows potential to be used, besides further applications, in the monitoring of (free) flaps and transplants and microcirculation monitoring of critically ill patients.

Optical coherence hyperspectral microscopy with a single supercontinuum light source

In the paper, we have developed an optical coherence hyperspectral microscopy with a single supercontinuum light source. The microscopy consists of optical coherence tomography (OCT) and hyperspectral imaging (HSI), which can visualize the structural and functional characteristics of biological tissues. The 500 to 700 nm band is selected for HSI and OCT imaging, where HSI enables imaging of oxygen saturation and hemoglobin (Hb) content, while OCT acquires structural characteristics to assess the morphology of biological tissues. The system performance of the optical coherence hyperspectral microscopy is verified by normal mice ears, and the practical applications of the microscopy is further performed in 4T1 and inflammation Balb/c mice ears in vivo. The experimental results demonstrate that the microscopy has potential to provide complementary information for clinical applications.

Hyperspectral autofluorescence characterization of drusen and sub-RPE deposits in age-related macular degeneration

Background:

Soft drusen and basal linear deposit (BLinD) are two forms of the same extracellular lipid rich material that together make up an Oil Spill on Bruch’s membrane (BrM). Drusen are focal and can be recognized clinically. In contrast BLinD is thin and diffusely distributed, and invisible clinically, even on highest resolution OCT, but has been detected on en face hyperspectral autofluorescence (AF) imaging ex vivo. We sought to optimize histologic hyperspectral AF imaging and image analysis for recognition of drusen and sub-RPE deposits (including BLinD and basal laminar deposit), for potential clinical application.

Methods:

Twenty locations specifically with drusen and 12 additional locations specifically from fovea, perifovea and mid-periphery from RPE/BrM flatmounts from 4 AMD donors underwent hyperspectral AF imaging with 4 excitation wavelengths (λ_ex 436, 450, 480 and 505 nm), and the resulting image cubes were simultaneously decomposed with our published non-negative matrix factorization (NMF). Rank 4 recovery of 4 emission spectra was chosen for each excitation wavelength.

Results:

A composite emission spectrum, sensitive and specific for drusen and presumed sub-RPE deposits (the SDr spectrum) was recovered with peak at 510–520 nm in all tissues with drusen, with greatest amplitudes at excitations λ_ex 436, 450 and 480 nm. The RPE spectra of combined sources Lipofuscin (LF)/Melanolipofuscin (MLF) were of comparable amplitude and consistently recapitulated the spectra S1, S2 and S3 previously reported from all tissues: tissues with drusen, foveal and extra-foveal locations.

Conclusions:

A clinical hyperspectral AF camera, with properly chosen excitation wavelengths in the blue range and a hyperspectral AF detector, should be capable of detecting and quantifying drusen and sub-RPE deposits, the earliest known lesions of AMD, before any other currently available imaging modality.

Fusing spectral and spatial information with 2-D stationary wavelet transform (SWT 2-D) for a deeper exploration of spectroscopic images

Nowadays, it is clear that there is an increasing importance in spectroscopic imaging in all fields of science. Obviously, one bulk analysis can no longer be satisfactory, as the interest focuses more on the chemical nature and the location of the compounds present within a given complex matrix. This is, evidently, due to the fact that for a more comprehensive exploration of complex samples, one single acquired hyperspectral data cube can provide both spectral and spatial information simultaneously. Although many techniques were proposed by the chemometric community in explorations of these specific datasets, unfortunately, they are almost always focusing on spectral information, even if chemical images were ultimately observed. In other words, spatial information is not well exploited, and therefore lost during the actual chemometric calculation phase. The goal of this short communication is to present a very simple and fast spectral/spatial fusion approach based on 2-D stationary wavelet transform (SWT 2-D) which is able to improve the obtainable information, compared with a classical data analysis, in which the spatial domain would not be considered nor used.

Computer Vision in the Operating Room: Opportunities and Caveats

Effectiveness of computer vision techniques has been demonstrated through a number of applications, both within and outside healthcare. The operating room environment specifically is a setting with rich data sources compatible with computational approaches and high potential for direct patient benefit. The aim of this review is to summarize major topics in computer vision for surgical domains. The major capabilities of computer vision are described as an aid to surgical teams to improve performance and contribute to enhanced patient safety. Literature was identified through leading experts in the fields of surgery, computational analysis and modeling in medicine, and computer vision in healthcare. The literature supports the application of computer vision principles to surgery. Potential applications within surgery include operating room vigilance, endoscopic vigilance, and individual and team-wide behavioral analysis. To advance the field, we recommend collecting and publishing carefully annotated datasets. Doing so will enable the surgery community to collectively define well-specified common objectives for automated systems, spur academic research, mobilize industry, and provide benchmarks with which we can track progress. Leveraging computer vision approaches through interdisciplinary collaboration and advanced approaches to data acquisition, modeling, interpretation, and integration promises a powerful impact on patient safety, public health, and financial costs.

Excitation-scanning hyperspectral video endoscopy: enhancing the light at the end of the tunnel

Colorectal cancer is the 3^rd leading cancer for incidence and mortality rates. Positive treatment outcomes have been associated with early detection; however, early stage lesions have limited contrast to surrounding mucosa. A potential technology to enhance early stagise detection is hyperspectral imaging (HSI). While HSI technologies have been previously utilized to detect colorectal cancer ex vivo or post-operation, they have been difficult to employ in real-time endoscopy scenarios. Here, we describe an LED-based multifurcated light guide and spectral light source that can provide illumination for spectral imaging at frame rates necessary for video-rate endoscopy. We also present an updated light source optical ray-tracing model that resulted in further optimization and provided a ∼10X light transmission increase compared to the initial prototype. Future work will iterate simulation and benchtop testing of the hyperspectral endoscopic system to achieve the goal of video-rate spectral endoscopy.

A review of the medical hyperspectral imaging systems and unmixing algorithms' in biological tissues

Hyperspectral fluorescence imaging (HFI) is a well-known technique in the medical research field and is considered a non-invasive tool for tissue diagnosis. This review article gives a brief introduction to acquisition methods, including the image preprocessing methods, feature selection and extraction methods, data classification techniques and medical image analysis along with recent relevant references. The process of fusion of unsupervised unmixing techniques with other classification methods, like the combination of support vector machine with an artificial neural network, the latest snapshot Hyperspectral imaging (HSI) and vortex analysis techniques are also outlined. Finally, the recent applications of hyperspectral images in cellular differentiation of various types of cancer are discussed.

Parallel Classification Pipelines for Skin Cancer Detection Exploiting Hyperspectral Imaging on Hybrid Systems

The early detection of skin cancer is of crucial importance to plan an effective therapy to treat the lesion. In routine medical practice, the diagnosis is based on the visual inspection of the lesion and it relies on the dermatologists’ expertise. After a first examination, the dermatologist may require a biopsy to confirm if the lesion is malignant or not. This methodology suffers from false positives and negatives issues, leading to unnecessary surgical procedures. Hyperspectral imaging is gaining relevance in this medical field since it is a non-invasive and non-ionizing technique, capable of providing higher accuracy than traditional imaging methods. Therefore, the development of an automatic classification system based on hyperspectral images could improve the medical practice to distinguish pigmented skin lesions from malignant, benign, and atypical lesions. Additionally, the system can assist general practitioners in first aid care to prevent noncritical lesions from reaching dermatologists, thereby alleviating the workload of medical specialists. In this paper is presented a parallel pipeline for skin cancer detection that exploits hyperspectral imaging. The computational times of the serial processing have been reduced by adopting multicore and many-core technologies, such as OpenMP and CUDA paradigms. Different parallel approaches have been combined, leading to the development of fifteen classification pipeline versions. Experimental results using in-vivo hyperspectral images show that a hybrid parallel approach is capable of classifying an image of 50 × 50 pixels with 125 bands in less than 1 s.

Dark-field hyperspectral imaging for label free detection of nano-bio-materials

Nanomaterials are playing an increasingly important role in cancer diagnosis and treatment. Nanoparticle (NP)-based technologies have been utilized for targeted drug delivery during chemotherapies, photodynamic therapy, and immunotherapy. Another active area of research is the toxicity studies of these nanomaterials to understand the cellular uptake and transport of these materials in cells, tissues, and environment. Traditional techniques such as transmission electron microscopy, and mass spectrometry to analyze NP-based cellular transport or toxicity effect are expensive, require extensive sample preparation, and are low-throughput. Dark-field hyperspectral imaging (DF-HSI), an integration of spectroscopy and microscopy/imaging, provides the ability to investigate cellular transport of these NPs and to quantify the distribution of them within bio-materials. DF-HSI also offers versatility in non-invasively monitoring microorganisms, single cell, and proteins. DF-HSI is a low-cost, label-free technique that is minimally invasive and is a viable choice for obtaining high-throughput quantitative molecular analyses. Multimodal imaging modalities such as Fourier transform infrared and Raman spectroscopy are also being integrated with HSI systems to enable chemical imaging of the samples. HSI technology is being applied in surgeries to obtain molecular information about the tissues in real-time. This article provides brief overview of fundamental principles of DF-HSI and its application for nanomaterials, protein-detection, single-cell analysis, microbiology, surgical procedures along with technical challenges and future integrative approach with other imaging and measurement modalities. This article is categorized under: Diagnostic Tools > in vitro Nanoparticle-Based Sensing Diagnostic Tools > in vivo Nanodiagnostics and Imaging Implantable Materials and Surgical Technologies > Nanoscale Tools and Techniques in Surgery.

Spectrally Tunable Neural Network-Assisted Segmentation of Microneurosurgical Anatomy

Background

Distinct tissue types are differentiated based on the surgeon’s knowledge and subjective visible information, typically assisted with white-light intraoperative imaging systems. Narrow-band imaging (NBI) assists in tissue identification and enables automated classifiers, but many anatomical details moderate computational predictions and cause bias. In particular, tissues’ light-source-dependent optical characteristics, anatomical location, and potentially hazardous microstructural changes such as peeling have been overlooked in previous literature.

Methods

Narrow-band images of five (n = 5) facial nerves (FNs) and internal carotid arteries (ICAs) were captured from freshly frozen temporal bones. The FNs were split into intracranial and intratemporal samples, and ICAs’ adventitia was peeled from the distal end. Three-dimensional (3D) spectral data were captured by a custom-built liquid crystal tunable filter (LCTF) spectral imaging (SI) system. We investigated the normal variance between the samples and utilized descriptive and machine learning analysis on the image stack hypercubes.

Results

Reflectance between intact and peeled arteries in lower-wavelength domains between 400 and 576 nm was significantly different (p < 0.05). Proximal FN could be differentiated from distal FN in a higher range, 490–720 nm (p < 0.001). ICA with intact tunica differed from proximal FN nearly thorough the VIS range, 412–592 nm (p < 0.001) and 664–720 nm (p < 0.05) as did its distal counterpart, 422–720 nm (p < 0.001). The availed U-Net algorithm classified 90.93% of the pixels correctly in comparison to tissue margins delineated by a specialist.

Conclusion

Selective NBI represents a promising method for assisting tissue identification and computational segmentation of surgical microanatomy. Further multidisciplinary research is required for its clinical applications and intraoperative integration.

Surgical spectral imaging

Recent technological developments have resulted in the availability of miniaturised spectral imaging sensors capable of operating in the multi- (MSI) and hyperspectral imaging (HSI) regimes. Simultaneous advances in image-processing techniques and artificial intelligence (AI), especially in machine learning and deep learning, have made these data-rich modalities highly attractive as a means of extracting biological information non-destructively. Surgery in particular is poised to benefit from this, as spectrally-resolved tissue optical properties can offer enhanced contrast as well as diagnostic and guidance information during interventions. This is particularly relevant for procedures where inherent contrast is low under standard white light visualisation. This review summarises recent work in surgical spectral imaging (SSI) techniques, taken from Pubmed, Google Scholar and arXiv searches spanning the period 2013-2019. New hardware, optimised for use in both open and minimally-invasive surgery (MIS), is described, and recent commercial activity is summarised. Computational approaches to extract spectral information from conventional colour images are reviewed, as tip-mounted cameras become more commonplace in MIS. Model-based and machine learning methods of data analysis are discussed in addition to simulation, phantom and clinical validation experiments. A wide variety of surgical pilot studies are reported but it is apparent that further work is needed to quantify the clinical value of MSI/HSI. The current trend toward data-driven analysis emphasises the importance of widely-available, standardised spectral imaging datasets, which will aid understanding of variability across organs and patients, and drive clinical translation.

Use of Fluorescent Dyes in Endoscopy and Diagnostic Investigation

Background

The advancement of innovative endoscopic technology in terms of improving the visualization of the mucosa has been of significant benefit.

Summary

Advancements in image resolution, software processing, and optical filter technology have resulted in several techniques complemental to traditional white light endoscopy. These new techniques provide a real-time optical diagnosis as well as virtual histology of detected lesions. Optical molecular imaging permits a functional assessment within cells.

Key Message

Optical molecular imaging provides an understanding of cellular processes and permits validation of the specificity of fluorescent tracers and the possibility of quantifying the signal.

A background correction method to compensate illumination variation in hyperspectral imaging

Hyperspectral imaging (HSI) can measure both spatial (morphological) and spectral (biochemical) information from biological tissues. While HSI appears promising for biomedical applications, interpretation of hyperspectral images can be challenging when data is acquired in complex biological environments. Variations in surface topology or optical power distribution at the sample, encountered for example during endoscopy, can lead to errors in post-processing of the HSI data, compromising disease diagnostic capabilities. Here, we propose a background correction method to compensate for such variations, which estimates the optical properties of illumination at the target based on the normalised spectral profile of the light source and the measured HSI intensity values at a fixed wavelength where the absorption characteristics of the sample are relatively low (in this case, 800 nm). We demonstrate the feasibility of the proposed method by imaging blood samples, tissue-mimicking phantoms, and ex vivo chicken tissue. Moreover, using synthetic HSI data composed from experimentally measured spectra, we show the proposed method would improve statistical analysis of HSI data. The proposed method could help the implementation of HSI techniques in practical clinical applications, where controlling the illumination pattern and power is difficult.

Detection of Tumoral Epithelial Lesions Using Hyperspectral Imaging and Deep Learning

We propose a new method for the analysis and classification of HSI images. The method uses deep learning to interpret the molecular vibrational behaviour of healthy and tumoral human epithelial tissue, based on data gathered via SWIR (short-wave infrared) spectroscopy. We analyzed samples of Melanoma, Dysplastic Nevus and healthy skin. Preliminary results show that human epithelial tissue is sensitive to SWIR to the point of making possible the differentiation between healthy and tumor tissues. We conclude that HSI-SWIR can be used to build new methods for tumor classification.

O’Shanahan A, Carrera D, Bisshopp S, Sosa C, Marquez M, Camacho R, Plaza MDLL, Morera J, M. Callico G. Most Relevant Spectral Bands Identification for Brain Cancer Detection Using Hyperspectral Imaging

Hyperspectral imaging (HSI) is a non-ionizing and non-contact imaging technique capable of obtaining more information than conventional RGB (red green blue) imaging. In the medical field, HSI has commonly been investigated due to its great potential for diagnostic and surgical guidance purposes. However, the large amount of information provided by HSI normally contains redundant or non-relevant information, and it is extremely important to identify the most relevant wavelengths for a certain application in order to improve the accuracy of the predictions and reduce the execution time of the classification algorithm. Additionally, some wavelengths can contain noise and removing such bands can improve the classification stage. The work presented in this paper aims to identify such relevant spectral ranges in the visual-and-near-infrared (VNIR) region for an accurate detection of brain cancer using in vivo hyperspectral images. A methodology based on optimization algorithms has been proposed for this task, identifying the relevant wavelengths to achieve the best accuracy in the classification results obtained by a supervised classifier (support vector machines), and employing the lowest possible number of spectral bands. The results demonstrate that the proposed methodology based on the genetic algorithm optimization slightly improves the accuracy of the tumor identification in ~5%, using only 48 bands, with respect to the reference results obtained with 128 bands, offering the possibility of developing customized acquisition sensors that could provide real-time HS imaging. The most relevant spectral ranges found comprise between 440.5-465.96 nm, 498.71-509.62 nm, 556.91-575.1 nm, 593.29-615.12 nm, 636.94-666.05 nm, 698.79-731.53 nm and 884.32-902.51 nm.

Rapid, label-free detection of cerebral ischemia in rats using hyperspectral imaging

BACKGROUND

Stroke is the third most common cause of disability and the second most common cause of death worldwide. Ischemia, one of the two broad categories of stroke, is characterized by a lack of sufficient amounts of blood in order to supply an adequate amount of oxygen and nutrients. It is important to assess the part of the brain that becomes ischemic and necrotic during neurosurgery or experiments in real time. However, there is currently no effective means to achieve this goal.

NEW METHOD

We proposed a method based on hyperspectral imaging (HSI) for the real-time detection of a varied range of ischemic brain tissues in vivo or ex vivo and assessed the practical utility of a model of ischemic stroke in rats.

RESULTS

The results showed that hyperspectral images processed with a ratio of spectral reflectance at 545 and 560 nm (R545/R560) could identify early brain ischemia and accurately show regions of ischemia.

COMPARISON WITH EXISTING METHODS

We verified the area imaged by HSI using hematoxylin and eosin (HE) and 2, 3, 5-triphenyltetrazolium chloride (TTC) staining methods. This technique could precisely image the ischemic part of the brain in vivo and ex vivo.

CONCLUSIONS

These results demonstrate the practical utility of HSI for the real-time detection of cerebral ischemia in rats. By providing rapid assessment of brain tissue perfusion, HSI may help doctors recognize ischemic regions quickly and precisely during surgery as well as have great utility in the experimental process.

Multi-wavelength spatial frequency domain diffuse optical tomography using single-pixel imaging based on lock-in photon counting

We present a spatial frequency domain (SFD) diffuse optical tomography for simultaneous acquisition of multi-wavelength tomographic images of turbid media. We propose a highly sensitive single-pixel SFD imaging system for simultaneously collecting multi-wavelength spatially modulated reflectance images, instead of using the expensive electron-multiplying charge-coupled device camera that requires switching between the multi-wavelength collections. The single-pixel SFD imaging system using three low-power light sources (455, 532 and 660 nm) that were intensity-modulated by square waves with three different frequencies for frequency encoding, and all the light sources were focused onto one digital micromirror device (DMD) for generating wide-field sinusoidal illumination patterns. Reflected light from the surface of the turbid media was modulated by the other DMD with many sampling patterns before being spatially integrated. Spatially integrated light signals were frequency decoded with a novel highly sensitive lock-in photon counting detection, then multi-wavelength spatially modulated reflectance images were recovered with the single-pixel imaging (SPI) method. We incorporated the two-dimensional discrete cosine transform (DCT) into the SPI method to reduce the number of sampling patterns, and, thereby, the proposed DCT-SPI scheme achieved a fast acquisition of SFD reflectance images that is desired for a dynamic SFD imaging application. Direct current (DC) and alternating current (AC) amplitudes at all the locations on the media surface were extracted from the recovered images. Multi-wavelength tomographic images were reconstructed with an inversion algorithm based on the first-order Rytov approximation of the diffusion equation, using both the extracted DC and AC amplitudes. We performed experiments using a series of tissue simulating phantoms to verify the performances of the proposed approach and compared the experimental results with those using a conventional camera-based SFD imaging system. The results demonstrate that our DCT-SPI based SFD-DOT approach is well suited for simultaneous reconstruction of multi-wavelength tomographic images to pave the way for many SFD imaging applications.

Spatial scanning hyperspectral imaging combining a rotating slit with a Dove prism

We present a new concept for spatial scanning hyperspectral imaging. Spatial scanning is one of the main methods used for hyperspectral data acquisition and can provide high spectral resolution over a wide spectral range. However, conventional techniques, such as the whiskbroom and the pushbroom techniques, suffer from the need for relative motion between the target and the imaging system, which increases the complexity on the hardware side and limits the application possibilities. Our new approach combines a rotating slit and a co-rotating Dove prism. The rotating slit scans the target image by selecting one line from the image at each angular position of the slit. The rotating Dove prism is used to synchronously re-align the transmitted light from the selected image line with respect to the transmission grating to allow the projection of the diffracted light over the same range of pixel columns of the image sensor to facilitate data acquisition and extraction of spectral information. The new approach enables the spatial scanning of the target image without the need for relative linear motion or the use of additional external equipment and therefore opens the door for more application scenarios.

In-Vivo and Ex-Vivo Tissue Analysis through Hyperspectral Imaging Techniques: Revealing the Invisible Features of Cancer

In contrast to conventional optical imaging modalities, hyperspectral imaging (HSI) is able to capture much more information from a certain scene, both within and beyond the visual spectral range (from 400 to 700 nm). This imaging modality is based on the principle that each material provides different responses to light reflection, absorption, and scattering across the electromagnetic spectrum. Due to these properties, it is possible to differentiate and identify the different materials/substances presented in a certain scene by their spectral signature. Over the last two decades, HSI has demonstrated potential to become a powerful tool to study and identify several diseases in the medical field, being a non-contact, non-ionizing, and a label-free imaging modality. In this review, the use of HSI as an imaging tool for the analysis and detection of cancer is presented. The basic concepts related to this technology are detailed. The most relevant, state-of-the-art studies that can be found in the literature using HSI for cancer analysis are presented and summarized, both in-vivo and ex-vivo. Lastly, we discuss the current limitations of this technology in the field of cancer detection, together with some insights into possible future steps in the improvement of this technology.

A clinically translatable hyperspectral endoscopy (HySE) system for imaging the gastrointestinal tract

Hyperspectral imaging (HSI) enables visualisation of morphological and biochemical information, which could improve disease diagnostic accuracy. Unfortunately, the wide range of image distortions that arise during flexible endoscopy in the clinic have made integration of HSI challenging. To address this challenge, we demonstrate a hyperspectral endoscope (HySE) that simultaneously records intrinsically co-registered hyperspectral and standard-of-care white light images, which allows image distortions to be compensated computationally and an accurate hyperspectral data cube to be reconstructed as the endoscope moves in the lumen. Evaluation of HySE performance shows excellent spatial, spectral and temporal resolution and high colour fidelity. Application of HySE enables: quantification of blood oxygenation levels in tissue mimicking phantoms; differentiation of spectral profiles from normal and pathological ex vivo human tissues; and recording of hyperspectral data under freehand motion within an intact ex vivo pig oesophagus model. HySE therefore shows potential for enabling HSI in clinical endoscopy.

Optical biopsy of head and neck cancer using hyperspectral imaging and convolutional neural networks

For patients undergoing surgical cancer resection of squamous cell carcinoma (SCCa), cancer-free surgical margins are essential for good prognosis. We developed a method to use hyperspectral imaging (HSI), a noncontact optical imaging modality, and convolutional neural networks (CNNs) to perform an optical biopsy of ex-vivo, surgical gross-tissue specimens, collected from 21 patients undergoing surgical cancer resection. Using a cross-validation paradigm with data from different patients, the CNN can distinguish SCCa from normal aerodigestive tract tissues with an area under the receiver operator curve (AUC) of 0.82. Additionally, normal tissue from the upper aerodigestive tract can be subclassified into squamous epithelium, muscle, and gland with an average AUC of 0.94. After separately training on thyroid tissue, the CNN can differentiate between thyroid carcinoma and normal thyroid with an AUC of 0.95, 92% accuracy, 92% sensitivity, and 92% specificity. Moreover, the CNN can discriminate medullary thyroid carcinoma from benign multinodular goiter (MNG) with an AUC of 0.93. Classical-type papillary thyroid carcinoma is differentiated from MNG with an AUC of 0.91. Our preliminary results demonstrate that an HSI-based optical biopsy method using CNNs can provide multicategory diagnostic information for normal and cancerous head-and-neck tissue, and more patient data are needed to fully investigate the potential and reliability of the proposed technique.

Miniature integrated micro-spectrometer array for snap shot multispectral sensing

An array of micro spectrometers for parallel spectral sensing is designed, set up and tested. It utilizes a planar prism grating combination to obtain an almost linear optical system of 6 mm length only. Arranging such micro spectrometers in an array configuration yields 2'000 spectrometers when utilizing a common 4/3" CCD image sensor well adapted to e.g. microscopic image dimensions. The application in microscopic imaging in the 450-900 nm spectral range is demonstrated as proof of concept, which can be adapted to massively parallel sensing in the frame of integrated sensor concepts.

Use of Hyperspectral/Multispectral Imaging in Gastroenterology. Shedding Some⁻Different⁻Light into the Dark

Hyperspectral/Multispectral imaging (HSI/MSI) technologies are able to sample from tens to hundreds of spectral channels within the electromagnetic spectrum, exceeding the capabilities of human vision. These spectral techniques are based on the principle that every material has a different response (reflection and absorption) to different wavelengths. Thereby, this technology facilitates the discrimination between different materials. HSI has demonstrated good discrimination capabilities for materials in fields, for instance, remote sensing, pollution monitoring, field surveillance, food quality, agriculture, astronomy, geological mapping, and currently, also in medicine. HSI technology allows tissue observation beyond the limitations of the human eye. Moreover, many researchers are using HSI as a new diagnosis tool to analyze optical properties of tissue. Recently, HSI has shown good performance in identifying human diseases in a non-invasive manner. In this paper, we show the potential use of these technologies in the medical domain, with emphasis in the current advances in gastroenterology. The main aim of this review is to provide an overview of contemporary concepts regarding HSI technology together with state-of-art systems and applications in gastroenterology. Finally, we discuss the current limitations and upcoming trends of HSI in gastroenterology.

Quantitative evaluation of comb-structure correction methods for multispectral fibrescopic imaging

Removing the comb artifact introduced by imaging fibre bundles, or 'fibrescopes', for example in medical endoscopy, is essential to provide high quality images to the observer. Multispectral imaging (MSI) is an emerging method that combines morphological (spatial) and chemical (spectral) information in a single data 'cube'. When a fibrescope is coupled to a spectrally resolved detector array (SRDA) to perform MSI, comb removal is complicated by the demosaicking step required to reconstruct the multispectral data cube. To understand the potential for using SRDAs as multispectral imaging sensors in medical endoscopy, we assessed five comb correction methods with respect to five performance metrics relevant to biomedical imaging applications: processing time, resolution, smoothness, signal and the accuracy of spectral reconstruction. By assigning weights to each metric, which are determined by the particular imaging application, our results can be used to select the correction method to achieve best overall performance. In most cases, interpolation gave the best compromise between the different performance metrics when imaging using an SRDA.

Real-time detection of breast cancer at the cellular level

Novel optoelectronic instrumentation has been developed for the multispectral imaging of autofluorescence emitted by metabolic fluorophores. The images resolve individual cells while spectra are collected for each pixel in the images. These datacubes are generated at a rate of 10 per second-fast enough for surgical guidance. The data is processed in real time to provide a single color-coded image to the surgeon. To date, the system has been applied to fresh, ex vivo, human surgical specimens and has distinguished breast cancer from benign tissue. The approach is applicable to in vivo measurements of surgical margins and needle-based optical biopsies. Ongoing work demonstrates that the system has great potential for translation to a hand-held probe with high sensitivity and specificity.

LED for hyperspectral imaging - a new selection method

Hyperspectral imaging (HSI) has become a sophisticated technique in modern applications such as food analyses, recycling technology, medicine, pharmacy and forensic science. It allows one to analyse both spatial and spectral information from an object. But hyperspectral cameras are still expensive due to their extended wavelength range. The development of new light-emitting diodes (LED) in the recent past enables another approach to HSI using a monochrome camera in combination with a LED-based illumination. However, such a system has a lower spectral resolution. Additionally, the growing supply of LED on the market complicates the selection of LED. In this paper, we propose a new time efficient selection method for the design process of an illumination. It chooses an optimised LED combination from an existing database to match a predefined spectral power distribution. Therefore, an algorithm is used to evaluate various LED combinations. Furthermore, the method considers the spectral behaviour of each LED in dependence of forward current and temperature of the solder point. Our method has already shown promise during the selection process for even spectral distributions which is demonstrated in the study. Additionally, we will show its potential for HSI illuminations.

Bimodal reflectance and fluorescence multispectral endoscopy based on spectrally resolving detector arrays

Emerging clinical interest in combining standard white light endoscopy with targeted near-infrared (NIR) fluorescent contrast agents for improved early cancer detection has created demand for multimodal imaging endoscopes. We used two spectrally resolving detector arrays (SRDAs) to realize a bimodal endoscope capable of simultaneous reflectance-based imaging in the visible spectral region and multiplexed fluorescence-based imaging in the NIR. The visible SRDA was composed of 16 spectral bands, with peak wavelengths in the range of 463 to 648 nm and full-width at half-maximum (FWHM) between 9 and 26 nm. The NIR SRDA was composed of 25 spectral bands, with peak wavelengths in the range 659 to 891 nm and FWHM 7 to 15 nm. The spectral endoscope design was based on a "babyscope" model using a commercially available imaging fiber bundle. We developed a spectral transmission model to select optical components and provide reference endmembers for linear spectral unmixing of the recorded image data. The technical characterization of the spectral endoscope is presented, including evaluation of the angular field-of-view, barrel distortion, spatial resolution and spectral fidelity, which showed encouraging performance. An agarose phantom containing oxygenated and deoxygenated blood with three fluorescent dyes was then imaged. After spectral unmixing, the different chemical components of the phantom could be successfully identified via majority decision with high signal-to-background ratio (>3). Imaging performance was further assessed in an ex vivo porcine esophagus model. Our preliminary imaging results demonstrate the capability to simultaneously resolve multiple biological components using a compact spectral endoscopy system.

Melanoma and Melanocyte Identification from Hyperspectral Pathology Images Using Object-Based Multiscale Analysis

Pathological skin imaging analysis is identified as an efficient technique to diagnose melanoma and provide necessary information for treatment. Automatic detection of melanoma and melanocytes in the epidermis area can be a challenging task as a result of the variability of melanocytes and similarity among cytological components. In order to develop a practical and reliable approach to address the issue, this paper proposed a melanoma and melanocyte detection method based on hyperspectral pathology images. Given the abundant and related spectral and spatial information associated with the hyperspectral skin pathological image, an object-based method was first used to construct the image into the object level; then a multiscale descriptor was performed to extract specific features of melanoma and melanocytes. A quantitative evaluation of 100 scenes of hyperspectral pathology images from 49 patients showed the optimal accuracy, sensitivity, and specificity of 94.29%, 95.57%, and 93.15%, respectively. The results can be interpreted that hyperspectral pathology imaging techniques help to detect the melanoma and melanocytes effectively and provide useful information for further segmentation and classification.

Hyperspectral imaging as a possible tool for visualization of changes in hemoglobin oxygenation in patients with deficient hemodynamics – proof of concept

There is a lack of imaging tools for the evaluation of spatial alterations in microcirculation including blood oxygen saturation and hemoglobin distribution but recent innovative developments in hyperspectral technology may offer a solution. We examined different hemodynamic disorders in patients suffering from scleroderma, Dupuytren surgery, chronic foot ulcera and skin infections. Superficial and deeper blood oxygen saturation, hemoglobin distribution and water content were determined using hyperspectral imaging (HSI). In the patient with scleroderma, distinct cutaneous low perfused regions correlated with macroscopic skin aspects and seem to be potential therapy control marker. With HSI accurate clinical evaluation of a macroscopic conspicuous wound after Dupuytren surgery was possible and influenced further surveillance decisions. HSI clearly revealed the spatial geometry and also the clinically related perfusion parameters of abscess formation and chronic ulcer wounds. The hemodynamically relevant parameters like blood oxygen saturation (1 mm to approx. 6 mm subcutaneous), total hemoglobin distribution and tissue water content can be easily determined and visualized with HSI in near real time. Hence, this technique seems to be suitable for routine diagnostics of acute and chronic wounds as well as for the examination of systemic hemodynamic disturbances. Special indications may be transplant surveillance and monitoring of therapeutical interventions.

Accelerating the K-Nearest Neighbors Filtering Algorithm to Optimize the Real-Time Classification of Human Brain Tumor in Hyperspectral Images

The use of hyperspectral imaging (HSI) in the medical field is an emerging approach to assist physicians in diagnostic or surgical guidance tasks. However, HSI data processing involves very high computational requirements due to the huge amount of information captured by the sensors. One of the stages with higher computational load is the K-Nearest Neighbors (KNN) filtering algorithm. The main goal of this study is to optimize and parallelize the KNN algorithm by exploiting the GPU technology to obtain real-time processing during brain cancer surgical procedures. This parallel version of the KNN performs the neighbor filtering of a classification map (obtained from a supervised classifier), evaluating the different classes simultaneously. The undertaken optimizations and the computational capabilities of the GPU device throw a speedup up to 66.18× when compared to a sequential implementation.