Compact and ultracompact spectral imagers: technology and applications in biomedical imaging

Parameter-based transfer learning for severity classification of atopic dermatitis using hyperspectral imaging

BACKGROUND/PURPOSE

Because atopic dermatitis (AD) is a chronic inflammatory skin condition that causes structural changes, there is a growing need for noninvasive research methods to evaluate this condition. Hyperspectral imaging (HSI) captures skin structure features by exploiting light wavelength variations in penetration depth. In this study, parameter-based transfer learning was deployed to classify the severity of AD using HSI. Therefore, we aimed to obtain an optimal combination of classification results from the four models after constructing different source- and target-domain datasets.

METHODS

We designated psoriasis, skin cancer, eczema, and AD datasets as the source datasets, and the set of images acquired via hyperspectral camera as the target dataset for wavelength-specific AD classification. We compared the severity classification performances of 96 combinations of sources, models, and targets.

RESULTS

The highest classification performance of 83% was achieved when ResNet50 was trained on the augmented psoriasis dataset as the source, with the resulting parameters used to train the model on the target Near-infrared radiation (NIR) dataset. The second highest classification accuracy of 81% was achieved when ResNet50 was trained on the unaugmented psoriasis dataset as the source, with the resulting parameters used to train the model on the target R dataset. ResNet50 demonstrated potential as a generalized model for both the source and target data, also confirming that the psoriasis dataset is an effective training resource.

CONCLUSION

The present study not only demonstrates the feasibility of the severity classification of AD based on hyperspectral images, but also showcases combinations and research scalability for domain exploration.

Hyperspectral push-broom imager using a volume Bragg grating as an angular filter

A hyperspectral push-broom imager has been designed, constructed, and tested. The narrow angular selectivity of a weakly index modulated volume Bragg grating is utilized to replace the objective lens, slit, and collimating lens of a conventional slit-based hyperspectral push-broom imager. The imager comprises a dispersion grating, an angular filter grating, a focusing lens, and an image sensor. The imager has a field of view (FOV) of 17 degrees in the spatial direction, a spectral range from 400 nm to 900 nm, and a spectral resolution of 2.1 nm. The acquired hyperspectral data cubes are presented, and the influence of wavelength-dependent incident angle errors is analyzed.

Nerve Detection and Visualization Using Hyperspectral Imaging for Surgical Guidance

During surgery of delicate regions, differentiation between nerve and surrounding tissue is crucial. Hyperspectral imaging (HSI) techniques can enhance the contrast between types of tissue beyond what the human eye can differentiate. Whereas an RGB image captures 3 bands within the visible light range (e.g., 400 nm to 700 nm), HSI can acquire many bands in wavelength increments that highlight regions of an image across a wavelength spectrum. We developed a workflow to identify nerve tissues from other similar tissues such as fat, bone, and muscle. Our workflow uses spectral angle mapper (SAM) and endmember selection. The method is robust for different types of environment and lighting conditions. We validated our workflow on two samples of human tissues. We used a compact HSI system that can image from 400 to 1700 nm to produce HSI of the samples. On these two samples, we achieved an intersection-over-union (IoU) segmentation score of 84.15% and 76.73%, respectively. We showed that our workflow identifies nerve segments that are not easily seen in RGB images. This method is fast, does not rely on special hardware, and can be applied in real time. The hyperspectral imaging and nerve detection approach may provide a powerful tool for image-guided surgery.

A High-Resolution Hyperspectral Imaging System for the Retina

In this study, we developed an imaging system that can acquire and produce high-resolution hyperspectral images of the retina. Our system combines the view from a high-resolution RGB camera and a snapshot hyperspectral camera together. The method is fast and can be constructed into a compact imaging device. We tested our system by imaging a calibrated color chart, biological tissues ex vivo, and a phantom of the human retina. By using image pansharpening methods, we were able to produce a high-resolution hyperspectral image. The images from the hyperspectral camera alone have a spatial resolution of 0.2 mm/pixel, whereas the pansharpened images have a spatial resolution of 0.1 mm/pixel, a 2x increase in spatial resolution. Our method has the potential to capture images of the retina rapidly. Our method preserves both the spatial and spectral fidelity, as shown by comparing the original hyperspectral images with the pansharpened images. The high-resolution hyperspectral imaging device can have a variety of applications in retina examinations.

Switchable LED-based laparoscopic multispectral system for rapid high-resolution perfusion imaging

Significance

Multispectral imaging (MSI) is an approach for real-time, quantitative, and non-invasive tissue perfusion measurements. Current laparoscopic systems based on mosaic sensors or filter wheels lack high spatial resolution or acceptable frame rates.

Aim

To develop a laparoscopic system for MSI-based color video and tissue perfusion imaging during gastrointestinal surgery without compromising spatial or temporal resolution.

Approach

The system was built with 14 switchable light-emitting diodes in the visible and near-infrared spectral range, a 4K image sensor, and a 10 mm laparoscope. Illumination patterns were created for tissue oxygenation and hemoglobin content monitoring. The system was calibrated to a clinically approved laparoscopic hyperspectral system using linear regression models and evaluated in an occlusion study with 36 volunteers.

Results

The root mean squared errors between the MSI and reference system were 0.073 for hemoglobin content, 0.039 for oxygenation in deeper tissue layers, and 0.093 for superficial oxygenation. The spatial resolution at a working distance of 45 mm was 156    μ m . The effective frame rate was 20 fps.

Conclusions

High-resolution perfusion monitoring was successfully achieved. Hardware optimizations will increase the frame rate. Parameter optimizations through alternative illumination patterns, regression, or assumed tissue models are planned. Intraoperative measurements must confirm the suitability during surgery.

Advances in photoacoustic imaging aided by nano contrast agents: special focus on role of lymphatic system imaging for cancer theranostics

Photoacoustic imaging (PAI) is a successful clinical imaging platform for management of cancer and other health conditions that has seen significant progress in the past decade. However, clinical translation of PAI based methods are still under scrutiny as the imaging quality and clinical information derived from PA images are not on par with other imaging methods. Hence, to improve PAI, exogenous contrast agents, in the form of nanomaterials, are being used to achieve better image with less side effects, lower accumulation, and improved target specificity. Nanomedicine has become inevitable in cancer management, as it contributes at every stage from diagnosis to therapy, surgery, and even in the postoperative care and surveillance for recurrence. Nanocontrast agents for PAI have been developed and are being explored for early and improved cancer diagnosis. The systemic stability and target specificity of the nanomaterials to render its theranostic property depends on various influencing factors such as the administration route and physico-chemical responsiveness. The recent focus in PAI is on targeting the lymphatic system and nodes for cancer diagnosis, as they play a vital role in cancer progression and metastasis. This review aims to discuss the clinical advancements of PAI using nanoparticles as exogenous contrast agents for cancer theranostics with emphasis on PAI of lymphatic system for diagnosis, cancer progression, metastasis, PAI guided tumor resection, and finally PAI guided drug delivery.

Column coded scanning aperture hyperspectral imaging system

The line scanning hyperspectral imaging system (LS-HIS), which relies on a mechanical slit or spatial light modulation device for single channel spatial scanning, is widely used in various fields such as biomedical imaging and remote sensing. However, in scenes that require low light illumination, a decrease in luminous flux will increase exposure time, leading to a significant decrease in scanning efficiency and signal-to-noise ratio (SNR). To address this issue, we present a flexible column coded scanning aperture hyperspectral imaging system (CCSA-HIS) using a spatial light modulator digital micromirror device (DMD). By introducing the concept of multiplex and constructing a multiplexing encoding matrix, we form a one-dimensional multi-column coded scanning aperture, which greatly improves scanning efficiency. Experimental comparisons demonstrate that this approach achieves higher SNR and equivalent spatial and spectral resolution in significantly less sampling time compared to LS-HIS. In short, our scheme provides a new imaging technology for the field of hyperspectral imaging with good theoretical value and engineering significance.