Rapid tissue oxygenation mapping from snapshot structured-light images with adversarial deep learning

Ultra-miniature dual-wavelength spatial frequency domain imaging for micro-endoscopy

Significance

There is a need for a cost-effective, quantitative imaging tool that can be deployed endoscopically to better detect early stage gastrointestinal cancers. Spatial frequency domain imaging (SFDI) is a low-cost imaging technique that produces near-real time, quantitative maps of absorption and reduced scattering coefficients, but most implementations are bulky and suitable only for use outside the body.

Aim

We aim to develop an ultra-miniature SFDI system comprising an optical fiber array (diameter 0.125 mm) and a micro camera (1 × 1    mm package) to displace conventionally bulky components, in particular, the projector.

Approach

First, we fabricated a prototype with an outer diameter of 3 mm, although the individual component dimensions could permit future packaging to a < 1.5    mm diameter. We developed a phase-tracking algorithm to rapidly extract images with fringe projections at three equispaced phase shifts to perform SFDI demodulation.

Results

To validate the performance, we first demonstrate comparable recovery of quantitative optical properties between our ultra-miniature system and a conventional bench-top SFDI system with an agreement of 15% and 6% for absorption and reduced scattering, respectively. Next, we demonstrate imaging of absorption and reduced scattering of tissue-mimicking phantoms providing enhanced contrast between simulated tissue types (healthy and tumour), done simultaneously at wavelengths of 515 and 660 nm. Using a support vector machine classifier, we estimate that sensitivity and specificity values of > 90 % are feasible for detecting simulated squamous cell carcinoma.

Conclusions

This device shows promise as a cost-effective, quantitative imaging tool to detect variations in optical absorption and scattering as indicators of cancer.

Optimal Hyperspectral Band Selection for Tissue Oxygenation Mapping with Generative Adversarial Network

Tissue oxygenation assessment using hyperspectral imaging is an emerging technique for the diagnosis and pre- and post-treatment monitoring of ischemic patients. However, the high spectral resolution of hyperspectral imaging leads to large data sizes and a long imaging time. In this study, we propose a method that utilizes multi-objective evolutionary algorithms to determine the optimal hyperspectral band combination when developing a deep learning model for predicting tissue oxygenation from hyperspectral images. Our results confirm that the deep learning model effectively predicts tissue oxygenation images for various oxygenation states. Moreover, we demonstrate that a high-performance prediction model can be developed using only a small number of spectral bands, indicating the potential for more efficient non-contact tissue oxygenation mapping with the proposed method.Clinical Relevance- The proposed method allows for the non-contact and efficient acquisition of two-dimensional tissue oxygenation information in various oxygenation states.

Training generative adversarial networks for optical property mapping using synthetic image data

We demonstrate the training of a generative adversarial network (GAN) for the prediction of optical property maps (scattering and absorption) using spatial frequency domain imaging (SFDI) image data sets that are generated synthetically with a free open-source 3D modelling and rendering software, Blender. The flexibility of Blender is exploited to simulate 5 models with real-life relevance to clinical SFDI of diseased tissue: flat samples containing a single material, flat samples containing 2 materials, flat samples containing 3 materials, flat samples with spheroidal tumours and cylindrical samples with spheroidal tumours. The last case is particularly relevant as it represents wide-field imaging inside a tubular organ e.g. the gastro-intestinal tract. In all 5 scenarios we show the GAN provides an accurate reconstruction of the optical properties from single SFDI images with a mean normalised error ranging from 1.0-1.2% for absorption and 1.1%-1.2% for scattering, resulting in visually improved contrast for tumour spheroid structures. This compares favourably with the ∼10% absorption error and ∼10% scattering error achieved using GANs on experimental SFDI data. Next, we perform a bi-directional cross-validation of our synthetically-trained GAN, retrained with 90% synthetic and 10% experimental data to encourage domain transfer, with a GAN trained fully on experimental data and observe visually accurate results with an error of 6.3%-10.3% for absorption and 6.6%-11.9% for scattering. Our synthetically trained GAN is therefore highly relevant to real experimental samples but provides the significant added benefits of large training datasets, perfect ground-truths and the ability to test realistic imaging geometries, e.g. inside cylinders, for which no conventional single-shot demodulation algorithms exist. In the future, we expect that the application of techniques such as domain adaptation or training on hybrid real-synthetic datasets will create a powerful tool for fast, accurate production of optical property maps for real clinical imaging systems.

Optical clearing and testing of lung tissue using inhalation aerosols: prospects for monitoring the action of viral infections

Optical clearing of the lung tissue aims to make it more transparent to light by minimizing light scattering, thus allowing reconstruction of the three-dimensional structure of the tissue with a much better resolution. This is of great importance for monitoring of viral infection impact on the alveolar structure of the tissue and oxygen transport. Optical clearing agents (OCAs) can provide not only lesser light scattering of tissue components but also may influence the molecular transport function of the alveolar membrane. Air-filled lungs present significant challenges for optical imaging including optical coherence tomography (OCT), confocal and two-photon microscopy, and Raman spectroscopy, because of the large refractive-index mismatch between alveoli walls and the enclosed air-filled region. During OCT imaging, the light is strongly backscattered at each air–tissue interface, such that image reconstruction is typically limited to a single alveolus. At the same time, the filling of these cavities with an OCA, to which water (physiological solution) can also be attributed since its refractive index is much higher than that of air will lead to much better tissue optical transmittance. This review presents general principles and advances in the field of tissue optical clearing (TOC) technology, OCA delivery mechanisms in lung tissue, studies of the impact of microbial and viral infections on tissue response, and antimicrobial and antiviral photodynamic therapies using methylene blue (MB) and indocyanine green (ICG) dyes as photosensitizers.

Rapid Quantification of Tissue Perfusion Properties with a Two-Stage Look-Up Table

Tissue perfusion properties reveal crucial information pertinent to clinical diagnosis and treatment. Multispectral spatial frequency domain imaging (SFDI) is an emerging imaging technique that has been widely used to quantify tissue perfusion properties. However, slow processing speed limits its usefulness in real-time imaging applications. In this study, we present a two-stage look-up table (LUT) approach that accurately and rapidly quantifies optical (absorption and reduced scattering maps) and perfusion (total hemoglobin and oxygen saturation maps) properties using stage-1 and stage-2 LUTs, respectively, based on reflectance images at 660 and 850 nm. The two-stage LUT can be implemented on both CPU and GPU computing platforms. Quantifying tissue perfusion properties using the simulated diffuse reflectance images, we achieved a quantification speed of 266, 174, and 74 frames per second for three image sizes 512 × 512, 1024 × 1024, and 2048 × 2048 pixels, respectively. Quantification of tissue perfusion properties was highly accurate with only 3.5% and 2.5% error for total hemoglobin and oxygen saturation quantification, respectively. The two-stage LUT has the potential to be integrated with dual-sensor cameras to enable real-time quantification of tissue hemodynamics.

Deep learning in macroscopic diffuse optical imaging

SIGNIFICANCE

Biomedical optics system design, image formation, and image analysis have primarily been guided by classical physical modeling and signal processing methodologies. Recently, however, deep learning (DL) has become a major paradigm in computational modeling and has demonstrated utility in numerous scientific domains and various forms of data analysis.

AIM

We aim to comprehensively review the use of DL applied to macroscopic diffuse optical imaging (DOI).

APPROACH

First, we provide a layman introduction to DL. Then, the review summarizes current DL work in some of the most active areas of this field, including optical properties retrieval, fluorescence lifetime imaging, and diffuse optical tomography.

RESULTS

The advantages of using DL for DOI versus conventional inverse solvers cited in the literature reviewed herein are numerous. These include, among others, a decrease in analysis time (often by many orders of magnitude), increased quantitative reconstruction quality, robustness to noise, and the unique capability to learn complex end-to-end relationships.

CONCLUSIONS

The heavily validated capability of DL's use across a wide range of complex inverse solving methodologies has enormous potential to bring novel DOI modalities, otherwise deemed impractical for clinical translation, to the patient's bedside.

Spatial Frequency Domain Imaging System Calibration, Correction and Application for Pear Surface Damage Detection

Spatial frequency domain imaging (SFDI) is a non-contact wide-field optical imaging technique for optical property detection. This study aimed to establish an SFDI system and investigate the effects of system calibration, error analysis and correction on the measurement of optical properties. Optical parameter characteristic measurements of normal pears with three different damage types were performed using the calibrated system. The obtained absorption coefficient μ_a and the reduced scattering coefficient μ'_s were used for discriminating pears with different surface damage using a linear discriminant analysis model. The results showed that at 527 nm and 675 nm, the pears' quadruple classification (normal, bruised, scratched and abraded) accuracy using the SFDI technique was 92.5% and 83.8%, respectively, which has an advantage compared with the conventional planar light classification results of 82.5% and 77.5%. The three-way classification (normal, minor damage and serious damage) SFDI technique was as high as 100% and 98.8% at 527 nm and 675 nm, respectively, while the classification accuracy of conventional planar light was 93.8% and 93.8%, respectively. The results of this study indicated that SFDI has the potential to detect different damage types in fruit and that the SFDI technique has a promising future in agricultural product quality inspection in further research.

Deep Learning in Biomedical Optics

This article reviews deep learning applications in biomedical optics with a particular emphasis on image formation. The review is organized by imaging domains within biomedical optics and includes microscopy, fluorescence lifetime imaging, in vivo microscopy, widefield endoscopy, optical coherence tomography, photoacoustic imaging, diffuse tomography, and functional optical brain imaging. For each of these domains, we summarize how deep learning has been applied and highlight methods by which deep learning can enable new capabilities for optics in medicine. Challenges and opportunities to improve translation and adoption of deep learning in biomedical optics are also summarized. Lasers Surg. Med. © 2021 Wiley Periodicals LLC.

Speckle illumination SFDI for projector-free optical property mapping

Spatial frequency domain imaging can map tissue scattering and absorption properties over a wide field of view, making it useful for clinical applications such as wound assessment and surgical guidance. This technique has previously required the projection of fully characterized illumination patterns. Here, we show that random and unknown speckle illumination can be used to sample the modulation transfer function of tissues at known spatial frequencies, allowing the quantitative mapping of optical properties with simple laser diode illumination. We compute low- and high-spatial frequency response parameters from the local power spectral density for each pixel and use a lookup table to accurately estimate absorption and scattering coefficients in tissue phantoms, in vivo human hand, and ex vivo swine esophagus. Because speckle patterns can be generated over a large depth of field and field of view with simple coherent illumination, this approach may enable optical property mapping in new form-factors and applications, including endoscopy.

Direct mapping from diffuse reflectance to chromophore concentrations in multi-fx spatial frequency domain imaging (SFDI) with a deep residual network (DRN)

Spatial frequency domain imaging (SFDI) is an emerging technology that enables label-free, non-contact, and wide-field mapping of tissue chromophore contents, such as oxy- and deoxy-hemoglobin concentrations. It has been shown that the use of more than two spatial frequencies (multi-fx ) can vastly improve measurement accuracy and reduce chromophore estimation uncertainties, but real-time multi-fx SFDI for chromophore monitoring has been limited in practice due to the slow speed of available chromophore inversion algorithms. Existing inversion algorithms have to first convert the multi-fx diffuse reflectance to optical absorptions, and then solve a set of linear equations to estimate chromophore concentrations. In this work, we present a deep learning framework, noted as a deep residual network (DRN), that is able to directly map from diffuse reflectance to chromophore concentrations. The proposed DRN is over 10x faster than the state-of-the-art method for chromophore inversion and enables 25x improvement on the frame rate for in vivo real-time oxygenation mapping. The proposed deep learning model will help enable real-time and highly accurate chromophore monitoring with multi-fx SFDI.