Rapid tissue oxygenation mapping from snapshot structured-light images with adversarial deep learning

Deep-learning-enabled spatial frequency domain imaging of the spatiotemporal dynamics of skin physiology

Significance

Spatial frequency domain imaging (SFDI) is an emerging optical imaging modality for visualizing tissue absorption and scattering properties. This approach is promising for noninvasive wide field-of-view (FOV) monitoring of biophysiological processes in vivo.

Aim

We aim to develop deep-learning-enabled spatial frequency domain imaging (SFDI-net) for real-time large FOV imaging of the optical, structural, and physiological properties and demonstrate its application for probing the spatiotemporal dynamics of skin physiology.

Approach

SFDI-net, based on mapping of a two-layer structure into an equivalent homogeneous medium for spatially modulated light and with a convolutional neural network architecture, produces two-dimensional maps of optical, structural, and physiological parameters for bilayered tissue, including cutaneous hemoglobin concentration, oxygen saturation, scattering properties (reduced scattering coefficient and scattering power), melanin content, surface roughness, and epidermal thickness, with visible spatially modulated light at the camera frame rate.

Results

Compared with traditional approaches, SFDI-net achieves a real-time inversion speed and significantly improves image quality by effectively suppressing noise while preserving tissue structure without oversmoothing. We demonstrate the application of the SFDI-net for monitoring the spatiotemporal dynamics of forearm skin physiology in reactive hyperemia and rhythmic respiration and reveal their intricate patterns in hemodynamics.

Conclusions

Deep-learning-enabled spatial frequency domain imaging and SFDI-net may offer insights into the cardiorespiratory system and have promising clinical utility for disease diagnosis, surveillance, and therapeutic assessment. Future hardware and software advancements will bring SFDI-net to clinical practice.

Complex conjugate removal in optical coherence tomography using phase aware generative adversarial network

Significance

Current methods for complex conjugate removal (CCR) in frequency-domain optical coherence tomography (FD-OCT) often require additional hardware components, which increase system complexity and cost. A software-based solution would provide a more efficient and cost-effective alternative.

Aim

We aim to develop a deep learning approach to effectively remove complex conjugate artifacts (CCAs) from OCT scans without the need for extra hardware components.

Approach

We introduce a deep learning method that employs generative adversarial networks to eliminate CCAs from OCT scans. Our model leverages both conventional intensity images and phase images from the OCT scans to enhance the artifact removal process.

Results

Our CCR-generative adversarial network models successfully converted conventional OCT scans with CCAs into artifact-free scans across various samples, including phantoms, human skin, and mouse eyes imaged in vivo with a phase-stable swept source-OCT prototype. The inclusion of phase images significantly improved the performance of the deep learning models in removing CCAs.

Conclusions

Our method provides a low-cost, data-driven, and software-based solution to enhance FD-OCT imaging capabilities by the removal of CCAs.

Soybean seed pest damage detection method based on spatial frequency domain imaging combined with RL-SVM

Soybean seeds are susceptible to damage from the Riptortus pedestris, which is a significant factor affecting the quality of soybean seeds. Currently, manual screening methods for soybean seeds are limited to visual inspection, making it difficult to identify seeds that are phenotypically defect-free but have been punctured by stink bugs on the sub-surface. To facilitate the convenient and efficient identification of healthy soybean seeds, this paper proposes a soybean seed pest detection method based on spatial frequency domain imaging combined with RL-SVM. Firstly, soybean optical data is obtained using single integration sphere technique, and the vigor index of soybean seeds is obtained through germination experiments. Then, based on the above two data items using feature extraction algorithms (the successive projections algorithm and the competitive adaptive reweighted sampling algorithm), the characteristic wavelengths of soybeans are identified. Subsequently, the spatial frequency domain imaging technique is used to obtain the sub-surface images of soybean seeds in a forward manner, and the optical coefficients such as the reduced scattering coefficientμ ' s and absorption coefficient μ a of soybean seeds are inverted. Finally, RL-MLR, RL-GRNN, and RL-SVM prediction models are established based on the ratio of the area of insect-damaged sub-surface to the entire seed, soybean varieties, and μ a at three wavelengths (502 nm, 813 nm, and 712 nm) for predicting and identifying soybean the stinging and sucking pest damage levels of soybean seeds. The experimental results show that the spatial frequency domain imaging technique yields small errors in the optical coefficients of soybean seeds, with errors of less than 15% for μ a and less than 10% forμ ' s . After parameter adjustment through reinforcement learning, the Macro-Recall metrics of each model have improved by 10%-15%, and the RL-SVM model achieves a high Macro-Recall value of 0.9635 for classifying the pest damage levels of soybean seeds.

Tutorial on methods for estimation of optical absorption and scattering properties of tissue

Significance

The estimation of tissue optical properties using diffuse optics has found a range of applications in disease detection, therapy monitoring, and general health care. Biomarkers derived from the estimated optical absorption and scattering coefficients can reflect the underlying progression of many biological processes in tissues.

Aim

Complex light-tissue interactions make it challenging to disentangle the absorption and scattering coefficients, so dedicated measurement systems are required. We aim to help readers understand the measurement principles and practical considerations needed when choosing between different estimation methods based on diffuse optics.

Approach

The estimation methods can be categorized as: steady state, time domain, time frequency domain (FD), spatial domain, and spatial FD. The experimental measurements are coupled with models of light-tissue interactions, which enable inverse solutions for the absorption and scattering coefficients from the measured tissue reflectance and/or transmittance.

Results

The estimation of tissue optical properties has been applied to characterize a variety of ex vivo and in vivo tissues, as well as tissue-mimicking phantoms. Choosing a specific estimation method for a certain application has to trade-off its advantages and limitations.

Conclusion

Optical absorption and scattering property estimation is an increasingly important and accessible approach for medical diagnosis and health monitoring.

Deep-learning approach to stratified reconstructions of tissue absorption and scattering in time-domain spatial frequency domain imaging

Significance

The conventional optical properties (OPs) reconstruction in spatial frequency domain (SFD) imaging, like the lookup table (LUT) method, causes OPs aliasing and yields only average OPs without depth resolution. Integrating SFD imaging with time-resolved (TR) measurements enhances space-TR information, enabling improved reconstruction of absorption (μ a ) and reduced scattering (μ s ' ) coefficients at various depths.

Aim

To achieve the stratified reconstruction of OPs and the separation between μ a and μ s ' , using deep learning workflow based on the temporal and spatial information provided by time-domain SFD imaging technique, while enhancing the reconstruction accuracy.

Approach

Two data processing methods are employed for the OPs reconstruction with TR-SFD imaging, one is full TR data, and the other is the featured data extracted from the full TR data (E , continuous-wave component, ⟨ t ⟩ , mean time of flight). We compared their performance using a series of simulation and phantom validations.

Results

Compared to the LUT approach, utilizing full TR, E and ⟨ t ⟩ datasets yield high-resolution OPs reconstruction results. Among the three datasets employed, full TR demonstrates the optimal accuracy.

Conclusions

Utilizing the data obtained from SFD and TR measurement techniques allows for achieving high-resolution separation reconstruction of μ a and μ s ' at different depths within 5 mm.

Ultra-miniature dual-wavelength spatial frequency domain imaging for micro-endoscopy

Significance

There is a need for a cost-effective, quantitative imaging tool that can be deployed endoscopically to better detect early stage gastrointestinal cancers. Spatial frequency domain imaging (SFDI) is a low-cost imaging technique that produces near-real time, quantitative maps of absorption and reduced scattering coefficients, but most implementations are bulky and suitable only for use outside the body.

Aim

We aim to develop an ultra-miniature SFDI system comprising an optical fiber array (diameter 0.125 mm) and a micro camera (1 × 1    mm package) to displace conventionally bulky components, in particular, the projector.

Approach

First, we fabricated a prototype with an outer diameter of 3 mm, although the individual component dimensions could permit future packaging to a < 1.5    mm diameter. We developed a phase-tracking algorithm to rapidly extract images with fringe projections at three equispaced phase shifts to perform SFDI demodulation.

Results

To validate the performance, we first demonstrate comparable recovery of quantitative optical properties between our ultra-miniature system and a conventional bench-top SFDI system with an agreement of 15% and 6% for absorption and reduced scattering, respectively. Next, we demonstrate imaging of absorption and reduced scattering of tissue-mimicking phantoms providing enhanced contrast between simulated tissue types (healthy and tumour), done simultaneously at wavelengths of 515 and 660 nm. Using a support vector machine classifier, we estimate that sensitivity and specificity values of > 90 % are feasible for detecting simulated squamous cell carcinoma.

Conclusions

This device shows promise as a cost-effective, quantitative imaging tool to detect variations in optical absorption and scattering as indicators of cancer.

Optimal Hyperspectral Band Selection for Tissue Oxygenation Mapping with Generative Adversarial Network

Tissue oxygenation assessment using hyperspectral imaging is an emerging technique for the diagnosis and pre- and post-treatment monitoring of ischemic patients. However, the high spectral resolution of hyperspectral imaging leads to large data sizes and a long imaging time. In this study, we propose a method that utilizes multi-objective evolutionary algorithms to determine the optimal hyperspectral band combination when developing a deep learning model for predicting tissue oxygenation from hyperspectral images. Our results confirm that the deep learning model effectively predicts tissue oxygenation images for various oxygenation states. Moreover, we demonstrate that a high-performance prediction model can be developed using only a small number of spectral bands, indicating the potential for more efficient non-contact tissue oxygenation mapping with the proposed method.Clinical Relevance- The proposed method allows for the non-contact and efficient acquisition of two-dimensional tissue oxygenation information in various oxygenation states.

Training generative adversarial networks for optical property mapping using synthetic image data

We demonstrate the training of a generative adversarial network (GAN) for the prediction of optical property maps (scattering and absorption) using spatial frequency domain imaging (SFDI) image data sets that are generated synthetically with a free open-source 3D modelling and rendering software, Blender. The flexibility of Blender is exploited to simulate 5 models with real-life relevance to clinical SFDI of diseased tissue: flat samples containing a single material, flat samples containing 2 materials, flat samples containing 3 materials, flat samples with spheroidal tumours and cylindrical samples with spheroidal tumours. The last case is particularly relevant as it represents wide-field imaging inside a tubular organ e.g. the gastro-intestinal tract. In all 5 scenarios we show the GAN provides an accurate reconstruction of the optical properties from single SFDI images with a mean normalised error ranging from 1.0-1.2% for absorption and 1.1%-1.2% for scattering, resulting in visually improved contrast for tumour spheroid structures. This compares favourably with the ∼10% absorption error and ∼10% scattering error achieved using GANs on experimental SFDI data. Next, we perform a bi-directional cross-validation of our synthetically-trained GAN, retrained with 90% synthetic and 10% experimental data to encourage domain transfer, with a GAN trained fully on experimental data and observe visually accurate results with an error of 6.3%-10.3% for absorption and 6.6%-11.9% for scattering. Our synthetically trained GAN is therefore highly relevant to real experimental samples but provides the significant added benefits of large training datasets, perfect ground-truths and the ability to test realistic imaging geometries, e.g. inside cylinders, for which no conventional single-shot demodulation algorithms exist. In the future, we expect that the application of techniques such as domain adaptation or training on hybrid real-synthetic datasets will create a powerful tool for fast, accurate production of optical property maps for real clinical imaging systems.

Optical clearing and testing of lung tissue using inhalation aerosols: prospects for monitoring the action of viral infections

Optical clearing of the lung tissue aims to make it more transparent to light by minimizing light scattering, thus allowing reconstruction of the three-dimensional structure of the tissue with a much better resolution. This is of great importance for monitoring of viral infection impact on the alveolar structure of the tissue and oxygen transport. Optical clearing agents (OCAs) can provide not only lesser light scattering of tissue components but also may influence the molecular transport function of the alveolar membrane. Air-filled lungs present significant challenges for optical imaging including optical coherence tomography (OCT), confocal and two-photon microscopy, and Raman spectroscopy, because of the large refractive-index mismatch between alveoli walls and the enclosed air-filled region. During OCT imaging, the light is strongly backscattered at each air–tissue interface, such that image reconstruction is typically limited to a single alveolus. At the same time, the filling of these cavities with an OCA, to which water (physiological solution) can also be attributed since its refractive index is much higher than that of air will lead to much better tissue optical transmittance. This review presents general principles and advances in the field of tissue optical clearing (TOC) technology, OCA delivery mechanisms in lung tissue, studies of the impact of microbial and viral infections on tissue response, and antimicrobial and antiviral photodynamic therapies using methylene blue (MB) and indocyanine green (ICG) dyes as photosensitizers.

Rapid Quantification of Tissue Perfusion Properties with a Two-Stage Look-Up Table

Tissue perfusion properties reveal crucial information pertinent to clinical diagnosis and treatment. Multispectral spatial frequency domain imaging (SFDI) is an emerging imaging technique that has been widely used to quantify tissue perfusion properties. However, slow processing speed limits its usefulness in real-time imaging applications. In this study, we present a two-stage look-up table (LUT) approach that accurately and rapidly quantifies optical (absorption and reduced scattering maps) and perfusion (total hemoglobin and oxygen saturation maps) properties using stage-1 and stage-2 LUTs, respectively, based on reflectance images at 660 and 850 nm. The two-stage LUT can be implemented on both CPU and GPU computing platforms. Quantifying tissue perfusion properties using the simulated diffuse reflectance images, we achieved a quantification speed of 266, 174, and 74 frames per second for three image sizes 512 × 512, 1024 × 1024, and 2048 × 2048 pixels, respectively. Quantification of tissue perfusion properties was highly accurate with only 3.5% and 2.5% error for total hemoglobin and oxygen saturation quantification, respectively. The two-stage LUT has the potential to be integrated with dual-sensor cameras to enable real-time quantification of tissue hemodynamics.

Deep learning in macroscopic diffuse optical imaging

SIGNIFICANCE

Biomedical optics system design, image formation, and image analysis have primarily been guided by classical physical modeling and signal processing methodologies. Recently, however, deep learning (DL) has become a major paradigm in computational modeling and has demonstrated utility in numerous scientific domains and various forms of data analysis.

AIM

We aim to comprehensively review the use of DL applied to macroscopic diffuse optical imaging (DOI).

APPROACH

First, we provide a layman introduction to DL. Then, the review summarizes current DL work in some of the most active areas of this field, including optical properties retrieval, fluorescence lifetime imaging, and diffuse optical tomography.

RESULTS

The advantages of using DL for DOI versus conventional inverse solvers cited in the literature reviewed herein are numerous. These include, among others, a decrease in analysis time (often by many orders of magnitude), increased quantitative reconstruction quality, robustness to noise, and the unique capability to learn complex end-to-end relationships.

CONCLUSIONS

The heavily validated capability of DL's use across a wide range of complex inverse solving methodologies has enormous potential to bring novel DOI modalities, otherwise deemed impractical for clinical translation, to the patient's bedside.

Spatial Frequency Domain Imaging System Calibration, Correction and Application for Pear Surface Damage Detection

Spatial frequency domain imaging (SFDI) is a non-contact wide-field optical imaging technique for optical property detection. This study aimed to establish an SFDI system and investigate the effects of system calibration, error analysis and correction on the measurement of optical properties. Optical parameter characteristic measurements of normal pears with three different damage types were performed using the calibrated system. The obtained absorption coefficient μ_a and the reduced scattering coefficient μ'_s were used for discriminating pears with different surface damage using a linear discriminant analysis model. The results showed that at 527 nm and 675 nm, the pears' quadruple classification (normal, bruised, scratched and abraded) accuracy using the SFDI technique was 92.5% and 83.8%, respectively, which has an advantage compared with the conventional planar light classification results of 82.5% and 77.5%. The three-way classification (normal, minor damage and serious damage) SFDI technique was as high as 100% and 98.8% at 527 nm and 675 nm, respectively, while the classification accuracy of conventional planar light was 93.8% and 93.8%, respectively. The results of this study indicated that SFDI has the potential to detect different damage types in fruit and that the SFDI technique has a promising future in agricultural product quality inspection in further research.

Deep Learning in Biomedical Optics

This article reviews deep learning applications in biomedical optics with a particular emphasis on image formation. The review is organized by imaging domains within biomedical optics and includes microscopy, fluorescence lifetime imaging, in vivo microscopy, widefield endoscopy, optical coherence tomography, photoacoustic imaging, diffuse tomography, and functional optical brain imaging. For each of these domains, we summarize how deep learning has been applied and highlight methods by which deep learning can enable new capabilities for optics in medicine. Challenges and opportunities to improve translation and adoption of deep learning in biomedical optics are also summarized. Lasers Surg. Med. © 2021 Wiley Periodicals LLC.

Speckle illumination SFDI for projector-free optical property mapping

Spatial frequency domain imaging can map tissue scattering and absorption properties over a wide field of view, making it useful for clinical applications such as wound assessment and surgical guidance. This technique has previously required the projection of fully characterized illumination patterns. Here, we show that random and unknown speckle illumination can be used to sample the modulation transfer function of tissues at known spatial frequencies, allowing the quantitative mapping of optical properties with simple laser diode illumination. We compute low- and high-spatial frequency response parameters from the local power spectral density for each pixel and use a lookup table to accurately estimate absorption and scattering coefficients in tissue phantoms, in vivo human hand, and ex vivo swine esophagus. Because speckle patterns can be generated over a large depth of field and field of view with simple coherent illumination, this approach may enable optical property mapping in new form-factors and applications, including endoscopy.

Direct mapping from diffuse reflectance to chromophore concentrations in multi-fx spatial frequency domain imaging (SFDI) with a deep residual network (DRN)

Spatial frequency domain imaging (SFDI) is an emerging technology that enables label-free, non-contact, and wide-field mapping of tissue chromophore contents, such as oxy- and deoxy-hemoglobin concentrations. It has been shown that the use of more than two spatial frequencies (multi-fx ) can vastly improve measurement accuracy and reduce chromophore estimation uncertainties, but real-time multi-fx SFDI for chromophore monitoring has been limited in practice due to the slow speed of available chromophore inversion algorithms. Existing inversion algorithms have to first convert the multi-fx diffuse reflectance to optical absorptions, and then solve a set of linear equations to estimate chromophore concentrations. In this work, we present a deep learning framework, noted as a deep residual network (DRN), that is able to directly map from diffuse reflectance to chromophore concentrations. The proposed DRN is over 10x faster than the state-of-the-art method for chromophore inversion and enables 25x improvement on the frame rate for in vivo real-time oxygenation mapping. The proposed deep learning model will help enable real-time and highly accurate chromophore monitoring with multi-fx SFDI.