Convolutional neural network for resolution enhancement and noise reduction in acoustic resolution photoacoustic microscopy

Highly sensitive self-focused ultrasound transducer with a bionic back-reflector for multiscale-resolution photoacoustic microscopy

In this study, we designed a self-focused ultrasonic transducer made of polyvinylidene fluoride (PVDF). This transducer involves a back-reflector, which is modeled after tapetum lucidum in the eyes of some nocturnal animals. The bionic structure reflects the ultrasound, which passes through the PVDF membrane, back to PVDF and provides a second chance for the PVDF to convert the ultrasound to electric signals. This design increases the amount of ultrasound absorbed by the PVDF, thereby improving the detection sensitivity. Both ultrasonic and photoacoustic (PA) experiments were conduct to characterize the performance of the transducer. The results show that the fabricated transducer has a center frequency of 13.07 MHz, and a bandwidth of 96% at -6 dB. With an acoustic numerical aperture (NA) of 0.64, the transducer provides a lateral resolution of 140µm. Importantly, the bionic design improves the detection sensitivity of the transducer about 30%. Finally, we apply the fabricated transducer to optical-resolution (OR) and acoustic-resolution photoacoustic microscopy (AR-PAM) to achieve multiscale-resolution PA imaging. Imaging of the bamboo leaf and the leaf skeleton demonstrates that the proposed transducer can provide high spatial resolution, better imaging intensity and contrast. Therefore, the proposed transducer design will be useful to enhance the performance of multiscale-resolution PAM.

4D spectral-spatial computational photoacoustic dermoscopy

Photoacoustic dermoscopy (PAD) is an emerging non-invasive imaging technology aids in the diagnosis of dermatological conditions by obtaining optical absorption information of skin tissues. Despite advances in PAD, it remains unclear how to obtain quantitative accuracy of the reconstructed PAD images according to the optical and acoustic properties of multilayered skin, the wavelength and distribution of excitation light, and the detection performance of ultrasound transducers. In this work, a computing method of four-dimensional (4D) spectral-spatial imaging for PAD is developed to enable quantitative analysis and optimization of structural and functional imaging of skin. This method takes the optical and acoustic properties of heterogeneous skin tissues into account, which can be used to correct the optical field of excitation light, detectable ultrasonic field, and provide accurate single-spectrum analysis or multi-spectral imaging solutions of PAD for multilayered skin tissues. A series of experiments were performed, and simulation datasets obtained from the computational model were used to train neural networks to further improve the imaging quality of the PAD system. All the results demonstrated the method could contribute to the development and optimization of clinical PADs by datasets with multiple variable parameters, and provide clinical predictability of photoacoustic (PA) data for human skin.

Noise-insensitive defocused signal and resolution enhancement for optical-resolution photoacoustic microscopy via deep learning

Optical-resolution photoacoustic microscopy suffers from narrow depth of field and a significant deterioration in defocused signal intensity and spatial resolution. Here, a method based on deep learning was proposed to enhance the defocused resolution and signal-to-noise ratio. A virtual optical-resolution photoacoustic microscopy based on k-wave was used to obtain the datasets of deep learning with different noise levels. A fully dense U-Net was trained with randomly distributed sources to improve the quality of photoacoustic images. The results show that the PSNR of defocused signal was enhanced by more than 1.2 times. An over 2.6-fold enhancement in lateral resolution and an over 3.4-fold enhancement in axial resolution of defocused regions were achieved. The large volumetric and high-resolution imaging of blood vessels further verified that the proposed method can effectively overcome the deterioration of the signal and the spatial resolution due to the narrow depth of field of optical-resolution photoacoustic microscopy.

Enhanced resolution and sensitivity acoustic-resolution photoacoustic microscopy with semi/unsupervised GANs

Acoustic-resolution photoacoustic microscopy (AR-PAM) enables visualization of biological tissues at depths of several millimeters with superior optical absorption contrast. However, the lateral resolution and sensitivity of AR-PAM are generally lower than those of optical-resolution PAM (OR-PAM) owing to the intrinsic physical acoustic focusing mechanism. Here, we demonstrate a computational strategy with two generative adversarial networks (GANs) to perform semi/unsupervised reconstruction with high resolution and sensitivity in AR-PAM by maintaining its imaging capability at enhanced depths. The b-scan PAM images were prepared as paired (for semi-supervised conditional GAN) and unpaired (for unsupervised CycleGAN) groups for label-free reconstructed AR-PAM b-scan image generation and training. The semi/unsupervised GANs successfully improved resolution and sensitivity in a phantom and in vivo mouse ear test with ground truth. We also confirmed that GANs could enhance resolution and sensitivity of deep tissues without the ground truth.

Applications of Deep Learning Algorithms to Ultrasound Imaging Analysis in Preclinical Studies on In Vivo Animals

BACKGROUND AND AIM

Ultrasound (US) imaging is increasingly preferred over other more invasive modalities in preclinical studies using animal models. However, this technique has some limitations, mainly related to operator dependence. To overcome some of the current drawbacks, sophisticated data processing models are proposed, in particular artificial intelligence models based on deep learning (DL) networks. This systematic review aims to overview the application of DL algorithms in assisting US analysis of images acquired in in vivo preclinical studies on animal models.

METHODS

A literature search was conducted using the Scopus and PubMed databases. Studies published from January 2012 to November 2022 that developed DL models on US images acquired in preclinical/animal experimental scenarios were eligible for inclusion. This review was conducted according to PRISMA guidelines.

RESULTS

Fifty-six studies were enrolled and classified into five groups based on the anatomical district in which the DL models were used. Sixteen studies focused on the cardiovascular system and fourteen on the abdominal organs. Five studies applied DL networks to images of the musculoskeletal system and eight investigations involved the brain. Thirteen papers, grouped under a miscellaneous category, proposed heterogeneous applications adopting DL systems. Our analysis also highlighted that murine models were the most common animals used in in vivo studies applying DL to US imaging.

CONCLUSION

DL techniques show great potential in terms of US images acquired in preclinical studies using animal models. However, in this scenario, these techniques are still in their early stages, and there is room for improvement, such as sample sizes, data preprocessing, and model interpretability.

Niche preclinical and clinical applications of photoacoustic imaging with endogenous contrast

In the past decade, photoacoustic (PA) imaging has attracted a great deal of popularity as an emergent diagnostic technology owing to its successful demonstration in both preclinical and clinical arenas by various academic and industrial research groups. Such steady growth of PA imaging can mainly be attributed to its salient features, including being non-ionizing, cost-effective, easily deployable, and having sufficient axial, lateral, and temporal resolutions for resolving various tissue characteristics and assessing the therapeutic efficacy. In addition, PA imaging can easily be integrated with the ultrasound imaging systems, the combination of which confers the ability to co-register and cross-reference various features in the structural, functional, and molecular imaging regimes. PA imaging relies on either an endogenous source of contrast (e.g., hemoglobin) or those of an exogenous nature such as nano-sized tunable optical absorbers or dyes that may boost imaging contrast beyond that provided by the endogenous sources. In this review, we discuss the applications of PA imaging with endogenous contrast as they pertain to clinically relevant niches, including tissue characterization, cancer diagnostics/therapies (termed as theranostics), cardiovascular applications, and surgical applications. We believe that PA imaging's role as a facile indicator of several disease-relevant states will continue to expand and evolve as it is adopted by an increasing number of research laboratories and clinics worldwide.

Restoring the imaging quality of circular transducer array-based PACT using synthetic aperture focusing technique integrated with 2nd-derivative-based back projection scheme

Circular-array-based photoacoustic computed tomography (CA-PACT) is a promising imaging tool owing to its broad acoustic detection coverage and fidelity. However, CA-PACT suffers from poor image quality outside the focal zone along both elevational and lateral dimensions. To address this challenge, we proposed a novel reconstruction strategy by integrating the synthetic aperture focusing technique (SAFT) with the 2nd derivative-based back projection (2nd D-BP) algorithm to restore the image quality outside the focal zone along both the elevational and lateral axes. The proposed solution is a two-phase reconstruction scheme. In the first phase, with the assistance of an acoustic lens, we designed a circular array-based SAFT algorithm to restore the resolution and SNR along the elevational axis. The acoustic lens pushes the boundary of the upper limit of the SAFT scheme to achieve enhanced elevational resolution. In the second phase, we proposed a 2nd D-BP scheme to improve the lateral resolution and suppress noises in 3D imaging results. The 2nd D-BP strategy enhances the image quality along the lateral dimension by up-converting the high spatial frequencies of the object's absorption pattern. We validated the effectiveness of the proposed strategy using both phantoms and in vivo human experiments. The experimental results demonstrated superior image quality (7-fold enhancement in elevational resolution, 3-fold enhancement in lateral resolution, and an 11-dB increase in SNR). This strategy provides a new paradigm in the PACT system as it significantly enhances the spatial resolution and imaging contrast in both the elevational and lateral dimensions while maintaining a large focal zone.

Deep learning-assisted frequency-domain photoacoustic microscopy

Frequency-domain photoacoustic microscopy (FD-PAM) constitutes a powerful cost-efficient imaging method integrating intensity-modulated laser beams for the excitation of single-frequency photoacoustic waves. Nevertheless, FD-PAM provides an extremely small signal-to-noise ratio (SNR), which can be up to two orders of magnitude lower than the conventional time-domain (TD) systems. To overcome this inherent SNR limitation of FD-PAM, we utilize a U-Net neural network aiming at image augmentation without the need for excessive averaging or the application of high optical power. In this context, we improve the accessibility of PAM as the system's cost is dramatically reduced, and we expand its applicability to demanding observations while retaining sufficiently high image quality standards.

De-Noising of Photoacoustic Microscopy Images by Attentive Generative Adversarial Network

As a hybrid imaging technology, photoacoustic microscopy (PAM) imaging suffers from noise due to the maximum permissible exposure of laser intensity, attenuation of ultrasound in the tissue, and the inherent noise of the transducer. De-noising is an image processing method to reduce noise, and PAM image quality can be recovered. However, previous de-noising techniques usually heavily rely on manually selected parameters, resulting in unsatisfactory and slow de-noising performance for different noisy images, which greatly hinders practical and clinical applications. In this work, we propose a deep learning-based method to remove noise from PAM images without manual selection of settings for different noisy images. An attention enhanced generative adversarial network is used to extract image features and adaptively remove various levels of Gaussian, Poisson, and Rayleigh noise. The proposed method is demonstrated on both synthetic and real datasets, including phantom (leaf veins) and in vivo (mouse ear blood vessels and zebrafish pigment) experiments. In the in vivo experiments using synthetic datasets, our method achieves the improvement of 6.53 dB and 0.26 in peak signal-to-noise ratio and structural similarity metrics, respectively. The results show that compared with previous PAM de-noising methods, our method exhibits good performance in recovering images qualitatively and quantitatively. In addition, the de-noising processing speed of 0.016 s is achieved for an image with 256×256 pixels, which has the potential for real-time applications. Our approach is effective and practical for the de-noising of PAM images.

Adaptive enhancement of acoustic resolution photoacoustic microscopy imaging via deep CNN prior

Acoustic resolution photoacoustic microscopy (AR-PAM) is a promising medical imaging modality that can be employed for deep bio-tissue imaging. However, its relatively low imaging resolution has greatly hindered its wide applications. Previous model-based or learning-based PAM enhancement algorithms either require design of complex handcrafted prior to achieve good performance or lack the interpretability and flexibility that can adapt to different degradation models. However, the degradation model of AR-PAM imaging is subject to both imaging depth and center frequency of ultrasound transducer, which varies in different imaging conditions and cannot be handled by a single neural network model. To address this limitation, an algorithm integrating both learning-based and model-based method is proposed here so that a single framework can deal with various distortion functions adaptively. The vasculature image statistics is implicitly learned by a deep convolutional neural network, which served as plug and play (PnP) prior. The trained network can be directly plugged into the model-based optimization framework for iterative AR-PAM image enhancement, which fitted for different degradation mechanisms. Based on physical model, the point spread function (PSF) kernels for various AR-PAM imaging situations are derived and used for the enhancement of simulation and in vivo AR-PAM images, which collectively proved the effectiveness of proposed method. Quantitatively, the PSNR and SSIM values have all achieve best performance with the proposed algorithm in all three simulation scenarios; The SNR and CNR values have also significantly raised from 6.34 and 5.79 to 35.37 and 29.66 respectively in an in vivo testing result with the proposed algorithm.

Deep Learning-Powered Bessel-Beam Multiparametric Photoacoustic Microscopy

Enabling simultaneous and high-resolution quantification of the total concentration of hemoglobin ( [Formula: see text]), oxygen saturation of hemoglobin (sO2), and cerebral blood flow (CBF), multi-parametric photoacoustic microscopy (PAM) has emerged as a promising tool for functional and metabolic imaging of the live mouse brain. However, due to the limited depth of focus imposed by the Gaussian-beam excitation, the quantitative measurements become inaccurate when the imaging object is out of focus. To address this problem, we have developed a hardware-software combined approach by integrating Bessel-beam excitation and conditional generative adversarial network (cGAN)-based deep learning. Side-by-side comparison of the new cGAN-powered Bessel-beam multi-parametric PAM against the conventional Gaussian-beam multi-parametric PAM shows that the new system enables high-resolution, quantitative imaging of [Formula: see text], sO2, and CBF over a depth range of [Formula: see text] in the live mouse brain, with errors 13-58 times lower than those of the conventional system. Better fulfilling the rigid requirement of light focusing for accurate hemodynamic measurements, the deep learning-powered Bessel-beam multi-parametric PAM may find applications in large-field functional recording across the uneven brain surface and beyond (e.g., tumor imaging).

Deep and Domain Transfer Learning Aided Photoacoustic Microscopy: Acoustic Resolution to Optical Resolution

Acoustic resolution photoacoustic micros- copy (AR-PAM) can achieve deeper imaging depth in biological tissue, with the sacrifice of imaging resolution compared with optical resolution photoacoustic microscopy (OR-PAM). Here we aim to enhance the AR-PAM image quality towards OR-PAM image, which specifically includes the enhancement of imaging resolution, restoration of micro-vasculatures, and reduction of artifacts. To address this issue, a network (MultiResU-Net) is first trained as generative model with simulated AR-OR image pairs, which are synthesized with physical transducer model. Moderate enhancement results can already be obtained when applying this model to in vivo AR imaging data. Nevertheless, the perceptual quality is unsatisfactory due to domain shift. Further, domain transfer learning technique under generative adversarial network (GAN) framework is proposed to drive the enhanced image's manifold towards that of real OR image. In this way, perceptually convincing AR to OR enhancement result is obtained, which can also be supported by quantitative analysis. Peak Signal to Noise Ratio (PSNR) and Structural Similarity Index (SSIM) values are significantly increased from 14.74 dB to 19.01 dB and from 0.1974 to 0.2937, respectively, validating the improvement of reconstruction correctness and overall perceptual quality. The proposed algorithm has also been validated across different imaging depths with experiments conducted in both shallow and deep tissue. The above AR to OR domain transfer learning with GAN (AODTL-GAN) framework has enabled the enhancement target with limited amount of matched in vivo AR-OR imaging data.

Multistage adaptive noise reduction technique for optical resolution photoacoustic microscopy

Photoacoustic microscopy has received great attention due to the benefits of the optical resolution contrast as well as its superior spatial resolution and relatively deep depth. Like other imaging modalities, photoacoustic images suffer from noise, and filtering techniques are required to remove them. To overcome the noise, we proposed a combination of filters, including an adaptive median filter, an effective filter for impulsive noise, and a nonlocal means filter, an effective filter for background noise, for noise removal and image quality enhancement. Our proposed method enhanced the signal-to-noise ratio by 16 dB in an in vivo study compared to the traditional image reconstruction approach and preserved the image detail with minimal blurring, which usually occurs when filtering. These experimental results verified that the proposed adaptive multistage denoising techniques could effectively improve image quality under noisy data acquisition conditions, providing a strong foundation for photoacoustic microscopy with limited laser power.

Deep learning methods hold promise for light fluence compensation in three-dimensional optoacoustic imaging

SIGNIFICANCE

Quantitative optoacoustic imaging (QOAI) continues to be a challenge due to the influence of nonlinear optical fluence distribution, which distorts the optoacoustic image representation. Nonlinear optical fluence correction in OA imaging is highly ill-posed, leading to the inaccurate recovery of optical absorption maps. This work aims to recover the optical absorption maps using deep learning (DL) approach by correcting for the fluence effect.

AIM

Different DL models were compared and investigated to enable optical absorption coefficient recovery at a particular wavelength in a nonhomogeneous foreground and background medium.

APPROACH

Data-driven models were trained with two-dimensional (2D) Blood vessel and three-dimensional (3D) numerical breast phantom with highly heterogeneous/realistic structures to correct for the nonlinear optical fluence distribution. The trained DL models such as U-Net, Fully Dense (FD) U-Net, Y-Net, FD Y-Net, Deep residual U-Net (Deep ResU-Net), and generative adversarial network (GAN) were tested to evaluate the performance of optical absorption coefficient recovery (or fluence compensation) with in-silico and in-vivo datasets.

RESULTS

The results indicated that FD U-Net-based deconvolution improves by about 10% over reconstructed optoacoustic images in terms of peak-signal-to-noise ratio. Further, it was observed that DL models can indeed highlight deep-seated structures with higher contrast due to fluence compensation. Importantly, the DL models were found to be about 17 times faster than solving diffusion equation for fluence correction.

CONCLUSIONS

The DL methods were able to compensate for nonlinear optical fluence distribution more effectively and improve the optoacoustic image quality.

Depth-extended acoustic-resolution photoacoustic microscopy based on a two-stage deep learning network

Acoustic resolution photoacoustic microscopy (AR-PAM) is a major modality of photoacoustic imaging. It can non-invasively provide high-resolution morphological and functional information about biological tissues. However, the image quality of AR-PAM degrades rapidly when the targets move far away from the focus. Although some works have been conducted to extend the high-resolution imaging depth of AR-PAM, most of them have a small focal point requirement, which is generally not satisfied in a regular AR-PAM system. Therefore, we propose a two-stage deep learning (DL) reconstruction strategy for AR-PAM to recover high-resolution photoacoustic images at different out-of-focus depths adaptively. The residual U-Net with attention gate was developed to implement the image reconstruction. We carried out phantom and in vivo experiments to optimize the proposed DL network and verify the performance of the proposed reconstruction method. Experimental results demonstrated that our approach extends the depth-of-focus of AR-PAM from 1mm to 3mm under the 4 mJ/cm² light energy used in the imaging system. In addition, the imaging resolution of the region 2 mm far away from the focus can be improved, similar to the in-focus area. The proposed method effectively improves the imaging ability of AR-PAM and thus could be used in various biomedical studies needing deeper depth.

Visualizing tumor angiogenesis and boundary with polygon-scanning multiscale photoacoustic microscopy

Recently, we developed an integrated optical-resolution (OR) and acoustic-resolution (AR) PAM, which has multiscale imaging capability using different resolutions. However, limited by the scanning method, a tradeoff exists between the imaging speed and field of view, which impedes its wider applications. Here, we present an improved multiscale PAM which achieves high-speed wide-field imaging based on a homemade polygon scanner. Encoder trigger mode was proposed to avoid jittering of the polygon scanner during imaging. Distortions caused by polygon scanning were analyzed theoretically and compared with traditional types of distortions in optical-scanning PAM. Then a depth correction method was proposed and verified to compensate for the distortions. System characterization of OR-PAM and AR-PAM was performed prior to in vivo imaging. Blood reperfusion of an in vivo mouse ear was imaged continuously to demonstrate the feasibility of the multiscale PAM for high-speed imaging. Results showed that the maximum B-scan rate could be 14.65 Hz in a fixed range of 10 mm. Compared with our previous multiscale system, the imaging speed of the improved system was increased by a factor of 12.35. In vivo imaging of a subcutaneously inoculated B-16 melanoma of a mouse was performed. Results showed that the blood vasculature around the melanoma could be resolved and the melanoma could be visualized at a depth up to 1.6 mm using the multiscale PAM.

High-fidelity deconvolution for acoustic-resolution photoacoustic microscopy enabled by convolutional neural networks

Acoustic-resolution photoacoustic microscopy (AR-PAM) image resolution is determined by the point spread function (PSF) of the imaging system. Previous algorithms, including Richardson-Lucy (R-L) deconvolution and model-based (MB) deconvolution, improve spatial resolution by taking advantage of the PSF as prior knowledge. However, these methods encounter the problems of inaccurate deconvolution, meaning the deconvolved feature size and the original one are not consistent (e.g., the former can be smaller than the latter). We present a novel deep convolution neural network (CNN)-based algorithm featuring high-fidelity recovery of multiscale feature size to improve lateral resolution of AR-PAM. The CNN is trained with simulated image pairs of line patterns, which is to mimic blood vessels. To investigate the suitable CNN model structure and elaborate on the effectiveness of CNN methods compared with non-learning methods, we select five different CNN models, while R-L and directional MB methods are also applied for comparison. Besides simulated data, experimental data including tungsten wires, leaf veins, and in vivo blood vessels are also evaluated. A custom-defined metric of relative size error (RSE) is used to quantify the multiscale feature recovery ability of different methods. Compared to other methods, enhanced deep super resolution (EDSR) network and residual in residual dense block network (RRDBNet) model show better recovery in terms of RSE for tungsten wires with diameters ranging from 30 μ m to 120 μ m . Moreover, AR-PAM images of leaf veins are tested to demonstrate the effectiveness of the optimized CNN methods (by EDSR and RRDBNet) for complex patterns. Finally, in vivo images of mouse ear blood vessels and rat ear blood vessels are acquired and then deconvolved, and the results show that the proposed CNN method (notably RRDBNet) enables accurate deconvolution of multiscale feature size and thus good fidelity.

High frame rate (∼3 Hz) circular photoacoustic tomography using single-element ultrasound transducer aided with deep learning

SIGNIFICANCE

In circular scanning photoacoustic tomography (PAT), it takes several minutes to generate an image of acceptable quality, especially with a single-element ultrasound transducer (UST). The imaging speed can be enhanced by faster scanning (with high repetition rate light sources) and using multiple-USTs. However, artifacts arising from the sparse signal acquisition and low signal-to-noise ratio at higher scanning speeds limit the imaging speed. Thus, there is a need to improve the imaging speed of the PAT systems without hampering the quality of the PAT image.

AIM

To improve the frame rate (or imaging speed) of the PAT system by using deep learning (DL).

APPROACH

For improving the frame rate (or imaging speed) of the PAT system, we propose a novel U-Net-based DL framework to reconstruct PAT images from fast scanning data.

RESULTS

The efficiency of the network was evaluated on both single- and multiple-UST-based PAT systems. Both phantom and in vivo imaging demonstrate that the network can improve the imaging frame rate by approximately sixfold in single-UST-based PAT systems and by approximately twofold in multi-UST-based PAT systems.

CONCLUSIONS

We proposed an innovative method to improve the frame rate (or imaging speed) by using DL and with this method, the fastest frame rate of ∼ 3    Hz imaging is achieved without hampering the quality of the reconstructed image.

Achieving depth-independent lateral resolution in AR-PAM using the synthetic-aperture focusing technique

Acoustic-resolution photoacoustic microscopy (AR-PAM) is a promising imaging modality that renders images with ultrasound resolution and extends the imaging depth beyond the optical ballistic regime. To achieve a high lateral resolution, a large numerical aperture (NA) of a focused transducer is usually applied for AR-PAM. However, AR-PAM fails to hold its performance in the out-of-focus region. The lateral resolution and signal-to-noise ratio (SNR) degrade substantially, thereby leading to a significantly deteriorated image quality outside the focal area. Based on the concept of the synthetic-aperture focusing technique (SAFT), various strategies have been developed to address this challenge. These include 1D-SAFT, 2D-SAFT, adaptive-SAFT, spatial impulse response (SIR)-based schemes, and delay-multiply-and-sum (DMAS) strategies. These techniques have shown progress in achieving depth-independent lateral resolution, while several challenges remain. This review aims to introduce these developments in SAFT-based approaches, highlight their fundamental mechanisms, underline the advantages and limitations of each approach, and discuss the outlook of the remaining challenges for future advances.

Deep-E: A Fully-Dense Neural Network for Improving the Elevation Resolution in Linear-Array-Based Photoacoustic Tomography

Linear-array-based photoacoustic tomography has shown broad applications in biomedical research and preclinical imaging. However, the elevational resolution of a linear array is fundamentally limited due to the weak cylindrical focus of the transducer element. While several methods have been proposed to address this issue, they have all handled the problem in a less time-efficient way. In this work, we propose to improve the elevational resolution of a linear array through Deep-E, a fully dense neural network based on U-net. Deep-E exhibits high computational efficiency by converting the three-dimensional problem into a two-dimension problem: it focused on training a model to enhance the resolution along elevational direction by only using the 2D slices in the axial and elevational plane and thereby reducing the computational burden in simulation and training. We demonstrated the efficacy of Deep-E using various datasets, including simulation, phantom, and human subject results. We found that Deep-E could improve elevational resolution by at least four times and recover the object's true size. We envision that Deep-E will have a significant impact in linear-array-based photoacoustic imaging studies by providing high-speed and high-resolution image enhancement.

High-resolution photoacoustic microscopy with deep penetration through learning

Optical-resolution photoacoustic microscopy (OR-PAM) enjoys superior spatial resolution and has received intense attention in recent years. The application, however, has been limited to shallow depths because of strong scattering of light in biological tissues. In this work, we propose to achieve deep-penetrating OR-PAM performance by using deep learning enabled image transformation on blurry living mouse vascular images that were acquired with an acoustic-resolution photoacoustic microscopy (AR-PAM) setup. A generative adversarial network (GAN) was trained in this study and improved the imaging lateral resolution of AR-PAM from 54.0 µm to 5.1 µm, comparable to that of a typical OR-PAM (4.7 µm). The feasibility of the network was evaluated with living mouse ear data, producing superior microvasculature images that outperforms blind deconvolution. The generalization of the network was validated with in vivo mouse brain data. Moreover, it was shown experimentally that the deep-learning method can retain high resolution at tissue depths beyond one optical transport mean free path. Whilst it can be further improved, the proposed method provides new horizons to expand the scope of OR-PAM towards deep-tissue imaging and wide applications in biomedicine.

Image enhancement in acoustic-resolution photoacoustic microscopy enabled by a novel directional algorithm

By considering the line pattern of acoustic-resolution photoacoustic microscopy (AR-PAM) vessel images, we develop modified algorithms for synthetic aperture focusing technique (SAFT) and deconvolution based on a directional approach to enhance images. The modified algorithms consist of Fourier accumulation SAFT (FA-SAFT) and directional model-based (D-MB) deconvolution. To evaluate the performance of our algorithms, we conduct a series of imaging experiments and apply our algorithms, and existing SAFT and deconvolution algorithms are also applied for side-by-side comparison. By imaging tungsten wire phantom, our algorithms enable full width at half maximum of 26 - 31 µm over depth of focus of 1.8 mm and minimum resolvable distance of 46 - 49 µm, besting existing SAFT and deconvolution algorithms. Imaging of leaf skeleton phantom and in vivo imaging of mouse blood vessels also prove that our algorithm is capable of providing high-resolution, high-signal-to-noise ratio, and good-fidelity results for complex structures and for in vivo applications, especially for the images with the line pattern. The proposed directional approach can not only be used in AR-PAM but also in other imaging modalities to deal with the line pattern, such as FA-SAFT for ultrasound imaging and D-MB deconvolution for optical coherence tomography angiography.

Light on osteoarthritic joint: from bench to bed

Osteoarthritis (OA) is one of the rapidly growing disability-associated conditions with population aging worldwide. There is a pressing need for precise diagnosis and timely intervention for OA in the early stage. Current clinical imaging modalities, including pain radiography, magnetic resonance imaging, ultrasound, and optical coherent tomography, are limited to provide structural changes when the damage has been established or advanced. It prompts further endeavors in search of novel functional and molecular imaging, which potentially enables early diagnosis and intervention of OA. A hybrid imaging modality based on photothermal effects, photoacoustic imaging, has drawn wide attention in recent years and has seen a variety of biomedical applications, due to its great performance in yielding high-contrast and high-resolution images from structure to function, from tissue down to molecular levels, from animals to human subjects. Photoacoustic imaging has witnessed gratifying potentials and preliminary effects in OA diagnosis. Regarding the treatment of OA, photothermal-triggered therapy has exhibited its attractions for enhanced therapeutic outcomes. In this narrative review, we will discuss photoacoustic imaging for the diagnosis and monitoring of OA at different stages. Structural, functional, and molecular parameter changes associated with OA joints captured by photoacoustics will be summarized, forming the diagnosis perspective of the review. Photothermal therapy applications related to OA will also be discussed herein. Lastly, relevant clinical applications and its potential solutions to extend photoacoustic imaging to deeper OA situations have been proposed. Although some aspects may not be covered, this mini review provides a better understanding of the diagnosis and treatment of OA with exciting innovations based on tissue photothermal effects. It may also inspire more explorations in the field towards earlier and better theranostics of OA.

Virtual optical-resolution photoacoustic microscopy using the k-Wave method

Deep learning has been widely used in image processing, quantitative analysis, and other applications in optical-resolution photoacoustic microscopy (OR-PAM). It requires a large amount of photoacoustic data for training and testing. However, due to the complex structure, high cost, slow imaging speed, and other factors of OR-PAM, it is difficult to obtain enough data required by deep learning, which limits the research of deep learning in OR-PAM to a certain extent. To solve this problem, a virtual OR-PAM based on k-Wave is proposed. The virtual photoacoustic microscopy mainly includes the setting of excitation light source and ultrasonic probe, scanning and signal processing, which can realize the common Gaussian-beam and Bessel-beam OR-PAMs. The system performance (lateral resolution, axial resolution, and depth of field) was tested by imaging a vertically tilted fiber, and the effectiveness and feasibility of the virtual simulation platform were verified by 3D imaging of the virtual vascular network. The ability to the generation of the dataset for deep learning was also verified. The construction of the virtual OR-PAM can promote the research of OR-PAM and the application of deep learning in OR-PAM.

Deep-learning-based multi-transducer photoacoustic tomography imaging without radius calibration

Pulsed laser diodes are used in photoacoustic tomography (PAT) as excitation sources because of their low cost, compact size, and high pulse repetition rate. In combination with multiple single-element ultrasound transducers (SUTs) the imaging speed of PAT can be improved. However, during PAT image reconstruction, the exact radius of each SUT is required for accurate reconstruction. Here we developed a novel deep learning approach to alleviate the need for radius calibration. We used a convolutional neural network (fully dense U-Net) aided with a convolutional long short-term memory block to reconstruct the PAT images. Our analysis on the test set demonstrates that the proposed network eliminates the need for radius calibration and improves the peak signal-to-noise ratio by ∼73% without compromising the image quality. In vivo imaging was used to verify the performance of the network.

Simultaneous Denoising and Localization Network for Photoacoustic Target Localization

A significant research problem of recent interest is the localization of targets like vessels, surgical needles, and tumors in photoacoustic (PA) images.To achieve accurate localization, a high photoacoustic signal-to-noise ratio (SNR) is required. However, this is not guaranteed for deep targets, as optical scattering causes an exponential decay in optical fluence with respect to tissue depth. To address this, we develop a novel deep learning method designed to explicitly exhibit robustness to noise present in photoacoustic radio-frequency (RF) data. More precisely, we describe and evaluate a deep neural network architecture consisting of a shared encoder and two parallel decoders. One decoder extracts the target coordinates from the input RF data while the other boosts the SNR and estimates clean RF data. The joint optimization of the shared encoder and dual decoders lends significant noise robustness to the features extracted by the encoder, which in turn enables the network to contain detailed information about deep targets that may be obscured by noise. Additional custom layers and newly proposed regularizers in the training loss function (designed based on observed RF data signal and noise behavior) serve to increase the SNR in the cleaned RF output and improve model performance. To account for depth-dependent strong optical scattering, our network was trained with simulated photoacoustic datasets of targets embedded at different depths inside tissue media of different scattering levels. The network trained on this novel dataset accurately locates targets in experimental PA data that is clinically relevant with respect to the localization of vessels, needles, or brachytherapy seeds. We verify the merits of the proposed architecture by outperforming the state of the art on both simulated and experimental datasets.

Deep learning-based autofocus method enhances image quality in light-sheet fluorescence microscopy

Light-sheet fluorescence microscopy (LSFM) is a minimally invasive and high throughput imaging technique ideal for capturing large volumes of tissue with sub-cellular resolution. A fundamental requirement for LSFM is a seamless overlap of the light-sheet that excites a selective plane in the specimen, with the focal plane of the objective lens. However, spatial heterogeneity in the refractive index of the specimen often results in violation of this requirement when imaging deep in the tissue. To address this issue, autofocus methods are commonly used to refocus the focal plane of the objective-lens on the light-sheet. Yet, autofocus techniques are slow since they require capturing a stack of images and tend to fail in the presence of spherical aberrations that dominate volume imaging. To address these issues, we present a deep learning-based autofocus framework that can estimate the position of the objective-lens focal plane relative to the light-sheet, based on two defocused images. This approach outperforms or provides comparable results with the best traditional autofocus method on small and large image patches respectively. When the trained network is integrated with a custom-built LSFM, a certainty measure is used to further refine the network's prediction. The network performance is demonstrated in real-time on cleared genetically labeled mouse forebrain and pig cochleae samples. Our study provides a framework that could improve light-sheet microscopy and its application toward imaging large 3D specimens with high spatial resolution.

Translational Photoacoustic Imaging for Disease Diagnosis, Monitoring, and Surgical Guidance: introduction to the feature issue

This feature issue of Biomedical Optics Express covered all aspects of translational photoacoustic research. Application areas include screening and diagnosis of diseases, imaging of disease progression and therapeutic response, and image-guided treatment, such as surgery, drug delivery, and photothermal/photodynamic therapy. The feature issue also covers relevant developments in photoacoustic instrumentation, contrast agents, image processing and reconstruction algorithms.

Sounding out the hidden data: A concise review of deep learning in photoacoustic imaging

The rapidly evolving field of photoacoustic tomography utilizes endogenous chromophores to extract both functional and structural information from deep within tissues. It is this power to perform precise quantitative measurements in vivo-with endogenous or exogenous contrast-that makes photoacoustic tomography highly promising for clinical translation in functional brain imaging, early cancer detection, real-time surgical guidance, and the visualization of dynamic drug responses. Considering photoacoustic tomography has benefited from numerous engineering innovations, it is of no surprise that many of photoacoustic tomography's current cutting-edge developments incorporate advances from the equally novel field of artificial intelligence. More specifically, alongside the growth and prevalence of graphical processing unit capabilities within recent years has emerged an offshoot of artificial intelligence known as deep learning. Rooted in the solid foundation of signal processing, deep learning typically utilizes a method of optimization known as gradient descent to minimize a loss function and update model parameters. There are already a number of innovative efforts in photoacoustic tomography utilizing deep learning techniques for a variety of purposes, including resolution enhancement, reconstruction artifact removal, undersampling correction, and improved quantification. Most of these efforts have proven to be highly promising in addressing long-standing technical obstacles where traditional solutions either completely fail or make only incremental progress. This concise review focuses on the history of applied artificial intelligence in photoacoustic tomography, presents recent advances at this multifaceted intersection of fields, and outlines the most exciting advances that will likely propagate into promising future innovations.

Perspective on fast-evolving photoacoustic tomography

SIGNIFICANCE

Acoustically detecting the rich optical absorption contrast in biological tissues, photoacoustic tomography (PAT) seamlessly bridges the functional and molecular sensitivity of optical excitation with the deep penetration and high scalability of ultrasound detection. As a result of continuous technological innovations and commercial development, PAT has been playing an increasingly important role in life sciences and patient care, including functional brain imaging, smart drug delivery, early cancer diagnosis, and interventional therapy guidance.

AIM

Built on our 2016 tutorial article that focused on the principles and implementations of PAT, this perspective aims to provide an update on the exciting technical advances in PAT.

APPROACH

This perspective focuses on the recent PAT innovations in volumetric deep-tissue imaging, high-speed wide-field microscopic imaging, high-sensitivity optical ultrasound detection, and machine-learning enhanced image reconstruction and data processing. Representative applications are introduced to demonstrate these enabling technical breakthroughs in biomedical research.

CONCLUSIONS

We conclude the perspective by discussing the future development of PAT technologies.

Deep learning for biomedical photoacoustic imaging: A review

Photoacoustic imaging (PAI) is a promising emerging imaging modality that enables spatially resolved imaging of optical tissue properties up to several centimeters deep in tissue, creating the potential for numerous exciting clinical applications. However, extraction of relevant tissue parameters from the raw data requires the solving of inverse image reconstruction problems, which have proven extremely difficult to solve. The application of deep learning methods has recently exploded in popularity, leading to impressive successes in the context of medical imaging and also finding first use in the field of PAI. Deep learning methods possess unique advantages that can facilitate the clinical translation of PAI, such as extremely fast computation times and the fact that they can be adapted to any given problem. In this review, we examine the current state of the art regarding deep learning in PAI and identify potential directions of research that will help to reach the goal of clinical applicability.

Another decade of photoacoustic imaging

Photoacoustic imaging - a hybrid biomedical imaging modality finding its way to clinical practices. Although the photoacoustic phenomenon was known more than a century back, only in the last two decades it has been widely researched and used for biomedical imaging applications. In this review we focus on the development and progress of the technology in the last decade (2010-2020). From becoming more and more user friendly, cheaper in cost, portable in size, photoacoustic imaging promises a wide range of applications, if translated to clinic. The growth of photoacoustic community is steady, and with several new directions researchers are exploring, it is inevitable that photoacoustic imaging will one day establish itself as a regular imaging system in the clinical practices.

Deep learning approach to improve tangential resolution in photoacoustic tomography

In circular scan photoacoustic tomography (PAT), the axial resolution is spatially invariant and is limited by the bandwidth of the detector. However, the tangential resolution is spatially variant and is dependent on the aperture size of the detector. In particular, the tangential resolution improves with the decreasing aperture size. However, using a detector with a smaller aperture reduces the sensitivity of the transducer. Thus, large aperture size detectors are widely preferred in circular scan PAT imaging systems. Although several techniques have been proposed to improve the tangential resolution, they have inherent limitations such as high cost and the need for customized detectors. Herein, we propose a novel deep learning architecture to counter the spatially variant tangential resolution in circular scanning PAT imaging systems. We used a fully dense U-Net based convolutional neural network architecture along with 9 residual blocks to improve the tangential resolution of the PAT images. The network was trained on the simulated datasets and its performance was verified by experimental in vivo imaging. Results show that the proposed deep learning network improves the tangential resolution by eight folds, without compromising the structural similarity and quality of image.