Deep-E: A Fully-Dense Neural Network for Improving the Elevation Resolution in Linear-Array-Based Photoacoustic Tomography

Image restoration for ring-array photoacoustic tomography based on an attention mechanism driven conditional generative adversarial network

Ring-Array photoacoustic tomography (PAT) systems have shown great promise in non-invasive biomedical imaging. However, images produced by these systems often suffer from quality degradation due to non-ideal imaging conditions, with common issues including blurring and streak artifacts. To address these challenges, we propose an image restoration method based on a conditional generative adversarial network (CGAN) framework. Our approach integrates a hybrid spatial and channel attention mechanism within a Residual Shifted Window Transformer Module (RSTM) to enhance the generator's performance. Additionally, we have developed a comprehensive loss function to balance pixel-level accuracy, detail preservation, and perceptual quality. We further incorporate a gamma correction module to enhance the contrast of the network's output. Experimental results on both simulated and in vivo data demonstrate that our method significantly improves resolution and restores overall image quality.

Enhanced clinical photoacoustic vascular imaging through a skin localization network and adaptive weighting

Photoacoustic tomography (PAT) is an emerging imaging modality with widespread applications in both preclinical and clinical studies. Despite its promising capabilities to provide high-resolution images, the visualization of vessels might be hampered by skin signals and attenuation in tissues. In this study, we have introduced a framework to retrieve deep vessels. It combines a deep learning network to segment skin layers and an adaptive weighting algorithm to compensate for attenuation. Evaluation of enhancement using vessel occupancy metrics and signal-to-noise ratio (SNR) demonstrates that the proposed method significantly recovers deep vessels across various body positions and skin tones. These findings indicate the method's potential to enhance quantitative analysis in preclinical and clinical photoacoustic research.

Three-dimensional diffractive acoustic tomography

Acoustically probing biological tissues with light or sound, photoacoustic and ultrasound imaging can provide anatomical, functional, and/or molecular information at depths far beyond the optical diffusion limit. However, most photoacoustic and ultrasound imaging systems rely on linear-array transducers with elevational focusing and are limited to two-dimensional imaging with anisotropic resolutions. Here, we present three-dimensional diffractive acoustic tomography (3D-DAT), which uses an off-the-shelf linear-array transducer with single-slit acoustic diffraction. Without jeopardizing its accessibility by general users, 3D-DAT has achieved simultaneous 3D photoacoustic and ultrasound imaging with optimal imaging performance in deep tissues, providing near-isotropic resolutions, high imaging speed, and a large field-of-view, as well as enhanced quantitative accuracy and detection sensitivity. Moreover, powered by the fast focal line volumetric reconstruction, 3D-DAT has achieved 50-fold faster reconstruction times than traditional photoacoustic imaging reconstruction. Using 3D-DAT on small animal models, we mapped the distribution of the biliverdin-binding serpin complex in glassfrogs, tracked gold nanoparticle accumulation in a mouse tumor model, imaged genetically-encoded photoswitchable tumors, and investigated polyfluoroalkyl substances exposure on developing embryos. With its enhanced imaging performance and high accessibility, 3D-DAT may find broad applications in fundamental life sciences and biomedical research.

Unsupervised denoising of photoacoustic images based on the Noise2Noise network

In this study, we implemented an unsupervised deep learning method, the Noise2Noise network, for the improvement of linear-array-based photoacoustic (PA) imaging. Unlike supervised learning, which requires a noise-free ground truth, the Noise2Noise network can learn noise patterns from a pair of noisy images. This is particularly important for in vivo PA imaging, where the ground truth is not available. In this study, we developed a method to generate noise pairs from a single set of PA images and verified our approach through simulation and experimental studies. Our results reveal that the method can effectively remove noise, improve signal-to-noise ratio, and enhance vascular structures at deeper depths. The denoised images show clear and detailed vascular structure at different depths, providing valuable insights for preclinical research and potential clinical applications.

Challenges and advances in two-dimensional photoacoustic computed tomography: a review

Significance

Photoacoustic computed tomography (PACT), a hybrid imaging modality combining optical excitation with acoustic detection, has rapidly emerged as a prominent biomedical imaging technique.

Aim

We review the challenges and advances of PACT, including (1) limited view, (2) anisotropy resolution, (3) spatial aliasing, (4) acoustic heterogeneity (speed of sound mismatch), and (5) fluence correction of spectral unmixing.

Approach

We performed a comprehensive literature review to summarize the key challenges in PACT toward practical applications and discuss various solutions.

Results

There is a wide range of contributions from both industry and academic spaces. Various approaches, including emerging deep learning methods, are proposed to improve the performance of PACT further.

Conclusions

We outline contemporary technologies aimed at tackling the challenges in PACT applications.

Dual-modal Photoacoustic and Ultrasound Imaging: from preclinical to clinical applications

Photoacoustic imaging is a novel biomedical imaging modality that has emerged over the recent decades. Due to the conversion of optical energy into the acoustic wave, photoacoustic imaging offers high-resolution imaging in depth beyond the optical diffusion limit. Photoacoustic imaging is frequently used in conjunction with ultrasound as a hybrid modality. The combination enables the acquisition of both optical and acoustic contrasts of tissue, providing functional, structural, molecular, and vascular information within the same field of view. In this review, we first described the principles of various photoacoustic and ultrasound imaging techniques and then classified the dual-modal imaging systems based on their preclinical and clinical imaging applications. The advantages of dual-modal imaging were thoroughly analyzed. Finally, the review ends with a critical discussion of existing developments and a look toward the future.

Photoacoustic imaging plus X: a review

Significance

Photoacoustic (PA) imaging (PAI) represents an emerging modality within the realm of biomedical imaging technology. It seamlessly blends the wealth of optical contrast with the remarkable depth of penetration offered by ultrasound. These distinctive features of PAI hold tremendous potential for various applications, including early cancer detection, functional imaging, hybrid imaging, monitoring ablation therapy, and providing guidance during surgical procedures. The synergy between PAI and other cutting-edge technologies not only enhances its capabilities but also propels it toward broader clinical applicability.

Aim

The integration of PAI with advanced technology for PA signal detection, signal processing, image reconstruction, hybrid imaging, and clinical applications has significantly bolstered the capabilities of PAI. This review endeavor contributes to a deeper comprehension of how the synergy between PAI and other advanced technologies can lead to improved applications.

Approach

An examination of the evolving research frontiers in PAI, integrated with other advanced technologies, reveals six key categories named "PAI plus X." These categories encompass a range of topics, including but not limited to PAI plus treatment, PAI plus circuits design, PAI plus accurate positioning system, PAI plus fast scanning systems, PAI plus ultrasound sensors, PAI plus advanced laser sources, PAI plus deep learning, and PAI plus other imaging modalities.

Results

After conducting a comprehensive review of the existing literature and research on PAI integrated with other technologies, various proposals have emerged to advance the development of PAI plus X. These proposals aim to enhance system hardware, improve imaging quality, and address clinical challenges effectively.

Conclusions

The progression of innovative and sophisticated approaches within each category of PAI plus X is positioned to drive significant advancements in both the development of PAI technology and its clinical applications. Furthermore, PAI not only has the potential to integrate with the above-mentioned technologies but also to broaden its applications even further.

Adaptive machine learning method for photoacoustic computed tomography based on sparse array sensor data

BACKGROUND AND OBJECTIVE

Photoacoustic computed tomography (PACT) is a non-invasive biomedical imaging technology that has developed rapidly in recent decades, especially has shown potential for small animal studies and early diagnosis of human diseases. To obtain high-quality images, the photoacoustic imaging system needs a high-element-density detector array. However, in practical applications, due to the cost limitation, manufacturing technology, and the system requirement in miniaturization and robustness, it is challenging to achieve sufficient elements and high-quality reconstructed images, which may even suffer from artifacts. Different from the latest machine learning methods based on removing distortions and artifacts to recover high-quality images, this paper proposes an adaptive machine learning method to firstly predict and complement the photoacoustic sensor channel data from sparse array sampling and then reconstruct images through conventional reconstruction algorithms.

METHODS

We develop an adaptive machine learning method to predict and complement the photoacoustic sensor channel data. The model consists of XGBoost and a neural network named SS-net. To handle data sets of different sizes and improve the generalization, a tunable parameter is used to control the weights of XGBoost and SS-net outputs.

RESULTS

The proposed method achieved superior performance as demonstrated by simulation, phantom experiments, and in vivo experiment results. Compared with linear interpolation, XGBoost, CAE, and U-net, the simulation results show that the SSIM value is increased by 12.83%, 6.78%, 21.46%, and 12.33%. Moreover, the median R2 is increased by 34.4%, 8.1%, 28.6%, and 84.1% with the in vivo data.

CONCLUSIONS

This model provides a framework to predict the missed photoacoustic sensor data on a sparse ring-shaped array for PACT imaging and has achieved considerable improvements in reconstructing the objects. Compared with linear interpolation and other deep learning methods qualitatively and quantitatively, our proposed methods can effectively suppress artifacts and improve image quality. The advantage of our methods is that there is no need for preparing a large number of images as the training dataset, and the data for training is directly from the sensors. It has the potential to be applied to a wide range of photoacoustic imaging detector arrays for low-cost and user-friendly clinical applications.

Dual-Scan Photoacoustic Tomography for the Imaging of Vascular Structure on Foot

Chronic leg ulcers are affecting approximately 6.5 million Americans, and they are associated with significant mortality, reduced quality of life, and high treatment costs. Since many chronic ulcers have underlying vascular insufficiency, accurate assessment of tissue perfusion is critical to treatment planning and monitoring. This study introduces a dual-scan photoacoustic (PA) tomography (PAT) system that can simultaneously image the dorsal and plantar sides of the foot to reduce imaging time. To account for the unique shape of the foot, the system employs height-adjustable and articulating baseball stages that can scan along the foot's contour. In vivo results from healthy volunteers demonstrate the system's ability to acquire clear images of foot vasculature, and results from patients indicate that the system can image patients with various ulcer conditions. We also investigated various PA features and examined their correlation with the foot condition. Our preliminary results indicate that vessel sharpness, occupancy, intensity, and density could all be used to assess tissue perfusion. This research demonstrated the potential of PAT for routine clinical tissue perfusion assessment.

Deep Learning Enhanced Volumetric Photoacoustic Imaging of Vasculature in Human

The development of high-performance imaging processing algorithms is a core area of photoacoustic tomography. While various deep learning based image processing techniques have been developed in the area, their applications in 3D imaging are still limited due to challenges in computational cost and memory allocation. To address those limitations, this work implements a 3D fully-dense (3DFD) U-net to linear array based photoacoustic tomography and utilizes volumetric simulation and mixed precision training to increase efficiency and training size. Through numerical simulation, phantom imaging, and in vivo experiments, this work demonstrates that the trained network restores the true object size, reduces the noise level and artifacts, improves the contrast at deep regions, and reveals vessels subject to limited view distortion. With these enhancements, 3DFD U-net successfully produces clear 3D vascular images of the palm, arms, breasts, and feet of human subjects. These enhanced vascular images offer improved capabilities for biometric identification, foot ulcer evaluation, and breast cancer imaging. These results indicate that the new algorithm will have a significant impact on preclinical and clinical photoacoustic tomography.

Integrated Photoacoustic, Ultrasound, and Angiographic Tomography (PAUSAT) for NonInvasive Whole-Brain Imaging of Ischemic Stroke

Presented here is an experimental ischemic stroke study using our newly developed noninvasive imaging system that integrates three acoustic-based imaging technologies: photoacoustic, ultrasound, and angiographic tomography (PAUSAT). Combining these three modalities helps acquire multi-spectral photoacoustic tomography (PAT) of the brain blood oxygenation, high-frequency ultrasound imaging of the brain tissue, and acoustic angiography of the cerebral blood perfusion. The multi-modal imaging platform allows the study of cerebral perfusion and oxygenation changes in the whole mouse brain after stroke. Two commonly used ischemic stroke models were evaluated: the permanent middle cerebral artery occlusion (pMCAO) model and the photothrombotic (PT) model. PAUSAT was used to image the same mouse brains before and after a stroke and quantitatively analyze both stroke models. This imaging system was able to clearly show the brain vascular changes after ischemic stroke, including significantly reduced blood perfusion and oxygenation in the stroke infarct region (ipsilateral) compared to the uninjured tissue (contralateral). The results were confirmed by both laser speckle contrast imaging and triphenyltetrazolium chloride (TTC) staining. Furthermore, stroke infarct volume in both stroke models was measured and validated by TTC staining as the ground truth. Through this study, we have demonstrated that PAUSAT can be a powerful tool in noninvasive and longitudinal preclinical studies of ischemic stroke.

Photoacoustic imaging with limited sampling: a review of machine learning approaches

Photoacoustic imaging combines high optical absorption contrast and deep acoustic penetration, and can reveal structural, molecular, and functional information about biological tissue non-invasively. Due to practical restrictions, photoacoustic imaging systems often face various challenges, such as complex system configuration, long imaging time, and/or less-than-ideal image quality, which collectively hinder their clinical application. Machine learning has been applied to improve photoacoustic imaging and mitigate the otherwise strict requirements in system setup and data acquisition. In contrast to the previous reviews of learned methods in photoacoustic computed tomography (PACT), this review focuses on the application of machine learning approaches to address the limited spatial sampling problems in photoacoustic imaging, specifically the limited view and undersampling issues. We summarize the relevant PACT works based on their training data, workflow, and model architecture. Notably, we also introduce the recent limited sampling works on the other major implementation of photoacoustic imaging, i.e., photoacoustic microscopy (PAM). With machine learning-based processing, photoacoustic imaging can achieve improved image quality with modest spatial sampling, presenting great potential for low-cost and user-friendly clinical applications.

Three-dimensional non-invasive brain imaging of ischemic stroke by integrated photoacoustic, ultrasound and angiographic tomography (PAUSAT)

We present an ischemic stroke study using our newly-developed PAUSAT system that integrates photoacoustic computed tomography (PACT), high-frequency ultrasound imaging, and acoustic angiographic tomography. PAUSAT is capable of three-dimensional (3D) imaging of the brain morphology, blood perfusion, and blood oxygenation. Using PAUSAT, we studied the hemodynamic changes in the whole mouse brain induced by two common ischemic stroke models: the permanent middle cerebral artery occlusion (pMCAO) model and the photothrombotic (PT) model. We imaged the same mouse brains before and after stroke, and quantitatively compared the two stroke models. We observed clear hemodynamic changes after ischemic stroke, including reduced blood perfusion and oxygenation. Such changes were spatially heterogenous. We also quantified the tissue infarct volume in both stroke models. The PAUSAT measurements were validated by laser speckle imaging and histology. Our results have collectively demonstrated that PAUSAT can be a valuable tool for non-invasive longitudinal studies of neurological diseases at the whole-brain scale.

Deep-E Enhanced Photoacoustic Tomography Using Three-Dimensional Reconstruction for High-Quality Vascular Imaging

Linear-array-based photoacoustic computed tomography (PACT) has been widely used in vascular imaging due to its low cost and high compatibility with current ultrasound systems. However, linear-array transducers have inherent limitations for three-dimensional imaging due to the poor elevation resolution. In this study, we introduced a deep learning-assisted data process algorithm to enhance the image quality in linear-array-based PACT. Compared to our earlier study where training was performed on 2D reconstructed data, here, we utilized 2D and 3D reconstructed data to train the two networks separately. We then fused the image data from both 2D and 3D training to get features from both algorithms. The numerical and in vivo validations indicate that our approach can improve elevation resolution, recover the true size of the object, and enhance deep vessels. Our deep learning-assisted approach can be applied to translational imaging applications that require detailed visualization of vascular features.

Semantic segmentation of multispectral photoacoustic images using deep learning

Photoacoustic (PA) imaging has the potential to revolutionize functional medical imaging in healthcare due to the valuable information on tissue physiology contained in multispectral photoacoustic measurements. Clinical translation of the technology requires conversion of the high-dimensional acquired data into clinically relevant and interpretable information. In this work, we present a deep learning-based approach to semantic segmentation of multispectral photoacoustic images to facilitate image interpretability. Manually annotated photoacoustic and ultrasound imaging data are used as reference and enable the training of a deep learning-based segmentation algorithm in a supervised manner. Based on a validation study with experimentally acquired data from 16 healthy human volunteers, we show that automatic tissue segmentation can be used to create powerful analyses and visualizations of multispectral photoacoustic images. Due to the intuitive representation of high-dimensional information, such a preprocessing algorithm could be a valuable means to facilitate the clinical translation of photoacoustic imaging.