Photoacoustic Tomography Image Restoration With Measured Spatially Variant Point Spread Functions

Image restoration for ring-array photoacoustic tomography based on an attention mechanism driven conditional generative adversarial network

Ring-Array photoacoustic tomography (PAT) systems have shown great promise in non-invasive biomedical imaging. However, images produced by these systems often suffer from quality degradation due to non-ideal imaging conditions, with common issues including blurring and streak artifacts. To address these challenges, we propose an image restoration method based on a conditional generative adversarial network (CGAN) framework. Our approach integrates a hybrid spatial and channel attention mechanism within a Residual Shifted Window Transformer Module (RSTM) to enhance the generator's performance. Additionally, we have developed a comprehensive loss function to balance pixel-level accuracy, detail preservation, and perceptual quality. We further incorporate a gamma correction module to enhance the contrast of the network's output. Experimental results on both simulated and in vivo data demonstrate that our method significantly improves resolution and restores overall image quality.

Organ-level instance segmentation enables continuous time-space-spectrum analysis of pre-clinical abdominal photoacoustic tomography images

Photoacoustic tomography (PAT), as a novel biomedical imaging technique, is able to capture temporal, spatial and spectral tomographic information from organisms. Organ-level multi-parametric analysis of continuous PAT images are of interest since it enables the quantification of organ specific morphological and functional parameters in small animals. Accurate organ delineation is imperative for organ-level image analysis, yet the low contrast and blurred organ boundaries in PAT images pose challenge for their precise segmentation. Fortunately, shared structural information among continuous images in the time-space-spectrum domain may be used to enhance segmentation. In this paper, we introduce a structure fusion enhanced graph convolutional network (SFE-GCN), which aims at automatically segmenting major organs including the body, liver, kidneys, spleen, vessel and spine of abdominal PAT image of mice. SFE-GCN enhances the structural feature of organs by fusing information in continuous image sequence captured at time, space and spectrum domains. As validated on large-scale datasets across different imaging scenarios, our method not only preserves fine structural details but also ensures anatomically aligned organ contours. Most importantly, this study explores the application of SFE-GCN in multi-dimensional organ image analysis, including organ-based dynamic morphological analysis, organ-wise light fluence correction and segmentation-enhanced spectral un-mixing. Code will be released at https://github.com/lzc-smu/SFEGCN.git.

Unsupervised adversarial neural network for enhancing vasculature in photoacoustic tomography images using optical coherence tomography angiography

Photoacoustic tomography (PAT) is a powerful imaging modality for visualizing tissue physiology and exogenous contrast agents. However, PAT faces challenges in visualizing deep-seated vascular structures due to light scattering, absorption, and reduced signal intensity with depth. Optical coherence tomography angiography (OCTA) offers high-contrast visualization of vasculature networks, yet its imaging depth is limited to a millimeter scale. Herein, we propose OCPA-Net, a novel unsupervised deep learning method that utilizes the rich vascular feature of OCTA to enhance PAT images. Trained on unpaired OCTA and PAT images, OCPA-Net incorporates a vessel-aware attention module to enhance deep-seated vessel details captured from OCTA. It leverages a domain-adversarial loss function to enforce structural consistency and a novel identity invariant loss to mitigate excessive image content generation. We validate the structural fidelity of OCPA-Net on simulation experiments, and then demonstrate its vascular enhancement performance on in vivo imaging experiments of tumor-bearing mice and contrast-enhanced pregnant mice. The results show the promise of our method for comprehensive vessel-related image analysis in preclinical research applications.

Spiral scanning and self-supervised image reconstruction enable ultra-sparse sampling multispectral photoacoustic tomography

Multispectral photoacoustic tomography (PAT) is an imaging modality that utilizes the photoacoustic effect to achieve non-invasive and high-contrast imaging of internal tissues but also molecular functional information derived from multi-spectral measurements. However, the hardware cost and computational demand of a multispectral PAT system consisting of up to thousands of detectors are huge. To address this challenge, we propose an ultra-sparse spiral sampling strategy for multispectral PAT, which we named U3S-PAT. Our strategy employs a sparse ring-shaped transducer that, when switching excitation wavelengths, simultaneously rotates and translates. This creates a spiral scanning pattern with multispectral angle-interlaced sampling. To solve the highly ill-conditioned image reconstruction problem, we propose a self-supervised learning method that is able to introduce structural information shared during spiral scanning. We simulate the proposed U3S-PAT method on a commercial PAT system and conduct in vivo animal experiments to verify its performance. The results show that even with a sparse sampling rate as low as 1/30, our U3S-PAT strategy achieves similar reconstruction and spectral unmixing accuracy as non-spiral dense sampling. Given its ability to dramatically reduce the time required for three-dimensional multispectral scanning, our U3S-PAT strategy has the potential to perform volumetric molecular imaging of dynamic biological activities.

Image restoration for ring-array photoacoustic tomography system based on blind spatially rotational deconvolution

Ring-array photoacoustic tomography (PAT) system has been widely used in noninvasive biomedical imaging. However, the reconstructed image usually suffers from spatially rotational blur and streak artifacts due to the non-ideal imaging conditions. To improve the reconstructed image towards higher quality, we propose a concept of spatially rotational convolution to formulate the image blur process, then we build a regularized restoration problem model accordingly and design an alternating minimization algorithm which is called blind spatially rotational deconvolution to achieve the restored image. Besides, we also present an image preprocessing method based on the proposed algorithm to remove the streak artifacts. We take experiments on phantoms and in vivo biological tissues for evaluation, the results show that our approach can significantly enhance the resolution of the image obtained from ring-array PAT system and remove the streak artifacts effectively.

Model-based quantitative photoacoustic tomography with directional total variation

In photoacoustic tomography (PAT), acoustic inversion aims to recover the spatial distribution of light energy deposition within the imaging object from the signals captured by detectors. To achieve quantitative imaging, optical inversion is further employed to derive absorption coefficient (AC) images. However, limitations such as restricted detection angles and inherent noise lead to substantial artifacts and degradation in the quality of PAT images, consequently affecting the accuracy of optical inversion results. In this study, we propose a directional total variation constrained optical inversion model to reconstruct the AC image. By incorporating anatomy prior information into the optical inversion process, our method can effectively suppress artifacts in AC images while maintaining structural integrity. Simulation, phantom, and in vivo experimental results demonstrate that our method significantly improves the reconstructed AC image quality. Our method provides a reliable foundation for achieving high-quality quantitative PAT imaging.

Unsupervised Fusion of Misaligned PAT and MRI Images via Mutually Reinforcing Cross-Modality Image Generation and Registration

Photoacoustic tomography (PAT) and magnetic resonance imaging (MRI) are two advanced imaging techniques widely used in pre-clinical research. PAT has high optical contrast and deep imaging range but poor soft tissue contrast, whereas MRI provides excellent soft tissue information but poor temporal resolution. Despite recent advances in medical image fusion with pre-aligned multimodal data, PAT-MRI image fusion remains challenging due to misaligned images and spatial distortion. To address these issues, we propose an unsupervised multi-stage deep learning framework called PAMRFuse for misaligned PAT and MRI image fusion. PAMRFuse comprises a multimodal to unimodal registration network to accurately align the input PAT-MRI image pairs and a self-attentive fusion network that selects information-rich features for fusion. We employ an end-to-end mutually reinforcing mode in our registration network, which enables joint optimization of cross-modality image generation and registration. To the best of our knowledge, this is the first attempt at information fusion for misaligned PAT and MRI. Qualitative and quantitative experimental results show the excellent performance of our method in fusing PAT-MRI images of small animals captured from commercial imaging systems.

Point spread function modeling for photoacoustic tomography - I: three-dimensional detection geometries

Photoacoustic computed tomography (PACT) has been under intensive investigation as a promising noninvasive biomedical imaging modality. Various acoustic detector arrays have been developed to achieve enhanced imaging performance. In this paper, we study the effect of the detection geometry on image quality through point spread function (PSF) modeling based on back-projection image reconstruction. Three commonly-used three-dimensional detection geometries, namely, spherical, cylindrical, and planar detector arrays, are investigated. The effect of detector bandwidth and aperture on PSF in these detection geometries is also studied. This work provides a performance evaluation tool for acoustic detector arrays used in PACT and can be helpful in the design and selection of detector arrays in practical imaging applications.

CylinGCN: Cylindrical structures segmentation in 3D biomedical optical imaging by a contour-based graph convolutional network

Cylindrical organs, e.g., blood vessels, airways, and intestines, are ubiquitous structures in biomedical optical imaging analysis. Image segmentation of these structures serves as a vital step in tissue physiology analysis. Traditional model-driven segmentation methods seek to fit the structure by constructing a corresponding topological geometry based on domain knowledge. Classification-based deep learning methods neglect the geometric features of the cylindrical structure and therefore cannot ensure the continuity of the segmentation surface. In this paper, by treating the cylindrical structures as a 3D graph, we introduce a novel contour-based graph neural network for 3D cylindrical structure segmentation in biomedical optical imaging. Our proposed method, which we named CylinGCN, adopts a novel learnable framework that extracts semantic features and complex topological relationships in the 3D volumetric data to achieve continuous and effective 3D segmentation. Our CylinGCN consists of a multiscale 3D semantic feature extractor for extracting inter-frame multiscale semantic features, and a residual graph convolutional network (GCN) contour generator that combines the semantic features and cylindrical topological priors to generate segmentation contours. We tested the CylinGCN framework on two types of optical tomographic imaging data, small animal whole body photoacoustic tomography (PAT) and endoscopic airway optical coherence tomography (OCT), and the results show that CylinGCN achieves state-of-the-art performance. Code will be released at https://github.com/lzc-smu/CylinGCN.git.

Learning spatially variant degradation for unsupervised blind photoacoustic tomography image restoration

Photoacoustic tomography (PAT) images contain inherent distortions due to the imaging system and heterogeneous tissue properties. Improving image quality requires the removal of these system distortions. While model-based approaches and data-driven techniques have been proposed for PAT image restoration, achieving accurate and robust image recovery remains challenging. Recently, deep-learning-based image deconvolution approaches have shown promise for image recovery. However, PAT imaging presents unique challenges, including spatially varying resolution and the absence of ground truth data. Consequently, there is a pressing need for a novel learning strategy specifically tailored for PAT imaging. Herein, we propose a configurable network model named Deep hybrid Image-PSF Prior (DIPP) that builds upon the physical image degradation model of PAT. DIPP is an unsupervised and deeply learned network model that aims to extract the ideal PAT image from complex system degradation. Our DIPP framework captures the degraded information solely from the acquired PAT image, without relying on ground truth or labeled data for network training. Additionally, we can incorporate the experimentally measured Point Spread Functions (PSFs) of the specific PAT system as a reference to further enhance performance. To evaluate the algorithm's effectiveness in addressing multiple degradations in PAT, we conduct extensive experiments using simulation images, publicly available datasets, phantom images, and in vivo small animal imaging data. Comparative analyses with classical analytical methods and state-of-the-art deep learning models demonstrate that our DIPP approach achieves significantly improved restoration results in terms of image details and contrast.

Spatially-variant image deconvolution for photoacoustic tomography

Photoacoustic tomography (PAT) system can reconstruct images of biological tissues with high resolution and contrast. However, in practice, the PAT images are usually degraded by spatially variant blur and streak artifacts due to the non-ideal imaging conditions and chosen reconstruction algorithms. Therefore, in this paper, we propose a two-phase restoration method to progressively improve the image quality. In the first phase, we design a precise device and measuring method to obtain spatially variant point spread function samples at preset positions of the PAT system in image domain, then we adopt principal component analysis and radial basis function interpolation to model the entire spatially variant point spread function. Afterwards, we propose a sparse logarithmic gradient regularized Richardson-Lucy (SLG-RL) algorithm to deblur the reconstructed PAT images. In the second phase, we present a novel method called deringing which is also based on SLG-RL to remove the streak artifacts. Finally, we evaluate our method with simulation, phantom and in vivo experiments, respectively. All the results show that our method can significantly improve the quality of PAT images.

Pixel-wise reconstruction of tissue absorption coefficients in photoacoustic tomography using a non-segmentation iterative method

In Photoacoustic Tomography (PAT), the acquired image represents a light energy deposition map of the imaging object. For quantitative imaging, the PAT image is converted into an absorption coefficient (μ a ) map by dividing the light fluence (LF). Previous methods usually assume a uniform tissueμ a distribution, and consequently degrade the LF correction results. Here, we propose a simple method to reconstruct the pixel-wiseμ a map. Our method is based on a non-segmentation-based iterative algorithm, which alternately optimizes the LF distribution and theμ a map. Using simulation data, as well as phantom and animal data, we implemented our algorithm and compared it to segmentation-based correction methods. The results show that our method can obtain accurate estimation of the LF distribution and therefore improve the image quality and feature visibility of theμ a map. Our method may facilitate efficient calculation of the concentration distributions of endogenous and exogenous agents in vivo.

MRI Information-Based Correction and Restoration of Photoacoustic Tomography

As an emerging molecular imaging modality, Photoacoustic Tomography (PAT) is capable of mapping tissue physiological metabolism and exogenous contrast agent information with high specificity. Due to its ultrasonic detection mechanism, the precise localization of targeted lesions has long been a challenge for PAT imaging. The poor soft-tissue contrast of the PAT image makes this process difficult and inaccurate. To meet this challenge, in this study, we first make use of the rich and clear structural information brought about by another advanced imaging modality, Magnetic Resonance Imaging (MRI), to assist organ segmentation and correct for the light fluence attenuation of PAT. We demonstrate improved feature visibility and enhanced localization of endogenous and exogenous agents in the fluence corrected PAT images. Compared with PAT-based methods, the contrast-to-noise ratio (CNR) of our MRI-assisted method increases by 29.1% in live animal experiments. Furthermore, we show that the co-registered MRI image can also be incorporated into PAT image restoration, and achieves improved anatomical landscape and soft-tissue contrast (CNR increased by 25.36%) while preserving similar spatial resolution. This PAT-MRI combination provides excellent structural, functional and molecular images of the subject, and may enable more comprehensive analysis of various preclinical research applications.