1
|
Li J, Meng YC. Multikernel positional embedding convolutional neural network for photoacoustic reconstruction with sparse data. APPLIED OPTICS 2023; 62:8506-8516. [PMID: 38037963 DOI: 10.1364/ao.504094] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/24/2023] [Accepted: 10/14/2023] [Indexed: 12/02/2023]
Abstract
Photoacoustic imaging (PAI) is an emerging noninvasive imaging modality that merges the high contrast of optical imaging with the high resolution of ultrasonic imaging. Low-quality photoacoustic reconstruction with sparse data due to sparse spatial sampling and limited view detection is a major obstacle to the popularization of PAI for medical applications. Deep learning has been considered as the best solution to this problem in the past decade. In this paper, we propose what we believe to be a novel architecture, named DPM-UNet, which consists of the U-Net backbone with additional position embedding block and two multi-kernel-size convolution blocks, a dilated dense block and dilated multi-kernel-size convolution block. Our method was experimentally validated with both simulated data and in vivo data, achieving a SSIM of 0.9824 and a PSNR of 33.2744 dB. Furthermore, the reconstructed images of our proposed method were compared with those obtained by other advanced methods. The results have shown that our proposed DPM-UNet has a great advantage in PAI over other methods with respect to the imaging effect and memory consumption.
Collapse
|
2
|
Mondal S, Paul S, Singh N, Saha RK. Deep learning on photoacoustic tomography to remove image distortion due to inaccurate measurement of the scanning radius. BIOMEDICAL OPTICS EXPRESS 2023; 14:5817-5832. [PMID: 38021110 PMCID: PMC10659812 DOI: 10.1364/boe.501277] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/24/2023] [Revised: 09/17/2023] [Accepted: 10/04/2023] [Indexed: 12/01/2023]
Abstract
Photoacoustic tomography (PAT) is a non-invasive, non-ionizing hybrid imaging modality that holds great potential for various biomedical applications and the incorporation with deep learning (DL) methods has experienced notable advancements in recent times. In a typical 2D PAT setup, a single-element ultrasound detector (USD) is used to collect the PA signals by making a 360° full scan of the imaging region. The traditional backprojection (BP) algorithm has been widely used to reconstruct the PAT images from the acquired signals. Accurate determination of the scanning radius (SR) is required for proper image reconstruction. Even a slight deviation from its nominal value can lead to image distortion compromising the quality of the reconstruction. To address this challenge, two approaches have been developed and examined herein. The first framework includes a modified version of dense U-Net (DUNet) architecture. The second procedure involves a DL-based convolutional neural network (CNN) for image classification followed by a DUNet. The first protocol was trained with heterogeneous simulated images generated from three different phantoms to learn the relationship between the reconstructed and the corresponding ground truth (GT) images. In the case of the second scheme, the first stage was trained with the same heterogeneous dataset to classify the image type and the second stage was trained individually with the appropriate images. The performance of these architectures has been tested on both simulated and experimental images. The first method can sustain SR deviation up to approximately 6% for simulated images and 5% for experimental images and can accurately reproduce the GTs. The proposed DL-approach extends the limits further (approximately 7% and 8% for simulated and experimental images, respectively). Our results suggest that classification-based DL method does not need a precise assessment of SR for accurate PAT image formation.
Collapse
Affiliation(s)
- Sudeep Mondal
- Department of Applied Sciences, Indian Institute of Information Technology Allahabad, Prayagraj, 211015, India
| | - Subhadip Paul
- Department of Applied Sciences, Indian Institute of Information Technology Allahabad, Prayagraj, 211015, India
| | - Navjot Singh
- Department of Information Technology, Indian Institute of Information Technology Allahabad, Prayagraj, 211015, India
| | - Ratan K Saha
- Department of Applied Sciences, Indian Institute of Information Technology Allahabad, Prayagraj, 211015, India
| |
Collapse
|
3
|
Hui X, Rajendran P, Zulkifli MAI, Ling T, Pramanik M. Android mobile-platform-based image reconstruction for photoacoustic tomography. JOURNAL OF BIOMEDICAL OPTICS 2023; 28:046009. [PMID: 37122476 PMCID: PMC10133999 DOI: 10.1117/1.jbo.28.4.046009] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 02/24/2023] [Revised: 04/10/2023] [Accepted: 04/11/2023] [Indexed: 05/03/2023]
Abstract
Significance In photoacoustic tomography (PAT), numerous reconstruction algorithms have been utilized to recover initial pressure rise distribution from the acquired pressure waves. In practice, most of these reconstructions are carried out on a desktop/workstation and the mobile-based reconstructions are far-flung. In recent years, mobile phones are becoming so ubiquitous, and most of them encompass a higher computing ability. Hence, realizing PAT image reconstruction on a mobile platform is intrinsic, and it will enhance the adaptability of PAT systems with point-of-care applications. Aim To implement PAT image reconstruction in Android-based mobile platforms. Approach For implementing PAT image reconstruction in Android-based mobile platforms, we proposed an Android-based application using Python to perform beamforming process in Android phones. Results The performance of the developed application was analyzed on different mobile platforms using both simulated and experimental datasets. The results demonstrate that the developed algorithm can accomplish the image reconstruction of in vivo small animal brain dataset in 2.4 s. Furthermore, the developed application reconstructs PAT images with comparable speed and no loss of image quality compared to that on a laptop. Employing a two-fold downsampling procedure could serve as a viable solution for reducing the time needed for beamforming while preserving image quality with minimal degradation. Conclusions We proposed an Android-based application that achieves image reconstruction on cheap, small, and universally available phones instead of relatively bulky expensive desktop computers/laptops/workstations. A beamforming speed of 2.4 s is achieved without hampering the quality of the reconstructed image.
Collapse
Affiliation(s)
- Xie Hui
- Nanyang Technological University, School of Chemistry, Chemical Engineering and Biotechnology, Singapore
| | | | | | - Tong Ling
- Nanyang Technological University, School of Chemistry, Chemical Engineering and Biotechnology, Singapore
| | - Manojit Pramanik
- Iowa State University, Department of Electrical and Computer Engineering, Ames, Iowa, United States
- Address all correspondence to Manojit Pramanik,
| |
Collapse
|
4
|
Zhang Z, Jin H, Zhang W, Lu W, Zheng Z, Sharma A, Pramanik M, Zheng Y. Adaptive enhancement of acoustic resolution photoacoustic microscopy imaging via deep CNN prior. PHOTOACOUSTICS 2023; 30:100484. [PMID: 37095888 PMCID: PMC10121479 DOI: 10.1016/j.pacs.2023.100484] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/11/2023] [Accepted: 03/29/2023] [Indexed: 05/03/2023]
Abstract
Acoustic resolution photoacoustic microscopy (AR-PAM) is a promising medical imaging modality that can be employed for deep bio-tissue imaging. However, its relatively low imaging resolution has greatly hindered its wide applications. Previous model-based or learning-based PAM enhancement algorithms either require design of complex handcrafted prior to achieve good performance or lack the interpretability and flexibility that can adapt to different degradation models. However, the degradation model of AR-PAM imaging is subject to both imaging depth and center frequency of ultrasound transducer, which varies in different imaging conditions and cannot be handled by a single neural network model. To address this limitation, an algorithm integrating both learning-based and model-based method is proposed here so that a single framework can deal with various distortion functions adaptively. The vasculature image statistics is implicitly learned by a deep convolutional neural network, which served as plug and play (PnP) prior. The trained network can be directly plugged into the model-based optimization framework for iterative AR-PAM image enhancement, which fitted for different degradation mechanisms. Based on physical model, the point spread function (PSF) kernels for various AR-PAM imaging situations are derived and used for the enhancement of simulation and in vivo AR-PAM images, which collectively proved the effectiveness of proposed method. Quantitatively, the PSNR and SSIM values have all achieve best performance with the proposed algorithm in all three simulation scenarios; The SNR and CNR values have also significantly raised from 6.34 and 5.79 to 35.37 and 29.66 respectively in an in vivo testing result with the proposed algorithm.
Collapse
Affiliation(s)
- Zhengyuan Zhang
- Nanyang Technological University, School of Electrical and Electronic Engineering, 639798, Singapore
| | - Haoran Jin
- Zhejiang University, College of Mechanical Engineering, The State Key Laboratory of Fluid Power and Mechatronic Systems, Hangzhou 310027, China
| | - Wenwen Zhang
- Nanyang Technological University, School of Electrical and Electronic Engineering, 639798, Singapore
| | - Wenhao Lu
- Nanyang Technological University, School of Electrical and Electronic Engineering, 639798, Singapore
| | - Zesheng Zheng
- Nanyang Technological University, School of Electrical and Electronic Engineering, 639798, Singapore
| | - Arunima Sharma
- Johns Hopkins University, Electrical and Computer Engineering, Baltimore, MD 21218, USA
| | - Manojit Pramanik
- Iowa State University, Department of Electrical and Computer Engineering, Ames, Iowa, USA
| | - Yuanjin Zheng
- Nanyang Technological University, School of Electrical and Electronic Engineering, 639798, Singapore
- Corresponding author.
| |
Collapse
|
5
|
Hakakzadeh S, Mozaffarzadeh M, Mostafavi SM, Kavehvash Z, Rajendran P, Verweij M, de Jong N, Pramanik M. Multi-angle data acquisition to compensate transducer finite size in photoacoustic tomography. PHOTOACOUSTICS 2022; 27:100373. [PMID: 35662895 PMCID: PMC9157198 DOI: 10.1016/j.pacs.2022.100373] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/27/2022] [Revised: 05/02/2022] [Accepted: 05/13/2022] [Indexed: 05/28/2023]
Abstract
In photoacoustic tomography (PAT) systems, the tangential resolution decreases due to the finite size of the transducer as the off-center distance increases. To address this problem, we propose a multi-angle detection approach in which the transducer used for data acquisition rotates around its center (with specific angles) as well as around the scanning center. The angles are calculated based on the central frequency and diameter of the transducer and the radius of the region-of-interest (ROI). Simulations with point-like absorbers (for point-spread-function evaluation) and a vasculature phantom (for quality assessment), and experiments with ten 0.5 mm-diameter pencil leads and a leaf skeleton phantom are used for evaluation of the proposed approach. The results show that a location-independent tangential resolution is achieved with 150 spatial sampling and central rotations with angles of ±8°/±16°. With further developments, the proposed detection strategy can replace the conventional detection (rotating a transducer around ROI) in PAT.
Collapse
Affiliation(s)
- Soheil Hakakzadeh
- Department of Electrical Engineering, Sharif University of Technology, Tehran, Iran
| | - Moein Mozaffarzadeh
- Laboratory of Medical Imaging, Department of Imaging Physics, Delft University of Technology, 2628 CJ Delft, The Netherlands
| | | | - Zahra Kavehvash
- Department of Electrical Engineering, Sharif University of Technology, Tehran, Iran
| | - Praveenbalaji Rajendran
- School of Chemical and Biomedical Engineering, Nanyang Technological University, 62 Nanyang Drive, Singapore
| | - Martin Verweij
- Laboratory of Medical Imaging, Department of Imaging Physics, Delft University of Technology, 2628 CJ Delft, The Netherlands
- Department Biomedical Engineering, Thoraxcenter, Erasmus Medical Center, 3015 GD Rotterdam, The Netherlands
| | - Nico de Jong
- Laboratory of Medical Imaging, Department of Imaging Physics, Delft University of Technology, 2628 CJ Delft, The Netherlands
- Department Biomedical Engineering, Thoraxcenter, Erasmus Medical Center, 3015 GD Rotterdam, The Netherlands
| | - Manojit Pramanik
- School of Chemical and Biomedical Engineering, Nanyang Technological University, 62 Nanyang Drive, Singapore
| |
Collapse
|
6
|
Nakshatri HS, Prakash J. Model resolution matrix based deconvolution improves over non-quadratic penalization in frequency-domain photoacoustic tomography. THE JOURNAL OF THE ACOUSTICAL SOCIETY OF AMERICA 2022; 152:1345. [PMID: 36182277 DOI: 10.1121/10.0013829] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/18/2022] [Accepted: 08/11/2022] [Indexed: 06/16/2023]
Abstract
Frequency domain photoacoustic tomography is becoming more attractive due to low-cost and compact light-sources being used; however, frequency-domain implementation suffers from lower signal to noise compared to time-domain implementation. In this work, we have developed a non-quadratic based penalization framework for frequency-domain photoacoustic imaging, and further proposed a two-step model-resolution matrix based deconvolution approach to improve the reconstruction image quality. The model-resolution matrix was developed in the context of different penalty functions like l2-norm, l1-norm, Cauchy, and Geman-McClure. These model-resolution matrices were then used to perform the deconvolution operation using split augmented Lagrangian shrinkage thresholding algorithm in both full-view and limited-view configurations. The results indicated that the two-step approach outperformed the different penalty function (prior constraint) based reconstruction, with an improvement of about 20% in terms of peak signal to noise ratio and 30% in terms of structural similarity index measure. The improved image quality provided using these algorithms will have a direct impact on realizing practical frequency-domain implementation in both limited-view and full-view configurations.
Collapse
Affiliation(s)
- Hemanth S Nakshatri
- Department of Instrumentation and Applied Physics, Indian Institute of Science, C. V. Raman Avenue, Bengaluru 560 012, India
| | - Jaya Prakash
- Department of Instrumentation and Applied Physics, Indian Institute of Science, C. V. Raman Avenue, Bengaluru 560 012, India
| |
Collapse
|
7
|
Rajendran P, Pramanik M. High frame rate (∼3 Hz) circular photoacoustic tomography using single-element ultrasound transducer aided with deep learning. JOURNAL OF BIOMEDICAL OPTICS 2022; 27:066005. [PMID: 36452448 PMCID: PMC9209813 DOI: 10.1117/1.jbo.27.6.066005] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/27/2022] [Accepted: 06/01/2022] [Indexed: 05/29/2023]
Abstract
SIGNIFICANCE In circular scanning photoacoustic tomography (PAT), it takes several minutes to generate an image of acceptable quality, especially with a single-element ultrasound transducer (UST). The imaging speed can be enhanced by faster scanning (with high repetition rate light sources) and using multiple-USTs. However, artifacts arising from the sparse signal acquisition and low signal-to-noise ratio at higher scanning speeds limit the imaging speed. Thus, there is a need to improve the imaging speed of the PAT systems without hampering the quality of the PAT image. AIM To improve the frame rate (or imaging speed) of the PAT system by using deep learning (DL). APPROACH For improving the frame rate (or imaging speed) of the PAT system, we propose a novel U-Net-based DL framework to reconstruct PAT images from fast scanning data. RESULTS The efficiency of the network was evaluated on both single- and multiple-UST-based PAT systems. Both phantom and in vivo imaging demonstrate that the network can improve the imaging frame rate by approximately sixfold in single-UST-based PAT systems and by approximately twofold in multi-UST-based PAT systems. CONCLUSIONS We proposed an innovative method to improve the frame rate (or imaging speed) by using DL and with this method, the fastest frame rate of ∼ 3 Hz imaging is achieved without hampering the quality of the reconstructed image.
Collapse
Affiliation(s)
| | - Manojit Pramanik
- Nanyang Technological University, School of Chemical and Biomedical Engineering, Singapore
| |
Collapse
|
8
|
Zou Q, Bao J, Yan X. Functional Nanomaterials Based on Self-Assembly of Endogenic NIR-Absorbing Pigments for Diagnostic and Therapeutic Applications. SMALL METHODS 2022; 6:e2101359. [PMID: 35142112 DOI: 10.1002/smtd.202101359] [Citation(s) in RCA: 10] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/30/2021] [Revised: 01/05/2022] [Indexed: 06/14/2023]
Abstract
Endogenic pigments derived from hemoglobin have been successfully applied in the clinic for both imaging and therapy based on their inherent photophysical and photochemical properties, including light absorption, fluorescence emission, and producing reactive oxygen species. However, the clinically approved endogenic pigments can be excited only by UV/vis light, restricting the penetration depth of in vivo applications. Recently, endogenic pigments with NIR-absorbing properties have been explored for constructing functional nanomaterials. Here, the overview of NIR-absorbing endogenic pigments, mainly bile pigments, and melanins, as emerging building blocks for supramolecular construction of diagnostic and therapeutic nanomaterials is provided. The endogenic origins, synthetic pathways, and structural characteristics of the NIR-absorbing endogenic pigments are described. The self-assembling approaches and noncovalent interactions in fabricating the nanomaterials are emphasized. Since bile pigments and melanins are inherently photothermal agents, the resulting nanomaterials are demonstrated as promising candidates for photoacoustic imaging and photothermal therapy. Integration of additional diagnostic and therapeutic agents by the nanomaterials through chemical conjugation or physical encapsulation toward synergetic effects is also included. Especially, the degradation behaviors of the nanomaterials in biological environments are summarized. Along with the challenges, future perspectives are discussed for accelerating the ration design and clinical translation of NIR-absorbing nanomaterials.
Collapse
Affiliation(s)
- Qianli Zou
- School of Pharmacy, Anhui Medical University, Hefei, 230032, P. R. China
| | - Jianwei Bao
- School of Pharmacy, Anhui Medical University, Hefei, 230032, P. R. China
| | - Xuehai Yan
- School of Pharmacy, Anhui Medical University, Hefei, 230032, P. R. China
- State Key Laboratory of Biochemical Engineering, Institute of Process Engineering, Chinese Academy of Sciences, Beijing, 100190, China
| |
Collapse
|
9
|
Photoacoustic imaging aided with deep learning: a review. Biomed Eng Lett 2021; 12:155-173. [DOI: 10.1007/s13534-021-00210-y] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/15/2021] [Revised: 10/19/2021] [Accepted: 11/07/2021] [Indexed: 12/21/2022] Open
|