1
|
Poimala J, Cox B, Hauptmann A. Compensating unknown speed of sound in learned fast 3D limited-view photoacoustic tomography. PHOTOACOUSTICS 2024; 37:100597. [PMID: 38425677 PMCID: PMC10901832 DOI: 10.1016/j.pacs.2024.100597] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 04/29/2022] [Revised: 08/15/2023] [Accepted: 02/16/2024] [Indexed: 03/02/2024]
Abstract
Real-time applications in three-dimensional photoacoustic tomography from planar sensors rely on fast reconstruction algorithms that assume the speed of sound (SoS) in the tissue is homogeneous. Moreover, the reconstruction quality depends on the correct choice for the constant SoS. In this study, we discuss the possibility of ameliorating the problem of unknown or heterogeneous SoS distributions by using learned reconstruction methods. This can be done by modelling the uncertainties in the training data. In addition, a correction term can be included in the learned reconstruction method. We investigate the influence of both and while a learned correction component can improve reconstruction quality further, we show that a careful choice of uncertainties in the training data is the primary factor to overcome unknown SoS. We support our findings with simulated and in vivo measurements in 3D.
Collapse
Affiliation(s)
- Jenni Poimala
- Research Unit of Mathematical Sciences, University of Oulu, Finland
| | - Ben Cox
- Department of Medical Physics and Biomedical Engineering, University College London, UK
| | - Andreas Hauptmann
- Research Unit of Mathematical Sciences, University of Oulu, Finland
- Department of Computer Science, University College London, UK
| |
Collapse
|
2
|
Vera M, González MG, Vega LR. Invariant representations in deep learning for optoacoustic imaging. THE REVIEW OF SCIENTIFIC INSTRUMENTS 2023; 94:2888187. [PMID: 37140340 DOI: 10.1063/5.0139286] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/19/2022] [Accepted: 04/24/2023] [Indexed: 05/05/2023]
Abstract
Image reconstruction in optoacoustic tomography (OAT) is a trending learning task highly dependent on measured physical magnitudes present at sensing time. A large number of different settings and also the presence of uncertainties or partial knowledge of parameters can lead to reconstruction algorithms that are specifically tailored and designed to a particular configuration, which could not be the one that will ultimately be faced in a final practical situation. Being able to learn reconstruction algorithms that are robust to different environments (e.g., the different OAT image reconstruction settings) or invariant to such environments is highly valuable because it allows us to focus on what truly matters for the application at hand and discard what are considered spurious features. In this work, we explore the use of deep learning algorithms based on learning invariant and robust representations for the OAT inverse problem. In particular, we consider the application of the ANDMask scheme due to its easy adaptation to the OAT problem. Numerical experiments are conducted showing that when out-of-distribution generalization (against variations in parameters such as the location of the sensors) is imposed, there is no degradation of the performance and, in some cases, it is even possible to achieve improvements with respect to standard deep learning approaches where invariance robustness is not explicitly considered.
Collapse
Affiliation(s)
- M Vera
- Universidad de Buenos Aires, Facultad de Ingeniería, Paseo Colón 850, C1063ACV Buenos Aires, Argentina
- Consejo Nacional de Investigaciones Científicas y Técnicas, (CONICET), Godoy Cruz, 2290, C1425FQB Buenos Aires, Argentina
| | - M G González
- Universidad de Buenos Aires, Facultad de Ingeniería, Paseo Colón 850, C1063ACV Buenos Aires, Argentina
- Consejo Nacional de Investigaciones Científicas y Técnicas, (CONICET), Godoy Cruz, 2290, C1425FQB Buenos Aires, Argentina
| | - L Rey Vega
- Universidad de Buenos Aires, Facultad de Ingeniería, Paseo Colón 850, C1063ACV Buenos Aires, Argentina
- Consejo Nacional de Investigaciones Científicas y Técnicas, (CONICET), Godoy Cruz, 2290, C1425FQB Buenos Aires, Argentina
| |
Collapse
|
3
|
Muller JW, Arabul MÜ, Schwab HM, Rutten MCM, van Sambeek MRHM, Wu M, Lopata RGP. Modeling toolchain for realistic simulation of photoacoustic data acquisition. JOURNAL OF BIOMEDICAL OPTICS 2022; 27:096005. [PMID: 36104838 PMCID: PMC9470848 DOI: 10.1117/1.jbo.27.9.096005] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 06/07/2022] [Accepted: 08/31/2022] [Indexed: 06/15/2023]
Abstract
SIGNIFICANCE Physics-based simulations of photoacoustic (PA) signals are used to validate new methods, to characterize PA setups and to generate training datasets for machine learning. However, a thoroughly validated PA simulation toolchain that can simulate realistic images is still lacking. AIM A quantitative toolchain was developed to model PA image acquisition in complex tissues, by simulating both the optical fluence and the acoustic wave propagation. APPROACH Sampling techniques were developed to decrease artifacts in acoustic simulations. The performance of the simulations was analyzed by measuring the point spread function (PSF) and using a rotatable three-channel phantom, filled with cholesterol, a human carotid plaque sample, and porcine blood. Ex vivo human plaque samples were simulated to validate the methods in more complex tissues. RESULTS The sampling techniques could enhance the quality of the simulated PA images effectively. The resolution and intensity of the PSF in the turbid medium matched the experimental data well. Overall, the appearance, signal-to-noise ratio and speckle of the images could be simulated accurately. CONCLUSIONS A PA toolchain was developed and validated, and the results indicate a great potential of PA simulations in more complex and heterogeneous media.
Collapse
Affiliation(s)
- Jan-Willem Muller
- Eindhoven University of Technology, Photoacoustics and Ultrasound Laboratory Eindhoven, Department of Biomedical Engineering, Eindhoven, The Netherlands
- Catharina Hospital, Department of Vascular Surgery, Eindhoven, The Netherlands
| | - Mustafa Ü. Arabul
- Eindhoven University of Technology, Photoacoustics and Ultrasound Laboratory Eindhoven, Department of Biomedical Engineering, Eindhoven, The Netherlands
| | - Hans-Martin Schwab
- Eindhoven University of Technology, Photoacoustics and Ultrasound Laboratory Eindhoven, Department of Biomedical Engineering, Eindhoven, The Netherlands
| | - Marcel C. M. Rutten
- Cardiovascular Biomechanics Group, Department of Biomedical Engineering, Eindhoven, The Netherlands
| | - Marc R. H. M. van Sambeek
- Eindhoven University of Technology, Photoacoustics and Ultrasound Laboratory Eindhoven, Department of Biomedical Engineering, Eindhoven, The Netherlands
- Catharina Hospital, Department of Vascular Surgery, Eindhoven, The Netherlands
| | - Min Wu
- Eindhoven University of Technology, Photoacoustics and Ultrasound Laboratory Eindhoven, Department of Biomedical Engineering, Eindhoven, The Netherlands
| | - Richard G. P. Lopata
- Eindhoven University of Technology, Photoacoustics and Ultrasound Laboratory Eindhoven, Department of Biomedical Engineering, Eindhoven, The Netherlands
| |
Collapse
|
4
|
Hirsch L, González MG, Rey Vega L. On the robustness of model-based algorithms for photoacoustic tomography: Comparison between time and frequency domains. THE REVIEW OF SCIENTIFIC INSTRUMENTS 2021; 92:114901. [PMID: 34852518 DOI: 10.1063/5.0065966] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/06/2021] [Accepted: 10/09/2021] [Indexed: 06/13/2023]
Abstract
For photoacoustic image reconstruction, certain parameters such as sensor positions and speed of sound have a major impact on the reconstruction process and must be carefully determined before data acquisition. Uncertainties in these parameters can lead to errors produced by a modeling mismatch, hindering the reconstruction process and severely affecting the resulting image quality. Therefore, in this work, we study how modeling errors arising from uncertainty in sensor locations affect the images obtained by matrix model-based reconstruction algorithms based on time domain and frequency domain models of the photoacoustic problem. The effects on the reconstruction performance with respect to the uncertainty in the knowledge of the sensors location are compared and analyzed both in a qualitative and quantitative fashion for both time and frequency models. Ultimately, our study shows that the frequency domain approach is more sensitive to this kind of modeling errors. These conclusions are supported by numerical experiments and a theoretical sensitivity analysis of the mathematical operator for the direct problem.
Collapse
Affiliation(s)
- L Hirsch
- Universidad de Buenos Aires, Facultad de Ingeniería, Paseo Colón 850, C1063ACV Buenos Aires, Argentina
| | - M G González
- Universidad de Buenos Aires, Facultad de Ingeniería, Paseo Colón 850, C1063ACV Buenos Aires, Argentina
| | - L Rey Vega
- Universidad de Buenos Aires, Facultad de Ingeniería, Paseo Colón 850, C1063ACV Buenos Aires, Argentina
| |
Collapse
|
5
|
Sahlstrom T, Pulkkinen A, Leskinen J, Tarvainen T. Computationally Efficient Forward Operator for Photoacoustic Tomography Based on Coordinate Transformations. IEEE TRANSACTIONS ON ULTRASONICS, FERROELECTRICS, AND FREQUENCY CONTROL 2021; 68:2172-2182. [PMID: 33600313 DOI: 10.1109/tuffc.2021.3060175] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/12/2023]
Abstract
Photoacoustic tomography (PAT) is an imaging modality that utilizes the photoacoustic effect. In PAT, a photoacoustic image is computed from measured data by modeling ultrasound propagation in the imaged domain and solving an inverse problem utilizing a discrete forward operator. However, in realistic measurement geometries with several ultrasound transducers and relatively large imaging volume, an explicit formation and use of the forward operator can be computationally prohibitively expensive. In this work, we propose a transformation-based approach for efficient modeling of photoacoustic signals and reconstruction of photoacoustic images. In the approach, the forward operator is constructed for a reference ultrasound transducer and expanded into a general measurement geometry using transformations that map the formulated forward operator in local coordinates to the global coordinates of the measurement geometry. The inverse problem is solved using a Bayesian framework. The approach is evaluated with numerical simulations and experimental data. The results show that the proposed approach produces accurate 3-D photoacoustic images with a significantly reduced computational cost both in memory requirements and time. In the studied cases, depending on the computational factors, such as discretization, over the 30-fold reduction in memory consumption was achieved without a reduction in image quality compared to a conventional approach.
Collapse
|
6
|
Gröhl J, Schellenberg M, Dreher K, Maier-Hein L. Deep learning for biomedical photoacoustic imaging: A review. PHOTOACOUSTICS 2021; 22:100241. [PMID: 33717977 PMCID: PMC7932894 DOI: 10.1016/j.pacs.2021.100241] [Citation(s) in RCA: 80] [Impact Index Per Article: 26.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/05/2020] [Revised: 01/18/2021] [Accepted: 01/20/2021] [Indexed: 05/04/2023]
Abstract
Photoacoustic imaging (PAI) is a promising emerging imaging modality that enables spatially resolved imaging of optical tissue properties up to several centimeters deep in tissue, creating the potential for numerous exciting clinical applications. However, extraction of relevant tissue parameters from the raw data requires the solving of inverse image reconstruction problems, which have proven extremely difficult to solve. The application of deep learning methods has recently exploded in popularity, leading to impressive successes in the context of medical imaging and also finding first use in the field of PAI. Deep learning methods possess unique advantages that can facilitate the clinical translation of PAI, such as extremely fast computation times and the fact that they can be adapted to any given problem. In this review, we examine the current state of the art regarding deep learning in PAI and identify potential directions of research that will help to reach the goal of clinical applicability.
Collapse
Affiliation(s)
- Janek Gröhl
- German Cancer Research Center, Computer Assisted Medical Interventions, Heidelberg, Germany
- Heidelberg University, Medical Faculty, Heidelberg, Germany
| | - Melanie Schellenberg
- German Cancer Research Center, Computer Assisted Medical Interventions, Heidelberg, Germany
| | - Kris Dreher
- German Cancer Research Center, Computer Assisted Medical Interventions, Heidelberg, Germany
- Heidelberg University, Faculty of Physics and Astronomy, Heidelberg, Germany
| | - Lena Maier-Hein
- German Cancer Research Center, Computer Assisted Medical Interventions, Heidelberg, Germany
- Heidelberg University, Medical Faculty, Heidelberg, Germany
- Heidelberg University, Faculty of Mathematics and Computer Science, Heidelberg, Germany
| |
Collapse
|
7
|
Godefroy G, Arnal B, Bossy E. Compensating for visibility artefacts in photoacoustic imaging with a deep learning approach providing prediction uncertainties. PHOTOACOUSTICS 2021; 21:100218. [PMID: 33364161 PMCID: PMC7750172 DOI: 10.1016/j.pacs.2020.100218] [Citation(s) in RCA: 18] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/24/2020] [Revised: 10/15/2020] [Accepted: 10/17/2020] [Indexed: 05/04/2023]
Abstract
Conventional photoacoustic imaging may suffer from the limited view and bandwidth of ultrasound transducers. A deep learning approach is proposed to handle these problems and is demonstrated both in simulations and in experiments on a multi-scale model of leaf skeleton. We employed an experimental approach to build the training and the test sets using photographs of the samples as ground truth images. Reconstructions produced by the neural network show a greatly improved image quality as compared to conventional approaches. In addition, this work aimed at quantifying the reliability of the neural network predictions. To achieve this, the dropout Monte-Carlo procedure is applied to estimate a pixel-wise degree of confidence on each predicted picture. Last, we address the possibility to use transfer learning with simulated data in order to drastically limit the size of the experimental dataset.
Collapse
Affiliation(s)
| | - Bastien Arnal
- Univ. Grenoble Alpes, CNRS, LIPhy, 38000 Grenoble, France
| | - Emmanuel Bossy
- Univ. Grenoble Alpes, CNRS, LIPhy, 38000 Grenoble, France
| |
Collapse
|
8
|
Sahlstrom T, Pulkkinen A, Tick J, Leskinen J, Tarvainen T. Modeling of Errors Due to Uncertainties in Ultrasound Sensor Locations in Photoacoustic Tomography. IEEE TRANSACTIONS ON MEDICAL IMAGING 2020; 39:2140-2150. [PMID: 31940525 DOI: 10.1109/tmi.2020.2966297] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/10/2023]
Abstract
Photoacoustic tomography is an imaging modality based on the photoacoustic effect caused by the absorption of an externally introduced light pulse. In the inverse problem of photoacoustic tomography, the initial pressure generated through the photoacoustic effect is estimated from a measured photoacoustic time-series utilizing a forward model for ultrasound propagation. Due to the ill-posedness of the inverse problem, errors in the forward model or measurements can result in significant errors in the solution of the inverse problem. In this work, we study modeling of errors caused by uncertainties in ultrasound sensor locations in photoacoustic tomography using a Bayesian framework. The approach is evaluated with simulated and experimental data. The results indicate that the inverse problem of photoacoustic tomography is sensitive even to small uncertainties in sensor locations. Furthermore, these uncertainties can lead to significant errors in the estimates and reduction of the quality of the photoacoustic images. In this work, we show that the errors due to uncertainties in ultrasound sensor locations can be modeled and compensated using Bayesian approximation error modeling.
Collapse
|