1
|
Zhong Y, Zhang S, Liu Z, Zhang X, Mo Z, Zhang Y, Hu H, Chen W, Qi L. Unsupervised Fusion of Misaligned PAT and MRI Images via Mutually Reinforcing Cross-Modality Image Generation and Registration. IEEE TRANSACTIONS ON MEDICAL IMAGING 2024; 43:1702-1714. [PMID: 38147426 DOI: 10.1109/tmi.2023.3347511] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/28/2023]
Abstract
Photoacoustic tomography (PAT) and magnetic resonance imaging (MRI) are two advanced imaging techniques widely used in pre-clinical research. PAT has high optical contrast and deep imaging range but poor soft tissue contrast, whereas MRI provides excellent soft tissue information but poor temporal resolution. Despite recent advances in medical image fusion with pre-aligned multimodal data, PAT-MRI image fusion remains challenging due to misaligned images and spatial distortion. To address these issues, we propose an unsupervised multi-stage deep learning framework called PAMRFuse for misaligned PAT and MRI image fusion. PAMRFuse comprises a multimodal to unimodal registration network to accurately align the input PAT-MRI image pairs and a self-attentive fusion network that selects information-rich features for fusion. We employ an end-to-end mutually reinforcing mode in our registration network, which enables joint optimization of cross-modality image generation and registration. To the best of our knowledge, this is the first attempt at information fusion for misaligned PAT and MRI. Qualitative and quantitative experimental results show the excellent performance of our method in fusing PAT-MRI images of small animals captured from commercial imaging systems.
Collapse
|
2
|
Bauer AQ, Gibson EA, Wang H, Srinivasan VJ. Introduction to the Optics and the Brain 2023 feature issue. BIOMEDICAL OPTICS EXPRESS 2024; 15:2110-2113. [PMID: 38633102 PMCID: PMC11019680 DOI: 10.1364/boe.517678] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/02/2024] [Indexed: 04/19/2024]
Abstract
A feature issue is being presented by a team of guest editors containing papers based on contributed submissions including studies presented at Optics and the Brain, held April 24-27, 2023 as part of Optica Biophotonics Congress: Optics in the Life Sciences, in Vancouver, Canada.
Collapse
Affiliation(s)
- Adam Q. Bauer
- Department of Radiology, Washington University in St. Louis, St. Louis, Missouri, USA
- Department of Biomedical Engineering, Washington University School of Medicine, St. Louis, Missouri, USA
| | - Emily A. Gibson
- Department of Bioengineering, University of Colorado Denver, Anschutz Medical Campus, Aurora, Colorado 80045, USA
| | - Hui Wang
- Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General Hospital/Harvard Medical School, Charlestown, MA 02129, USA
| | - Vivek J. Srinivasan
- Tech4Health Institute, NYU Langone Health, New York, New York 10010, USA
- Department of Ophthalmology, NYU Langone Health, New York, New York 10017, USA
| |
Collapse
|
3
|
Ren W, Deán-Ben XL, Skachokova Z, Augath MA, Ni R, Chen Z, Razansky D. Monitoring mouse brain perfusion with hybrid magnetic resonance optoacoustic tomography. BIOMEDICAL OPTICS EXPRESS 2023; 14:1192-1204. [PMID: 36950237 PMCID: PMC10026577 DOI: 10.1364/boe.482205] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/01/2022] [Revised: 01/10/2023] [Accepted: 01/10/2023] [Indexed: 06/18/2023]
Abstract
Progress in brain research critically depends on the development of next-generation multi-modal imaging tools capable of capturing transient functional events and multiplexed contrasts noninvasively and concurrently, thus enabling a holistic view of dynamic events in vivo. Here we report on a hybrid magnetic resonance and optoacoustic tomography (MROT) system for murine brain imaging, which incorporates an MR-compatible spherical matrix array transducer and fiber-based light illumination into a 9.4 T small animal scanner. An optimized radiofrequency coil has further been devised for whole-brain interrogation. System's utility is showcased by acquiring complementary angiographic and soft tissue anatomical contrast along with simultaneous dual-modality visualization of contrast agent dynamics in vivo.
Collapse
Affiliation(s)
- Wuwei Ren
- Institute for Biomedical Engineering and Institute of Pharmacology and Toxicology, Faculty of Medicine, University of Zurich, Zurich 8052, Switzerland
- Institute for Biomedical Engineering, Department of Information Technology and Electrical Engineering, ETH Zurich, Zurich 8093, Switzerland
- Present address: School of Information Science and Technology, ShanghaiTech University, Shanghai 201210, China
- equal contribution
| | - Xosé Luís Deán-Ben
- Institute for Biomedical Engineering and Institute of Pharmacology and Toxicology, Faculty of Medicine, University of Zurich, Zurich 8052, Switzerland
- Institute for Biomedical Engineering, Department of Information Technology and Electrical Engineering, ETH Zurich, Zurich 8093, Switzerland
- equal contribution
| | - Zhiva Skachokova
- Institute for Biomedical Engineering and Institute of Pharmacology and Toxicology, Faculty of Medicine, University of Zurich, Zurich 8052, Switzerland
| | - Mark-Aurel Augath
- Institute for Biomedical Engineering and Institute of Pharmacology and Toxicology, Faculty of Medicine, University of Zurich, Zurich 8052, Switzerland
- Institute for Biomedical Engineering, Department of Information Technology and Electrical Engineering, ETH Zurich, Zurich 8093, Switzerland
| | - Ruiqing Ni
- Institute for Biomedical Engineering and Institute of Pharmacology and Toxicology, Faculty of Medicine, University of Zurich, Zurich 8052, Switzerland
- Zurich Neuroscience Center, University of Zurich and ETH Zurich, Zurich 8093, Switzerland
- Institute for Regenerative Medicine, Faculty of Medicine, University of Zurich, Zurich 8952, Switzerland
| | - Zhenyue Chen
- Institute for Biomedical Engineering and Institute of Pharmacology and Toxicology, Faculty of Medicine, University of Zurich, Zurich 8052, Switzerland
- Institute for Biomedical Engineering, Department of Information Technology and Electrical Engineering, ETH Zurich, Zurich 8093, Switzerland
| | - Daniel Razansky
- Institute for Biomedical Engineering and Institute of Pharmacology and Toxicology, Faculty of Medicine, University of Zurich, Zurich 8052, Switzerland
- Institute for Biomedical Engineering, Department of Information Technology and Electrical Engineering, ETH Zurich, Zurich 8093, Switzerland
- Zurich Neuroscience Center, University of Zurich and ETH Zurich, Zurich 8093, Switzerland
| |
Collapse
|