Deep learning facilitates fully automated brain image registration of optoacoustic tomography and magnetic resonance imaging

Unsupervised Fusion of Misaligned PAT and MRI Images via Mutually Reinforcing Cross-Modality Image Generation and Registration

Photoacoustic tomography (PAT) and magnetic resonance imaging (MRI) are two advanced imaging techniques widely used in pre-clinical research. PAT has high optical contrast and deep imaging range but poor soft tissue contrast, whereas MRI provides excellent soft tissue information but poor temporal resolution. Despite recent advances in medical image fusion with pre-aligned multimodal data, PAT-MRI image fusion remains challenging due to misaligned images and spatial distortion. To address these issues, we propose an unsupervised multi-stage deep learning framework called PAMRFuse for misaligned PAT and MRI image fusion. PAMRFuse comprises a multimodal to unimodal registration network to accurately align the input PAT-MRI image pairs and a self-attentive fusion network that selects information-rich features for fusion. We employ an end-to-end mutually reinforcing mode in our registration network, which enables joint optimization of cross-modality image generation and registration. To the best of our knowledge, this is the first attempt at information fusion for misaligned PAT and MRI. Qualitative and quantitative experimental results show the excellent performance of our method in fusing PAT-MRI images of small animals captured from commercial imaging systems.

Introduction to the Optics and the Brain 2023 feature issue

A feature issue is being presented by a team of guest editors containing papers based on contributed submissions including studies presented at Optics and the Brain, held April 24-27, 2023 as part of Optica Biophotonics Congress: Optics in the Life Sciences, in Vancouver, Canada.

Monitoring mouse brain perfusion with hybrid magnetic resonance optoacoustic tomography

Progress in brain research critically depends on the development of next-generation multi-modal imaging tools capable of capturing transient functional events and multiplexed contrasts noninvasively and concurrently, thus enabling a holistic view of dynamic events in vivo. Here we report on a hybrid magnetic resonance and optoacoustic tomography (MROT) system for murine brain imaging, which incorporates an MR-compatible spherical matrix array transducer and fiber-based light illumination into a 9.4 T small animal scanner. An optimized radiofrequency coil has further been devised for whole-brain interrogation. System's utility is showcased by acquiring complementary angiographic and soft tissue anatomical contrast along with simultaneous dual-modality visualization of contrast agent dynamics in vivo.