1
|
Wang S, Wu R, Jia S, Diakite A, Li C, Liu Q, Zheng H, Ying L. Knowledge-driven deep learning for fast MR imaging: Undersampled MR image reconstruction from supervised to un-supervised learning. Magn Reson Med 2024; 92:496-518. [PMID: 38624162 DOI: 10.1002/mrm.30105] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/03/2023] [Revised: 03/19/2024] [Accepted: 03/20/2024] [Indexed: 04/17/2024]
Abstract
Deep learning (DL) has emerged as a leading approach in accelerating MRI. It employs deep neural networks to extract knowledge from available datasets and then applies the trained networks to reconstruct accurate images from limited measurements. Unlike natural image restoration problems, MRI involves physics-based imaging processes, unique data properties, and diverse imaging tasks. This domain knowledge needs to be integrated with data-driven approaches. Our review will introduce the significant challenges faced by such knowledge-driven DL approaches in the context of fast MRI along with several notable solutions, which include learning neural networks and addressing different imaging application scenarios. The traits and trends of these techniques have also been given which have shifted from supervised learning to semi-supervised learning, and finally, to unsupervised learning methods. In addition, MR vendors' choices of DL reconstruction have been provided along with some discussions on open questions and future directions, which are critical for the reliable imaging systems.
Collapse
Affiliation(s)
- Shanshan Wang
- Paul C Lauterbur Research Center for Biomedical Imaging, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
| | - Ruoyou Wu
- Paul C Lauterbur Research Center for Biomedical Imaging, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
| | - Sen Jia
- Paul C Lauterbur Research Center for Biomedical Imaging, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
| | - Alou Diakite
- Paul C Lauterbur Research Center for Biomedical Imaging, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
- University of Chinese Academy of Sciences, Beijing, China
| | - Cheng Li
- Paul C Lauterbur Research Center for Biomedical Imaging, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
| | - Qiegen Liu
- Department of Electronic Information Engineering, Nanchang University, Nanchang, China
| | - Hairong Zheng
- Paul C Lauterbur Research Center for Biomedical Imaging, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
| | - Leslie Ying
- Department of Biomedical Engineering and Department of Electrical Engineering, The State University of New York, Buffalo, New York, USA
| |
Collapse
|
2
|
Singh SB, Sarrami AH, Gatidis S, Varniab ZS, Chaudhari A, Daldrup-Link HE. Applications of Artificial Intelligence for Pediatric Cancer Imaging. AJR Am J Roentgenol 2024. [PMID: 38809123 DOI: 10.2214/ajr.24.31076] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/30/2024]
Abstract
Artificial intelligence (AI) is transforming medical imaging of adult patients. However, its utilization in pediatric oncology imaging remains constrained, in part due to the inherent data scarcity associated with childhood cancers. Pediatric cancers are rare, and imaging technologies are evolving rapidly, leading to insufficient data of a particular type to effectively train these algorithms. The small market size of pediatrics compared to adults could also contribute to this challenge, as market size is a driver of commercialization. This article provides an overview of the current state of AI applications for pediatric cancer imaging, including applications for medical image acquisition, processing, reconstruction, segmentation, diagnosis, staging, and treatment response monitoring. While current developments are promising, impediments due to diverse anatomies of growing children and nonstandardized imaging protocols have led to limited clinical translation thus far. Opportunities include leveraging reconstruction algorithms to achieve accelerated low-dose imaging and automating the generation of metric-based staging and treatment monitoring scores. Transfer-learning of adult-based AI models to pediatric cancers, multi-institutional data sharing, and ethical data privacy practices for pediatric patients with rare cancers will be keys to unlocking AI's full potential for clinical translation and improved outcomes for these young patients.
Collapse
Affiliation(s)
- Shashi B Singh
- Department of Radiology, Division of Pediatric Radiology, Stanford University School of Medicine, Stanford, CA, 94305
| | - Amir H Sarrami
- Department of Radiology, Division of Pediatric Radiology, Stanford University School of Medicine, Stanford, CA, 94305
| | - Sergios Gatidis
- Department of Radiology, Division of Pediatric Radiology, Stanford University School of Medicine, Stanford, CA, 94305
| | - Zahra S Varniab
- Department of Radiology, Division of Pediatric Radiology, Stanford University School of Medicine, Stanford, CA, 94305
| | - Akshay Chaudhari
- Department of Radiology, Integrative Biomedical Imaging Informatics (IBIIS), Stanford University School of Medicine, Stanford University, Stanford, CA, 94304
- Department of Biomedical Data Science, Stanford University School of Medicine, Stanford University, Stanford, CA, 94304
| | - Heike E Daldrup-Link
- Department of Radiology, Division of Pediatric Radiology, Stanford University School of Medicine, Stanford, CA, 94305
- Department of Pediatrics, Pediatric Hematology/Oncology, Lucile Packard Children's Hospital, Stanford University, Stanford, CA, 94305
| |
Collapse
|
3
|
Chen S, Duan J, Ren X, Wang J, Liu Y. DFUSNN: zero-shot dual-domain fusion unsupervised neural network for parallel MRI reconstruction. Phys Med Biol 2024; 69:105028. [PMID: 38604186 DOI: 10.1088/1361-6560/ad3dbc] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/23/2023] [Accepted: 04/11/2024] [Indexed: 04/13/2024]
Abstract
Objective. Recently, deep learning models have been used to reconstruct parallel magnetic resonance (MR) images from undersampled k-space data. However, most existing approaches depend on large databases of fully sampled MR data for training, which can be challenging or sometimes infeasible to acquire in certain scenarios. The goal is to develop an effective alternative for improved reconstruction quality that does not rely on external training datasets.Approach. We introduce a novel zero-shot dual-domain fusion unsupervised neural network (DFUSNN) for parallel MR imaging reconstruction without any external training datasets. We employ the Noise2Noise (N2N) network for the reconstruction in the k-space domain, integrate phase and coil sensitivity smoothness priors into the k-space N2N network, and use an early stopping criterion to prevent overfitting. Additionally, we propose a dual-domain fusion method based on Bayesian optimization to enhance reconstruction quality efficiently.Results. Simulation experiments conducted on three datasets with different undersampling patterns showed that the DFUSNN outperforms all other competing unsupervised methods and the one-shot Hankel-k-space generative model (HKGM). The DFUSNN also achieves comparable results to the supervised Deep-SLR method.Significance. The novel DFUSNN model offers a viable solution for reconstructing high-quality MR images without the need for external training datasets, thereby overcoming a major hurdle in scenarios where acquiring fully sampled MR data is difficult.
Collapse
Affiliation(s)
- Shengyi Chen
- Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming 650500, People's Republic of China
| | - Jizhong Duan
- Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming 650500, People's Republic of China
| | - Xinmin Ren
- Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming 650500, People's Republic of China
| | - Junfeng Wang
- Department of Hepatobiliary Surgery, First People's Hospital of Yunnan Province, Kunming 650030, People's Republic of China
| | - Yu Liu
- School of Microelectronics, Tianjin University, Tianjin 300072, People's Republic of China
| |
Collapse
|
4
|
Yoon MA, Gold GE, Chaudhari AS. Accelerated Musculoskeletal Magnetic Resonance Imaging. J Magn Reson Imaging 2023. [PMID: 38156716 DOI: 10.1002/jmri.29205] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/24/2023] [Revised: 12/13/2023] [Accepted: 12/14/2023] [Indexed: 01/03/2024] Open
Abstract
With a substantial growth in the use of musculoskeletal MRI, there has been a growing need to improve MRI workflow, and faster imaging has been suggested as one of the solutions for a more efficient examination process. Consequently, there have been considerable advances in accelerated MRI scanning methods. This article aims to review the basic principles and applications of accelerated musculoskeletal MRI techniques including widely used conventional acceleration methods, more advanced deep learning-based techniques, and new approaches to reduce scan time. Specifically, conventional accelerated MRI techniques, including parallel imaging, compressed sensing, and simultaneous multislice imaging, and deep learning-based accelerated MRI techniques, including undersampled MR image reconstruction, super-resolution imaging, artifact correction, and generation of unacquired contrast images, are discussed. Finally, new approaches to reduce scan time, including synthetic MRI, novel sequences, and new coil setups and designs, are also reviewed. We believe that a deep understanding of these fast MRI techniques and proper use of combined acceleration methods will synergistically improve scan time and MRI workflow in daily practice. EVIDENCE LEVEL: 3 TECHNICAL EFFICACY: Stage 1.
Collapse
Affiliation(s)
- Min A Yoon
- Department of Radiology, Stanford University, Stanford, California, USA
- Department of Radiology and Research Institute of Radiology, University of Ulsan College of Medicine, Asan Medical Center, Seoul, South Korea
| | - Garry E Gold
- Department of Radiology, Stanford University, Stanford, California, USA
- Department of Orthopaedic Surgery, Stanford University, Stanford, California, USA
- Department of Bioengineering, Stanford University, Stanford, California, USA
| | | |
Collapse
|
5
|
Millard C, Chiew M. A Theoretical Framework for Self-Supervised MR Image Reconstruction Using Sub-Sampling via Variable Density Noisier2Noise. IEEE TRANSACTIONS ON COMPUTATIONAL IMAGING 2023; 9:707-720. [PMID: 37600280 PMCID: PMC7614963 DOI: 10.1109/tci.2023.3299212] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 08/22/2023]
Abstract
In recent years, there has been attention on leveraging the statistical modeling capabilities of neural networks for reconstructing sub-sampled Magnetic Resonance Imaging (MRI) data. Most proposed methods assume the existence of a representative fully-sampled dataset and use fully-supervised training. However, for many applications, fully sampled training data is not available, and may be highly impractical to acquire. The development and understanding of self-supervised methods, which use only sub-sampled data for training, are therefore highly desirable. This work extends the Noisier2Noise framework, which was originally constructed for self-supervised denoising tasks, to variable density sub-sampled MRI data. We use the Noisier2Noise framework to analytically explain the performance of Self-Supervised Learning via Data Undersampling (SSDU), a recently proposed method that performs well in practice but until now lacked theoretical justification. Further, we propose two modifications of SSDU that arise as a consequence of the theoretical developments. Firstly, we propose partitioning the sampling set so that the subsets have the same type of distribution as the original sampling mask. Secondly, we propose a loss weighting that compensates for the sampling and partitioning densities. On the fastMRI dataset we show that these changes significantly improve SSDU's image restoration quality and robustness to the partitioning parameters.
Collapse
Affiliation(s)
- Charles Millard
- the Wellcome Centre for Integrative Neuroimaging, FMRIB, Nuffield Department of Clinical Neurosciences, University of Oxford, OX3 9DU Oxford, U.K
| | - Mark Chiew
- the Wellcome Centre for Integrative Neuroimaging, FMRIB, Nuffield Department of Clinical Neurosciences, University of Oxford, OX3 9DU Oxford, U.K., and with the Department of Medical Biophysics, University of Toronto, Toronto, ON M5S 1A1, Canada, and also with the Canada and Physical Sciences, Sunnybrook Research Institute, Toronto, ON M4N 3M5, Canada
| |
Collapse
|