1
|
Huang J, Wu Y, Wang F, Fang Y, Nan Y, Alkan C, Abraham D, Liao C, Xu L, Gao Z, Wu W, Zhu L, Chen Z, Lally P, Bangerter N, Setsompop K, Guo Y, Rueckert D, Wang G, Yang G. Data- and Physics-Driven Deep Learning Based Reconstruction for Fast MRI: Fundamentals and Methodologies. IEEE Rev Biomed Eng 2025; 18:152-171. [PMID: 39437302 DOI: 10.1109/rbme.2024.3485022] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/25/2024]
Abstract
Magnetic Resonance Imaging (MRI) is a pivotal clinical diagnostic tool, yet its extended scanning times often compromise patient comfort and image quality, especially in volumetric, temporal and quantitative scans. This review elucidates recent advances in MRI acceleration via data and physics-driven models, leveraging techniques from algorithm unrolling models, enhancement-based methods, and plug-and-play models to the emerging full spectrum of generative model-based methods. We also explore the synergistic integration of data models with physics-based insights, encompassing the advancements in multi-coil hardware accelerations like parallel imaging and simultaneous multi-slice imaging, and the optimization of sampling patterns. We then focus on domain-specific challenges and opportunities, including image redundancy exploitation, image integrity, evaluation metrics, data heterogeneity, and model generalization. This work also discusses potential solutions and future research directions, with an emphasis on the role of data harmonization and federated learning for further improving the general applicability and performance of these methods in MRI reconstruction.
Collapse
|
2
|
Lin DJ, Johnson PM, Knoll F, Lui YW. Artificial Intelligence for MR Image Reconstruction: An Overview for Clinicians. J Magn Reson Imaging 2021; 53:1015-1028. [PMID: 32048372 PMCID: PMC7423636 DOI: 10.1002/jmri.27078] [Citation(s) in RCA: 114] [Impact Index Per Article: 28.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/17/2019] [Revised: 01/15/2020] [Accepted: 01/17/2020] [Indexed: 12/22/2022] Open
Abstract
Artificial intelligence (AI) shows tremendous promise in the field of medical imaging, with recent breakthroughs applying deep-learning models for data acquisition, classification problems, segmentation, image synthesis, and image reconstruction. With an eye towards clinical applications, we summarize the active field of deep-learning-based MR image reconstruction. We review the basic concepts of how deep-learning algorithms aid in the transformation of raw k-space data to image data, and specifically examine accelerated imaging and artifact suppression. Recent efforts in these areas show that deep-learning-based algorithms can match and, in some cases, eclipse conventional reconstruction methods in terms of image quality and computational efficiency across a host of clinical imaging applications, including musculoskeletal, abdominal, cardiac, and brain imaging. This article is an introductory overview aimed at clinical radiologists with no experience in deep-learning-based MR image reconstruction and should enable them to understand the basic concepts and current clinical applications of this rapidly growing area of research across multiple organ systems.
Collapse
Affiliation(s)
- Dana J. Lin
- Department of Radiology, NYU School of Medicine / NYU Langone Health
| | | | - Florian Knoll
- New York University School of Medicine, Center for Biomedical Imaging
| | - Yvonne W. Lui
- Department of Radiology, NYU School of Medicine / NYU Langone Health
| |
Collapse
|
3
|
Nencka AS, Arpinar VE, Bhave S, Yang B, Banerjee S, McCrea M, Mickevicius NJ, Muftuler LT, Koch KM. Split-slice training and hyperparameter tuning of RAKI networks for simultaneous multi-slice reconstruction. Magn Reson Med 2020; 85:3272-3280. [PMID: 33331002 DOI: 10.1002/mrm.28634] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/09/2020] [Revised: 11/11/2020] [Accepted: 11/12/2020] [Indexed: 01/07/2023]
Abstract
PURPOSE Simultaneous multi-slice acquisitions are essential for modern neuroimaging research, enabling high temporal resolution functional and high-resolution q-space sampling diffusion acquisitions. Recently, deep learning reconstruction techniques have been introduced for unaliasing these accelerated acquisitions, and robust artificial-neural-networks for k-space interpolation (RAKI) have shown promising capabilities. This study systematically examines the impacts of hyperparameter selections for RAKI networks, and introduces a novel technique for training data generation which is analogous to the split-slice formalism used in slice-GRAPPA. METHODS RAKI networks were developed with variable hyperparameters and with and without split-slice training data generation. Each network was trained and applied to five different datasets including acquisitions harmonized with Human Connectome Project lifespan protocol. Unaliasing performance was assessed through L1 errors computed between unaliased and calibration frequency-space data. RESULTS Split-slice training significantly improved network performance in nearly all hyperparameter configurations. Best unaliasing results were achieved with three layer RAKI networks using at least 64 convolutional filters with receptive fields of 7 voxels, 128 single-voxel filters in the penultimate RAKI layer, batch normalization, and no training dropout with the split-slice augmented training dataset. Networks trained without the split-slice technique showed symptoms of network over-fitting. CONCLUSIONS Split-slice training for simultaneous multi-slice RAKI networks positively impacts network performance. Hyperparameter tuning of such reconstruction networks can lead to further improvements in unaliasing performance.
Collapse
Affiliation(s)
- Andrew S Nencka
- Department of Radiology, Medical College of Wisconsin, Milwaukee, WI, USA.,Center for Imaging Research, Medical College of Wisconsin, Milwaukee, WI, USA
| | - Volkan E Arpinar
- Center for Imaging Research, Medical College of Wisconsin, Milwaukee, WI, USA
| | | | | | | | - Michael McCrea
- Department of Neurosurgery, Medical College of Wisconsin, Milwaukee, WI, USA
| | | | - L Tugan Muftuler
- Center for Imaging Research, Medical College of Wisconsin, Milwaukee, WI, USA.,Department of Neurosurgery, Medical College of Wisconsin, Milwaukee, WI, USA
| | - Kevin M Koch
- Department of Radiology, Medical College of Wisconsin, Milwaukee, WI, USA.,Center for Imaging Research, Medical College of Wisconsin, Milwaukee, WI, USA
| |
Collapse
|
4
|
Cunha GM, Hasenstab KA, Higaki A, Wang K, Delgado T, Brunsing RL, Schlein A, Schwartzman A, Hsiao A, Sirlin CB, Fowler KJ. Convolutional neural network-automated hepatobiliary phase adequacy evaluation may optimize examination time. Eur J Radiol 2020; 124:108837. [PMID: 31958630 PMCID: PMC7309446 DOI: 10.1016/j.ejrad.2020.108837] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/20/2019] [Accepted: 01/10/2020] [Indexed: 02/08/2023]
Abstract
PURPOSE To develop and evaluate the performance of a fully-automated convolutional neural network (CNN)-based algorithm to evaluate hepatobiliary phase (HBP) adequacy of gadoxetate disodium (EOB)-enhanced MRI. Secondarily, we explored the potential of the proposed CNN algorithm to reduce examination length by applying it to EOB-MRI examinations. METHODS We retrospectively identified EOB-enhanced MRI-HBP series from examinations performed 2011-2018 (internal and external datasets). Our algorithm, comprising a liver segmentation and classification CNN, produces an adequacy score. Two abdominal radiologists independently classified series as adequate or suboptimal. The consensus determination of HBP adequacy was used as ground truth for CNN model training and validation. Reader agreement was evaluated with Cohen's kappa. Performance of the algorithm was assessed by receiver operating characteristics (ROC) analysis and computation of the area under the ROC curve (AUC). Potential examination duration reduction was evaluated descriptively. RESULTS 1408 HBP series from 484 patients were included. Reader kappa agreement was 0.67 (internal dataset) and 0.80 (external dataset). AUCs were 0.97 (0.96-0.99) for internal and 0.95 (0.92-96) for external and were not significantly different from each other (p = 0.24). 48 % (50/105) examinations could have been shorter by applying the algorithm. CONCLUSION A proposed CNN-based algorithm achieves higher than 95 % AUC for classifying HBP images as adequate versus suboptimal. The application of this algorithm could potentially shorten examination time and aid radiologists in recognizing technically suboptimal images, avoiding diagnostic pitfalls.
Collapse
Affiliation(s)
- Guilherme Moura Cunha
- Liver Imaging Group, Department of Radiology, University of California San Diego, La Jolla, CA, United States.
| | - Kyle A Hasenstab
- Liver Imaging Group, Department of Radiology, University of California San Diego, La Jolla, CA, United States; AiDA Laboratory, Department of Radiology, University of California San Diego, La Jolla, CA, United States; Department of Family Medicine and Public Health, University of California San Diego, La Jolla, CA, United States
| | - Atsushi Higaki
- Liver Imaging Group, Department of Radiology, University of California San Diego, La Jolla, CA, United States
| | - Kang Wang
- Liver Imaging Group, Department of Radiology, University of California San Diego, La Jolla, CA, United States; AiDA Laboratory, Department of Radiology, University of California San Diego, La Jolla, CA, United States
| | - Timo Delgado
- Liver Imaging Group, Department of Radiology, University of California San Diego, La Jolla, CA, United States
| | - Ryan L Brunsing
- Radiology, Stanford University, Palo Alto, CA, United States
| | - Alexandra Schlein
- Liver Imaging Group, Department of Radiology, University of California San Diego, La Jolla, CA, United States
| | - Armin Schwartzman
- Department of Family Medicine and Public Health, University of California San Diego, La Jolla, CA, United States
| | - Albert Hsiao
- AiDA Laboratory, Department of Radiology, University of California San Diego, La Jolla, CA, United States
| | - Claude B Sirlin
- Liver Imaging Group, Department of Radiology, University of California San Diego, La Jolla, CA, United States
| | - Katie J Fowler
- Liver Imaging Group, Department of Radiology, University of California San Diego, La Jolla, CA, United States
| |
Collapse
|
5
|
Knoll F, Hammernik K, Zhang C, Moeller S, Pock T, Sodickson DK, Akçakaya M. Deep-Learning Methods for Parallel Magnetic Resonance Imaging Reconstruction: A Survey of the Current Approaches, Trends, and Issues. IEEE SIGNAL PROCESSING MAGAZINE 2020; 37:128-140. [PMID: 33758487 PMCID: PMC7982984 DOI: 10.1109/msp.2019.2950640] [Citation(s) in RCA: 150] [Impact Index Per Article: 30.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/26/2023]
Abstract
Following the success of deep learning in a wide range of applications, neural network-based machine learning techniques have received interest as a means of accelerating magnetic resonance imaging (MRI). A number of ideas inspired by deep learning techniques from computer vision and image processing have been successfully applied to non-linear image reconstruction in the spirit of compressed sensing for both low dose computed tomography and accelerated MRI. The additional integration of multi-coil information to recover missing k-space lines in the MRI reconstruction process, is still studied less frequently, even though it is the de-facto standard for currently used accelerated MR acquisitions. This manuscript provides an overview of the recent machine learning approaches that have been proposed specifically for improving parallel imaging. A general background introduction to parallel MRI is given that is structured around the classical view of image space and k-space based methods. Both linear and non-linear methods are covered, followed by a discussion of recent efforts to further improve parallel imaging using machine learning, and specifically using artificial neural networks. Image-domain based techniques that introduce improved regularizers are covered as well as k-space based methods, where the focus is on better interpolation strategies using neural networks. Issues and open problems are discussed as well as recent efforts for producing open datasets and benchmarks for the community.
Collapse
Affiliation(s)
- Florian Knoll
- F. Knoll and D. K. Sodickson are with the Center for Biomedical Imaging, Department of Radiology, New York University. K. Hammernik is with the Department of Computing, Imperial College London. T. Pock is with the Institute of Computer Graphics and Vision, Graz University of Technology. C. Zhang and M. Akçakaya are with the Department of Electrical and Computer Engineering, and Center for Magnetic Resonance Research, University of Minnesota, Minneapolis, MN. S. Moeller is with the Center for Magnetic Resonance Research, University of Minnesota, Minneapolis, MN
| | - Kerstin Hammernik
- F. Knoll and D. K. Sodickson are with the Center for Biomedical Imaging, Department of Radiology, New York University. K. Hammernik is with the Department of Computing, Imperial College London. T. Pock is with the Institute of Computer Graphics and Vision, Graz University of Technology. C. Zhang and M. Akçakaya are with the Department of Electrical and Computer Engineering, and Center for Magnetic Resonance Research, University of Minnesota, Minneapolis, MN. S. Moeller is with the Center for Magnetic Resonance Research, University of Minnesota, Minneapolis, MN
| | - Chi Zhang
- F. Knoll and D. K. Sodickson are with the Center for Biomedical Imaging, Department of Radiology, New York University. K. Hammernik is with the Department of Computing, Imperial College London. T. Pock is with the Institute of Computer Graphics and Vision, Graz University of Technology. C. Zhang and M. Akçakaya are with the Department of Electrical and Computer Engineering, and Center for Magnetic Resonance Research, University of Minnesota, Minneapolis, MN. S. Moeller is with the Center for Magnetic Resonance Research, University of Minnesota, Minneapolis, MN
| | - Steen Moeller
- F. Knoll and D. K. Sodickson are with the Center for Biomedical Imaging, Department of Radiology, New York University. K. Hammernik is with the Department of Computing, Imperial College London. T. Pock is with the Institute of Computer Graphics and Vision, Graz University of Technology. C. Zhang and M. Akçakaya are with the Department of Electrical and Computer Engineering, and Center for Magnetic Resonance Research, University of Minnesota, Minneapolis, MN. S. Moeller is with the Center for Magnetic Resonance Research, University of Minnesota, Minneapolis, MN
| | - Thomas Pock
- F. Knoll and D. K. Sodickson are with the Center for Biomedical Imaging, Department of Radiology, New York University. K. Hammernik is with the Department of Computing, Imperial College London. T. Pock is with the Institute of Computer Graphics and Vision, Graz University of Technology. C. Zhang and M. Akçakaya are with the Department of Electrical and Computer Engineering, and Center for Magnetic Resonance Research, University of Minnesota, Minneapolis, MN. S. Moeller is with the Center for Magnetic Resonance Research, University of Minnesota, Minneapolis, MN
| | - Daniel K Sodickson
- F. Knoll and D. K. Sodickson are with the Center for Biomedical Imaging, Department of Radiology, New York University. K. Hammernik is with the Department of Computing, Imperial College London. T. Pock is with the Institute of Computer Graphics and Vision, Graz University of Technology. C. Zhang and M. Akçakaya are with the Department of Electrical and Computer Engineering, and Center for Magnetic Resonance Research, University of Minnesota, Minneapolis, MN. S. Moeller is with the Center for Magnetic Resonance Research, University of Minnesota, Minneapolis, MN
| | - Mehmet Akçakaya
- F. Knoll and D. K. Sodickson are with the Center for Biomedical Imaging, Department of Radiology, New York University. K. Hammernik is with the Department of Computing, Imperial College London. T. Pock is with the Institute of Computer Graphics and Vision, Graz University of Technology. C. Zhang and M. Akçakaya are with the Department of Electrical and Computer Engineering, and Center for Magnetic Resonance Research, University of Minnesota, Minneapolis, MN. S. Moeller is with the Center for Magnetic Resonance Research, University of Minnesota, Minneapolis, MN
| |
Collapse
|