Deep Residual Learning for Accelerated MRI Using Magnitude and Phase Networks

EPRI sparse reconstruction method based on deep learning

Electron paramagnetic resonance imaging (EPRI) is an advanced tumor oxygen concentration imaging method. Now, the bottleneck problem of EPRI is that the scanning time is too long. Sparse reconstruction is an effective and fast imaging method, which means reconstructing images from sparse-view projections. However, the EPRI images sparsely reconstructed by the classic filtered back projection (FBP) algorithm often contain severe streak artifacts, which affect subsequent image processing. In this work, we propose a feature pyramid attention-based, residual, dense, deep convolutional network (FRD-Net) to suppress the streak artifacts in the FBP-reconstructed images. This network combines residual connection, attention mechanism, dense connections and introduces perceptual loss. The EPRI image with streak artifacts is used as the input of the network and the output-label is the corresponding high-quality image densely reconstructed by the FBP algorithm. After training, the FRD-Net gets the capability of suppressing streak artifacts. The real data reconstruction experiments show that the FRD-Net can better improve the sparse reconstruction accuracy, compared with three existing representative deep networks.

High-Resolution 3D Magnetic Resonance Fingerprinting With a Graph Convolutional Network

Magnetic resonance fingerprinting (MRF) is a novel quantitative imaging framework for rapid and simultaneous quantification of multiple tissue properties. 3D MRF allows higher through-plane resolution, but the acquisition process is slow when whole-brain coverage is needed. Existing methods for acceleration mainly rely on GRAPPA for k-space interpolation in the partition-encoding direction, limiting the acceleration factor to 2 or 3. In this work, we replace GRAPPA with a deep learning approach for accurate tissue quantification with greater acceleration. Specifically, a graph convolution network (GCN) is developed to cater to the non-Cartesian spiral sampling trajectories typical in MRF acquisition. The GCN maintains high quantification accuracy with up to 6-fold acceleration and allows 1mm isotropic resolution whole-brain 3D MRF data to be acquired in 3min and submillimeter 3D MRF (0.8mm) in 5min, greatly improving the feasibility of MRF in clinical settings.

A densely interconnected network for deep learning accelerated MRI

OBJECTIVE

To improve accelerated MRI reconstruction through a densely connected cascading deep learning reconstruction framework.

MATERIALS AND METHODS

A cascading deep learning reconstruction framework (reference model) was modified by applying three architectural modifications: input-level dense connections between cascade inputs and outputs, an improved deep learning sub-network, and long-range skip-connections between subsequent deep learning networks. An ablation study was performed, where five model configurations were trained on the NYU fastMRI neuro dataset with an end-to-end scheme conjunct on four- and eightfold acceleration. The trained models were evaluated by comparing their respective structural similarity index measure (SSIM), normalized mean square error (NMSE), and peak signal to noise ratio (PSNR).

RESULTS

The proposed densely interconnected residual cascading network (DIRCN), utilizing all three suggested modifications achieved a SSIM improvement of 8% and 11%, a NMSE improvement of 14% and 23%, and a PSNR improvement of 2% and 3% for four- and eightfold acceleration, respectively. In an ablation study, the individual architectural modifications all contributed to this improvement for both acceleration factors, by improving the SSIM, NMSE, and PSNR with approximately 2-4%, 4-9%, and 0.5-1%, respectively.

CONCLUSION

The proposed architectural modifications allow for simple adjustments on an already existing cascading framework to further improve the resulting reconstructions.

Manifold Learning via Linear Tangent Space Alignment (LTSA) for Accelerated Dynamic MRI With Sparse Sampling

The spatial resolution and temporal frame-rate of dynamic magnetic resonance imaging (MRI) can be improved by reconstructing images from sparsely sampled k -space data with mathematical modeling of the underlying spatiotemporal signals. These models include sparsity models, linear subspace models, and non-linear manifold models. This work presents a novel linear tangent space alignment (LTSA) model-based framework that exploits the intrinsic low-dimensional manifold structure of dynamic images for accelerated dynamic MRI. The performance of the proposed method was evaluated and compared to state-of-the-art methods using numerical simulation studies as well as 2D and 3D in vivo cardiac imaging experiments. The proposed method achieved the best performance in image reconstruction among all the compared methods. The proposed method could prove useful for accelerating many MRI applications, including dynamic MRI, multi-parametric MRI, and MR spectroscopic imaging.

Respiratory Motion Management in Abdominal MRI: Radiology In Training

A 96-year-old woman had a suboptimal evaluation of liver observations at abdominal MRI due to significant respiratory motion. State-of-the-art strategies to minimize respiratory motion during clinical abdominal MRI are discussed.

De-Aliasing and Accelerated Sparse Magnetic Resonance Image Reconstruction Using Fully Dense CNN with Attention Gates

When sparsely sampled data are used to accelerate magnetic resonance imaging (MRI), conventional reconstruction approaches produce significant artifacts that obscure the content of the image. To remove aliasing artifacts, we propose an advanced convolutional neural network (CNN) called fully dense attention CNN (FDA-CNN). We updated the Unet model with the fully dense connectivity and attention mechanism for MRI reconstruction. The main benefit of FDA-CNN is that an attention gate in each decoder layer increases the learning process by focusing on the relevant image features and provides a better generalization of the network by reducing irrelevant activations. Moreover, densely interconnected convolutional layers reuse the feature maps and prevent the vanishing gradient problem. Additionally, we also implement a new, proficient under-sampling pattern in the phase direction that takes low and high frequencies from the k-space both randomly and non-randomly. The performance of FDA-CNN was evaluated quantitatively and qualitatively with three different sub-sampling masks and datasets. Compared with five current deep learning-based and two compressed sensing MRI reconstruction techniques, the proposed method performed better as it reconstructed smoother and brighter images. Furthermore, FDA-CNN improved the mean PSNR by 2 dB, SSIM by 0.35, and VIFP by 0.37 compared with Unet for the acceleration factor of 5.

A Systematic Literature Review on Applications of GAN-Synthesized Images for Brain MRI

With the advances in brain imaging, magnetic resonance imaging (MRI) is evolving as a popular radiological tool in clinical diagnosis. Deep learning (DL) methods can detect abnormalities in brain images without an extensive manual feature extraction process. Generative adversarial network (GAN)-synthesized images have many applications in this field besides augmentation, such as image translation, registration, super-resolution, denoising, motion correction, segmentation, reconstruction, and contrast enhancement. The existing literature was reviewed systematically to understand the role of GAN-synthesized dummy images in brain disease diagnosis. Web of Science and Scopus databases were extensively searched to find relevant studies from the last 6 years to write this systematic literature review (SLR). Predefined inclusion and exclusion criteria helped in filtering the search results. Data extraction is based on related research questions (RQ). This SLR identifies various loss functions used in the above applications and software to process brain MRIs. A comparative study of existing evaluation metrics for GAN-synthesized images helps choose the proper metric for an application. GAN-synthesized images will have a crucial role in the clinical sector in the coming years, and this paper gives a baseline for other researchers in the field.

MR image reconstruction from undersampled data for image-guided radiation therapy using a patient-specific deep manifold image prior

Introduction

Recent advancements in radiotherapy (RT) have allowed for the integration of a Magnetic Resonance (MR) imaging scanner with a medical linear accelerator to use MR images for image guidance to position tumors against the treatment beam. Undersampling in MR acquisition is desired to accelerate the imaging process, but unavoidably deteriorates the reconstructed image quality. In RT, a high-quality MR image of a patient is available for treatment planning. In light of this unique clinical scenario, we proposed to exploit the patient-specific image prior to facilitate high-quality MR image reconstruction.

Methods

Utilizing the planning MR image, we established a deep auto-encoder to form a manifold of image patches of the patient. The trained manifold was then incorporated as a regularization to restore MR images of the same patient from undersampled data. We performed a simulation study using a patient case, a real patient study with three liver cancer patient cases, and a phantom experimental study using data acquired on an in-house small animal MR scanner. We compared the performance of the proposed method with those of the Fourier transform method, a tight-frame based Compressive Sensing method, and a deep learning method with a patient-generic manifold as the image prior.

Results

In the simulation study with 12.5% radial undersampling and 15% increase in noise, our method improved peak-signal-to-noise ratio by 4.46dB and structural similarity index measure by 28% compared to the patient-generic manifold method. In the experimental study, our method outperformed others by producing reconstructions of visually improved image quality.

[Applications of Artificial Intelligence in MR Image Acquisition and Reconstruction]

Recently, artificial intelligence (AI) technology has shown potential clinical utility in a wide range of MRI fields. In particular, AI models for improving the efficiency of the image acquisition process and the quality of reconstructed images are being actively developed by the MR research community. AI is expected to further reduce acquisition times in various MRI protocols used in clinical practice when compared to current parallel imaging techniques. Additionally, AI can help with tasks such as planning, parameter optimization, artifact reduction, and quality assessment. Furthermore, AI is being actively applied to automate MR image analysis such as image registration, segmentation, and object detection. For this reason, it is important to consider the effects of protocols or devices in MR image analysis. In this review article, we briefly introduced issues related to AI application of MR image acquisition and reconstruction.

Pyramid Convolutional RNN for MRI Image Reconstruction

Fast and accurate MRI image reconstruction from undersampled data is crucial in clinical practice. Deep learning based reconstruction methods have shown promising advances in recent years. However, recovering fine details from undersampled data is still challenging. In this paper, we introduce a novel deep learning based method, Pyramid Convolutional RNN (PC-RNN), to reconstruct images from multiple scales. Based on the formulation of MRI reconstruction as an inverse problem, we design the PC-RNN model with three convolutional RNN (ConvRNN) modules to iteratively learn the features in multiple scales. Each ConvRNN module reconstructs images at different scales and the reconstructed images are combined by a final CNN module in a pyramid fashion. The multi-scale ConvRNN modules learn a coarse-to-fine image reconstruction. Unlike other common reconstruction methods for parallel imaging, PC-RNN does not employ coil sensitive maps for multi-coil data and directly model the multiple coils as multi-channel inputs. The coil compression technique is applied to standardize data with various coil numbers, leading to more efficient training. We evaluate our model on the fastMRI knee and brain datasets and the results show that the proposed model outperforms other methods and can recover more details. The proposed method is one of the winner solutions in the 2019 fastMRI competition.

Multiple B-Value Model-Based Residual Network (MORN) for Accelerated High-Resolution Diffusion-Weighted Imaging

Single-Shot Echo Planar Imaging (SSEPI) based Diffusion Weighted Imaging (DWI) has shortcomings such as low resolution and severe distortions. In contrast, Multi-Shot EPI (MSEPI) provides optimal spatial resolution but increases scan time. This study proposed a Multiple b-value mOdel-based Residual Network (MORN) model to reconstruct multiple b-value high-resolution DWI from undersampled k-space data simultaneously. We incorporated Parallel Imaging (PI) into a residual U-net to reconstruct multiple b-value multi-coil data with the supervision of MUltiplexed Sensitivity-Encoding (MUSE) reconstructed Multi-Shot DWI (MSDWI). Moreover, asymmetric concatenations among different b-values and the combined loss to back propagate helped the feature transfer. After training and validation of the MORN in a dataset of 32 healthy cases, additional assessments were performed on 6 patients with different tumor types. The experimental results demonstrated that the MORN model outperformed conventional PI reconstruction (i.e. SENSE) and two state-of-the-art deep learning methods (SENSE-GAN and VSNet) in terms of PSNR (Peak Signal-to-Noise Ratio), SSIM (Structual SIMilarity) and apparent diffusion coefficient maps. In addition, using the pre-trained model under DWI, the MORN achieved consistent fractional anisotrophy and mean diffusivity reconstructed from multiple diffusion directions. Hence, the proposed method shows potential in clinical application according to the observations on tumor patients as well as images of multiple diffusion directions.

Multi-Coil MRI Reconstruction Challenge-Assessing Brain MRI Reconstruction Models and Their Generalizability to Varying Coil Configurations

Deep-learning-based brain magnetic resonance imaging (MRI) reconstruction methods have the potential to accelerate the MRI acquisition process. Nevertheless, the scientific community lacks appropriate benchmarks to assess the MRI reconstruction quality of high-resolution brain images, and evaluate how these proposed algorithms will behave in the presence of small, but expected data distribution shifts. The multi-coil MRI (MC-MRI) reconstruction challenge provides a benchmark that aims at addressing these issues, using a large dataset of high-resolution, three-dimensional, T1-weighted MRI scans. The challenge has two primary goals: (1) to compare different MRI reconstruction models on this dataset and (2) to assess the generalizability of these models to data acquired with a different number of receiver coils. In this paper, we describe the challenge experimental design and summarize the results of a set of baseline and state-of-the-art brain MRI reconstruction models. We provide relevant comparative information on the current MRI reconstruction state-of-the-art and highlight the challenges of obtaining generalizable models that are required prior to broader clinical adoption. The MC-MRI benchmark data, evaluation code, and current challenge leaderboard are publicly available. They provide an objective performance assessment for future developments in the field of brain MRI reconstruction.

Deep learning for fast low-field MRI acquisitions

Low-field (LF) MRI research currently gains momentum from its potential to offer reduced costs and reduced footprints translating into wider accessibility. However, the impeded signal-to-noise ratio inherent to lower magnetic fields can have a significant impact on acquisition times that challenges LF clinical relevance. Undersampling is an effective way to speed up acquisitions in MRI, and recent work has shown encouraging results when combined with deep learning (DL). Yet, training DL models generally requires large databases that are not yet available at LF regimes. Here, we demonstrate the capability of Residual U-net combined with data augmentation to reconstruct magnitude and phase information of undersampled LF MRI scans at 0.1 T with a limited training dataset (n = 10). The model performance was first evaluated in a retrospective study for different acceleration rates and sampling patterns. Ultimately, the DL approach was validated on prospectively acquired, fivefold undersampled LF data. With varying performances associated to the adopted sampling scheme, our results show that the approach investigated can preserve the global structure and the details sharpness in the reconstructed magnitude and phase images. Overall, promising results could be obtained on acquired LF MR images that may bring this research closer to clinical implementation.

Unsupervised MRI Reconstruction via Zero-Shot Learned Adversarial Transformers

Supervised reconstruction models are characteristically trained on matched pairs of undersampled and fully-sampled data to capture an MRI prior, along with supervision regarding the imaging operator to enforce data consistency. To reduce supervision requirements, the recent deep image prior framework instead conjoins untrained MRI priors with the imaging operator during inference. Yet, canonical convolutional architectures are suboptimal in capturing long-range relationships, and priors based on randomly initialized networks may yield suboptimal performance. To address these limitations, here we introduce a novel unsupervised MRI reconstruction method based on zero-Shot Learned Adversarial TransformERs (SLATER). SLATER embodies a deep adversarial network with cross-attention transformers to map noise and latent variables onto coil-combined MR images. During pre-training, this unconditional network learns a high-quality MRI prior in an unsupervised generative modeling task. During inference, a zero-shot reconstruction is then performed by incorporating the imaging operator and optimizing the prior to maximize consistency to undersampled data. Comprehensive experiments on brain MRI datasets clearly demonstrate the superior performance of SLATER against state-of-the-art unsupervised methods.

EPI phase error correction with deep learning (PEC-DL) at 7 T

PURPOSE

The phase mismatch between odd and even echoes in EPI causes Nyquist ghost artifacts. Existing ghost correction methods often suffer from severe residual artifacts and are ineffective with k-space undersampling data. This study proposed a deep learning-based method (PEC-DL) to correct phase errors for DWI at 7 Tesla.

METHODS

The acquired k-space data were divided into 2 independent undersampled datasets according to their readout polarities. Then the proposed PEC-DL network reconstructed 2 ghost-free images using the undersampled data without calibration and navigator data. The network was trained with fully sampled images and applied to two- and fourfold accelerated data. Healthy volunteers and patients with Moyamoya disease were recruited to validate the efficacy of the PEC-DL method.

RESULTS

The PEC-DL method was capable to mitigate the ghost artifacts in DWI in healthy volunteers as well as patients with Moyamoya disease. The fourfold accelerated results showed much less distortion in the lesions of the Moyamoya patient using high b-value DWI and the corresponding ADC maps. The ghost-to-signal ratios were significantly lower in PEC-DL images compared to conventional linear phase corrections, mini-entropy, and PEC-GRAPPA algorithms.

CONCLUSION

The proposed method can effectively eliminate ghost artifacts for full sampled and up to fourfold accelerated EPI data without calibration and navigator data.

Simultaneously optimizing sampling pattern for joint acceleration of multi-contrast MRI using model-based deep learning

BACKGROUND

Acceleration of MR imaging (MRI) is a popular research area, and usage of deep learning for acceleration has become highly widespread in the MR community. Joint acceleration of multiple-acquisition MRI was proven to be effective over a single-acquisition approach. Also, optimization in the sampling pattern demonstrated its advantage over conventional undersampling pattern. However, optimizing the sampling patterns for joint acceleration of multiple-acquisition MRI has not been investigated well.

PURPOSE

To develop a model-based deep learning scheme to optimize sampling patterns for a joint acceleration of multi-contrast MRI.

METHODS

The proposed scheme combines sampling pattern optimization and multi-contrast MRI reconstruction. It was extended from the physics-guided method of the joint model-based deep learning (J-MoDL) scheme to optimize the separate sampling pattern for each of multiple contrasts simultaneously for their joint reconstruction. Tests were performed with three contrasts of T2-weighted, FLAIR, and T1-weighted images. The proposed multi-contrast method was compared to (i) single-contrast method with sampling optimization (baseline J-MoDL), (ii) multi-contrast method without sampling optimization, and (iii) multi-contrast method with single common sampling optimization for all contrasts. The optimized sampling patterns were analyzed for sampling location overlap across contrasts. The scheme was also tested in a data-driven scenario, where the inversion between input and label was learned from the under-sampled data directly and tested on knee datasets for generalization test.

RESULTS

The proposed scheme demonstrated a quantitative and qualitative advantage over the single-contrast scheme with sampling pattern optimization and the multi-contrast scheme without sampling pattern optimization. Optimizing the separate sampling pattern for each of the multi-contrasts was superior to optimizing only one common sampling pattern for all contrasts. The proposed scheme showed less overlap in sampling locations than the single-contrast scheme. The main hypothesis was also held in the data-driven situation as well. The brain-trained model worked well on the knee images, demonstrating its generalizability.

CONCLUSION

Our study introduced an effective scheme that combines the sampling optimization and the multi-contrast acceleration. The seamless combination resulted in superior performance over the other existing methods.

Multi-Domain Neumann Network with Sensitivity Maps for Parallel MRI Reconstruction

MRI is an imaging technology that non-invasively obtains high-quality medical images for diagnosis. However, MRI has the major disadvantage of long scan times which cause patient discomfort and image artifacts. As one of the methods for reducing the long scan time of MRI, the parallel MRI method for reconstructing a high-fidelity MR image from under-sampled multi-coil k-space data is widely used. In this study, we propose a method to reconstruct a high-fidelity MR image from under-sampled multi-coil k-space data using deep-learning. The proposed multi-domain Neumann network with sensitivity maps (MDNNSM) is based on the Neumann network and uses a forward model including coil sensitivity maps for parallel MRI reconstruction. The MDNNSM consists of three main structures: the CNN-based sensitivity reconstruction block estimates coil sensitivity maps from multi-coil under-sampled k-space data; the recursive MR image reconstruction block reconstructs the MR image; and the skip connection accumulates each output and produces the final result. Experiments using the fastMRI T1-weighted brain image dataset were conducted at acceleration factors of 2, 4, and 8. Qualitative and quantitative experimental results show that the proposed MDNNSM method reconstructs MR images more accurately than other methods, including the generalized autocalibrating partially parallel acquisitions (GRAPPA) method and the original Neumann network.

Residual RAKI: A hybrid linear and non-linear approach for scan-specific k-space deep learning

Parallel imaging is the most clinically used acceleration technique for magnetic resonance imaging (MRI) in part due to its easy inclusion into routine acquisitions. In k-space based parallel imaging reconstruction, sub-sampled k-space data are interpolated using linear convolutions. At high acceleration rates these methods have inherent noise amplification and reduced image quality. On the other hand, non-linear deep learning methods provide improved image quality at high acceleration, but the availability of training databases for different scans, as well as their interpretability hinder their adaptation. In this work, we present an extension of Robust Artificial-neural-networks for k-space Interpolation (RAKI), called residual-RAKI (rRAKI), which achieves scan-specific machine learning reconstruction using a hybrid linear and non-linear methodology. In rRAKI, non-linear CNNs are trained jointly with a linear convolution implemented via a skip connection. In effect, the linear part provides a baseline reconstruction, while the non-linear CNN that runs in parallel provides further reduction of artifacts and noise arising from the linear part. The explicit split between the linear and non-linear aspects of the reconstruction also help improve interpretability compared to purely non-linear methods. Experiments were conducted on the publicly available fastMRI datasets, as well as high-resolution anatomical imaging, comparing GRAPPA and its variants, compressed sensing, RAKI, Scan Specific Artifact Reduction in K-space (SPARK) and the proposed rRAKI. Additionally, highly-accelerated simultaneous multi-slice (SMS) functional MRI reconstructions were also performed, where the proposed rRAKI was compred to Read-out SENSE-GRAPPA and RAKI. Our results show that the proposed rRAKI method substantially improves the image quality compared to conventional parallel imaging, and offers sharper images compared to SPARK and ℓ₁-SPIRiT. Furthermore, rRAKI shows improved preservation of time-varying dynamics compared to both parallel imaging and RAKI in highly-accelerated SMS fMRI.

Varying undersampling directions for accelerating multiple acquisition magnetic resonance imaging

In this study, we propose a new sampling strategy for efficiently accelerating multiple acquisition MRI. The new sampling strategy is to obtain data along different phase-encoding directions across multiple acquisitions. The proposed sampling strategy was evaluated in multicontrast MR imaging (T1, T2, proton density) and multiple phase-cycled (PC) balanced steady-state free precession (bSSFP) imaging by using convolutional neural networks with central and random sampling patterns. In vivo MRI acquisitions as well as a public database were used to test the concept. Based on both visual inspection and quantitative analysis, the proposed sampling strategy showed better performance than sampling along the same phase-encoding direction in both multicontrast MR imaging and multiple PC-bSSFP imaging, regardless of sampling pattern (central, random) or datasets (public, retrospective and prospective in vivo). For the prospective in vivo applications, acceleration was performed by sampling along different phase-encoding directions at the time of acquisition with a conventional rectangular field of view, which demonstrated the advantage of the proposed sampling strategy in the real environment. Preliminary trials on compressed sensing (CS) also demonstrated improvement of CS with the proposed idea. Sampling along different phase-encoding directions across multiple acquisitions is advantageous for accelerating multiacquisition MRI, irrespective of sampling pattern or datasets, with further improvement through transfer learning.

Neuroimaging in the Era of Artificial Intelligence: Current Applications

BACKGROUND

Artificial intelligence (AI) in medicine has shown significant promise, particularly in neuroimaging. AI increases efficiency and reduces errors, making it a valuable resource for physicians. With the increasing amount of data processing and image interpretation required, the ability to use AI to augment and aid the radiologist could improve the quality of patient care.

OBSERVATIONS

AI can predict patient wait times, which may allow more efficient patient scheduling. Additionally, AI can save time for repeat magnetic resonance neuroimaging and reduce the time spent during imaging. AI has the ability to read computed tomography, magnetic resonance imaging, and positron emission tomography with reduced or without contrast without significant loss in sensitivity for detecting lesions. Neuroimaging does raise important ethical considerations and is subject to bias. It is vital that users understand the practical and ethical considerations of the technology.

CONCLUSIONS

The demonstrated applications of AI in neuroimaging are numerous and varied, and it is reasonable to assume that its implementation will increase as the technology matures. AI's use for detecting neurologic conditions holds promise in combatting ever increasing imaging volumes and providing timely diagnoses.

A Review of Deep Learning Methods for Compressed Sensing Image Reconstruction and Its Medical Applications

Compressed sensing (CS) and its medical applications are active areas of research. In this paper, we review recent works using deep learning method to solve CS problem for images or medical imaging reconstruction including computed tomography (CT), magnetic resonance imaging (MRI) and positron-emission tomography (PET). We propose a novel framework to unify traditional iterative algorithms and deep learning approaches. In short, we define two projection operators toward image prior and data consistency, respectively, and any reconstruction algorithm can be decomposed to the two parts. Though deep learning methods can be divided into several categories, they all satisfies the framework. We built the relationship between different reconstruction methods of deep learning, and connect them to traditional methods through the proposed framework. It also indicates that the key to solve CS problem and its medical applications is how to depict the image prior. Based on the framework, we analyze the current deep learning methods and point out some important directions of research in the future.

A simultaneous multi-slice T2 mapping framework based on overlapping-echo detachment planar imaging and deep learning reconstruction

PURPOSE

Quantitative MRI (qMRI) is of great importance to clinical medicine and scientific research. However, most qMRI techniques are time-consuming and sensitive to motion, especially when a large 3D volume is imaged. To accelerate the acquisition, a framework is proposed to realize reliable simultaneous multi-slice T₂ mapping.

METHODS

The simultaneous multi-slice T₂ mapping framework is based on overlapping-echo detachment (OLED) planar imaging (dubbed SMS-OLED). Multi-slice overlapping-echo signals were generated by multiple excitation pulses together with echo-shifting gradients. The signals were excited and acquired with a single-channel coil. U-Net was used to reconstruct T₂ maps from the acquired overlapping-echo image.

RESULTS

Single-shot double-slice and two-shot triple-slice SMS-OLED scan schemes were designed according to the framework for evaluation. Simulations, water phantom, and in vivo rat brain experiments were carried out. Overlapping-echo signals were acquired, and T₂ maps were reconstructed and compared with references. The results demonstrate the superior performance of our method.

CONCLUSION

Two slices of T₂ maps can be obtained in a single shot within hundreds of milliseconds. Higher quality multi-slice T₂ maps can be obtained via multiple shots. SMS-OLED provides a lower specific absorption rate scheme compared with conventional SMS methods with a coil with only a single receiver channel. The new method is of potential in dynamic qMRI and functional qMRI where temporal resolution is vital.

Review and Prospect: Artificial Intelligence in Advanced Medical Imaging

Artificial intelligence (AI) as an emerging technology is gaining momentum in medical imaging. Recently, deep learning-based AI techniques have been actively investigated in medical imaging, and its potential applications range from data acquisition and image reconstruction to image analysis and understanding. In this review, we focus on the use of deep learning in image reconstruction for advanced medical imaging modalities including magnetic resonance imaging (MRI), computed tomography (CT), and positron emission tomography (PET). Particularly, recent deep learning-based methods for image reconstruction will be emphasized, in accordance with their methodology designs and performances in handling volumetric imaging data. It is expected that this review can help relevant researchers understand how to adapt AI for medical imaging and which advantages can be achieved with the assistance of AI.

Artificial intelligence with deep learning in nuclear medicine and radiology

The use of deep learning in medical imaging has increased rapidly over the past few years, finding applications throughout the entire radiology pipeline, from improved scanner performance to automatic disease detection and diagnosis. These advancements have resulted in a wide variety of deep learning approaches being developed, solving unique challenges for various imaging modalities. This paper provides a review on these developments from a technical point of view, categorizing the different methodologies and summarizing their implementation. We provide an introduction to the design of neural networks and their training procedure, after which we take an extended look at their uses in medical imaging. We cover the different sections of the radiology pipeline, highlighting some influential works and discussing the merits and limitations of deep learning approaches compared to other traditional methods. As such, this review is intended to provide a broad yet concise overview for the interested reader, facilitating adoption and interdisciplinary research of deep learning in the field of medical imaging.

Deep Learning Reconstruction Enables Highly Accelerated Biparametric MR Imaging of the Prostate

BACKGROUND

Early diagnosis and treatment of prostate cancer (PCa) can be curative; however, prostate-specific antigen is a suboptimal screening test for clinically significant PCa. While prostate magnetic resonance imaging (MRI) has demonstrated value for the diagnosis of PCa, the acquisition time is too long for a first-line screening modality.

PURPOSE

To accelerate prostate MRI exams, utilizing a variational network (VN) for image reconstruction.

STUDY TYPE

Retrospective.

SUBJECTS

One hundred and thirteen subjects (train/val/test: 70/13/30) undergoing prostate MRI.

FIELD STRENGTH/SEQUENCE

3.0 T; a T2 turbo spin echo (TSE) T2-weighted image (T2WI) sequence in axial and coronal planes, and axial echo-planar diffusion-weighted imaging (DWI).

ASSESSMENT

Four abdominal radiologists evaluated the image quality of VN reconstructions of retrospectively under-sampled biparametric MRIs (bp-MRI), and standard bp-MRI reconstructions for 20 test subjects (studies). The studies included axial and coronal T2WI, DWI B50 seconds/mm² and B1000 seconds/mm (4-fold T2WI, 3-fold DWI), all of which were evaluated separately for image quality on a Likert scale (1: non-diagnostic to 5: excellent quality). In another 10 test subjects, three readers graded lesions on bp-MRI-which additionally included calculated B1500 seconds/mm² , and apparent diffusion coefficient map-according to the Prostate Imaging Reporting and Data System (PI-RADS v2.1), for both VN and standard reconstructions. Accuracy of PI-RADS ≥3 for clinically significant cancer was computed. Projected scan time of the retrospectively under-sampled biparametric exam was also computed.

STATISTICAL TESTS

One-sided Wilcoxon signed-rank test was used for comparison of image quality. Sensitivity, specificity, positive predictive value, and negative predictive value were calculated for lesion detection and grading. Generalized estimating equation with cluster effect was used to compare differences between standard and VN bp-MRI. A P-value of <0.05 was considered statistically significant.

RESULTS

Three of four readers rated no significant difference for overall quality between the standard and VN axial T2WI (Reader 1: 4.00 ± 0.56 (Standard), 3.90 ± 0.64 (VN) P = 0.33; Reader 2: 4.35 ± 0.74 (Standard), 3.80 ± 0.89 (VN) P = 0.003; Reader 3: 4.60 ± 0.50 (Standard), 4.55 ± 0.60 (VN) P = 0.39; Reader 4: 3.65 ± 0.99 (Standard), 3.60 ± 1.00 (VN) P = 0.38). All four readers rated no significant difference for overall quality between standard and VN DWI B1000 seconds/mm² (Reader 1: 2.25 ± 0.62 (Standard), 2.45 ± 0.75 (VN) P = 0.96; Reader 2: 3.60 ± 0.92 (Standard), 3.55 ± 0.82 (VN) P = 0.40; Reader 3: 3.85 ± 0.72 (Standard), 3.55 ± 0.89 (VN) P = 0.07; Reader 4: 4.70 ± 0.76 (Standard); 4.60 ± 0.73 (VN) P = 0.17) and three of four readers rated no significant difference for overall quality between standard and VN DWI B50 seconds/mm² (Reader 1: 3.20 ± 0.70 (Standard), 3.40 ± 0.75 (VN) P = 0.98; Reader 2: 2.85 ± 0.81 (Standard), 3.00 ± 0.79 (VN) P = 0.93; Reader 3: 4.45 ± 0.72 (Standard), 4.05 ± 0.69 (VN) P = 0.02; Reader 4: 4.50 ± 0.69 (Standard), 4.45 ± 0.76 (VN) P = 0.50). In the lesion evaluation study, there was no significant difference in the number of PI-RADS ≥3 lesions identified on standard vs. VN bp-MRI (P = 0.92, 0.59, 0.87) with similar sensitivity and specificity for clinically significant cancer. The average scan time of the standard clinical biparametric exam was 11.8 minutes, and this was projected to be 3.2 minutes for the accelerated exam.

DATA CONCLUSION

Diagnostic accelerated biparametric prostate MRI exams can be performed using deep learning methods in <4 minutes, potentially enabling rapid screening prostate MRI.

LEVEL OF EVIDENCE

3 TECHNICAL EFFICACY: Stage 5.

Deep learning based multiplexed sensitivity-encoding (DL-MUSE) for high-resolution multi-shot DWI

PURPOSE

A phase correction method for high-resolution multi-shot (MSH) diffusion weighted imaging (DWI) is proposed. The efficacy and generalization capability of the method were validated on both healthy volunteers and patients.

THEORY AND METHODS

Conventionally, inter-shot phase variations for MSH echo-planar imaging (EPI) DWI are corrected by model-based algorithms. However, many acquisition imperfections are hard to measure accurately for conventional model-based methods, making the phase estimation and artifacts suppression unreliable. We propose a deep learning multiplexed sensitivity-encoding (DL-MUSE) framework to improve the phase estimations based on convolutional neural network (CNN) reconstruction. Aliasing-free single-shot (SSH) DW images, which have been used routinely in clinical settings, were used for training before the aliasing correction of MSH-DWI images. A dual-channel U-net comprising multiple convolutional layers was used for the phase estimation of MSH-EPI. The network was trained on a dataset containing 30 healthy volunteers and tested on another dataset of 52 healthy subjects and 15 patients with lesions or tumors with different shot numbers (4, 6 and 8). To further validate the generalization capability of our network, we acquired a dataset with different numbers of shots, TEs, partial Fourier factors, resolutions, ETLs, FOVs, coil numbers, and image orientations from two sites. We also compared the reconstruction performance of our proposed method with that of the conventional MUSE and SSH-EPI qualitatively and quantitatively.

RESULTS

Our results show that DL-MUSE is capable of correcting inter-shot phase errors with high and robust performance. Compared to conventional model-based MUSE, our method, by applying deep learning-based phase corrections, showed reduced distortion, noise level, and signal loss in high b-value DWIs. The improvements of image quality become more evident as the shot number increases from 4 to 8, especially in those central regions of the images, where g-factor artifacts are severe. Furthermore, the proposed method could provide the information about the orientation of the white matter with better consistency and achieve finer fibers delineation compared to the SSH-EPI method. Besides, the experiments on volunteers and patients from two different sites demonstrated the generalizability of our proposed method preliminarily.

CONCLUSION

A deep learning-based reconstruction algorithm for MSH-EPI images, which helps improve image quality greatly, was proposed. Results from healthy volunteers and tumor patients demonstrated the feasibility and generalization performances of our method for high-resolution MSH-EPI DWI, which can be used for routine clinical applications as well as neuroimaging research.

Phase2Phase: Respiratory Motion-Resolved Reconstruction of Free-Breathing Magnetic Resonance Imaging Using Deep Learning Without a Ground Truth for Improved Liver Imaging

OBJECTIVES

Respiratory binning of free-breathing magnetic resonance imaging data reduces motion blurring; however, it exacerbates noise and introduces severe artifacts due to undersampling. Deep neural networks can remove artifacts and noise but usually require high-quality ground truth images for training. This study aimed to develop a network that can be trained without this requirement.

MATERIALS AND METHODS

This retrospective study was conducted on 33 participants enrolled between November 2016 and June 2019. Free-breathing magnetic resonance imaging was performed using a radial acquisition. Self-navigation was used to bin the k-space data into 10 respiratory phases. To simulate short acquisitions, subsets of radial spokes were used in reconstructing images with multicoil nonuniform fast Fourier transform (MCNUFFT), compressed sensing (CS), and 2 deep learning methods: UNet3DPhase and Phase2Phase (P2P). UNet3DPhase was trained using a high-quality ground truth, whereas P2P was trained using noisy images with streaking artifacts. Two radiologists blinded to the reconstruction methods independently reviewed the sharpness, contrast, and artifact-freeness of the end-expiration images reconstructed from data collected at 16% of the Nyquist sampling rate. The generalized estimating equation method was used for statistical comparison. Motion vector fields were derived to examine the respiratory motion range of 4-dimensional images reconstructed using different methods.

RESULTS

A total of 15 healthy participants and 18 patients with hepatic malignancy (50 ± 15 years, 6 women) were enrolled. Both reviewers found that the UNet3DPhase and P2P images had higher contrast (P < 0.01) and fewer artifacts (P < 0.01) than the CS images. The UNet3DPhase and P2P images were reported to be sharper than the CS images by 1 reviewer (P < 0.01) but not by the other reviewer (P = 0.22, P = 0.18). UNet3DPhase and P2P were similar in sharpness and contrast, whereas UNet3DPhase had fewer artifacts (P < 0.01). The motion vector lengths for the MCNUFFT800 and P2P800 images were comparable (10.5 ± 4.2 mm and 9.9 ± 4.0 mm, respectively), whereas both were significantly larger than CS2000 (7.0 ± 3.9 mm; P < 0.0001) and UNnet3DPhase800 (6.9 ± 3.2; P < 0.0001) images.

CONCLUSIONS

Without a ground truth, P2P can reconstruct sharp, artifact-free, and high-contrast respiratory motion-resolved images from highly undersampled data. Unlike the CS and UNet3DPhase methods, P2P did not artificially reduce the respiratory motion range.

DeepSENSE: Learning coil sensitivity functions for SENSE reconstruction using deep learning

PURPOSE

To improve the estimation of coil sensitivity functions from limited auto-calibration signals (ACS) in SENSE-based reconstruction for brain imaging.

METHODS

We propose to use deep learning to estimate coil sensitivity functions by leveraging information from previous scans obtained using the same RF receiver system. Specifically, deep convolutional neural networks were designed to learn an end-to-end mapping from the initial sensitivity to the high-resolution counterpart. Sensitivity alignment was further proposed to reduce the geometric variation caused by different subject positions and imaging FOVs. Cross-validation with a small set of datasets was performed to validate the learned neural network. Iterative SENSE reconstruction was adopted to evaluate the utility of the sensitivity functions from the proposed and conventional methods.

RESULTS

The proposed method produced improved sensitivity estimates and SENSE reconstructions compared to the conventional methods in terms of aliasing and noise suppression with very limited ACS data. Cross-validation with a small set of data demonstrated the feasibility of learning coil sensitivity functions for brain imaging. The network learned on the spoiled GRE data can be applied to predict sensitivity functions for spin-echo and MPRAGE datasets.

CONCLUSION

A deep learning-based method has been proposed for improving the estimation of coil sensitivity functions. Experimental results have demonstrated the feasibility and potential of the proposed method for improving SENSE-based reconstructions especially when the ACS data are limited.

Learning a preconditioner to accelerate compressed sensing reconstructions in MRI

Purpose

To learn a preconditioner that accelerates parallel imaging (PI) and compressed sensing (CS) reconstructions.

Methods

A convolutional neural network (CNN) with residual connections was used to train a preconditioning operator. Training and validation data were simulated using 50% brain images and 50% white Gaussian noise images. Each multichannel training example contains a simulated sampling mask, complex coil sensitivity maps, and two regularization parameter maps. The trained model was integrated in the preconditioned conjugate gradient (PCG) method as part of the split Bregman CS method. The acceleration performance was compared with that of a circulant PI‐CS preconditioner for varying undersampling factors, number of coil elements and anatomies.

Results

The learned preconditioner reduces the number of PCG iterations by a factor of 4, yielding a similar acceleration as an efficient circulant preconditioner. The method generalizes well to different sampling schemes, coil configurations and anatomies.

Conclusion

It is possible to learn adaptable preconditioners for PI and CS reconstructions that meet the performance of state‐of‐the‐art preconditioners. Further acceleration could be achieved by optimizing the network architecture and the training set. Such a preconditioner could also be integrated in fully learned reconstruction methods to accelerate the training process of unrolled networks.

Learning Data Consistency and its Application to Dynamic MR Imaging

Magnetic resonance (MR) image reconstruction from undersampled k-space data can be formulated as a minimization problem involving data consistency and image prior. Existing deep learning (DL)-based methods for MR reconstruction employ deep networks to exploit the prior information and integrate the prior knowledge into the reconstruction under the explicit constraint of data consistency, without considering the real distribution of the noise. In this work, we propose a new DL-based approach termed Learned DC that implicitly learns the data consistency with deep networks, corresponding to the actual probability distribution of system noise. The data consistency term and the prior knowledge are both embedded in the weights of the networks, which provides an utterly implicit manner of learning reconstruction model. We evaluated the proposed approach with highly undersampled dynamic data, including the dynamic cardiac cine data with up to 24-fold acceleration and dynamic rectum data with the acceleration factor equal to the number of phases. Experimental results demonstrate the superior performance of the Learned DC both quantitatively and qualitatively than the state-of-the-art.

Upstream Machine Learning in Radiology

Machine learning (ML) and Artificial intelligence (AI) has the potential to dramatically improve radiology practice at multiple stages of the imaging pipeline. Most of the attention has been garnered by applications focused on improving the end of the pipeline: image interpretation. However, this article reviews how AI/ML can be applied to improve upstream components of the imaging pipeline, including exam modality selection, hardware design, exam protocol selection, data acquisition, image reconstruction, and image processing. A breadth of applications and their potential for impact is shown across multiple imaging modalities, including ultrasound, computed tomography, and MRI.

MAGnitude-Image-to-Complex K-space (MAGIC-K) Net: A Data Augmentation Network for Image Reconstruction

Deep learning has demonstrated superior performance in image reconstruction compared to most conventional iterative algorithms. However, their effectiveness and generalization capability are highly dependent on the sample size and diversity of the training data. Deep learning-based reconstruction requires multi-coil raw k-space data, which are not collected by routine scans. On the other hand, large amounts of magnitude images are readily available in hospitals. Hence, we proposed the MAGnitude Images to Complex K-space (MAGIC-K) Net to generate multi-coil k-space data from existing magnitude images and a limited number of required raw k-space data to facilitate the reconstruction. Compared to some basic data augmentation methods applying global intensity and displacement transformations to the source images, the MAGIC-K Net can generate more realistic intensity variations and displacements from pairs of anatomical Digital Imaging and Communications in Medicine (DICOM) images. The reconstruction performance was validated in 30 healthy volunteers and 6 patients with different types of tumors. The experimental results demonstrated that the high-resolution Diffusion Weighted Image (DWI) reconstruction benefited from the proposed augmentation method. The MAGIC-K Net enabled the deep learning network to reconstruct images with superior performance in both healthy and tumor patients, qualitatively and quantitatively.

Visualizing Trace Pollutants in Solids at Nanoscale via Electron Tomography

Visualizing trace pollutants such as toxic metals and viruses in environmental solids such as soils, sediments, aerosols, and suspended particles in water has long been the holy grail for scientists and engineers. In this Perspective, progress on the state-of-the-art electron tomography is highlighted as an increasingly indispensable tool for visualizing contaminant distribution and transformation in three-dimension (3D), including environmental pollutants at the water-minerals interfaces, toxicology assessment, environmental behavior of viruses in heterogeneous environmental media, etc. Adding a third dimension to the pollutant characterization will surely enrich our understanding on the complex and emerging environmental issues facing the global society, and provide vital support to the ongoing research and development of life-saving mitigation technologies from air filtration, to drinking water purification, to virus disinfection.

Over-and-Under Complete Convolutional RNN for MRI Reconstruction

Reconstructing magnetic resonance (MR) images from under-sampled data is a challenging problem due to various artifacts introduced by the under-sampling operation. Recent deep learning-based methods for MR image reconstruction usually leverage a generic auto-encoder architecture which captures low-level features at the initial layers and high-level features at the deeper layers. Such networks focus much on global features which may not be optimal to reconstruct the fully-sampled image. In this paper, we propose an Over-and-Under Complete Convolutional Recurrent Neural Network (OUCR), which consists of an overcomplete and an undercomplete Convolutional Recurrent Neural Network (CRNN). The overcomplete branch gives special attention in learning local structures by restraining the receptive field of the network. Combining it with the undercomplete branch leads to a network which focuses more on low-level features without losing out on the global structures. Extensive experiments on two datasets demonstrate that the proposed method achieves significant improvements over the compressed sensing and popular deep learning-based methods with less number of trainable parameters.

Feasibility and Implementation of a Deep Learning MR Reconstruction for TSE Sequences in Musculoskeletal Imaging

Magnetic Resonance Imaging (MRI) of the musculoskeletal system is one of the most common examinations in clinical routine. The application of Deep Learning (DL) reconstruction for MRI is increasingly gaining attention due to its potential to improve the image quality and reduce the acquisition time simultaneously. However, the technology has not yet been implemented in clinical routine for turbo spin echo (TSE) sequences in musculoskeletal imaging. The aim of this study was therefore to assess the technical feasibility and evaluate the image quality. Sixty examinations of knee, hip, ankle, shoulder, hand, and lumbar spine in healthy volunteers at 3 T were included in this prospective, internal-review-board-approved study. Conventional (TSE_S) and DL-based TSE sequences (TSE_DL) were compared regarding image quality, anatomical structures, and diagnostic confidence. Overall image quality was rated to be excellent, with a significant improvement in edge sharpness and reduced noise compared to TSE_S (p < 0.001). No difference was found concerning the extent of artifacts, the delineation of anatomical structures, and the diagnostic confidence comparing TSE_S and TSE_DL (p > 0.05). Therefore, DL image reconstruction for TSE sequences in MSK imaging is feasible, enabling a remarkable time saving (up to 75%), whilst maintaining excellent image quality and diagnostic confidence.

Domain knowledge augmentation of parallel MR image reconstruction using deep learning

A deep learning (DL) method for accelerated magnetic resonance (MR) imaging is presented that incorporates domain knowledge of parallel MR imaging to augment the DL networks for accurate and stable image reconstruction. The proposed DL method employs a novel loss function consisting of a combination of mean absolute error, structural similarity, and sobel edge loss. The DL model takes both original measurements and images reconstructed by the parallel imaging method as inputs to the network. The accuracy of the proposed method was evaluated using two anatomical regions and six MRI contrasts and was compared with state-of-the-art parallel imaging and deep learning methods. The proposed method significantly outperformed the other methods for all the six different contrasts in terms of structural similarity, peak signal to noise ratio, and normalized mean squared error. The out-of-sample performance of the proposed method was assessed for a truly "unseen" case in a volunteer scan. The method produced images without any artificial features, often occurring in the DL image reconstruction methods. A stability analysis was performed by adding perturbations to the input, which demonstrated that the proposed method is robust and stable with respect to small structural changes, and different undersampling ratios. Comprehensive validation on large datasets demonstrated that incorporation of domain knowledge sufficiently regularizes the DL based image reconstruction and produces accurate and stable image enhancement.

Deep Semisupervised Multitask Learning Model and Its Interpretability for Survival Analysis

Survival analysis is a commonly used method in the medical field to analyze and predict the time of events. In medicine, this approach plays a key role in determining the course of treatment, developing new drugs, and improving hospital procedures. Most of the existing work in this area has addressed the problem by making strong assumptions about the underlying stochastic process. However, these assumptions are usually violated in the real-world data. This paper proposed a semisupervised multitask learning (SSMTL) method based on deep learning for survival analysis with or without competing risks. SSMTL transforms the survival analysis problem into a multitask learning problem that includes semisupervised learning and multipoint survival probability prediction. The distribution of survival times and the relationship between covariates and outcomes were modeled directly without any assumptions. Semisupervised loss and ranking loss are used to deal with censored data and the prior knowledge of the nonincreasing trend of the survival probability. Additionally, the importance of prognostic factors is determined, and the time-dependent and nonlinear effects of these factors on survival outcomes are visualized. The prediction performance of SSMTL is better than that of previous models in settings with or without competing risks, and the effects of predictors are successfully described. This study is of great significance for the exploration and application of deep learning methods involving medical structured data and provides an effective deep-learning-based method for survival analysis with complex-structured clinical data.

Optimizing multicontrast MRI reconstruction with shareable feature aggregation and selection

This paper proposes a new method for optimizing feature sharing in deep neural network-based, rapid, multicontrast magnetic resonance imaging (MC-MRI). Using the shareable information of MC images for accelerated MC-MRI reconstruction, current algorithms stack the MC images or features without optimizing the sharing protocols, leading to suboptimal reconstruction results. In this paper, we propose a novel feature aggregation and selection scheme in a deep neural network to better leverage the MC features and improve the reconstruction results. First, we propose to extract and use the shareable information by mapping the MC images into multiresolution feature maps with multilevel layers of the neural network. In this way, the extracted features capture complementary image properties, including local patterns from the shallow layers and semantic information from the deep layers. Then, an explicit selection module is designed to compile the extracted features optimally. That is, larger weights are learned to incorporate the constructive, shareable features; and smaller weights are assigned to the unshareable information. We conduct comparative studies on publicly available T2-weighted and T2-weighted fluid attenuated inversion recovery brain images, and the results show that the proposed network consistently outperforms existing algorithms. In addition, the proposed method can recover the images with high fidelity under 16 times acceleration. The ablation studies are conducted to evaluate the effectiveness of the proposed feature aggregation and selection mechanism. The results and the visualization of the weighted features show that the proposed method does effectively improve the usage of the useful features and suppress useless information, leading to overall enhanced reconstruction results. Additionally, the selection module can zero-out repeated and redundant features and improve network efficiency.

Reducing SAR in 7T brain fMRI by circumventing fat suppression while removing the lipid signal through a parallel acquisition approach

Ultra-high-field functional magnetic resonance imaging (fMRI) offers a way to new insights while increasing the spatial and temporal resolution. However, a crucial concern in 7T human MRI is the increase in power deposition, supervised through the specific absorption rate (SAR). The SAR limitation can restrict the brain coverage or the minimal repetition time of fMRI experiments. In the majority of today’s studies fMRI relies on the well-known gradient-echo echo-planar imaging (GRE-EPI) sequence, which offers ultrafast acquisition. Commonly, the GRE-EPI sequence comprises two pulses: fat suppression and excitation. This work provides the means for a significant reduction in the SAR by circumventing the fat-suppression pulse. Without this fat-suppression, however, lipid signal can result in artifacts due to the chemical shift between the lipid and water signals. Our approach exploits a reconstruction similar to the simultaneous-multi-slice method to separate the lipid and water images, thus avoiding undesired lipid artifacts in brain images. The lipid-water separation is based on the known spatial shift of the lipid signal, which can be detected by the multi-channel coils sensitivity profiles. Our study shows robust human imaging, offering greater flexibility to reduce the SAR, shorten the repetition time or increase the volume coverage with substantial benefit for brain functional studies.

Accelerating quantitative susceptibility and R2* mapping using incoherent undersampling and deep neural network reconstruction

Quantitative susceptibility mapping (QSM) and R2* mapping are MRI post-processing methods that quantify tissue magnetic susceptibility and transverse relaxation rate distributions. However, QSM and R2* acquisitions are relatively slow, even with parallel imaging. Incoherent undersampling and compressed sensing reconstruction techniques have been used to accelerate traditional magnitude-based MRI acquisitions; however, most do not recover the full phase signal, as required by QSM, due to its non-convex nature. In this study, a learning-based Deep Complex Residual Network (DCRNet) is proposed to recover both the magnitude and phase images from incoherently undersampled data, enabling high acceleration of QSM and R2* acquisition. Magnitude, phase, R2*, and QSM results from DCRNet were compared with two iterative and one deep learning methods on retrospectively undersampled acquisitions from six healthy volunteers, one intracranial hemorrhage and one multiple sclerosis patients, as well as one prospectively undersampled healthy subject using a 7T scanner. Peak signal to noise ratio (PSNR), structural similarity (SSIM), root-mean-squared error (RMSE), and region-of-interest susceptibility and R2* measurements are reported for numerical comparisons. The proposed DCRNet method substantially reduced artifacts and blurring compared to the other methods and resulted in the highest PSNR, SSIM, and RMSE on the magnitude, R2*, local field, and susceptibility maps. Compared to two iterative and one deep learning methods, the DCRNet method demonstrated a 3.2% to 9.1% accuracy improvement in deep grey matter susceptibility when accelerated by a factor of four. The DCRNet also dramatically shortened the reconstruction time of single 2D brain images from 36-140 seconds using conventional approaches to only 15-70 milliseconds.

Temporally aware volumetric generative adversarial network-based MR image reconstruction with simultaneous respiratory motion compensation: Initial feasibility in 3D dynamic cine cardiac MRI

PURPOSE

Develop a novel three-dimensional (3D) generative adversarial network (GAN)-based technique for simultaneous image reconstruction and respiratory motion compensation of 4D MRI. Our goal was to enable high-acceleration factors 10.7X-15.8X, while maintaining robust and diagnostic image quality superior to state-of-the-art self-gating (SG) compressed sensing wavelet (CS-WV) reconstruction at lower acceleration factors 3.5X-7.9X.

METHODS

Our GAN was trained based on pixel-wise content loss functions, adversarial loss function, and a novel data-driven temporal aware loss function to maintain anatomical accuracy and temporal coherence. Besides image reconstruction, our network also performs respiratory motion compensation for free-breathing scans. A novel progressive growing-based strategy was adapted to make the training process possible for the proposed GAN-based structure. The proposed method was developed and thoroughly evaluated qualitatively and quantitatively based on 3D cardiac cine data from 42 patients.

RESULTS

Our proposed method achieved significantly better scores in general image quality and image artifacts at 10.7X-15.8X acceleration than the SG CS-WV approach at 3.5X-7.9X acceleration (4.53 ± 0.540 vs. 3.13 ± 0.681 for general image quality, 4.12 ± 0.429 vs. 2.97 ± 0.434 for image artifacts, P < .05 for both). No spurious anatomical structures were observed in our images. The proposed method enabled similar cardiac-function quantification as conventional SG CS-WV. The proposed method achieved faster central processing unit-based image reconstruction (6 s/cardiac phase) than the SG CS-WV (312 s/cardiac phase).

CONCLUSION

The proposed method showed promising potential for high-resolution (1 mm³ ) free-breathing 4D MR data acquisition with simultaneous respiratory motion compensation and fast reconstruction time.

Deep learning in magnetic resonance image reconstruction

Magnetic resonance (MR) imaging visualises soft tissue contrast in exquisite detail without harmful ionising radiation. In this work, we provide a state-of-the-art review on the use of deep learning in MR image reconstruction from different image acquisition types involving compressed sensing techniques, parallel image acquisition and multi-contrast imaging. Publications with deep learning-based image reconstruction for MR imaging were identified from the literature (PubMed and Google Scholar), and a comprehensive description of each of the works was provided. A detailed comparison that highlights the differences, the data used and the performance of each of these works were also made. A discussion of the potential use cases for each of these methods is provided. The sparse image reconstruction methods were found to be most popular in using deep learning for improved performance, accelerating acquisitions by around 4-8 times. Multi-contrast image reconstruction methods rely on at least one pre-acquired image, but can achieve 16-fold, and even up to 32- to 50-fold acceleration depending on the set-up. Parallel imaging provides frameworks to be integrated in many of these methods for additional speed-up potential. The successful use of compressed sensing techniques and multi-contrast imaging with deep learning and parallel acquisition methods could yield significant MR acquisition speed-ups within clinical routines in the near future.

Systematic evaluation of iterative deep neural networks for fast parallel MRI reconstruction with sensitivity-weighted coil combination

PURPOSE

To systematically investigate the influence of various data consistency layers and regularization networks with respect to variations in the training and test data domain, for sensitivity-encoded accelerated parallel MR image reconstruction.

THEORY AND METHODS

Magnetic resonance (MR) image reconstruction is formulated as a learned unrolled optimization scheme with a down-up network as regularization and varying data consistency layers. The proposed networks are compared to other state-of-the-art approaches on the publicly available fastMRI knee and neuro dataset and tested for stability across different training configurations regarding anatomy and number of training samples.

RESULTS

Data consistency layers and expressive regularization networks, such as the proposed down-up networks, form the cornerstone for robust MR image reconstruction. Physics-based reconstruction networks outperform post-processing methods substantially for R = 4 in all cases and for R = 8 when the training and test data are aligned. At R = 8, aligning training and test data is more important than architectural choices.

CONCLUSION

In this work, we study how dataset sizes affect single-anatomy and cross-anatomy training of neural networks for MRI reconstruction. The study provides insights into the robustness, properties, and acceleration limits of state-of-the-art networks, and our proposed down-up networks. These key insights provide essential aspects to successfully translate learning-based MRI reconstruction to clinical practice, where we are confronted with limited datasets and various imaged anatomies.

Reconstruction of Compressed-sensing MR Imaging Using Deep Residual Learning in the Image Domain

Purpose:

A deep residual learning convolutional neural network (DRL-CNN) was applied to improve image quality and speed up the reconstruction of compressed sensing magnetic resonance imaging. The reconstruction performances of the proposed method was compared with iterative reconstruction methods.

Methods:

The proposed method adopted a DRL-CNN to learn the residual component between the input and output images (i.e., aliasing artifacts) for image reconstruction. The CNN-based reconstruction was compared with iterative reconstruction methods. To clarify the reconstruction performance of the proposed method, reconstruction experiments using 1D-, 2D-random under-sampling and sampling patterns that mix random and non-random under-sampling were executed. The peak-signal-to-noise ratio (PSNR) and the structural similarity index (SSIM) were examined for various numbers of training images, sampling rates, and numbers of training epochs.

Results:

The experimental results demonstrated that reconstruction time is drastically reduced to 0.022 s per image compared with that for conventional iterative reconstruction. The PSNR and SSIM were improved as the coherence of the sampling pattern increases. These results indicate that a deep CNN can learn coherent artifacts and is effective especially for cases where the randomness of k-space sampling is rather low. Simulation studies showed that variable density non-random under-sampling was a promising sampling pattern in 1D-random under-sampling of 2D image acquisition.

Conclusion:

A DRL-CNN can recognize and predict aliasing artifacts with low incoherence. It was demonstrated that reconstruction time is significantly reduced and the improvement in the PSNR and SSIM is higher in 1D-random under-sampling than in 2D. The requirement of incoherence for aliasing artifacts is different from that for iterative reconstruction.

Multi-institutional Collaborations for Improving Deep Learning-based Magnetic Resonance Image Reconstruction Using Federated Learning

Fast and accurate reconstruction of magnetic resonance (MR) images from under-sampled data is important in many clinical applications. In recent years, deep learning-based methods have been shown to produce superior performance on MR image reconstruction. However, these methods require large amounts of data which is difficult to collect and share due to the high cost of acquisition and medical data privacy regulations. In order to overcome this challenge, we propose a federated learning (FL) based solution in which we take advantage of the MR data available at different institutions while preserving patients' privacy. However, the generalizability of models trained with the FL setting can still be suboptimal due to domain shift, which results from the data collected at multiple institutions with different sensors, disease types, and acquisition protocols, etc. With the motivation of circumventing this challenge, we propose a cross-site modeling for MR image reconstruction in which the learned intermediate latent features among different source sites are aligned with the distribution of the latent features at the target site. Extensive experiments are conducted to provide various insights about FL for MR image reconstruction. Experimental results demonstrate that the proposed framework is a promising direction to utilize multi-institutional data without compromising patients' privacy for achieving improved MR image reconstruction. Our code is available at https://github.com/guopengf/FL-MRCM.

Adaptive convolutional neural networks for accelerating magnetic resonance imaging via k-space data interpolation

Deep learning in k-space has demonstrated great potential for image reconstruction from undersampled k-space data in fast magnetic resonance imaging (MRI). However, existing deep learning-based image reconstruction methods typically apply weight-sharing convolutional neural networks (CNNs) to k-space data without taking into consideration the k-space data's spatial frequency properties, leading to ineffective learning of the image reconstruction models. Moreover, complementary information of spatially adjacent slices is often ignored in existing deep learning methods. To overcome such limitations, we have developed a deep learning algorithm, referred to as adaptive convolutional neural networks for k-space data interpolation (ACNN-k-Space), which adopts a residual Encoder-Decoder network architecture to interpolate the undersampled k-space data by integrating spatially contiguous slices as multi-channel input, along with k-space data from multiple coils if available. The network is enhanced by self-attention layers to adaptively focus on k-space data at different spatial frequencies and channels. We have evaluated our method on two public datasets and compared it with state-of-the-art existing methods. Ablation studies and experimental results demonstrate that our method effectively reconstructs images from undersampled k-space data and achieves significantly better image reconstruction performance than current state-of-the-art techniques. Source code of the method is available at https://gitlab.com/qgpmztmf/acnn-k-space.

Two-step training deep learning framework for computational imaging without physics priors

Deep learning (DL) is a powerful tool in computational imaging for many applications. A common strategy is to use a preprocessor to reconstruct a preliminary image as the input to a neural network to achieve an optimized image. Usually, the preprocessor incorporates knowledge of the physics priors in the imaging model. One outstanding challenge, however, is errors that arise from imperfections in the assumed model. Model mismatches degrade the quality of the preliminary image and therefore affect the DL predictions. Another main challenge is that many imaging inverse problems are ill-posed and the networks are over-parameterized; DL networks have flexibility to extract features from the data that are not directly related to the imaging model. This can lead to suboptimal training and poorer image reconstruction results. To solve these challenges, a two-step training DL (TST-DL) framework is proposed for computational imaging without physics priors. First, a single fully-connected layer (FCL) is trained to directly learn the inverse model with the raw measurement data as the inputs and the images as the outputs. Then, this pre-trained FCL is fixed and concatenated with an un-trained deep convolutional network with a U-Net architecture for a second-step training to optimize the output image. This approach has the advantage that does not rely on an accurate representation of the imaging physics since the first-step training directly learns the inverse model. Furthermore, the TST-DL approach mitigates network over-parameterization by separately training the FCL and U-Net. We demonstrate this framework using a linear single-pixel camera imaging model. The results are quantitatively compared with those from other frameworks. The TST-DL approach is shown to perform comparable to approaches which incorporate perfect knowledge of the imaging model, to be robust to noise and model ill-posedness, and to be more robust to model mismatch than approaches which incorporate imperfect knowledge of the imaging model. Furthermore, TST-DL yields better results than end-to-end training while suffering from less overfitting. Overall, this TST-DL framework is a flexible approach for image reconstruction without physics priors, applicable to diverse computational imaging systems.

Two-stage deep learning for accelerated 3D time-of-flight MRA without matched training data

Time-of-flight magnetic resonance angiography (TOF-MRA) is one of the most widely used non-contrast MR imaging methods to visualize blood vessels, but due to the 3-D volume acquisition highly accelerated acquisition is necessary. Accordingly, high quality reconstruction from undersampled TOF-MRA is an important research topic for deep learning. However, most existing deep learning works require matched reference data for supervised training, which are often difficult to obtain. By extending the recent theoretical understanding of cycleGAN from the optimal transport theory, here we propose a novel two-stage unsupervised deep learning approach, which is composed of the multi-coil reconstruction network along the coronal plane followed by a multi-planar refinement network along the axial plane. Specifically, the first network is trained in the square-root of sum of squares (SSoS) domain to achieve high quality parallel image reconstruction, whereas the second refinement network is designed to efficiently learn the characteristics of highly-activated blood flow using double-headed projection discriminator. Extensive experiments demonstrate that the proposed learning process without matched reference exceeds performance of state-of-the-art compressed sensing (CS)-based method and provides comparable or even better results than supervised learning approaches.