Results of the 2020 fastMRI Challenge for Machine Learning MR Image Reconstruction

B-Spline Parameterized Joint Optimization of Reconstruction and K-Space Trajectories (BJORK) for Accelerated 2D MRI

Optimizing k-space sampling trajectories is a promising yet challenging topic for fast magnetic resonance imaging (MRI). This work proposes to optimize a reconstruction method and sampling trajectories jointly concerning image reconstruction quality in a supervised learning manner. We parameterize trajectories with quadratic B-spline kernels to reduce the number of parameters and apply multi-scale optimization, which may help to avoid sub-optimal local minima. The algorithm includes an efficient non-Cartesian unrolled neural network-based reconstruction and an accurate approximation for backpropagation through the non-uniform fast Fourier transform (NUFFT) operator to accurately reconstruct and back-propagate multi-coil non-Cartesian data. Penalties on slew rate and gradient amplitude enforce hardware constraints. Sampling and reconstruction are trained jointly using large public datasets. To correct for possible eddy-current effects introduced by the curved trajectory, we use a pencil-beam trajectory mapping technique. In both simulations and in- vivo experiments, the learned trajectory demonstrates significantly improved image quality compared to previous model-based and learning-based trajectory optimization methods for 10× acceleration factors. Though trained with neural network-based reconstruction, the proposed trajectory also leads to improved image quality with compressed sensing-based reconstruction.

Revisiting [Formula: see text]-wavelet compressed-sensing MRI in the era of deep learning

Following their success in numerous imaging and computer vision applications, deep-learning (DL) techniques have emerged as one of the most prominent strategies for accelerated MRI reconstruction. These methods have been shown to outperform conventional regularized methods based on compressed sensing (CS). However, in most comparisons, CS is implemented with two or three hand-tuned parameters, while DL methods enjoy a plethora of advanced data science tools. In this work, we revisit [Formula: see text]-wavelet CS reconstruction using these modern tools. Using ideas such as algorithm unrolling and advanced optimization methods over large databases that DL algorithms utilize, along with conventional insights from wavelet representations and CS theory, we show that [Formula: see text]-wavelet CS can be fine-tuned to a level close to DL reconstruction for accelerated MRI. The optimized [Formula: see text]-wavelet CS method uses only 128 parameters compared to >500,000 for DL, employs a convex reconstruction at inference time, and performs within <1% of a DL approach that has been used in multiple studies in terms of quantitative quality metrics.

Pyramid Convolutional RNN for MRI Image Reconstruction

Fast and accurate MRI image reconstruction from undersampled data is crucial in clinical practice. Deep learning based reconstruction methods have shown promising advances in recent years. However, recovering fine details from undersampled data is still challenging. In this paper, we introduce a novel deep learning based method, Pyramid Convolutional RNN (PC-RNN), to reconstruct images from multiple scales. Based on the formulation of MRI reconstruction as an inverse problem, we design the PC-RNN model with three convolutional RNN (ConvRNN) modules to iteratively learn the features in multiple scales. Each ConvRNN module reconstructs images at different scales and the reconstructed images are combined by a final CNN module in a pyramid fashion. The multi-scale ConvRNN modules learn a coarse-to-fine image reconstruction. Unlike other common reconstruction methods for parallel imaging, PC-RNN does not employ coil sensitive maps for multi-coil data and directly model the multiple coils as multi-channel inputs. The coil compression technique is applied to standardize data with various coil numbers, leading to more efficient training. We evaluate our model on the fastMRI knee and brain datasets and the results show that the proposed model outperforms other methods and can recover more details. The proposed method is one of the winner solutions in the 2019 fastMRI competition.

Multi-Coil MRI Reconstruction Challenge-Assessing Brain MRI Reconstruction Models and Their Generalizability to Varying Coil Configurations

Deep-learning-based brain magnetic resonance imaging (MRI) reconstruction methods have the potential to accelerate the MRI acquisition process. Nevertheless, the scientific community lacks appropriate benchmarks to assess the MRI reconstruction quality of high-resolution brain images, and evaluate how these proposed algorithms will behave in the presence of small, but expected data distribution shifts. The multi-coil MRI (MC-MRI) reconstruction challenge provides a benchmark that aims at addressing these issues, using a large dataset of high-resolution, three-dimensional, T1-weighted MRI scans. The challenge has two primary goals: (1) to compare different MRI reconstruction models on this dataset and (2) to assess the generalizability of these models to data acquired with a different number of receiver coils. In this paper, we describe the challenge experimental design and summarize the results of a set of baseline and state-of-the-art brain MRI reconstruction models. We provide relevant comparative information on the current MRI reconstruction state-of-the-art and highlight the challenges of obtaining generalizable models that are required prior to broader clinical adoption. The MC-MRI benchmark data, evaluation code, and current challenge leaderboard are publicly available. They provide an objective performance assessment for future developments in the field of brain MRI reconstruction.

NC-PDNet: A Density-Compensated Unrolled Network for 2D and 3D Non-Cartesian MRI Reconstruction

Deep Learning has become a very promising avenue for magnetic resonance image (MRI) reconstruction. In this work, we explore the potential of unrolled networks for non-Cartesian acquisition settings. We design the NC-PDNet (Non-Cartesian Primal Dual Netwok), the first density-compensated (DCp) unrolled neural network, and validate the need for its key components via an ablation study. Moreover, we conduct some generalizability experiments to test this network in out-of-distribution settings, for example training on knee data and validating on brain data. The results show that NC-PDNet outperforms baseline (U-Net, Deep image prior) models both visually and quantitatively in all settings. In particular, in the 2D multi-coil acquisition scenario, the NC-PDNet provides up to a 1.2 dB improvement in peak signal-to-noise ratio (PSNR) over baseline networks, while also allowing a gain of at least 1dB in PSNR in generalization settings. We provide the open-source implementation of NC-PDNet, and in particular the Non-uniform Fourier Transform in TensorFlow, tested on 2D multi-coil and 3D single-coil k-space data.

Assessment of data consistency through cascades of independently recurrent inference machines for fast and robust accelerated MRI reconstruction

Objective. Machine Learning methods can learn how to reconstruct magnetic resonance images (MRI) and thereby accelerate acquisition, which is of paramount importance to the clinical workflow. Physics-informed networks incorporate the forward model of accelerated MRI reconstruction in the learning process. With increasing network complexity, robustness is not ensured when reconstructing data unseen during training. We aim to embed data consistency (DC) in deep networks while balancing the degree of network complexity. While doing so, we will assess whether either explicit or implicit enforcement of DC in varying network architectures is preferred to optimize performance. Approach. We propose a scheme called Cascades of Independently Recurrent Inference Machines (CIRIM) to assess DC through unrolled optimization. Herein we assess DC both implicitly by gradient descent and explicitly by a designed term. Extensive comparison of the CIRIM to compressed sensing as well as other Machine Learning methods is performed: the End-to-End Variational Network (E2EVN), CascadeNet, KIKINet, LPDNet, RIM, IRIM, and UNet. Models were trained and evaluated on T1-weighted and FLAIR contrast brain data, and T2-weighted knee data. Both 1D and 2D undersampling patterns were evaluated. Robustness was tested by reconstructing 7.5× prospectively undersampled 3D FLAIR MRI data of multiple sclerosis (MS) patients with white matter lesions. Main results. The CIRIM performed best when implicitly enforcing DC, while the E2EVN required an explicit DC formulation. Through its cascades, the CIRIM was able to score higher on structural similarity and PSNR compared to other methods, in particular under heterogeneous imaging conditions. In reconstructing MS patient data, prospectively acquired with a sampling pattern unseen during model training, the CIRIM maintained lesion contrast while efficiently denoising the images. Significance. The CIRIM showed highly promising generalization capabilities maintaining a very fair trade-off between reconstructed image quality and fast reconstruction times, which is crucial in the clinical workflow.

Real-time cardiac MRI using an undersampled spiral k-space trajectory and a reconstruction based on a variational network

PURPOSE

Cardiac MRI represents the gold standard to determine myocardial function. However, the current clinical standard protocol, a segmented Cartesian acquisition, is time-consuming and can lead to compromised image quality in the case of arrhythmia or dyspnea. In this article, a machine learning-based reconstruction of undersampled spiral k-space data is presented to enable free breathing real-time cardiac MRI with good image quality and short reconstruction times.

METHODS

Data were acquired in free breathing with a 2D spiral trajectory corrected by the gradient system transfer function. Undersampled data were reconstructed by a variational network (VN), which was specifically adapted to the non-Cartesian sampling pattern. The network was trained with data from 11 subjects. Subsequently, the imaging technique was validated in 14 subjects by quantifying the difference to a segmented reference acquisition, an expert reader study, and by comparing derived volumes and functional parameters with values obtained using the current clinical gold standard.

RESULTS

The scan time for the entire heart was below 1 min. The VN reconstructed data in about 0.9 s per image, which is considerably shorter than conventional model-based approaches. The VN furthermore performed better than a U-Net and not inferior to a low-rank plus sparse model in terms of achieved image quality. Functional parameters agreed, on average, with reference data.

CONCLUSIONS

The proposed VN method enables real-time cardiac imaging with both high spatial and temporal resolution in free breathing and with short reconstruction time.

Convolutional neural network-based reconstruction for acceleration of prostate T2 weighted MR imaging: a retro- and prospective study

OBJECTIVE

The aim of this study was to develop a deep neural network (DNN)-based parallel imaging reconstruction for highly accelerated 2D turbo spin echo (TSE) data in prostate MRI without quality degradation compared to conventional scans.

METHODS

155 participant data were acquired for training and testing. Two DNN models were generated according to the number of acquisitions (NAQ) of input images: DNN-N1 for NAQ = 1 and DNN-N2 for NAQ = 2. In the test data, DNN and TSE images were compared by quantitative error metrics. The visual appropriateness of DNN reconstructions on accelerated scans (DNN-N1 and DNN-N2) and conventional scans (TSE-Conv) was assessed for nine parameters by two radiologists. The lesion detection was evaluated at DNNs and TES-Conv by prostate imaging-reporting and data system.

RESULTS

The scan time was reduced by 71% at NAQ = 1, and 42% at NAQ = 2. Quantitative evaluation demonstrated the better error metrics of DNN images (29-43% lower NRMSE, 4-13% higher structure similarity index, and 2.8-4.8 dB higher peak signal-to-noise ratio; p < 0.001) than TSE images. In the assessment of the visual appropriateness, both radiologists evaluated that DNN-N2 showed better or comparable performance in all parameters compared to TSE-Conv. In the lesion detection, DNN images showed almost perfect agreement (κ > 0.9) scores with TSE-Conv.

CONCLUSIONS

DNN-based reconstruction in highly accelerated prostate TSE imaging showed comparable quality to conventional TSE.

ADVANCES IN KNOWLEDGE

Our framework reduces the scan time by 42% of conventional prostate TSE imaging without sequence modification, revealing great potential for clinical application.

Blip up-down acquisition for spin- and gradient-echo imaging (BUDA-SAGE) with self-supervised denoising enables efficient T2 , T2 *, para- and dia-magnetic susceptibility mapping

PURPOSE

To rapidly obtain high resolution T₂ , T₂ *, and quantitative susceptibility mapping (QSM) source separation maps with whole-brain coverage and high geometric fidelity.

METHODS

We propose Blip Up-Down Acquisition for Spin And Gradient Echo imaging (BUDA-SAGE), an efficient EPI sequence for quantitative mapping. The acquisition includes multiple T₂ *-, T₂ '-, and T₂ -weighted contrasts. We alternate the phase-encoding polarities across the interleaved shots in this multi-shot navigator-free acquisition. A field map estimated from interim reconstructions was incorporated into the joint multi-shot EPI reconstruction with a structured low rank constraint to eliminate distortion. A self-supervised neural network (NN), MR-Self2Self (MR-S2S), was used to perform denoising to boost SNR. Using Slider encoding allowed us to reach 1 mm isotropic resolution by performing super-resolution reconstruction on volumes acquired with 2 mm slice thickness. Quantitative T₂ (=1/R₂ ) and T₂ * (=1/R₂ *) maps were obtained using Bloch dictionary matching on the reconstructed echoes. QSM was estimated using nonlinear dipole inversion on the gradient echoes. Starting from the estimated R₂ /R₂ * maps, R₂ ' information was derived and used in source separation QSM reconstruction, which provided additional para- and dia-magnetic susceptibility maps.

RESULTS

In vivo results demonstrate the ability of BUDA-SAGE to provide whole-brain, distortion-free, high-resolution, multi-contrast images and quantitative T₂ /T₂ * maps, as well as yielding para- and dia-magnetic susceptibility maps. Estimated quantitative maps showed comparable values to conventional mapping methods in phantom and in vivo measurements.

CONCLUSION

BUDA-SAGE acquisition with self-supervised denoising and Slider encoding enables rapid, distortion-free, whole-brain T₂ /T₂ * mapping at 1 mm isotropic resolution under 90 s.

Implicit data crimes: Machine learning bias arising from misuse of public data

SignificancePublic databases are an important resource for machine learning research, but their growing availability sometimes leads to "off-label" usage, where data published for one task are used for another. This work reveals that such off-label usage could lead to biased, overly optimistic results of machine-learning algorithms. The underlying cause is that public data are processed with hidden processing pipelines that alter the data features. Here we study three well-known algorithms developed for image reconstruction from magnetic resonance imaging measurements and show they could produce biased results with up to 48% artificial improvement when applied to public databases. We relate to the publication of such results as implicit "data crimes" to raise community awareness of this growing big data problem.

Highly accelerated 3D MPRAGE using deep neural network-based reconstruction for brain imaging in children and young adults

OBJECTIVES

This study aimed to accelerate the 3D magnetization-prepared rapid gradient-echo (MPRAGE) sequence for brain imaging through the deep neural network (DNN).

METHODS

This retrospective study used the k-space data of 240 scans (160 for the training set, mean ± standard deviation age, 93 ± 80 months, 94 males; 80 for the test set, 106 ± 83 months, 44 males) of conventional MPRAGE (C-MPRAGE) and 102 scans (77 ± 74 months, 52 males) of both C-MPRAGE and accelerated MPRAGE. All scans were acquired with 3T scanners. DNN was developed with simulated-acceleration data generated by under-sampling. Quantitative error metrics were compared between images reconstructed with DNN, GRAPPA, and E-SPIRIT using the paired t-test. Qualitative image quality was compared between C-MPRAGE and accelerated MPRAGE reconstructed with DNN (DNN-MPRAGE) by two readers. Lesions were segmented and the agreement between C-MPRAGE and DNN-MPRAGE was assessed using linear regression.

RESULTS

Accelerated MPRAGE reduced scan times by 38% compared to C-MPRAGE (142 s vs. 320 s). For quantitative error metrics, DNN showed better performance than GRAPPA and E-SPIRIT (p < 0.001). For qualitative evaluation, overall image quality of DNN-MPRAGE was comparable (p > 0.999) or better (p = 0.025) than C-MPRAGE, depending on the reader. Pixelation was reduced in DNN-MPRAGE (p < 0.001). Other qualitative parameters were comparable (p > 0.05). Lesions in C-MPRAGE and DNN-MPRAGE showed good agreement for the dice similarity coefficient (= 0.68) and linear regression (R² = 0.97; p < 0.001).

CONCLUSIONS

DNN-MPRAGE reduced acquisition time by 38% and revealed comparable image quality to C-MPRAGE.

KEY POINTS

• DNN-MPRAGE reduced acquisition times by 38%. • DNN-MPRAGE outperformed conventional reconstruction on accelerated scans (SSIM of DNN-MPRAGE = 0.96, GRAPPA = 0.43, E-SPIRIT = 0.88; p < 0.001). • Compared to C-MPRAGE scans, DNN-MPRAGE showed improved mean scores for overall image quality (2.46 vs. 2.52; p < 0.001) or comparable perceived SNR (2.56 vs. 2.58; p = 0.08).

The difficulty of computing stable and accurate neural networks: On the barriers of deep learning and Smale's 18th problem

Instability is the Achilles’ heel of modern artificial intelligence (AI) and a paradox, with training algorithms finding unstable neural networks (NNs) despite the existence of stable ones. This foundational issue relates to Smale’s 18th mathematical problem for the 21st century on the limits of AI. By expanding methodologies initiated by Gödel and Turing, we demonstrate limitations on the existence of (even randomized) algorithms for computing NNs. Despite numerous existence results of NNs with great approximation properties, only in specific cases do there also exist algorithms that can compute them. We initiate a classification theory on which NNs can be trained and introduce NNs that—under suitable conditions—are robust to perturbations and exponentially accurate in the number of hidden layers.

Deep learning (DL) has had unprecedented success and is now entering scientific computing with full force. However, current DL methods typically suffer from instability, even when universal approximation properties guarantee the existence of stable neural networks (NNs). We address this paradox by demonstrating basic well-conditioned problems in scientific computing where one can prove the existence of NNs with great approximation qualities; however, there does not exist any algorithm, even randomized, that can train (or compute) such a NN. For any positive integers K>2 and L, there are cases where simultaneously 1) no randomized training algorithm can compute a NN correct to K digits with probability greater than 1/2; 2) there exists a deterministic training algorithm that computes a NN with K –1 correct digits, but any such (even randomized) algorithm needs arbitrarily many training data; and 3) there exists a deterministic training algorithm that computes a NN with K –2 correct digits using no more than L training samples. These results imply a classification theory describing conditions under which (stable) NNs with a given accuracy can be computed by an algorithm. We begin this theory by establishing sufficient conditions for the existence of algorithms that compute stable NNs in inverse problems. We introduce fast iterative restarted networks (FIRENETs), which we both prove and numerically verify are stable. Moreover, we prove that only O(|log (ϵ)|) layers are needed for an ϵ-accurate solution to the inverse problem.

A multispeaker dataset of raw and reconstructed speech production real-time MRI video and 3D volumetric images

Real-time magnetic resonance imaging (RT-MRI) of human speech production is enabling significant advances in speech science, linguistics, bio-inspired speech technology development, and clinical applications. Easy access to RT-MRI is however limited, and comprehensive datasets with broad access are needed to catalyze research across numerous domains. The imaging of the rapidly moving articulators and dynamic airway shaping during speech demands high spatio-temporal resolution and robust reconstruction methods. Further, while reconstructed images have been published, to-date there is no open dataset providing raw multi-coil RT-MRI data from an optimized speech production experimental setup. Such datasets could enable new and improved methods for dynamic image reconstruction, artifact correction, feature extraction, and direct extraction of linguistically-relevant biomarkers. The present dataset offers a unique corpus of 2D sagittal-view RT-MRI videos along with synchronized audio for 75 participants performing linguistically motivated speech tasks, alongside the corresponding public domain raw RT-MRI data. The dataset also includes 3D volumetric vocal tract MRI during sustained speech sounds and high-resolution static anatomical T2-weighted upper airway MRI for each participant.