Learning the Sampling Pattern for MRI

Knowledge-driven deep learning for fast MR imaging: Undersampled MR image reconstruction from supervised to un-supervised learning

Deep learning (DL) has emerged as a leading approach in accelerating MRI. It employs deep neural networks to extract knowledge from available datasets and then applies the trained networks to reconstruct accurate images from limited measurements. Unlike natural image restoration problems, MRI involves physics-based imaging processes, unique data properties, and diverse imaging tasks. This domain knowledge needs to be integrated with data-driven approaches. Our review will introduce the significant challenges faced by such knowledge-driven DL approaches in the context of fast MRI along with several notable solutions, which include learning neural networks and addressing different imaging application scenarios. The traits and trends of these techniques have also been given which have shifted from supervised learning to semi-supervised learning, and finally, to unsupervised learning methods. In addition, MR vendors' choices of DL reconstruction have been provided along with some discussions on open questions and future directions, which are critical for the reliable imaging systems.

On retrospective k-space subsampling schemes for deep MRI reconstruction

Acquiring fully-sampled MRI k-space data is time-consuming, and collecting accelerated data can reduce the acquisition time. Employing 2D Cartesian-rectilinear subsampling schemes is a conventional approach for accelerated acquisitions; however, this often results in imprecise reconstructions, even with the use of Deep Learning (DL), especially at high acceleration factors. Non-rectilinear or non-Cartesian trajectories can be implemented in MRI scanners as alternative subsampling options. This work investigates the impact of the k-space subsampling scheme on the quality of reconstructed accelerated MRI measurements produced by trained DL models. The Recurrent Variational Network (RecurrentVarNet) was used as the DL-based MRI-reconstruction architecture. Cartesian, fully-sampled multi-coil k-space measurements from three datasets were retrospectively subsampled with different accelerations using eight distinct subsampling schemes: four Cartesian-rectilinear, two Cartesian non-rectilinear, and two non-Cartesian. Experiments were conducted in two frameworks: scheme-specific, where a distinct model was trained and evaluated for each dataset-subsampling scheme pair, and multi-scheme, where for each dataset a single model was trained on data randomly subsampled by any of the eight schemes and evaluated on data subsampled by all schemes. In both frameworks, RecurrentVarNets trained and evaluated on non-rectilinearly subsampled data demonstrated superior performance, particularly for high accelerations. In the multi-scheme setting, reconstruction performance on rectilinearly subsampled data improved when compared to the scheme-specific experiments. Our findings demonstrate the potential for using DL-based methods, trained on non-rectilinearly subsampled measurements, to optimize scan time and image quality.

Intravoxel incoherent motion magnetic resonance imaging reconstruction from highly under-sampled diffusion-weighted PROPELLER acquisition data via physics-informed residual feedback unrolled network

Objective. The acquisition of diffusion-weighted images for intravoxel incoherent motion (IVIM) imaging is time consuming. This work aims to accelerate the scan through a highly under-sampling diffusion-weighted turbo spin echo PROPELLER (DW-TSE-PROPELLER) scheme and to develop a reconstruction method for accurate IVIM parameter mapping from the under-sampled data.Approach.The proposed under-sampling DW-TSE-PROPELLER scheme for IVIM imaging is that a few blades perb-value are acquired and rotated along theb-value dimension to cover high-frequency information. A physics-informed residual feedback unrolled network (PIRFU-Net) is proposed to directly estimate distortion-free and artifact-free IVIM parametric maps (i.e., the perfusion-free diffusion coefficientDand the perfusion fractionf) from highly under-sampled DW-TSE-PROPELLER data. PIRFU-Net used an unrolled convolution network to explore data redundancy in the k-q space to remove under-sampling artifacts. An empirical IVIM physical constraint was incorporated into the network to ensure that the signal evolution curves along theb-value follow a bi-exponential decay. The residual between the realistic and estimated measurements was fed into the network to refine the parametric maps. Meanwhile, the use of synthetic training data eliminated the need for genuine DW-TSE-PROPELLER data.Main results.The experimental results show that the DW-TSE-PROPELLER acquisition was six times faster than full k-space coverage PROPELLER acquisition and within a clinically acceptable time. Compared with the state-of-the-art methods, the distortion-freeDandfmaps estimated by PIRFU-Net were more accurate and had better-preserved tissue boundaries on a simulated human brain and realistic phantom/rat brain/human brain data.Significance.Our proposed method greatly accelerates IVIM imaging. It is capable of directly and simultaneously reconstructing distortion-free, artifact-free, and accurateDandfmaps from six-fold under-sampled DW-TSE-PROPELLER data.

AI-driven and automated MRI sequence optimization in scanner-independent MRI sequences formulated by a domain-specific language

Introduction

The complexity of Magnetic Resonance Imaging (MRI) sequences requires expert knowledge about the underlying contrast mechanisms to select from the wide range of available applications and protocols. Automation of this process using machine learning (ML) can support the radiologists and MR technicians by complementing their experience and finding the optimal MRI sequence and protocol for certain applications.

Methods

We define domain-specific languages (DSL) both for describing MRI sequences and for formulating clinical demands for sequence optimization. By using various abstraction levels, we allow different key users exact definitions of MRI sequences and make them more accessible to ML. We use a vendor-independent MRI framework (gammaSTAR) to build sequences that are formulated by the DSL and export them using the generic file format introduced by the Pulseq framework, making it possible to simulate phantom data using the open-source MR simulation framework JEMRIS to build a training database that relates input MRI sequences to output sets of metrics. Utilizing ML techniques, we learn this correspondence to allow efficient optimization of MRI sequences meeting the clinical demands formulated as a starting point.

Results

ML methods are capable of capturing the relation of input and simulated output parameters. Evolutionary algorithms show promising results in finding optimal MRI sequences with regards to the training data. Simulated and acquired MRI data show high correspondence to the initial set of requirements.

Discussion

This work has the potential to offer optimal solutions for different clinical scenarios, potentially reducing exam times by preventing suboptimal MRI protocol settings. Future work needs to cover additional DSL layers of higher flexibility as well as an optimization of the underlying MRI simulation process together with an extension of the optimization method.

Dual-domain accelerated MRI reconstruction using transformers with learning-based undersampling

Acceleration in MRI has garnered much attention from the deep-learning community in recent years, particularly for imaging large anatomical volumes such as the abdomen or moving targets such as the heart. A variety of deep learning approaches have been investigated, with most existing works using convolutional neural network (CNN)-based architectures as the reconstruction backbone, paired with fixed, rather than learned, k-space undersampling patterns. In both image domain and k-space, CNN-based architectures may not be optimal for reconstruction due to its limited ability to capture long-range dependencies. Furthermore, fixed undersampling patterns, despite ease of implementation, may not lead to optimal reconstruction. Lastly, few deep learning models to date have leveraged temporal correlation across dynamic MRI data to improve reconstruction. To address these gaps, we present a dual-domain (image and k-space), transformer-based reconstruction network, paired with learning-based undersampling that accepts temporally correlated sequences of MRI images for dynamic reconstruction. We call our model DuDReTLU-net. We train the network end-to-end against fully sampled ground truth dataset. Human cardiac CINE images undersampled at different factors (5-100) were tested. Reconstructed images were assessed both visually and quantitatively via the structural similarity index, mean squared error, and peak signal-to-noise. Experimental results show superior performance of DuDReTLU-net over state-of-the-art methods (LOUPE, k-t SLR, BM3D-MRI) in accelerated MRI reconstruction; ablation studies show that transformer-based reconstruction outperformed CNN-based reconstruction in both image domain and k-space; dual-domain reconstruction architectures outperformed single-domain reconstruction architectures regardless of reconstruction backbone (CNN or transformer); and dynamic sequence input leads to more accurate reconstructions than single frame input. We expect our results to encourage further research in the use of dual-domain architectures, transformer-based architectures, and learning-based undersampling, in the setting of accelerated MRI reconstruction. The code for this project is made freely available at https://github.com/william2343/dual-domain-mri-recon-nets (Hong et al., 2022).

Efficient Approximation of Jacobian Matrices Involving a Non-Uniform Fast Fourier Transform (NUFFT)

There is growing interest in learning Fourier domain sampling strategies (particularly for magnetic resonance imaging, MRI) using optimization approaches. For non-Cartesian sampling, the system models typically involve non-uniform fast Fourier transform (NUFFT) operations. Commonly used NUFFT algorithms contain frequency domain interpolation, which is not differentiable with respect to the sampling pattern, complicating the use of gradient methods. This paper describes an efficient and accurate approach for computing approximate gradients involving NUFFTs. Multiple numerical experiments validate the improved accuracy and efficiency of the proposed approximation. As an application to computational imaging, the NUFFT Jacobians were used to optimize non-Cartesian MRI sampling trajectories via data-driven stochastic optimization. Specifically, the sampling patterns were learned with respect to various model-based image reconstruction (MBIR) algorithms. The proposed approach enables sampling optimization for image sizes that are infeasible with standard auto-differentiation methods due to memory limits. The synergistic acquisition and reconstruction design leads to remarkably improved image quality. In fact, we show that model-based image reconstruction methods with suitably optimized imaging parameters can perform nearly as well as CNN-based methods.

Undersampling raster scans in spectromicroscopy for a reduced dose and faster measurements

Combinations of spectroscopic analysis and microscopic techniques are used across many disciplines of scientific research, including material science, chemistry and biology. X-ray spectromicroscopy, in particular, is a powerful tool used for studying chemical state distributions at the micro and nano scales. With the beam fixed, a specimen is typically rastered through the probe with continuous motion and a range of multimodal data is collected at fixed time intervals. The application of this technique is limited in some areas due to: long scanning times to collect the data, either because of the area/volume under study or the compositional properties of the specimen; and material degradation due to the dose absorbed during the measurement. In this work, we propose a novel approach for reducing the dose and scanning times by undersampling the raster data. This is achieved by skipping rows within scans and reconstructing the x-ray spectromicroscopic measurements using low-rank matrix completion. The new method is robust and allows for 5 to 6-fold reduction in sampling. Experimental results obtained on real data are illustrated.

Joint optimization of Cartesian sampling patterns and reconstruction for single-contrast and multi-contrast fast magnetic resonance imaging

BACKGROUND AND OBJECTIVE

Compressed sensing (CS) has gained increased attention in magnetic resonance imaging (MRI), leveraging its efficacy to accelerate image acquisition. Incoherence measurement and non-linear reconstruction are the most crucial guarantees of accurate restoration. However, the loose link between measurement and reconstruction hinders the further improvement of reconstruction quality, i.e., the default sampling pattern is not adaptively tailored to the downstream reconstruction method. When single-contrast reconstruction (SCR) has been upgraded to its multi-contrast reconstruction (MCR) variant, the identical morphologic information as a priori source could be integrated into the reconstruction procedure. How to measure less and reconstruct effectively by using the shareable morphologic information of various contrast images is an attractive topic.

METHODS

An adaptive sampling (AS) based end-to-end framework (ASSCR or ASMCR) is proposed to address this issue, which simultaneously optimizes sampling patterns and reconstruction from under-sampled data in SCR or MCR scenarios. Several deep probabilistic subsampling (DPS) modules are used in AS network to construct a sampling pattern generator. In SCR and MCR, a convolution block and a data consistency layer are iteratively applied in the reconstruction network. Specifically, the learned optimal sampling pattern output from the trained AS sub-net is used for under-sampling. Incoherence measurement for single-contrast images and the combination of sampling patterns for multi-contrast data are guided by the SCR/MCR sub-net.

RESULTS

Experiments were conducted on two single-contrast and one multi-contrast public MRI datasets. Compared with several state-of-the-art reconstruction methods, SCR results show that a learned sampling pattern brings the quality of the reconstructed image closer to the fully-sampled reference. With the addition of different contrast images, under-sampled images with higher acceleration factors could be well recovered.

CONCLUSION

The proposed method could improve the reconstruction quality of under-sampled images by using adaptive sampling patterns and learning-based reconstruction.

Joint Optimization of k-t Sampling Pattern and Reconstruction of DCE MRI for Pharmacokinetic Parameter Estimation

This work proposes to develop and evaluate a deep learning framework that jointly optimizes k-t sampling patterns and reconstruction for head and neck dynamic contrast-enhanced (DCE) MRI aiming to reduce bias and uncertainty of pharmacokinetic (PK) parameter estimation. 2D Cartesian phase encoding k-space subsampling patterns for a 3D spoiled gradient recalled echo (SPGR) sequence along a time course of DCE MRI were jointly optimized in a deep learning-based dynamic MRI reconstruction network by a loss function concerning both reconstruction image quality and PK parameter estimation accuracy. During training, temporal k-space data sharing scheme was optimized as well. The proposed method was trained and tested by multi-coil complex digital reference objects of DCE images (mcDROs). The PK parameters estimated by the proposed method were compared with two published iterative DCE MRI reconstruction schemes using normalized root mean squared errors (NRMSEs) and Bland-Altman analysis at temporal resolutions of [Formula: see text] = 2s, 3s, 4s, and 5s, which correspond to undersampling rates of R = 50, 34, 25, and 20. The proposed method achieved low PK parameter NRMSEs at all four temporal resolutions compared with the benchmark methods on testing mcDROs. The Bland-Altman plots demonstrated that the proposed method reduced PK parameter estimation bias and uncertainty in tumor regions at temporal resolution of 2s. The proposed method also showed robustness to contrast arrival timing variations across patients. This work provides a potential way to increase PK parameter estimation accuracy and precision, and thus facilitate the clinical translation of DCE MRI.

B-Spline Parameterized Joint Optimization of Reconstruction and K-Space Trajectories (BJORK) for Accelerated 2D MRI

Optimizing k-space sampling trajectories is a promising yet challenging topic for fast magnetic resonance imaging (MRI). This work proposes to optimize a reconstruction method and sampling trajectories jointly concerning image reconstruction quality in a supervised learning manner. We parameterize trajectories with quadratic B-spline kernels to reduce the number of parameters and apply multi-scale optimization, which may help to avoid sub-optimal local minima. The algorithm includes an efficient non-Cartesian unrolled neural network-based reconstruction and an accurate approximation for backpropagation through the non-uniform fast Fourier transform (NUFFT) operator to accurately reconstruct and back-propagate multi-coil non-Cartesian data. Penalties on slew rate and gradient amplitude enforce hardware constraints. Sampling and reconstruction are trained jointly using large public datasets. To correct for possible eddy-current effects introduced by the curved trajectory, we use a pencil-beam trajectory mapping technique. In both simulations and in- vivo experiments, the learned trajectory demonstrates significantly improved image quality compared to previous model-based and learning-based trajectory optimization methods for 10× acceleration factors. Though trained with neural network-based reconstruction, the proposed trajectory also leads to improved image quality with compressed sensing-based reconstruction.

Deep learning in macroscopic diffuse optical imaging

SIGNIFICANCE

Biomedical optics system design, image formation, and image analysis have primarily been guided by classical physical modeling and signal processing methodologies. Recently, however, deep learning (DL) has become a major paradigm in computational modeling and has demonstrated utility in numerous scientific domains and various forms of data analysis.

AIM

We aim to comprehensively review the use of DL applied to macroscopic diffuse optical imaging (DOI).

APPROACH

First, we provide a layman introduction to DL. Then, the review summarizes current DL work in some of the most active areas of this field, including optical properties retrieval, fluorescence lifetime imaging, and diffuse optical tomography.

RESULTS

The advantages of using DL for DOI versus conventional inverse solvers cited in the literature reviewed herein are numerous. These include, among others, a decrease in analysis time (often by many orders of magnitude), increased quantitative reconstruction quality, robustness to noise, and the unique capability to learn complex end-to-end relationships.

CONCLUSIONS

The heavily validated capability of DL's use across a wide range of complex inverse solving methodologies has enormous potential to bring novel DOI modalities, otherwise deemed impractical for clinical translation, to the patient's bedside.

Alternating Learning Approach for Variational Networks and Undersampling Pattern in Parallel MRI Applications

This work proposes an alternating learning approach to learn the sampling pattern (SP) and the parameters of variational networks (VN) in accelerated parallel magnetic resonance imaging (MRI). We investigate four variations of the learning approach, that alternates between improving the SP, using bias-accelerated subset selection, and improving parameters of the VN, using ADAM. The variations include the use of monotone or non-monotone alternating steps and systematic reduction of learning rates. The algorithms learn an effective pair to be used in future scans, including an SP that captures fewer k-space samples in which the generated undersampling artifacts are removed by the VN reconstruction. The quality of the VNs and SPs obtained by the proposed approaches is compared against different methods, including other kinds of joint learning methods and state-of-art reconstructions, on two different datasets at various acceleration factors (AF). We observed improvements visually and in three different figures of merit commonly used in deep learning (RMSE, SSIM, and HFEN) on AFs from 2 to 20 with brain and knee joint datasets when compared to the other approaches. The improvements ranged from 1% to 62% over the next best approach tested with VNs. The proposed approach has shown stable performance, obtaining similar learned SPs under different initial training conditions. We observe that the improvement is not only due to the learned sampling density, it is also due to the learned position of samples in k-space. The proposed approach was able to learn effective pairs of SPs and reconstruction VNs, improving 3D Cartesian accelerated parallel MRI applications.

Fast data-driven learning of parallel MRI sampling patterns for large scale problems

In this study, a fast data-driven optimization approach, named bias-accelerated subset selection (BASS), is proposed for learning efficacious sampling patterns (SPs) with the purpose of reducing scan time in large-dimensional parallel MRI. BASS is applicable when Cartesian fully-sampled k-space measurements of specific anatomy are available for training and the reconstruction method for undersampled measurements is specified; such information is used to define the efficacy of any SP for recovering the values at the non-sampled k-space points. BASS produces a sequence of SPs with the aim of finding one of a specified size with (near) optimal efficacy. BASS was tested with five reconstruction methods for parallel MRI based on low-rankness and sparsity that allow a free choice of the SP. Three datasets were used for testing, two of high-resolution brain images ([Formula: see text]-weighted images and, respectively, [Formula: see text]-weighted images) and another of knee images for quantitative mapping of the cartilage. The proposed approach has low computational cost and fast convergence; in the tested cases it obtained SPs up to 50 times faster than the currently best greedy approach. Reconstruction quality increased by up to 45% over that provided by variable density and Poisson disk SPs, for the same scan time. Optionally, the scan time can be nearly halved without loss of reconstruction quality. Quantitative MRI and prospective accelerated MRI results show improvements. Compared with greedy approaches, BASS rapidly learns effective SPs for various reconstruction methods, using larger SPs and larger datasets; enabling better selection of sampling-reconstruction pairs for specific MRI problems.

MRzero - Automated discovery of MRI sequences using supervised learning

PURPOSE

A supervised learning framework is proposed to automatically generate MR sequences and corresponding reconstruction based on the target contrast of interest. Combined with a flexible, task-driven cost function this allows for an efficient exploration of novel MR sequence strategies.

METHODS

The scanning and reconstruction process is simulated end-to-end in terms of RF events, gradient moment events in x and y, and delay times, acting on the input model spin system given in terms of proton density, T 1 and T 2 , and Δ B 0 . As a proof of concept, we use both conventional MR images and T 1 maps as targets and optimize from scratch using the loss defined by data fidelity, SAR penalty, and scan time.

RESULTS

In a first attempt, MRzero learns gradient and RF events from zero, and is able to generate a target image produced by a conventional gradient echo sequence. Using a neural network within the reconstruction module allows arbitrary targets to be learned successfully. Experiments could be translated to image acquisition at the real system (3T Siemens, PRISMA) and could be verified in the measurements of phantoms and a human brain in vivo.

CONCLUSIONS

Automated MR sequence generation is possible based on differentiable Bloch equation simulations and a supervised learning approach.