M-MRI: A Manifold-based Framework to Highly Accelerated Dynamic Magnetic Resonance Imaging

Subspace Model-Assisted Deep Learning for Improved Image Reconstruction

Image reconstruction from limited and/or sparse data is known to be an ill-posed problem and a priori information/constraints have played an important role in solving the problem. Early constrained image reconstruction methods utilize image priors based on general image properties such as sparsity, low-rank structures, spatial support bound, etc. Recent deep learning-based reconstruction methods promise to produce even higher quality reconstructions by utilizing more specific image priors learned from training data. However, learning high-dimensional image priors requires huge amounts of training data that are currently not available in medical imaging applications. As a result, deep learning-based reconstructions often suffer from two known practical issues: a) sensitivity to data perturbations (e.g., changes in data sampling scheme), and b) limited generalization capability (e.g., biased reconstruction of lesions). This paper proposes a new method to address these issues. The proposed method synergistically integrates model-based and data-driven learning in three key components. The first component uses the linear vector space framework to capture global dependence of image features; the second exploits a deep network to learn the mapping from a linear vector space to a nonlinear manifold; the third is an unrolling-based deep network that captures local residual features with the aid of a sparsity model. The proposed method has been evaluated with magnetic resonance imaging data, demonstrating improved reconstruction in the presence of data perturbation and/or novel image features. The method may enhance the practical utility of deep learning-based image reconstruction.

MR image reconstruction from undersampled data for image-guided radiation therapy using a patient-specific deep manifold image prior

Introduction

Recent advancements in radiotherapy (RT) have allowed for the integration of a Magnetic Resonance (MR) imaging scanner with a medical linear accelerator to use MR images for image guidance to position tumors against the treatment beam. Undersampling in MR acquisition is desired to accelerate the imaging process, but unavoidably deteriorates the reconstructed image quality. In RT, a high-quality MR image of a patient is available for treatment planning. In light of this unique clinical scenario, we proposed to exploit the patient-specific image prior to facilitate high-quality MR image reconstruction.

Methods

Utilizing the planning MR image, we established a deep auto-encoder to form a manifold of image patches of the patient. The trained manifold was then incorporated as a regularization to restore MR images of the same patient from undersampled data. We performed a simulation study using a patient case, a real patient study with three liver cancer patient cases, and a phantom experimental study using data acquired on an in-house small animal MR scanner. We compared the performance of the proposed method with those of the Fourier transform method, a tight-frame based Compressive Sensing method, and a deep learning method with a patient-generic manifold as the image prior.

Results

In the simulation study with 12.5% radial undersampling and 15% increase in noise, our method improved peak-signal-to-noise ratio by 4.46dB and structural similarity index measure by 28% compared to the patient-generic manifold method. In the experimental study, our method outperformed others by producing reconstructions of visually improved image quality.

NC-PDNet: A Density-Compensated Unrolled Network for 2D and 3D Non-Cartesian MRI Reconstruction

Deep Learning has become a very promising avenue for magnetic resonance image (MRI) reconstruction. In this work, we explore the potential of unrolled networks for non-Cartesian acquisition settings. We design the NC-PDNet (Non-Cartesian Primal Dual Netwok), the first density-compensated (DCp) unrolled neural network, and validate the need for its key components via an ablation study. Moreover, we conduct some generalizability experiments to test this network in out-of-distribution settings, for example training on knee data and validating on brain data. The results show that NC-PDNet outperforms baseline (U-Net, Deep image prior) models both visually and quantitatively in all settings. In particular, in the 2D multi-coil acquisition scenario, the NC-PDNet provides up to a 1.2 dB improvement in peak signal-to-noise ratio (PSNR) over baseline networks, while also allowing a gain of at least 1dB in PSNR in generalization settings. We provide the open-source implementation of NC-PDNet, and in particular the Non-uniform Fourier Transform in TensorFlow, tested on 2D multi-coil and 3D single-coil k-space data.

Time-Dependent Deep Image Prior for Dynamic MRI

We propose a novel unsupervised deep-learning-based algorithm for dynamic magnetic resonance imaging (MRI) reconstruction. Dynamic MRI requires rapid data acquisition for the study of moving organs such as the heart. We introduce a generalized version of the deep-image-prior approach, which optimizes the weights of a reconstruction network to fit a sequence of sparsely acquired dynamic MRI measurements. Our method needs neither prior training nor additional data. In particular, for cardiac images, it does not require the marking of heartbeats or the reordering of spokes. The key ingredients of our method are threefold: 1) a fixed low-dimensional manifold that encodes the temporal variations of images; 2) a network that maps the manifold into a more expressive latent space; and 3) a convolutional neural network that generates a dynamic series of MRI images from the latent variables and that favors their consistency with the measurements in k -space. Our method outperforms the state-of-the-art methods quantitatively and qualitatively in both retrospective and real fetal cardiac datasets. To the best of our knowledge, this is the first unsupervised deep-learning-based method that can reconstruct the continuous variation of dynamic MRI sequences with high spatial resolution.

Multi-scale Unrolled Deep Learning Framework for Accelerated Magnetic Resonance Imaging

Accelerating data acquisition in magnetic resonance imaging (MRI) has been of perennial interest due to its prohibitively slow data acquisition process. Recent trends in accelerating MRI employ data-centric deep learning frameworks due to its fast inference time and 'one-parameter-fit-all' principle unlike in traditional model-based acceleration techniques. Unrolled deep learning framework that combines the deep priors and model knowledge are robust compared to naive deep learning based framework. In this paper, we propose a novel multi-scale unrolled deep learning framework which learns deep image priors through multi-scale CNN and is combined with unrolled framework to enforce data-consistency and model knowledge. Essentially, this framework combines the best of both learning paradigms:model-based and data-centric learning paradigms. Proposed method is verified using several experiments on numerous data sets.

Bi-Linear Modeling of Data Manifolds for Dynamic-MRI Recovery

This paper puts forth a novel bi-linear modeling framework for data recovery via manifold-learning and sparse-approximation arguments and considers its application to dynamic magnetic-resonance imaging (dMRI). Each temporal-domain MR image is viewed as a point that lies onto or close to a smooth manifold, and landmark points are identified to describe the point cloud concisely. To facilitate computations, a dimensionality reduction module generates low-dimensional/compressed renditions of the landmark points. Recovery of high-fidelity MRI data is realized by solving a non-convex minimization task for the linear decompression operator and affine combinations of landmark points which locally approximate the latent manifold geometry. An algorithm with guaranteed convergence to stationary solutions of the non-convex minimization task is also provided. The aforementioned framework exploits the underlying spatio-temporal patterns and geometry of the acquired data without any prior training on external data or information. Extensive numerical results on simulated as well as real cardiac-cine MRI data illustrate noteworthy improvements of the advocated machine-learning framework over state-of-the-art reconstruction techniques.

MLS: Joint Manifold-Learning and Sparsity-Aware Framework for Highly Accelerated Dynamic Magnetic Resonance Imaging

Manifold-based models have been recently exploited for accelerating dynamic magnetic resonance imaging (dMRI). While manifold-based models have shown to be more efficient than conventional low-rank approaches, joint low-rank and sparsity-aware modeling still appears to be very efficient due to the inherent sparsity within dMR images. In this paper, we propose a joint manifold-learning and sparsity-aware framework for dMRI. The proposed method establishes a link between the recently developed manifold models and conventional sparsity-aware models. Dynamic MR images are modeled as points located on or close to a smooth manifold, and a novel data-driven manifold-learning approach, which preserves affine relation among images, is used to learn the low-dimensional embedding of the dynamic images. The temporal basis learnt from such an approach efficiently captures the inherent periodicity of dynamic images and hence sparsity along temporal direction is enforced during reconstruction. The proposed framework is validated by extensive numerical tests on phantom and in-vivo data sets.

Bi-Linear Modeling of Manifold-Data Geometry for Dynamic-MRI Recovery

This paper establishes a modeling framework for data located onto or close to (unknown) smooth manifolds, embedded in Euclidean spaces, and considers its application to dynamic magnetic resonance imaging (dMRI). The framework comprises several modules: First, a set of landmark points is identified to describe concisely a data cloud formed by highly under-sampled dMRI data, and second, low-dimensional renditions of the landmark points are computed. Searching for the linear operator that decompresses low-dimensional data to high-dimensional ones, and for those combinations of landmark points which approximate the manifold data by affine patches, leads to a bi-linear model of the dMRI data, cognizant of the intrinsic data geometry. Preliminary numerical tests on synthetically generated dMRI phantoms, and comparisons with state-of-the-art reconstruction techniques, underline the rich potential of the proposed method for the recovery of highly under-sampled dMRI data.