Deep Residual Learning for Accelerated MRI Using Magnitude and Phase Networks

Analysis of deep complex-valued convolutional neural networks for MRI reconstruction and phase-focused applications

PURPOSE

Deep learning has had success with MRI reconstruction, but previously published works use real-valued networks. The few works which have tried complex-valued networks have not fully assessed their impact on phase. Therefore, the purpose of this work is to fully investigate end-to-end complex-valued convolutional neural networks (CNNs) for accelerated MRI reconstruction and in several phase-based applications in comparison to 2-channel real-valued networks.

METHODS

Several complex-valued activation functions for MRI reconstruction were implemented, and their performance was compared. Complex-valued convolution was implemented and tested on an unrolled network architecture and a U-Net-based architecture over a wide range of network widths and depths with knee, body, and phase-contrast datasets.

RESULTS

Quantitative and qualitative results demonstrated that complex-valued CNNs with complex-valued convolutions provided superior reconstructions compared to real-valued convolutions with the same number of trainable parameters for both an unrolled network architecture and a U-Net-based architecture, and for 3 different datasets. Complex-valued CNNs consistently had superior normalized RMS error, structural similarity index, and peak SNR compared to real-valued CNNs.

CONCLUSION

Complex-valued CNNs can enable superior accelerated MRI reconstruction and phase-based applications such as fat-water separation, and flow quantification compared to real-valued convolutional neural networks.

Fractional deep neural network via constrained optimization

This paper introduces a novel algorithmic framework for a deep neural network (DNN), which in a mathematically rigorous manner, allows us to incorporate history (or memory) into the network—it ensures all layers are connected to one another. This DNN, called Fractional-DNN, can be viewed as a time-discretization of a fractional in time non-linear ordinary differential equation (ODE). The learning problem then is a minimization problem subject to that fractional ODE as constraints. We emphasize that an analogy between the existing DNN and ODEs, with standard time derivative, is well-known by now. The focus of our work is the Fractional-DNN. Using the Lagrangian approach, we provide a derivation of the backward propagation and the design equations. We test our network on several datasets for classification problems. Fractional-DNN offers various advantages over the existing DNN. The key benefits are a significant improvement to the vanishing gradient issue due to the memory effect, and better handling of nonsmooth data due to the network’s ability to approximate non-smooth functions.

Deep model-based magnetic resonance parameter mapping network (DOPAMINE) for fast T1 mapping using variable flip angle method

Quantitative tissue characteristics, which provide valuable diagnostic information, can be represented by magnetic resonance (MR) parameter maps using magnetic resonance imaging (MRI); however, a long scan time is necessary to acquire them, which prevents the application of quantitative MR parameter mapping to real clinical protocols. For fast MR parameter mapping, we propose a deep model-based MR parameter mapping network called DOPAMINE that combines a deep learning network with a model-based method to reconstruct MR parameter maps from undersampled multi-channel k-space data. DOPAMINE consists of two networks: 1) an MR parameter mapping network that uses a deep convolutional neural network (CNN) that estimates initial parameter maps from undersampled k-space data (CNN-based mapping), and 2) a reconstruction network that removes aliasing artifacts in the parameter maps with a deep CNN (CNN-based reconstruction) and an interleaved data consistency layer by an embedded MR model-based optimization procedure. We demonstrated the performance of DOPAMINE in brain T₁ map reconstruction with a variable flip angle (VFA) model. To evaluate the performance of DOPAMINE, we compared it with conventional parallel imaging, low-rank based reconstruction, model-based reconstruction, and state-of-the-art deep-learning-based mapping methods for three different reduction factors (R = 3, 5, and 7) and two different sampling patterns (1D Cartesian and 2D Poisson-disk). Quantitative metrics indicated that DOPAMINE outperformed other methods in reconstructing T₁ maps for all sampling patterns and reduction factors. DOPAMINE exhibited quantitatively and qualitatively superior performance to that of conventional methods in reconstructing MR parameter maps from undersampled multi-channel k-space data. The proposed method can thus reduce the scan time of quantitative MR parameter mapping that uses a VFA model.

On the regularization of feature fusion and mapping for fast MR multi-contrast imaging via iterative networks

Multi-contrast (MC) Magnetic Resonance Imaging (MRI) of the same patient usually requires long scanning times, despite the images sharing redundant information. In this work, we propose a new iterative network that utilizes the sharable information among MC images for MRI acceleration. The proposed network has reinforced data fidelity control and anatomy guidance through an iterative optimization procedure of Gradient Descent, leading to reduced uncertainties and improved reconstruction results. Through a convolutional network, the new method incorporates a learnable regularization unit that is capable of extracting, fusing, and mapping shareable information among different contrasts. Specifically, a dilated inception block is proposed to promote multi-scale feature extractions and increase the receptive field diversity for contextual information incorporation. Lastly, an optimal MC information feeding protocol is built through the design of a complementary feature extractor block. Comprehensive experiments demonstrated the superiority of the proposed network, both qualitatively and quantitatively.

Unpaired Training of Deep Learning tMRA for Flexible Spatio-Temporal Resolution

Time-resolved MR angiography (tMRA) has been widely used for dynamic contrast enhanced MRI (DCE-MRI) due to its highly accelerated acquisition. In tMRA, the periphery of the k -space data are sparsely sampled so that neighbouring frames can be merged to construct one temporal frame. However, this view-sharing scheme fundamentally limits the temporal resolution, and it is not possible to change the view-sharing number to achieve different spatio-temporal resolution trade-offs. Although many deep learning approaches have been recently proposed for MR reconstruction from sparse samples, the existing approaches usually require matched fully sampled k -space reference data for supervised training, which is not suitable for tMRA due to the lack of high spatio-temporal resolution ground-truth images. To address this problem, here we propose a novel unpaired training scheme for deep learning using optimal transport driven cycle-consistent generative adversarial network (cycleGAN). In contrast to the conventional cycleGAN with two pairs of generator and discriminator, the new architecture requires just a single pair of generator and discriminator, which makes the training much simpler but still improves the performance. Reconstruction results using in vivo tMRA and simulation data set confirm that the proposed method can immediately generate high quality reconstruction results at various choices of view-sharing numbers, allowing us to exploit better trade-off between spatial and temporal resolution in time-resolved MR angiography.

Rapid reconstruction of highly undersampled, non-Cartesian real-time cine k-space data using a perceptual complex neural network (PCNN)

Highly accelerated real-time cine MRI using compressed sensing (CS) is a promising approach to achieve high spatio-temporal resolution and clinically acceptable image quality in patients with arrhythmia and/or dyspnea. However, its lengthy image reconstruction time may hinder its clinical translation. The purpose of this study was to develop a neural network for reconstruction of non-Cartesian real-time cine MRI k-space data faster (<1 min per slice with 80 frames) than graphics processing unit (GPU)-accelerated CS reconstruction, without significant loss in image quality or accuracy in left ventricular (LV) functional parameters. We introduce a perceptual complex neural network (PCNN) that trains on complex-valued MRI signal and incorporates a perceptual loss term to suppress incoherent image details. This PCNN was trained and tested with multi-slice, multi-phase, cine images from 40 patients (20 for training, 20 for testing), where the zero-filled images were used as input and the corresponding CS reconstructed images were used as practical ground truth. The resulting images were compared using quantitative metrics (structural similarity index (SSIM) and normalized root mean square error (NRMSE)) and visual scores (conspicuity, temporal fidelity, artifacts, and noise scores), individually graded on a five-point scale (1, worst; 3, acceptable; 5, best), and LV ejection fraction (LVEF). The mean processing time per slice with 80 frames for PCNN was 23.7 ± 1.9 s for pre-processing (Step 1, same as CS) and 0.822 ± 0.004 s for dealiasing (Step 2, 166 times faster than CS). Our PCNN produced higher data fidelity metrics (SSIM = 0.88 ± 0.02, NRMSE = 0.014 ± 0.004) compared with CS. While all the visual scores were significantly different (P < 0.05), the median scores were all 4.0 or higher for both CS and PCNN. LVEFs measured from CS and PCNN were strongly correlated (R² = 0.92) and in good agreement (mean difference = -1.4% [2.3% of mean]; limit of agreement = 10.6% [17.6% of mean]). The proposed PCNN is capable of rapid reconstruction (25 s per slice with 80 frames) of non-Cartesian real-time cine MRI k-space data, without significant loss in image quality or accuracy in LV functional parameters.

Deep learning for brain disorders: from data processing to disease treatment

In order to reach precision medicine and improve patients' quality of life, machine learning is increasingly used in medicine. Brain disorders are often complex and heterogeneous, and several modalities such as demographic, clinical, imaging, genetics and environmental data have been studied to improve their understanding. Deep learning, a subpart of machine learning, provides complex algorithms that can learn from such various data. It has become state of the art in numerous fields, including computer vision and natural language processing, and is also growingly applied in medicine. In this article, we review the use of deep learning for brain disorders. More specifically, we identify the main applications, the concerned disorders and the types of architectures and data used. Finally, we provide guidelines to bridge the gap between research studies and clinical routine.

Deep learning using a biophysical model for robust and accelerated reconstruction of quantitative, artifact-free and denoised R2* images

PURPOSE

To introduce a novel deep learning method for Robust and Accelerated Reconstruction (RoAR) of quantitative and B0-inhomogeneity-corrected R 2 * maps from multi-gradient recalled echo (mGRE) MRI data.

METHODS

RoAR trains a convolutional neural network (CNN) to generate quantitative R 2 ∗ maps free from field inhomogeneity artifacts by adopting a self-supervised learning strategy given (a) mGRE magnitude images, (b) the biophysical model describing mGRE signal decay, and (c) preliminary-evaluated F-function accounting for contribution of macroscopic B0 field inhomogeneities. Importantly, no ground-truth R 2 * images are required and F-function is only needed during RoAR training but not application.

RESULTS

We show that RoAR preserves all features of R 2 * maps while offering significant improvements over existing methods in computation speed (seconds vs. hours) and reduced sensitivity to noise. Even for data with SNR = 5 RoAR produced R 2 * maps with accuracy of 22% while voxel-wise analysis accuracy was 47%. For SNR = 10 the RoAR accuracy increased to 17% vs. 24% for direct voxel-wise analysis.

CONCLUSIONS

RoAR is trained to recognize the macroscopic magnetic field inhomogeneities directly from the input magnitude-only mGRE data and eliminate their effect on R 2 ∗ measurements. RoAR training is based on the biophysical model and does not require ground-truth R 2 * maps. Since RoAR utilizes signal information not just from individual voxels but also accounts for spatial patterns of the signals in the images, it reduces the sensitivity of R 2 * maps to the noise in the data. These features plus high computational speed provide significant benefits for the potential usage of RoAR in clinical settings.

Reconstruction for Diverging-Wave Imaging Using Deep Convolutional Neural Networks

In recent years, diverging wave (DW) ultrasound imaging has become a very promising methodology for cardiovascular imaging due to its high temporal resolution. However, if they are limited in number, DW transmits provide lower image quality compared with classical focused schemes. A conventional reconstruction approach consists in summing series of ultrasound signals coherently, at the expense of frame rate, data volume, and computation time. To deal with this limitation, we propose a convolutional neural network (CNN) architecture, Inception for DW Network (IDNet), for high-quality reconstruction of DW ultrasound images using a small number of transmissions. In order to cope with the specificities induced by the sectorial geometry associated with DW imaging, we adopted the inception model composed of the concatenation of multiscale convolution kernels. Incorporating inception modules aims at capturing different image features with multiscale receptive fields. A mapping between low-quality images and corresponding high-quality compounded reconstruction was learned by training the network using in vitro and in vivo samples. The performance of the proposed approach was evaluated in terms of contrast ratio (CR), contrast-to-noise ratio (CNR), and lateral resolution (LR), and compared with standard compounding method and conventional CNN methods. The results demonstrated that our method could produce high-quality images using only 3 DWs, yielding an image quality equivalent to that obtained with compounding of 31 DWs and outperforming more conventional CNN architectures in terms of complexity, inference time, and image quality.

Self-supervised learning of physics-guided reconstruction neural networks without fully sampled reference data

PURPOSE

To develop a strategy for training a physics-guided MRI reconstruction neural network without a database of fully sampled data sets.

METHODS

Self-supervised learning via data undersampling (SSDU) for physics-guided deep learning reconstruction partitions available measurements into two disjoint sets, one of which is used in the data consistency (DC) units in the unrolled network and the other is used to define the loss for training. The proposed training without fully sampled data is compared with fully supervised training with ground-truth data, as well as conventional compressed-sensing and parallel imaging methods using the publicly available fastMRI knee database. The same physics-guided neural network is used for both proposed SSDU and supervised training. The SSDU training is also applied to prospectively two-fold accelerated high-resolution brain data sets at different acceleration rates, and compared with parallel imaging.

RESULTS

Results on five different knee sequences at an acceleration rate of 4 shows that the proposed self-supervised approach performs closely with supervised learning, while significantly outperforming conventional compressed-sensing and parallel imaging, as characterized by quantitative metrics and a clinical reader study. The results on prospectively subsampled brain data sets, in which supervised learning cannot be used due to lack of ground-truth reference, show that the proposed self-supervised approach successfully performs reconstruction at high acceleration rates (4, 6, and 8). Image readings indicate improved visual reconstruction quality with the proposed approach compared with parallel imaging at acquisition acceleration.

CONCLUSION

The proposed SSDU approach allows training of physics-guided deep learning MRI reconstruction without fully sampled data, while achieving comparable results with supervised deep learning MRI trained on fully sampled data.

Deep Generalization of Structured Low-Rank Algorithms (Deep-SLR)

Structured low-rank (SLR) algorithms, which exploit annihilation relations between the Fourier samples of a signal resulting from different properties, is a powerful image reconstruction framework in several applications. This scheme relies on low-rank matrix completion to estimate the annihilation relations from the measurements. The main challenge with this strategy is the high computational complexity of matrix completion. We introduce a deep learning (DL) approach to significantly reduce the computational complexity. Specifically, we use a convolutional neural network (CNN)-based filterbank that is trained to estimate the annihilation relations from imperfect (under-sampled and noisy) k-space measurements of Magnetic Resonance Imaging (MRI). The main reason for the computational efficiency is the pre-learning of the parameters of the non-linear CNN from exemplar data, compared to SLR schemes that learn the linear filterbank parameters from the dataset itself. Experimental comparisons show that the proposed scheme can enable calibration-less parallel MRI; it can offer performance similar to SLR schemes while reducing the runtime by around three orders of magnitude. Unlike pre-calibrated and self-calibrated approaches, the proposed uncalibrated approach is insensitive to motion errors and affords higher acceleration. The proposed scheme also incorporates image domain priors that are complementary, thus significantly improving the performance over that of SLR schemes.

Deblurring for spiral real-time MRI using convolutional neural networks

PURPOSE

To develop and evaluate a fast and effective method for deblurring spiral real-time MRI (RT-MRI) using convolutional neural networks.

METHODS

We demonstrate a 3-layer residual convolutional neural networks to correct image domain off-resonance artifacts in speech production spiral RT-MRI without the knowledge of field maps. The architecture is motivated by the traditional deblurring approaches. Spatially varying off-resonance blur is synthetically generated by using discrete object approximation and field maps with data augmentation from a large database of 2D human speech production RT-MRI. The effect of off-resonance range, shift-invariance of blur, and readout durations on deblurring performance are investigated. The proposed method is validated using synthetic and real data with longer readouts, quantitatively using image quality metrics and qualitatively via visual inspection, and with a comparison to conventional deblurring methods.

RESULTS

Deblurring performance was found superior to a current autocalibrated method for in vivo data and only slightly worse than an ideal reconstruction with perfect knowledge of the field map for synthetic test data. Convolutional neural networks deblurring made it possible to visualize articulator boundaries with readouts up to 8 ms at 1.5 T, which is 3-fold longer than the current standard practice. The computation time was 12.3 ± 2.2 ms per frame, enabling low-latency processing for RT-MRI applications.

CONCLUSION

Convolutional neural networks deblurring is a practical, efficient, and field map-free approach for the deblurring of spiral RT-MRI. In the context of speech production imaging, this can enable 1.7-fold improvement in scan efficiency and the use of spiral readouts at higher field strengths such as 3 T.

Convolutional neural network for discriminating nasopharyngeal carcinoma and benign hyperplasia on MRI

OBJECTIVES

A convolutional neural network (CNN) was adapted to automatically detect early-stage nasopharyngeal carcinoma (NPC) and discriminate it from benign hyperplasia on a non-contrast-enhanced MRI sequence for potential use in NPC screening programs.

METHODS

We retrospectively analyzed 412 patients who underwent T2-weighted MRI, 203 of whom had biopsy-proven primary NPC confined to the nasopharynx (stage T1) and 209 had benign hyperplasia without NPC. Thirteen patients were sampled randomly to monitor the training process. We applied the Residual Attention Network architecture, adapted for three-dimensional MR images, and incorporated a slice-attention mechanism, to produce a CNN score of 0-1 for NPC probability. Threefold cross-validation was performed in 399 patients. CNN scores between the NPC and benign hyperplasia groups were compared using Student's t test. Receiver operating characteristic with the area under the curve (AUC) was performed to identify the optimal CNN score threshold.

RESULTS

In each fold, significant differences were observed in the CNN scores between the NPC and benign hyperplasia groups (p < .01). The AUCs ranged from 0.95 to 0.97 with no significant differences between the folds (p = .35 to .92). The combined AUC from all three folds (n = 399) was 0.96, with an optimal CNN score threshold of > 0.71, producing a sensitivity, specificity, and accuracy of 92.4%, 90.6%, and 91.5%, respectively, for NPC detection.

CONCLUSION

Our CNN method applied to T2-weighted MRI could discriminate between malignant and benign tissues in the nasopharynx, suggesting that it as a promising approach for the automated detection of early-stage NPC.

KEY POINTS

• The convolutional neural network (CNN)-based algorithm could automatically discriminate between malignant and benign diseases using T2-weighted fat-suppressed MR images. • The CNN-based algorithm had an accuracy of 91.5% with an area under the receiver operator characteristic curve of 0.96 for discriminating early-stage T1 nasopharyngeal carcinoma from benign hyperplasia. • The CNN-based algorithm had a sensitivity of 92.4% and specificity of 90.6% for detecting early-stage nasopharyngeal carcinoma.

Multiple Slice k-space Deep Learning for Magnetic Resonance Imaging Reconstruction

Magnetic resonance imaging (MRI) has been one of the most powerful and valuable imaging methods for medical diagnosis and staging of disease. Due to the long scan time of MRI acquisition, k-space under-samplings is required during the acquisition processing. Thus, MRI reconstruction, which transfers undersampled k-space data to high-quality magnetic resonance imaging, becomes an important and meaningful task. There have been many explorations on k-space interpolation for MRI reconstruction. However, most of these methods ignore the strong correlation between target slice and its adjacent slices. Inspired by this, we propose a fully data-driven deep learning algorithm for k-space interpolation, utilizing the correlation information between the target slice and its neighboring slices. A novel network is proposed, which models the inter-dependencies between different slices. In addition, the network is easily implemented and expended. Experiments show that our methods consistently surpass existing image-domain and k-space-domain magnetic resonance imaging reconstructing methods.

Dense Recurrent Neural Networks for Accelerated MRI: History-Cognizant Unrolling of Optimization Algorithms

Inverse problems for accelerated MRI typically incorporate domain-specific knowledge about the forward encoding operator in a regularized reconstruction framework. Recently physics-driven deep learning (DL) methods have been proposed to use neural networks for data-driven regularization. These methods unroll iterative optimization algorithms to solve the inverse problem objective function, by alternating between domain-specific data consistency and data-driven regularization via neural networks. The whole unrolled network is then trained end-to-end to learn the parameters of the network. Due to simplicity of data consistency updates with gradient descent steps, proximal gradient descent (PGD) is a common approach to unroll physics-driven DL reconstruction methods. However, PGD methods have slow convergence rates, necessitating a higher number of unrolled iterations, leading to memory issues in training and slower reconstruction times in testing. Inspired by efficient variants of PGD methods that use a history of the previous iterates, we propose a history-cognizant unrolling of the optimization algorithm with dense connections across iterations for improved performance. In our approach, the gradient descent steps are calculated at a trainable combination of the outputs of all the previous regularization units. We also apply this idea to unrolling variable splitting methods with quadratic relaxation. Our results in reconstruction of the fastMRI knee dataset show that the proposed history-cognizant approach reduces residual aliasing artifacts compared to its conventional unrolled counterpart without requiring extra computational power or increasing reconstruction time.

Transfer learning in deep neural network based under-sampled MR image reconstruction

In Magnetic Resonance Imaging (MRI), the success of deep learning-based under-sampled MR image reconstruction depends on: (i) size of the training dataset, (ii) generalization capabilities of the trained neural network. Whenever there is a mismatch between the training and testing data, there is a need to retrain the neural network from scratch with thousands of MR images obtained using the same protocol. This may not be possible in MRI as it is costly and time consuming to acquire data. In this research, a transfer learning approach i.e. end-to-end fine tuning is proposed for U-Net to address the data scarcity and generalization problems of deep learning-based MR image reconstruction. First the generalization capabilities of a pre-trained U-Net (initially trained on the human brain images of 1.5 T scanner) are assessed for: (a) MR images acquired from MRI scanners of different magnetic field strengths, (b) MR images of different anatomies and (c) MR images under-sampled by different acceleration factors. Later, end-to-end fine tuning of the pre-trained U-Net is proposed for the reconstruction of the above-mentioned MR images (i.e. (a), (b) and (c)). The results show successful reconstructions obtained from the proposed method as reflected by the Structural SIMilarity index, Root Mean Square Error, Peak Signal-to-Noise Ratio and central line profile of the reconstructed images.

Neural Architecture Search for compressed sensing Magnetic Resonance image reconstruction

Recent works have demonstrated that deep learning (DL) based compressed sensing (CS) implementation can accelerate Magnetic Resonance (MR) Imaging by reconstructing MR images from sub-sampled k-space data. However, network architectures adopted in previous methods are all designed by handcraft. Neural Architecture Search (NAS) algorithms can automatically build neural network architectures which have outperformed human designed ones in several vision tasks. Inspired by this, here we proposed a novel and efficient network for the MR image reconstruction problem via NAS instead of manual attempts. Particularly, a specific cell structure, which was integrated into the model-driven MR reconstruction pipeline, was automatically searched from a flexible pre-defined operation search space in a differentiable manner. Experimental results show that our searched network can produce better reconstruction results compared to previous state-of-the-art methods in terms of PSNR and SSIM with 4∼6 times fewer computation resources. Extensive experiments were conducted to analyze how hyper-parameters affect reconstruction performance and the searched structures. The generalizability of the searched architecture was also evaluated on different organ MR datasets. Our proposed method can reach a better trade-off between computation cost and reconstruction performance for MR reconstruction problem with good generalizability and offer insights to design neural networks for other medical image applications. The evaluation code will be available at https://github.com/yjump/NAS-for-CSMRI.

Deep-learning-based image quality enhancement of compressed sensing magnetic resonance imaging of vessel wall: comparison of self-supervised and unsupervised approaches

While high-resolution proton density-weighted magnetic resonance imaging (MRI) of intracranial vessel walls is significant for a precise diagnosis of intracranial artery disease, its long acquisition time is a clinical burden. Compressed sensing MRI is a prospective technology with acceleration factors that could potentially reduce the scan time. However, high acceleration factors result in degraded image quality. Although recent advances in deep-learning-based image restoration algorithms can alleviate this problem, clinical image pairs used in deep learning training typically do not align pixel-wise. Therefore, in this study, two different deep-learning-based denoising algorithms-self-supervised learning and unsupervised learning-are proposed; these algorithms are applicable to clinical datasets that are not aligned pixel-wise. The two approaches are compared quantitatively and qualitatively. Both methods produced promising results in terms of image denoising and visual grading. While the image noise and signal-to-noise ratio of self-supervised learning were superior to those of unsupervised learning, unsupervised learning was preferable over self-supervised learning in terms of radiomic feature reproducibility.

CINENet: deep learning-based 3D cardiac CINE MRI reconstruction with multi-coil complex-valued 4D spatio-temporal convolutions

Cardiac CINE magnetic resonance imaging is the gold-standard for the assessment of cardiac function. Imaging accelerations have shown to enable 3D CINE with left ventricular (LV) coverage in a single breath-hold. However, 3D imaging remains limited to anisotropic resolution and long reconstruction times. Recently deep learning has shown promising results for computationally efficient reconstructions of highly accelerated 2D CINE imaging. In this work, we propose a novel 4D (3D + time) deep learning-based reconstruction network, termed 4D CINENet, for prospectively undersampled 3D Cartesian CINE imaging. CINENet is based on (3 + 1)D complex-valued spatio-temporal convolutions and multi-coil data processing. We trained and evaluated the proposed CINENet on in-house acquired 3D CINE data of 20 healthy subjects and 15 patients with suspected cardiovascular disease. The proposed CINENet network outperforms iterative reconstructions in visual image quality and contrast (+ 67% improvement). We found good agreement in LV function (bias ± 95% confidence) in terms of end-systolic volume (0 ± 3.3 ml), end-diastolic volume (− 0.4 ± 2.0 ml) and ejection fraction (0.1 ± 3.2%) compared to clinical gold-standard 2D CINE, enabling single breath-hold isotropic 3D CINE in less than 10 s scan and ~ 5 s reconstruction time.

Adaptive and Compressive Beamforming Using Deep Learning for Medical Ultrasound

In ultrasound (US) imaging, various types of adaptive beamforming techniques have been investigated to improve the resolution and the contrast-to-noise ratio of the delay and sum (DAS) beamformers. Unfortunately, the performance of these adaptive beamforming approaches degrades when the underlying model is not sufficiently accurate and the number of channels decreases. To address this problem, here, we propose a deep-learning-based beamformer to generate significantly improved images over widely varying measurement conditions and channel subsampling patterns. In particular, our deep neural network is designed to directly process full or subsampled radio frequency (RF) data acquired at various subsampling rates and detector configurations so that it can generate high-quality US images using a single beamformer. The origin of such input-dependent adaptivity is also theoretically analyzed. Experimental results using the B-mode focused US confirm the efficacy of the proposed methods.

Artificial Intelligence in Neuroradiology: Current Status and Future Directions

Fueled by new techniques, computational tools, and broader availability of imaging data, artificial intelligence has the potential to transform the practice of neuroradiology. The recent exponential increase in publications related to artificial intelligence and the central focus on artificial intelligence at recent professional and scientific radiology meetings underscores the importance. There is growing momentum behind leveraging artificial intelligence techniques to improve workflow and diagnosis and treatment and to enhance the value of quantitative imaging techniques. This article explores the reasons why neuroradiologists should care about the investments in new artificial intelligence applications, highlights current activities and the roles neuroradiologists are playing, and renders a few predictions regarding the near future of artificial intelligence in neuroradiology.

Deep complex convolutional network for fast reconstruction of 3D late gadolinium enhancement cardiac MRI

Several deep-learning models have been proposed to shorten MRI scan time. Prior deep-learning models that utilize real-valued kernels have limited capability to learn rich representations of complex MRI data. In this work, we utilize a complex-valued convolutional network (ℂNet) for fast reconstruction of highly under-sampled MRI data and evaluate its ability to rapidly reconstruct 3D late gadolinium enhancement (LGE) data. ℂNet preserves the complex nature and optimal combination of real and imaginary components of MRI data throughout the reconstruction process by utilizing complex-valued convolution, novel radial batch normalization, and complex activation function layers in a U-Net architecture. A prospectively under-sampled 3D LGE cardiac MRI dataset of 219 patients (17 003 images) at acceleration rates R = 3 through R = 5 was used to evaluate ℂNet. The dataset was further retrospectively under-sampled to a maximum of R = 8 to simulate higher acceleration rates. We created three reconstructions of the 3D LGE dataset using (1) ℂNet, (2) a compressed-sensing-based low-dimensional-structure self-learning and thresholding algorithm (LOST), and (3) a real-valued U-Net (realNet) with the same number of parameters as ℂNet. LOST-reconstructed data were considered the reference for training and evaluation of all models. The reconstructed images were quantitatively evaluated using mean-squared error (MSE) and the structural similarity index measure (SSIM), and subjectively evaluated by three independent readers. Quantitatively, ℂNet-reconstructed images had significantly improved MSE and SSIM values compared with realNet (MSE, 0.077 versus 0.091; SSIM, 0.876 versus 0.733, respectively; p < 0.01). Subjective quality assessment showed that ℂNet-reconstructed image quality was similar to that of compressed sensing and significantly better than that of realNet. ℂNet reconstruction was also more than 300 times faster than compressed sensing. Retrospective under-sampled images demonstrate the potential of ℂNet at higher acceleration rates. ℂNet enables fast reconstruction of highly accelerated 3D MRI with superior performance to real-valued networks, and achieves faster reconstruction than compressed sensing.

High-Fidelity Accelerated MRI Reconstruction by Scan-Specific Fine-Tuning of Physics-Based Neural Networks

Long scan duration remains a challenge for high-resolution MRI. Deep learning has emerged as a powerful means for accelerated MRI reconstruction by providing data-driven regularizers that are directly learned from data. These data-driven priors typically remain unchanged for future data in the testing phase once they are learned during training. In this study, we propose to use a transfer learning approach to fine-tune these regularizers for new subjects using a self-supervision approach. While the proposed approach can compromise the extremely fast reconstruction time of deep learning MRI methods, our results on knee MRI indicate that such adaptation can substantially reduce the remaining artifacts in reconstructed images. In addition, the proposed approach has the potential to reduce the risks of generalization to rare pathological conditions, which may be unavailable in the training data.

Artificial Intelligence in Head and Neck Imaging: A Glimpse into the Future

Artificial intelligence, specifically machine learning and deep learning, is a rapidly developing field in imaging sciences with the potential to improve the efficiency and effectiveness of radiologists. This review covers common technical terms and basic concepts in imaging artificial intelligence and briefly reviews the application of these techniques to general imaging as well as head and neck imaging. Artificial intelligence has the potential to contribute improvements to all areas of patient care, including image acquisition, processing, segmentation, automated detection of findings, integration of clinical information, quality improvement, and research. Numerous challenges remain, however, before widespread imaging clinical adoption and integration occur.

Rising role of artificial intelligence in image reconstruction for biomedical imaging

In this editorial, we review recent progress on the applications of artificial intelligence (AI) in image reconstruction for biomedical imaging. Because it abandons prior information of traditional artificial design and adopts a completely data-driven mode to obtain deeper prior information via learning, AI technology plays an increasingly important role in biomedical image reconstruction. The combination of AI technology and the biomedical image reconstruction method has become a hotspot in the field. Favoring AI, the performance of biomedical image reconstruction has been improved in terms of accuracy, resolution, imaging speed, etc. We specifically focus on how to use AI technology to improve the performance of biomedical image reconstruction, and propose possible future directions in this field.

High-dimensional embedding network derived prior for compressive sensing MRI reconstruction

Although recent deep learning methodology has shown promising performance in fast imaging, the network needs to be retrained for specific sampling patterns and ratios. Therefore, how to explore the network as a general prior and leverage it into the observation constraint flexibly is urgent. In this work, we present a multi-channel enhanced Deep Mean-Shift Prior (MEDMSP) to address the highly under-sampled magnetic resonance imaging reconstruction problem. By extending the naive DMSP via integration of multi-model aggregation and multi-channel network learning, a high-dimensional embedding network derived prior is formed. Then, we apply the learned prior to single-channel image reconstruction via variable augmentation technique. The resulting model is tackled by proximal gradient descent and alternative iteration. Experimental results under various sampling trajectories and acceleration factors consistently demonstrated the superiority of the proposed prior.

Accelerated coronary MRI with sRAKI: A database-free self-consistent neural network k-space reconstruction for arbitrary undersampling

Purpose

To accelerate coronary MRI acquisitions with arbitrary undersampling patterns by using a novel reconstruction algorithm that applies coil self-consistency using subject-specific neural networks.

Methods

Self-consistent robust artificial-neural-networks for k-space interpolation (sRAKI) performs iterative parallel imaging reconstruction by enforcing self-consistency among coils. The approach bears similarity to SPIRiT, but extends the linear convolutions in SPIRiT to nonlinear interpolation using convolutional neural networks (CNNs). These CNNs are trained individually for each scan using the scan-specific autocalibrating signal (ACS) data. Reconstruction is performed by imposing the learned self-consistency and data-consistency, which enables sRAKI to support random undersampling patterns. Fully-sampled targeted right coronary artery MRI was acquired in six healthy subjects. The data were retrospectively undersampled, and reconstructed using SPIRiT, l₁-SPIRiT and sRAKI for acceleration rates of 2 to 5. Additionally, prospectively undersampled whole-heart coronary MRI was acquired to further evaluate reconstruction performance.

Results

sRAKI reduces noise amplification and blurring artifacts compared with SPIRiT and l₁-SPIRiT, especially at high acceleration rates in targeted coronary MRI. Quantitative analysis shows that sRAKI outperforms these techniques in terms of normalized mean-squared-error (~44% and ~21% over SPIRiT and l1-SPIRiT at rate 5) and vessel sharpness (~10% and ~20% over SPIRiT and l₁-SPIRiT at rate 5). Whole-heart data shows the sharpest coronary arteries when resolved using sRAKI, with 11% and 15% improvement in vessel sharpness over SPIRiT and l₁-SPIRiT, respectively.

Conclusion

sRAKI is a database-free neural network-based reconstruction technique that may further accelerate coronary MRI with arbitrary undersampling patterns, while improving noise resilience over linear parallel imaging and image sharpness over l₁ regularization techniques.

k -Space Deep Learning for Accelerated MRI

The annihilating filter-based low-rank Hankel matrix approach (ALOHA) is one of the state-of-the-art compressed sensing approaches that directly interpolates the missing k -space data using low-rank Hankel matrix completion. The success of ALOHA is due to the concise signal representation in the k -space domain, thanks to the duality between structured low-rankness in the k -space domain and the image domain sparsity. Inspired by the recent mathematical discovery that links convolutional neural networks to Hankel matrix decomposition using data-driven framelet basis, here we propose a fully data-driven deep learning algorithm for k -space interpolation. Our network can be also easily applied to non-Cartesian k -space trajectories by simply adding an additional regridding layer. Extensive numerical experiments show that the proposed deep learning method consistently outperforms the existing image-domain deep learning approaches.

Deep Magnetic Resonance Image Reconstruction: Inverse Problems Meet Neural Networks

Image reconstruction from undersampled k-space data has been playing an important role in fast MRI. Recently, deep learning has demonstrated tremendous success in various fields and has also shown potential in significantly accelerating MRI reconstruction with fewer measurements. This article provides an overview of the deep learning-based image reconstruction methods for MRI. Two types of deep learning-based approaches are reviewed: those based on unrolled algorithms and those which are not. The main structure of both approaches are explained, respectively. Several signal processing issues for maximizing the potential of deep reconstruction in fast MRI are discussed. The discussion may facilitate further development of the networks and the analysis of performance from a theoretical point of view.

Image Reconstruction: From Sparsity to Data-adaptive Methods and Machine Learning

The field of medical image reconstruction has seen roughly four types of methods. The first type tended to be analytical methods, such as filtered back-projection (FBP) for X-ray computed tomography (CT) and the inverse Fourier transform for magnetic resonance imaging (MRI), based on simple mathematical models for the imaging systems. These methods are typically fast, but have suboptimal properties such as poor resolution-noise trade-off for CT. A second type is iterative reconstruction methods based on more complete models for the imaging system physics and, where appropriate, models for the sensor statistics. These iterative methods improved image quality by reducing noise and artifacts. The FDA-approved methods among these have been based on relatively simple regularization models. A third type of methods has been designed to accommodate modified data acquisition methods, such as reduced sampling in MRI and CT to reduce scan time or radiation dose. These methods typically involve mathematical image models involving assumptions such as sparsity or low-rank. A fourth type of methods replaces mathematically designed models of signals and systems with data-driven or adaptive models inspired by the field of machine learning. This paper focuses on the two most recent trends in medical image reconstruction: methods based on sparsity or low-rank models, and data-driven methods based on machine learning techniques.

Utilizing wavelet deep learning network to classify different states of task-fMRI for verifying activation regions

Purpose: We propose a convolutional neural network (CNN) based on wavelet for verifying the activation regions decided with statistical analysis. Because the functional magnetic resonance imaging (fMRI) data contains lots of noises, it is difficult to get the data of blood-oxygen-level dependent (BOLD) signal directly for intervention testing like animal studies. So it is difficult to effectively verify these activation regions. Based on the rapid development of deep learning technology. Materials and methods: We select the task fMRI data of presenting food and nonfood pictures to volunteer subjects from open public data, whose website is https://www.openfmri.org/dataset/ds000157/. Firstly, the brain activation regions are obtained by utilizing the method of statistical analysis. Then the spatial coordinates are acquired from the activation regions by checking the atlas table. The P-value of the activation regions are less 0.05. The activation regions are the most responsive to perceive the differences of BOLD in the brain between the two states, presenting food and nonfood pictures. We select the part task fMRI data of from the activation regions, for preparing the training and validation samples. Then we design a deep leaning network based on wavelet to classify the task fMRI data between food and nonfood.Results and conclusions: The classification accuracy is 80.23%. However, when we select the spatial coordinates of other inactivation regions, the classification accuracy is only 60%. The differences of classification accuracy between the activation regions and the inactivation regions prove that the activation regions selected with statistical analysis method are accurate and effective. The two methods of deep learning and statistical analysis can be cross-validated for the study of human being brain.

Optimized fast GPU implementation of robust artificial-neural-networks for k-space interpolation (RAKI) reconstruction

BACKGROUND

Robust Artificial-neural-networks for k-space Interpolation (RAKI) is a recently proposed deep-learning-based reconstruction algorithm for parallel imaging. Its main premise is to perform k-space interpolation using convolutional neural networks (CNNs) trained on subject-specific autocalibration signal (ACS) data. Since training is performed individually for each subject, the reconstruction time is longer than approaches that pre-train on databases. In this study, we sought to reduce the computational time of RAKI.

METHODS

RAKI was implemented using CPU multi-processing and process pooling to maximize the utility of GPU resources. We also proposed an alternative CNN architecture that interpolates all output channels jointly for specific skipped k-space lines. This new architecture was compared to the original CNN architecture in RAKI, as well as to GRAPPA in phantom, brain and knee MRI datasets, both qualitatively and quantitatively.

RESULTS

The optimized GPU implementations were approximately 2-to-5-fold faster than a simple GPU implementation. The new CNN architecture further improved the computational time by 4-to-5-fold compared to the optimized GPU implementation using the original RAKI CNN architecture. It also provided significant improvement over GRAPPA both visually and quantitatively, although it performed slightly worse than the original RAKI CNN architecture.

CONCLUSIONS

The proposed implementations of RAKI bring the computational time towards clinically acceptable ranges. The new CNN architecture yields faster training, albeit at a slight performance loss, which may be acceptable for faster visualization in some settings.

Applications of Deep Learning to Neuro-Imaging Techniques

Many clinical applications based on deep learning and pertaining to radiology have been proposed and studied in radiology for classification, risk assessment, segmentation tasks, diagnosis, prognosis, and even prediction of therapy responses. There are many other innovative applications of AI in various technical aspects of medical imaging, particularly applied to the acquisition of images, ranging from removing image artifacts, normalizing/harmonizing images, improving image quality, lowering radiation and contrast dose, and shortening the duration of imaging studies. This article will address this topic and will seek to present an overview of deep learning applied to neuroimaging techniques.

Undersampled MR image reconstruction using an enhanced recursive residual network

When using aggressive undersampling, it is difficult to recover the high quality image with reliably fine features. In this paper, we propose an enhanced recursive residual network (ERRN) that improves the basic recursive residual network with a high-frequency feature guidance, an error-correction unit and dense connections. The feature guidance is designed to predict the underlying anatomy based on image a priori learned from the label data, playing a complementary role to the residual learning. The ERRN is adapted for two important applications: compressed sensing (CS) MRI and super resolution (SR) MRI, while an application-specific error-correction unit is added into the framework, i.e. data consistency for CS-MRI and back projection for SR-MRI due to their different sampling schemes. Our proposed network was evaluated using a real-valued brain dataset, a complex-valued knee dataset, pathological brain data and in vivo rat brain data with different undersampling masks and rates. Experimental results demonstrated that ERRN presented superior reconstructions at all cases with distinctly restored structural features and highest image quality metrics compared to both the state-of-the-art convolutional neural networks and the conventional optimization-based methods, particularly for the undersampling rate over 5-fold. Thus, an excellent framework design can endow the network with a flexible architecture, fewer parameters, outstanding performances for various undersampling schemes, and reduced overfitting in generalization, which will facilitate real-time reconstruction on MRI scanners.

ACCELERATED CORONARY MRI USING 3D SPIRIT-RAKI WITH SPARSITY REGULARIZATION

Coronary MRI is a non-invasive radiation-free imaging tool for the diagnosis of coronary artery disease. One of its limitations is the long scan time, due to the need for high resolution imaging in the presence of respiratory and cardiac motions. Machine learning (ML) methods have been recently utilized to accelerate MRI. In particular, a scan-specific ML technique, called Robust Artifical-neural-network for k-space Interpolation (RAKI) has shown promise in cardiac MRI. However, it requires uniform undersampling. In this study, we sought to extend this approach to arbitrary sampling patterns, using coil self-consistency. This technique, called SPIRiT-RAKI, utilizes scan-specific convolutional neural networks to nonlinearly enforce coil self-consistency. Additionally, regularization terms can also be incorporated. SPIRiT-RAKI was used to accelerate right coronary MRI. Reconstructions were compared to SPIRiT for different undersampling patterns and acceleration rates. Results show SPIRiT-RAKI reduces residual aliasing and blurring artifacts compared to SPIRiT.

Conditional generative adversarial network for 3D rigid-body motion correction in MRI

PURPOSE

Subject motion in MRI remains an unsolved problem; motion during image acquisition may cause blurring and artifacts that severely degrade image quality. In this work, we approach motion correction as an image-to-image translation problem, which refers to the approach of training a deep neural network to predict an image in 1 domain from an image in another domain. Specifically, the purpose of this work was to develop and train a conditional generative adversarial network to predict artifact-free brain images from motion-corrupted data.

METHODS

An open source MRI data set comprising T₂ *-weighted, FLASH magnitude, and phase brain images for 53 patients was used to generate complex image data for motion simulation. To simulate rigid motion, rotations and translations were applied to the image data based on randomly generated motion profiles. A conditional generative adversarial network, comprising a generator and discriminator networks, was trained using the motion-corrupted and corresponding ground truth (original) images as training pairs.

RESULTS

The images predicted by the conditional generative adversarial network have improved image quality compared to the motion-corrupted images. The mean absolute error between the motion-corrupted and ground-truth images of the test set was 16.4% of the image mean value, whereas the mean absolute error between the conditional generative adversarial network-predicted and ground-truth images was 10.8% The network output also demonstrated improved peak SNR and structural similarity index for all test-set images.

CONCLUSION

The images predicted by the conditional generative adversarial network have quantitatively and qualitatively improved image quality compared to the motion-corrupted images.

Accelerated Simultaneous Multi-Slice MRI using Subject-Specific Convolutional Neural Networks

Simultaneous multi-slice or multi-band (SMS/MB) imaging allows accelerated coverage in magnetic resonance imaging (MRI). Multiple slices are excited and acquired at the same time, and reconstructed using the redundancies in receiver coil arrays, similar to parallel imaging. SMS/MB reconstruction is currently performed with linear reconstruction techniques. Recently, a nonlinear reconstruction method for parallel imaging, Robust Artificial-neural-networks for k-space Interpolation (RAKI) was proposed and shown to improve upon linear methods. This method uses convolutional neural networks (CNN) trained solely on subject-specific calibration data. In this study, we sought to extend RAKI to SMS/MB imaging reconstruction. CNN training was performed on calibration data acquired prior to SMS/MB imaging, in a manner consistent with the existing linear methods. These CNNs were used to reconstruct a time series of functional MRI (fMRI) data. CNN network parameters were optimized using an extensive search of the parameter space. With these optimal parameters, RAKI substantially improves image quality compared to a commonly used linear reconstruction algorithm, especially for high acceleration rates.