ACCELERATING MAGNETIC RESONANCE IMAGING VIA DEEP LEARNING

High-Fidelity Accelerated MRI Reconstruction by Scan-Specific Fine-Tuning of Physics-Based Neural Networks

Long scan duration remains a challenge for high-resolution MRI. Deep learning has emerged as a powerful means for accelerated MRI reconstruction by providing data-driven regularizers that are directly learned from data. These data-driven priors typically remain unchanged for future data in the testing phase once they are learned during training. In this study, we propose to use a transfer learning approach to fine-tune these regularizers for new subjects using a self-supervision approach. While the proposed approach can compromise the extremely fast reconstruction time of deep learning MRI methods, our results on knee MRI indicate that such adaptation can substantially reduce the remaining artifacts in reconstructed images. In addition, the proposed approach has the potential to reduce the risks of generalization to rare pathological conditions, which may be unavailable in the training data.

Fidelity imposed network edit (FINE) for solving ill-posed image reconstruction

Deep learning (DL) is increasingly used to solve ill-posed inverse problems in medical imaging, such as reconstruction from noisy and/or incomplete data, as DL offers advantages over conventional methods that rely on explicit image features and hand engineered priors. However, supervised DL-based methods may achieve poor performance when the test data deviates from the training data, for example, when it has pathologies not encountered in the training data. Furthermore, DL-based image reconstructions do not always incorporate the underlying forward physical model, which may improve performance. Therefore, in this work we introduce a novel approach, called fidelity imposed network edit (FINE), which modifies the weights of a pre-trained reconstruction network for each case in the testing dataset. This is achieved by minimizing an unsupervised fidelity loss function that is based on the forward physical model. FINE is applied to two important inverse problems in neuroimaging: quantitative susceptibility mapping (QSM) and under-sampled image reconstruction in MRI. Our experiments demonstrate that FINE can improve reconstruction accuracy.

Artificial intelligence in diagnostic imaging: impact on the radiography profession

The arrival of artificially intelligent systems into the domain of medical imaging has focused attention and sparked much debate on the role and responsibilities of the radiologist. However, discussion about the impact of such technology on the radiographer role is lacking. This paper discusses the potential impact of artificial intelligence (AI) on the radiography profession by assessing current workflow and cross-mapping potential areas of AI automation such as procedure planning, image acquisition and processing. We also highlight the opportunities that AI brings including enhancing patient-facing care, increased cross-modality education and working, increased technological expertise and expansion of radiographer responsibility into AI-supported image reporting and auditing roles.

Multi-scale Unrolled Deep Learning Framework for Accelerated Magnetic Resonance Imaging

Accelerating data acquisition in magnetic resonance imaging (MRI) has been of perennial interest due to its prohibitively slow data acquisition process. Recent trends in accelerating MRI employ data-centric deep learning frameworks due to its fast inference time and 'one-parameter-fit-all' principle unlike in traditional model-based acceleration techniques. Unrolled deep learning framework that combines the deep priors and model knowledge are robust compared to naive deep learning based framework. In this paper, we propose a novel multi-scale unrolled deep learning framework which learns deep image priors through multi-scale CNN and is combined with unrolled framework to enforce data-consistency and model knowledge. Essentially, this framework combines the best of both learning paradigms:model-based and data-centric learning paradigms. Proposed method is verified using several experiments on numerous data sets.

Compressed-Sensing Magnetic Resonance Image Reconstruction Using an Iterative Convolutional Neural Network Approach

Convolutional neural networks (CNNs) demonstrate excellent performance when employed to reconstruct the images obtained by compressed-sensing magnetic resonance imaging (CS-MRI). Our study aimed to enhance image quality by developing a novel iterative reconstruction approach that utilizes image-based CNNs and k-space correction to preserve original k-space data. In the proposed method, CNNs represent a priori information concerning image spaces. First, the CNNs are trained to map zero-filling images onto corresponding full-sampled images. Then, they recover the zero-filled part of the k-space data. Subsequently, k-space corrections, which involve the replacement of unfilled regions by original k-space data, are implemented to preserve the original k-space data. The above-mentioned processes are used iteratively. The performance of the proposed method was validated using a T2-weighted brain-image dataset, and experiments were conducted with several sampling masks. Finally, the proposed method was compared with other noniterative approaches to demonstrate its effectiveness. The aliasing artifacts in the reconstructed images obtained using the proposed approach were reduced compared to those using other state-of-the-art techniques. In addition, the quantitative results obtained in the form of the peak signal-to-noise ratio and structural similarity index demonstrated the effectiveness of the proposed method. The proposed CS-MRI method enhanced MR image quality with high-throughput examinations.

Emerging Advanced Technologies Developed by IPR for Bio Medical Applications ‑.A Review

Over the last decade, research has intensified worldwide on the use of low-temperature plasmas in medicine and healthcare. Researchers have discovered many methods of applying plasmas to living tissues to deactivate pathogens; to end the flow of blood without damaging healthy tissue; to sanitize wounds and accelerate its healing; and to selectively kill malignant cancer cells. This review paper presents the latest development of advanced and plasma-based technologies used for applications in neurology in particular. Institute for Plasma Research (IPR), an aided institute of the Department of Atomic Energy (DAE), has also developed various technologies in some of these areas. One of these is an Atmospheric Pressure Plasma Jet (APPJ). This device is being studied to treat skin diseases, for coagulation of blood at faster rates and its interaction with oral, lung, and brain cancer cells. In certain cases, in-vitro studies have yielded encouraging results and limited in-vivo studies have been initiated. Plasma activated water has been produced in the laboratory for microbial disinfection, with potential applications in the health sector. Recently, plasmonic nanoparticle arrays which allow detection of very low concentrations of chemicals is studied in detail to allow early-stage detection of diseases. IPR has also been developing AI-based software called DeepCXR and AIBacilli for automated, high-speed screening and detection of footprints of tuberculosis (TB) in Chest X-ray images and for recognizing single/multiple TB bacilli in sputum smear test images, respectively. Deep Learning systems are increasingly being used around the world for analyzing electroencephalogram (EEG) signals for emotion recognition, mental workload, and seizure detection.

Bi-Linear Modeling of Data Manifolds for Dynamic-MRI Recovery

This paper puts forth a novel bi-linear modeling framework for data recovery via manifold-learning and sparse-approximation arguments and considers its application to dynamic magnetic-resonance imaging (dMRI). Each temporal-domain MR image is viewed as a point that lies onto or close to a smooth manifold, and landmark points are identified to describe the point cloud concisely. To facilitate computations, a dimensionality reduction module generates low-dimensional/compressed renditions of the landmark points. Recovery of high-fidelity MRI data is realized by solving a non-convex minimization task for the linear decompression operator and affine combinations of landmark points which locally approximate the latent manifold geometry. An algorithm with guaranteed convergence to stationary solutions of the non-convex minimization task is also provided. The aforementioned framework exploits the underlying spatio-temporal patterns and geometry of the acquired data without any prior training on external data or information. Extensive numerical results on simulated as well as real cardiac-cine MRI data illustrate noteworthy improvements of the advocated machine-learning framework over state-of-the-art reconstruction techniques.

Spatio-Temporal Deep Learning-Based Undersampling Artefact Reduction for 2D Radial Cine MRI With Limited Training Data

In this work we reduce undersampling artefacts in two-dimensional (2D) golden-angle radial cine cardiac MRI by applying a modified version of the U-net. The network is trained on 2D spatio-temporal slices which are previously extracted from the image sequences. We compare our approach to two 2D and a 3D deep learning-based post processing methods, three iterative reconstruction methods and two recently proposed methods for dynamic cardiac MRI based on 2D and 3D cascaded networks. Our method outperforms the 2D spatially trained U-net and the 2D spatio-temporal U-net. Compared to the 3D spatio-temporal U-net, our method delivers comparable results, but requiring shorter training times and less training data. Compared to the compressed sensing-based methods kt-FOCUSS and a total variation regularized reconstruction approach, our method improves image quality with respect to all reported metrics. Further, it achieves competitive results when compared to the iterative reconstruction method based on adaptive regularization with dictionary learning and total variation and when compared to the methods based on cascaded networks, while only requiring a small fraction of the computational and training time. A persistent homology analysis demonstrates that the data manifold of the spatio-temporal domain has a lower complexity than the one of the spatial domain and therefore, the learning of a projection-like mapping is facilitated. Even when trained on only one single subject without data-augmentation, our approach yields results which are similar to the ones obtained on a large training dataset. This makes the method particularly suitable for training a network on limited training data. Finally, in contrast to the spatial 2D U-net, our proposed method is shown to be naturally robust with respect to image rotation in image space and almost achieves rotation-equivariance where neither data-augmentation nor a particular network design are required.

ADMM-CSNet: A Deep Learning Approach for Image Compressive Sensing

Compressive sensing (CS) is an effective technique for reconstructing image from a small amount of sampled data. It has been widely applied in medical imaging, remote sensing, image compression, etc. In this paper, we propose two versions of a novel deep learning architecture, dubbed as ADMM-CSNet, by combining the traditional model-based CS method and data-driven deep learning method for image reconstruction from sparsely sampled measurements. We first consider a generalized CS model for image reconstruction with undetermined regularizations in undetermined transform domains, and then two efficient solvers using Alternating Direction Method of Multipliers (ADMM) algorithm for optimizing the model are proposed. We further unroll and generalize the ADMM algorithm to be two deep architectures, in which all parameters of the CS model and the ADMM algorithm are discriminatively learned by end-to-end training. For both applications of fast CS complex-valued MR imaging and CS imaging of real-valued natural images, the proposed ADMM-CSNet achieved favorable reconstruction accuracy in fast computational speed compared with the traditional and the other deep learning methods.

A deep error correction network for compressed sensing MRI

BACKGROUND

CS-MRI (compressed sensing for magnetic resonance imaging) exploits image sparsity properties to reconstruct MRI from very few Fourier k-space measurements. Due to imperfect modelings in the inverse imaging, state-of-the-art CS-MRI methods tend to leave structural reconstruction errors. Compensating such errors in the reconstruction could help further improve the reconstruction quality.

RESULTS

In this work, we propose a DECN (deep error correction network) for CS-MRI. The DECN model consists of three parts, which we refer to as modules: a guide, or template, module, an error correction module, and a data fidelity module. Existing CS-MRI algorithms can serve as the template module for guiding the reconstruction. Using this template as a guide, the error correction module learns a CNN (convolutional neural network) to map the k-space data in a way that adjusts for the reconstruction error of the template image. We propose a deep error correction network. Our experimental results show the proposed DECN CS-MRI reconstruction framework can considerably improve upon existing inversion algorithms by supplementing with an error-correcting CNN.

CONCLUSIONS

In the proposed a deep error correction framework, any off-the-shelf CS-MRI algorithm can be used as template generation. Then a deep neural network is used to compensate reconstruction errors. The promising experimental results validate the effectiveness and utility of the proposed framework.

Accelerated coronary MRI with sRAKI: A database-free self-consistent neural network k-space reconstruction for arbitrary undersampling

Purpose

To accelerate coronary MRI acquisitions with arbitrary undersampling patterns by using a novel reconstruction algorithm that applies coil self-consistency using subject-specific neural networks.

Methods

Self-consistent robust artificial-neural-networks for k-space interpolation (sRAKI) performs iterative parallel imaging reconstruction by enforcing self-consistency among coils. The approach bears similarity to SPIRiT, but extends the linear convolutions in SPIRiT to nonlinear interpolation using convolutional neural networks (CNNs). These CNNs are trained individually for each scan using the scan-specific autocalibrating signal (ACS) data. Reconstruction is performed by imposing the learned self-consistency and data-consistency, which enables sRAKI to support random undersampling patterns. Fully-sampled targeted right coronary artery MRI was acquired in six healthy subjects. The data were retrospectively undersampled, and reconstructed using SPIRiT, l₁-SPIRiT and sRAKI for acceleration rates of 2 to 5. Additionally, prospectively undersampled whole-heart coronary MRI was acquired to further evaluate reconstruction performance.

Results

sRAKI reduces noise amplification and blurring artifacts compared with SPIRiT and l₁-SPIRiT, especially at high acceleration rates in targeted coronary MRI. Quantitative analysis shows that sRAKI outperforms these techniques in terms of normalized mean-squared-error (~44% and ~21% over SPIRiT and l1-SPIRiT at rate 5) and vessel sharpness (~10% and ~20% over SPIRiT and l₁-SPIRiT at rate 5). Whole-heart data shows the sharpest coronary arteries when resolved using sRAKI, with 11% and 15% improvement in vessel sharpness over SPIRiT and l₁-SPIRiT, respectively.

Conclusion

sRAKI is a database-free neural network-based reconstruction technique that may further accelerate coronary MRI with arbitrary undersampling patterns, while improving noise resilience over linear parallel imaging and image sharpness over l₁ regularization techniques.

k -Space Deep Learning for Accelerated MRI

The annihilating filter-based low-rank Hankel matrix approach (ALOHA) is one of the state-of-the-art compressed sensing approaches that directly interpolates the missing k -space data using low-rank Hankel matrix completion. The success of ALOHA is due to the concise signal representation in the k -space domain, thanks to the duality between structured low-rankness in the k -space domain and the image domain sparsity. Inspired by the recent mathematical discovery that links convolutional neural networks to Hankel matrix decomposition using data-driven framelet basis, here we propose a fully data-driven deep learning algorithm for k -space interpolation. Our network can be also easily applied to non-Cartesian k -space trajectories by simply adding an additional regridding layer. Extensive numerical experiments show that the proposed deep learning method consistently outperforms the existing image-domain deep learning approaches.

Current applications and future directions of deep learning in musculoskeletal radiology

Deep learning with convolutional neural networks (CNN) is a rapidly advancing subset of artificial intelligence that is ideally suited to solving image-based problems. There are an increasing number of musculoskeletal applications of deep learning, which can be conceptually divided into the categories of lesion detection, classification, segmentation, and non-interpretive tasks. Numerous examples of deep learning achieving expert-level performance in specific tasks in all four categories have been demonstrated in the past few years, although comprehensive interpretation of imaging examinations has not yet been achieved. It is important for the practicing musculoskeletal radiologist to understand the current scope of deep learning as it relates to musculoskeletal radiology. Interest in deep learning from researchers, radiology leadership, and industry continues to increase, and it is likely that these developments will impact the daily practice of musculoskeletal radiology in the near future.

fastMRI: A Publicly Available Raw k-Space and DICOM Dataset of Knee Images for Accelerated MR Image Reconstruction Using Machine Learning

A publicly available dataset containing k-space data as well as Digital Imaging and Communications in Medicine image data of knee images for accelerated MR image reconstruction using machine learning is presented.

Accelerating MR Imaging via Deep Chambolle-Pock Network

Compressed sensing (CS) has been introduced to accelerate data acquisition in MR Imaging. However, CS-MRI methods suffer from detail loss with large acceleration and complicated parameter selection. To address the limitations of existing CS-MRI methods, a model-driven MR reconstruction is proposed that trains a deep network, named CP-net, which is derived from the Chambolle-Pock algorithm to reconstruct the in vivo MR images of human brains from highly undersampled complex k-space data acquired on different types of MR scanners. The proposed deep network can learn the proximal operator and parameters among the Chambolle-Pock algorithm. All of the experiments show that the proposed CP-net achieves more accurate MR reconstruction results, outperforming state-of-the-art methods across various quantitative metrics.

A Network-Driven Prior Induced Bregman Model for Parallel MR Imaging

Compressed sensing based parallel imaging (CS-PI) has attracted great attention in fast magnetic resonance imaging (MRI) community. In particular, Bregman iterative model has shown encouraging performance in solving this problem. However, its regularization term still has large room for improvement. In this work, we propose a network-driven prior induced Bregman model, dubbed as Breg-EDAEP, for CS-PI task. In the present model, the implicit property among different channel MR images is preliminarily explored by the network to obtain more structure details in iterative reconstruction procedure. Experiments on various acceleration factors and sampling patterns have shown that the proposed method outperforms the state-of-the-art algorithms. Breg-EDAEP possesses strong capability to restore image details and preserves well structure information.

Deep Magnetic Resonance Image Reconstruction: Inverse Problems Meet Neural Networks

Image reconstruction from undersampled k-space data has been playing an important role in fast MRI. Recently, deep learning has demonstrated tremendous success in various fields and has also shown potential in significantly accelerating MRI reconstruction with fewer measurements. This article provides an overview of the deep learning-based image reconstruction methods for MRI. Two types of deep learning-based approaches are reviewed: those based on unrolled algorithms and those which are not. The main structure of both approaches are explained, respectively. Several signal processing issues for maximizing the potential of deep reconstruction in fast MRI are discussed. The discussion may facilitate further development of the networks and the analysis of performance from a theoretical point of view.

Image Reconstruction: From Sparsity to Data-adaptive Methods and Machine Learning

The field of medical image reconstruction has seen roughly four types of methods. The first type tended to be analytical methods, such as filtered back-projection (FBP) for X-ray computed tomography (CT) and the inverse Fourier transform for magnetic resonance imaging (MRI), based on simple mathematical models for the imaging systems. These methods are typically fast, but have suboptimal properties such as poor resolution-noise trade-off for CT. A second type is iterative reconstruction methods based on more complete models for the imaging system physics and, where appropriate, models for the sensor statistics. These iterative methods improved image quality by reducing noise and artifacts. The FDA-approved methods among these have been based on relatively simple regularization models. A third type of methods has been designed to accommodate modified data acquisition methods, such as reduced sampling in MRI and CT to reduce scan time or radiation dose. These methods typically involve mathematical image models involving assumptions such as sparsity or low-rank. A fourth type of methods replaces mathematically designed models of signals and systems with data-driven or adaptive models inspired by the field of machine learning. This paper focuses on the two most recent trends in medical image reconstruction: methods based on sparsity or low-rank models, and data-driven methods based on machine learning techniques.

A Deep Information Sharing Network for Multi-Contrast Compressed Sensing MRI Reconstruction

Compressed sensing (CS) theory can accelerate multi-contrast magnetic resonance imaging (MRI) by sampling fewer measurements within each contrast. However, conventional optimization-based reconstruction models suffer several limitations, including a strict assumption of shared sparse support, time-consuming optimization, and "shallow" models with difficulties in encoding the patterns contained in massive MRI data. In this paper, we propose the first deep learning model for multi-contrast CS-MRI reconstruction. We achieve information sharing through feature sharing units, which significantly reduces the number of model parameters. The feature sharing unit combines with a data fidelity unit to comprise an inference block, which are then cascaded with dense connections, allowing for efficient information transmission across different depths of the network. Experiments on various multi-contrast MRI datasets show that the proposed model outperforms both state-of-the-art single-contrast and multi-contrast MRI methods in accuracy and efficiency. We demonstrate that improved reconstruction quality can bring benefits to subsequent medical image analysis. Furthermore, the robustness of the proposed model to misregistration shows its potential in real MRI applications.

PET image denoising using unsupervised deep learning

PURPOSE

Image quality of positron emission tomography (PET) is limited by various physical degradation factors. Our study aims to perform PET image denoising by utilizing prior information from the same patient. The proposed method is based on unsupervised deep learning, where no training pairs are needed.

METHODS

In this method, the prior high-quality image from the patient was employed as the network input and the noisy PET image itself was treated as the training label. Constrained by the network structure and the prior image input, the network was trained to learn the intrinsic structure information from the noisy image and output a restored PET image. To validate the performance of the proposed method, a computer simulation study based on the BrainWeb phantom was first performed. A 68Ga-PRGD2 PET/CT dataset containing 10 patients and a 18F-FDG PET/MR dataset containing 30 patients were later on used for clinical data evaluation. The Gaussian, non-local mean (NLM) using CT/MR image as priors, BM4D, and Deep Decoder methods were included as reference methods. The contrast-to-noise ratio (CNR) improvements were used to rank different methods based on Wilcoxon signed-rank test.

RESULTS

For the simulation study, contrast recovery coefficient (CRC) vs. standard deviation (STD) curves showed that the proposed method achieved the best performance regarding the bias-variance tradeoff. For the clinical PET/CT dataset, the proposed method achieved the highest CNR improvement ratio (53.35% ± 21.78%), compared with the Gaussian (12.64% ± 6.15%, P = 0.002), NLM guided by CT (24.35% ± 16.30%, P = 0.002), BM4D (38.31% ± 20.26%, P = 0.002), and Deep Decoder (41.67% ± 22.28%, P = 0.002) methods. For the clinical PET/MR dataset, the CNR improvement ratio of the proposed method achieved 46.80% ± 25.23%, higher than the Gaussian (18.16% ± 10.02%, P < 0.0001), NLM guided by MR (25.36% ± 19.48%, P < 0.0001), BM4D (37.02% ± 21.38%, P < 0.0001), and Deep Decoder (30.03% ± 20.64%, P < 0.0001) methods. Restored images for all the datasets demonstrate that the proposed method can effectively smooth out the noise while recovering image details.

CONCLUSION

The proposed unsupervised deep learning framework provides excellent image restoration effects, outperforming the Gaussian, NLM methods, BM4D, and Deep Decoder methods.

SANTIS: Sampling-Augmented Neural neTwork with Incoherent Structure for MR image reconstruction

PURPOSE

To develop and evaluate a novel deep learning-based reconstruction framework called SANTIS (Sampling-Augmented Neural neTwork with Incoherent Structure) for efficient MR image reconstruction with improved robustness against sampling pattern discrepancy.

METHODS

With a combination of data cycle-consistent adversarial network, end-to-end convolutional neural network mapping, and data fidelity enforcement for reconstructing undersampled MR data, SANTIS additionally utilizes a sampling-augmented training strategy by extensively varying undersampling patterns during training, so that the network is capable of learning various aliasing structures and thereby removing undersampling artifacts more effectively and robustly. The performance of SANTIS was demonstrated for accelerated knee imaging and liver imaging using a Cartesian trajectory and a golden-angle radial trajectory, respectively. Quantitative metrics were used to assess its performance against different references. The feasibility of SANTIS in reconstructing dynamic contrast-enhanced images was also demonstrated using transfer learning.

RESULTS

Compared to conventional reconstruction that exploits image sparsity, SANTIS achieved consistently improved reconstruction performance (lower errors and greater image sharpness). Compared to standard learning-based methods without sampling augmentation (e.g., training with a fixed undersampling pattern), SANTIS provides comparable reconstruction performance, but significantly improved robustness, against sampling pattern discrepancy. SANTIS also achieved encouraging results for reconstructing liver images acquired at different contrast phases.

CONCLUSION

By extensively varying undersampling patterns, the sampling-augmented training strategy in SANTIS can remove undersampling artifacts more robustly. The novel concept behind SANTIS can particularly be useful for improving the robustness of deep learning-based image reconstruction against discrepancy between training and inference, an important, but currently less explored, topic.

Machine Friendly Machine Learning: Interpretation of Computed Tomography Without Image Reconstruction

Recent advancements in deep learning for automated image processing and classification have accelerated many new applications for medical image analysis. However, most deep learning algorithms have been developed using reconstructed, human-interpretable medical images. While image reconstruction from raw sensor data is required for the creation of medical images, the reconstruction process only uses a partial representation of all the data acquired. Here, we report the development of a system to directly process raw computed tomography (CT) data in sinogram-space, bypassing the intermediary step of image reconstruction. Two classification tasks were evaluated for their feasibility of sinogram-space machine learning: body region identification and intracranial hemorrhage (ICH) detection. Our proposed SinoNet, a convolutional neural network optimized for interpreting sinograms, performed favorably compared to conventional reconstructed image-space-based systems for both tasks, regardless of scanning geometries in terms of projections or detectors. Further, SinoNet performed significantly better when using sparsely sampled sinograms than conventional networks operating in image-space. As a result, sinogram-space algorithms could be used in field settings for triage (presence of ICH), especially where low radiation dose is desired. These findings also demonstrate another strength of deep learning where it can analyze and interpret sinograms that are virtually impossible for human experts.

Reconstruction techniques for cardiac cine MRI

The present survey describes the state-of-the-art techniques for dynamic cardiac magnetic resonance image reconstruction. Additionally, clinical relevance, main challenges, and future trends of this image modality are outlined. Thus, this paper aims to provide a general vision about cine MRI as the standard procedure in functional evaluation of the heart, focusing on technical methodologies.

ReconSocket: a low-latency raw data streaming interface for real-time MRI-guided radiotherapy

With the recent advent of hybrid MRI-guided radiotherapy systems, continuous intra-fraction MR imaging for motion monitoring has become feasible. The ability to perform real-time custom image reconstructions is however often lacking. In this work we present a low-latency streaming solution, ReconSocket, which provides a real-time stream of k-space data from the magnetic resonance imaging (MRI) to custom reconstruction servers. We determined the performance of the data streaming by measuring the streaming latency (i.e. non-zero time delay due to data transfer and processing) and jitter (i.e. deviations from periodicity) using an ultra-fast 1D MRI acquisition of a moving phantom. Simultaneously, its position was recorded with near-zero time delay. The feasibility of low-latency custom reconstructions was tested by measuring the imaging latency (i.e. time delay between physical change and appearance of that change on the image) for several non-Cartesian 2D and 3D acquisitions using an in-house implemented reconstruction server. The measured streaming latency of the ReconSocket interface was [Formula: see text] ms. 98% of the incoming data packets arrived within a jitter range of 367 [Formula: see text]s. This shows that the ReconSocket interface can provide reliable real-time access to MRI data, acquired during the course of a MRI-guided radiotherapy fraction. The total imaging latency was measured to be 221 ms (2D) and 3889 ms (3D) for exemplary acquisitions, using the custom image reconstruction server. These imaging latencies are approximately equal to half of the temporal footprint (T _acq /2) of the respective 2D and 3D golden-angle radial sequences. For radial sequences, it was previously showed that T _acq /2 is the expected contribution of only the data acquisition to the total imaging latency. Indeed, the contribution of the non-Cartesian reconstruction to the total imaging latency was minor (<10%): 21 ms for 2D, 300 ms for 3D, indicating that the acquisition, i.e. the physical encoding of the image itself is the major contributor to the total imaging latency.

Deep Learning Based Noise Reduction for Brain MR Imaging: Tests on Phantoms and Healthy Volunteers

Purpose:

To test whether our proposed denoising approach with deep learning-based reconstruction (dDLR) can effectively denoise brain MR images.

Methods:

In an initial experimental study, we obtained brain images from five volunteers and added different artificial noise levels. Denoising was applied to the modified images using a denoising convolutional neural network (DnCNN), a shrinkage convolutional neural network (SCNN), and dDLR. Using these brain MR images, we compared the structural similarity (SSIM) index and peak signal-to-noise ratio (PSNR) between the three denoising methods. Two neuroradiologists assessed the image quality of the three types of images. In the clinical study, we evaluated the denoising effect of dDLR in brain images with different levels of actual noise such as thermal noise. Specifically, we obtained 2D-T₂-weighted image, 2D-fluid-attenuated inversion recovery (FLAIR) and 3D-magnetization-prepared rapid acquisition with gradient echo (MPRAGE) from 15 healthy volunteers at two different settings for the number of image acquisitions (NAQ): NAQ2 and NAQ5. We reconstructed dDLR-processed NAQ2 from NAQ2, then compared with SSIM and PSNR. Two neuroradiologists separately assessed the image quality of NAQ5, NAQ2 and dDLR-NAQ2. Statistical analysis was performed in the experimental and clinical study. In the clinical study, the inter-observer agreement was also assessed.

Results:

In the experimental study, PSNR and SSIM for dDLR were statistically higher than those of DnCNN and SCNN (P < 0.001). The image quality of dDLR was also superior to DnCNN and SCNN. In the clinical study, dDLR-NAQ2 was significantly better than NAQ2 images for SSIM and PSNR in all three sequences (P < 0.05), except for PSNR in FLAIR. For all qualitative items, dDLR-NAQ2 had equivalent or better image quality than NAQ5, and superior quality to that of NAQ2 (P < 0.05), for all criteria except artifact. The inter-observer agreement ranged from substantial to near perfect.

Conclusion:

dDLR reduces image noise while preserving image quality on brain MR images.

Applications of Deep Learning to Neuro-Imaging Techniques

Many clinical applications based on deep learning and pertaining to radiology have been proposed and studied in radiology for classification, risk assessment, segmentation tasks, diagnosis, prognosis, and even prediction of therapy responses. There are many other innovative applications of AI in various technical aspects of medical imaging, particularly applied to the acquisition of images, ranging from removing image artifacts, normalizing/harmonizing images, improving image quality, lowering radiation and contrast dose, and shortening the duration of imaging studies. This article will address this topic and will seek to present an overview of deep learning applied to neuroimaging techniques.

Robust Single-Shot T2 Mapping via Multiple Overlapping-Echo Acquisition and Deep Neural Network

Quantitative magnetic resonance imaging (MRI) is of great value to both clinical diagnosis and scientific research. However, most MRI experiments remain qualitative, especially dynamic MRI, because repeated sampling with variable weighting parameter makes quantitative imaging time-consuming and sensitive to motion artifacts. A single-shot quantitative T₂ mapping method based on multiple overlapping-echo acquisition (dubbed MOLED-4) was proposed to obtain reliable T₂ mapping in milliseconds. Different from traditional MRI acceleration methods, such as compressed sensing and parallel imaging, MOLED-4 accelerates quantitative T₂ mapping via synchronized multisampling and then deep learning to map the complex nonlinear relationship that is difficult to solve by traditional optimization-based methods. The results of simulation, phantom, and in vivo human brain experiments show the great performance of the proposed method. The principle of MOLED-4 may be extended to other ultrafast quantitative parameter mappings and potentially lead to new dynamic MRI with high efficiency to catch quantitative variation of tissue properties.

Fast and accurate reconstruction of human lung gas MRI with deep learning

PURPOSE

To fast and accurately reconstruct human lung gas MRI from highly undersampled k-space using deep learning.

METHODS

The scheme was comprised of coarse-to-fine nets (C-net and F-net). Zero-filling images from retrospectively undersampled k-space at an acceleration factor of 4 were used as input for C-net, and then output intermediate results which were fed into F-net. During training, a L2 loss function was adopted in C-net, while a function that united L2 loss with proton prior knowledge was used in F-net. The 871 hyperpolarized ¹²⁹ Xe pulmonary ventilation images from 72 volunteers were randomly arranged as training (90%) and testing (10%) data. Ventilation defect percentage comparisons were implemented using a paired 2-tailed Student's t-test and correlation analysis. Furthermore, prospective acquisitions were demonstrated in 5 healthy subjects and 5 asymptomatic smokers.

RESULTS

Each image with size of 96 × 84 could be reconstructed within 31 ms (mean absolute error was 4.35% and structural similarity was 0.7558). Compared with conventional compressed sensing MRI, the mean absolute error decreased by 17.92%, but the structural similarity increased by 6.33%. For ventilation defect percentage, there were no significant differences between the fully sampled and reconstructed images through the proposed algorithm (P = 0.932), but had significant correlations (r = 0.975; P < 0.001). The prospectively undersampled results validated a good agreement with fully sampled images, with no significant differences in ventilation defect percentage but significantly higher signal-to-noise ratio values.

CONCLUSION

The proposed algorithm outperformed classical undersampling methods, paving the way for future use of deep learning in real-time and accurate reconstruction of gas MRI.

ACCELERATED CORONARY MRI USING 3D SPIRIT-RAKI WITH SPARSITY REGULARIZATION

Coronary MRI is a non-invasive radiation-free imaging tool for the diagnosis of coronary artery disease. One of its limitations is the long scan time, due to the need for high resolution imaging in the presence of respiratory and cardiac motions. Machine learning (ML) methods have been recently utilized to accelerate MRI. In particular, a scan-specific ML technique, called Robust Artifical-neural-network for k-space Interpolation (RAKI) has shown promise in cardiac MRI. However, it requires uniform undersampling. In this study, we sought to extend this approach to arbitrary sampling patterns, using coil self-consistency. This technique, called SPIRiT-RAKI, utilizes scan-specific convolutional neural networks to nonlinearly enforce coil self-consistency. Additionally, regularization terms can also be incorporated. SPIRiT-RAKI was used to accelerate right coronary MRI. Reconstructions were compared to SPIRiT for different undersampling patterns and acceleration rates. Results show SPIRiT-RAKI reduces residual aliasing and blurring artifacts compared to SPIRiT.

MR Image Reconstruction Using Deep Density Priors

Algorithms for magnetic resonance (MR) image reconstruction from undersampled measurements exploit prior information to compensate for missing k-space data. Deep learning (DL) provides a powerful framework for extracting such information from existing image datasets, through learning, and then using it for reconstruction. Leveraging this, recent methods employed DL to learn mappings from undersampled to fully sampled images using paired datasets, including undersampled and corresponding fully sampled images, integrating prior knowledge implicitly. In this letter, we propose an alternative approach that learns the probability distribution of fully sampled MR images using unsupervised DL, specifically variational autoencoders (VAE), and use this as an explicit prior term in reconstruction, completely decoupling the encoding operation from the prior. The resulting reconstruction algorithm enjoys a powerful image prior to compensate for missing k-space data without requiring paired datasets for training nor being prone to associated sensitivities, such as deviations in undersampling patterns used in training and test time or coil settings. We evaluated the proposed method with T1 weighted images from a publicly available dataset, multi-coil complex images acquired from healthy volunteers ( N=8 ), and images with white matter lesions. The proposed algorithm, using the VAE prior, produced visually high quality reconstructions and achieved low RMSE values, outperforming most of the alternative methods on the same dataset. On multi-coil complex data, the algorithm yielded accurate magnitude and phase reconstruction results. In the experiments on images with white matter lesions, the method faithfully reconstructed the lesions.

PET Image Reconstruction Using Deep Image Prior

Recently, deep neural networks have been widely and successfully applied in computer vision tasks and have attracted growing interest in medical imaging. One barrier for the application of deep neural networks to medical imaging is the need for large amounts of prior training pairs, which is not always feasible in clinical practice. This is especially true for medical image reconstruction problems, where raw data are needed. Inspired by the deep image prior framework, in this paper, we proposed a personalized network training method where no prior training pairs are needed, but only the patient's own prior information. The network is updated during the iterative reconstruction process using the patient-specific prior information and measured data. We formulated the maximum-likelihood estimation as a constrained optimization problem and solved it using the alternating direction method of multipliers algorithm. Magnetic resonance imaging guided positron emission tomography reconstruction was employed as an example to demonstrate the effectiveness of the proposed framework. Quantification results based on simulation and real data show that the proposed reconstruction framework can outperform Gaussian post-smoothing and anatomically guided reconstructions using the kernel method or the neural-network penalty.

Parallel imaging and convolutional neural network combined fast MR image reconstruction: Applications in low-latency accelerated real-time imaging

PURPOSE

To develop and evaluate a parallel imaging and convolutional neural network combined image reconstruction framework for low-latency and high-quality accelerated real-time MR imaging.

METHODS

Conventional Parallel Imaging reconstruction resolved as gradient descent steps was compacted as network layers and interleaved with convolutional layers in a general convolutional neural network. All parameters of the network were determined during the offline training process, and applied to unseen data once learned. The proposed network was first evaluated for real-time cardiac imaging at 1.5 T and real-time abdominal imaging at 0.35 T, using threefold to fivefold retrospective undersampling for cardiac imaging and threefold retrospective undersampling for abdominal imaging. Then, prospective undersampling with fourfold acceleration was performed on cardiac imaging to compare the proposed method with standard clinically available GRAPPA method and the state-of-the-art L1-ESPIRiT method.

RESULTS

Both retrospective and prospective evaluations confirmed that the proposed network was able to images with a lower noise level and reduced aliasing artifacts in comparison with the single-coil based and L1-ESPIRiT reconstructions for cardiac imaging at 1.5 T, and the GRAPPA and L1-ESPIRiT reconstructions for abdominal imaging at 0.35 T. Using the proposed method, each frame can be reconstructed in less than 100 ms, suggesting its clinical compatibility.

CONCLUSION

The proposed Parallel Imaging and convolutional neural network combined reconstruction framework is a promising technique that allows low-latency and high-quality real-time MR imaging.

Fast learning of fiber orientation distribution function for MR tractography using convolutional neural network

PURPOSE

In diffusion-weighted magnetic resonance imaging (DW-MRI), the fiber orientation distribution function (fODF) is of great importance for solving complex fiber configurations to achieve reliable tractography throughout the brain, which ultimately facilitates the understanding of brain connectivity and exploration of neurological dysfunction. Recently, multi-shell multi-tissue constrained spherical deconvolution (MSMT-CSD) method has been explored for reconstructing full fODFs. To achieve a reliable fitting, similar to other model-based approaches, a large number of diffusion measurements is typically required for MSMT-CSD method. The prolonged acquisition is, however, not feasible in practical clinical routine and is prone to motion artifacts. To accelerate the acquisition, we proposed a method to reconstruct the fODF from downsampled diffusion-weighted images (DWIs) by leveraging the strong inference ability of the deep convolutional neural network (CNN).

METHODS

The method treats spherical harmonics (SH)-represented DWI signals and fODF coefficients as inputs and outputs, respectively. To compensate for the reduced gradient directions with reduced number of DWIs in acquisition in each voxel, its surrounding voxels are incorporated by the network for exploiting their spatial continuity. The resulting fODF coefficients are fitted with applying the CNN in a multi-target regression model. The network is composed of two convolutional layers and three fully connected layers. To obtain an initial evaluation of the method, we quantitatively measured its performance on a simulated dataset. Then, for in vivo tests, we employed data from 24 subjects from the Human Connectome Project (HCP) as training set and six subjects as test set. The performance of the proposed method was primarily compared to the super-resolved MSMT-CSD with the decreasing number of DWIs. The fODFs reconstructed by MSMT-CSD from all available 288 DWIs were used as training labels and the reference standard. The performance was quantitatively measured by the angular correlation coefficient (ACC) and the mean angular error (MAE).

RESULTS

For the simulated dataset, the proposed method exhibited the potential advantage over the model reconstruction. For the in vivo dataset, it achieved superior results over the MSMT-CSD in all the investigated cases, with its advantage more obvious when a limited number of DWIs were used. As the number of DWIs was reduced from 95 to 25, the median ACC ranged from 0.96 to 0.91 for the CNN, but 0.93 to 0.77 for the MSMT-CSD (with perfect score of 1). The angular error in the typical regions of interest (ROIs) was also much lower, especially in multi-fiber regions. The average MAE for the CNN method in regions containing one, two, three fibers was, respectively, 1.09°, 2.75°, and 8.35° smaller than the MSMT-CSD method. The visual inception of the fODF further confirmed this superiority. Moreover, the tractography results validated the effectiveness of the learned fODF, in preserving known major branching fibers with only 25 DWIs.

CONCLUSION

Experiments on HCP datasets demonstrated the feasibility of the proposed method in recovering fODFs from up to 11-fold reduced number of DWIs. The proposed method offers a new streamlined reconstruction procedure and exhibits promising potential in acquisition acceleration for the reconstruction of fODFs with good accuracy.

Compressed sensing MRI: a review from signal processing perspective

Magnetic resonance imaging (MRI) is an inherently slow imaging modality, since it acquires multi-dimensional k-space data through 1-D free induction decay or echo signals. This often limits the use of MRI, especially for high resolution or dynamic imaging. Accordingly, many investigators has developed various acceleration techniques to allow fast MR imaging. For the last two decades, one of the most important breakthroughs in this direction is the introduction of compressed sensing (CS) that allows accurate reconstruction from sparsely sampled k-space data. The recent FDA approval of compressed sensing products for clinical scans clearly reflect the maturity of this technology. Therefore, this paper reviews the basic idea of CS and how this technology have been evolved for various MR imaging problems.

MANTIS: Model-Augmented Neural neTwork with Incoherent k-space Sampling for efficient MR parameter mapping

PURPOSE

To develop and evaluate a novel deep learning-based image reconstruction approach called MANTIS (Model-Augmented Neural neTwork with Incoherent k-space Sampling) for efficient MR parameter mapping.

METHODS

MANTIS combines end-to-end convolutional neural network (CNN) mapping, incoherent k-space undersampling, and a physical model as a synergistic framework. The CNN mapping directly converts a series of undersampled images straight into MR parameter maps using supervised training. Signal model fidelity is enforced by adding a pathway between the undersampled k-space and estimated parameter maps to ensure that the parameter maps produced synthesized k-space consistent with the acquired undersampling measurements. The MANTIS framework was evaluated on the T₂ mapping of the knee at different acceleration rates and was compared with 2 other CNN mapping methods and conventional sparsity-based iterative reconstruction approaches. Global quantitative assessment and regional T₂ analysis for the cartilage and meniscus were performed to demonstrate the reconstruction performance of MANTIS.

RESULTS

MANTIS achieved high-quality T₂ mapping at both moderate (R = 5) and high (R = 8) acceleration rates. Compared to conventional reconstruction approaches that exploited image sparsity, MANTIS yielded lower errors (normalized root mean square error of 6.1% for R = 5 and 7.1% for R = 8) and higher similarity (structural similarity index of 86.2% at R = 5 and 82.1% at R = 8) to the reference in the T₂ estimation. MANTIS also achieved superior performance compared to direct CNN mapping and a 2-step CNN method.

CONCLUSION

The MANTIS framework, with a combination of end-to-end CNN mapping, signal model-augmented data consistency, and incoherent k-space sampling, is a promising approach for efficient and robust estimation of quantitative MR parameters.

Iterative PET Image Reconstruction Using Convolutional Neural Network Representation

PET image reconstruction is challenging due to the ill-poseness of the inverse problem and limited number of detected photons. Recently, the deep neural networks have been widely and successfully used in computer vision tasks and attracted growing interests in medical imaging. In this paper, we trained a deep residual convolutional neural network to improve PET image quality by using the existing inter-patient information. An innovative feature of the proposed method is that we embed the neural network in the iterative reconstruction framework for image representation, rather than using it as a post-processing tool. We formulate the objective function as a constrained optimization problem and solve it using the alternating direction method of multipliers algorithm. Both simulation data and hybrid real data are used to evaluate the proposed method. Quantification results show that our proposed iterative neural network method can outperform the neural network denoising and conventional penalized maximum likelihood methods.

PET Image Denoising Using a Deep Neural Network Through Fine Tuning

Positron emission tomography (PET) is a functional imaging modality widely used in clinical diagnosis. In this work, we trained a deep convolutional neural network (CNN) to improve PET image quality. Perceptual loss based on features derived from a pre-trained VGG network, instead of the conventional mean squared error, was employed as the training loss function to preserve image details. As the number of real patient data set for training is limited, we propose to pre-train the network using simulation data and fine-tune the last few layers of the network using real data sets. Results from simulation, real brain and lung data sets show that the proposed method is more effective in removing noise than the traditional Gaussian filtering method.

Accelerated Simultaneous Multi-Slice MRI using Subject-Specific Convolutional Neural Networks

Simultaneous multi-slice or multi-band (SMS/MB) imaging allows accelerated coverage in magnetic resonance imaging (MRI). Multiple slices are excited and acquired at the same time, and reconstructed using the redundancies in receiver coil arrays, similar to parallel imaging. SMS/MB reconstruction is currently performed with linear reconstruction techniques. Recently, a nonlinear reconstruction method for parallel imaging, Robust Artificial-neural-networks for k-space Interpolation (RAKI) was proposed and shown to improve upon linear methods. This method uses convolutional neural networks (CNN) trained solely on subject-specific calibration data. In this study, we sought to extend RAKI to SMS/MB imaging reconstruction. CNN training was performed on calibration data acquired prior to SMS/MB imaging, in a manner consistent with the existing linear methods. These CNNs were used to reconstruct a time series of functional MRI (fMRI) data. CNN network parameters were optimized using an extensive search of the parameter space. With these optimal parameters, RAKI substantially improves image quality compared to a commonly used linear reconstruction algorithm, especially for high acceleration rates.

Highly accelerated, model-free diffusion tensor MRI reconstruction using neural networks

PURPOSE

The purpose of this study was to develop a neural network that accurately performs diffusion tensor imaging (DTI) reconstruction from highly accelerated scans.

MATERIALS AND METHODS

This retrospective study was conducted using data acquired between 2013 and 2018 and was approved by the local institutional review board. DTI acquired in healthy volunteers (N = 10) was used to train a neural network, DiffNet, to reconstruct fractional anisotropy (FA) and mean diffusivity (MD) maps from small subsets of acquired DTI data with between 3 and 20 diffusion-encoding directions. FA and MD maps were then reconstructed in volunteers and in patients with glioblastoma multiforme (GBM, N = 12) using both DiffNet and conventional reconstructions. Accuracy and precision were quantified in volunteer scans and compared between reconstructions. The accuracy of tumor delineation was compared between reconstructed patient data by evaluating agreement between DTI-derived tumor volumes and volumes defined by contrast-enhanced T1-weighted MRI. Comparisons were performed using areas under the receiver operating characteristic curves (AUC).

RESULTS

DiffNet FA reconstructions were more accurate and precise compared with conventional reconstructions for all acceleration factors. DiffNet permitted reconstruction with only three diffusion-encoding directions with significantly lower bias than the conventional method using six directions (0.01 ± 0.01 vs 0.06 ± 0.01, P < 0.001). While MD-based tumor delineation was not substantially different with DiffNet (AUC range: 0.888-0.902), DiffNet FA had higher AUC than conventional reconstructions for fixed scan time and achieved similar performance with shorter scans (conventional, six directions: AUC = 0.926, DiffNet, three directions: AUC = 0.920).

CONCLUSION

DiffNet improved DTI reconstruction accuracy, precision, and tumor delineation performance in GBM while permitting reconstruction from only three diffusion-encoding directions.&!#6.

Clinical Potential of a New Approach to MRI Acceleration

Fast ROtary Nonlinear Spatial ACquisition (FRONSAC) was recently introduced as a new strategy that applies nonlinear gradients as a small perturbation to improve image quality in highly undersampled MRI. In addition to experimentally showing the previously simulated improvement to image quality, this work introduces the insight that Cartesian-FRONSAC retains many desirable features of Cartesian imaging. Cartesian-FRONSAC preserves the existing linear gradient waveforms of the Cartesian sequence while adding oscillating nonlinear gradient waveforms. Experiments show that performance is essentially identical to Cartesian imaging in terms of (1) resilience to experimental imperfections, like timing errors or off-resonance spins, (2) accommodating scan geometry changes without the need for recalibration or additional field mapping, (3) contrast generation, as in turbo spin echo. Despite these similarities to Cartesian imaging, which provides poor parallel imaging performance, Cartesian-FRONSAC consistently shows reduced undersampling artifacts and better response to advanced reconstruction techniques. A final experiment shows that hardware requirements are also flexible. Cartesian-FRONSAC improves accelerated imaging while retaining the robustness and flexibility critical to real clinical use.

Noncontrast MR angiography: An update

Both computed tomography (CT) angiography (CTA) and contrast-enhanced MR angiography (CEMRA) have proven to be useful and accurate cross-sectional imaging modalities over a wide range of vascular territories and vascular disorders. A key advantage of MRA is that, unlike CTA, it can be performed without the administration of a contrast agent. In this review article we consider the motivations for using noncontrast MRA, potential contrast mechanisms, imaging techniques, advantages, and drawbacks with respect to CTA and CEMRA, and the level of evidence for using the various MRA techniques. In addition, we explore new developments that promise to expand the reliability and range of clinical applications for noncontrast MRA, along with functional MRA capabilities not available with CTA or CEMRA. Level of Evidence: 1 Technical Efficacy: Stage 1 J. Magn. Reson. Imaging 2019;49:355-373.