Deep Generative Adversarial Neural Networks for Compressive Sensing MRI

Accelerating susceptibility-weighted imaging with deep learning by complex-valued convolutional neural network (ComplexNet): validation in clinical brain imaging

OBJECTIVES

Susceptibility-weighted imaging (SWI) is crucial for the characterization of intracranial hemorrhage and mineralization, but has the drawback of long acquisition times. We aimed to propose a deep learning model to accelerate SWI, and evaluate the clinical feasibility of this approach.

METHODS

A complex-valued convolutional neural network (ComplexNet) was developed to reconstruct high-quality SWI from highly accelerated k-space data. ComplexNet can leverage the inherently complex-valued nature of SWI data and learn richer representations by using complex-valued network. SWI data were acquired from 117 participants who underwent clinical brain MRI examination between 2019 and 2021, including patients with tumor, stroke, hemorrhage, traumatic brain injury, etc. Reconstruction quality was evaluated using quantitative image metrics and image quality scores, including overall image quality, signal-to-noise ratio, sharpness, and artifacts.

RESULTS

The average reconstruction time of ComplexNet was 19 ms per section (1.33 s per participant). ComplexNet achieved significantly improved quantitative image metrics compared to a conventional compressed sensing method and a real-valued network with acceleration rates of 5 and 8 (p < 0.001). Meanwhile, there was no significant difference between fully sampled and ComplexNet approaches in terms of overall image quality and artifacts (p > 0.05) at both acceleration rates. Furthermore, ComplexNet showed comparable diagnostic performance to the fully sampled SWI for visualizing a wide range of pathology, including hemorrhage, cerebral microbleeds, and brain tumor.

CONCLUSIONS

ComplexNet can effectively accelerate SWI while providing superior performance in terms of overall image quality and visualization of pathology for routine clinical brain imaging.

KEY POINTS

• The complex-valued convolutional neural network (ComplexNet) allowed fast and high-quality reconstruction of highly accelerated SWI data, with an average reconstruction time of 19 ms per section. • ComplexNet achieved significantly improved quantitative image metrics compared to a conventional compressed sensing method and a real-valued network with acceleration rates of 5 and 8 (p < 0.001). • ComplexNet showed comparable diagnostic performance to the fully sampled SWI for visualizing a wide range of pathology, including hemorrhage, cerebral microbleeds, and brain tumor.

A Review of Deep Learning Methods for Compressed Sensing Image Reconstruction and Its Medical Applications

Compressed sensing (CS) and its medical applications are active areas of research. In this paper, we review recent works using deep learning method to solve CS problem for images or medical imaging reconstruction including computed tomography (CT), magnetic resonance imaging (MRI) and positron-emission tomography (PET). We propose a novel framework to unify traditional iterative algorithms and deep learning approaches. In short, we define two projection operators toward image prior and data consistency, respectively, and any reconstruction algorithm can be decomposed to the two parts. Though deep learning methods can be divided into several categories, they all satisfies the framework. We built the relationship between different reconstruction methods of deep learning, and connect them to traditional methods through the proposed framework. It also indicates that the key to solve CS problem and its medical applications is how to depict the image prior. Based on the framework, we analyze the current deep learning methods and point out some important directions of research in the future.

Inpainted Image Reconstruction Using an Extended Hopfield Neural Network Based Machine Learning System

This paper considers the use of a machine learning system for the reconstruction and recognition of distorted or damaged patterns, in particular, images of faces partially covered with masks. The most up-to-date image reconstruction structures are based on constrained optimization algorithms and suitable regularizers. In contrast with the above-mentioned image processing methods, the machine learning system presented in this paper employs the superposition of system vectors setting up asymptotic centers of attraction. The structure of the system is implemented using Hopfield-type neural network-based biorthogonal transformations. The reconstruction property gives rise to a superposition processor and reversible computations. Moreover, this paper's distorted image reconstruction sets up associative memories where images stored in memory are retrieved by distorted/inpainted key images.

Undersampled Multi-contrast MRI Reconstruction Based on Double-domain Generative Adversarial Network

Multi-contrast magnetic resonance imaging can provide comprehensive information for clinical diagnosis. However, multi-contrast imaging suffers from long acquisition time, which makes it inhibitive for daily clinical practice. Subsampling k-space is one of the main methods to speed up scan time. Missing k-space samples will lead to inevitable serious artifacts and noise. Considering the assumption that different contrast modalities share some mutual information, it may be possible to exploit this redundancy to accelerate multi-contrast imaging acquisition. Recently, generative adversarial network shows superior performance in image reconstruction and synthesis. Some studies based on k-space reconstruction also exhibit superior performance over conventional state-of-art method. In this study, we propose a cross-domain two-stage generative adversarial network for multi-contrast images reconstruction based on prior full-sampled contrast and undersampled information. The new approach integrates reconstruction and synthesis, which estimates and completes the missing k-space and then refines in image space. It takes one fully-sampled contrast modality data and highly undersampled data from several other modalities as input, and outputs high quality images for each contrast simultaneously. The network is trained and tested on a public brain dataset from healthy subjects. Quantitative comparisons against baseline clearly indicate that the proposed method can effectively reconstruct undersampled images. Even under high acceleration, the network still can recover texture details and reduce artifacts.

A simultaneous multi-slice T2 mapping framework based on overlapping-echo detachment planar imaging and deep learning reconstruction

PURPOSE

Quantitative MRI (qMRI) is of great importance to clinical medicine and scientific research. However, most qMRI techniques are time-consuming and sensitive to motion, especially when a large 3D volume is imaged. To accelerate the acquisition, a framework is proposed to realize reliable simultaneous multi-slice T₂ mapping.

METHODS

The simultaneous multi-slice T₂ mapping framework is based on overlapping-echo detachment (OLED) planar imaging (dubbed SMS-OLED). Multi-slice overlapping-echo signals were generated by multiple excitation pulses together with echo-shifting gradients. The signals were excited and acquired with a single-channel coil. U-Net was used to reconstruct T₂ maps from the acquired overlapping-echo image.

RESULTS

Single-shot double-slice and two-shot triple-slice SMS-OLED scan schemes were designed according to the framework for evaluation. Simulations, water phantom, and in vivo rat brain experiments were carried out. Overlapping-echo signals were acquired, and T₂ maps were reconstructed and compared with references. The results demonstrate the superior performance of our method.

CONCLUSION

Two slices of T₂ maps can be obtained in a single shot within hundreds of milliseconds. Higher quality multi-slice T₂ maps can be obtained via multiple shots. SMS-OLED provides a lower specific absorption rate scheme compared with conventional SMS methods with a coil with only a single receiver channel. The new method is of potential in dynamic qMRI and functional qMRI where temporal resolution is vital.

2D probabilistic undersampling pattern optimization for MR image reconstruction

With 3D magnetic resonance imaging (MRI), a tradeoff exists between higher image quality and shorter scan time. One way to solve this problem is to reconstruct high-quality MRI images from undersampled k-space. There have been many recent studies exploring effective k-space undersampling patterns and designing MRI reconstruction methods from undersampled k-space, which are two necessary steps. Most studies separately considered these two steps, although in theory, their performance is dependent on each other. In this study, we propose a joint optimization model, trained end-to-end, to simultaneously optimize the undersampling pattern in the Fourier domain and the reconstruction model in the image domain. A 2D probabilistic undersampling layer was designed to optimize the undersampling pattern and probability distribution in a differentiable manner. A 2D inverse Fourier transform layer was implemented to connect the Fourier domain and the image domain during the forward and back propagation. Finally, we discovered an optimized relationship between the probability distribution of the undersampling pattern and its corresponding sampling rate. Further testing was performed using 3D T1-weighted MR images of the brain from the MICCAI 2013 Grand Challenge on Multi-Atlas Labeling dataset and locally acquired brain 3D T1-weighted MR images of healthy volunteers and contrast-enhanced 3D T1-weighted MR images of high-grade glioma patients. The results showed that the recovered MR images using our 2D probabilistic undersampling pattern (with or without the reconstruction network) significantly outperformed those using the existing start-of-the-art undersampling strategies for both qualitative and quantitative comparison, suggesting the advantages and some extent of the generalization of our proposed method.

Evaluation on the generalization of a learned convolutional neural network for MRI reconstruction

Recently, deep learning approaches with various network architectures have drawn significant attention from the magnetic resonance imaging (MRI) community because of their great potential for image reconstruction from undersampled k-space data in fast MRI. However, the robustness of a trained network when applied to test data deviated from training data is still an important open question. In this work, we focus on quantitatively evaluating the influence of image contrast, human anatomy, sampling pattern, undersampling factor, and noise level on the generalization of a trained network composed by a cascade of several CNNs and a data consistency layer, called a deep cascade of convolutional neural network (DC-CNN). The DC-CNN is trained from datasets with different image contrast, human anatomy, sampling pattern, undersampling factor, and noise level, and then applied to test datasets consistent or inconsistent with the training datasets to assess the generalizability of the learned DC-CNN network. The results of our experiments show that reconstruction quality from the DC-CNN network is highly sensitive to sampling pattern, undersampling factor, and noise level, which are closely related to signal-to-noise ratio (SNR), and is relatively less sensitive to the image contrast. We also show that a deviation of human anatomy between training and test data leads to a substantial reduction of image quality for the brain dataset, whereas comparable performance for the chest and knee dataset having fewer anatomy details than brain images. This work further provides some empirical understanding of the generalizability of trained networks when there are deviations between training and test data. It also demonstrates the potential of transfer learning for image reconstruction from datasets different from those used in training the network.

High-fidelity fast volumetric brain MRI using synergistic wave-controlled aliasing in parallel imaging and a hybrid denoising generative adversarial network (HDnGAN)

PURPOSE

The goal of this study is to leverage an advanced fast imaging technique, wave-controlled aliasing in parallel imaging (Wave-CAIPI), and a generative adversarial network (GAN) for denoising to achieve accelerated high-quality high-signal-to-noise-ratio (SNR) volumetric MRI.

METHODS

Three-dimensional (3D) T₂ -weighted fluid-attenuated inversion recovery (FLAIR) image data were acquired on 33 multiple sclerosis (MS) patients using a prototype Wave-CAIPI sequence (acceleration factor R = 3×2, 2.75 minutes) and a standard T₂ -SPACE FLAIR sequence (R = 2, 7.25 minutes). A hybrid denoising GAN entitled "HDnGAN" consisting of a 3D generator and a 2D discriminator was proposed to denoise highly accelerated Wave-CAIPI images. HDnGAN benefits from the improved image synthesis performance provided by the 3D generator and increased training samples from a limited number of patients for training the 2D discriminator. HDnGAN was trained and validated on data from 25 MS patients with the standard FLAIR images as the target and evaluated on data from 8 MS patients not seen during training. HDnGAN was compared to other denoising methods including AONLM, BM4D, MU-Net, and 3D GAN in qualitative and quantitative analysis of output images using the mean squared error (MSE) and VGG perceptual loss compared to standard FLAIR images, and a reader assessment by two neuroradiologists regarding sharpness, SNR, lesion conspicuity, and overall quality. Finally, the performance of these denoising methods was compared at higher noise levels using simulated data with added Rician noise.

RESULTS

HDnGAN effectively denoised low-SNR Wave-CAIPI images with sharpness and rich textural details, which could be adjusted by controlling the contribution of the adversarial loss to the total loss when training the generator. Quantitatively, HDnGAN (λ = 10^-3 ) achieved low MSE and the lowest VGG perceptual loss. The reader study showed that HDnGAN (λ = 10^-3 ) significantly improved the SNR of Wave-CAIPI images (P<0.001), outperformed AONLM (P = 0.015), BM4D (P<0.001), MU-Net (P<0.001) and 3D GAN (λ = 10^-3 ) (P<0.001) regarding image sharpness, and outperformed MU-Net (P<0.001) and 3D GAN (λ = 10^-3 ) (P = 0.001) regarding lesion conspicuity. The overall quality score of HDnGAN (λ = 10^-3 ) (4.25±0.43) was significantly higher than those from Wave-CAIPI (3.69±0.46, P = 0.003), BM4D (3.50±0.71, P = 0.001), MU-Net (3.25±0.75, P<0.001), and 3D GAN (λ = 10^-3 ) (3.50±0.50, P<0.001), with no significant difference compared to standard FLAIR images (4.38±0.48, P = 0.333). The advantages of HDnGAN over other methods were more obvious at higher noise levels.

CONCLUSION

HDnGAN provides robust and feasible denoising while preserving rich textural detail in empirical volumetric MRI data. Our study using empirical patient data and systematic evaluation supports the use of HDnGAN in combination with modern fast imaging techniques such as Wave-CAIPI to achieve high-fidelity fast volumetric MRI and represents an important step to the clinical translation of GANs. This article is protected by copyright. All rights reserved.

Compressed Sensing in Sodium Magnetic Resonance Imaging: Techniques, Applications, and Future Prospects

Sodium (²³ Na) yields the second strongest nuclear magnetic resonance (NMR) signal in biological tissues and plays a vital role in cell physiology. Sodium magnetic resonance imaging (MRI) can provide insights into cell integrity and tissue viability relative to pathologies without significant anatomical alternations, and thus it is considered to be a potential surrogate biomarker that provides complementary information for standard hydrogen (¹ H) MRI in a noninvasive and quantitative manner. However, sodium MRI suffers from a relatively low signal-to-noise ratio and long acquisition times due to its relatively low NMR sensitivity. Compressed sensing-based (CS-based) methods have been shown to accelerate sodium imaging and/or improve sodium image quality significantly. In this manuscript, the basic concepts of CS and how CS might be applied to improve sodium MRI are described, and the historical milestones of CS-based sodium MRI are briefly presented. Representative advanced techniques and evaluation methods are discussed in detail, followed by an expose of clinical applications in multiple anatomical regions and diseases as well as thoughts and suggestions on potential future research prospects of CS in sodium MRI. EVIDENCE LEVEL: 5 TECHNICAL EFFICACY: Stage 1.

Review and Prospect: Artificial Intelligence in Advanced Medical Imaging

Artificial intelligence (AI) as an emerging technology is gaining momentum in medical imaging. Recently, deep learning-based AI techniques have been actively investigated in medical imaging, and its potential applications range from data acquisition and image reconstruction to image analysis and understanding. In this review, we focus on the use of deep learning in image reconstruction for advanced medical imaging modalities including magnetic resonance imaging (MRI), computed tomography (CT), and positron emission tomography (PET). Particularly, recent deep learning-based methods for image reconstruction will be emphasized, in accordance with their methodology designs and performances in handling volumetric imaging data. It is expected that this review can help relevant researchers understand how to adapt AI for medical imaging and which advantages can be achieved with the assistance of AI.

Anisotropic neural deblurring for MRI acceleration

PURPOSE

MRI has become the tool of choice for brain imaging, providing unrivalled contrast between soft tissues, as well as a wealth of information about anatomy, function, and neurochemistry. Image quality, in terms of spatial resolution and noise, is strongly dependent on acquisition duration. A typical brain MRI scan may last several minutes, with total protocol duration often exceeding 30 minutes. Long scan duration leads to poor patient experience, long waiting time for appointments, and high costs. Therefore, shortening MRI scans is crucial. In this paper, we investigate the enhancement of low-resolution (LR) brain MRI scanning, to enable shorter acquisition times without compromising the diagnostic value of the images.

METHODS

We propose a novel fully convolutional neural enhancement approach. It is optimized for accelerated LR MRI acquisitions obtained by reducing the acquisition matrix size only along phase encoding direction. The network is trained to transform the LR acquisitions into corresponding high-resolution (HR) counterparts in an end-to-end manner. In contrast to previous neural-based MRI enhancement algorithms, such as DAGAN, the LR images used for training are real acquisitions rather than smoothed, downsampled versions of the HR images.

RESULTS

The proposed method is validated qualitatively and quantitatively for an acceleration factor of 4. Favourable comparison is demonstrated against the state-of-the-art DeblurGAN and DAGAN algorithms in terms of PSNR and SSIM scores. The result was further confirmed by an image quality rating experiment performed by four senior neuroradiologists.

CONCLUSIONS

The proposed method may become a valuable tool for scan time reduction in brain MRI. In continuation of this research, the validation should be extended to larger datasets acquired for different imaging protocols, and considering several MRI machines produced by different vendors.

Time-Dependent Deep Image Prior for Dynamic MRI

We propose a novel unsupervised deep-learning-based algorithm for dynamic magnetic resonance imaging (MRI) reconstruction. Dynamic MRI requires rapid data acquisition for the study of moving organs such as the heart. We introduce a generalized version of the deep-image-prior approach, which optimizes the weights of a reconstruction network to fit a sequence of sparsely acquired dynamic MRI measurements. Our method needs neither prior training nor additional data. In particular, for cardiac images, it does not require the marking of heartbeats or the reordering of spokes. The key ingredients of our method are threefold: 1) a fixed low-dimensional manifold that encodes the temporal variations of images; 2) a network that maps the manifold into a more expressive latent space; and 3) a convolutional neural network that generates a dynamic series of MRI images from the latent variables and that favors their consistency with the measurements in k -space. Our method outperforms the state-of-the-art methods quantitatively and qualitatively in both retrospective and real fetal cardiac datasets. To the best of our knowledge, this is the first unsupervised deep-learning-based method that can reconstruct the continuous variation of dynamic MRI sequences with high spatial resolution.

Artificial intelligence in image-guided radiotherapy: a review of treatment target localization

Modern conformal beam delivery techniques require image-guidance to ensure the prescribed dose to be delivered as planned. Recent advances in artificial intelligence (AI) have greatly augmented our ability to accurately localize the treatment target while sparing the normal tissues. In this paper, we review the applications of AI-based algorithms in image-guided radiotherapy (IGRT), and discuss the indications of these applications to the future of clinical practice of radiotherapy. The benefits, limitations and some important trends in research and development of the AI-based IGRT techniques are also discussed. AI-based IGRT techniques have the potential to monitor tumor motion, reduce treatment uncertainty and improve treatment precision. Particularly, these techniques also allow more healthy tissue to be spared while keeping tumor coverage the same or even better.

DeepSENSE: Learning coil sensitivity functions for SENSE reconstruction using deep learning

PURPOSE

To improve the estimation of coil sensitivity functions from limited auto-calibration signals (ACS) in SENSE-based reconstruction for brain imaging.

METHODS

We propose to use deep learning to estimate coil sensitivity functions by leveraging information from previous scans obtained using the same RF receiver system. Specifically, deep convolutional neural networks were designed to learn an end-to-end mapping from the initial sensitivity to the high-resolution counterpart. Sensitivity alignment was further proposed to reduce the geometric variation caused by different subject positions and imaging FOVs. Cross-validation with a small set of datasets was performed to validate the learned neural network. Iterative SENSE reconstruction was adopted to evaluate the utility of the sensitivity functions from the proposed and conventional methods.

RESULTS

The proposed method produced improved sensitivity estimates and SENSE reconstructions compared to the conventional methods in terms of aliasing and noise suppression with very limited ACS data. Cross-validation with a small set of data demonstrated the feasibility of learning coil sensitivity functions for brain imaging. The network learned on the spoiled GRE data can be applied to predict sensitivity functions for spin-echo and MPRAGE datasets.

CONCLUSION

A deep learning-based method has been proposed for improving the estimation of coil sensitivity functions. Experimental results have demonstrated the feasibility and potential of the proposed method for improving SENSE-based reconstructions especially when the ACS data are limited.

Robust brain MR image compressive sensing via re-weighted total variation and sparse regression

Total variation (TV) and non-local self-similarity (NSS) are powerful tools for successfully enhancing compressive sensing performance. However, standard TV approaches often over-smooth detailed edges in the image, due to the uniform regularization of gradient magnitude. In this paper, a novel compressed sensing method for the reconstruction of medical images is proposed, the image edges are well preserved with the proposed reweighted TV. The redundancy of the NSS patch also is leveraged through the sparse regression model. The proposed model was solved with an efficient strategy of the Alternating Direction Method of Multipliers (ADMM) algorithm. Experimental results on thesimulated phantom, brain Magnetic resonance imaging (MRI) show that the proposed method outperforms the state-of-the-art compressed sensing approaches.

Learning Data Consistency and its Application to Dynamic MR Imaging

Magnetic resonance (MR) image reconstruction from undersampled k-space data can be formulated as a minimization problem involving data consistency and image prior. Existing deep learning (DL)-based methods for MR reconstruction employ deep networks to exploit the prior information and integrate the prior knowledge into the reconstruction under the explicit constraint of data consistency, without considering the real distribution of the noise. In this work, we propose a new DL-based approach termed Learned DC that implicitly learns the data consistency with deep networks, corresponding to the actual probability distribution of system noise. The data consistency term and the prior knowledge are both embedded in the weights of the networks, which provides an utterly implicit manner of learning reconstruction model. We evaluated the proposed approach with highly undersampled dynamic data, including the dynamic cardiac cine data with up to 24-fold acceleration and dynamic rectum data with the acceleration factor equal to the number of phases. Experimental results demonstrate the superior performance of the Learned DC both quantitatively and qualitatively than the state-of-the-art.

Upstream Machine Learning in Radiology

Machine learning (ML) and Artificial intelligence (AI) has the potential to dramatically improve radiology practice at multiple stages of the imaging pipeline. Most of the attention has been garnered by applications focused on improving the end of the pipeline: image interpretation. However, this article reviews how AI/ML can be applied to improve upstream components of the imaging pipeline, including exam modality selection, hardware design, exam protocol selection, data acquisition, image reconstruction, and image processing. A breadth of applications and their potential for impact is shown across multiple imaging modalities, including ultrasound, computed tomography, and MRI.

Blind Primed Supervised (BLIPS) Learning for MR Image Reconstruction

This paper examines a combined supervised-unsupervised framework involving dictionary-based blind learning and deep supervised learning for MR image reconstruction from under-sampled k-space data. A major focus of the work is to investigate the possible synergy of learned features in traditional shallow reconstruction using adaptive sparsity-based priors and deep prior-based reconstruction. Specifically, we propose a framework that uses an unrolled network to refine a blind dictionary learning-based reconstruction. We compare the proposed method with strictly supervised deep learning-based reconstruction approaches on several datasets of varying sizes and anatomies. We also compare the proposed method to alternative approaches for combining dictionary-based methods with supervised learning in MR image reconstruction. The improvements yielded by the proposed framework suggest that the blind dictionary-based approach preserves fine image details that the supervised approach can iteratively refine, suggesting that the features learned using the two methods are complementary.

Vessel Wall MR Imaging in the Pediatric Head and Neck

Vessel wall MR imaging (VWI) is a technique that progressively has gained traction in clinical diagnostic applications for evaluation of intracranial and extracranial vasculopathies, with increasing use in pediatric populations. The technique has shown promise in detection, differentiation, and characterization of both inflammatory and noninflammatory vasculopathies. In this article, optimal techniques for intracranial and extracranial VWI as well as applications and value for pediatric vascular disease evaluation are discussed.

Feasibility of Brain Imaging Using a Digital Surround Technology Body Coil: A Study Based on SRGAN-VGG Convolutional Neural Networks. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2021;2021:3734-3737. [PMID: 34892048 DOI: 10.1109/embc46164.2021.9630816] [Citation(s) in RCA: 0] [Impact Index Per Article: 0]

Brain imaging using conventional head coils presents several problems in routine magnetic resonance (MR) examination, such as anxiety and claustrophobic reactions during scanning with a head coil, photon attenuation caused by the MRI head coil in positron emission tomography (PET)/MRI, and coil constraints in intraoperative MRI or MRI-guided radiotherapy. In this paper, we propose a super resolution generative adversarial (SRGAN-VGG) network-based approach to enhance low-quality brain images scanned with body coils. Two types of T1 fluid-attenuated inversion recovery (FLAIR) images scanned with different coils were obtained in this study: joint images of the head-neck coil and digital surround technology body coil (H+B images) and body coil images (B images). The deep learning (DL) model was trained using images acquired from 36 subjects and tested in 4 subjects. Both quantitative and qualitative image quality assessment methods were performed during evaluation. Wilcoxon signed-rank tests were used for statistical analysis. Quantitative image quality assessment showed an improved structural similarity index (SSIM) and peak signal-to-noise ratio (PSNR) in gray matter and cerebrospinal fluid (CSF) tissues for DL images compared with B images (P <.01), while the mean square error (MSE) was significantly decreased (P <.05). The analysis also showed that the natural image quality evaluator (NIQE) and blind image quality index (BIQI) were significantly lower for DL images than for B images (P <.0001). Qualitative scoring results indicated that DL images showed an improved SNR, image contrast and sharpness (P<.0001). The outcomes of this study preliminarily indicate that body coils can be used in brain imaging, making it possible to expand the application of MR-based brain imaging.

Compressed Sensing MRI with ℓ1-Wavelet Reconstruction Revisited Using Modern Data Science Tools

Deep learning (DL) has emerged as a powerful tool for improving the reconstruction quality of accelerated MRI. These methods usually show enhanced performance compared to conventional methods, such as compressed sensing (CS) and parallel imaging. However, in most scenarios, CS is implemented with two or three empirically-tuned hyperparameters, while a plethora of advanced data science tools are used in DL. In this work, we revisit ℓ1 -wavelet CS for accelerated MRI using modern data science tools. By using tools like algorithm unrolling and end-to-end training with stochastic gradient descent over large databases that DL algorithms utilize, and combining these with conventional concepts like wavelet sub-band processing and reweighted ℓ1 minimization, we show that ℓ1-wavelet CS can be fine-tuned to a level comparable to DL methods. While DL uses hundreds of thousands of parameters, the proposed optimized ℓ1-wavelet CS with sub-band training and reweighting uses only 128 parameters, and employs a fully-explainable convex reconstruction model.

Enhancement-constrained acceleration: A robust reconstruction framework in breast DCE-MRI

In patients with dense breasts or at high risk of breast cancer, dynamic contrast enhanced MRI (DCE-MRI) is a highly sensitive diagnostic tool. However, its specificity is highly variable and sometimes low; quantitative measurements of contrast uptake parameters may improve specificity and mitigate this issue. To improve diagnostic accuracy, data need to be captured at high spatial and temporal resolution. While many methods exist to accelerate MRI temporal resolution, not all are optimized to capture breast DCE-MRI dynamics. We propose a novel, flexible, and powerful framework for the reconstruction of highly-undersampled DCE-MRI data: enhancement-constrained acceleration (ECA). Enhancement-constrained acceleration uses an assumption of smooth enhancement at small time-scale to estimate points of smooth enhancement curves in small time intervals at each voxel. This method is tested in silico with physiologically realistic virtual phantoms, simulating state-of-the-art ultrafast acquisitions at 3.5s temporal resolution reconstructed at 0.25s temporal resolution (demo code available here). Virtual phantoms were developed from real patient data and parametrized in continuous time with arterial input function (AIF) models and lesion enhancement functions. Enhancement-constrained acceleration was compared to standard ultrafast reconstruction in estimating the bolus arrival time and initial slope of enhancement from reconstructed images. We found that the ECA method reconstructed images at 0.25s temporal resolution with no significant loss in image fidelity, a 4x reduction in the error of bolus arrival time estimation in lesions (p < 0.01) and 11x error reduction in blood vessels (p < 0.01). Our results suggest that ECA is a powerful and versatile tool for breast DCE-MRI.

Deep Convolutional Neural Network for Compressive Sensing of Magnetic Resonance Images

Compressive Sensing for Magnetic Resonance Imaging (CS-MRI) aims to reconstruct Magnetic Resonance (MR) images from under-sampled raw data. There are two challenges to improve CS-MRI methods, i.e. designing an under-sampling algorithm to achieve optimal sampling, as well as designing fast and small deep neural networks to obtain reconstructed MR images with superior quality. To improve the reconstruction quality of MR images, we propose a novel deep convolutional neural network architecture for CS-MRI named MRCSNet. The MRCSNet consists of three sub-networks, a compressive sensing sampling sub-network, an initial reconstruction sub-network, and a refined reconstruction sub-network. Experimental results demonstrate that MRCSNet generates high-quality reconstructed MR images at various under-sampling ratios, and also meets the requirements of real-time CS-MRI applications. Compared to state-of-the-art CS-MRI approaches, MRCSNet offers a significant improvement in reconstruction accuracies, such as Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity (SSIM). Besides, it reduces the reconstruction error evaluated by the Normalized Root-Mean-Square Error (NRMSE). The source codes are available at https://github.com/TaihuLight/MRCSNet .

Efficient Computer-Aided Design of Dental Inlay Restoration: A Deep Adversarial Framework

Restoring the normal masticatory function of broken teeth is a challenging task primarily due to the defect location and size of a patient's teeth. In recent years, although some representative image-to-image transformation methods (e.g. Pix2Pix) can be potentially applicable to restore the missing crown surface, most of them fail to generate dental inlay surface with realistic crown details (e.g. occlusal groove) that are critical to the restoration of defective teeth with varying shapes. In this article, we design a computer-aided Deep Adversarial-driven dental Inlay reStoration (DAIS) framework to automatically reconstruct a realistic surface for a defective tooth. Specifically, DAIS consists of a Wasserstein generative adversarial network (WGAN) with a specially designed loss measurement, and a new local-global discriminator mechanism. The local discriminator focuses on missing regions to ensure the local consistency of a generated occlusal surface, while the global discriminator aims at defective teeth and adjacent teeth to assess if it is coherent as a whole. Experimental results demonstrate that DAIS is highly efficient to deal with a large area of missing teeth in arbitrary shapes and generate realistic occlusal surface completion. Moreover, the designed watertight inlay prostheses have enough anatomical morphology, thus providing higher clinical applicability compared with more state-of-the-art methods.

Accurate and robust sparse-view angle CT image reconstruction using deep learning and prior image constrained compressed sensing (DL-PICCS)

BACKGROUND

Sparse-view CT image reconstruction problems encountered in dynamic CT acquisitions are technically challenging. Recently, many deep learning strategies have been proposed to reconstruct CT images from sparse-view angle acquisitions showing promising results. However, two fundamental problems with these deep learning reconstruction methods remain to be addressed: (1) limited reconstruction accuracy for individual patients and (2) limited generalizability for patient statistical cohorts.

PURPOSE

The purpose of this work is to address the previously mentioned challenges in current deep learning methods.

METHODS

A method that combines a deep learning strategy with prior image constrained compressed sensing (PICCS) was developed to address these two problems. In this method, the sparse-view CT data were reconstructed by the conventional filtered backprojection (FBP) method first, and then processed by the trained deep neural network to eliminate streaking artifacts. The outputs of the deep learning architecture were then used as the needed prior image in PICCS to reconstruct the image. If the noise level from the PICCS reconstruction is not satisfactory, another light duty deep neural network can then be used to reduce noise level. Both extensive numerical simulation data and human subject data have been used to quantitatively and qualitatively assess the performance of the proposed DL-PICCS method in terms of reconstruction accuracy and generalizability.

RESULTS

Extensive evaluation studies have demonstrated that: (1) quantitative reconstruction accuracy of DL-PICCS for individual patient is improved when it is compared with the deep learning methods and CS-based methods; (2) the false-positive lesion-like structures and false negative missing anatomical structures in the deep learning approaches can be effectively eliminated in the DL-PICCS reconstructed images; and (3) DL-PICCS enables a deep learning scheme to relax its working conditions to enhance its generalizability.

CONCLUSIONS

DL-PICCS offers a promising opportunity to achieve personalized reconstruction with improved reconstruction accuracy and enhanced generalizability.

Improvement of megavoltage computed tomography image quality for adaptive helical tomotherapy using cycleGAN-based image synthesis with small datasets

PURPOSE

Megavoltage computed tomography (MVCT) offers an opportunity for adaptive helical tomotherapy. However, high noise and reduced contrast in the MVCT images due to a decrease in the imaging dose to patients limits its usability. Therefore, we propose an algorithm to improve the image quality of MVCT.

METHODS

The proposed algorithm generates kilovoltage CT (kVCT)-like images from MVCT images using a cycle-consistency generative adversarial network (cycleGAN)-based image synthesis model. Data augmentation using an affine transformation was applied to the training data to overcome the lack of data diversity in the network training. The mean absolute error (MAE), root-mean-square error (RMSE), peak signal-to-noise ratio (PSNR), and structural similarity index measure (SSIM) were used to quantify the correction accuracy of the images generated by the proposed algorithm. The proposed method was validated by comparing the images generated with those obtained from conventional and deep learning-based image processing method through non-augmented datasets.

RESULTS

The average MAE, RMSE, PSNR, and SSIM values were 18.91 HU, 69.35 HU, 32.73 dB, and 95.48 using the proposed method, respectively, whereas cycleGAN with non-augmented data showed inferior results (19.88 HU, 70.55 HU, 32.62 dB, 95.19, respectively). The voxel values of the image obtained by the proposed method also indicated similar distributions to those of the kVCT image. The dose-volume histogram of the proposed method was also similar to that of electron density corrected MVCT.

CONCLUSIONS

The proposed algorithm generates synthetic kVCT images from MVCT images using cycleGAN with small patient datasets. The image quality achieved by the proposed method was correspondingly improved to the level of a kVCT image while maintaining the anatomical structure of an MVCT image. The evaluation of dosimetric effectiveness of the proposed method indicates the applicability of accurate treatment planning in adaptive radiation therapy.

Feasibility and Implementation of a Deep Learning MR Reconstruction for TSE Sequences in Musculoskeletal Imaging

Magnetic Resonance Imaging (MRI) of the musculoskeletal system is one of the most common examinations in clinical routine. The application of Deep Learning (DL) reconstruction for MRI is increasingly gaining attention due to its potential to improve the image quality and reduce the acquisition time simultaneously. However, the technology has not yet been implemented in clinical routine for turbo spin echo (TSE) sequences in musculoskeletal imaging. The aim of this study was therefore to assess the technical feasibility and evaluate the image quality. Sixty examinations of knee, hip, ankle, shoulder, hand, and lumbar spine in healthy volunteers at 3 T were included in this prospective, internal-review-board-approved study. Conventional (TSE_S) and DL-based TSE sequences (TSE_DL) were compared regarding image quality, anatomical structures, and diagnostic confidence. Overall image quality was rated to be excellent, with a significant improvement in edge sharpness and reduced noise compared to TSE_S (p < 0.001). No difference was found concerning the extent of artifacts, the delineation of anatomical structures, and the diagnostic confidence comparing TSE_S and TSE_DL (p > 0.05). Therefore, DL image reconstruction for TSE sequences in MSK imaging is feasible, enabling a remarkable time saving (up to 75%), whilst maintaining excellent image quality and diagnostic confidence.

A Survey of Soft Computing Approaches in Biomedical Imaging

Medical imaging is an essential technique for the diagnosis and treatment of diseases in modern clinics. Soft computing plays a major role in the recent advances in medical imaging. It handles uncertainties and improves the qualities of an image. Until now, various soft computing approaches have been proposed for medical applications. This paper discusses various medical imaging modalities and presents a short review of soft computing approaches such as fuzzy logic, artificial neural network, genetic algorithm, machine learning, and deep learning. We also studied and compared each approach used for other imaging modalities based on the certain parameter used for the system evaluation. Finally, based on comparative analysis, the possible research strategies for further development are proposed. As far as we know, no previous work examined this issue.

Prospective Deployment of Deep Learning in MRI: A Framework for Important Considerations, Challenges, and Recommendations for Best Practices

Artificial intelligence algorithms based on principles of deep learning (DL) have made a large impact on the acquisition, reconstruction, and interpretation of MRI data. Despite the large number of retrospective studies using DL, there are fewer applications of DL in the clinic on a routine basis. To address this large translational gap, we review the recent publications to determine three major use cases that DL can have in MRI, namely, that of model-free image synthesis, model-based image reconstruction, and image or pixel-level classification. For each of these three areas, we provide a framework for important considerations that consist of appropriate model training paradigms, evaluation of model robustness, downstream clinical utility, opportunities for future advances, as well recommendations for best current practices. We draw inspiration for this framework from advances in computer vision in natural imaging as well as additional healthcare fields. We further emphasize the need for reproducibility of research studies through the sharing of datasets and software. LEVEL OF EVIDENCE: 5 TECHNICAL EFFICACY STAGE: 2.

Temporally aware volumetric generative adversarial network-based MR image reconstruction with simultaneous respiratory motion compensation: Initial feasibility in 3D dynamic cine cardiac MRI

PURPOSE

Develop a novel three-dimensional (3D) generative adversarial network (GAN)-based technique for simultaneous image reconstruction and respiratory motion compensation of 4D MRI. Our goal was to enable high-acceleration factors 10.7X-15.8X, while maintaining robust and diagnostic image quality superior to state-of-the-art self-gating (SG) compressed sensing wavelet (CS-WV) reconstruction at lower acceleration factors 3.5X-7.9X.

METHODS

Our GAN was trained based on pixel-wise content loss functions, adversarial loss function, and a novel data-driven temporal aware loss function to maintain anatomical accuracy and temporal coherence. Besides image reconstruction, our network also performs respiratory motion compensation for free-breathing scans. A novel progressive growing-based strategy was adapted to make the training process possible for the proposed GAN-based structure. The proposed method was developed and thoroughly evaluated qualitatively and quantitatively based on 3D cardiac cine data from 42 patients.

RESULTS

Our proposed method achieved significantly better scores in general image quality and image artifacts at 10.7X-15.8X acceleration than the SG CS-WV approach at 3.5X-7.9X acceleration (4.53 ± 0.540 vs. 3.13 ± 0.681 for general image quality, 4.12 ± 0.429 vs. 2.97 ± 0.434 for image artifacts, P < .05 for both). No spurious anatomical structures were observed in our images. The proposed method enabled similar cardiac-function quantification as conventional SG CS-WV. The proposed method achieved faster central processing unit-based image reconstruction (6 s/cardiac phase) than the SG CS-WV (312 s/cardiac phase).

CONCLUSION

The proposed method showed promising potential for high-resolution (1 mm³ ) free-breathing 4D MR data acquisition with simultaneous respiratory motion compensation and fast reconstruction time.

Complementary time-frequency domain networks for dynamic parallel MR image reconstruction

PURPOSE

To introduce a novel deep learning-based approach for fast and high-quality dynamic multicoil MR reconstruction by learning a complementary time-frequency domain network that exploits spatiotemporal correlations simultaneously from complementary domains.

THEORY AND METHODS

Dynamic parallel MR image reconstruction is formulated as a multivariable minimization problem, where the data are regularized in combined temporal Fourier and spatial (x-f) domain as well as in spatiotemporal image (x-t) domain. An iterative algorithm based on variable splitting technique is derived, which alternates among signal de-aliasing steps in x-f and x-t spaces, a closed-form point-wise data consistency step and a weighted coupling step. The iterative model is embedded into a deep recurrent neural network which learns to recover the image via exploiting spatiotemporal redundancies in complementary domains.

RESULTS

Experiments were performed on two datasets of highly undersampled multicoil short-axis cardiac cine MRI scans. Results demonstrate that our proposed method outperforms the current state-of-the-art approaches both quantitatively and qualitatively. The proposed model can also generalize well to data acquired from a different scanner and data with pathologies that were not seen in the training set.

CONCLUSION

The work shows the benefit of reconstructing dynamic parallel MRI in complementary time-frequency domains with deep neural networks. The method can effectively and robustly reconstruct high-quality images from highly undersampled dynamic multicoil data ( 16 × and 24 × yielding 15 s and 10 s scan times respectively) with fast reconstruction speed (2.8 seconds). This could potentially facilitate achieving fast single-breath-hold clinical 2D cardiac cine imaging.

Deep learning in magnetic resonance image reconstruction

Magnetic resonance (MR) imaging visualises soft tissue contrast in exquisite detail without harmful ionising radiation. In this work, we provide a state-of-the-art review on the use of deep learning in MR image reconstruction from different image acquisition types involving compressed sensing techniques, parallel image acquisition and multi-contrast imaging. Publications with deep learning-based image reconstruction for MR imaging were identified from the literature (PubMed and Google Scholar), and a comprehensive description of each of the works was provided. A detailed comparison that highlights the differences, the data used and the performance of each of these works were also made. A discussion of the potential use cases for each of these methods is provided. The sparse image reconstruction methods were found to be most popular in using deep learning for improved performance, accelerating acquisitions by around 4-8 times. Multi-contrast image reconstruction methods rely on at least one pre-acquired image, but can achieve 16-fold, and even up to 32- to 50-fold acceleration depending on the set-up. Parallel imaging provides frameworks to be integrated in many of these methods for additional speed-up potential. The successful use of compressed sensing techniques and multi-contrast imaging with deep learning and parallel acquisition methods could yield significant MR acquisition speed-ups within clinical routines in the near future.

Deep Convolutional Neural Network With Adversarial Training for Denoising Digital Breast Tomosynthesis Images

Digital breast tomosynthesis (DBT) is a quasi-three-dimensional imaging modality that can reduce false negatives and false positives in mass lesion detection caused by overlapping breast tissue in conventional two-dimensional (2D) mammography. The patient dose of a DBT scan is similar to that of a single 2D mammogram, while acquisition of each projection view adds detector readout noise. The noise is propagated to the reconstructed DBT volume, possibly obscuring subtle signs of breast cancer such as microcalcifications (MCs). This study developed a deep convolutional neural network (DCNN) framework for denoising DBT images with a focus on improving the conspicuity of MCs as well as preserving the ill-defined margins of spiculated masses and normal tissue textures. We trained the DCNN using a weighted combination of mean squared error (MSE) loss and adversarial loss. We configured a dedicated x-ray imaging simulator in combination with digital breast phantoms to generate realistic in silico DBT data for training. We compared the DCNN training between using digital phantoms and using real physical phantoms. The proposed denoising method improved the contrast-to-noise ratio (CNR) and detectability index (d') of the simulated MCs in the validation phantom DBTs. These performance measures improved with increasing training target dose and training sample size. Promising denoising results were observed on the transferability of the digital-phantom-trained denoiser to DBT reconstructed with different techniques and on a small independent test set of human subject DBT images.

Transfer learning enhanced generative adversarial networks for multi-channel MRI reconstruction

Deep learning based generative adversarial networks (GAN) can effectively perform image reconstruction with under-sampled MR data. In general, a large number of training samples are required to improve the reconstruction performance of a certain model. However, in real clinical applications, it is difficult to obtain tens of thousands of raw patient data to train the model since saving k-space data is not in the routine clinical flow. Therefore, enhancing the generalizability of a network based on small samples is urgently needed. In this study, three novel applications were explored based on parallel imaging combined with the GAN model (PI-GAN) and transfer learning. The model was pre-trained with public Calgary brain images and then fine-tuned for use in (1) patients with tumors in our center; (2) different anatomies, including knee and liver; (3) different k-space sampling masks with acceleration factors (AFs) of 2 and 6. As for the brain tumor dataset, the transfer learning results could remove the artifacts found in PI-GAN and yield smoother brain edges. The transfer learning results for the knee and liver were superior to those of the PI-GAN model trained with its own dataset using a smaller number of training cases. However, the learning procedure converged more slowly in the knee datasets compared to the learning in the brain tumor datasets. The reconstruction performance was improved by transfer learning both in the models with AFs of 2 and 6. Of these two models, the one with AF = 2 showed better results. The results also showed that transfer learning with the pre-trained model could solve the problem of inconsistency between the training and test datasets and facilitate generalization to unseen data.

Which GAN? A comparative study of generative adversarial network-based fast MRI reconstruction

Fast magnetic resonance imaging (MRI) is crucial for clinical applications that can alleviate motion artefacts and increase patient throughput. K-space undersampling is an obvious approach to accelerate MR acquisition. However, undersampling of k-space data can result in blurring and aliasing artefacts for the reconstructed images. Recently, several studies have been proposed to use deep learning-based data-driven models for MRI reconstruction and have obtained promising results. However, the comparison of these methods remains limited because the models have not been trained on the same datasets and the validation strategies may be different. The purpose of this work is to conduct a comparative study to investigate the generative adversarial network (GAN)-based models for MRI reconstruction. We reimplemented and benchmarked four widely used GAN-based architectures including DAGAN, ReconGAN, RefineGAN and KIGAN. These four frameworks were trained and tested on brain, knee and liver MRI images using twofold, fourfold and sixfold accelerations, respectively, with a random undersampling mask. Both quantitative evaluations and qualitative visualization have shown that the RefineGAN method has achieved superior performance in reconstruction with better accuracy and perceptual quality compared to other GAN-based methods. This article is part of the theme issue 'Synergistic tomographic image reconstruction: part 1'.

Free-breathing Accelerated Cardiac MRI Using Deep Learning: Validation in Children and Young Adults

Background Obtaining ventricular volumetry and mass is key to most cardiac MRI but challenged by long multibreath-hold acquisitions. Purpose To assess the image quality and performance of a highly accelerated, free-breathing, two-dimensional cine cardiac MRI sequence incorporating deep learning (DL) reconstruction compared with reference standard balanced steady-state free precession (bSSFP). Materials and Methods A DL algorithm was developed to reconstruct custom 12-fold accelerated bSSFP cardiac MRI cine images from coil sensitivity maps using 15 iterations of separable three-dimensional convolutions and data consistency steps. The model was trained, validated, and internally tested in 10, two, and 10 adult human volunteers, respectively, based on vendor partner-supplied fully sampled bSSFP acquisitions. For prospective external clinical validation, consecutive children and young adults undergoing cardiac MRI from September through December 2019 at a single children's hospital underwent both conventional and highly accelerated short-axis bSSFP cine acquisitions in one MRI examination. Two radiologists scored overall and volumetric three-dimensional mesh image quality of all short-axis stacks on a five-point Likert scale and manually segmented endocardial and epicardial contours. Scan times and image quality were compared using the Wilcoxon rank sum test. Measurement agreement was assessed with intraclass correlation coefficient and Bland-Altman analysis. Results Fifty participants (mean age, 16 years ± 4 [standard deviation]; range, 5-30 years; 29 men) were evaluated. The mean prescribed acquisition times of accelerated scans (non-breath-held) and bSSFP (excluding breath-hold time) were 0.9 minute ± 0.3 versus 3.0 minutes ± 1.9 (P < .001). Overall and three-dimensional mesh image quality scores were, respectively, 3.8 ± 0.6 versus 4.3 ± 0.6 (P < .001) and 4.0 ± 1.0 versus 4.4 ± 0.8 (P < .001). Raters had strong agreement between all bSSFP and DL measurements, with intraclass correlation coefficients of 0.76 to 0.97, near-zero mean differences, and narrow limits of agreement. Conclusion With slightly lower image quality yet much faster speed, deep learning reconstruction may allow substantially shorter acquisition times of cardiac MRI compared with conventional balanced steady-state free precession MRI performed for ventricular volumetry. © RSNA, 2021 Online supplemental material is available for this article.

Systematic evaluation of iterative deep neural networks for fast parallel MRI reconstruction with sensitivity-weighted coil combination

PURPOSE

To systematically investigate the influence of various data consistency layers and regularization networks with respect to variations in the training and test data domain, for sensitivity-encoded accelerated parallel MR image reconstruction.

THEORY AND METHODS

Magnetic resonance (MR) image reconstruction is formulated as a learned unrolled optimization scheme with a down-up network as regularization and varying data consistency layers. The proposed networks are compared to other state-of-the-art approaches on the publicly available fastMRI knee and neuro dataset and tested for stability across different training configurations regarding anatomy and number of training samples.

RESULTS

Data consistency layers and expressive regularization networks, such as the proposed down-up networks, form the cornerstone for robust MR image reconstruction. Physics-based reconstruction networks outperform post-processing methods substantially for R = 4 in all cases and for R = 8 when the training and test data are aligned. At R = 8, aligning training and test data is more important than architectural choices.

CONCLUSION

In this work, we study how dataset sizes affect single-anatomy and cross-anatomy training of neural networks for MRI reconstruction. The study provides insights into the robustness, properties, and acceleration limits of state-of-the-art networks, and our proposed down-up networks. These key insights provide essential aspects to successfully translate learning-based MRI reconstruction to clinical practice, where we are confronted with limited datasets and various imaged anatomies.

Magnetic resonance parameter mapping using model-guided self-supervised deep learning

PURPOSE

To develop a model-guided self-supervised deep learning MRI reconstruction framework called reference-free latent map extraction (RELAX) for rapid quantitative MR parameter mapping.

METHODS

Two physical models are incorporated for network training in RELAX, including the inherent MR imaging model and a quantitative model that is used to fit parameters in quantitative MRI. By enforcing these physical model constraints, RELAX eliminates the need for full sampled reference data sets that are required in standard supervised learning. Meanwhile, RELAX also enables direct reconstruction of corresponding MR parameter maps from undersampled k-space. Generic sparsity constraints used in conventional iterative reconstruction, such as the total variation constraint, can be additionally included in the RELAX framework to improve reconstruction quality. The performance of RELAX was tested for accelerated T1 and T2 mapping in both simulated and actually acquired MRI data sets and was compared with supervised learning and conventional constrained reconstruction for suppressing noise and/or undersampling-induced artifacts.

RESULTS

In the simulated data sets, RELAX generated good T1 /T2 maps in the presence of noise and/or undersampling artifacts, comparable to artifact/noise-free ground truth. The inclusion of a spatial total variation constraint helps improve image quality. For the in vivo T1 /T2 mapping data sets, RELAX achieved superior reconstruction quality compared with conventional iterative reconstruction, and similar reconstruction performance to supervised deep learning reconstruction.

CONCLUSION

This work has demonstrated the initial feasibility of rapid quantitative MR parameter mapping based on self-supervised deep learning. The RELAX framework may also be further extended to other quantitative MRI applications by incorporating corresponding quantitative imaging models.

Using 5D flow MRI to decode the effects of rhythm on left atrial 3D flow dynamics in patients with atrial fibrillation

PURPOSE

This study used a 5D flow framework to explore the influence of arrhythmia on thrombogenic hemodynamic parameters in patients with atrial fibrillation (AF).

METHODS

A fully self-gated, 3D radial, highly accelerated free-running 5D flow sequence with interleaved four-point velocity-encoding was acquired using an in vitro arrhythmic flow phantom and in 25 patients with a history of AF (68 ± 8 y, 6 female). Self-gating signals were used to calculate AF burden, bin data, and tag each k-space line with its RRLength . Data were binned as an RR-resolved dataset with four RR-interval bins (RR1-RR4, short-to-long) for compressed sensing reconstruction. AF burden was calculated as interquartile range of all intrascan RR-intervals divided by median RR-interval, and left atrial (LA) stasis as the percent of the cardiac cycle where the velocity was <0.1 m/s.

RESULTS

In vitro results demonstrated successful recovery of RR-binned flow curves using RR-resolved 5D flow compared to a real-time PC reference standard. In vivo, 5D flow was acquired in 8:48 minutes. AF burden was significantly correlated with 5D flow-derived peak (PV) and mean (MV) velocity and stasis (|ρ| = 0.54-0.75, P < .001). Sensitivity analyses determined a threshold for low versus high AF burden at 9.7%. High burden patients had increased LA mean stasis (up to +42%, P < .01), and lower MV and PV (-30%, -40.6%, respectively, P < .01). RR4 deviated furthest from respiratory-resolved reconstruction (end-expiration) with increased mean stasis (7.6% ± 14.0%, P = .10) and decreased PV (-12.7 ± 14.2%, P = .09).

CONCLUSIONS

RR-resolved 5D flow can capture temporal and RR-resolved 3D hemodynamics in <10 minutes and offers a novel approach to investigate arrhythmias.

Adversarial Data Augmentation on Breast MRI Segmentation

The scarcity of balanced and annotated datasets has been a recurring problem in medical image analysis. Several researchers have tried to fill this gap employing dataset synthesis with adversarial networks (GANs). Breast magnetic resonance imaging (MRI) provides complex, texture-rich medical images, with the same annotation shortage issues, for which, to the best of our knowledge, no previous work tried synthesizing data. Within this context, our work addresses the problem of synthesizing breast MRI images from corresponding annotations and evaluate the impact of this data augmentation strategy on a semantic segmentation task. We explored variations of image-to-image translation using conditional GANs, namely fitting the generator’s architecture with residual blocks and experimenting with cycle consistency approaches. We studied the impact of these changes on visual verisimilarity and how an U-Net segmentation model is affected by the usage of synthetic data. We achieved sufficiently realistic-looking breast MRI images and maintained a stable segmentation score even when completely replacing the dataset with the synthetic set. Our results were promising, especially when concerning to Pix2PixHD and Residual CycleGAN architectures.

Narrative review of generative adversarial networks in medical and molecular imaging

Recent years have witnessed a rapidly expanding use of artificial intelligence and machine learning in medical imaging. Generative adversarial networks (GANs) are techniques to synthesize images based on artificial neural networks and deep learning. In addition to the flexibility and versatility inherent in deep learning on which the GANs are based, the potential problem-solving ability of the GANs has attracted attention and is being vigorously studied in the medical and molecular imaging fields. Here this narrative review provides a comprehensive overview for GANs and discuss their usefulness in medical and molecular imaging on the following topics: (I) data augmentation to increase training data for AI-based computer-aided diagnosis as a solution for the data-hungry nature of such training sets; (II) modality conversion to complement the shortcomings of a single modality that reflects certain physical measurement principles, such as from magnetic resonance (MR) to computed tomography (CT) images or vice versa; (III) de-noising to realize less injection and/or radiation dose for nuclear medicine and CT; (IV) image reconstruction for shortening MR acquisition time while maintaining high image quality; (V) super-resolution to produce a high-resolution image from low-resolution one; (VI) domain adaptation which utilizes knowledge such as supervised labels and annotations from a source domain to the target domain with no or insufficient knowledge; and (VII) image generation with disease severity and radiogenomics. GANs are promising tools for medical and molecular imaging. The progress of model architectures and their applications should continue to be noteworthy.

Compressed medical imaging based on average sparsity model and reweighted analysis of multiple basis pursuit

In medical imaging and applications, efficient image sampling and transfer are some of the key fields of research. The compressed sensing (CS) theory has shown that such compression can be performed during the data retrieval process and that the uncompressed image can be retrieved using a computationally flexible optimization method. The objective of this study is to propose compressed medical imaging for a different type of medical images, based on the combination of the average sparsity model and reweighted analysis of multiple basis pursuit (M-BP) reconstruction methods, referred to as multiple basis reweighted analysis (M-BRA). The proposed algorithm includes the joint multiple sparsity averaging to improves the signal sparsity in M-BP. In this study, four types of medical images are opted to fill the gap of lacking a detailed analysis of M-BRA in medical images. The medical dataset consists of magnetic resonance imaging (MRI) data, computed tomography (CT) data, colonoscopy data, and endoscopy data. Employing the proposed approach, a signal-to-noise ratio (SNR) of 30 dB was achieved for MRI data on a sampling ratio of M/N=0.3. SNR of 34, 30, and 34 dB are corresponding to CT, colonoscopy, and endoscopy data on the same sampling ratio of M/N=0.15. The proposed M-BRA performance indicates the potential for compressed medical imaging analysis with high reconstruction image quality.

Deep Generative Adversarial Networks: Applications in Musculoskeletal Imaging

In recent years, deep learning techniques have been applied in musculoskeletal radiology to increase the diagnostic potential of acquired images. Generative adversarial networks (GANs), which are deep neural networks that can generate or transform images, have the potential to aid in faster imaging by generating images with a high level of realism across multiple contrast and modalities from existing imaging protocols. This review introduces the key architectures of GANs as well as their technical background and challenges. Key research trends are highlighted, including: (a) reconstruction of high-resolution MRI; (b) image synthesis with different modalities and contrasts; (c) image enhancement that efficiently preserves high-frequency information suitable for human interpretation; (d) pixel-level segmentation with annotation sharing between domains; and (e) applications to different musculoskeletal anatomies. In addition, an overview is provided of the key issues wherein clinical applicability is challenging to capture with conventional performance metrics and expert evaluation. When clinically validated, GANs have the potential to improve musculoskeletal imaging. Keywords: Adults and Pediatrics, Computer Aided Diagnosis (CAD), Computer Applications-General (Informatics), Informatics, Skeletal-Appendicular, Skeletal-Axial, Soft Tissues/Skin © RSNA, 2021.

Artificial Intelligence for MR Image Reconstruction: An Overview for Clinicians

Artificial intelligence (AI) shows tremendous promise in the field of medical imaging, with recent breakthroughs applying deep-learning models for data acquisition, classification problems, segmentation, image synthesis, and image reconstruction. With an eye towards clinical applications, we summarize the active field of deep-learning-based MR image reconstruction. We review the basic concepts of how deep-learning algorithms aid in the transformation of raw k-space data to image data, and specifically examine accelerated imaging and artifact suppression. Recent efforts in these areas show that deep-learning-based algorithms can match and, in some cases, eclipse conventional reconstruction methods in terms of image quality and computational efficiency across a host of clinical imaging applications, including musculoskeletal, abdominal, cardiac, and brain imaging. This article is an introductory overview aimed at clinical radiologists with no experience in deep-learning-based MR image reconstruction and should enable them to understand the basic concepts and current clinical applications of this rapidly growing area of research across multiple organ systems.