Compressed Sensing MRI Reconstruction Using a Generative Adversarial Network With a Cyclic Loss

Fast and low-dose medical imaging generation empowered by hybrid deep-learning and iterative reconstruction

Fast and low-dose reconstructions of medical images are highly desired in clinical routines. We propose a hybrid deep-learning and iterative reconstruction (hybrid DL-IR) framework and apply it for fast magnetic resonance imaging (MRI), fast positron emission tomography (PET), and low-dose computed tomography (CT) image generation tasks. First, in a retrospective MRI study (6,066 cases), we demonstrate its capability of handling 3- to 10-fold under-sampled MR data, enabling organ-level coverage with only 10- to 100-s scan time; second, a low-dose CT study (142 cases) shows that our framework can successfully alleviate the noise and streak artifacts in scans performed with only 10% radiation dose (0.61 mGy); and last, a fast whole-body PET study (131 cases) allows us to faithfully reconstruct tumor-induced lesions, including small ones (<4 mm), from 2- to 4-fold-accelerated PET acquisition (30-60 s/bp). This study offers a promising avenue for accurate and high-quality image reconstruction with broad clinical value.

Commercially Available Deep-learning-reconstruction of MR Imaging of the Knee at 1.5T Has Higher Image Quality Than Conventionally-reconstructed Imaging at 3T: A Normal Volunteer Study

PURPOSE

This study aimed to evaluate whether the image quality of 1.5T magnetic resonance imaging (MRI) of the knee is equal to or higher than that of 3T MRI by applying deep learning reconstruction (DLR).

METHODS

Proton density-weighted images of the right knee of 27 healthy volunteers were obtained by 3T and 1.5T MRI scanners using similar imaging parameters (21 for high resolution image and 6 for normal resolution image). Commercially available DLR was applied to the 1.5T images to obtain 1.5T/DLR images. The 3T and 1.5T/DLR images were compared subjectively for visibility of structures, image noise, artifacts, and overall diagnostic acceptability and objectively. One-way ANOVA and Friedman tests were used for the statistical analyses.

RESULTS

For the high resolution images, all of the anatomical structures, except for bone, were depicted significantly better on the 1.5T/DLR compared with 3T images. Image noise scored statistically lower and overall diagnostic acceptability scored higher on the 1.5T/DLR images. The contrast between lateral meniscus and articular cartilage of the 1.5T/DLR images was significantly higher (5.89 ± 1.30 vs. 4.34 ± 0.87, P < 0.001), and also the contrast between medial meniscus and articular cartilage of the 1.5T/DLR images was significantly higher (5.12 ± 0.93 vs. 3.87 ± 0.56, P < 0.001). Similar image quality improvement by DLR was observed for the normal resolution images.

CONCLUSION

The 1.5T/DLR images can achieve less noise, more precise visualization of the meniscus and ligaments, and higher overall image quality compared with the 3T images acquired using a similar protocol.

Evaluation of a deep learning-based reconstruction method for denoising and image enhancement of shoulder MRI in patients with shoulder pain

OBJECTIVES

To evaluate the diagnostic performance of an automated reconstruction algorithm combining MR imaging acquired using compressed SENSE (CS) with deep learning (DL) in order to reconstruct denoised high-quality images from undersampled MR images in patients with shoulder pain.

METHODS

Prospectively, thirty-eight patients (14 women, mean age 40.0 ± 15.2 years) with shoulder pain underwent morphological MRI using a pseudo-random, density-weighted k-space scheme with an acceleration factor of 2.5 using CS only. An automated DL-based algorithm (CS DL) was used to create reconstructions of the same k-space data as used for CS reconstructions. Images were analyzed by two radiologists and assessed for pathologies, image quality, and visibility of anatomical landmarks using a 4-point Likert scale.

RESULTS

Overall agreement for the detection of pathologies between the CS DL reconstructions and CS images was substantial to almost perfect (κ 0.95 (95% confidence interval 0.82-1.00)). Image quality and the visibility of the rotator cuff, articular cartilage, and axillary recess were overall rated significantly higher for CS DL images compared to CS (p < 0.03). Contrast-to-noise ratios were significantly higher for cartilage/fluid (CS DL 198 ± 24.3, CS 130 ± 32.2, p = 0.02) and ligament/fluid (CS DL 184 ± 17.3, CS 141 ± 23.5, p = 0.03) and SNR values were significantly higher for ligaments and muscle of the CS DL reconstructions (p < 0.04).

CONCLUSION

Evaluation of shoulder pathologies was feasible using a DL-based algorithm for MRI reconstruction and denoising. In clinical routine, CS DL may be beneficial in particular for reducing image noise and may be useful for the detection and better discrimination of discrete pathologies. Assessment of shoulder pathologies was feasible with improved image quality as well as higher SNR using a compressed sensing deep learning-based framework for image reconstructions and denoising.

KEY POINTS

• Automated deep learning-based reconstructions showed a significant increase in signal-to-noise ratio and contrast-to-noise ratio (p < 0.04) with only a slight increase of reconstruction time of 40 s compared to CS. • All pathologies were accurately detected with no loss of diagnostic information or prolongation of the scan time. • Significant improvements of the image quality as well as the visibility of the rotator cuff, articular cartilage, and axillary recess were detected.

Improving accelerated 3D imaging in MRI-guided radiotherapy for prostate cancer using a deep learning method

PURPOSE

This study was to improve image quality for high-speed MR imaging using a deep learning method for online adaptive radiotherapy in prostate cancer. We then evaluated its benefits on image registration.

METHODS

Sixty pairs of 1.5 T MR images acquired with an MR-linac were enrolled. The data included low-speed, high-quality (LSHQ), and high-speed low-quality (HSLQ) MR images. We proposed a CycleGAN, which is based on the data augmentation technique, to learn the mapping between the HSLQ and LSHQ images and then generate synthetic LSHQ (synLSHQ) images from the HSLQ images. Five-fold cross-validation was employed to test the CycleGAN model. The normalized mean absolute error (nMAE), peak signal-to-noise ratio (PSNR), structural similarity index measurement (SSIM), and edge keeping index (EKI) were calculated to determine image quality. The Jacobian determinant value (JDV), Dice similarity coefficient (DSC), and mean distance to agreement (MDA) were used to analyze deformable registration.

RESULTS

Compared with the LSHQ, the proposed synLSHQ achieved comparable image quality and reduced imaging time by ~ 66%. Compared with the HSLQ, the synLSHQ had better image quality with improvement of 57%, 3.4%, 26.9%, and 3.6% for nMAE, SSIM, PSNR, and EKI, respectively. Furthermore, the synLSHQ enhanced registration accuracy with a superior mean JDV (6%) and preferable DSC and MDA values compared with HSLQ.

CONCLUSION

The proposed method can generate high-quality images from high-speed scanning sequences. As a result, it shows potential to shorten the scan time while ensuring the accuracy of radiotherapy.

Need for objective task-based evaluation of deep learning-based denoising methods: A study in the context of myocardial perfusion SPECT

BACKGROUND

Artificial intelligence-based methods have generated substantial interest in nuclear medicine. An area of significant interest has been the use of deep-learning (DL)-based approaches for denoising images acquired with lower doses, shorter acquisition times, or both. Objective evaluation of these approaches is essential for clinical application.

PURPOSE

DL-based approaches for denoising nuclear-medicine images have typically been evaluated using fidelity-based figures of merit (FoMs) such as root mean squared error (RMSE) and structural similarity index measure (SSIM). However, these images are acquired for clinical tasks and thus should be evaluated based on their performance in these tasks. Our objectives were to: (1) investigate whether evaluation with these FoMs is consistent with objective clinical-task-based evaluation; (2) provide a theoretical analysis for determining the impact of denoising on signal-detection tasks; and (3) demonstrate the utility of virtual imaging trials (VITs) to evaluate DL-based methods.

METHODS

A VIT to evaluate a DL-based method for denoising myocardial perfusion SPECT (MPS) images was conducted. To conduct this evaluation study, we followed the recently published best practices for the evaluation of AI algorithms for nuclear medicine (the RELAINCE guidelines). An anthropomorphic patient population modeling clinically relevant variability was simulated. Projection data for this patient population at normal and low-dose count levels (20%, 15%, 10%, 5%) were generated using well-validated Monte Carlo-based simulations. The images were reconstructed using a 3-D ordered-subsets expectation maximization-based approach. Next, the low-dose images were denoised using a commonly used convolutional neural network-based approach. The impact of DL-based denoising was evaluated using both fidelity-based FoMs and area under the receiver operating characteristic curve (AUC), which quantified performance on the clinical task of detecting perfusion defects in MPS images as obtained using a model observer with anthropomorphic channels. We then provide a mathematical treatment to probe the impact of post-processing operations on signal-detection tasks and use this treatment to analyze the findings of this study.

RESULTS

Based on fidelity-based FoMs, denoising using the considered DL-based method led to significantly superior performance. However, based on ROC analysis, denoising did not improve, and in fact, often degraded detection-task performance. This discordance between fidelity-based FoMs and task-based evaluation was observed at all the low-dose levels and for different cardiac-defect types. Our theoretical analysis revealed that the major reason for this degraded performance was that the denoising method reduced the difference in the means of the reconstructed images and of the channel operator-extracted feature vectors between the defect-absent and defect-present cases.

CONCLUSIONS

The results show the discrepancy between the evaluation of DL-based methods with fidelity-based metrics versus the evaluation on clinical tasks. This motivates the need for objective task-based evaluation of DL-based denoising approaches. Further, this study shows how VITs provide a mechanism to conduct such evaluations computationally, in a time and resource-efficient setting, and avoid risks such as radiation dose to the patient. Finally, our theoretical treatment reveals insights into the reasons for the limited performance of the denoising approach and may be used to probe the effect of other post-processing operations on signal-detection tasks.

Linear fine-tuning: a linear transformation based transfer strategy for deep MRI reconstruction

Introduction

Fine-tuning (FT) is a generally adopted transfer learning method for deep learning-based magnetic resonance imaging (MRI) reconstruction. In this approach, the reconstruction model is initialized with pre-trained weights derived from a source domain with ample data and subsequently updated with limited data from the target domain. However, the direct full-weight update strategy can pose the risk of "catastrophic forgetting" and overfitting, hindering its effectiveness. The goal of this study is to develop a zero-weight update transfer strategy to preserve pre-trained generic knowledge and reduce overfitting.

Methods

Based on the commonality between the source and target domains, we assume a linear transformation relationship of the optimal model weights from the source domain to the target domain. Accordingly, we propose a novel transfer strategy, linear fine-tuning (LFT), which introduces scaling and shifting (SS) factors into the pre-trained model. In contrast to FT, LFT only updates SS factors in the transfer phase, while the pre-trained weights remain fixed.

Results

To evaluate the proposed LFT, we designed three different transfer scenarios and conducted a comparative analysis of FT, LFT, and other methods at various sampling rates and data volumes. In the transfer scenario between different contrasts, LFT outperforms typical transfer strategies at various sampling rates and considerably reduces artifacts on reconstructed images. In transfer scenarios between different slice directions or anatomical structures, LFT surpasses the FT method, particularly when the target domain contains a decreasing number of training images, with a maximum improvement of up to 2.06 dB (5.89%) in peak signal-to-noise ratio.

Discussion

The LFT strategy shows great potential to address the issues of "catastrophic forgetting" and overfitting in transfer scenarios for MRI reconstruction, while reducing the reliance on the amount of data in the target domain. Linear fine-tuning is expected to shorten the development cycle of reconstruction models for adapting complicated clinical scenarios, thereby enhancing the clinical applicability of deep MRI reconstruction.

Adaptive diffusion priors for accelerated MRI reconstruction

Deep MRI reconstruction is commonly performed with conditional models that de-alias undersampled acquisitions to recover images consistent with fully-sampled data. Since conditional models are trained with knowledge of the imaging operator, they can show poor generalization across variable operators. Unconditional models instead learn generative image priors decoupled from the operator to improve reliability against domain shifts related to the imaging operator. Recent diffusion models are particularly promising given their high sample fidelity. Nevertheless, inference with a static image prior can perform suboptimally. Here we propose the first adaptive diffusion prior for MRI reconstruction, AdaDiff, to improve performance and reliability against domain shifts. AdaDiff leverages an efficient diffusion prior trained via adversarial mapping over large reverse diffusion steps. A two-phase reconstruction is executed following training: a rapid-diffusion phase that produces an initial reconstruction with the trained prior, and an adaptation phase that further refines the result by updating the prior to minimize data-consistency loss. Demonstrations on multi-contrast brain MRI clearly indicate that AdaDiff outperforms competing conditional and unconditional methods under domain shifts, and achieves superior or on par within-domain performance.

Hierarchical Perception Adversarial Learning Framework for Compressed Sensing MRI

The long acquisition time has limited the accessibility of magnetic resonance imaging (MRI) because it leads to patient discomfort and motion artifacts. Although several MRI techniques have been proposed to reduce the acquisition time, compressed sensing in magnetic resonance imaging (CS-MRI) enables fast acquisition without compromising SNR and resolution. However, existing CS-MRI methods suffer from the challenge of aliasing artifacts. This challenge results in the noise-like textures and missing the fine details, thus leading to unsatisfactory reconstruction performance. To tackle this challenge, we propose a hierarchical perception adversarial learning framework (HP-ALF). HP-ALF can perceive the image information in the hierarchical mechanism: image-level perception and patch-level perception. The former can reduce the visual perception difference in the entire image, and thus achieve aliasing artifact removal. The latter can reduce this difference in the regions of the image, and thus recover fine details. Specifically, HP-ALF achieves the hierarchical mechanism by utilizing multilevel perspective discrimination. This discrimination can provide the information from two perspectives (overall and regional) for adversarial learning. It also utilizes a global and local coherent discriminator to provide structure information to the generator during training. In addition, HP-ALF contains a context-aware learning block to effectively exploit the slice information between individual images for better reconstruction performance. The experiments validated on three datasets demonstrate the effectiveness of HP-ALF and its superiority to the comparative methods.

Medical image super-resolution reconstruction algorithms based on deep learning: A survey

BACKGROUND AND OBJECTIVE

With the high-resolution (HR) requirements of medical images in clinical practice, super-resolution (SR) reconstruction algorithms based on low-resolution (LR) medical images have become a research hotspot. This type of method can significantly improve image SR without improving hardware equipment, so it is of great significance to review it.

METHODS

Aiming at the unique SR reconstruction algorithms in the field of medical images, based on subdivided medical fields such as magnetic resonance (MR) images, computed tomography (CT) images, and ultrasound images. Firstly, we deeply analyzed the research progress of SR reconstruction algorithms, and summarized and compared the different types of algorithms. Secondly, we introduced the evaluation indicators corresponding to the SR reconstruction algorithms. Finally, we prospected the development trend of SR reconstruction technology in the medical field.

RESULTS

The medical image SR reconstruction technology based on deep learning can provide more abundant lesion information, relieve the expert's diagnosis pressure, and improve the diagnosis efficiency and accuracy.

CONCLUSION

The medical image SR reconstruction technology based on deep learning helps to improve the quality of medicine, provides help for the diagnosis of experts, and lays a solid foundation for the subsequent analysis and identification tasks of the computer, which is of great significance for improving the diagnosis efficiency of experts and realizing intelligent medical care.

Deep learning based MRI reconstruction with transformer

Magnetic resonance imaging (MRI) has become one of the most powerful imaging techniques in medical diagnosis, yet the prolonged scanning time becomes a bottleneck for application. Reconstruction methods based on compress sensing (CS) have made progress in reducing this cost by acquiring fewer points in the k-space. Traditional CS methods impose restrictions from different sparse domains to regularize the optimization that always requires balancing time with accuracy. Neural network techniques enable learning a better prior from sample pairs and generating the results in an analytic way. In this paper, we propose a deep learning based reconstruction method to restore high-quality MRI images from undersampled k-space data in an end-to-end style. Unlike prior literature adopting convolutional neural networks (CNN), advanced Swin Transformer is used as the backbone of our work, which proved to be powerful in extracting deep features of the image. In addition, we combined the k-space consistency in the output and further improved the quality. We compared our models with several reconstruction methods and variants, and the experiment results proved that our model achieves the best results in samples at low sampling rates. The source code of KTMR could be acquired at https://github.com/BITwzl/KTMR.

MEDL-Net: A model-based neural network for MRI reconstruction with enhanced deep learned regularizers

PURPOSE

To improve the MRI reconstruction performance of model-based networks and to alleviate their large demand for GPU memory.

METHODS

A model-based neural network with enhanced deep learned regularizers (MEDL-Net) was proposed. The MEDL-Net is separated into several submodules, each of which consists of several cascades to mimic the optimization steps in conventional MRI reconstruction algorithms. Information from shallow cascades is densely connected to latter ones to enrich their inputs in each submodule, and additional revising blocks (RB) are stacked at the end of the submodules to bring more flexibility. Moreover, a composition loss function was designed to explicitly supervise RBs.

RESULTS

Network performance was evaluated on a publicly available dataset. The MEDL-Net quantitatively outperforms the state-of-the-art methods on different MR image sequences with different acceleration rates (four-fold and six-fold). Moreover, the reconstructed images showed that the detailed textures are better preserved. In addition, fewer cascades are required when achieving the same reconstruction results compared with other model-based networks.

CONCLUSION

In this study, a more efficient model-based deep network was proposed to reconstruct MR images. The experimental results indicate that the proposed method improves reconstruction performance with fewer cascades, which alleviates the large demand for GPU memory.

Momentum-Net: Fast and Convergent Iterative Neural Network for Inverse Problems

Iterative neural networks (INN) are rapidly gaining attention for solving inverse problems in imaging, image processing, and computer vision. INNs combine regression NNs and an iterative model-based image reconstruction (MBIR) algorithm, often leading to both good generalization capability and outperforming reconstruction quality over existing MBIR optimization models. This paper proposes the first fast and convergent INN architecture, Momentum-Net, by generalizing a block-wise MBIR algorithm that uses momentum and majorizers with regression NNs. For fast MBIR, Momentum-Net uses momentum terms in extrapolation modules, and noniterative MBIR modules at each iteration by using majorizers, where each iteration of Momentum-Net consists of three core modules: image refining, extrapolation, and MBIR. Momentum-Net guarantees convergence to a fixed-point for general differentiable (non)convex MBIR functions (or data-fit terms) and convex feasible sets, under two asymptomatic conditions. To consider data-fit variations across training and testing samples, we also propose a regularization parameter selection scheme based on the "spectral spread" of majorization matrices. Numerical experiments for light-field photography using a focal stack and sparse-view computational tomography demonstrate that, given identical regression NN architectures, Momentum-Net significantly improves MBIR speed and accuracy over several existing INNs; it significantly improves reconstruction quality compared to a state-of-the-art MBIR method in each application.

A Hybrid Residual Attention Convolutional Neural Network for Compressed Sensing Magnetic Resonance Image Reconstruction

We propose a dual-domain deep learning technique for accelerating compressed sensing magnetic resonance image reconstruction. An advanced convolutional neural network with residual connectivity and an attention mechanism was developed for frequency and image domains. First, the sensor domain subnetwork estimates the unmeasured frequencies of k-space to reduce aliasing artifacts. Second, the image domain subnetwork performs a pixel-wise operation to remove blur and noisy artifacts. The skip connections efficiently concatenate the feature maps to alleviate the vanishing gradient problem. An attention gate in each decoder layer enhances network generalizability and speeds up image reconstruction by eliminating irrelevant activations. The proposed technique reconstructs real-valued clinical images from sparsely sampled k-spaces that are identical to the reference images. The performance of this novel approach was compared with state-of-the-art direct mapping, single-domain, and multi-domain methods. With acceleration factors (AFs) of 4 and 5, our method improved the mean peak signal-to-noise ratio (PSNR) to 8.67 and 9.23, respectively, compared with the single-domain Unet model; similarly, our approach increased the average PSNR to 3.72 and 4.61, respectively, compared with the multi-domain W-net. Remarkably, using an AF of 6, it enhanced the PSNR by 9.87 ± 1.55 and 6.60 ± 0.38 compared with Unet and W-net, respectively.

DSMENet: Detail and Structure Mutually Enhancing Network for under-sampled MRI reconstruction

Reconstructing zero-filled MR images (ZF) from partial k-space by convolutional neural networks (CNN) is an important way to accelerate MRI. However, due to the lack of attention to different components in ZF, it is challenging to learn the mapping from ZF to targets effectively. To ameliorate this issue, we propose a Detail and Structure Mutually Enhancing Network (DSMENet), which benefits from the complementary of the Structure Reconstruction UNet (SRUN) and the Detail Feature Refinement Module (DFRM). The SRUN learns structure-dominated information at multiple scales. And the DRFM enriches detail-dominated information from coarse to fine. The bidirectional alternate connections then exchange information between them. Moreover, the Detail Representation Construction Module (DRCM) extracts valuable initial detail representation for DFRM. And the Detail Guided Fusion Module (DGFM) facilitates the deep fusion of these complementary information. With the help of them, various components in ZF can be applied with discriminative attentions and mutually enhanced. In addition, the performance can be further improved by the Deep Enhanced Restoration (DER), a strategy based on recursion and constrain. Extensive experiments on fastMRI and CC-359 datasets demonstrate that DSMENet has robustness in terms of various body parts, under-sampling rates, and masks. Furthermore, DSMENet can achieve promising performance on qualitative and quantitative results, especially the competitive NMSE of 0.0268, PSNE of 33.7, and SSIM of 0.7808 on fastMRI 4 × single-coil knee leaderboard.

Magnetic resonance imaging reconstruction using a deep energy-based model

Although recent deep energy-based generative models (EBMs) have shown encouraging results in many image-generation tasks, how to take advantage of self-adversarial cogitation in deep EBMs to boost the performance of magnetic resonance imaging (MRI) reconstruction is still desired. With the successful application of deep learning in a wide range of MRI reconstructions, a line of emerging research involves formulating an optimization-based reconstruction method in the space of a generative model. Leveraging this, a novel regularization strategy is introduced in this article that takes advantage of self-adversarial cogitation of the deep energy-based model. More precisely, we advocate alternating learning by a more powerful energy-based model with maximum likelihood estimation to obtain the deep energy-based information, represented as a prior image. Simultaneously, implicit inference with Langevin dynamics is a unique property of reconstruction. In contrast to other generative models for reconstruction, the proposed method utilizes deep energy-based information as the image prior in reconstruction to improve the quality of image. Experimental results imply the proposed technique can obtain remarkable performance in terms of high reconstruction accuracy that is competitive with state-of-the-art methods, and which does not suffer from mode collapse. Algorithmically, an iterative approach is presented to strengthen EBM training with the gradient of energy network. The robustness and reproducibility of the algorithm were also experimentally validated. More importantly, the proposed reconstruction framework can be generalized for most MRI reconstruction scenarios.

Deep compressed sensing MRI via a gradient-enhanced fusion model

BACKGROUND

Compressed sensing has been employed to accelerate magnetic resonance imaging by sampling fewer measurements. However, conventional iterative optimization-based CS-MRI are time-consuming for iterative calculations and often share poor generalization ability on multicontrast datasets. Most deep-learning-based CS-MRI focus on learning an end-to-end mapping while ignoring some prior knowledge existed in MR images.

PURPOSE

We propose an iterative fusion model to integrate the image and gradient-based priors into reconstruction via convolutional neural network models while maintaining high quality and preserving the detailed information as well.

METHODS

We propose a gradient-enhanced fusion network (GFN) for fast and accurate MRI reconstruction, in which dense blocks with dilated convolution and dense residual learnings are used to capture abundant features with fewer parameters. Meanwhile, decomposed gradient maps containing obvious structural information are introduced into the network to enhance the reconstructed images. Besides this, gradient-based priors along directions X and Y are exploited to learn adaptive tight frames for reconstructing the desired image contrast and edges by respective gradient fusion networks. After that, both image and gradient priors are fused in the proposed optimization model, in which we employ the l₂ -norm to promote the sparsity of gradient priors. The proposed fusion model can effectively help to capture edge structures in the gradient images and to preserve more detailed information of MR images.

RESULTS

Experimental results demonstrate that the proposed method outperforms several CS-MRI methods in terms of peak signal-to-noise (PSNR), the structural similarity index (SSIM), and visualizations on three sampling masks with different rates. Noteworthy, to evaluate the generalization ability, the proposed model conducts cross-center training and testing experiments for all three modalities and shares more stable performance compared than other approaches. In addition, the proposed fusion model is applied to other comparable deep learning methods. The quantitative results show that the reconstruction results of these methods are obviously improved.

CONCLUSIONS

The gradient-based priors reconstructed from GFNs can effectively enhance the edges and details of under-sampled data. The proposed fusion model integrates image and gradient priors using l₂ -norm can effectively improve the generalization ability on multicontrast datasets reconstruction.

Multi-channel GAN-based calibration-free diffusion-weighted liver imaging with simultaneous coil sensitivity estimation and reconstruction

INTRODUCTION

Diffusion-weighted imaging (DWI) with parallel reconstruction may suffer from a mismatch between the coil calibration scan and imaging scan due to motions, especially for abdominal imaging.

METHODS

This study aimed to construct an iterative multichannel generative adversarial network (iMCGAN)-based framework for simultaneous sensitivity map estimation and calibration-free image reconstruction. The study included 106 healthy volunteers and 10 patients with tumors.

RESULTS

The performance of iMCGAN was evaluated in healthy participants and patients and compared with the SAKE, ALOHA-net, and DeepcomplexMRI reconstructions. The peak signal-to-noise ratio (PSNR), structural similarity index measure (SSIM), root mean squared error (RMSE), and histograms of apparent diffusion coefficient (ADC) maps were calculated for assessing image qualities. The proposed iMCGAN outperformed the other methods in terms of the PSNR (iMCGAN: 41.82 ± 2.14; SAKE: 17.38 ± 1.78; ALOHA-net: 20.43 ± 2.11 and DeepcomplexMRI: 39.78 ± 2.78) for b = 800 DWI with an acceleration factor of 4. Besides, the ghosting artifacts in the SENSE due to the mismatch between the DW image and the sensitivity maps were avoided using the iMCGAN model.

DISCUSSION

The current model iteratively refined the sensitivity maps and the reconstructed images without additional acquisitions. Thus, the quality of the reconstructed image was improved, and the aliasing artifact was alleviated when motions occurred during the imaging procedure.

SwinGAN: A dual-domain Swin Transformer-based generative adversarial network for MRI reconstruction

Magnetic resonance imaging (MRI) is one of the most important modalities for clinical diagnosis. However, the main disadvantages of MRI are the long scanning time and the moving artifact caused by patient movement during prolonged imaging. It can also lead to patient anxiety and discomfort, so accelerated imaging is indispensable for MRI. Convolutional neural network (CNN) based methods have become the fact standard for medical image reconstruction, and generative adversarial network (GAN) have also been widely used. Nevertheless, due to the limited ability of CNN to capture long-distance information, it may lead to defects in the structure of the reconstructed images such as blurry contour. In this paper, we propose a novel Swin Transformer-based dual-domain generative adversarial network (SwinGAN) for accelerated MRI reconstruction. The SwinGAN consists of two generators: a frequency-domain generator and an image-domain generator. Both the generators utilize Swin Transformer as backbone for effectively capturing the long-distance dependencies. A contextual image relative position encoder (ciRPE) is designed to enhance the ability to capture local information. We extensively evaluate the method on the IXI brain dataset, MICCAI 2013 dataset and MRNet knee dataset. Compared with KIGAN, the peak signal-to-noise ratio (PSNR) and structural similarity index measure (SSIM) are improved by 6.1% and 1.49% to 37.64 dB and 0.98 on IXI dataset respectively, which demonstrates that our model can sufficiently utilize the local and global information of image. The model shows promising performance and robustness under different undersampling masks, different acceleration rates and different datasets. But it needs high hardware requirements with the increasing of the network parameters. The code is available at: https://github.com/learnerzx/SwinGAN.

SuperMAP: Deep ultrafast MR relaxometry with joint spatiotemporal undersampling

PURPOSE

To develop an ultrafast and robust MR parameter mapping network using deep learning.

THEORY AND METHODS

We design a deep learning framework called SuperMAP that directly converts a series of undersampled (both in k-space and parameter-space) parameter-weighted images into several quantitative maps, bypassing the conventional exponential fitting procedure. We also present a novel technique to simultaneously reconstruct T1rho and T2 relaxation maps within a single scan. Full data were acquired and retrospectively undersampled for training and testing using traditional and state-of-the-art techniques for comparison. Prospective data were also collected to evaluate the trained network. The performance of all methods is evaluated using the parameter qualification errors and other metrics in the segmented regions of interest.

RESULTS

SuperMAP achieved accurate T1rho and T2 mapping with high acceleration factors (R = 24 and R = 32). It exploited both spatial and temporal information and yielded low error (normalized mean square error of 2.7% at R = 24 and 2.8% at R = 32) and high resemblance (structural similarity of 97% at R = 24 and 96% at R = 32) to the gold standard. The network trained with retrospectively undersampled data also works well for the prospective data (with a slightly lower acceleration factor). SuperMAP is also superior to conventional methods.

CONCLUSION

Our results demonstrate the feasibility of generating superfast MR parameter maps through very few undersampled parameter-weighted images. SuperMAP can simultaneously generate T1rho and T2 relaxation maps in a short scan time.

Undersampling artifact reduction for free-breathing 3D stack-of-radial MRI based on a deep adversarial learning network

PURPOSE

Stack-of-radial MRI allows free-breathing abdominal scans, however, it requires relatively long acquisition time. Undersampling reduces scan time but can cause streaking artifacts and degrade image quality. This study developed deep learning networks with adversarial loss and evaluated the performance of reducing streaking artifacts and preserving perceptual image sharpness.

METHODS

A 3D generative adversarial network (GAN) was developed for reducing streaking artifacts in stack-of-radial abdominal scans. Training and validation datasets were self-gated to 5 respiratory states to reduce motion artifacts and to effectively augment the data. The network used a combination of three loss functions to constrain the anatomy and preserve image quality: adversarial loss, mean-squared-error loss and structural similarity index loss. The performance of the network was investigated for 3-5 times undersampled data from 2 institutions. The performance of the GAN for 5 times accelerated images was compared with a 3D U-Net and evaluated using quantitative NMSE, SSIM and region of interest (ROI) measurements as well as qualitative scores of radiologists.

RESULTS

The 3D GAN showed similar NMSE (0.0657 vs. 0.0559, p = 0.5217) and significantly higher SSIM (0.841 vs. 0.798, p < 0.0001) compared to U-Net. ROI analysis showed GAN removed streaks in both the background air and the tissue and was not significantly different from the reference mean and variations. Radiologists' scores showed GAN had a significant improvement of 1.6 point (p = 0.004) on a 4-point scale in streaking score while no significant difference in sharpness score compared to the input.

CONCLUSION

3D GAN removes streaking artifacts and preserves perceptual image details.

Artificial Intelligence-Driven Ultra-Fast Superresolution MRI: 10-Fold Accelerated Musculoskeletal Turbo Spin Echo MRI Within Reach

ABSTRACT

Magnetic resonance imaging (MRI) is the keystone of modern musculoskeletal imaging; however, long pulse sequence acquisition times may restrict patient tolerability and access. Advances in MRI scanners, coil technology, and innovative pulse sequence acceleration methods enable 4-fold turbo spin echo pulse sequence acceleration in clinical practice; however, at this speed, conventional image reconstruction approaches the signal-to-noise limits of temporal, spatial, and contrast resolution. Novel deep learning image reconstruction methods can minimize signal-to-noise interdependencies to better advantage than conventional image reconstruction, leading to unparalleled gains in image speed and quality when combined with parallel imaging and simultaneous multislice acquisition. The enormous potential of deep learning-based image reconstruction promises to facilitate the 10-fold acceleration of the turbo spin echo pulse sequence, equating to a total acquisition time of 2-3 minutes for entire MRI examinations of joints without sacrificing spatial resolution or image quality. Current investigations aim for a better understanding of stability and failure modes of image reconstruction networks, validation of network reconstruction performance with external data sets, determination of diagnostic performances with independent reference standards, establishing generalizability to other centers, scanners, field strengths, coils, and anatomy, and building publicly available benchmark data sets to compare methods and foster innovation and collaboration between the clinical and image processing community. In this article, we review basic concepts of deep learning-based acquisition and image reconstruction techniques for accelerating and improving the quality of musculoskeletal MRI, commercially available and developing deep learning-based MRI solutions, superresolution, denoising, generative adversarial networks, and combined strategies for deep learning-driven ultra-fast superresolution musculoskeletal MRI. This article aims to equip radiologists and imaging scientists with the necessary practical knowledge and enthusiasm to meet this exciting new era of musculoskeletal MRI.

De-Aliasing and Accelerated Sparse Magnetic Resonance Image Reconstruction Using Fully Dense CNN with Attention Gates

When sparsely sampled data are used to accelerate magnetic resonance imaging (MRI), conventional reconstruction approaches produce significant artifacts that obscure the content of the image. To remove aliasing artifacts, we propose an advanced convolutional neural network (CNN) called fully dense attention CNN (FDA-CNN). We updated the Unet model with the fully dense connectivity and attention mechanism for MRI reconstruction. The main benefit of FDA-CNN is that an attention gate in each decoder layer increases the learning process by focusing on the relevant image features and provides a better generalization of the network by reducing irrelevant activations. Moreover, densely interconnected convolutional layers reuse the feature maps and prevent the vanishing gradient problem. Additionally, we also implement a new, proficient under-sampling pattern in the phase direction that takes low and high frequencies from the k-space both randomly and non-randomly. The performance of FDA-CNN was evaluated quantitatively and qualitatively with three different sub-sampling masks and datasets. Compared with five current deep learning-based and two compressed sensing MRI reconstruction techniques, the proposed method performed better as it reconstructed smoother and brighter images. Furthermore, FDA-CNN improved the mean PSNR by 2 dB, SSIM by 0.35, and VIFP by 0.37 compared with Unet for the acceleration factor of 5.

Semi-Supervised Learning of MRI Synthesis Without Fully-Sampled Ground Truths

Learning-based translation between MRI contrasts involves supervised deep models trained using high-quality source- and target-contrast images derived from fully-sampled acquisitions, which might be difficult to collect under limitations on scan costs or time. To facilitate curation of training sets, here we introduce the first semi-supervised model for MRI contrast translation (ssGAN) that can be trained directly using undersampled k-space data. To enable semi-supervised learning on undersampled data, ssGAN introduces novel multi-coil losses in image, k-space, and adversarial domains. The multi-coil losses are selectively enforced on acquired k-space samples unlike traditional losses in single-coil synthesis models. Comprehensive experiments on retrospectively undersampled multi-contrast brain MRI datasets are provided. Our results demonstrate that ssGAN yields on par performance to a supervised model, while outperforming single-coil models trained on coil-combined magnitude images. It also outperforms cascaded reconstruction-synthesis models where a supervised synthesis model is trained following self-supervised reconstruction of undersampled data. Thus, ssGAN holds great promise to improve the feasibility of learning-based multi-contrast MRI synthesis.

A Systematic Literature Review on Applications of GAN-Synthesized Images for Brain MRI

With the advances in brain imaging, magnetic resonance imaging (MRI) is evolving as a popular radiological tool in clinical diagnosis. Deep learning (DL) methods can detect abnormalities in brain images without an extensive manual feature extraction process. Generative adversarial network (GAN)-synthesized images have many applications in this field besides augmentation, such as image translation, registration, super-resolution, denoising, motion correction, segmentation, reconstruction, and contrast enhancement. The existing literature was reviewed systematically to understand the role of GAN-synthesized dummy images in brain disease diagnosis. Web of Science and Scopus databases were extensively searched to find relevant studies from the last 6 years to write this systematic literature review (SLR). Predefined inclusion and exclusion criteria helped in filtering the search results. Data extraction is based on related research questions (RQ). This SLR identifies various loss functions used in the above applications and software to process brain MRIs. A comparative study of existing evaluation metrics for GAN-synthesized images helps choose the proper metric for an application. GAN-synthesized images will have a crucial role in the clinical sector in the coming years, and this paper gives a baseline for other researchers in the field.

Synthetic Time of Flight Magnetic Resonance Angiography Generation Model Based on Cycle-Consistent Generative Adversarial Network Using PETRA-MRA in the Patients With Treated Intracranial Aneurysm

BACKGROUND

Pointwise encoding time reduction with radial acquisition (PETRA) magnetic resonance angiography (MRA) is useful for evaluating intracranial aneurysm recurrence, but the problem of severe background noise and low peripheral signal-to-noise ratio (SNR) remain. Deep learning could reduce noise using high- and low-quality images.

PURPOSE

To develop a cycle-consistent generative adversarial network (cycleGAN)-based deep learning model to generate synthetic TOF (synTOF) using PETRA.

STUDY TYPE

Retrospective.

POPULATION

A total of 377 patients (mean age: 60 ± 11; 293 females) with treated intracranial aneurysms who underwent both PETRA and TOF from October 2017 to January 2021. Data were randomly divided into training (49.9%, 188/377) and validation (50.1%, 189/377) groups.

FIELD STRENGTH/SEQUENCE

Ultra-short echo time and TOF-MRA on a 3-T MR system.

ASSESSMENT

For the cycleGAN model, the peak SNR (PSNR) and structural similarity (SSIM) were evaluated. Image quality was compared qualitatively (5-point Likert scale) and quantitatively (SNR). A multireader diagnostic optimality evaluation was performed with 17 radiologists (experience of 1-18 years).

STATISTICAL TESTS

Generalized estimating equation analysis, Friedman's test, McNemar test, and Spearman's rank correlation. P < 0.05 indicated statistical significance.

RESULTS

The PSNR and SSIM between synTOF and TOF were 17.51 [16.76; 18.31] dB and 0.71 ± 0.02. The median values of overall image quality, noise, sharpness, and vascular conspicuity were significantly higher for synTOF than for PETRA (4.00 [4.00; 5.00] vs. 4.00 [3.00; 4.00]; 5.00 [4.00; 5.00] vs. 3.00 [2.00; 4.00]; 4.00 [4.00; 4.00] vs. 4.00 [3.00; 4.00]; 3.00 [3.00; 4.00] vs. 3.00 [2.00; 3.00]). The SNRs of the middle cerebral arteries were the highest for synTOF (synTOF vs. TOF vs. PETRA; 63.67 [43.25; 105.00] vs. 52.42 [32.88; 74.67] vs. 21.05 [12.34; 37.88]). In the multireader evaluation, there was no significant difference in diagnostic optimality or preference between synTOF and TOF (19.00 [18.00; 19.00] vs. 20.00 [18.00; 20.00], P = 0.510; 8.00 [6.00; 11.00] vs. 11.00 [9.00, 14.00], P = 1.000).

DATA CONCLUSION

The cycleGAN-based deep learning model provided synTOF free from background artifact. The synTOF could be a versatile alternative to TOF in patients who have undergone PETRA for evaluating treated aneurysms.

EVIDENCE LEVEL

4 TECHNICAL EFFICACY: Stage 1.

How Machine Learning is Powering Neuroimaging to Improve Brain Health

This report presents an overview of how machine learning is rapidly advancing clinical translational imaging in ways that will aid in the early detection, prediction, and treatment of diseases that threaten brain health. Towards this goal, we aresharing the information presented at a symposium, "Neuroimaging Indicators of Brain Structure and Function - Closing the Gap Between Research and Clinical Application", co-hosted by the McCance Center for Brain Health at Mass General Hospital and the MIT HST Neuroimaging Training Program on February 12, 2021. The symposium focused on the potential for machine learning approaches, applied to increasingly large-scale neuroimaging datasets, to transform healthcare delivery and change the trajectory of brain health by addressing brain care earlier in the lifespan. While not exhaustive, this overview uniquely addresses many of the technical challenges from image formation, to analysis and visualization, to synthesis and incorporation into the clinical workflow. Some of the ethical challenges inherent to this work are also explored, as are some of the regulatory requirements for implementation. We seek to educate, motivate, and inspire graduate students, postdoctoral fellows, and early career investigators to contribute to a future where neuroimaging meaningfully contributes to the maintenance of brain health.

A cascade of preconditioned conjugate gradient networks for accelerated magnetic resonance imaging

BACKGROUND AND OBJECTIVE

Recent unfolding based compressed sensing magnetic resonance imaging (CS-MRI) methods only reinterpret conventional CS-MRI optimization algorithms and, consequently, inherit the weaknesses of the alternating optimization strategy. In order to avoid the structural complexity of the alternating optimization strategy and achieve better reconstruction performance, we propose to directly optimize the ℓ₁ regularized convex optimization problem using a deep learning approach.

METHOD

In order to achieve direct optimization, a system of equations solving the ℓ₁ regularized optimization problem is constructed from the optimality conditions of a novel primal-dual form proposed for the effective training of the sparsifying transform. The optimal solution is obtained by a cascade of unfolding networks of the preconditioned conjugate gradient (PCG) algorithm trained to minimize the mean element-wise absolute difference (ℓ₁ loss) between the terminal output and ground truth image in an end-to-end manner. The performance of the proposed method was compared with that of U-Net, PD-Net, ISTA-Net+, and the recently proposed projection-based cascaded U-Net, using single-coil knee MR images of the fastMRI dataset.

RESULTS

In our experiment, the proposed network outperformed existing unfolding-based networks and the complex version of U-Net in several subsampling scenarios. In particular, when using the random Cartesian subsampling mask with 25 % sampling rate, the proposed model outperformed PD-Net by 0.76 dB, ISTA-Net+ by 0.43 dB, and U-Net by 1.21 dB on the positron density without suppression (PD) dataset in term of peak signal to noise ratio. In comparison with the projection-based cascade U-Net, the proposed algorithm achieved approximately the same performance when the sampling rate was 25% with only 1.62% number of network parameters at the cost of a longer reconstruction time (approximately twice).

CONCLUSION

A cascade of unfolding networks of the PCG algorithm was proposed to directly optimize the ℓ₁ regularized CS-MRI optimization problem. The proposed network achieved improved reconstruction performance compared with U-Net, PD-Net, and ISTA-Net+, and achieved approximately the same performance as the projection-based cascaded U-Net while using significantly fewer network parameters.

Self-Training Strategy Based on Finite Element Method for Adaptive Bioluminescence Tomography Reconstruction

Bioluminescence tomography (BLT) is a promising pre-clinical imaging technique for a wide variety of biomedical applications, which can non-invasively reveal functional activities inside living animal bodies through the detection of visible or near-infrared light produced by bioluminescent reactions. Recently, reconstruction approaches based on deep learning have shown great potential in optical tomography modalities. However, these reports only generate data with stationary patterns of constant target number, shape, and size. The neural networks trained by these data sets are difficult to reconstruct the patterns outside the data sets. This will tremendously restrict the applications of deep learning in optical tomography reconstruction. To address this problem, a self-training strategy is proposed for BLT reconstruction in this paper. The proposed strategy can fast generate large-scale BLT data sets with random target numbers, shapes, and sizes through an algorithm named random seed growth algorithm and the neural network is automatically self-trained. In addition, the proposed strategy uses the neural network to build a map between photon densities on surface and inside the imaged object rather than an end-to-end neural network that directly infers the distribution of sources from the photon density on surface. The map of photon density is further converted into the distribution of sources through the multiplication with stiffness matrix. Simulation, phantom, and mouse studies are carried out. Results show the availability of the proposed self-training strategy.

Virtual contrast enhancement for CT scans of abdomen and pelvis

Contrast agents are commonly used to highlight blood vessels, organs, and other structures in magnetic resonance imaging (MRI) and computed tomography (CT) scans. However, these agents may cause allergic reactions or nephrotoxicity, limiting their use in patients with kidney dysfunctions. In this paper, we propose a generative adversarial network (GAN) based framework to automatically synthesize contrast-enhanced CTs directly from the non-contrast CTs in the abdomen and pelvis region. The respiratory and peristaltic motion can affect the pixel-level mapping of contrast-enhanced learning, which makes this task more challenging than other body parts. A perceptual loss is introduced to compare high-level semantic differences of the enhancement areas between the virtual contrast-enhanced and actual contrast-enhanced CT images. Furthermore, to accurately synthesize the intensity details as well as remain texture structures of CT images, a dual-path training schema is proposed to learn the texture and structure features simultaneously. Experiment results on three contrast phases (i.e. arterial, portal, and delayed phase) show the potential to synthesize virtual contrast-enhanced CTs directly from non-contrast CTs of the abdomen and pelvis for clinical evaluation.

Improving high frequency image features of deep learning reconstructions via k-space refinement with null-space kernel

PURPOSE

Deep learning (DL) based reconstruction using unrolled neural networks has shown great potential in accelerating MRI. However, one of the major drawbacks is the loss of high-frequency details and textures in the output. The purpose of the study is to propose a novel refinement method that uses null-space kernel to refine k-space and improve blurred image details and textures.

METHODS

The proposed method constrains the output of the DL to comply to the linear neighborhood relationship calibrated in the auto-calibration lines. To demonstrate efficacy, we tested our refinement method on the DL reconstruction under a variety of conditions (i.e., dataset, unrolled neural networks, and under-sampling scheme). Specifically, the method was tested on three large-scale public datasets (knee and brain) from fastMRI's multi-coil track.

RESULTS

The proposed scheme visually reduces the structural error in the k-space domain, enhance the homogeneity of the k-space intensity. Consequently, reconstructed image shows sharper images with enhanced details and textures. The proposed method is also successful in improving high-frequency image details (SSIM, GMSD) without sacrificing overall image error (PSNR).

CONCLUSION

Our findings imply that refining DL output using the proposed method may generally improve DL reconstruction as tested with various large-scale dataset and networks.

Generation of Digital Brain Phantom for Machine Learning Application of Dopamine Transporter Radionuclide Imaging

While machine learning (ML) methods may significantly improve image quality for SPECT imaging for the diagnosis and monitoring of Parkinson's disease (PD), they require a large amount of data for training. It is often difficult to collect a large population of patient data to support the ML research, and the ground truth of lesion is also unknown. This paper leverages a generative adversarial network (GAN) to generate digital brain phantoms for training ML-based PD SPECT algorithms. A total of 594 PET 3D brain models from 155 patients (113 male and 42 female) were reviewed and 1597 2D slices containing the full or a portion of the striatum were selected. Corresponding attenuation maps were also generated based on these images. The data were then used to develop a GAN for generating 2D brain phantoms, where each phantom consisted of a radioactivity image and the corresponding attenuation map. Statistical methods including histogram, Fréchet distance, and structural similarity were used to evaluate the generator based on 10,000 generated phantoms. When the generated phantoms and training dataset were both passed to the discriminator, similar normal distributions were obtained, which indicated the discriminator was unable to distinguish the generated phantoms from the training datasets. The generated digital phantoms can be used for 2D SPECT simulation and serve as the ground truth to develop ML-based reconstruction algorithms. The cumulated experience from this work also laid the foundation for building a 3D GAN for the same application.

Federated Learning in Medical Imaging: Part II: Methods, Challenges, and Considerations

Federated learning is a machine learning method that allows decentralized training of deep neural networks among multiple clients while preserving the privacy of each client's data. Federated learning is instrumental in medical imaging because of the privacy considerations of medical data. Setting up federated networks in hospitals comes with unique challenges, primarily because medical imaging data and federated learning algorithms each have their own set of distinct characteristics. This article introduces federated learning algorithms in medical imaging and discusses technical challenges and considerations of real-world implementation of them.

Dielectric Breast Phantoms by Generative Adversarial Network

In order to conduct the research of machine-learning (ML) based microwave breast imaging (MBI), a large number of digital dielectric breast phantoms that can be used as training data (ground truth) are required but are difficult to be achieved from practice. Although a few dielectric breast phantoms have been developed for research purpose, the number and the diversity are limited and is far inadequate to develop a robust ML algorithm for MBI. This paper presents a neural network method to generate 2D virtual breast phantoms that are similar to the real ones, which can be used to develop ML-based MBI in the future. The generated phantoms are similar but are different from those used in training. Each phantom consists of several images with each representing the distribution of a dielectric parameter in the breast map. Statistical analysis was performed over 10,000 generated phantoms to investigate the performance of the generative network. With the generative network, one may generate unlimited number of breast images with more variations, so the ML-based MBI will be more ready to deploy.

Pyramid Convolutional RNN for MRI Image Reconstruction

Fast and accurate MRI image reconstruction from undersampled data is crucial in clinical practice. Deep learning based reconstruction methods have shown promising advances in recent years. However, recovering fine details from undersampled data is still challenging. In this paper, we introduce a novel deep learning based method, Pyramid Convolutional RNN (PC-RNN), to reconstruct images from multiple scales. Based on the formulation of MRI reconstruction as an inverse problem, we design the PC-RNN model with three convolutional RNN (ConvRNN) modules to iteratively learn the features in multiple scales. Each ConvRNN module reconstructs images at different scales and the reconstructed images are combined by a final CNN module in a pyramid fashion. The multi-scale ConvRNN modules learn a coarse-to-fine image reconstruction. Unlike other common reconstruction methods for parallel imaging, PC-RNN does not employ coil sensitive maps for multi-coil data and directly model the multiple coils as multi-channel inputs. The coil compression technique is applied to standardize data with various coil numbers, leading to more efficient training. We evaluate our model on the fastMRI knee and brain datasets and the results show that the proposed model outperforms other methods and can recover more details. The proposed method is one of the winner solutions in the 2019 fastMRI competition.

Multiple B-Value Model-Based Residual Network (MORN) for Accelerated High-Resolution Diffusion-Weighted Imaging

Single-Shot Echo Planar Imaging (SSEPI) based Diffusion Weighted Imaging (DWI) has shortcomings such as low resolution and severe distortions. In contrast, Multi-Shot EPI (MSEPI) provides optimal spatial resolution but increases scan time. This study proposed a Multiple b-value mOdel-based Residual Network (MORN) model to reconstruct multiple b-value high-resolution DWI from undersampled k-space data simultaneously. We incorporated Parallel Imaging (PI) into a residual U-net to reconstruct multiple b-value multi-coil data with the supervision of MUltiplexed Sensitivity-Encoding (MUSE) reconstructed Multi-Shot DWI (MSDWI). Moreover, asymmetric concatenations among different b-values and the combined loss to back propagate helped the feature transfer. After training and validation of the MORN in a dataset of 32 healthy cases, additional assessments were performed on 6 patients with different tumor types. The experimental results demonstrated that the MORN model outperformed conventional PI reconstruction (i.e. SENSE) and two state-of-the-art deep learning methods (SENSE-GAN and VSNet) in terms of PSNR (Peak Signal-to-Noise Ratio), SSIM (Structual SIMilarity) and apparent diffusion coefficient maps. In addition, using the pre-trained model under DWI, the MORN achieved consistent fractional anisotrophy and mean diffusivity reconstructed from multiple diffusion directions. Hence, the proposed method shows potential in clinical application according to the observations on tumor patients as well as images of multiple diffusion directions.

Sparse signal reconstruction via collaborative neurodynamic optimization

In this paper, we formulate a mixed-integer problem for sparse signal reconstruction and reformulate it as a global optimization problem with a surrogate objective function subject to underdetermined linear equations. We propose a sparse signal reconstruction method based on collaborative neurodynamic optimization with multiple recurrent neural networks for scattered searches and a particle swarm optimization rule for repeated repositioning. We elaborate on experimental results to demonstrate the outperformance of the proposed approach against ten state-of-the-art algorithms for sparse signal reconstruction.

Real-time MRI motion estimation through an unsupervised k-space-driven deformable registration network (KS-RegNet)

Purpose. Real-time three-dimensional (3D) magnetic resonance (MR) imaging is challenging because of slow MR signal acquisition, leading to highly under-sampled k-space data. Here, we proposed a deep learning-based, k-space-driven deformable registration network (KS-RegNet) for real-time 3D MR imaging. By incorporating prior information, KS-RegNet performs a deformable image registration between a fully-sampled prior image and on-board images acquired from highly-under-sampled k-space data, to generate high-quality on-board images for real-time motion tracking. Methods. KS-RegNet is an end-to-end, unsupervised network consisting of an input data generation block, a subsequent U-Net core block, and following operations to compute data fidelity and regularization losses. The input data involved a fully-sampled, complex-valued prior image, and the k-space data of an on-board, real-time MR image (MRI). From the k-space data, under-sampled real-time MRI was reconstructed by the data generation block to input into the U-Net core. In addition, to train the U-Net core to learn the under-sampling artifacts, the k-space data of the prior image was intentionally under-sampled using the same readout trajectory as the real-time MRI, and reconstructed to serve an additional input. The U-Net core predicted a deformation vector field that deforms the prior MRI to on-board real-time MRI. To avoid adverse effects of quantifying image similarity on the artifacts-ridden images, the data fidelity loss of deformation was evaluated directly in k-space. Results. Compared with Elastix and other deep learning network architectures, KS-RegNet demonstrated better and more stable performance. The average (±s.d.) DICE coefficients of KS-RegNet on a cardiac dataset for the 5- , 9- , and 13-spoke k-space acquisitions were 0.884 ± 0.025, 0.889 ± 0.024, and 0.894 ± 0.022, respectively; and the corresponding average (±s.d.) center-of-mass errors (COMEs) were 1.21 ± 1.09, 1.29 ± 1.22, and 1.01 ± 0.86 mm, respectively. KS-RegNet also provided the best performance on an abdominal dataset. Conclusion. KS-RegNet allows real-time MRI generation with sub-second latency. It enables potential real-time MR-guided soft tissue tracking, tumor localization, and radiotherapy plan adaptation.

Multi-Coil MRI Reconstruction Challenge-Assessing Brain MRI Reconstruction Models and Their Generalizability to Varying Coil Configurations

Deep-learning-based brain magnetic resonance imaging (MRI) reconstruction methods have the potential to accelerate the MRI acquisition process. Nevertheless, the scientific community lacks appropriate benchmarks to assess the MRI reconstruction quality of high-resolution brain images, and evaluate how these proposed algorithms will behave in the presence of small, but expected data distribution shifts. The multi-coil MRI (MC-MRI) reconstruction challenge provides a benchmark that aims at addressing these issues, using a large dataset of high-resolution, three-dimensional, T1-weighted MRI scans. The challenge has two primary goals: (1) to compare different MRI reconstruction models on this dataset and (2) to assess the generalizability of these models to data acquired with a different number of receiver coils. In this paper, we describe the challenge experimental design and summarize the results of a set of baseline and state-of-the-art brain MRI reconstruction models. We provide relevant comparative information on the current MRI reconstruction state-of-the-art and highlight the challenges of obtaining generalizable models that are required prior to broader clinical adoption. The MC-MRI benchmark data, evaluation code, and current challenge leaderboard are publicly available. They provide an objective performance assessment for future developments in the field of brain MRI reconstruction.

Unsupervised MRI Reconstruction via Zero-Shot Learned Adversarial Transformers

Supervised reconstruction models are characteristically trained on matched pairs of undersampled and fully-sampled data to capture an MRI prior, along with supervision regarding the imaging operator to enforce data consistency. To reduce supervision requirements, the recent deep image prior framework instead conjoins untrained MRI priors with the imaging operator during inference. Yet, canonical convolutional architectures are suboptimal in capturing long-range relationships, and priors based on randomly initialized networks may yield suboptimal performance. To address these limitations, here we introduce a novel unsupervised MRI reconstruction method based on zero-Shot Learned Adversarial TransformERs (SLATER). SLATER embodies a deep adversarial network with cross-attention transformers to map noise and latent variables onto coil-combined MR images. During pre-training, this unconditional network learns a high-quality MRI prior in an unsupervised generative modeling task. During inference, a zero-shot reconstruction is then performed by incorporating the imaging operator and optimizing the prior to maximize consistency to undersampled data. Comprehensive experiments on brain MRI datasets clearly demonstrate the superior performance of SLATER against state-of-the-art unsupervised methods.

Assessment of data consistency through cascades of independently recurrent inference machines for fast and robust accelerated MRI reconstruction

Objective. Machine Learning methods can learn how to reconstruct magnetic resonance images (MRI) and thereby accelerate acquisition, which is of paramount importance to the clinical workflow. Physics-informed networks incorporate the forward model of accelerated MRI reconstruction in the learning process. With increasing network complexity, robustness is not ensured when reconstructing data unseen during training. We aim to embed data consistency (DC) in deep networks while balancing the degree of network complexity. While doing so, we will assess whether either explicit or implicit enforcement of DC in varying network architectures is preferred to optimize performance. Approach. We propose a scheme called Cascades of Independently Recurrent Inference Machines (CIRIM) to assess DC through unrolled optimization. Herein we assess DC both implicitly by gradient descent and explicitly by a designed term. Extensive comparison of the CIRIM to compressed sensing as well as other Machine Learning methods is performed: the End-to-End Variational Network (E2EVN), CascadeNet, KIKINet, LPDNet, RIM, IRIM, and UNet. Models were trained and evaluated on T1-weighted and FLAIR contrast brain data, and T2-weighted knee data. Both 1D and 2D undersampling patterns were evaluated. Robustness was tested by reconstructing 7.5× prospectively undersampled 3D FLAIR MRI data of multiple sclerosis (MS) patients with white matter lesions. Main results. The CIRIM performed best when implicitly enforcing DC, while the E2EVN required an explicit DC formulation. Through its cascades, the CIRIM was able to score higher on structural similarity and PSNR compared to other methods, in particular under heterogeneous imaging conditions. In reconstructing MS patient data, prospectively acquired with a sampling pattern unseen during model training, the CIRIM maintained lesion contrast while efficiently denoising the images. Significance. The CIRIM showed highly promising generalization capabilities maintaining a very fair trade-off between reconstructed image quality and fast reconstruction times, which is crucial in the clinical workflow.

Simultaneously optimizing sampling pattern for joint acceleration of multi-contrast MRI using model-based deep learning

BACKGROUND

Acceleration of MR imaging (MRI) is a popular research area, and usage of deep learning for acceleration has become highly widespread in the MR community. Joint acceleration of multiple-acquisition MRI was proven to be effective over a single-acquisition approach. Also, optimization in the sampling pattern demonstrated its advantage over conventional undersampling pattern. However, optimizing the sampling patterns for joint acceleration of multiple-acquisition MRI has not been investigated well.

PURPOSE

To develop a model-based deep learning scheme to optimize sampling patterns for a joint acceleration of multi-contrast MRI.

METHODS

The proposed scheme combines sampling pattern optimization and multi-contrast MRI reconstruction. It was extended from the physics-guided method of the joint model-based deep learning (J-MoDL) scheme to optimize the separate sampling pattern for each of multiple contrasts simultaneously for their joint reconstruction. Tests were performed with three contrasts of T2-weighted, FLAIR, and T1-weighted images. The proposed multi-contrast method was compared to (i) single-contrast method with sampling optimization (baseline J-MoDL), (ii) multi-contrast method without sampling optimization, and (iii) multi-contrast method with single common sampling optimization for all contrasts. The optimized sampling patterns were analyzed for sampling location overlap across contrasts. The scheme was also tested in a data-driven scenario, where the inversion between input and label was learned from the under-sampled data directly and tested on knee datasets for generalization test.

RESULTS

The proposed scheme demonstrated a quantitative and qualitative advantage over the single-contrast scheme with sampling pattern optimization and the multi-contrast scheme without sampling pattern optimization. Optimizing the separate sampling pattern for each of the multi-contrasts was superior to optimizing only one common sampling pattern for all contrasts. The proposed scheme showed less overlap in sampling locations than the single-contrast scheme. The main hypothesis was also held in the data-driven situation as well. The brain-trained model worked well on the knee images, demonstrating its generalizability.

CONCLUSION

Our study introduced an effective scheme that combines the sampling optimization and the multi-contrast acceleration. The seamless combination resulted in superior performance over the other existing methods.

Quality assessment of anatomical MRI images from generative adversarial networks: Human assessment and image quality metrics

BACKGROUND

Generative Adversarial Networks (GANs) can synthesize brain images from image or noise input. So far, the gold standard for assessing the quality of the generated images has been human expert ratings. However, due to limitations of human assessment in terms of cost, scalability, and the limited sensitivity of the human eye to more subtle statistical relationships, a more automated approach towards evaluating GANs is required.

NEW METHOD

We investigated to what extent visual quality can be assessed using image quality metrics and we used group analysis and spatial independent components analysis to verify that the GAN reproduces multivariate statistical relationships found in real data. Reference human data was obtained by recruiting neuroimaging experts to assess real Magnetic Resonance (MR) images and images generated by a GAN. Image quality was manipulated by exporting images at different stages of GAN training.

RESULTS

Experts were sensitive to changes in image quality as evidenced by ratings and reaction times, and the generated images reproduced group effects (age, gender) and spatial correlations moderately well. We also surveyed a number of image quality metrics. Overall, Fréchet Inception Distance (FID), Maximum Mean Discrepancy (MMD) and Naturalness Image Quality Evaluator (NIQE) showed sensitivity to image quality and good correspondence with the human data, especially for lower-quality images (i.e., images from early stages of GAN training). However, only a Deep Quality Assessment (QA) model trained on human ratings was able to reproduce the subtle differences between higher-quality images.

CONCLUSIONS

We recommend a combination of group analyses, spatial correlation analyses, and both distortion metrics (FID, MMD, NIQE) and perceptual models (Deep QA) for a comprehensive evaluation and comparison of brain images produced by GANs.

Hierarchical Amortized GAN for 3D High Resolution Medical Image Synthesis

Generative Adversarial Networks (GAN) have many potential medical imaging applications, including data augmentation, domain adaptation, and model explanation. Due to the limited memory of Graphical Processing Units (GPUs), most current 3D GAN models are trained on low-resolution medical images, these models either cannot scale to high-resolution or are prone to patchy artifacts. In this work, we propose a novel end-to-end GAN architecture that can generate high-resolution 3D images. We achieve this goal by using different configurations between training and inference. During training, we adopt a hierarchical structure that simultaneously generates a low-resolution version of the image and a randomly selected sub-volume of the high-resolution image. The hierarchical design has two advantages: First, the memory demand for training on high-resolution images is amortized among sub-volumes. Furthermore, anchoring the high-resolution sub-volumes to a single low-resolution image ensures anatomical consistency between sub-volumes. During inference, our model can directly generate full high-resolution images. We also incorporate an encoder with a similar hierarchical structure into the model to extract features from the images. Experiments on 3D thorax CT and brain MRI demonstrate that our approach outperforms state of the art in image generation. We also demonstrate clinical applications of the proposed model in data augmentation and clinical-relevant feature extraction.