Spatio-Temporal Deep Learning-Based Undersampling Artefact Reduction for 2D Radial Cine MRI With Limited Training Data

Deep learning-based whole-brain B1 +-mapping at 7T

PURPOSE

This study investigates the feasibility of using complex-valued neural networks (NNs) to estimate quantitative transmit magnetic RF field (B1 +) maps from multi-slice localizer scans with different slice orientations in the human head at 7T, aiming to accelerate subject-specific B1 +-calibration using parallel transmission (pTx).

METHODS

Datasets containing channel-wise B1 +-maps and corresponding multi-slice localizers were acquired in axial, sagittal, and coronal orientation in 15 healthy subjects utilizing an eight-channel pTx transceiver head coil. Training included five-fold cross-validation for four network configurations:NN cx tra $$ {\mathrm{NN}}_{\mathrm{cx}}^{\mathrm{tra}} $$ used transversal,NN cx sag $$ {\mathrm{NN}}_{\mathrm{cx}}^{\mathrm{sag}} $$ sagittal,NN cx cor $$ {\mathrm{NN}}_{\mathrm{cx}}^{\mathrm{cor}} $$ coronal data, andNN cx all $$ {\mathrm{NN}}_{\mathrm{cx}}^{\mathrm{all}} $$ was trained on all slice orientations. The resulting maps were compared to B1 +-reference scans using different quality metrics. The proposed network was applied in-vivo at 7T in two unseen test subjects using dynamic kt-point pulses.

RESULTS

Predicted B1 +-maps demonstrated a high similarity with measured B1 +-maps across multiple orientations. The estimation matched the reference with a mean relative error in the magnitude of (2.70 ± 2.86)% and mean absolute phase difference of (6.70 ± 1.99)° for transversal, (1.82 ± 0.69)% and (4.25 ± 1.62)° for sagittal (NN cx sag $$ {\mathrm{NN}}_{\mathrm{cx}}^{\mathrm{sag}} $$ ), as well as (1.33 ± 0.27)% and (2.66 ± 0.60)° for coronal slices (NN cx cor $$ {\mathrm{NN}}_{\mathrm{cx}}^{\mathrm{cor}} $$ ) considering brain tissue.NN cx all $$ {\mathrm{NN}}_{\mathrm{cx}}^{\mathrm{all}} $$ trained on all orientations enables a robust prediction of B1 +-maps across different orientations. Achieving a homogenous excitation over the whole brain for an in-vivo application displayed the approach's feasibility.

CONCLUSION

This study demonstrates the feasibility of utilizing complex-valued NNs to estimate multi-slice B1 +-maps in different slice orientations from localizer scans in the human brain at 7T.

The state-of-the-art in cardiac MRI reconstruction: Results of the CMRxRecon challenge in MICCAI 2023

Cardiac magnetic resonance imaging (MRI) provides detailed and quantitative evaluation of the heart's structure, function, and tissue characteristics with high-resolution spatial-temporal imaging. However, its slow imaging speed and motion artifacts are notable limitations. Undersampling reconstruction, especially data-driven algorithms, has emerged as a promising solution to accelerate scans and enhance imaging performance using highly under-sampled data. Nevertheless, the scarcity of publicly available cardiac k-space datasets and evaluation platform hinder the development of data-driven reconstruction algorithms. To address this issue, we organized the Cardiac MRI Reconstruction Challenge (CMRxRecon) in 2023, in collaboration with the 26th International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI). CMRxRecon presented an extensive k-space dataset comprising cine and mapping raw data, accompanied by detailed annotations of cardiac anatomical structures. With overwhelming participation, the challenge attracted more than 285 teams and over 600 participants. Among them, 22 teams successfully submitted Docker containers for the testing phase, with 7 teams submitted for both cine and mapping tasks. All teams use deep learning based approaches, indicating that deep learning has predominately become a promising solution for the problem. The first-place winner of both tasks utilizes the E2E-VarNet architecture as backbones. In contrast, U-Net is still the most popular backbone for both multi-coil and single-coil reconstructions. This paper provides a comprehensive overview of the challenge design, presents a summary of the submitted results, reviews the employed methods, and offers an in-depth discussion that aims to inspire future advancements in cardiac MRI reconstruction models. The summary emphasizes the effective strategies observed in Cardiac MRI reconstruction, including backbone architecture, loss function, pre-processing techniques, physical modeling, and model complexity, thereby providing valuable insights for further developments in this field.

Attention incorporated network for sharing low-rank, image and k-space information during MR image reconstruction to achieve single breath-hold cardiac Cine imaging

Cardiac Cine Magnetic Resonance Imaging (MRI) provides an accurate assessment of heart morphology and function in clinical practice. However, MRI requires long acquisition times, with recent deep learning-based methods showing great promise to accelerate imaging and enhance reconstruction quality. Existing networks exhibit some common limitations that constrain further acceleration possibilities, including single-domain learning, reliance on a single regularization term, and equal feature contribution. To address these limitations, we propose to embed information from multiple domains, including low-rank, image, and k-space, in a novel deep learning network for MRI reconstruction, which we denote as A-LIKNet. A-LIKNet adopts a parallel-branch structure, enabling independent learning in the k-space and image domain. Coupled information sharing layers realize the information exchange between domains. Furthermore, we introduce attention mechanisms into the network to assign greater weights to more critical coils or important temporal frames. Training and testing were conducted on an in-house dataset, including 91 cardiovascular patients and 38 healthy subjects scanned with 2D cardiac Cine using retrospective undersampling. Additionally, we evaluated A-LIKNet on the real-time prospectively undersampled data from the OCMR dataset. The results demonstrate that our proposed A-LIKNet outperforms existing methods and provides high-quality reconstructions. The network can effectively reconstruct highly retrospectively undersampled dynamic MR images up to 24× accelerations, indicating its potential for single breath-hold imaging.

Accelerated cardiac cine with spatio-coil regularized deep learning reconstruction

PURPOSE

To develop an iterative deep learning (DL) reconstruction with spatio-coil regularization and multichannel k-space data consistency for accelerated cine imaging.

METHODS

This study proposes a Spatio-Coil Regularized DL (SCR-DL) approach for iterative deep learning reconstruction incorporating multicoil information in data consistency and regularizer. SCR-DL uses shift-invariant convolutional kernels to interpolate missing k-space lines and reconstruct individual coil images, followed by a regularizer that operates simultaneously across spatial and coil dimensions using learned image priors. At 8-fold acceleration, SCR-DL was compared with Generalized Autocalibrating Partially Parallel Acquisition (GRAPPA), sensitivity encoding (SENSE)-based DL and spatio-temporal regularized (STR)-DL reconstruction. In the retrospective undersampled cine, images were quantitatively evaluated using normalized mean square error (NMSE) and structural similarity index measure (SSIM). Additionally, agreement for left-ventricular ejection fraction and left-ventricular mass were assessed using prospectively accelerated cine images at 2-fold and 8-fold accelerations.

RESULTS

The SCR-DL algorithm successfully reconstructed highly accelerated cine images. SCR-DL had significant improvements in NMSE (0.03 ± 0.02) and SSIM (91.4% ± 2.7%) compared with GRAPPA (NMSE: 0.09 ± 0.04, SSIM: 69.9% ± 11.1%; p < 0.001), SENSE-DL (NMSE: 0.07 ± 0.04, SSIM: 86.9% ± 3.2%; p < 0.001), and STR-DL (NMSE: 0.04 ± 0.03, SSIM: 90.0% ± 2.5%; p < 0.001) with retrospective undersampled cine. Despite the 3-fold reduction in scan time, there was no difference between left-ventricular ejection fraction (59.8 ± 4.5 vs. 60.8 ± 4.8, p = 0.46) or left-ventricular mass (73.6 ± 19.4 g vs. 73.2 ± 19.7 g, p = 0.95) between R = 2 and R = 8 prospectively accelerated cine images.

CONCLUSIONS

SCR-DL enabled highly accelerated cardiac cine imaging, significantly reducing breath-hold time. Compared with GRAPPA or SENSE-DL, images reconstructed with SCR-DL showed superior NMSE and SSIM.

Unsupervised reconstruction of accelerated cardiac cine MRI using neural fields

BACKGROUND

Cardiac cine MRI is the gold standard for cardiac functional assessment, but the inherently slow acquisition process creates the necessity of reconstruction approaches for accelerated undersampled acquisitions. Several regularization approaches that exploit spatial-temporal redundancy have been proposed to reconstruct undersampled cardiac cine MRI. More recently, methods based on supervised deep learning have been also proposed to further accelerate acquisition and reconstruction. However, these techniques rely on usually large dataset for training, which are not always available and might introduce biases.

METHODS

In this work we propose NF-cMRI, an unsupervised approach based on implicit neural field representations for cardiac cine MRI. We evaluate our method in in-vivo undersampled golden-angle radial multi-coil acquisitions for undersampling factors of 13x, 17x and 26x.

RESULTS

The proposed method achieves excellent scores in sharpness and robustness to artifacts and comparable or improved spatial-temporal depiction than state-of-the-art conventional and unsupervised deep learning reconstruction techniques.

CONCLUSIONS

We have demonstrated NF-cMRI potential for cardiac cine MRI reconstruction with highly undersampled data.

Knowledge-driven deep learning for fast MR imaging: Undersampled MR image reconstruction from supervised to un-supervised learning

Deep learning (DL) has emerged as a leading approach in accelerating MRI. It employs deep neural networks to extract knowledge from available datasets and then applies the trained networks to reconstruct accurate images from limited measurements. Unlike natural image restoration problems, MRI involves physics-based imaging processes, unique data properties, and diverse imaging tasks. This domain knowledge needs to be integrated with data-driven approaches. Our review will introduce the significant challenges faced by such knowledge-driven DL approaches in the context of fast MRI along with several notable solutions, which include learning neural networks and addressing different imaging application scenarios. The traits and trends of these techniques have also been given which have shifted from supervised learning to semi-supervised learning, and finally, to unsupervised learning methods. In addition, MR vendors' choices of DL reconstruction have been provided along with some discussions on open questions and future directions, which are critical for the reliable imaging systems.

The beating heart: artificial intelligence for cardiovascular application in the clinic

Artificial intelligence (AI) integration in cardiac magnetic resonance imaging presents new and exciting avenues for advancing patient care, automating post-processing tasks, and enhancing diagnostic precision and outcomes. The use of AI significantly streamlines the examination workflow through the reduction of acquisition and postprocessing durations, coupled with the automation of scan planning and acquisition parameters selection. This has led to a notable improvement in examination workflow efficiency, a reduction in operator variability, and an enhancement in overall image quality. Importantly, AI unlocks new possibilities to achieve spatial resolutions that were previously unattainable in patients. Furthermore, the potential for low-dose and contrast-agent-free imaging represents a stride toward safer and more patient-friendly diagnostic procedures. Beyond these benefits, AI facilitates precise risk stratification and prognosis evaluation by adeptly analysing extensive datasets. This comprehensive review article explores recent applications of AI in the realm of cardiac magnetic resonance imaging, offering insights into its transformative potential in the field.

Predictive uncertainty in deep learning-based MR image reconstruction using deep ensembles: Evaluation on the fastMRI data set

PURPOSE

To estimate pixel-wise predictive uncertainty for deep learning-based MR image reconstruction and to examine the impact of domain shifts and architecture robustness.

METHODS

Uncertainty prediction could provide a measure for robustness of deep learning (DL)-based MR image reconstruction from undersampled data. DL methods bear the risk of inducing reconstruction errors like in-painting of unrealistic structures or missing pathologies. These errors may be obscured by visual realism of DL reconstruction and thus remain undiscovered. Furthermore, most methods are task-agnostic and not well calibrated to domain shifts. We propose a strategy that estimates aleatoric (data) and epistemic (model) uncertainty, which entails training a deep ensemble (epistemic) with nonnegative log-likelihood (aleatoric) loss in addition to the conventional applied losses terms. The proposed procedure can be paired with any DL reconstruction, enabling investigations of their predictive uncertainties on a pixel level. Five different architectures were investigated on the fastMRI database. The impact on the examined uncertainty of in-distributional and out-of-distributional data with changes to undersampling pattern, imaging contrast, imaging orientation, anatomy, and pathology were explored.

RESULTS

Predictive uncertainty could be captured and showed good correlation to normalized mean squared error. Uncertainty was primarily focused along the aliased anatomies and on hyperintense and hypointense regions. The proposed uncertainty measure was able to detect disease prevalence shifts. Distinct predictive uncertainty patterns were observed for changing network architectures.

CONCLUSION

The proposed approach enables aleatoric and epistemic uncertainty prediction for DL-based MR reconstruction with an interpretable examination on a pixel level.

Improving the efficiency and accuracy of cardiovascular magnetic resonance with artificial intelligence-review of evidence and proposition of a roadmap to clinical translation

BACKGROUND

Cardiovascular magnetic resonance (CMR) is an important imaging modality for the assessment of heart disease; however, limitations of CMR include long exam times and high complexity compared to other cardiac imaging modalities. Recently advancements in artificial intelligence (AI) technology have shown great potential to address many CMR limitations. While the developments are remarkable, translation of AI-based methods into real-world CMR clinical practice remains at a nascent stage and much work lies ahead to realize the full potential of AI for CMR.

METHODS

Herein we review recent cutting-edge and representative examples demonstrating how AI can advance CMR in areas such as exam planning, accelerated image reconstruction, post-processing, quality control, classification and diagnosis.

RESULTS

These advances can be applied to speed up and simplify essentially every application including cine, strain, late gadolinium enhancement, parametric mapping, 3D whole heart, flow, perfusion and others. AI is a unique technology based on training models using data. Beyond reviewing the literature, this paper discusses important AI-specific issues in the context of CMR, including (1) properties and characteristics of datasets for training and validation, (2) previously published guidelines for reporting CMR AI research, (3) considerations around clinical deployment, (4) responsibilities of clinicians and the need for multi-disciplinary teams in the development and deployment of AI in CMR, (5) industry considerations, and (6) regulatory perspectives.

CONCLUSIONS

Understanding and consideration of all these factors will contribute to the effective and ethical deployment of AI to improve clinical CMR.

JUST-Net: Jointly unrolled cross-domain optimization based spatio-temporal reconstruction network for accelerated 3D myelin water imaging

PURPOSE

We introduced a novel reconstruction network, jointly unrolled cross-domain optimization-based spatio-temporal reconstruction network (JUST-Net), aimed at accelerating 3D multi-echo gradient-echo (mGRE) data acquisition and improving the quality of resulting myelin water imaging (MWI) maps.

METHOD

An unrolled cross-domain spatio-temporal reconstruction network was designed. The main idea is to combine frequency and spatio-temporal image feature representations and to sequentially implement convolution layers in both domains. The k-space subnetwork utilizes shared information from adjacent frames, whereas the image subnetwork applies separate convolutions in both spatial and temporal dimensions. The proposed reconstruction network was evaluated for both retrospectively and prospectively accelerated acquisition. Furthermore, it was assessed in simulation studies and real-world cases with k-space corruptions to evaluate its potential for motion artifact reduction.

RESULTS

The proposed JUST-Net enabled highly reproducible and accelerated 3D mGRE acquisition for whole-brain MWI, reducing the acquisition time from fully sampled 15:23 to 2:22 min within a 3-min reconstruction time. The normalized root mean squared error of the reconstructed mGRE images increased by less than 4.0%, and the correlation coefficients for MWI showed a value of over 0.68 when compared to the fully sampled reference. Additionally, the proposed method demonstrated a mitigating effect on both simulated and clinical motion-corrupted cases.

CONCLUSION

The proposed JUST-Net has demonstrated the capability to achieve high acceleration factors for 3D mGRE-based MWI, which is expected to facilitate widespread clinical applications of MWI.

High-resolution spiral real-time cardiac cine imaging with deep learning-based rapid image reconstruction and quantification

The objective of the current study was to develop and evaluate a DEep learning-based rapid Spiral Image REconstruction (DESIRE) and deep learning (DL)-based segmentation approach to quantify the left ventricular ejection fraction (LVEF) for high-resolution spiral real-time cine imaging, including 2D balanced steady-state free precession imaging at 1.5 T and gradient echo (GRE) imaging at 1.5 and 3 T. A 3D U-Net-based image reconstruction network and 2D U-Net-based image segmentation network were proposed and evaluated. Low-rank plus sparse (L+S) served as the reference for the image reconstruction network and manual contouring of the left ventricle was the reference of the segmentation network. To assess the image reconstruction quality, structural similarity index, peak signal-to-noise ratio, normalized root-mean-square error, and blind grading by two experienced cardiologists (5: excellent; 1: poor) were performed. To assess the segmentation performance, quantification of the LVEF on GRE imaging at 3 T was compared with the quantification from manual contouring. Excellent performance was demonstrated by the proposed technique. In terms of image quality, there was no difference between L+S and the proposed DESIRE technique. For quantification analysis, the proposed DL method was not different to the manual segmentation method (p > 0.05) in terms of quantification of LVEF. The reconstruction time for DESIRE was ~32 s (including nonuniform fast Fourier transform [NUFFT]) per dynamic series (40 frames), while the reconstruction time of L+S with GPU acceleration was approximately 3 min. The DL segmentation takes less than 5 s. In conclusion, the proposed DL-based image reconstruction and quantification techniques enabled 1-min image reconstruction for the whole heart and quantification with automatic reconstruction and quantification of the left ventricle function for high-resolution spiral real-time cine imaging with excellent performance.

HyperSLICE: HyperBand optimized spiral for low-latency interactive cardiac examination

PURPOSE

Interactive cardiac MRI is used for fast scan planning and MR-guided interventions. However, the requirement for real-time acquisition and near-real-time visualization constrains the achievable spatio-temporal resolution. This study aims to improve interactive imaging resolution through optimization of undersampled spiral sampling and leveraging of deep learning for low-latency reconstruction (deep artifact suppression).

METHODS

A variable density spiral trajectory was parametrized and optimized via HyperBand to provide the best candidate trajectory for rapid deep artifact suppression. Training data consisted of 692 breath-held CINEs. The developed interactive sequence was tested in simulations and prospectively in 13 subjects (10 for image evaluation, 2 during catheterization, 1 during exercise). In the prospective study, the optimized framework-HyperSLICE- was compared with conventional Cartesian real-time and breath-hold CINE imaging in terms quantitative and qualitative image metrics. Statistical differences were tested using Friedman chi-squared tests with post hoc Nemenyi test (p < 0.05).

RESULTS

In simulations the normalized RMS error, peak SNR, structural similarity, and Laplacian energy were all statistically significantly higher using optimized spiral compared to radial and uniform spiral sampling, particularly after scan plan changes (structural similarity: 0.71 vs. 0.45 and 0.43). Prospectively, HyperSLICE enabled a higher spatial and temporal resolution than conventional Cartesian real-time imaging. The pipeline was demonstrated in patients during catheter pull back, showing sufficiently fast reconstruction for interactive imaging.

CONCLUSION

HyperSLICE enables high spatial and temporal resolution interactive imaging. Optimizing the spiral sampling enabled better overall image quality and superior handling of image transitions compared with radial and uniform spiral trajectories.

A deep unrolled neural network for real-time MRI-guided brain intervention

Accurate navigation and targeting are critical for neurological interventions including biopsy and deep brain stimulation. Real-time image guidance further improves surgical planning and MRI is ideally suited for both pre- and intra-operative imaging. However, balancing spatial and temporal resolution is a major challenge for real-time interventional MRI (i-MRI). Here, we proposed a deep unrolled neural network, dubbed as LSFP-Net, for real-time i-MRI reconstruction. By integrating LSFP-Net and a custom-designed, MR-compatible interventional device into a 3 T MRI scanner, a real-time MRI-guided brain intervention system is proposed. The performance of the system was evaluated using phantom and cadaver studies. 2D/3D real-time i-MRI was achieved with temporal resolutions of 80/732.8 ms, latencies of 0.4/3.66 s including data communication, processing and reconstruction time, and in-plane spatial resolution of 1 × 1 mm2. The results demonstrated that the proposed method enables real-time monitoring of the remote-controlled brain intervention, and showed the potential to be readily integrated into diagnostic scanners for image-guided neurosurgery.

Data-efficient Bayesian learning for radial dynamic MR reconstruction

BACKGROUND

Cardiac MRI has become the gold-standard imaging technique for assessing cardiovascular morphology and function. In spite of this, its slow data acquisition process presents imaging challenges due to the motion from heartbeats, respiration, and blood flow. In recent studies, deep learning (DL) algorithms have shown promising results for the task of image reconstruction. However, there have been instances where they have introduced artifacts that may be misinterpreted as pathologies or may obscure the detection of pathologies. Therefore, it is important to obtain a metric, such as the uncertainty of the network output, that identifies such artifacts. However, this can be quite challenging for large-scale image reconstruction problems such as dynamic multi-coil non-Cartesian MRI.

PURPOSE

To efficiently quantify uncertainties of a physics-informed DL-based image reconstruction method for a large-scale accelerated 2D multi-coil dynamic radial MRI reconstruction problem, and demonstrate the benefits of physics-informed DL over model-agnostic DL in reducing uncertainties while at the same time improving image quality.

METHODS

We extended a recently proposed physics-informed 2D U-Net that learns spatio-temporal slices (named XT-YT U-Net), and employed it for the task of uncertainty quantification (UQ) by using Monte Carlo dropout and a Gaussian negative log-likelihood loss function. Our data comprised 2D dynamic MR images acquired with a radial balanced steady-state free precession sequence. The XT-YT U-Net, which allows for training with a limited amount of data, was trained and validated on a dataset of 15 healthy volunteers, and further tested on data from four patients. An extensive comparison between physics-informed and model-agnostic neural networks (NNs) concerning the obtained image quality and uncertainty estimates was performed. Further, we employed calibration plots to assess the quality of the UQ.

RESULTS

The inclusion of the MR-physics model of data acquisition as a building block in the NN architecture led to higher image quality (NRMSE:- 33 ± 8.2 % $-33 \pm 8.2 \%$ , PSNR:6.3 ± 1.3 % $6.3 \pm 1.3 \%$ , and SSIM:1.9 ± 0.96 % $1.9 \pm 0.96 \%$ ), lower uncertainties (- 46 ± 8.7 % $-46 \pm 8.7 \%$ ), and, based on the calibration plots, an improved UQ compared to its model-agnostic counterpart. Furthermore, the UQ information can be used to differentiate between anatomical structures (e.g., coronary arteries, ventricle boundaries) and artifacts.

CONCLUSIONS

Using an XT-YT U-Net, we were able to quantify uncertainties of a physics-informed NN for a high-dimensional and computationally demanding 2D multi-coil dynamic MR imaging problem. In addition to improving the image quality, embedding the acquisition model in the network architecture decreased the reconstruction uncertainties as well as quantitatively improved the UQ. The UQ provides additional information to assess the performance of different network approaches.

Accelerated respiratory-resolved 4D-MRI with separable spatio-temporal neural networks

BACKGROUND

Respiratory-resolved four-dimensional magnetic resonance imaging (4D-MRI) provides essential motion information for accurate radiation treatments of mobile tumors. However, obtaining high-quality 4D-MRI suffers from long acquisition and reconstruction times.

PURPOSE

To develop a deep learning architecture to quickly acquire and reconstruct high-quality 4D-MRI, enabling accurate motion quantification for MRI-guided radiotherapy (MRIgRT).

METHODS

A small convolutional neural network called MODEST is proposed to reconstruct 4D-MRI by performing a spatial and temporal decomposition, omitting the need for 4D convolutions to use all the spatio-temporal information present in 4D-MRI. This network is trained on undersampled 4D-MRI after respiratory binning to reconstruct high-quality 4D-MRI obtained by compressed sensing reconstruction. The network is trained, validated, and tested on 4D-MRI of 28 lung cancer patients acquired with a T1-weighted golden-angle radial stack-of-stars (GA-SOS) sequence. The 4D-MRI of 18, 5, and 5 patients were used for training, validation, and testing. Network performances are evaluated on image quality measured by the structural similarity index (SSIM) and motion consistency by comparing the position of the lung-liver interface on undersampled 4D-MRI before and after respiratory binning. The network is compared to conventional architectures such as a U-Net, which has 30 times more trainable parameters.

RESULTS

MODEST can reconstruct high-quality 4D-MRI with higher image quality than a U-Net, despite a thirty-fold reduction in trainable parameters. High-quality 4D-MRI can be obtained using MODEST in approximately 2.5 min, including acquisition, processing, and reconstruction.

CONCLUSION

High-quality accelerated 4D-MRI can be obtained using MODEST, which is particularly interesting for MRIgRT.

A computationally-inexpensive strategy in CT image data augmentation for robust deep learning classification in the early stages of an outbreak

Coronavirus disease 2019 (COVID-19) has spread globally for over three years, and chest computed tomography (CT) has been used to diagnose COVID-19 and identify lung damage in COVID-19 patients. Given its widespread, CT will remain a common diagnostic tool in future pandemics, but its effectiveness at the beginning of any pandemic will depend strongly on the ability to classify CT scans quickly and correctly when only limited resources are available, as it will happen inevitably again in future pandemics. Here, we resort into the transfer learning procedure and limited hyperparameters to use as few computing resources as possible for COVID-19 CT images classification. Advanced Normalisation Tools (ANTs) are used to synthesise images as augmented/independent data and trained on EfficientNet to investigate the effect of synthetic images. On the COVID-CT dataset, classification accuracy increases from 91.15% to 95.50% and Area Under the Receiver Operating Characteristic (AUC) from 96.40% to 98.54%. We also customise a small dataset to simulate data collected in the early stages of the outbreak and report an improvement in accuracy from 85.95% to 94.32% and AUC from 93.21% to 98.61%. This study provides a feasible Low-Threshold, Easy-To-Deploy and Ready-To-Use solution with a relatively low computational cost for medical image classification at an early stage of an outbreak in which scarce data are available and traditional data augmentation may fail. Hence, it would be most suitable for low-resource settings.

Automated diagnosis of cardiovascular diseases from cardiac magnetic resonance imaging using deep learning models: A review

In recent years, cardiovascular diseases (CVDs) have become one of the leading causes of mortality globally. At early stages, CVDs appear with minor symptoms and progressively get worse. The majority of people experience symptoms such as exhaustion, shortness of breath, ankle swelling, fluid retention, and other symptoms when starting CVD. Coronary artery disease (CAD), arrhythmia, cardiomyopathy, congenital heart defect (CHD), mitral regurgitation, and angina are the most common CVDs. Clinical methods such as blood tests, electrocardiography (ECG) signals, and medical imaging are the most effective methods used for the detection of CVDs. Among the diagnostic methods, cardiac magnetic resonance imaging (CMRI) is increasingly used to diagnose, monitor the disease, plan treatment and predict CVDs. Coupled with all the advantages of CMR data, CVDs diagnosis is challenging for physicians as each scan has many slices of data, and the contrast of it might be low. To address these issues, deep learning (DL) techniques have been employed in the diagnosis of CVDs using CMR data, and much research is currently being conducted in this field. This review provides an overview of the studies performed in CVDs detection using CMR images and DL techniques. The introduction section examined CVDs types, diagnostic methods, and the most important medical imaging techniques. The following presents research to detect CVDs using CMR images and the most significant DL methods. Another section discussed the challenges in diagnosing CVDs from CMRI data. Next, the discussion section discusses the results of this review, and future work in CVDs diagnosis from CMR images and DL techniques are outlined. Finally, the most important findings of this study are presented in the conclusion section.

Uncertainty-aware physics-driven deep learning network for free-breathing liver fat and R2 * quantification using self-gated stack-of-radial MRI

PURPOSE

To develop a deep learning-based method for rapid liver proton-density fat fraction (PDFF) and R2 * quantification with built-in uncertainty estimation using self-gated free-breathing stack-of-radial MRI.

METHODS

This work developed an uncertainty-aware physics-driven deep learning network (UP-Net) to (1) suppress radial streaking artifacts because of undersampling after self-gating, (2) calculate accurate quantitative maps, and (3) provide pixel-wise uncertainty maps. UP-Net incorporated a phase augmentation strategy, generative adversarial network architecture, and an MRI physics loss term based on a fat-water and R2 * signal model. UP-Net was trained and tested using free-breathing multi-echo stack-of-radial MRI data from 105 subjects. UP-Net uncertainty scores were calibrated in a validation dataset and used to predict quantification errors for liver PDFF and R2 * in a testing dataset.

RESULTS

Compared with images reconstructed using compressed sensing (CS), UP-Net achieved structural similarity index >0.87 and normalized root mean squared error <0.18. Compared with reference quantitative maps generated using CS and graph-cut (GC) algorithms, UP-Net achieved low mean differences (MD) for liver PDFF (-0.36%) and R2 * (-0.37 s-1 ). Compared with breath-holding Cartesian MRI results, UP-Net achieved low MD for liver PDFF (0.53%) and R2 * (6.75 s-1 ). UP-Net uncertainty scores predicted absolute liver PDFF and R2 * errors with low MD of 0.27% and 0.12 s-1 compared to CS + GC results. The computational time for UP-Net was 79 ms/slice, whereas CS + GC required 3.2 min/slice.

CONCLUSION

UP-Net rapidly calculates accurate liver PDFF and R2 * maps from self-gated free-breathing stack-of-radial MRI. The pixel-wise uncertainty maps from UP-Net predict quantification errors in the liver.

Fast Deformable Image Registration for Real-Time Target Tracking During Radiation Therapy Using Cine MRI and Deep Learning

PURPOSE

We developed a deep learning (DL) model for fast deformable image registration using 2-dimensional sagittal cine magnetic resonance imaging (MRI) acquired during radiation therapy and evaluated its potential for real-time target tracking compared with conventional image registration methods.

METHODS AND MATERIALS

Our DL model uses a pair of cine MRI images as input and provides a motion vector field (MVF) as output. The MVF is then applied to align the input images. A retrospective study was conducted to train and evaluate our model using cine MRI data from patients undergoing treatment for abdominal and thoracic tumors. For each treatment fraction, MR-linear accelerator delivery log files, tracking videos, and cine image files were analyzed. Individual MRI frames were temporally sampled to construct a large set of image registration pairs used to evaluate multiple methods. The DL model was optimized using 5-fold cross validation, and model outputs (transformed images and MVFs) using test set images were saved for comparison with 3 conventional registration methods (affine, b-spline, and demons). Evaluation metrics were 3-fold: (1) registration error, (2) MVF stability (both spatial and temporal), and (3) average computation time.

RESULTS

We analyzed >21 hours of cine MRI (>629,000 frames) acquired during 86 treatment fractions from 21 patients. In a test set of 10,320 image registration pairs, DL registration outperformed conventional methods in both registration error (affine, b-spline, demons, DL; root mean square error: 0.067, 0.040, 0.036, 0.032; paired t test demons vs DL: t[20] = 4.2, P < .001) and computation time per frame (51, 1150, 4583, 8 ms). Among deformable methods, spatial stability of resulting MVFs was comparable; however, the DL model had significantly improved temporal consistency.

CONCLUSIONS

DL-based image registration can leverage large-scale MR cine data sets to outperform conventional registration methods and is a promising solution for real-time deformable motion estimation in radiation therapy.

Rapid estimation of 2D relative B 1 + -maps from localizers in the human heart at 7T using deep learning

PURPOSE

Subject-tailored parallel transmission pulses for ultra-high fields body applications are typically calculated based on subject-specific B 1 + $$ {\mathrm{B}}_1^{+} $$ -maps of all transmit channels, which require lengthy adjustment times. This study investigates the feasibility of using deep learning to estimate complex, channel-wise, relative 2D B 1 + $$ {\mathrm{B}}_1^{+} $$ -maps from a single gradient echo localizer to overcome long calibration times.

METHODS

126 channel-wise, complex, relative 2D B 1 + $$ {\mathrm{B}}_1^{+} $$ -maps of the human heart from 44 subjects were acquired at 7T using a Cartesian, cardiac gradient-echo sequence obtained under breath-hold to create a library for network training and cross-validation. The deep learning predicted maps were qualitatively compared to the ground truth. Phase-only B 1 + $$ {\mathrm{B}}_1^{+} $$ -shimming was subsequently performed on the estimated B 1 + $$ {\mathrm{B}}_1^{+} $$ -maps for a region of interest covering the heart. The proposed network was applied at 7T to 3 unseen test subjects.

RESULTS

The deep learning-based B 1 + $$ {\mathrm{B}}_1^{+} $$ -maps, derived in approximately 0.2 seconds, match the ground truth for the magnitude and phase. The static, phase-only pulse design performs best when maximizing the mean transmission efficiency. In-vivo application of the proposed network to unseen subjects demonstrates the feasibility of this approach: the network yields predicted B 1 + $$ {\mathrm{B}}_1^{+} $$ -maps comparable to the acquired ground truth and anatomical scans reflect the resulting B 1 + $$ {\mathrm{B}}_1^{+} $$ -pattern using the deep learning-based maps.

CONCLUSION

The feasibility of estimating 2D relative B 1 + $$ {\mathrm{B}}_1^{+} $$ -maps from initial localizer scans of the human heart at 7T using deep learning is successfully demonstrated. Because the technique requires only sub-seconds to derive channel-wise B 1 + $$ {\mathrm{B}}_1^{+} $$ -maps, it offers high potential for advancing clinical body imaging at ultra-high fields.

Undersampling artifact reduction for free-breathing 3D stack-of-radial MRI based on a deep adversarial learning network

PURPOSE

Stack-of-radial MRI allows free-breathing abdominal scans, however, it requires relatively long acquisition time. Undersampling reduces scan time but can cause streaking artifacts and degrade image quality. This study developed deep learning networks with adversarial loss and evaluated the performance of reducing streaking artifacts and preserving perceptual image sharpness.

METHODS

A 3D generative adversarial network (GAN) was developed for reducing streaking artifacts in stack-of-radial abdominal scans. Training and validation datasets were self-gated to 5 respiratory states to reduce motion artifacts and to effectively augment the data. The network used a combination of three loss functions to constrain the anatomy and preserve image quality: adversarial loss, mean-squared-error loss and structural similarity index loss. The performance of the network was investigated for 3-5 times undersampled data from 2 institutions. The performance of the GAN for 5 times accelerated images was compared with a 3D U-Net and evaluated using quantitative NMSE, SSIM and region of interest (ROI) measurements as well as qualitative scores of radiologists.

RESULTS

The 3D GAN showed similar NMSE (0.0657 vs. 0.0559, p = 0.5217) and significantly higher SSIM (0.841 vs. 0.798, p < 0.0001) compared to U-Net. ROI analysis showed GAN removed streaks in both the background air and the tissue and was not significantly different from the reference mean and variations. Radiologists' scores showed GAN had a significant improvement of 1.6 point (p = 0.004) on a 4-point scale in streaking score while no significant difference in sharpness score compared to the input.

CONCLUSION

3D GAN removes streaking artifacts and preserves perceptual image details.

Signal intensity informed multi-coil encoding operator for physics-guided deep learning reconstruction of highly accelerated myocardial perfusion CMR

PURPOSE

To develop a physics-guided deep learning (PG-DL) reconstruction strategy based on a signal intensity informed multi-coil (SIIM) encoding operator for highly-accelerated simultaneous multislice (SMS) myocardial perfusion cardiac MRI (CMR).

METHODS

First-pass perfusion CMR acquires highly-accelerated images with dynamically varying signal intensity/SNR following the administration of a gadolinium-based contrast agent. Thus, using PG-DL reconstruction with a conventional multi-coil encoding operator leads to analogous signal intensity variations across different time-frames at the network output, creating difficulties in generalization for varying SNR levels. We propose to use a SIIM encoding operator to capture the signal intensity/SNR variations across time-frames in a reformulated encoding operator. This leads to a more uniform/flat contrast at the output of the PG-DL network, facilitating generalizability across time-frames. PG-DL reconstruction with the proposed SIIM encoding operator is compared to PG-DL with conventional encoding operator, split slice-GRAPPA, locally low-rank (LLR) regularized reconstruction, low-rank plus sparse (L + S) reconstruction, and regularized ROCK-SPIRiT.

RESULTS

Results on highly accelerated free-breathing first pass myocardial perfusion CMR at three-fold SMS and four-fold in-plane acceleration show that the proposed method improves upon the reconstruction methods use for comparison. Substantial noise reduction is achieved compared to split slice-GRAPPA, and aliasing artifacts reduction compared to LLR regularized reconstruction, L + S reconstruction and PG-DL with conventional encoding. Furthermore, a qualitative reader study indicated that proposed method outperformed all methods.

CONCLUSION

PG-DL reconstruction with the proposed SIIM encoding operator improves generalization across different time-frames /SNRs in highly accelerated perfusion CMR.

FlowRAU-Net: Accelerated 4D Flow MRI of Aortic Valvular Flows With a Deep 2D Residual Attention Network

In this work, we propose a novel deep learning reconstruction framework for rapid and accurate reconstruction of 4D flow MRI data. Reconstruction is performed on a slice-by-slice basis by reducing artifacts in zero-filled reconstructed complex images obtained from undersampled k-space. A deep residual attention network FlowRAU-Net is proposed, trained separately for each encoding direction with 2D complex image slices extracted from complex 4D images at each temporal frame and slice position. The network was trained and tested on 4D flow MRI data of aortic valvular flow in 18 human subjects. Performance of the reconstructions was measured in terms of image quality, 3-D velocity vector accuracy, and accuracy in hemodynamic parameters. Reconstruction performance was measured for three different k-space undersamplings and compared with one state of the art compressed sensing reconstruction method and three deep learning-based reconstruction methods. The proposed method outperforms state of the art methods in all performance measures for all three different k-space undersamplings. Hemodynamic parameters such as blood flow rate and peak velocity from the proposed technique show good agreement with reference flow parameters. Visualization of the reconstructed image and velocity magnitude also shows excellent agreement with the fully sampled reference dataset. Moreover, the proposed method is computationally fast. Total 4D flow data (including all slices in space and time) for a subject can be reconstructed in 69 seconds on a single GPU. Although the proposed method has been applied to 4D flow MRI of aortic valvular flows, given a sufficient number of training samples, it should be applicable to other arterial flows.

3D Extrusion Printing of Biphasic Anthropomorphic Brain Phantoms Mimicking MR Relaxation Times Based on Alginate-Agarose-Carrageenan Blends

The availability of adapted phantoms mimicking different body parts is fundamental to establishing the stability and reliability of magnetic resonance imaging (MRI) methods. The primary purpose of such phantoms is the mimicking of physiologically relevant, contrast-creating relaxation times T1 and T2. For the head, frequently examined by MRI, an anthropomorphic design of brain phantoms would imply the discrimination of gray matter and white matter (WM) within defined, spatially distributed compartments. Multichannel extrusion printing allows the layer-by-layer fabrication of multiple pastelike materials in a spatially defined manner with a predefined shape. In this study, the advantages of this method are used to fabricate biphasic brain phantoms mimicking MR relaxation times and anthropomorphic geometry. The printable ink was based on purely naturally derived polymers: alginate as a calcium-cross-linkable gelling agent, agarose, ι-carrageenan, and GdCl3 in different concentrations (0-280 μmol kg-1) as the paramagnetic component. The suggested inks (e.g., 3Alg-1Agar-6Car) fulfilled the requirements of viscoelastic behavior and printability of large constructs (>150 mL). The microstructure and distribution of GdCl3 were assessed by scanning electron microscopy (SEM) with energy-dispersive X-ray spectroscopy (EDX). In closely monitored steps of technological development and characterization, from monophasic and biphasic samples as printable inks and cross-linked gels, we describe the construction of large-scale phantom models whose relaxation times were characterized and checked for stability over time.

IMPULSED model based cytological feature estimation with U-Net: Application to human brain tumor at 3T

PURPOSE

This work introduces and validates a deep-learning-based fitting method, which can rapidly provide accurate and robust estimation of cytological features of brain tumor based on the IMPULSED (imaging microstructural parameters using limited spectrally edited diffusion) model fitting with diffusion-weighted MRI data.

METHODS

The U-Net was applied to rapidly quantify extracellular diffusion coefficient (D_ex ), cell size (d), and intracellular volume fraction (v_in ) of brain tumor. At the training stage, the image-based training data, synthesized by randomizing quantifiable microstructural parameters within specific ranges, was used to train U-Net. At the test stage, the pre-trained U-Net was applied to estimate the microstructural parameters from simulated data and the in vivo data acquired on patients at 3T. The U-Net was compared with conventional non-linear least-squares (NLLS) fitting in simulations in terms of estimation accuracy and precision.

RESULTS

Our results confirm that the proposed method yields better fidelity in simulations and is more robust to noise than the NLLS fitting. For in vivo data, the U-Net yields obvious quality improvement in parameter maps, and the estimations of all parameters are in good agreement with the NLLS fitting. Moreover, our method is several orders of magnitude faster than the NLLS fitting (from about 5 min to <1 s).

CONCLUSION

The image-based training scheme proposed herein helps to improve the quality of the estimated parameters. Our deep-learning-based fitting method can estimate the cell microstructural parameters fast and accurately.

Accelerated 3D myelin water imaging using joint spatio-temporal reconstruction

PURPOSE

To enable acceleration in 3D multi-echo gradient echo (mGRE) acquisition for myelin water imaging (MWI) by combining joint parallel imaging (JPI) and joint deep learning (JDL).

METHODS

We implemented a multistep reconstruction process using both advanced parallel imaging and deep learning network which can utilize joint spatiotemporal components between the multi-echo images to further accelerate 3D mGRE acquisition for MWI. In the first step, JPI was performed to estimate missing k-space lines. Next, JDL was implemented to reduce residual artifacts and produce high-fidelity reconstruction by using variable splitting optimization consisting of spatiotemporal denoiser block, data consistency block, and weighted average block. The proposed method was evaluated for MWI with 2D Cartesian uniform under-sampling for each echo, enabling scan times of up to approximately 2 min for 2 mm × 2 mm × 2 mm $2\ {\rm mm} \times 2\ {\rm mm} \times 2\ {\rm mm}$ 3D coverage.

RESULTS

The proposed method showed acceptable MWI quality with improved quantitative values compared to both JPI and JDL methods individually. The improved performance of the proposed method was demonstrated by the low normalized mean-square error and high-frequency error norm values of the reconstruction with high similarity to the fully sampled MWI.

CONCLUSION

Joint spatiotemporal reconstruction approach by combining JPI and JDL can achieve high acceleration factors for 3D mGRE-based MWI.

DEep learning-based rapid Spiral Image REconstruction (DESIRE) for high-resolution spiral first-pass myocardial perfusion imaging

The objective of the current study was to develop and evaluate a DEep learning-based rapid Spiral Image REconstruction (DESIRE) technique for high-resolution spiral first-pass myocardial perfusion imaging with whole-heart coverage, to provide fast and accurate image reconstruction for both single-slice (SS) and simultaneous multislice (SMS) acquisitions. Three-dimensional U-Net-based image enhancement architectures were evaluated for high-resolution spiral perfusion imaging at 3 T. The SS and SMS MB = 2 networks were trained on SS perfusion images from 156 slices from 20 subjects. Structural similarity index (SSIM), peak signal-to-noise ratio (PSNR), and normalized root mean square error (NRMSE) were assessed, and prospective images were blindly graded by two experienced cardiologists (5: excellent; 1: poor). Excellent performance was demonstrated for the proposed technique. For SS, SSIM, PSNR, and NRMSE were 0.977 [0.972, 0.982], 42.113 [40.174, 43.493] dB, and 0.102 [0.080, 0.125], respectively, for the best network. For SMS MB = 2 retrospective data, SSIM, PSNR, and NRMSE were 0.961 [0.950, 0.969], 40.834 [39.619, 42.004] dB, and 0.107 [0.086, 0.133], respectively, for the best network. The image quality scores were 4.5 [4.1, 4.8], 4.5 [4.3, 4.6], 3.5 [3.3, 4], and 3.5 [3.3, 3.8] for SS DESIRE, SS L1-SPIRiT, MB = 2 DESIRE, and MB = 2 SMS-slice-L1-SPIRiT, respectively, showing no statistically significant difference (p = 1 and p = 1 for SS and SMS, respectively) between L1-SPIRiT and the proposed DESIRE technique. The network inference time was ~100 ms per dynamic perfusion series with DESIRE, while the reconstruction time of L1-SPIRiT with GPU acceleration was ~ 30 min. It was concluded that DESIRE enabled fast and high-quality image reconstruction for both SS and SMS MB = 2 whole-heart high-resolution spiral perfusion imaging.

Cardiac MR: From Theory to Practice

Cardiovascular disease (CVD) is the leading single cause of morbidity and mortality, causing over 17. 9 million deaths worldwide per year with associated costs of over $800 billion. Improving prevention, diagnosis, and treatment of CVD is therefore a global priority. Cardiovascular magnetic resonance (CMR) has emerged as a clinically important technique for the assessment of cardiovascular anatomy, function, perfusion, and viability. However, diversity and complexity of imaging, reconstruction and analysis methods pose some limitations to the widespread use of CMR. Especially in view of recent developments in the field of machine learning that provide novel solutions to address existing problems, it is necessary to bridge the gap between the clinical and scientific communities. This review covers five essential aspects of CMR to provide a comprehensive overview ranging from CVDs to CMR pulse sequence design, acquisition protocols, motion handling, image reconstruction and quantitative analysis of the obtained data. (1) The basic MR physics of CMR is introduced. Basic pulse sequence building blocks that are commonly used in CMR imaging are presented. Sequences containing these building blocks are formed for parametric mapping and functional imaging techniques. Commonly perceived artifacts and potential countermeasures are discussed for these methods. (2) CMR methods for identifying CVDs are illustrated. Basic anatomy and functional processes are described to understand the cardiac pathologies and how they can be captured by CMR imaging. (3) The planning and conduct of a complete CMR exam which is targeted for the respective pathology is shown. Building blocks are illustrated to create an efficient and patient-centered workflow. Further strategies to cope with challenging patients are discussed. (4) Imaging acceleration and reconstruction techniques are presented that enable acquisition of spatial, temporal, and parametric dynamics of the cardiac cycle. The handling of respiratory and cardiac motion strategies as well as their integration into the reconstruction processes is showcased. (5) Recent advances on deep learning-based reconstructions for this purpose are summarized. Furthermore, an overview of novel deep learning image segmentation and analysis methods is provided with a focus on automatic, fast and reliable extraction of biomarkers and parameters of clinical relevance.

Automated segmentation of biventricular contours in tissue phase mapping using deep learning

Tissue phase mapping (TPM) is an MRI technique for quantification of regional biventricular myocardial velocities. Despite its potential, clinical use is limited due to the requisite labor-intensive manual segmentation of cardiac contours for all time frames. The purpose of this study was to develop a deep learning (DL) network for automated segmentation of TPM images, without significant loss in segmentation and myocardial velocity quantification accuracy compared with manual segmentation. We implemented a multi-channel 3D (three dimensional; 2D + time) dense U-Net that trained on magnitude and phase images and combined cross-entropy, Dice, and Hausdorff distance loss terms to improve the segmentation accuracy and suppress unnatural boundaries. The dense U-Net was trained and tested with 150 multi-slice, multi-phase TPM scans (114 scans for training, 36 for testing) from 99 heart transplant patients (44 females, 1-4 scans/patient), where the magnitude and velocity-encoded (V_x , V_y , V_z ) images were used as input and the corresponding manual segmentation masks were used as reference. The accuracy of DL segmentation was evaluated using quantitative metrics (Dice scores, Hausdorff distance) and linear regression and Bland-Altman analyses on the resulting peak radial and longitudinal velocities (V_r and V_z ). The mean segmentation time was about 2 h per patient for manual and 1.9 ± 0.3 s for DL. Our network produced good accuracy (median Dice = 0.85 for left ventricle (LV), 0.64 for right ventricle (RV), Hausdorff distance = 3.17 pixels) compared with manual segmentation. Peak V_r and V_z measured from manual and DL segmentations were strongly correlated (R ≥ 0.88) and in good agreement with manual analysis (mean difference and limits of agreement for V_z and V_r were -0.05 ± 0.98 cm/s and -0.06 ± 1.18 cm/s for LV, and -0.21 ± 2.33 cm/s and 0.46 ± 4.00 cm/s for RV, respectively). The proposed multi-channel 3D dense U-Net was capable of reducing the segmentation time by 3,600-fold, without significant loss in accuracy in tissue velocity measurements.

Deep Learning Applications in Magnetic Resonance Imaging: Has the Future Become Present?

Deep learning technologies and applications demonstrate one of the most important upcoming developments in radiology. The impact and influence of these technologies on image acquisition and reporting might change daily clinical practice. The aim of this review was to present current deep learning technologies, with a focus on magnetic resonance image reconstruction. The first part of this manuscript concentrates on the basic technical principles that are necessary for deep learning image reconstruction. The second part highlights the translation of these techniques into clinical practice. The third part outlines the different aspects of image reconstruction techniques, and presents a review of the current literature regarding image reconstruction and image post-processing in MRI. The promising results of the most recent studies indicate that deep learning will be a major player in radiology in the upcoming years. Apart from decision and diagnosis support, the major advantages of deep learning magnetic resonance imaging reconstruction techniques are related to acquisition time reduction and the improvement of image quality. The implementation of these techniques may be the solution for the alleviation of limited scanner availability via workflow acceleration. It can be assumed that this disruptive technology will change daily routines and workflows permanently.

Learning Data Consistency and its Application to Dynamic MR Imaging

Magnetic resonance (MR) image reconstruction from undersampled k-space data can be formulated as a minimization problem involving data consistency and image prior. Existing deep learning (DL)-based methods for MR reconstruction employ deep networks to exploit the prior information and integrate the prior knowledge into the reconstruction under the explicit constraint of data consistency, without considering the real distribution of the noise. In this work, we propose a new DL-based approach termed Learned DC that implicitly learns the data consistency with deep networks, corresponding to the actual probability distribution of system noise. The data consistency term and the prior knowledge are both embedded in the weights of the networks, which provides an utterly implicit manner of learning reconstruction model. We evaluated the proposed approach with highly undersampled dynamic data, including the dynamic cardiac cine data with up to 24-fold acceleration and dynamic rectum data with the acceleration factor equal to the number of phases. Experimental results demonstrate the superior performance of the Learned DC both quantitatively and qualitatively than the state-of-the-art.

Deep-learning based super-resolution for 3D isotropic coronary MR angiography in less than a minute

PURPOSE

To develop and evaluate a novel and generalizable super-resolution (SR) deep-learning framework for motion-compensated isotropic 3D coronary MR angiography (CMRA), which allows free-breathing acquisitions in less than a minute.

METHODS

Undersampled motion-corrected reconstructions have enabled free-breathing isotropic 3D CMRA in ~5-10 min acquisition times. In this work, we propose a deep-learning-based SR framework, combined with non-rigid respiratory motion compensation, to shorten the acquisition time to less than 1 min. A generative adversarial network (GAN) is proposed consisting of two cascaded Enhanced Deep Residual Network generator, a trainable discriminator, and a perceptual loss network. A 16-fold increase in spatial resolution is achieved by reconstructing a high-resolution (HR) isotropic CMRA (0.9 mm3 or 1.2 mm3 ) from a low-resolution (LR) anisotropic CMRA (0.9 × 3.6 × 3.6 mm3 or 1.2 × 4.8 × 4.8 mm3 ). The impact and generalization of the proposed SRGAN approach to different input resolutions and operation on image and patch-level is investigated. SRGAN was evaluated on a retrospective downsampled cohort of 50 patients and on 16 prospective patients that were scanned with LR-CMRA in ~50 s under free-breathing. Vessel sharpness and length of the coronary arteries from the SR-CMRA is compared against the HR-CMRA.

RESULTS

SR-CMRA showed statistically significant (P < .001) improved vessel sharpness 34.1% ± 12.3% and length 41.5% ± 8.1% compared with LR-CMRA. Good generalization to input resolution and image/patch-level processing was found. SR-CMRA enabled recovery of coronary stenosis similar to HR-CMRA with comparable qualitative performance.

CONCLUSION

The proposed SR-CMRA provides a 16-fold increase in spatial resolution with comparable image quality to HR-CMRA while reducing the predictable scan time to <1 min.

Real-time deep artifact suppression using recurrent U-Nets for low-latency cardiac MRI

PURPOSE

Real-time low latency MRI is performed to guide various cardiac interventions. Real-time acquisitions often require iterative image reconstruction strategies, which lead to long reconstruction times. In this study, we aim to reconstruct highly undersampled radial real-time data with low latency using deep learning.

METHODS

A 2D U-Net with convolutional long short-term memory layers is proposed to exploit spatial and preceding temporal information to reconstruct highly accelerated tiny golden radial data with low latency. The network was trained using a dataset of breath-hold CINE data (including 770 time series from 7 different orientations). Synthetic paired data were created by retrospectively undersampling the magnitude images, and the network was trained to recover the target images. In the spirit of interventional imaging, the network was trained and tested for varying acceleration rates and orientations. Data were prospectively acquired and reconstructed in real time in 1 healthy subject interactively and in 3 patients who underwent catheterization. Images were visually compared to sliding window and compressed sensing reconstructions and a conventional Cartesian real-time sequence.

RESULTS

The proposed network generalized well to different acceleration rates and unseen orientations for all considered metrics in simulated data (less than 4% reduction in structural similarity index compared to similar acceleration and orientation-specific networks). The proposed reconstruction was demonstrated interactively, successfully depicting catheters in vivo with low latency (39 ms, including 19 ms for deep artifact suppression) and an image quality comparing favorably to other reconstructions.

CONCLUSION

Deep artifact suppression was successfully demonstrated in the time-critical application of non-Cartesian real-time interventional cardiac MR.

Tile-net for undersampled cardiovascular CINE magnetic resonance imaging

We propose the "Tile-net" method based on dividing an image into smaller tiles. Using the tile as the input to the neural network, the network is simplified substantially. The Tile-net learns at a much faster rate than the networks without tiling. The training and reconstruction times for the Tile-net are reduced by 40% and 33%, respectively compared to the networks without tiling. The Tile-net performance is evaluated through the normalized mean square error (NMSE), peak signal to noise ratio (PSNR), structure similarity index measure (SSIM) and the quality of the reconstructed image for test datasets. The Tile-net does not degrade performance; however, it reduces the NMSE by 0.3% compared to the networks without tiling.

Cine Cardiac MRI Motion Artifact Reduction Using a Recurrent Neural Network

Cine cardiac magnetic resonance imaging (MRI) is widely used for the diagnosis of cardiac diseases thanks to its ability to present cardiovascular features in excellent contrast. As compared to computed tomography (CT), MRI, however, requires a long scan time, which inevitably induces motion artifacts and causes patients' discomfort. Thus, there has been a strong clinical motivation to develop techniques to reduce both the scan time and motion artifacts. Given its successful applications in other medical imaging tasks such as MRI super-resolution and CT metal artifact reduction, deep learning is a promising approach for cardiac MRI motion artifact reduction. In this paper, we propose a novel recurrent generative adversarial network model for cardiac MRI motion artifact reduction. This model utilizes bi-directional convolutional long short-term memory (ConvLSTM) and multi-scale convolutions to improve the performance of the proposed network, in which bi-directional ConvLSTMs handle long-range temporal features while multi-scale convolutions gather both local and global features. We demonstrate a decent generalizability of the proposed method thanks to the novel architecture of our deep network that captures the essential relationship of cardiovascular dynamics. Indeed, our extensive experiments show that our method achieves better image quality for cine cardiac MRI images than existing state-of-the-art methods. In addition, our method can generate reliable missing intermediate frames based on their adjacent frames, improving the temporal resolution of cine cardiac MRI sequences.

Temporally aware volumetric generative adversarial network-based MR image reconstruction with simultaneous respiratory motion compensation: Initial feasibility in 3D dynamic cine cardiac MRI

PURPOSE

Develop a novel three-dimensional (3D) generative adversarial network (GAN)-based technique for simultaneous image reconstruction and respiratory motion compensation of 4D MRI. Our goal was to enable high-acceleration factors 10.7X-15.8X, while maintaining robust and diagnostic image quality superior to state-of-the-art self-gating (SG) compressed sensing wavelet (CS-WV) reconstruction at lower acceleration factors 3.5X-7.9X.

METHODS

Our GAN was trained based on pixel-wise content loss functions, adversarial loss function, and a novel data-driven temporal aware loss function to maintain anatomical accuracy and temporal coherence. Besides image reconstruction, our network also performs respiratory motion compensation for free-breathing scans. A novel progressive growing-based strategy was adapted to make the training process possible for the proposed GAN-based structure. The proposed method was developed and thoroughly evaluated qualitatively and quantitatively based on 3D cardiac cine data from 42 patients.

RESULTS

Our proposed method achieved significantly better scores in general image quality and image artifacts at 10.7X-15.8X acceleration than the SG CS-WV approach at 3.5X-7.9X acceleration (4.53 ± 0.540 vs. 3.13 ± 0.681 for general image quality, 4.12 ± 0.429 vs. 2.97 ± 0.434 for image artifacts, P < .05 for both). No spurious anatomical structures were observed in our images. The proposed method enabled similar cardiac-function quantification as conventional SG CS-WV. The proposed method achieved faster central processing unit-based image reconstruction (6 s/cardiac phase) than the SG CS-WV (312 s/cardiac phase).

CONCLUSION

The proposed method showed promising potential for high-resolution (1 mm³ ) free-breathing 4D MR data acquisition with simultaneous respiratory motion compensation and fast reconstruction time.

Synergistic multi-contrast cardiac magnetic resonance image reconstruction

Cardiac magnetic resonance imaging (CMR) is an important tool for the non-invasive diagnosis of a variety of cardiovascular diseases. Parametric mapping with multi-contrast CMR is able to quantify tissue alterations in myocardial disease and promises to improve patient care. However, magnetic resonance imaging is an inherently slow imaging modality, resulting in long acquisition times for parametric mapping which acquires a series of cardiac images with different contrasts for signal fitting or dictionary matching. Furthermore, extra efforts to deal with respiratory and cardiac motion by triggering and gating further increase the scan time. Several techniques have been developed to speed up CMR acquisitions, which usually acquire less data than that required by the Nyquist-Shannon sampling theorem, followed by regularized reconstruction to mitigate undersampling artefacts. Recent advances in CMR parametric mapping speed up CMR by synergistically exploiting spatial-temporal and contrast redundancies. In this article, we will review the recent developments in multi-contrast CMR image reconstruction for parametric mapping with special focus on low-rank and model-based reconstructions. Deep learning-based multi-contrast reconstruction has recently been proposed in other magnetic resonance applications. These developments will be covered to introduce the general methodology. Current technical limitations and potential future directions are discussed. This article is part of the theme issue 'Synergistic tomographic image reconstruction: part 1'.

A review of medical image data augmentation techniques for deep learning applications

Research in artificial intelligence for radiology and radiotherapy has recently become increasingly reliant on the use of deep learning-based algorithms. While the performance of the models which these algorithms produce can significantly outperform more traditional machine learning methods, they do rely on larger datasets being available for training. To address this issue, data augmentation has become a popular method for increasing the size of a training dataset, particularly in fields where large datasets aren't typically available, which is often the case when working with medical images. Data augmentation aims to generate additional data which is used to train the model and has been shown to improve performance when validated on a separate unseen dataset. This approach has become commonplace so to help understand the types of data augmentation techniques used in state-of-the-art deep learning models, we conducted a systematic review of the literature where data augmentation was utilised on medical images (limited to CT and MRI) to train a deep learning model. Articles were categorised into basic, deformable, deep learning or other data augmentation techniques. As artificial intelligence models trained using augmented data make their way into the clinic, this review aims to give an insight to these techniques and confidence in the validity of the models produced.

Ground-truth-free deep learning for artefacts reduction in 2D radial cardiac cine MRI using a synthetically generated dataset

In this work, we consider the task of image reconstruction in 2D radial cardiac cine MRI using deep learning (DL)-based regularization. As the regularization is achieved by employing an image-prior predicted by a pre-trained convolutional neural network (CNN), the quality of the image-prior is of essential importance. The achievable performance of any DL-based method is limited by the amount and the quality of the available training data. For fast dynamic processes, obtaining good-quality MR data is challenging because of technical and physiological reasons. In this work, we try to overcome these problems by a transfer-learning approach which is motivated by a previously presented DL-method (XT,YT U-Net). There, instead of training the network on the whole 2D dynamic images, it is trained on 2D spatio-temporal profiles (xt,yt-slices) which show the temporal changes of the imaged object. Therefore, for the training and test data, it is more important that their spatio-temporal profiles share similar local features rather than being images of the same anatomy. This allows us to equip arbitrary data with simulated motion that resembles the cardiac motion and use it as training data. By doing so, it is possible to train a CNN which is applicable to cardiac cine MR data without using ground-truth cine MR images for training. We demonstrate that combining XT,YT U-Net with the proposed transfer-learning strategy delivers comparable performance to CNNs trained on cardiac cine MR images and in some cases even qualitatively surpasses these. Additionally, the transfer-learning strategy was investigated for a 2D and 3D U-Net. The images processed by the the CNNs were used as image-priors in the CNN-regularized iterative reconstruction. The XT,YT U-Net yielded visibly better results than the 2D U-Net and slightly better results than the 3D U-Net when used in combination with the presented transfer learning-strategy.

An end-to-end-trainable iterative network architecture for accelerated radial multi-coil 2D cine MR image reconstruction

PURPOSE

Iterative convolutional neural networks (CNNs) which resemble unrolled learned iterative schemes have shown to consistently deliver state-of-the-art results for image reconstruction problems across different imaging modalities. However, because these methods include the forward model in the architecture, their applicability is often restricted to either relatively small reconstruction problems or to problems with operators which are computationally cheap to compute. As a consequence, they have not been applied to dynamic non-Cartesian multi-coil reconstruction problems so far.

METHODS

In this work, we propose a CNN architecture for image reconstruction of accelerated 2D radial cine MRI with multiple receiver coils. The network is based on a computationally light CNN component and a subsequent conjugate gradient (CG) method which can be jointly trained end-to-end using an efficient training strategy. We investigate the proposed training strategy and compare our method with other well-known reconstruction techniques with learned and non-learned regularization methods.

RESULTS

Our proposed method outperforms all other methods based on non-learned regularization. Further, it performs similar or better than a CNN-based method employing a 3D U-Net and a method using adaptive dictionary learning. In addition, we empirically demonstrate that even by training the network with only iteration, it is possible to increase the length of the network at test time and further improve the results.

CONCLUSIONS

End-to-end training allows to highly reduce the number of trainable parameters of and stabilize the reconstruction network. Further, because it is possible to change the length of the network at the test time, the need to find a compromise between the complexity of the CNN-block and the number of iterations in each CG-block becomes irrelevant.

Deep Learning-Based Parameter Mapping with Uncertainty Estimation for Fat Quantification using Accelerated Free-Breathing Radial MRI

Deep learning has been applied to remove artifacts from undersampled MRI and to replace time-consuming signal fitting in quantitative MRI, but these have usually been treated as separate tasks, which does not fully exploit the shared information. This work proposes a new two-stage framework that completes these two tasks in a concerted approach and also estimates the pixel-wise uncertainty levels. Results from accelerated free-breathing radial MRI for liver fat quantification demonstrate that the proposed framework can achieve high image quality from undersampled radial data, high accuracy for liver fat quantification, and detect uncertainty caused by noisy input data. The proposed framework achieved 3-fold acceleration to <1 min scan time and reduced the computational time for signal fitting to <100 ms/slice in free-breathing liver fat quantification.

Deep convolution neural networks based artifact suppression in under-sampled radial acquisitions of myocardial T 1 mapping images

We developed a deep convolutional neural network (CNN) based method to remove streaking artefact from accelerated radial acquisitions of myocardial T ₁-mapping images. A deep CNN based on a modified U-Net architecture was developed and trained to remove the streaking artefacts from under-sampled T ₁ mapping images. A total of 2090 T ₁-weighted images for 33 patients (55 ± 15 years, 19 males) and five healthy subjects (30 ± 14 years, 2 males) were used for training and testing the network. The images were acquired using radial slice interleaved T ₁ mapping sequence (STONE) and retrospectively under-sampled to achieve acceleration rate of 4 (corresponding to 48 spokes). The dataset was split into training and testing subsets with 23 subjects (60%) and 15 subjects (40%), respectively. For generating voxel-wise T ₁ maps, a two-parameter fitting model was used. Network performance was evaluated using normalized mean square error (NMSE), structural similarity index (SSIM) and peak signal-to-noise ratio (PSNR) metrics. The proposed network allowed fast (<0.3 s/image) removal of the artefact from all T ₁-weighted testing images and the corresponding T ₁ maps with PSNR = 64.3 ± 1.02, NMSE = 0.2 ± 0.09 and SSIM = 0.9 ± 0.3 × 10^-4. There was no statistically significant difference between the measured T ₁ maps for both per-subject (reference: 1085 ± 37 ms, CNN: 1088 ± 37 ms, p = 0.4) and per-segment (reference: 1084 ± 48 ms, CNN: 1083 ± 58 ms, p = 0.9) analyses. In summary, deep CNN allows fast and reliable removal of streaking artefact from under-sampled radial T ₁ mapping images. Our results show that the highly non-linear operations of deep CNN processing of T ₁ mapping images do not impact accurate reconstruction of myocardial T ₁ maps.

Adaptive sparsity level and dictionary size estimation for image reconstruction in accelerated 2D radial cine MRI

PURPOSE

In the past, dictionary learning (DL) and sparse coding (SC) have been proposed for the regularization of image reconstruction problems. The regularization is given by a sparse approximation of all image patches using a learned dictionary, that is, an overcomplete set of basis functions learned from data. Despite its competitiveness, DL and SC require the tuning of two essential hyperparameters: the sparsity level S - the number of basis functions of the dictionary, called atoms, which are used to approximate each patch, and K - the overall number of such atoms in the dictionary. These two hyperparameters usually have to be chosen a priori and are determined by repetitive and computationally expensive experiments. Furthermore, the final reported values vary depending on the specific situation. As a result, the clinical application of the method is limited, as standardized reconstruction protocols have to be used.

METHODS

In this work, we use adaptive DL and propose a novel adaptive sparse coding algorithm for two-dimensional (2D) radial cine MR image reconstruction. Using adaptive DL and adaptive SC, the optimal dictionary size K as well as the optimal sparsity level S are chosen dependent on the considered data.

RESULTS

Our three main results are the following: First, adaptive DL and adaptive SC deliver results which are comparable or better than the most widely used nonadaptive version of DL and SC. Second, the time needed for the regularization is accelerated due to the fact that the sparsity level S is never overestimated. Finally, the a priori choice of S and K is no longer needed but is optimally chosen dependent on the data under consideration.

CONCLUSIONS

Adaptive DL and adaptive SC can highly facilitate the application of DL- and SC-based regularization methods. While in this work we focused on 2D radial cine MR image reconstruction, we expect the method to be applicable to different imaging modalities as well.

Multi-domain convolutional neural network (MD-CNN) for radial reconstruction of dynamic cardiac MRI

PURPOSE

Cardiac MR cine imaging allows accurate and reproducible assessment of cardiac function. However, its long scan time not only limits the spatial and temporal resolutions but is challenging in patients with breath-holding difficulty or non-sinus rhythms. To reduce scan time, we propose a multi-domain convolutional neural network (MD-CNN) for fast reconstruction of highly undersampled radial cine images.

METHODS

MD-CNN is a complex-valued network that processes MR data in k-space and image domains via k-space interpolation and image-domain subnetworks for residual artifact suppression. MD-CNN exploits spatio-temporal correlations across timeframes and multi-coil redundancies to enable high acceleration. Radial cine data were prospectively collected in 108 subjects (50 ± 17 y, 72 males) using retrospective-gated acquisition with 80%:20% split for training/testing. Images were reconstructed by MD-CNN and k-t Radial Sparse-Sense(kt-RASPS) using an undersampled dataset (14 of 196 acquired views; relative acceleration rate = 14). MD-CNN images were evaluated quantitatively using mean-squared-error (MSE) and structural similarity index (SSIM) relative to reference images, and qualitatively by three independent readers for left ventricular (LV) border sharpness and temporal fidelity using 5-point Likert-scale (1-non-diagnostic, 2-poor, 3-fair, 4-good, and 5-excellent).

RESULTS

MD-CNN showed improved MSE and SSIM compared to kt-RASPS (0.11 ± 0.10 vs. 0.61 ± 0.51, and 0.87 ± 0.07 vs. 0.72 ± 0.07, respectively; P < .01). Qualitatively, MD-CCN significantly outperformed kt-RASPS in LV border sharpness (3.87 ± 0.66 vs. 2.71 ± 0.58 at end-diastole, and 3.57 ± 0.6 vs. 2.56 ± 0.6 at end-systole, respectively; P < .01) and temporal fidelity (3.27 ± 0.65 vs. 2.59 ± 0.59; P < .01).

CONCLUSION

MD-CNN reduces the scan time of cine imaging by a factor of 23.3 and provides superior image quality compared to kt-RASPS.

Neural networks-based regularization for large-scale medical image reconstruction

In this paper we present a generalized Deep Learning-based approach for solving ill-posed large-scale inverse problems occuring in medical image reconstruction. Recently, Deep Learning methods using iterative neural networks (NNs) and cascaded NNs have been reported to achieve state-of-the-art results with respect to various quantitative quality measures as PSNR, NRMSE and SSIM across different imaging modalities. However, the fact that these approaches employ the application of the forward and adjoint operators repeatedly in the network architecture requires the network to process the whole images or volumes at once, which for some applications is computationally infeasible. In this work, we follow a different reconstruction strategy by strictly separating the application of the NN, the regularization of the solution and the consistency with the measured data. The regularization is given in the form of an image prior obtained by the output of a previously trained NN which is used in a Tikhonov regularization framework. By doing so, more complex and sophisticated network architectures can be used for the removal of the artefacts or noise than it is usually the case in iterative NNs. Due to the large scale of the considered problems and the resulting computational complexity of the employed networks, the priors are obtained by processing the images or volumes as patches or slices. We evaluated the method for the cases of 3D cone-beam low dose CT and undersampled 2D radial cine MRI and compared it to a total variation-minimization-based reconstruction algorithm as well as to a method with regularization based on learned overcomplete dictionaries. The proposed method outperformed all the reported methods with respect to all chosen quantitative measures and further accelerates the regularization step in the reconstruction by several orders of magnitude.

Rising role of artificial intelligence in image reconstruction for biomedical imaging

In this editorial, we review recent progress on the applications of artificial intelligence (AI) in image reconstruction for biomedical imaging. Because it abandons prior information of traditional artificial design and adopts a completely data-driven mode to obtain deeper prior information via learning, AI technology plays an increasingly important role in biomedical image reconstruction. The combination of AI technology and the biomedical image reconstruction method has become a hotspot in the field. Favoring AI, the performance of biomedical image reconstruction has been improved in terms of accuracy, resolution, imaging speed, etc. We specifically focus on how to use AI technology to improve the performance of biomedical image reconstruction, and propose possible future directions in this field.