LRTV: MR Image Super-Resolution With Low-Rank and Total Variation Regularizations

Accelerated Musculoskeletal Magnetic Resonance Imaging

With a substantial growth in the use of musculoskeletal MRI, there has been a growing need to improve MRI workflow, and faster imaging has been suggested as one of the solutions for a more efficient examination process. Consequently, there have been considerable advances in accelerated MRI scanning methods. This article aims to review the basic principles and applications of accelerated musculoskeletal MRI techniques including widely used conventional acceleration methods, more advanced deep learning-based techniques, and new approaches to reduce scan time. Specifically, conventional accelerated MRI techniques, including parallel imaging, compressed sensing, and simultaneous multislice imaging, and deep learning-based accelerated MRI techniques, including undersampled MR image reconstruction, super-resolution imaging, artifact correction, and generation of unacquired contrast images, are discussed. Finally, new approaches to reduce scan time, including synthetic MRI, novel sequences, and new coil setups and designs, are also reviewed. We believe that a deep understanding of these fast MRI techniques and proper use of combined acceleration methods will synergistically improve scan time and MRI workflow in daily practice. EVIDENCE LEVEL: 3 TECHNICAL EFFICACY: Stage 1.

Learned Tensor Neural Network Texture Prior for Photon-Counting CT Reconstruction

Photon-counting computed tomography (PCCT) reconstructs multiple energy-channel images to describe the same object, where there exists a strong correlation among different channel images. In addition, reconstruction of each channel image suffers photon count starving problem. To make full use of the correlation among different channel images to suppress the data noise and enhance the texture details in reconstructing each channel image, this paper proposes a tensor neural network (TNN) architecture to learn a multi-channel texture prior for PCCT reconstruction. Specifically, we first learn a spatial texture prior in each individual channel image by modeling the relationship between the center pixels and its corresponding neighbor pixels using a neural network. Then, we merge the single channel spatial texture prior into multi-channel neural network to learn the spectral local correlation information among different channel images. Since our proposed TNN is trained on a series of unpaired small spatial-spectral cubes which are extracted from one single reference multi-channel image, the local correlation in the spatial-spectral cubes is considered by TNN. To boost the TNN performance, a low-rank representation is also employed to consider the global correlation among different channel images. Finally, we integrate the learned TNN and the low-rank representation as priors into Bayesian reconstruction framework. To evaluate the performance of the proposed method, four references are considered. One is simulated images from ultra-high-resolution CT. One is spectral images from dual-energy CT. The other two are animal tissue and preclinical mouse images from a custom-made PCCT systems. Our TNN prior Bayesian reconstruction demonstrated better performance than other state-of-the-art competing algorithms, in terms of not only preserving texture feature but also suppressing image noise in each channel image.

MDRN: Multi-distillation residual network for efficient MR image super-resolution

Super-resolution (SR) of magnetic resonance imaging (MRI) is gaining increasing attention for being able to provide detailed anatomical information. However, current SR methods often use the complex convolutional network for feature extraction, which is difficult to train and not suitable for limited computation resources in the medical scenario. To tackle these bottlenecks, we propose a multi-distillation residual network (MDRN) for more differential feature refinement, which has a superior trade-off between reconstruction accuracy and computation cost. Specifically, a novel feature multi-distillation residual block with a contrast-aware channel attention module was designed to make the residual features more focused on low-vision information, which maximizes the power of MDRN. Comprehensive experiments demonstrate the superiority of our MDRN over state-of-the-art methods in reconstruction quality and efficiency. Our method outperforms other existing methods in peak signal-noise ratio by up to 0.44-1.82 dB in 4× scale when GPU memory and runtime are lower than in other SR methods. The source code will be available at https://github.com/Jennieyy/MDRN.

Isotropic Brain MRI Reconstruction from Orthogonal Scans Using 3D Convolutional Neural Network

As an alternative to true isotropic 3D imaging, image super-resolution (SR) has been applied to reconstruct an isotropic 3D volume from multiple anisotropic scans. However, traditional SR methods struggle with inadequate performance, prolonged processing times, and the necessity for manual feature extraction. Motivated by the exceptional representational ability and automatic feature extraction of convolutional neural networks (CNNs), in this work, we present an end-to-end isotropic MRI reconstruction strategy based on deep learning. The proposed method is based on 3D convolutional neural networks (3D CNNs), which can effectively capture the 3D structural features of MRI volumes and accurately predict potential structure. In addition, the proposed method takes multiple orthogonal scans as input and thus enables the model to use more complementary information from different dimensions for precise inference. Experimental results show that the proposed algorithm achieves promising performance in terms of both quantitative and qualitative assessments. In addition, it can process a 3D volume with a size of 256 × 256 × 256 in less than 1 min with the support of an NVIDIA GeForce GTX 1080Ti GPU, which suggests that it is not only a quantitatively superior method but also a practical one.

Beyond Nyquist: A Comparative Analysis of 3D Deep Learning Models Enhancing MRI Resolution

High-spatial resolution MRI produces abundant structural information, enabling highly accurate clinical diagnosis and image-guided therapeutics. However, the acquisition of high-spatial resolution MRI data typically can come at the expense of less spatial coverage, lower signal-to-noise ratio (SNR), and longer scan time due to physical, physiological and hardware limitations. In order to overcome these limitations, super-resolution MRI deep-learning-based techniques can be utilised. In this work, different state-of-the-art 3D convolution neural network models for super resolution (RRDB, SPSR, UNet, UNet-MSS and ShuffleUNet) were compared for the super-resolution task with the goal of finding the best model in terms of performance and robustness. The public IXI dataset (only structural images) was used. Data were artificially downsampled to obtain lower-resolution spatial MRIs (downsampling factor varying from 8 to 64). When assessing performance using the SSIM metric in the test set, all models performed well. In particular, regardless of the downsampling factor, the UNet consistently obtained the top results. On the other hand, the SPSR model consistently performed worse. In conclusion, UNet and UNet-MSS achieved overall top performances while RRDB performed relatively poorly compared to the other models.

High-Dimensional MR Reconstruction Integrating Subspace and Adaptive Generative Models

OBJECTIVE

To develop a new method that integrates subspace and generative image models for high-dimensional MR image reconstruction.

METHODS

We proposed a formulation that synergizes a low-dimensional subspace model of high-dimensional images, an adaptive generative image prior serving as spatial constraints on the sequence of "contrast-weighted" images or spatial coefficients of the subspace model, and a conventional sparsity regularization. A special pretraining plus subject-specific network adaptation strategy was proposed to construct an accurate generative-network-based representation for images with varying contrasts. An iterative algorithm was introduced to jointly update the subspace coefficients and the multi-resolution latent space of the generative image model that leveraged an recently proposed intermediate layer optimization technique for network inversion.

RESULTS

We evaluated the utility of the proposed method for two high-dimensional imaging applications: accelerated MR parameter mapping and high-resolution MR spectroscopic imaging. Improved performance over state-of-the-art subspace-based methods was demonstrated in both cases.

CONCLUSION

The proposed method provided a new way to address high-dimensional MR image reconstruction problems by incorporating an adaptive generative model as a data-driven spatial prior for constraining subspace reconstruction.

SIGNIFICANCE

Our work demonstrated the potential of integrating data-driven and adaptive generative priors with canonical low-dimensional modeling for high-dimensional imaging problems.

3D-MRI super-resolution reconstruction using multi-modality based on multi-resolution CNN

BACKGROUND AND OBJECTIVE

High-resolution (HR) MR images provide rich structural detail to assist physicians in clinical diagnosis and treatment plan. However, it is arduous to acquire HR MRI due to equipment limitations, scanning time or patient comfort. Instead, HR MRI could be obtained through a number of computer assisted post-processing methods that have proven to be effective and reliable. This paper aims to develop a convolutional neural network (CNN) based super-resolution reconstruction framework for low-resolution (LR) T2w images.

METHOD

In this paper, we propose a novel multi-modal HR MRI generation framework based on deep learning techniques. Specifically, we construct a CNN based on multi-resolution analysis to learn an end-to-end mapping between LR T2w and HR T2w, where HR T1w is fed into the network to offer detailed a priori information to help generate HR T2w. Furthermore, a low-frequency filtering module is introduced to filter out the interference from HR-T1w during high-frequency information extraction. Based on the idea of multi-resolution analysis, detailed features extracted from HR T1w and LR T2w are fused at two scales in the network and then HR T2w is reconstructed by upsampling and dense connectivity module.

RESULTS

Extensive quantitative and qualitative evaluations demonstrate that the proposed method enhances the recovered HR T2w details and outperforms other state-of-the-art methods. In addition, the experimental results also suggest that our network has a lightweight structure and favorable generalization performance.

CONCLUSION

The results show that the proposed method is capable of reconstructing HR T2w with higher accuracy. Meanwhile, the super-resolution reconstruction results on other dataset illustrate the excellent generalization ability of the method.

Multi-level feature extraction and reconstruction for 3D MRI image super-resolution

Magnetic resonance imaging (MRI) is an essential radiology technique in clinical diagnosis, but its spatial resolution may not suffice to meet the growing need for precise diagnosis due to hardware limitations and thicker slice thickness. Therefore, it is crucial to explore suitable methods to increase the resolution of MRI images. Recently, deep learning has yielded many impressive results in MRI image super-resolution (SR) reconstruction. However, current SR networks mainly use convolutions to extract relatively single image features, which may not be optimal for further enhancing the quality of image reconstruction. In this work, we propose a multi-level feature extraction and reconstruction (MFER) method to restore the degraded high-resolution details of MRI images. Specifically, to comprehensively extract different types of features, we design the triple-mixed convolution by leveraging the strengths and uniqueness of different filter operations. For the features of each level, we then apply deconvolutions to upsample them separately at the tail of the network, followed by the feature calibration of spatial and channel attention. Besides, we also use a soft cross-scale residual operation to improve the effectiveness of parameter optimization. Experiments on lesion-free and glioma datasets indicate that our method obtains superior quantitative performance and visual effects when compared with state-of-the-art MRI image SR methods.

Tensor Ring Based Image Enhancement

Background

Image enhancement, including image de-noising, super-resolution, registration, reconstruction, in-painting, and so on, is an important issue in different research areas. Different methods which have been exploited for image analysis were mostly based on matrix or low order analysis. However, recent researches show the superior power of tensor-based methods for image enhancement.

Method

In this article, a new method for image super-resolution using Tensor Ring decomposition has been proposed. The proposed image super-resolution technique has been derived for the super-resolution of low resolution and noisy images. The new approach is based on a modification and extension of previous tensor-based approaches used for super-resolution of datasets. In this method, a weighted combination of the original and the resulting image of the previous stage has been computed and used to provide a new input to the algorithm.

Result

This enables the method to do the super-resolution and de-noising simultaneously.

Conclusion

Simulation results show the effectiveness of the proposed approach, especially in highly noisy situations.

High-resolution MRI synthesis using a data-driven framework with denoising diffusion probabilistic modeling

Objective. High-resolution magnetic resonance imaging (MRI) can enhance lesion diagnosis, prognosis, and delineation. However, gradient power and hardware limitations prohibit recording thin slices or sub-1 mm resolution. Furthermore, long scan time is not clinically acceptable. Conventional high-resolution images generated using statistical or analytical methods include the limitation of capturing complex, high-dimensional image data with intricate patterns and structures. This study aims to harness cutting-edge diffusion probabilistic deep learning techniques to create a framework for generating high-resolution MRI from low-resolution counterparts, improving the uncertainty of denoising diffusion probabilistic models (DDPM).Approach. DDPM includes two processes. The forward process employs a Markov chain to systematically introduce Gaussian noise to low-resolution MRI images. In the reverse process, a U-Net model is trained to denoise the forward process images and produce high-resolution images conditioned on the features of their low-resolution counterparts. The proposed framework was demonstrated using T2-weighted MRI images from institutional prostate patients and brain patients collected in the Brain Tumor Segmentation Challenge 2020 (BraTS2020).Main results. For the prostate dataset, the bicubic interpolation model (Bicubic), conditional generative-adversarial network (CGAN), and our proposed DDPM framework improved the noise quality measure from low-resolution images by 4.4%, 5.7%, and 12.8%, respectively. Our method enhanced the signal-to-noise ratios by 11.7%, surpassing Bicubic (9.8%) and CGAN (8.1%). In the BraTS2020 dataset, the proposed framework and Bicubic enhanced peak signal-to-noise ratio from resolution-degraded images by 9.1% and 5.8%. The multi-scale structural similarity indexes were 0.970 ± 0.019, 0.968 ± 0.022, and 0.967 ± 0.023 for the proposed method, CGAN, and Bicubic, respectively.Significance. This study explores a deep learning-based diffusion probabilistic framework for improving MR image resolution. Such a framework can be used to improve clinical workflow by obtaining high-resolution images without penalty of the long scan time. Future investigation will likely focus on prospectively testing the efficacy of this framework with different clinical indications.

Cross modality generative learning framework for anatomical transitive Magnetic Resonance Imaging (MRI) from Electrical Impedance Tomography (EIT) image

This paper presents a cross-modality generative learning framework for transitive magnetic resonance imaging (MRI) from electrical impedance tomography (EIT). The proposed framework is aimed at converting low-resolution EIT images to high-resolution wrist MRI images using a cascaded cycle generative adversarial network (CycleGAN) model. This model comprises three main components: the collection of initial EIT from the medical device, the generation of a high-resolution transitive EIT image from the corresponding MRI image for domain adaptation, and the coalescence of two CycleGAN models for cross-modality generation. The initial EIT image was generated at three different frequencies (70 kHz, 140 kHz, and 200 kHz) using a 16-electrode belt. Wrist T1-weighted images were acquired on a 1.5T MRI. A total of 19 normal volunteers were imaged using both EIT and MRI, which resulted in 713 paired EIT and MRI images. The cascaded CycleGAN, end-to-end CycleGAN, and Pix2Pix models were trained and tested on the same cohort. The proposed method achieved the highest accuracy in bone detection, with 0.97 for the proposed cascaded CycleGAN, 0.68 for end-to-end CycleGAN, and 0.70 for the Pix2Pix model. Visual inspection showed that the proposed method reduced bone-related errors in the MRI-style anatomical reference compared with end-to-end CycleGAN and Pix2Pix. Multifrequency EIT inputs reduced the testing normalized root mean squared error of MRI-style anatomical reference from 67.9% ± 12.7% to 61.4% ± 8.8% compared with that of single-frequency EIT. The mean conductivity values of fat and bone from regularized EIT were 0.0435 ± 0.0379 S/m and 0.0183 ± 0.0154 S/m, respectively, when the anatomical prior was employed. These results demonstrate that the proposed framework is able to generate MRI-style anatomical references from EIT images with a good degree of accuracy.

Exact Decomposition of Joint Low Rankness and Local Smoothness Plus Sparse Matrices

It is known that the decomposition in low-rank and sparse matrices (L+S for short) can be achieved by several Robust PCA techniques. Besides the low rankness, the local smoothness (LSS) is a vitally essential prior for many real-world matrix data such as hyperspectral images and surveillance videos, which makes such matrices have low-rankness and local smoothness property at the same time. This poses an interesting question: Can we make a matrix decomposition in terms of L&LSS +S form exactly? To address this issue, we propose in this paper a new RPCA model based on three-dimensional correlated total variation regularization (3DCTV-RPCA for short) by fully exploiting and encoding the prior expression underlying such joint low-rank and local smoothness matrices. Specifically, using a modification of Golfing scheme, we prove that under some mild assumptions, the proposed 3DCTV-RPCA model can decompose both components exactly, which should be the first theoretical guarantee among all such related methods combining low rankness and local smoothness. In addition, by utilizing Fast Fourier Transform (FFT), we propose an efficient ADMM algorithm with a solid convergence guarantee for solving the resulting optimization problem. Finally, a series of experiments on both simulations and real applications are carried out to demonstrate the general validity of the proposed 3DCTV-RPCA model.

Low-field magnetic resonance image enhancement via stochastic image quality transfer

Low-field (<1T) magnetic resonance imaging (MRI) scanners remain in widespread use in low- and middle-income countries (LMICs) and are commonly used for some applications in higher income countries e.g. for small child patients with obesity, claustrophobia, implants, or tattoos. However, low-field MR images commonly have lower resolution and poorer contrast than images from high field (1.5T, 3T, and above). Here, we present Image Quality Transfer (IQT) to enhance low-field structural MRI by estimating from a low-field image the image we would have obtained from the same subject at high field. Our approach uses (i) a stochastic low-field image simulator as the forward model to capture uncertainty and variation in the contrast of low-field images corresponding to a particular high-field image, and (ii) an anisotropic U-Net variant specifically designed for the IQT inverse problem. We evaluate the proposed algorithm both in simulation and using multi-contrast (T1-weighted, T2-weighted, and fluid attenuated inversion recovery (FLAIR)) clinical low-field MRI data from an LMIC hospital. We show the efficacy of IQT in improving contrast and resolution of low-field MR images. We demonstrate that IQT-enhanced images have potential for enhancing visualisation of anatomical structures and pathological lesions of clinical relevance from the perspective of radiologists. IQT is proved to have capability of boosting the diagnostic value of low-field MRI, especially in low-resource settings.

Magnetic resonance image restoration via least absolute deviations measure with isotropic total variation constraint

This paper presents a magnetic resonance image deblurring and denoising model named the isotropic total variation regularized least absolute deviations measure (LADTV). More specifically, the least absolute deviations term is first adopted to measure the violation of the relation between the desired magnetic resonance image and the observed image, and to simultaneously suppress the noise that may corrupt the desired image. Then, in order to preserve the smoothness of the desired image, we introduce an isotropic total variation constraint, yielding the proposed restoration model LADTV. Finally, an alternating optimization algorithm is developed to solve the associated minimization problem. Comparative experiments on clinical data demonstrate the effectiveness of our approach to synchronously deblur and denoise magnetic resonance image.

An Arbitrary Scale Super-Resolution Approach for 3D MR Images via Implicit Neural Representation

High Resolution (HR) medical images provide rich anatomical structure details to facilitate early and accurate diagnosis. In magnetic resonance imaging (MRI), restricted by hardware capacity, scan time, and patient cooperation ability, isotropic 3-dimensional (3D) HR image acquisition typically requests long scan time and, results in small spatial coverage and low signal-to-noise ratio (SNR). Recent studies showed that, with deep convolutional neural networks, isotropic HR MR images could be recovered from low-resolution (LR) input via single image super-resolution (SISR) algorithms. However, most existing SISR methods tend to approach scale-specific projection between LR and HR images, thus these methods can only deal with fixed up-sampling rates. In this paper, we propose ArSSR, an Arbitrary Scale Super-Resolution approach for recovering 3D HR MR images. In the ArSSR model, the LR image and the HR image are represented using the same implicit neural voxel function with different sampling rates. Due to the continuity of the learned implicit function, a single ArSSR model is able to achieve arbitrary and infinite up-sampling rate reconstructions of HR images from any input LR image. Then the SR task is converted to approach the implicit voxel function via deep neural networks from a set of paired HR and LR training examples. The ArSSR model consists of an encoder network and a decoder network. Specifically, the convolutional encoder network is to extract feature maps from the LR input images and the fully-connected decoder network is to approximate the implicit voxel function. Experimental results on three datasets show that the ArSSR model can achieve state-of-the-art SR performance for 3D HR MR image reconstruction while using a single trained model to achieve arbitrary up-sampling scales.

Spectral CT reconstruction via low-rank representation and structure preserving regularization

Objective:With the development of computed tomography (CT) imaging technology, it is possible to acquire multi-energy data by spectral CT. Being different from conventional CT, the X-ray energy spectrum of spectral CT is cut into several narrow bins which leads to the result that only a part of photon can be collected in each individual energy channel.This can severely degrade the image qualities. To address this problem, we propose a spectral CT reconstruction algorithm based on low-rank representation and structure preserving regularization in this paper.Approach:To make full use of the prior knowledge about both the inter-channel correlation and the sparsity in gradient domain of inner-channel data, this paper combines a low-rank correlation descriptor with a structure extraction operator as priori regularization terms for spectral CT reconstruction. Furthermore, a split-Bregman based iterative algorithm is developed to solve the reconstruction model. Finally, we propose a multi-channel adaptive parameters generation strategy according to CT values of each individual energy channel.Main results: Experimental results on numerical simulations and real mouse data indicate that the proposed algorithm achieves higher accuracy on both reconstruction and material decomposition than the methods based on simultaneous algebraic reconstruction technique (SART), total variation minimization (TVM), total variation with low-rank (LRTV), and spatial-spectral cube matching frame (SSCMF). Compared with SART, our algorithm improves the feature similarity (FSIM) by 40.4% on average for numerical simulation reconstruction, whereas TVM, LRTV, and SSCMF correspond to 26.1%, 28.2%, and 29.5%, respectively.Significance: We outline a multi-channel reconstruction algorithm tailored for spectral CT. The qualitative and quantitative comparisons present a significant improvement of image quality, indicating its promising potential in spectral CT imaging.

Multi-modal brain MRI images enhancement based on framelet and local weights super-resolution

Magnetic resonance (MR) image enhancement technology can reconstruct high-resolution image from a low-resolution image, which is of great significance for clinical application and scientific research. T1 weighting and T2 weighting are the two common magnetic resonance imaging modes, each of which has its own advantages, but the imaging time of T2 is much longer than that of T1. Related studies have shown that they have very similar anatomical structures in brain images, which can be utilized to enhance the resolution of low-resolution T2 images by using the edge information of high-resolution T1 images that can be rapidly imaged, so as to shorten the imaging time needed for T2 images. In order to overcome the inflexibility of traditional methods using fixed weights for interpolation and the inaccuracy of using gradient threshold to determine edge regions, we propose a new model based on previous studies on multi-contrast MR image enhancement. Our model uses framelet decomposition to finely separate the edge structure of the T2 brain image, and uses the local regression weights calculated from T1 image to construct a global interpolation matrix, so that our model can not only guide the edge reconstruction more accurately where the weights are shared, but also carry out collaborative global optimization for the remaining pixels and their interpolated weights. Experimental results on a set of simulated MR data and two sets of real MR images show that the enhanced images obtained by the proposed method are superior to the compared methods in terms of visual sharpness or qualitative indicators.

Accurate and lightweight MRI super-resolution via multi-scale bidirectional fusion attention network

High-resolution magnetic resonance (MR) imaging has attracted much attention due to its contribution to clinical diagnoses and treatment. However, because of the interference of noise and the limitation of imaging equipment, it is expensive to generate a satisfactory image. Super-resolution (SR) is a technique that enhances an imaging system's resolution, which is effective and cost-efficient for MR imaging. In recent years, deep learning-based SR methods have made remarkable progress on natural images but not on medical images. Most existing medical images SR algorithms focus on the spatial information of a single image but ignore the temporal correlation between medical images sequence. We proposed two novel architectures for single medical image and sequential medical images, respectively. The multi-scale back-projection network (MSBPN) is constructed of several different scale back-projection units which consist of iterative up- and down-sampling layers. The multi-scale machine extracts different scale spatial information and strengthens the information fusion for a single image. Based on MSBPN, we proposed an accurate and lightweight Multi-Scale Bidirectional Fusion Attention Network(MSBFAN) that combines temporal information iteratively. That supplementary temporal information is extracted from the adjacent image sequence of the target image. The MSBFAN can effectively learn both the spatio-temporal dependencies and the iterative refinement process with only a lightweight number of parameters. Experimental results demonstrate that our MSBPN and MSBFAN are outperforming current SR methods in terms of reconstruction accuracy and parameter quantity of the model.

Development of a Super-Resolution Scheme for Pediatric Magnetic Resonance Brain Imaging Through Convolutional Neural Networks

Pediatric medical imaging represents a real challenge for physicians, as children who are patients often move during the examination, and it causes the appearance of different artifacts in the images. Thus, it is not possible to obtain good quality images for this target population limiting the possibility of evaluation and diagnosis in certain pathological conditions. Specifically, magnetic resonance imaging (MRI) is a technique that requires long acquisition times and, therefore, demands the use of sedation or general anesthesia to avoid the movement of the patient, which is really damaging in this specific population. Because ALARA (as low as reasonably achievable) principles should be considered for all imaging studies, one of the most important reasons for establishing novel MRI imaging protocols is to avoid the harmful effects of anesthesia/sedation. In this context, ground-breaking concepts and novel technologies, such as artificial intelligence, can help to find a solution to these challenges while helping in the search for underlying disease mechanisms. The use of new MRI protocols and new image acquisition and/or pre-processing techniques can aid in the development of neuroimaging studies for children evaluation, and their translation to pediatric populations. In this paper, a novel super-resolution method based on a convolutional neural network (CNN) in two and three dimensions to automatically increase the resolution of pediatric brain MRI acquired in a reduced time scheme is proposed. Low resolution images have been generated from an original high resolution dataset and used as the input of the CNN, while several scaling factors have been assessed separately. Apart from a healthy dataset, we also tested our model with pathological pediatric MRI, and it successfully recovers the original image quality in both visual and quantitative ways, even for available examples of dysplasia lesions. We hope then to establish the basis for developing an innovative free-sedation protocol in pediatric anatomical MRI acquisition.

Artificial intelligence in multi-parametric magnetic resonance imaging: A review

Multi-parametric magnetic resonance imaging (mpMRI) is an indispensable tool in the clinical workflow for the diagnosis and treatment planning of various diseases. Machine learning-based artificial intelligence (AI) methods, especially those adopting the deep learning technique, have been extensively employed to perform mpMRI image classification, segmentation, registration, detection, reconstruction, and super-resolution. The current availability of increasing computational power and fast-improving AI algorithms have empowered numerous computer-based systems for applying mpMRI to disease diagnosis, imaging-guided radiotherapy, patient risk and overall survival time prediction, and the development of advanced quantitative imaging technology for magnetic resonance fingerprinting. However, the wide application of these developed systems in the clinic is still limited by a number of factors, including robustness, reliability, and interpretability. This survey aims to provide an overview for new researchers in the field as well as radiologists with the hope that they can understand the general concepts, main application scenarios, and remaining challenges of AI in mpMRI. This article is protected by copyright. All rights reserved.

Rank-One Network: An Effective Framework for Image Restoration

The principal rank-one (RO) components of an image represent the self-similarity of the image, which is an important property for image restoration. However, the RO components of a corrupted image could be decimated by the procedure of image denoising. We suggest that the RO property should be utilized and the decimation should be avoided in image restoration. To achieve this, we propose a new framework comprised of two modules, i.e., the RO decomposition and RO reconstruction. The RO decomposition is developed to decompose a corrupted image into the RO components and residual. This is achieved by successively applying RO projections to the image or its residuals to extract the RO components. The RO projections, based on neural networks, extract the closest RO component of an image. The RO reconstruction is aimed to reconstruct the important information, respectively from the RO components and residual, as well as to restore the image from this reconstructed information. Experimental results on four tasks, i.e., noise-free image super-resolution (SR), realistic image SR, gray-scale image denoising, and color image denoising, show that the method is effective and efficient for image restoration, and it delivers superior performance for realistic image SR and color image denoising. Our source code is available online.

Nonconvex Nonlocal Tucker Decomposition for 3D Medical Image Super-Resolution

Limited by hardware conditions, imaging devices, transmission efficiency, and other factors, high-resolution (HR) images cannot be obtained directly in clinical settings. It is expected to obtain HR images from low-resolution (LR) images for more detailed information. In this article, we propose a novel super-resolution model for single 3D medical images. In our model, nonlocal low-rank tensor Tucker decomposition is applied to exploit the nonlocal self-similarity prior knowledge of data. Different from the existing methods that use a convex optimization for tensor Tucker decomposition, we use a tensor folded-concave penalty to approximate a nonlocal low-rank tensor. Weighted 3D total variation (TV) is used to maintain the local smoothness across different dimensions. Extensive experiments show that our method outperforms some state-of-the-art (SOTA) methods on different kinds of medical images, including MRI data of the brain and prostate and CT data of the abdominal and dental.

Deep learning-based single image super-resolution for low-field MR brain images

Low-field MRI scanners are significantly less expensive than their high-field counterparts, which gives them the potential to make MRI technology more accessible all around the world. In general, images acquired using low-field MRI scanners tend to be of a relatively low resolution, as signal-to-noise ratios are lower. The aim of this work is to improve the resolution of these images. To this end, we present a deep learning-based approach to transform low-resolution low-field MR images into high-resolution ones. A convolutional neural network was trained to carry out single image super-resolution reconstruction using pairs of noisy low-resolution images and their noise-free high-resolution counterparts, which were obtained from the publicly available NYU fastMRI database. This network was subsequently applied to noisy images acquired using a low-field MRI scanner. The trained convolutional network yielded sharp super-resolution images in which most of the high-frequency components were recovered. In conclusion, we showed that a deep learning-based approach has great potential when it comes to increasing the resolution of low-field MR images.

Exploiting Non-Local Priors via Self-Convolution for Highly-Efficient Image Restoration

Constructing effective priors is critical to solving ill-posed inverse problems in image processing and computational imaging. Recent works focused on exploiting non-local similarity by grouping similar patches for image modeling, and demonstrated state-of-the-art results in many image restoration applications. However, compared to classic methods based on filtering or sparsity, non-local algorithms are more time-consuming, mainly due to the highly inefficient block matching step, i.e., distance between every pair of overlapping patches needs to be computed. In this work, we propose a novel Self-Convolution operator to exploit image non-local properties in a unified framework. We prove that the proposed Self-Convolution based formulation can generalize the commonly-used non-local modeling methods, as well as produce results equivalent to standard methods, but with much cheaper computation. Furthermore, by applying Self-Convolution, we propose an effective multi-modality image restoration scheme, which is much more efficient than conventional block matching for non-local modeling. Experimental results demonstrate that (1) Self-Convolution with fast Fourier transform implementation can significantly speed up most of the popular non-local image restoration algorithms, with two-fold to nine-fold faster block matching, and (2) the proposed online multi-modality image restoration scheme achieves superior denoising results than competing methods in both efficiency and effectiveness on RGB-NIR images. The code for this work is publicly available at https://github.com/GuoLanqing/Self-Convolution.

Deep robust residual network for super-resolution of 2D fetal brain MRI

Spatial resolution is a key factor of quantitatively evaluating the quality of magnetic resonance imagery (MRI). Super-resolution (SR) approaches can improve its spatial resolution by reconstructing high-resolution (HR) images from low-resolution (LR) ones to meet clinical and scientific requirements. To increase the quality of brain MRI, we study a robust residual-learning SR network (RRLSRN) to generate a sharp HR brain image from an LR input. Due to the Charbonnier loss can handle outliers well, and Gradient Difference Loss (GDL) can sharpen an image, we combined the Charbonnier loss and GDL to improve the robustness of the model and enhance the texture information of SR results. Two MRI datasets of adult brain, Kirby 21 and NAMIC, were used to train and verify the effectiveness of our model. To further verify the generalizability and robustness of the proposed model, we collected eight clinical fetal brain MRI 2D data for evaluation. The experimental results have shown that the proposed deep residual-learning network achieved superior performance and high efficiency over other compared methods.

Realizing the Effective Detection of Tumor in Magnetic Resonance Imaging using Cluster-Sparse Assisted Super-Resolution

Recently, significant research has been done in Super-Resolution (SR) methods for augmenting the spatial resolution of the Magnetic Resonance (MR) images, which aids the physician in improved disease diagnoses. Single SR methods have drawbacks; they fail to capture self-similarity in non-local patches and are not robust to noise. To exploit the non-local self-similarity and intrinsic sparsity in MR images, this paper proposes the use of Cluster-Sparse Assisted Super-Resolution. This SR method effectively captures similarity in non-locally positioned patches by training on clusters of patches using a self-adaptive dictionary. This method of training also leads to better edge and texture detection. Experiments show that using Cluster-Sparse Assisted Super-Resolution for brain MR images results in enhanced detection of lesions leading to better diagnosis.

Assessing the utility of low resolution brain imaging: treatment of infant hydrocephalus

As low-field MRI technology is being disseminated into clinical settings around the world, it is important to assess the image quality required to properly diagnose and treat a given disease and evaluate the role of machine learning algorithms, such as deep learning, in the enhancement of lower quality images. In this post hoc analysis of an ongoing randomized clinical trial, we assessed the diagnostic utility of reduced-quality and deep learning enhanced images for hydrocephalus treatment planning. CT images of post-infectious infant hydrocephalus were degraded in terms of spatial resolution, noise, and contrast between brain and CSF and enhanced using deep learning algorithms. Both degraded and enhanced images were presented to three experienced pediatric neurosurgeons accustomed to working in low- to middle-income countries (LMIC) for assessment of clinical utility in treatment planning for hydrocephalus. In addition, enhanced images were presented alongside their ground-truth CT counterparts in order to assess whether reconstruction errors caused by the deep learning enhancement routine were acceptable to the evaluators. Results indicate that image resolution and contrast-to-noise ratio between brain and CSF predict the likelihood of an image being characterized as useful for hydrocephalus treatment planning. Deep learning enhancement substantially increases contrast-to-noise ratio improving the apparent likelihood of the image being useful; however, deep learning enhancement introduces structural errors which create a substantial risk of misleading clinical interpretation. We find that images with lower quality than is customarily acceptable can be useful for hydrocephalus treatment planning. Moreover, low quality images may be preferable to images enhanced with deep learning, since they do not introduce the risk of misleading information which could misguide treatment decisions. These findings advocate for new standards in assessing acceptable image quality for clinical use.

Gradient-Guided Isotropic MRI Reconstruction from Anisotropic Acquisitions

The trade-off between image resolution, signal-to-noise ratio (SNR), and scan time in any magnetic resonance imaging (MRI) protocol is inevitable and unavoidable. Super-resolution reconstruction (SRR) has been shown effective in mitigating these factors, and thus, has become an important approach in addressing the current limitations of MRI. In this work, we developed a novel, image-based MRI SRR approach based on anisotropic acquisition schemes, which utilizes a new gradient guidance regularization method that guides the high-resolution (HR) reconstruction via a spatial gradient estimate. Further, we designed an analytical solution to propagate the spatial gradient fields from the low-resolution (LR) images to the HR image space and exploited these gradient fields over multiple scales with a dynamic update scheme for more accurate edge localization in the reconstruction. We also established a forward model of image formation and inverted it along with the proposed gradient guidance. The proposed SRR method allows subject motion between volumes and is able to incorporate various acquisition schemes where the LR images are acquired with arbitrary orientations and displacements, such as orthogonal and through-plane origin-shifted scans. We assessed our proposed approach on simulated data as well as on the data acquired on a Siemens 3T MRI scanner containing 45 MRI scans from 14 subjects. Our experimental results demonstrate that our approach achieved superior reconstructions compared to state-of-the-art methods, both in terms of local spatial smoothness and edge preservation, while, in parallel, at reduced, or at the same cost as scans delivered with direct HR acquisition.

Recycling diagnostic MRI for empowering brain morphometric research - Critical & practical assessment on learning-based image super-resolution

Preliminary studies have shown the feasibility of deep learning (DL)-based super-resolution (SR) technique for reconstructing thick-slice/gap diagnostic MR images into high-resolution isotropic data, which would be of great significance for brain research field if the vast amount of diagnostic MRI data could be successively put into brain morphometric study. However, less evidence has addressed the practicability of the strategy, because lack of a large-sample available real data for constructing DL model. In this work, we employed a large cohort (n = 2052) of peculiar data with both low through-plane resolution diagnostic and high-resolution isotropic brain MR images from identical subjects. By leveraging a series of SR approaches, including a proposed novel DL algorithm of Structure Constrained Super Resolution Network (SCSRN), the diagnostic images were transformed to high-resolution isotropic data to meet the criteria of brain research in voxel-based and surface-based morphometric analyses. We comprehensively assessed image quality and the practicability of the reconstructed data in a variety of morphometric analysis scenarios. We further compared the performance of SR approaches to the ground truth high-resolution isotropic data. The results showed (i) DL-based SR algorithms generally improve the quality of diagnostic images and render morphometric analysis more accurate, especially, with the most superior performance of the novel approach of SCSRN. (ii) Accuracies vary across brain structures and methods, and (iii) performance increases were higher for voxel than for surface based approaches. This study supports that DL-based image super-resolution potentially recycle huge amount of routine diagnostic brain MRI deposited in sleeping state, and turning them into useful data for neurometric research.

MRI super-resolution via realistic downsampling with adversarial learning

Many deep learning (DL) frameworks have demonstrated state-of-the-art performance in the super-resolution (SR) task of magnetic resonance imaging, but most performances have been achieved with simulated low-resolution (LR) images rather than LR images from real acquisition. Due to the limited generalizability of the SR network, enhancement is not guaranteed for real LR images because of the unreality of the training LR images. In this study, we proposed a DL-based SR framework with an emphasis on data construction to achieve better performance on real LR MR images. The framework comprised two steps: (a) downsampling training using a generative adversarial network (GAN) to construct more realistic and perfectly matched LR/high-resolution (HR) pairs. The downsampling GAN input was real LR and HR images. The generator translated the HR images to LR images and the discriminator distinguished the patch-level difference between the synthetic and real LR images. (b) SR training was performed using an enhance4d deep super-resolution network (EDSR). In the controlled experiments, three EDSRs were trained using our proposed method, Gaussian blur, and k-space zero-filling. As for the data, liver MR images were obtained from 24 patients using breath-hold serial LR and HR scans (only HR images were used in the conventional methods). The k-space zero-filling group delivered almost zero enhancement on the real LR images and the Gaussian group produced a considerable number of artifacts. The proposed method exhibited significantly better resolution enhancement and fewer artifacts compared with the other two networks. Our method outperformed the Gaussian method by an improvement of 0.111 ± 0.016 in the structural similarity index and 2.76 ± 0.98 dB in the peak signal-to-noise ratio. The blind/reference-less image spatial quality evaluator metric of the conventional Gaussian method and proposed method were 46.6 ± 4.2 and 34.1 ± 2.4, respectively.

3D dense convolutional neural network for fast and accurate single MR image super-resolution

Super-resolution (SR) MR image reconstruction has shown to be a very promising direction to improve the spatial resolution of low-resolution (LR) MR images. In this paper, we presented a novel MR image SR method based on a dense convolutional neural network (DDSR), and its enhanced version called EDDSR. There are three major innovations: first, we re-designed dense modules to extract hierarchical features directly from LR images and propagate the extracted feature maps through dense connections. Therefore, unlike other CNN-based SR MR techniques that upsample LR patches in the initial phase, our methods take the original LR images or patches as input. This effectively reduces computational complexity and speeds up SR reconstruction. Second, a final deconvolution filter in our model automatically learns filters to fuse and upscale all hierarchical feature maps to generate HR MR images. Using this, EDDSR can perform SR reconstructions at different upscale factors using a single model with one stride fixed deconvolution operation. Third, to further improve SR reconstruction accuracy, we exploited a geometric self-ensemble strategy. Experimental results on three benchmark datasets demonstrate that our methods, DDSR and EDDSR, achieved superior performance compared to state-of-the-art MR image SR methods with less computational load and memory usage.

Robust Single-Image Super-Resolution via CNNs and TV-TV Minimization

Single-image super-resolution is the process of increasing the resolution of an image, obtaining a high-resolution (HR) image from a low-resolution (LR) one. By leveraging large training datasets, convolutional neural networks (CNNs) currently achieve the state-of-the-art performance in this task. Yet, during testing/deployment, they fail to enforce consistency between the HR and LR images: if we downsample the output HR image, it never matches its LR input. Based on this observation, we propose to post-process the CNN outputs with an optimization problem that we call TV-TV minimization, which enforces consistency. As our extensive experiments show, such post-processing not only improves the quality of the images, in terms of PSNR and SSIM, but also makes the super-resolution task robust to operator mismatch, i.e., when the true downsampling operator is different from the one used to create the training dataset.

Joint super-resolution and synthesis of 1 mm isotropic MP-RAGE volumes from clinical MRI exams with scans of different orientation, resolution and contrast

Most existing algorithms for automatic 3D morphometry of human brain MRI scans are designed for data with near-isotropic voxels at approximately 1 mm resolution, and frequently have contrast constraints as well-typically requiring T1-weighted images (e.g., MP-RAGE scans). This limitation prevents the analysis of millions of MRI scans acquired with large inter-slice spacing in clinical settings every year. In turn, the inability to quantitatively analyze these scans hinders the adoption of quantitative neuro imaging in healthcare, and also precludes research studies that could attain huge sample sizes and hence greatly improve our understanding of the human brain. Recent advances in convolutional neural networks (CNNs) are producing outstanding results in super-resolution and contrast synthesis of MRI. However, these approaches are very sensitive to the specific combination of contrast, resolution and orientation of the input images, and thus do not generalize to diverse clinical acquisition protocols - even within sites. In this article, we present SynthSR, a method to train a CNN that receives one or more scans with spaced slices, acquired with different contrast, resolution and orientation, and produces an isotropic scan of canonical contrast (typically a 1 mm MP-RAGE). The presented method does not require any preprocessing, beyond rigid coregistration of the input scans. Crucially, SynthSR trains on synthetic input images generated from 3D segmentations, and can thus be used to train CNNs for any combination of contrasts, resolutions and orientations without high-resolution real images of the input contrasts. We test the images generated with SynthSR in an array of common downstream analyses, and show that they can be reliably used for subcortical segmentation and volumetry, image registration (e.g., for tensor-based morphometry), and, if some image quality requirements are met, even cortical thickness morphometry. The source code is publicly available at https://github.com/BBillot/SynthSR.

Multi-slice low-rank tensor decomposition based multi-atlas segmentation: Application to automatic pathological liver CT segmentation

Liver segmentation from abdominal CT images is an essential step for liver cancer computer-aided diagnosis and surgical planning. However, both the accuracy and robustness of existing liver segmentation methods cannot meet the requirements of clinical applications. In particular, for the common clinical cases where the liver tissue contains major pathology, current segmentation methods show poor performance. In this paper, we propose a novel low-rank tensor decomposition (LRTD) based multi-atlas segmentation (MAS) framework that achieves accurate and robust pathological liver segmentation of CT images. Firstly, we propose a multi-slice LRTD scheme to recover the underlying low-rank structure embedded in 3D medical images. It performs the LRTD on small image segments consisting of multiple consecutive image slices. Then, we present an LRTD-based atlas construction method to generate tumor-free liver atlases that mitigates the performance degradation of liver segmentation due to the presence of tumors. Finally, we introduce an LRTD-based MAS algorithm to derive patient-specific liver atlases for each test image, and to achieve accurate pairwise image registration and label propagation. Extensive experiments on three public databases of pathological liver cases validate the effectiveness of the proposed method. Both qualitative and quantitative results demonstrate that, in the presence of major pathology, the proposed method is more accurate and robust than state-of-the-art methods.

Super Resolution of Magnetic Resonance Images

In this work, novel denoising and super resolution (SR) approaches for magnetic resonance (MR) images are addressed, and are integrated in a unified framework, which do not require example low resolution (LR)/high resolution (HR)/cross-modality/noise-free images and prior information of noise-noise variance. The proposed method categorizes the patches as either smooth or textured and then denoises them by deploying different denoising strategies for efficient denoising. The denoising algorithm is integrated into the SR approach, which uses a gradient profile-based constraint in a sparse representation-based framework to improve the resolution of MR images with reduced smearing of image details. This constraint regularizes the estimation of HR images such that the estimated HR image has gradient profiles similar to the gradient profiles of the original HR image. For this, the gradient profile sharpness (GPS) values of an unknown HR image are estimated using an approximated piece-wise linear relation among GPS values of LR and upsampled LR images. The experiments are performed on three different publicly available datasets. The proposed SR approach outperforms the existing unsupervised SR approach addressed for real MR images that exploits low rank and total variation (LRTV) regularization, by an average peak signal to noise ratio (PSNR) of 0.73 dB and 0.38 dB for upsampling factors 2 and 3, respectively. For the super resolution of noisy real MR images (degraded with 2% noise), the proposed approach outperforms the LRTV approach by an average PSNR of 0.54 dB and 0.46 dB for upsampling factors 2 and 3, respectively. The qualitative analysis is shown for real MR images from healthy subjects and subjects with Alzheimer's disease and structural deformity, i.e., cavernoma.

Fast and High-Resolution Neonatal Brain MRI Through Super-Resolution Reconstruction From Acquisitions With Variable Slice Selection Direction

The brain of neonates is small in comparison to adults. Imaging at typical resolutions such as one cubic mm incurs more partial voluming artifacts in a neonate than in an adult. The interpretation and analysis of MRI of the neonatal brain benefit from a reduction in partial volume averaging that can be achieved with high spatial resolution. Unfortunately, direct acquisition of high spatial resolution MRI is slow, which increases the potential for motion artifact, and suffers from reduced signal-to-noise ratio. The purpose of this study is thus that using super-resolution reconstruction in conjunction with fast imaging protocols to construct neonatal brain MRI images at a suitable signal-to-noise ratio and with higher spatial resolution than can be practically obtained by direct Fourier encoding. We achieved high quality brain MRI at a spatial resolution of isotropic 0.4 mm with 6 min of imaging time, using super-resolution reconstruction from three short duration scans with variable directions of slice selection. Motion compensation was achieved by aligning the three short duration scans together. We applied this technique to 20 newborns and assessed the quality of the images we reconstructed. Experiments show that our approach to super-resolution reconstruction achieved considerable improvement in spatial resolution and signal-to-noise ratio, while, in parallel, substantially reduced scan times, as compared to direct high-resolution acquisitions. The experimental results demonstrate that our approach allowed for fast and high-quality neonatal brain MRI for both scientific research and clinical studies.

Autoencoder-Inspired Convolutional Network-Based Super-Resolution Method in MRI

OBJECTIVE

To introduce an MRI in-plane resolution enhancement method that estimates High-Resolution (HR) MRIs from Low-Resolution (LR) MRIs.

METHOD & MATERIALS

Previous CNN-based MRI super-resolution methods cause loss of input image information due to the pooling layer. An Autoencoder-inspired Convolutional Network-based Super-resolution (ACNS) method was developed with the deconvolution layer that extrapolates the missing spatial information by the convolutional neural network-based nonlinear mapping between LR and HR features of MRI. Simulation experiments were conducted with virtual phantom images and thoracic MRIs from four volunteers. The Peak Signal-to-Noise Ratio (PSNR), Structure SIMilarity index (SSIM), Information Fidelity Criterion (IFC), and computational time were compared among: ACNS; Super-Resolution Convolutional Neural Network (SRCNN); Fast Super-Resolution Convolutional Neural Network (FSRCNN); Deeply-Recursive Convolutional Network (DRCN).

RESULTS

ACNS achieved comparable PSNR, SSIM, and IFC results to SRCNN, FSRCNN, and DRCN. However, the average computation speed of ACNS was 6, 4, and 35 times faster than SRCNN, FSRCNN, and DRCN, respectively under the computer setup used with the actual average computation time of 0.15 s per [Formula: see text].

3D MRI Reconstruction Based on 2D Generative Adversarial Network Super-Resolution

The diagnosis of brain pathologies usually involves imaging to analyze the condition of the brain. Magnetic resonance imaging (MRI) technology is widely used in brain disorder diagnosis. The image quality of MRI depends on the magnetostatic field strength and scanning time. Scanners with lower field strengths have the disadvantages of a low resolution and high imaging cost, and scanning takes a long time. The traditional super-resolution reconstruction method based on MRI generally states an optimization problem in terms of prior information. It solves the problem using an iterative approach with a large time cost. Many methods based on deep learning have emerged to replace traditional methods. MRI super-resolution technology based on deep learning can effectively improve MRI resolution through a three-dimensional convolutional neural network; however, the training costs are relatively high. In this paper, we propose the use of two-dimensional super-resolution technology for the super-resolution reconstruction of MRI images. In the first reconstruction, we choose a scale factor of 2 and simulate half the volume of MRI slices as input. We utilize a receiving field block enhanced super-resolution generative adversarial network (RFB-ESRGAN), which is superior to other super-resolution technologies in terms of texture and frequency information. We then rebuild the super-resolution reconstructed slices in the MRI. In the second reconstruction, the image after the first reconstruction is composed of only half of the slices, and there are still missing values. In our previous work, we adopted the traditional interpolation method, and there was still a gap in the visual effect of the reconstructed images. Therefore, we propose a noise-based super-resolution network (nESRGAN). The noise addition to the network can provide additional texture restoration possibilities. We use nESRGAN to further restore MRI resolution and high-frequency information. Finally, we achieve the 3D reconstruction of brain MRI images through two super-resolution reconstructions. Our proposed method is superior to 3D super-resolution technology based on deep learning in terms of perception range and image quality evaluation standards.

PSR: Unified Framework of Parameter-Learning-Based MR Image Superresolution

Magnetic resonance imaging has significant applications for disease diagnosis. Due to the particularity of its imaging mechanism, hardware imaging suffers from resolution and reaches its limit, and higher radiation intensity and longer radiation time will cause damage to the human body. The problem is expected to be solved by a superresolution algorithm, especially the image superresolution based on sparse reconstruction has good performance. Dictionary generation is a key issue that affects the performance of superresolution algorithms, and dictionary performance is affected by dictionary construction parameters: balance parameters, dictionary size, overlapping block size, and a number of training sample blocks. In response to this problem, we propose an optimal dictionary construction parameter search method through the experiment to find the optimal dictionary construction parameters on the MR image and compare them with the dictionary obtained by multiple sets of random dictionary construction parameters. The dictionary we searched for the optimal parameters of the dictionary construction training has more powerful feature expressions, which can improve the superresolution effect of MR images.

SMORE: A Self-Supervised Anti-Aliasing and Super-Resolution Algorithm for MRI Using Deep Learning

High resolution magnetic resonance (MR) images are desired in many clinical and research applications. Acquiring such images with high signal-to-noise (SNR), however, can require a long scan duration, which is difficult for patient comfort, is more costly, and makes the images susceptible to motion artifacts. A very common practical compromise for both 2D and 3D MR imaging protocols is to acquire volumetric MR images with high in-plane resolution, but lower through-plane resolution. In addition to having poor resolution in one orientation, 2D MRI acquisitions will also have aliasing artifacts, which further degrade the appearance of these images. This paper presents an approach SMORE1 based on convolutional neural networks (CNNs) that restores image quality by improving resolution and reducing aliasing in MR images.2 This approach is self-supervised, which requires no external training data because the high-resolution and low-resolution data that are present in the image itself are used for training. For 3D MRI, the method consists of only one self-supervised super-resolution (SSR) deep CNN that is trained from the volumetric image data. For 2D MRI, there is a self-supervised anti-aliasing (SAA) deep CNN that precedes the SSR CNN, also trained from the volumetric image data. Both methods were evaluated on a broad collection of MR data, including filtered and downsampled images so that quantitative metrics could be computed and compared, and actual acquired low resolution images for which visual and sharpness measures could be computed and compared. The super-resolution method is shown to be visually and quantitatively superior to previously reported methods.

Image quality improvement of single-shot turbo spin-echo magnetic resonance imaging of female pelvis using a convolutional neural network

We have developed a deep learning-based approach to improve image quality of single-shot turbo spin-echo (SSTSE) images of female pelvis. We aimed to compare the deep learning-based single-shot turbo spin-echo (DL-SSTSE) images of female pelvis with turbo spin-echo (TSE) and conventional SSTSE images in terms of image quality.One hundred five and 21 subjects were used as training and test sets, respectively. We performed 6-fold cross validation. In the training process, low-quality images were generated from TSE images as input. TSE images were used as ground truth images. In the test process, the trained convolutional neural network was applied to SSTSE images. The output images were denoted as DL-SSTSE images. Apart from DL-SSTSE images, classical filtering methods were adopted to SSTSE images. Generated images were denoted as F-SSTSE images. Contrast ratio (CR) of gluteal fat and myometrium and signal-to-noise ratio (SNR) of gluteal fat were measured for all images. Two radiologists graded these images using a 5-point scale and evaluated the image quality with regard to overall image quality, contrast, noise, motion artifact, boundary sharpness of layers in the uterus, and the conspicuity of the ovaries. CRs, SNRs, and image quality scores were compared using the Steel-Dwass multiple comparison tests.CRs and SNRs were significantly higher in DL-SSTSE, F-SSTSE, and TSE images than in SSTSE images. Scores with regard to overall image quality, contrast, noise, and boundary sharpness of layers in the uterus were significantly higher on DL-SSTSE and TSE images than on SSTSE images. There were no significant differences in the CRs, SNRs, and respective scores between DL-SSTSE and TSE images. The score with regard to motion artifacts was significantly higher on DL-SSTSE, F-SSTSE, and SSTSE images than on TSE images. The score with regard to the conspicuity of ovaries was significantly higher on DL-SSTSE images than on F-SSTSE, SSTSE, and TSE images (P < .001).DL-SSTSE images showed higher image quality as compared with SSTSE images. In comparison with conventional TSE images, DL-SSTSE images had acceptable image quality while keeping the advantage of the motion artifact-robustness and acquisition time efficiency in SSTSE imaging.

Adversarial Confidence Learning for Medical Image Segmentation and Synthesis

Generative adversarial networks (GAN) are widely used in medical image analysis tasks, such as medical image segmentation and synthesis. In these works, adversarial learning is directly applied to the original supervised segmentation (synthesis) networks. The usage of adversarial learning is effective in improving visual perception performance since adversarial learning works as realistic regularization for supervised generators. However, the quantitative performance often cannot improve as much as the qualitative performance, and it can even become worse in some cases. In this paper, we explore how we can take better advantage of adversarial learning in supervised segmentation (synthesis) models and propose an adversarial confidence learning framework to better model these problems. We analyze the roles of discriminator in the classic GANs and compare them with those in supervised adversarial systems. Based on this analysis, we propose adversarial confidence learning, i.e., besides the adversarial learning for emphasizing visual perception, we use the confidence information provided by the adversarial network to enhance the design of supervised segmentation (synthesis) network. In particular, we propose using a fully convolutional adversarial network for confidence learning to provide voxel-wise and region-wise confidence information for the segmentation (synthesis) network. With these settings, we propose a difficulty-aware attention mechanism to properly handle hard samples or regions by taking structural information into consideration so that we can better deal with the irregular distribution of medical data. Furthermore, we investigate the loss functions of various GANs and propose using the binary cross entropy loss to train the proposed adversarial system so that we can retain the unlimited modeling capacity of the discriminator. Experimental results on clinical and challenge datasets show that our proposed network can achieve state-of-the-art segmentation (synthesis) accuracy. Further analysis also indicates that adversarial confidence learning can both improve the visual perception performance and the quantitative performance.

Spectral CT Reconstruction via Low-Rank Representation and Region-Specific Texture Preserving Markov Random Field Regularization

Photon-counting spectral computed tomography (CT) is capable of material characterization and can improve diagnostic performance over traditional clinical CT. However, it suffers from photon count starving for each individual energy channel which may cause severe artifacts in the reconstructed images. Furthermore, since the images in different energy channels describe the same object, there are high correlations among different channels. To make full use of the inter-channel correlations and minimize the count starving effect while maintaining clinically meaningful texture information, this paper combines a region-specific texture model with a low-rank correlation descriptor as an a priori regularization to explore a superior texture preserving Bayesian reconstruction of spectral CT. Specifically, the inter-channel correlations are characterized by the low-rank representation, and the inner-channel regional textures are modeled by a texture preserving Markov random field. In other words, this paper integrates the spectral and spatial information into a unified Bayesian reconstruction framework. The widely-used Split-Bregman algorithm is employed to minimize the objective function because of the non-differentiable property of the low-rank representation. To evaluate the tissue texture preserving performance of the proposed method for each channel, three references are built for comparison: one is the traditional CT image from energy integration detection. The second one is spectral images from dual-energy CT. The third one is individual channels images from custom-made photon-counting spectral CT. As expected, the proposed method produced promising results in terms of not only preserving texture features but also suppressing image noise in each channel, comparing to existing methods of total variation (TV), low-rank TV and tensor dictionary learning, by both visual inspection and quantitative indexes of root mean square error, peak signal to noise ratio, structural similarity and feature similarity.