LRTV: MR Image Super-Resolution With Low-Rank and Total Variation Regularizations

PathSRGAN: Multi-Supervised Super-Resolution for Cytopathological Images Using Generative Adversarial Network

In the cytopathology screening of cervical cancer, high-resolution digital cytopathological slides are critical for the interpretation of lesion cells. However, the acquisition of high-resolution digital slides requires high-end imaging equipment and long scanning time. In the study, we propose a GAN-based progressive multi-supervised super-resolution model called PathSRGAN (pathology super-resolution GAN) to learn the mapping of real low-resolution and high-resolution cytopathological images. With respect to the characteristics of cytopathological images, we design a new two-stage generator architecture with two supervision terms. The generator of the first stage corresponds to a densely-connected U-Net and achieves 4× to 10× super resolution. The generator of the second stage corresponds to a residual-in-residual DenseBlock and achieves 10× to 20× super resolution. The designed generator alleviates the difficulty in learning the mapping from 4× images to 20× images caused by the great numerical aperture difference and generates high quality high-resolution images. We conduct a series of comparison experiments and demonstrate the superiority of PathSRGAN to mainstream CNN-based and GAN-based super-resolution methods in cytopathological images. Simultaneously, the reconstructed high-resolution images by PathSRGAN improve the accuracy of computer-aided diagnosis tasks effectively. It is anticipated that the study will help increase the penetration rate of cytopathology screening in remote and impoverished areas that lack high-end imaging equipment.

A trusted medical image super-resolution method based on feedback adaptive weighted dense network

High-resolution (HR) medical images are preferred in clinical diagnoses and subsequent analysis. However, the acquisition of HR medical images is easily affected by hardware devices. As an effective and trusted alternative method, the super-resolution (SR) technology is introduced to improve the image resolution. Compared with traditional SR methods, the deep learning-based SR methods can obtain more clear and trusted HR images. In this paper, we propose a trusted deep convolutional neural network-based SR method named feedback adaptive weighted dense network (FAWDN) for HR medical image reconstruction. Specifically, the proposed FAWDN can transmit the information of the output image to the low-level features by a feedback connection. To explore advanced feature representation and reduce the feature redundancy in dense blocks, an adaptive weighted dense block (AWDB) is introduced to adaptively select the informative features. Experimental results demonstrate that our FAWDN outperforms the state-of-the-art image SR methods and can obtain more clear and trusted medical images than comparative methods.

Whole-heart, ungated, free-breathing, cardiac-phase-resolved myocardial perfusion MRI by using Continuous Radial Interleaved simultaneous Multi-slice acquisitions at sPoiled steady-state (CRIMP)

PURPOSE

To develop a whole-heart, free-breathing, non-electrocardiograph (ECG)-gated, cardiac-phase-resolved myocardial perfusion MRI framework (CRIMP; Continuous Radial Interleaved simultaneous Multi-slice acquisitions at sPoiled steady-state) and test its quantification feasibility.

METHODS

CRIMP used interleaved radial simultaneous multi-slice (SMS) slice groups to cover the whole heart in 9 or 12 short-axis slices. The sequence continuously acquired data without magnetization preparation, ECG gating or breath-holding, and captured multiple cardiac phases. Images were reconstructed by a motion-compensated patch-based locally low-rank reconstruction. Bloch simulations were performed to study the signal-to-noise ratio/contrast-to-noise ratio (SNR/CNR) for CRIMP and to study the steady-state signal under motion. Seven patients were scanned with CRIMP at stress and rest to develop the sequence. One human and two dogs were scanned at rest with a dual-bolus method to test the quantification feasibility of CRIMP. The dual-bolus scans were performed using both CRIMP and an ungated radial SMS saturation recovery (SMS-SR) sequence with injection dose = 0.075 mmol/kg to compare the sequences in terms of SNR, cardiac phase resolution and quantitative myocardial blood flow (MBF).

RESULTS

Perfusion images with multiple cardiac phases in all image slices with a temporal resolution of 72 ms/frame were obtained. Simulations and in-vivo acquisitions showed CRIMP kept the inner slices in steady-state regardless of motion. CRIMP outperformed SMS-SR in slice coverage (9 over 6), SNR (mean 20% improvement), and provided cardiac phase resolution. CRIMP and SMS-SR sequences provided comparable MBF values (rest systolic CRIMP = 0.58 ± 0.07, SMS-SR = 0.61 ± 0.16).

CONCLUSION

CRIMP allows for whole-heart, cardiac-phase-resolved myocardial perfusion images without ECG-gating or breath-holding. The sequence can provide MBF if an accurate arterial input function is obtained separately.

Synthesized 7T MRI from 3T MRI via deep learning in spatial and wavelet domains

Ultra-high field 7T MRI scanners, while producing images with exceptional anatomical details, are cost prohibitive and hence highly inaccessible. In this paper, we introduce a novel deep learning network that fuses complementary information from spatial and wavelet domains to synthesize 7T T1-weighted images from their 3T counterparts. Our deep learning network leverages wavelet transformation to facilitate effective multi-scale reconstruction, taking into account both low-frequency tissue contrast and high-frequency anatomical details. Our network utilizes a novel wavelet-based affine transformation (WAT) layer, which modulates feature maps from the spatial domain with information from the wavelet domain. Extensive experimental results demonstrate the capability of the proposed method in synthesizing high-quality 7T images with better tissue contrast and greater details, outperforming state-of-the-art methods.

Super-Resolution Whole-Brain 3D MR Spectroscopic Imaging for Mapping D-2-Hydroxyglutarate and Tumor Metabolism in Isocitrate Dehydrogenase 1-mutated Human Gliomas

Background Isocitrate dehydrogenase (IDH) mutations are highly frequent in glioma, producing high levels of the oncometabolite D-2-hydroxyglutarate (D-2HG). Hence, D-2HG represents a valuable imaging marker for IDH-mutated human glioma. Purpose To develop and evaluate a super-resolution three-dimensional (3D) MR spectroscopic imaging strategy to map D-2HG and tumor metabolism in IDH-mutated human glioma. Materials and Methods Between March and September 2018, participants with IDH1-mutated gliomas and healthy participants were prospectively scanned with a 3-T whole-brain 3D MR spectroscopic imaging protocol optimized for D-2HG. The acquired D-2HG maps with a voxel size of 5.2 × 5.2 × 12 mm were upsampled to a voxel size of 1.7 × 1.7 × 3 mm using a super-resolution method that combined weighted total variation, feature-based nonlocal means, and high-spatial-resolution anatomic imaging priors. Validation with simulated healthy and patient data and phantom measurements was also performed. The Mann-Whitney U test was used to check that the proposed super-resolution technique yields the highest peak signal-to-noise ratio and structural similarity index. Results Three participants with IDH1-mutated gliomas (mean age, 50 years ± 21 [standard deviation]; two men) and three healthy participants (mean age, 32 years ± 3; two men) were scanned. Twenty healthy participants (mean age, 33 years ± 5; 16 men) underwent a simulation of upsampled MR spectroscopic imaging. Super-resolution upsampling improved peak signal-to-noise ratio and structural similarity index by 62% (P < .05) and 7.3% (P < .05), respectively, for simulated data when compared with spline interpolation. Correspondingly, the proposed method significantly improved tissue contrast and structural information for the acquired 3D MR spectroscopic imaging data. Conclusion High-spatial-resolution whole-brain D-2-hydroxyglutarate imaging is possible in isocitrate dehydrogenase 1-mutated human glioma by using a super-resolution framework to upsample three-dimensional MR spectroscopic images acquired at lower resolution. © RSNA, 2020 Online supplemental material is available for this article. See also the editorial by Huang and Lin in this issue.

Fusion of Magnetic Resonance and Ultrasound Images for Endometriosis Detection

This paper introduces a new fusion method for magnetic resonance (MR) and ultrasound (US) images, which aims at combining the advantages of each modality, i.e., good contrast and signal to noise ratio for the MR image and good spatial resolution for the US image. The proposed algorithm is based on two inverse problems, performing a super-resolution of the MR image and a denoising of the US image. A polynomial function is introduced to model the relationships between the gray levels of the two modalities. The resulting inverse problem is solved using a proximal alternating linearized minimization framework. The accuracy and the interest of the fusion algorithm are shown quantitatively and qualitatively via evaluations on synthetic and experimental phantom data.

Specular Reflections Removal for Endoscopic Image Sequences With Adaptive-RPCA Decomposition

Specular reflections (i.e., highlight) always exist in endoscopic images, and they can severely disturb surgeons' observation and judgment. In an augmented reality (AR)-based surgery navigation system, the highlight may also lead to the failure of feature extraction or registration. In this paper, we propose an adaptive robust principal component analysis (Adaptive-RPCA) method to remove the specular reflections in endoscopic image sequences. It can iteratively optimize the sparse part parameter during RPCA decomposition. In this new approach, we first adaptively detect the highlight image based on pixels. With the proposed distance metric algorithm, it then automatically measures the similarity distance between the sparse result image and the detected highlight image. Finally, the low-rank and sparse results are obtained by enforcing the similarity distance between the two types of images to fall within a certain range. Our method has been verified by multiple different types of endoscopic image sequences in minimally invasive surgery (MIS). The experiments and clinical blind tests demonstrate that the new Adaptive-RPCA method can obtain the optimal sparse decomposition parameters directly and can generate robust highlight removal results. Compared with the state-of-the-art approaches, the proposed method not only achieves the better highlight removal results but also can adaptively process image sequences.

Brain Lesion Segmentation Based on Joint Constraints of Low-Rank Representation and Sparse Representation

The segmentation of brain lesions from a brain magnetic resonance (MR) image is of great significance for the clinical diagnosis and follow-up treatment. An automatic segmentation method for brain lesions is proposed based on the low-rank representation (LRR) and the sparse representation (SR) theory. The proposed method decomposes the brain image into the background part composed of brain tissue and the brain lesion part. Considering that each pixel in the brain tissue can be represented by the background dictionary, a low-rank representation that incorporates sparsity-inducing regularization term is adopted to model the part. Then, the linearized alternating direction method with adaptive penalty (LADMAP) was selected to solve the model, and the brain lesions can be obtained by the response of the residual matrix. The presented model not only reflects the global structure of the image but also preserves the local information of the pixels, thus improving the representation accuracy. The experimental results on the data of brain tumor patients and multiple sclerosis patients revealed that the proposed method is superior to several existing methods in terms of segmentation accuracy while realizing the segmentation automatically.

Globally Optimized Super-Resolution of Diffusion MRI Data via Fiber Continuity

In this paper, we introduce a technique for super-resolution reconstruction of diffusion MRI, harnessing fiber-continuity (FC) as a constraint in a global whole-brain optimization framework. FC is a biologically-motivated constraint that relates orientation information between neighboring voxels. We show that it can be used to effectively constrain the inverse problem of recovering high-resolution data from low-resolution data. Since voxels are inter-related by FC, we devise a global optimization framework that allows solutions pertaining to all voxels to be solved simultaneously. We demonstrate that the proposed super-resolution framework is effective for diffusion MRI data of a glioma patient, a healthy subject, and a macaque.

Applications of a deep learning method for anti-aliasing and super-resolution in MRI

Magnetic resonance (MR) images with both high resolutions and high signal-to-noise ratios (SNRs) are desired in many clinical and research applications. However, acquiring such images takes a long time, which is both costly and susceptible to motion artifacts. Acquiring MR images with good in-plane resolution and poor through-plane resolution is a common strategy that saves imaging time, preserves SNR, and provides one viewpoint with good resolution in two directions. Unfortunately, this strategy also creates orthogonal viewpoints that have poor resolution in one direction and, for 2D MR acquisition protocols, also creates aliasing artifacts. A deep learning approach called SMORE that carries out both anti-aliasing and super-resolution on these types of acquisitions using no external atlas or exemplars has been previously reported but not extensively validated. This paper reviews the SMORE algorithm and then demonstrates its performance in four applications with the goal to demonstrate its potential for use in both research and clinical scenarios. It is first shown to improve the visualization of brain white matter lesions in FLAIR images acquired from multiple sclerosis patients. Then it is shown to improve the visualization of scarring in cardiac left ventricular remodeling after myocardial infarction. Third, its performance on multi-view images of the tongue is demonstrated and finally it is shown to improve performance in parcellation of the brain ventricular system. Both visual and selected quantitative metrics of resolution enhancement are demonstrated.

Gradient-Guided Convolutional Neural Network for MRI Image Super-Resolution

Super-resolution (SR) technology is essential for improving image quality in magnetic resonance imaging (MRI). The main challenge of MRI SR is to reconstruct high-frequency (HR) details from a low-resolution (LR) image. To address this challenge, we develop a gradient-guided convolutional neural network for improving the reconstruction accuracy of high-frequency image details from the LR image. A gradient prior is fully explored to supply the information of high-frequency details during the super-resolution process, thereby leading to a more accurate reconstructed image. Experimental results of image super-resolution on public MRI databases demonstrate that the gradient-guided convolutional neural network achieves better performance over the published state-of-art approaches.

Deep MR Brain Image Super-Resolution Using Spatio-Structural Priors

High resolution Magnetic Resonance (MR) images are desired for accurate diagnostics. In practice, image resolution is restricted by factors like hardware and processing constraints. Recently, deep learning methods have been shown to produce compelling state-of-the-art results for image enhancement/super-resolution. Paying particular attention to desired hi-resolution MR image structure, we propose a new regularized network that exploits image priors, namely a low-rank structure and a sharpness prior to enhance deep MR image super-resolution (SR). Our contributions are then incorporating these priors in an analytically tractable fashion as well as towards a novel prior guided network architecture that accomplishes the super-resolution task. This is particularly challenging for the low rank prior since the rank is not a differentiable function of the image matrix (and hence the network parameters), an issue we address by pursuing differentiable approximations of the rank. Sharpness is emphasized by the variance of the Laplacian which we show can be implemented by a fixed feedback layer at the output of the network. As a key extension, we modify the fixed feedback (Laplacian) layer by learning a new set of training data driven filters that are optimized for enhanced sharpness. Experiments performed on publicly available MR brain image databases and comparisons against existing state-of-the-art methods show that the proposed prior guided network offers significant practical gains in terms of improved SNR/image quality measures. Because our priors are on output images, the proposed method is versatile and can be combined with a wide variety of existing network architectures to further enhance their performance.

An Entropy-Based Algorithm with Nonlocal Residual Learning for Image Compressive Sensing Recovery

Image recovery from compressive sensing (CS) measurement data, especially noisy data has always been challenging due to its implicit ill-posed nature, thus, to seek a domain where a signal can exhibit a high degree of sparsity and to design an effective algorithm have drawn increasingly more attention. Among various sparsity-based models, structured or group sparsity often leads to more powerful signal reconstruction techniques. In this paper, we propose a novel entropy-based algorithm for CS recovery to enhance image sparsity through learning the group sparsity of residual. To reduce the residual of similar packed patches, the group sparsity of residual is described by a Laplacian scale mixture (LSM) model, therefore, each singular value of the residual of similar packed patches is modeled as a Laplacian distribution with a variable scale parameter, to exploit the benefits of high-order dependency among sparse coefficients. Due to the latent variables, the maximum a posteriori (MAP) estimation of the sparse coefficients cannot be obtained, thus, we design a loss function for expectation–maximization (EM) method based on relative entropy. In the frame of EM iteration, the sparse coefficients can be estimated with the denoising-based approximate message passing (D-AMP) algorithm. Experimental results have shown that the proposed algorithm can significantly outperform existing CS techniques for image recovery.

Multiscale brain MRI super-resolution using deep 3D convolutional networks

The purpose of super-resolution approaches is to overcome the hardware limitations and the clinical requirements of imaging procedures by reconstructing high-resolution images from low-resolution acquisitions using post-processing methods. Super-resolution techniques could have strong impacts on structural magnetic resonance imaging when focusing on cortical surface or fine-scale structure analysis for instance. In this paper, we study deep three-dimensional convolutional neural networks for the super-resolution of brain magnetic resonance imaging data. First, our work delves into the relevance of several factors in the performance of the purely convolutional neural network-based techniques for the monomodal super-resolution: optimization methods, weight initialization, network depth, residual learning, filter size in convolution layers, number of the filters, training patch size and number of training subjects. Second, our study also highlights that one single network can efficiently handle multiple arbitrary scaling factors based on a multiscale training approach. Third, we further extend our super-resolution networks to the multimodal super-resolution using intermodality priors. Fourth, we investigate the impact of transfer learning skills onto super-resolution performance in terms of generalization among different datasets. Lastly, the learnt models are used to enhance real clinical low-resolution images. Results tend to demonstrate the potential of deep neural networks with respect to practical medical image applications.

Undersampled MR image reconstruction using an enhanced recursive residual network

When using aggressive undersampling, it is difficult to recover the high quality image with reliably fine features. In this paper, we propose an enhanced recursive residual network (ERRN) that improves the basic recursive residual network with a high-frequency feature guidance, an error-correction unit and dense connections. The feature guidance is designed to predict the underlying anatomy based on image a priori learned from the label data, playing a complementary role to the residual learning. The ERRN is adapted for two important applications: compressed sensing (CS) MRI and super resolution (SR) MRI, while an application-specific error-correction unit is added into the framework, i.e. data consistency for CS-MRI and back projection for SR-MRI due to their different sampling schemes. Our proposed network was evaluated using a real-valued brain dataset, a complex-valued knee dataset, pathological brain data and in vivo rat brain data with different undersampling masks and rates. Experimental results demonstrated that ERRN presented superior reconstructions at all cases with distinctly restored structural features and highest image quality metrics compared to both the state-of-the-art convolutional neural networks and the conventional optimization-based methods, particularly for the undersampling rate over 5-fold. Thus, an excellent framework design can endow the network with a flexible architecture, fewer parameters, outstanding performances for various undersampling schemes, and reduced overfitting in generalization, which will facilitate real-time reconstruction on MRI scanners.

On Single-Image Super-Resolution in 3D Brain Magnetic Resonance Imaging

The objective of this work is to apply 3D super resolution (SR) techniques to brain magnetic resonance (MR) image restoration. Two 3D SR methods are considered following different trends: one recently proposed tensor-based approach and one inverse problem algorithm based on total variation and low rank regularization. The evaluation of their effectiveness is assessed through the segmentation of brain compartments: gray matter, white matter and cerebrospinal fluid. The two algorithms are qualitatively and quantitatively evaluated on simulated images with ground truth available and on experimental data. The originality of this work is to consider the SR methods as an initial step towards the final segmentation task. The results show the ability of both methods to overcome the loss of spatial resolution and to facilitate the segmentation of brain structures with improved accuracy compared to native low-resolution MR images. Both algorithms achieved almost equivalent results with a highly reduced computational time cost for the tensor-based approach.

Super-resolution reconstruction of neonatal brain magnetic resonance images via residual structured sparse representation

Magnetic resonance images of neonates, compared with toddlers, exhibit lower signal-to-noise ratio and spatial resolution. In this paper, we propose a novel method for super-resolution reconstruction of neonate images with the help of toddler images, using residual-structured sparse representation with convex regularization. Specifically, we introduce a two-layer image representation, consisting of a base layer and a detail layer, to cater to signal variation across scanners and sites. The base layer consists of the smoothed version of the image obtained via Gaussian filtering. The detail layer is the difference between the original image and the base layer. High-frequency details in the detail layer are borrowed across subjects for super-resolution reconstruction. Experimental results on T1 and T2 images demonstrate that the proposed algorithm can recover fine anatomical structures, and generally outperform the state-of-the-art methods both qualitatively and quantitatively.

A Tensor Factorization Method for 3-D Super Resolution With Application to Dental CT

Available super-resolution techniques for 3-D images are either computationally inefficient prior-knowledge-based iterative techniques or deep learning methods which require a large database of known low-resolution and high-resolution image pairs. A recently introduced tensor-factorization-based approach offers a fast solution without the use of known image pairs or strict prior assumptions. In this paper, this factorization framework is investigated for single image resolution enhancement with an offline estimate of the system point spread function. The technique is applied to 3-D cone beam computed tomography for dental image resolution enhancement. To demonstrate the efficiency of our method, it is compared to a recent state-of-the-art iterative technique using low-rank and total variation regularizations. In contrast to this comparative technique, the proposed reconstruction technique gives a 2-order-of-magnitude improvement in running time-2 min compared to 2 h for a dental volume of 282×266×392 voxels. Furthermore, it also offers slightly improved quantitative results (peak signal-to-noise ratio and segmentation quality). Another advantage of the presented technique is the low number of hyperparameters. As demonstrated in this paper, the framework is not sensitive to small changes in its parameters, proposing an ease of use.

Longitudinally Guided Super-Resolution of Neonatal Brain Magnetic Resonance Images

Neonatal magnetic resonance (MR) images typically have low spatial resolution and insufficient tissue contrast. Interpolation methods are commonly used to upsample the images for the subsequent analysis. However, the resulting images are often blurry and susceptible to partial volume effects. In this paper, we propose a novel longitudinally guided super-resolution (SR) algorithm for neonatal images. This is motivated by the fact that anatomical structures evolve slowly and smoothly as the brain develops after birth. We propose a strategy involving longitudinal regularization, similar to bilateral filtering, in combination with low-rank and total variation constraints to solve the ill-posed inverse problem associated with image SR. Experimental results on neonatal MR images demonstrate that the proposed algorithm recovers clear structural details and outperforms state-of-the-art methods both qualitatively and quantitatively.

DEEP MR IMAGE SUPER-RESOLUTION USING STRUCTURAL PRIORS

High resolution magnetic resonance (MR) images are desired for accurate diagnostics. In practice, image resolution is restricted by factors like hardware, cost and processing constraints. Recently, deep learning methods have been shown to produce compelling state of the art results for image superresolution. Paying particular attention to desired hi-resolution MR image structure, we propose a new regularized network that exploits image priors, namely a low-rank structure and a sharpness prior to enhance deep MR image superresolution. Our contributions are then incorporating these priors in an analytically tractable fashion in the learning of a convolutional neural network (CNN) that accomplishes the super-resolution task. This is particularly challenging for the low rank prior, since the rank is not a differentiable function of the image matrix (and hence the network parameters), an issue we address by pursuing differentiable approximations of the rank. Sharpness is emphasized by the variance of the Laplacian which we show can be implemented by a fixed feedback layer at the output of the network. Experiments performed on two publicly available MR brain image databases exhibit promising results particularly when training imagery is limited.

Super-Resolution of Magnetic Resonance Images via Convex Optimization with Local and Global Prior Regularization and Spectrum Fitting

Given a low-resolution image, there are many challenges to obtain a super-resolved, high-resolution image. Many of those approaches try to simultaneously upsample and deblur an image in signal domain. However, the nature of the super-resolution is to restore high-frequency components in frequency domain rather than upsampling in signal domain. In that sense, there is a close relationship between super-resolution of an image and extrapolation of the spectrum. In this study, we propose a novel framework for super-resolution, where the high-frequency components are theoretically restored with respect to the frequency fidelities. This framework helps to introduce multiple simultaneous regularizers in both signal and frequency domains. Furthermore, we propose a new super-resolution model where frequency fidelity, low-rank (LR) prior, low total variation (TV) prior, and boundary prior are considered at once. The proposed method is formulated as a convex optimization problem which can be solved by the alternating direction method of multipliers. The proposed method is the generalized form of the multiple super-resolution methods such as TV super-resolution, LR and TV super-resolution, and the Gerchberg method. Experimental results show the utility of the proposed method comparing with some existing methods using both simulational and practical images.

Isotropic Reconstruction of MR Images Using 3D Patch-Based Self-Similarity Learning

Isotropic three-dimensional (3D) acquisition is a challenging task in magnetic resonance imaging (MRI). Particularly in cardiac MRI, due to hardware and time limitations, current 3D acquisitions are limited by low-resolution, especially in the through-plane direction, leading to poor image quality in that dimension. To overcome this problem, super-resolution (SR) techniques have been proposed to reconstruct a single isotropic 3D volume from multiple anisotropic acquisitions. Previously, local regularization techniques such as total variation have been applied to limit noise amplification while preserving sharp edges and small features in the images. In this paper, inspired by the recent progress in patch-based reconstruction, we propose a novel isotropic 3D reconstruction scheme that integrates non-local and self-similarity information from 3D patch neighborhoods. By grouping 3D patches with similar structures, we enforce the natural sparsity of MR images, which can be expressed by a low-rank structure, leading to robust image reconstruction with high signal-to-noise ratio efficiency. An Augmented Lagrangian formulation of the problem is proposed to efficiently decompose the optimization into a low-rank volume denoising and a SR reconstruction. Experimental results in simulations, brain imaging and clinical cardiac MRI, demonstrate that the proposed joint SR and self-similarity learning framework outperforms current state-of-the-art methods. The proposed reconstruction of isotropic 3D volumes may be particularly useful for cardiac applications, such as myocardial infarction scar assessment by late gadolinium enhancement MRI.

MR Image Super-Resolution via Wide Residual Networks With Fixed Skip Connection

Spatial resolution is a critical imaging parameter in magnetic resonance imaging. The image super-resolution (SR) is an effective and cost efficient alternative technique to improve the spatial resolution of MR images. Over the past several years, the convolutional neural networks (CNN)-based SR methods have achieved state-of-the-art performance. However, CNNs with very deep network structures usually suffer from the problems of degradation and diminishing feature reuse, which add difficulty to network training and degenerate the transmission capability of details for SR. To address these problems, in this work, a progressive wide residual network with a fixed skip connection (named FSCWRN) based SR algorithm is proposed to reconstruct MR images, which combines the global residual learning and the shallow network based local residual learning. The strategy of progressive wide networks is adopted to replace deeper networks, which can partially relax the above-mentioned problems, while a fixed skip connection helps provide rich local details at high frequencies from a fixed shallow layer network to subsequent networks. The experimental results on one simulated MR image database and three real MR image databases show the effectiveness of the proposed FSCWRN SR algorithm, which achieves improved reconstruction performance compared with other algorithms.

A New 4-D Nonlocal Transform-Domain Filter for 3-D Magnetic Resonance Images Denoising

The simultaneous removal of noise and preservation of the integrity of 3-D magnetic resonance (MR) images is a difficult and important task. In this paper, we consider characterizing MR images with 3-D operators, and present a novel 4-D transform-domain method termed 'modified nonlocal tensor-SVD (MNL-tSVD)' for MR image denoising. The proposed method is based on the grouping, hard-thresholding and aggregation paradigms, and can be viewed as a generalized nonlocal extension of tensor-SVD (t-SVD). By keeping MR images in its natural three-dimensional form, and collaboratively filtering similar patches, MNL-tSVD utilizes both the self-similarity property and 3-D structure of MR images to preserve more actual details and minimize the introduction of new artifacts. We show the adaptability of MNL-tSVD by incorporating it into a two-stage denoising strategy with a few adjustments. In addition, analysis of the relationship between MNL-tSVD and current the state-of-the-art 4-D transforms is given. Experimental comparisons over simulated and real brain data sets at different Rician noise levels show that MNL-tSVD can produce competitive performance compared with related approaches.

Reweighted Low-Rank Matrix Analysis With Structural Smoothness for Image Denoising

In this paper, we develop a new low-rank matrix recovery algorithm for image denoising. We incorporate the total variation (TV) norm and the pixel range constraint into the existing reweighted low-rank matrix analysis to achieve structural smoothness and to significantly improve quality in the recovered image. Our proposed mathematical formulation of the low-rank matrix recovery problem combines the nuclear norm, TV norm, and norm, thereby allowing us to exploit the low-rank property of natural images, enhance the structural smoothness, and detect and remove large sparse noise. Using the iterative alternating direction and fast gradient projection methods, we develop an algorithm to solve the proposed challenging non-convex optimization problem. We conduct extensive performance evaluations on single-image denoising, hyper-spectral image denoising, and video background modeling from corrupted images. Our experimental results demonstrate that the proposed method outperforms the state-of-the-art low-rank matrix recovery methods, particularly for large random noise. For example, when the density of random sparse noise is 30%, for single-image denoising, our proposed method is able to improve the quality of the restored image by up to 4.21 dB over existing methods.

An efficient algorithm for dynamic MRI using low-rank and total variation regularizations

In this paper, we propose an efficient algorithm for dynamic magnetic resonance (MR) image reconstruction. With the total variation (TV) and the nuclear norm (NN) regularization, the TVNNR model can utilize both spatial and temporal redundancy in dynamic MR images. Such prior knowledge can help model dynamic MRI data significantly better than a low-rank or a sparse model alone. However, it is very challenging to efficiently minimize the energy function due to the non-smoothness and non-separability of both TV and NN terms. To address this issue, we propose an efficient algorithm by solving a primal-dual form of the original problem. We theoretically prove that the proposed algorithm achieves a convergence rate of O(1/N) for N iterations. In comparison with state-of-the-art methods, extensive experiments on single-coil and multi-coil dynamic MR data demonstrate the superior performance of the proposed method in terms of both reconstruction accuracy and time complexity.

7T-guided super-resolution of 3T MRI

PURPOSE

High-resolution MR images can depict rich details of brain anatomical structures and show subtle changes in longitudinal data. 7T MRI scanners can acquire MR images with higher resolution and better tissue contrast than the routine 3T MRI scanners. However, 7T MRI scanners are currently more expensive and less available in clinical and research centers. To this end, we propose a method to generate super-resolution 3T MRI that resembles 7T MRI, which is called as 7T-like MR image in this paper.

METHODS

First, we propose a mapping from 3T MRI to 7T MRI space, using regression random forest. The mapped 3T MR images serve as intermediate results with similar appearance as 7T MR images. Second, we predict the final higher resolution 7T-like MR images based on sparse representation, using paired local dictionaries for both the mapped 3T MR images and 7T MR images.

RESULTS

Based on 15 subjects with both 3T and 7T MR images, the predicted 7T-like MR images by our method can best match the ground-truth 7T MR images, compared to other methods. Meanwhile, the experiment on brain tissue segmentation shows that our 7T-like MR images lead to the highest accuracy in the segmentation of WM, GM, and CSF brain tissues, compared to segmentations of 3T MR images as well as the reconstructed 7T-like MR images by other methods.

CONCLUSIONS

We propose a novel method for prediction of high-resolution 7T-like MR images from low-resolution 3T MR images. Our predicted 7T-like MR images demonstrate better spatial resolution compared to 3T MR images, as well as prediction results by other comparison methods. Such high-quality 7T-like MR images could better facilitate disease diagnosis and intervention.

Fast single image super-resolution using estimated low-frequency k-space data in MRI

PURPOSE

Single image super-resolution (SR) is highly desired in many fields but obtaining it is often technically limited in practice. The purpose of this study was to propose a simple, rapid and robust single image SR method in magnetic resonance (MR) imaging (MRI).

METHODS

The idea is based on the mathematical formulation of the intrinsic link in k-space between a given (modulus) low-resolution (LR) image and the desired SR image. The method consists of two steps: 1) estimating the low-frequency k-space data of the desired SR image from a single LR image; 2) reconstructing the SR image using the estimated low-frequency and zero-filled high-frequency k-space data. The method was evaluated on digital phantom images, physical phantom MR images and real brain MR images, and compared with existing SR methods.

RESULTS

The proposed SR method exhibited a good robustness by reaching a clearly higher PSNR (25.77dB) and SSIM (0.991) averaged over different noise levels in comparison with existing edge-guided nonlinear interpolation (EGNI) (PSNR=23.78dB, SSIM=0.983), zero-filling (ZF) (PSNR=24.09dB, SSIM=0.985) and total variation (TV) (PSNR=24.54dB, SSIM=0.987) methods while presenting the same order of computation time as the ZF method but being much faster than the EGNI or TV method. The average PSNR or SSIM over different slice images of the proposed method (PSNR=26.33 dB or SSIM=0.955) was also higher than the EGNI (PSNR=25.07dB or SSIM=0.952), ZF (PSNR=24.97dB or SSIM=0.950) and TV (PSNR=25.70dB or SSIM=0.953) methods, demonstrating its good robustness to variation in anatomical structure of the images. Meanwhile, the proposed method always produced less ringing artifacts than the ZF method, gave a clearer image than the EGNI method, and did not exhibit any blocking effect presented in the TV method. In addition, the proposed method yielded the highest spatial consistency in the inter-slice dimension among the four methods.

CONCLUSIONS

This study proposed a fast, robust and efficient single image SR method with high spatial consistency in the inter-slice dimension for clinical MR images by estimating the low-frequency k-space data of the desired SR image from a single spatial modulus LR image.

Reconstruction of 7T-Like Images From 3T MRI

In the recent MRI scanning, ultra-high-field (7T) MR imaging provides higher resolution and better tissue contrast compared to routine 3T MRI, which may help in more accurate and early brain diseases diagnosis. However, currently, 7T MRI scanners are more expensive and less available at clinical and research centers. These motivate us to propose a method for the reconstruction of images close to the quality of 7T MRI, called 7T-like images, from 3T MRI, to improve the quality in terms of resolution and contrast. By doing so, the post-processing tasks, such as tissue segmentation, can be done more accurately and brain tissues details can be seen with higher resolution and contrast. To do this, we have acquired a unique dataset which includes paired 3T and 7T images scanned from same subjects, and then propose a hierarchical reconstruction based on group sparsity in a novel multi-level Canonical Correlation Analysis (CCA) space, to improve the quality of 3T MR image to be 7T-like MRI. First, overlapping patches are extracted from the input 3T MR image. Then, by extracting the most similar patches from all the aligned 3T and 7T images in the training set, the paired 3T and 7T dictionaries are constructed for each patch. It is worth noting that, for the training, we use pairs of 3T and 7T MR images from each training subject. Then, we propose multi-level CCA to map the paired 3T and 7T patch sets to a common space to increase their correlations. In such space, each input 3T MRI patch is sparsely represented by the 3T dictionary and then the obtained sparse coefficients are used together with the corresponding 7T dictionary to reconstruct the 7T-like patch. Also, to have the structural consistency between adjacent patches, the group sparsity is employed. This reconstruction is performed with changing patch sizes in a hierarchical framework. Experiments have been done using 13 subjects with both 3T and 7T MR images. The results show that our method outperforms previous methods and is able to recover better structural details. Also, to place our proposed method in a medical application context, we evaluated the influence of post-processing methods such as brain tissue segmentation on the reconstructed 7T-like MR images. Results show that our 7T-like images lead to higher accuracy in segmentation of white matter (WM), gray matter (GM), cerebrospinal fluid (CSF), and skull, compared to segmentation of 3T MR images.

Global denoising for 3D MRI

BACKGROUND

Denoising is the primary preprocessing step for subsequent application of MRI. However, most commonly-used patch-based denoising methods are heavily dependent on the degree of patch matching. Due to the large number of voxels in the 3D MRI dataset, the procedure of searching sufficient similarity patches was limited by the empirical compromising between computational efficiency and estimation accuracy, and cannot fulfill the application in multimodal MRI dataset with different SNR and resolutions.

METHODS

In this study, we propose a modified global filtering framework for 3D MRI. For each denoising voxel, the similarity weighting matrix is computed using the reference patch and other patches from the whole dataset. This large weighting matrix is then approximated using the k-means clustering Nyström method to achieve computational viability.

RESULTS

Experiments on both synthetic and in vivo MRI datasets demonstrated that the proposed adaptive Nyström low-rank approximation could achieve competitive estimation compared with exact global filter while reducing the sampling rate by four orders of magnitude. In addition, the corresponding global filter improved patches-based method in both spatial and transform domain.

CONCLUSION

We propose a global denoising framework for 3D MRI which extracts information from the entire dataset to restore each voxel. This large weighting matrix of the global filter is approximated using Nyström low-rank approximation with an adaptive k-means clustering sampling scheme, which significantly reduce the sampling rate as well as the running time. The proposed method is capable of denoising in multimodal MRI dataset and can be used to improve currently used patch-based methods.

Super-Resolution Reconstruction of Diffusion-Weighted Images using 4D Low-Rank and Total Variation

Diffusion-weighted imaging (DWI) provides invaluable information in white matter microstructure and is widely applied in neurological applications. However, DWI is largely limited by its relatively low spatial resolution. In this paper, we propose an image post-processing method, referred to as super-resolution reconstruction, to estimate a high spatial resolution DWI from the input low-resolution DWI, e.g., at a factor of 2. Instead of requiring specially designed DWI acquisition of multiple shifted or orthogonal scans, our method needs only a single DWI scan. To do that, we propose to model both the blurring and downsampling effects in the image degradation process where the low-resolution image is observed from the latent high-resolution image, and recover the latent high-resolution image with the help of two regularizations. The first regularization is 4-dimensional (4D) low-rank, proposed to gather self-similarity information from both the spatial domain and the diffusion domain of 4D DWI. The second regularization is total variation, proposed to depress noise and preserve local structures such as edges in the image recovery process. Extensive experiments were performed on 20 subjects, and results show that the proposed method is able to recover the fine details of white matter structures, and outperform other approaches such as interpolation methods, non-local means based upsampling, and total variation based upsampling.