Compressed Sensing MRI Reconstruction Using a Generative Adversarial Network With a Cyclic Loss

Systematic evaluation of iterative deep neural networks for fast parallel MRI reconstruction with sensitivity-weighted coil combination

PURPOSE

To systematically investigate the influence of various data consistency layers and regularization networks with respect to variations in the training and test data domain, for sensitivity-encoded accelerated parallel MR image reconstruction.

THEORY AND METHODS

Magnetic resonance (MR) image reconstruction is formulated as a learned unrolled optimization scheme with a down-up network as regularization and varying data consistency layers. The proposed networks are compared to other state-of-the-art approaches on the publicly available fastMRI knee and neuro dataset and tested for stability across different training configurations regarding anatomy and number of training samples.

RESULTS

Data consistency layers and expressive regularization networks, such as the proposed down-up networks, form the cornerstone for robust MR image reconstruction. Physics-based reconstruction networks outperform post-processing methods substantially for R = 4 in all cases and for R = 8 when the training and test data are aligned. At R = 8, aligning training and test data is more important than architectural choices.

CONCLUSION

In this work, we study how dataset sizes affect single-anatomy and cross-anatomy training of neural networks for MRI reconstruction. The study provides insights into the robustness, properties, and acceleration limits of state-of-the-art networks, and our proposed down-up networks. These key insights provide essential aspects to successfully translate learning-based MRI reconstruction to clinical practice, where we are confronted with limited datasets and various imaged anatomies.

Robust Single-Probe Quantitative Ultrasonic Imaging System with a Target-Aware Deep Neural Network

OBJECTIVE

The speed of sound (SoS) has great potential as a quantitative imaging biomarker since it is sensitive to pathological changes in tissues. In this paper, a target-aware deep neural (TAD) network reconstructing an SoS image quantitatively from pulse-echo phase-shift maps gathered from a single conventional ultrasound probe is presented.

METHODS

In the proposed TAD network, the reconstruction process is guided by feature maps created from segmented target images for accuracy and contrast. In addition, the feature extraction process utilizes phase difference information instead of direct pulse-echo radio frequency (RF) data for robust image reconstruction against noise in the pulse-echo data.

RESULTS

The TAD network outperforms the fully convolutional network in root mean square error (RMSE), contrast-to-noise ratio (CNR), and structural similarity index (SSIM) in the presence of nearby reflectors. The measured RMSE and CNR are 5.4 m/s and 22 dB, respectively with the tissue attenuation coefficient of 2 dB/cm/MHz, which are 72% and 13 dB improvement over the state of the art design in RMSE and CNR, respectively. In the in-vivo test, the proposed method classifies the tissues in the neck area using SoS with a p-value below 0.025.

CONCLUSION

The proposed TAD network is the most accurate and robust single-probe SoS image reconstruction method reported to date.

SIGNIFICANCE

The accuracy and robustness demonstrated by the proposed SoS imaging method open up the possibilities of wide-spread clinical application of the single-probe SoS imaging system.

A deep cascade of ensemble of dual domain networks with gradient-based T1 assistance and perceptual refinement for fast MRI reconstruction

Deep learning networks have shown promising results in fast magnetic resonance imaging (MRI) reconstruction. In our work, we develop deep networks to further improve the quantitative and the perceptual quality of reconstruction. To begin with, we propose reconsynergynet (RSN), a network that combines the complementary benefits of independently operating on both the image and the Fourier domain. For a single-coil acquisition, we introduce deep cascade RSN (DC-RSN), a cascade of RSN blocks interleaved with data fidelity (DF) units. Secondly, we improve the structure recovery of DC-RSN for T2 weighted Imaging (T2WI) through assistance of T1 weighted imaging (T1WI), a sequence with short acquisition time. T1 assistance is provided to DC-RSN through a gradient of log feature (GOLF) fusion. Furthermore, we propose perceptual refinement network (PRN) to refine the reconstructions for better visual information fidelity (VIF), a metric highly correlated to radiologist's opinion on the image quality. Lastly, for multi-coil acquisition, we propose variable splitting RSN (VS-RSN), a deep cascade of blocks, each block containing RSN, multi-coil DF unit, and a weighted average module. We extensively validate our models DC-RSN and VS-RSN for single-coil and multi-coil acquisitions and report the state-of-the-art performance. We obtain a SSIM of 0.768, 0.923, and 0.878 for knee single-coil-4x, multi-coil-4x, and multi-coil-8x in fastMRI, respectively. We also conduct experiments to demonstrate the efficacy of GOLF based T1 assistance and PRN.

Adaptive convolutional neural networks for accelerating magnetic resonance imaging via k-space data interpolation

Deep learning in k-space has demonstrated great potential for image reconstruction from undersampled k-space data in fast magnetic resonance imaging (MRI). However, existing deep learning-based image reconstruction methods typically apply weight-sharing convolutional neural networks (CNNs) to k-space data without taking into consideration the k-space data's spatial frequency properties, leading to ineffective learning of the image reconstruction models. Moreover, complementary information of spatially adjacent slices is often ignored in existing deep learning methods. To overcome such limitations, we have developed a deep learning algorithm, referred to as adaptive convolutional neural networks for k-space data interpolation (ACNN-k-Space), which adopts a residual Encoder-Decoder network architecture to interpolate the undersampled k-space data by integrating spatially contiguous slices as multi-channel input, along with k-space data from multiple coils if available. The network is enhanced by self-attention layers to adaptively focus on k-space data at different spatial frequencies and channels. We have evaluated our method on two public datasets and compared it with state-of-the-art existing methods. Ablation studies and experimental results demonstrate that our method effectively reconstructs images from undersampled k-space data and achieves significantly better image reconstruction performance than current state-of-the-art techniques. Source code of the method is available at https://gitlab.com/qgpmztmf/acnn-k-space.

Machine learning applications to neuroimaging for glioma detection and classification: An artificial intelligence augmented systematic review

Glioma is the most common primary intraparenchymal tumor of the brain and the 5-year survival rate of high-grade glioma is poor. Magnetic resonance imaging (MRI) is essential for detecting, characterizing and monitoring brain tumors but definitive diagnosis still relies on surgical pathology. Machine learning has been applied to the analysis of MRI data in glioma research and has the potential to change clinical practice and improve patient outcomes. This systematic review synthesizes and analyzes the current state of machine learning applications to glioma MRI data and explores the use of machine learning for systematic review automation. Various datapoints were extracted from the 153 studies that met inclusion criteria and analyzed. Natural language processing (NLP) analysis involved keyword extraction, topic modeling and document classification. Machine learning has been applied to tumor grading and diagnosis, tumor segmentation, non-invasive genomic biomarker identification, detection of progression and patient survival prediction. Model performance was generally strong (AUC = 0.87 ± 0.09; sensitivity = 0.87 ± 0.10; specificity = 0.0.86 ± 0.10; precision = 0.88 ± 0.11). Convolutional neural network, support vector machine and random forest algorithms were top performers. Deep learning document classifiers yielded acceptable performance (mean 5-fold cross-validation AUC = 0.71). Machine learning tools and data resources were synthesized and summarized to facilitate future research. Machine learning has been widely applied to the processing of MRI data in glioma research and has demonstrated substantial utility. NLP and transfer learning resources enabled the successful development of a replicable method for automating the systematic review article screening process, which has potential for shortening the time from discovery to clinical application in medicine.

FusionNet: A Deep Fully Residual Convolutional Neural Network for Image Segmentation in Connectomics

Cellular-resolution connectomics is an ambitious research direction with the goal of generating comprehensive brain connectivity maps using high-throughput, nano-scale electron microscopy. One of the main challenges in connectomics research is developing scalable image analysis algorithms that require minimal user intervention. Deep learning has provided exceptional performance in image classification tasks in computer vision, leading to a recent explosion in popularity. Similarly, its application to connectomic analyses holds great promise. Here, we introduce a deep neural network architecture, FusionNet, with a focus on its application to accomplish automatic segmentation of neuronal structures in connectomics data. FusionNet combines recent advances in machine learning, such as semantic segmentation and residual neural networks, with summation-based skip connections. This results in a much deeper network architecture and improves segmentation accuracy. We demonstrate the performance of the proposed method by comparing it with several other popular electron microscopy segmentation methods. We further illustrate its flexibility through segmentation results for two different tasks: cell membrane segmentation and cell nucleus segmentation.

Narrative review of generative adversarial networks in medical and molecular imaging

Recent years have witnessed a rapidly expanding use of artificial intelligence and machine learning in medical imaging. Generative adversarial networks (GANs) are techniques to synthesize images based on artificial neural networks and deep learning. In addition to the flexibility and versatility inherent in deep learning on which the GANs are based, the potential problem-solving ability of the GANs has attracted attention and is being vigorously studied in the medical and molecular imaging fields. Here this narrative review provides a comprehensive overview for GANs and discuss their usefulness in medical and molecular imaging on the following topics: (I) data augmentation to increase training data for AI-based computer-aided diagnosis as a solution for the data-hungry nature of such training sets; (II) modality conversion to complement the shortcomings of a single modality that reflects certain physical measurement principles, such as from magnetic resonance (MR) to computed tomography (CT) images or vice versa; (III) de-noising to realize less injection and/or radiation dose for nuclear medicine and CT; (IV) image reconstruction for shortening MR acquisition time while maintaining high image quality; (V) super-resolution to produce a high-resolution image from low-resolution one; (VI) domain adaptation which utilizes knowledge such as supervised labels and annotations from a source domain to the target domain with no or insufficient knowledge; and (VII) image generation with disease severity and radiogenomics. GANs are promising tools for medical and molecular imaging. The progress of model architectures and their applications should continue to be noteworthy.

Compressed medical imaging based on average sparsity model and reweighted analysis of multiple basis pursuit

In medical imaging and applications, efficient image sampling and transfer are some of the key fields of research. The compressed sensing (CS) theory has shown that such compression can be performed during the data retrieval process and that the uncompressed image can be retrieved using a computationally flexible optimization method. The objective of this study is to propose compressed medical imaging for a different type of medical images, based on the combination of the average sparsity model and reweighted analysis of multiple basis pursuit (M-BP) reconstruction methods, referred to as multiple basis reweighted analysis (M-BRA). The proposed algorithm includes the joint multiple sparsity averaging to improves the signal sparsity in M-BP. In this study, four types of medical images are opted to fill the gap of lacking a detailed analysis of M-BRA in medical images. The medical dataset consists of magnetic resonance imaging (MRI) data, computed tomography (CT) data, colonoscopy data, and endoscopy data. Employing the proposed approach, a signal-to-noise ratio (SNR) of 30 dB was achieved for MRI data on a sampling ratio of M/N=0.3. SNR of 34, 30, and 34 dB are corresponding to CT, colonoscopy, and endoscopy data on the same sampling ratio of M/N=0.15. The proposed M-BRA performance indicates the potential for compressed medical imaging analysis with high reconstruction image quality.

SPECTnet: a deep learning neural network for SPECT image reconstruction

Background

Single photon emission computed tomography (SPECT) is an important functional tool for clinical diagnosis and scientific research of brain disorders, but suffers from limited spatial resolution and high noise due to hardware design and imaging physics. The present study is to develop a deep learning technique for SPECT image reconstruction that directly converts raw projection data to image with high resolution and low noise, while an efficient training method specifically applicable to medical image reconstruction is presented.

Methods

A custom software was developed to generate 20,000 2-D brain phantoms, of which 16,000 were used to train the neural network, 2,000 for validation, and the final 2,000 for testing. To reduce development difficulty, a two-step training strategy for network design was adopted. We first compressed full-size activity image (128×128 pixels) to a one-D vector consisting of 256×1 pixels, accomplished by an autoencoder (AE) consisting of an encoder and a decoder. The vector is a good representation of the full-size image in a lower-dimensional space and was used as a compact label to develop the second network that maps between the projection-data domain and the vector domain. Since the label had 256 pixels only, the second network was compact and easy to converge. The second network, when successfully developed, was connected to the decoder (a portion of AE) to decompress the vector to a regular 128×128 image. Therefore, a complex network was essentially divided into two compact neural networks trained separately in sequence but eventually connectable.

Results

A total of 2,000 test examples, a synthetic brain phantom, and de-identified patient data were used to validate SPECTnet. Results obtained from SPECTnet were compared with those obtained from our clinic OS-EM method. Images with lower noise and more accurate information in the uptake areas were obtained by SPECTnet.

Conclusions

The challenge of developing a complex deep neural network is reduced by training two separate compact connectable networks. The combination of the two networks forms the full version of SPECTnet. Results show that the developed neural network can produce more accurate SPECT images.

Generative Adversarial Networks: A Primer for Radiologists

Artificial intelligence techniques involving the use of artificial neural networks-that is, deep learning techniques-are expected to have a major effect on radiology. Some of the most exciting applications of deep learning in radiology make use of generative adversarial networks (GANs). GANs consist of two artificial neural networks that are jointly optimized but with opposing goals. One neural network, the generator, aims to synthesize images that cannot be distinguished from real images. The second neural network, the discriminator, aims to distinguish these synthetic images from real images. These deep learning models allow, among other applications, the synthesis of new images, acceleration of image acquisitions, reduction of imaging artifacts, efficient and accurate conversion between medical images acquired with different modalities, and identification of abnormalities depicted on images. The authors provide an introduction to GANs and adversarial deep learning methods. In addition, the different ways in which GANs can be used for image synthesis and image-to-image translation tasks, as well as the principles underlying conditional GANs and cycle-consistent GANs, are described. Illustrated examples of GAN applications in radiologic image analysis for different imaging modalities and different tasks are provided. The clinical potential of GANs, future clinical GAN applications, and potential pitfalls and caveats that radiologists should be aware of also are discussed in this review. The online slide presentation from the RSNA Annual Meeting is available for this article. ©RSNA, 2021.

Deep Generative Adversarial Networks: Applications in Musculoskeletal Imaging

In recent years, deep learning techniques have been applied in musculoskeletal radiology to increase the diagnostic potential of acquired images. Generative adversarial networks (GANs), which are deep neural networks that can generate or transform images, have the potential to aid in faster imaging by generating images with a high level of realism across multiple contrast and modalities from existing imaging protocols. This review introduces the key architectures of GANs as well as their technical background and challenges. Key research trends are highlighted, including: (a) reconstruction of high-resolution MRI; (b) image synthesis with different modalities and contrasts; (c) image enhancement that efficiently preserves high-frequency information suitable for human interpretation; (d) pixel-level segmentation with annotation sharing between domains; and (e) applications to different musculoskeletal anatomies. In addition, an overview is provided of the key issues wherein clinical applicability is challenging to capture with conventional performance metrics and expert evaluation. When clinically validated, GANs have the potential to improve musculoskeletal imaging. Keywords: Adults and Pediatrics, Computer Aided Diagnosis (CAD), Computer Applications-General (Informatics), Informatics, Skeletal-Appendicular, Skeletal-Axial, Soft Tissues/Skin © RSNA, 2021.

Feasibility of compressed sensing technique for isotropic dynamic contrast-enhanced liver magnetic resonance imaging

PURPOSE

To investigate whether an isotropic T1-weighted gradient echo (T1-GRE) sequence using a compressed sensing (CS) technique during liver magnetic resonance imaging (MRI) can improve the image quality compared to that using a standard parallel imaging (PI) technique in patients with hepatocellular carcinoma (HCC).

METHODS

Forty-nine patients with single pathologically confirmed HCC were included in the prospective study, who underwent a 3.0 T MRI including the two T1-GRE sequences (CS and PI). Qualitative analysis including the relative contrast (RC) of liver-to-lesion, liver-to-portal vein and liver-to-hepatic vein on pre-contrast and postcontrast (delayed phase) images were calculated. Respiratory motion artifact, gastrointestinal motion artifact and overall image quality were scored by using a 4-point scale.

RESULTS

RC of liver-to-lesion, liver-to-portal vein and liver-to-hepatic vein measured on both pre-contrast and postcontrast phase images were significantly higher for CS than for PI. The scores of overall image quality was comparable between PI and CS (3.98 ± 0.10vs 3.96 ± 0.13, P = 0.083 for pre-contrast; 3.96 ± 0.16 vs 3.93 ± 0.17, P = 0.132 for postcontrast, respectively). The scores of gastrointestinal motion artifact was significantly higher for PI than for CS (3.92 ± 0.21 vs 3.69 ± 0.33 for pre-contrast; 3.86 ± 0.21 vs 3.59 ± 0.30 for postcontrast, P < 0.001 for both). The scores of respiratory motion artifact was significantly higher for PI only in pre-contrast sequence (3.97±0.11 vs 3.89 ± 0.22, P = 0.002 for pre-contrast; 3.95 ± 0.18 vs 3.90 ± 0.22, P = 0.083 for postcontrast, respectively).

CONCLUSIONS

Compared to the standard PI sequence, the CS technique can provide greater contrast in displaying HCCs and hepatic vessels in MRI without compromise of overall image quality.

Two-stage deep learning for accelerated 3D time-of-flight MRA without matched training data

Time-of-flight magnetic resonance angiography (TOF-MRA) is one of the most widely used non-contrast MR imaging methods to visualize blood vessels, but due to the 3-D volume acquisition highly accelerated acquisition is necessary. Accordingly, high quality reconstruction from undersampled TOF-MRA is an important research topic for deep learning. However, most existing deep learning works require matched reference data for supervised training, which are often difficult to obtain. By extending the recent theoretical understanding of cycleGAN from the optimal transport theory, here we propose a novel two-stage unsupervised deep learning approach, which is composed of the multi-coil reconstruction network along the coronal plane followed by a multi-planar refinement network along the axial plane. Specifically, the first network is trained in the square-root of sum of squares (SSoS) domain to achieve high quality parallel image reconstruction, whereas the second refinement network is designed to efficiently learn the characteristics of highly-activated blood flow using double-headed projection discriminator. Extensive experiments demonstrate that the proposed learning process without matched reference exceeds performance of state-of-the-art compressed sensing (CS)-based method and provides comparable or even better results than supervised learning approaches.

A dual-task dual-domain model for blind MRI reconstruction

MRI reconstruction is the key technology to accelerate MR acquisition. Recent cascade models have gained satisfactory results, however, they deeply rely on the known sample mask, which we call it mask prior. To restore the MR image without mask prior, we designed an auxiliary network to estimate the mask from sampled k-space data. Experimentally, the sample mask can be completely estimated by the proposed network and be used to input to the cascade models. Moreover, we rethink the MRI reconstruction model as a k-space inpainting task. A dual-domain cascade network, which utilized partial convolutional layers to inpaint features in k-space, was presented to restore the MR image. Without the mask prior, our blind reconstruction model demonstrates the best reconstruction ability in both 4x acceleration and 8x acceleration.

Artificial Intelligence for MR Image Reconstruction: An Overview for Clinicians

Artificial intelligence (AI) shows tremendous promise in the field of medical imaging, with recent breakthroughs applying deep-learning models for data acquisition, classification problems, segmentation, image synthesis, and image reconstruction. With an eye towards clinical applications, we summarize the active field of deep-learning-based MR image reconstruction. We review the basic concepts of how deep-learning algorithms aid in the transformation of raw k-space data to image data, and specifically examine accelerated imaging and artifact suppression. Recent efforts in these areas show that deep-learning-based algorithms can match and, in some cases, eclipse conventional reconstruction methods in terms of image quality and computational efficiency across a host of clinical imaging applications, including musculoskeletal, abdominal, cardiac, and brain imaging. This article is an introductory overview aimed at clinical radiologists with no experience in deep-learning-based MR image reconstruction and should enable them to understand the basic concepts and current clinical applications of this rapidly growing area of research across multiple organ systems.

SR-Net: A sequence offset fusion net and refine net for undersampled multislice MR image reconstruction

BACKGROUND AND OBJECTIVE

The study of deep learning-based fast magnetic resonance imaging (MRI) reconstruction methods has become popular in recent years. However, there is still a challenge when MRI results undersample large acceleration factors. The objective of this study was to improve the reconstruction quality of undersampled MR images by exploring data redundancy among slices.

METHODS

There are two aspects of redundancy in multislice MR images including correlations inside a single slice and correlations among slices. Thus, we built two subnets for the two kinds of redundancy. For correlations among slices, we built a bidirectional recurrent convolutional neural network, named Sequence Offset Fusion Net (S-Net). In S-Net, we used a deformable convolution module to construct a neighbor slice feature extractor. For the correlation inside a single slice, we built a Refine Net (R-Net), which has 5 layers of 2D convolutions. In addition, we used a data consistency (DC) operation to maintain data fidelity in k-space. Finally, we treated the reconstruction task as a dealiasing problem in the image domain, and S-Net and R-Net are applied alternately and iteratively to generate the final reconstructions.

RESULTS

The proposed algorithm was evaluated using two online public MRI datasets. Compared with several state-of-the-art methods, the proposed method achieved better reconstruction results in terms of dealiasing and restoring tissue structure. Moreover, with over 14 slices per second reconstruction speed on 256x256 pixel images, the proposed method can meet the need for real-time processing.

CONCLUSION

With spatial correlation among slices as additional prior information, the proposed method dramatically improves the reconstruction quality of undersampled MR images.

Prediction and analysis of multiple protein lysine modified sites based on conditional wasserstein generative adversarial networks

BACKGROUND

Protein post-translational modification (PTM) is a key issue to investigate the mechanism of protein's function. With the rapid development of proteomics technology, a large amount of protein sequence data has been generated, which highlights the importance of the in-depth study and analysis of PTMs in proteins.

METHOD

We proposed a new multi-classification machine learning pipeline MultiLyGAN to identity seven types of lysine modified sites. Using eight different sequential and five structural construction methods, 1497 valid features were remained after the filtering by Pearson correlation coefficient. To solve the data imbalance problem, Conditional Generative Adversarial Network (CGAN) and Conditional Wasserstein Generative Adversarial Network (CWGAN), two influential deep generative methods were leveraged and compared to generate new samples for the types with fewer samples. Finally, random forest algorithm was utilized to predict seven categories.

RESULTS

In the tenfold cross-validation, accuracy (Acc) and Matthews correlation coefficient (MCC) were 0.8589 and 0.8376, respectively. In the independent test, Acc and MCC were 0.8549 and 0.8330, respectively. The results indicated that CWGAN better solved the existing data imbalance and stabilized the training error. Alternatively, an accumulated feature importance analysis reported that CKSAAP, PWM and structural features were the three most important feature-encoding schemes. MultiLyGAN can be found at https://github.com/Lab-Xu/MultiLyGAN .

CONCLUSIONS

The CWGAN greatly improved the predictive performance in all experiments. Features derived from CKSAAP, PWM and structure schemes are the most informative and had the greatest contribution to the prediction of PTM.

Enhancing magnetic resonance imaging-driven Alzheimer's disease classification performance using generative adversarial learning

BACKGROUND

Generative adversarial networks (GAN) can produce images of improved quality but their ability to augment image-based classification is not fully explored. We evaluated if a modified GAN can learn from magnetic resonance imaging (MRI) scans of multiple magnetic field strengths to enhance Alzheimer's disease (AD) classification performance.

METHODS

T1-weighted brain MRI scans from 151 participants of the Alzheimer's Disease Neuroimaging Initiative (ADNI), who underwent both 1.5-Tesla (1.5-T) and 3-Tesla imaging at the same time were selected to construct a GAN model. This model was trained along with a three-dimensional fully convolutional network (FCN) using the generated images (3T*) as inputs to predict AD status. Quality of the generated images was evaluated using signal to noise ratio (SNR), Blind/Referenceless Image Spatial Quality Evaluator (BRISQUE) and Natural Image Quality Evaluator (NIQE). Cases from the Australian Imaging, Biomarker & Lifestyle Flagship Study of Ageing (AIBL, n = 107) and the National Alzheimer's Coordinating Center (NACC, n = 565) were used for model validation.

RESULTS

The 3T*-based FCN classifier performed better than the FCN model trained using the 1.5-T scans. Specifically, the mean area under curve increased from 0.907 to 0.932, from 0.934 to 0.940, and from 0.870 to 0.907 on the ADNI test, AIBL, and NACC datasets, respectively. Additionally, we found that the mean quality of the generated (3T*) images was consistently higher than the 1.5-T images, as measured using SNR, BRISQUE, and NIQE on the validation datasets.

CONCLUSION

This study demonstrates a proof of principle that GAN frameworks can be constructed to augment AD classification performance and improve image quality.

Reference-Driven Undersampled MR Image Reconstruction Using Wavelet Sparsity-Constrained Deep Image Prior

Deep learning has shown potential in significantly improving performance for undersampled magnetic resonance (MR) image reconstruction. However, one challenge for the application of deep learning to clinical scenarios is the requirement of large, high-quality patient-based datasets for network training. In this paper, we propose a novel deep learning-based method for undersampled MR image reconstruction that does not require pre-training procedure and pre-training datasets. The proposed reference-driven method using wavelet sparsity-constrained deep image prior (RWS-DIP) is based on the DIP framework and thereby reduces the dependence on datasets. Moreover, RWS-DIP explores and introduces structure and sparsity priors into network learning to improve the efficiency of learning. By employing a high-resolution reference image as the network input, RWS-DIP incorporates structural information into network. RWS-DIP also uses the wavelet sparsity to further enrich the implicit regularization of traditional DIP by formulating the training of network parameters as a constrained optimization problem, which is solved using the alternating direction method of multipliers (ADMM) algorithm. Experiments on in vivo MR scans have demonstrated that the RWS-DIP method can reconstruct MR images more accurately and preserve features and textures from undersampled k-space measurements.

High quality and fast compressed sensing MRI reconstruction via edge-enhanced dual discriminator generative adversarial network

Generative adversarial networks (GAN) are widely used for fast compressed sensing magnetic resonance imaging (CSMRI) reconstruction. However, most existing methods are difficult to make an effective trade-off between abstract global high-level features and edge features. It easily causes problems, such as significant remaining aliasing artifacts and clearly over-smoothed reconstruction details. To tackle these issues, we propose a novel edge-enhanced dual discriminator generative adversarial network architecture called EDDGAN for CSMRI reconstruction with high quality. In this model, we extract effective edge features by fusing edge information from different depths. Then, leveraging the relationship between abstract global high-level features and edge features, a three-player game is introduced to control the hallucination of details and stabilize the training process. The resulting EDDGAN can offer more focus on edge restoration and de-aliasing. Extensive experimental results demonstrate that our method consistently outperforms state-of-the-art methods and obtains reconstructed images with rich edge details. In addition, our method also shows remarkable generalization, and its time consumption for each 256 × 256 image reconstruction is approximately 8.39 ms.

PIC-GAN: A Parallel Imaging Coupled Generative Adversarial Network for Accelerated Multi-Channel MRI Reconstruction

In this study, we proposed a model combing parallel imaging (PI) with generative adversarial network (GAN) architecture (PIC-GAN) for accelerated multi-channel magnetic resonance imaging (MRI) reconstruction. This model integrated data fidelity and regularization terms into the generator to benefit from multi-coils information and provide an "end-to-end" reconstruction. Besides, to better preserve image details during reconstruction, we combined the adversarial loss with pixel-wise loss in both image and frequency domains. The proposed PIC-GAN framework was evaluated on abdominal and knee MRI images using 2, 4 and 6-fold accelerations with different undersampling patterns. The performance of the PIC-GAN was compared to the sparsity-based parallel imaging (L1-ESPIRiT), the variational network (VN), and conventional GAN with single-channel images as input (zero-filled (ZF)-GAN). Experimental results show that our PIC-GAN can effectively reconstruct multi-channel MR images at a low noise level and improved structure similarity of the reconstructed images. PIC-GAN has yielded the lowest Normalized Mean Square Error (in ×10-5) (PIC-GAN: 0.58 ± 0.37, ZF-GAN: 1.93 ± 1.41, VN: 1.87 ± 1.28, L1-ESPIRiT: 2.49 ± 1.04 for abdominal MRI data and PIC-GAN: 0.80 ± 0.26, ZF-GAN: 0.93 ± 0.29, VN:1.18 ± 0.31, L1-ESPIRiT: 1.28 ± 0.24 for knee MRI data) and the highest Peak Signal to Noise Ratio (PIC-GAN: 34.43 ± 1.92, ZF-GAN: 31.45 ± 4.0, VN: 29.26 ± 2.98, L1-ESPIRiT: 25.40 ± 1.88 for abdominal MRI data and PIC-GAN: 34.10 ± 1.09, ZF-GAN: 31.47 ± 1.05, VN: 30.01 ± 1.01, L1-ESPIRiT: 28.01 ± 0.98 for knee MRI data) compared to ZF-GAN, VN and L1-ESPIRiT with an under-sampling factor of 6. The proposed PIC-GAN framework has shown superior reconstruction performance in terms of reducing aliasing artifacts and restoring tissue structures as compared to other conventional and state-of-the-art reconstruction methods.

A review on medical imaging synthesis using deep learning and its clinical applications

This paper reviewed the deep learning-based studies for medical imaging synthesis and its clinical application. Specifically, we summarized the recent developments of deep learning-based methods in inter- and intra-modality image synthesis by listing and highlighting the proposed methods, study designs, and reported performances with related clinical applications on representative studies. The challenges among the reviewed studies were then summarized with discussion.

Wasserstein GANs for MR Imaging: From Paired to Unpaired Training

Lack of ground-truth MR images impedes the common supervised training of neural networks for image reconstruction. To cope with this challenge, this article leverages unpaired adversarial training for reconstruction networks, where the inputs are undersampled k-space and naively reconstructed images from one dataset, and the labels are high-quality images from another dataset. The reconstruction networks consist of a generator which suppresses the input image artifacts, and a discriminator using a pool of (unpaired) labels to adjust the reconstruction quality. The generator is an unrolled neural network - a cascade of convolutional and data consistency layers. The discriminator is also a multilayer CNN that plays the role of a critic scoring the quality of reconstructed images based on the Wasserstein distance. Our experiments with knee MRI datasets demonstrate that the proposed unpaired training enables diagnostic-quality reconstruction when high-quality image labels are not available for the input types of interest, or when the amount of labels is small. In addition, our adversarial training scheme can achieve better image quality (as rated by expert radiologists) compared with the paired training schemes with pixel-wise loss.

Uncertainty Quantification in Deep MRI Reconstruction

Reliable MRI is crucial for accurate interpretation in therapeutic and diagnostic tasks. However, undersampling during MRI acquisition as well as the overparameterized and non-transparent nature of deep learning (DL) leaves substantial uncertainty about the accuracy of DL reconstruction. With this in mind, this study aims to quantify the uncertainty in image recovery with DL models. To this end, we first leverage variational autoencoders (VAEs) to develop a probabilistic reconstruction scheme that maps out (low-quality) short scans with aliasing artifacts to the diagnostic-quality ones. The VAE encodes the acquisition uncertainty in a latent code and naturally offers a posterior of the image from which one can generate pixel variance maps using Monte-Carlo sampling. Accurately predicting risk requires knowledge of the bias as well, for which we leverage Stein's Unbiased Risk Estimator (SURE) as a proxy for mean-squared-error (MSE). A range of empirical experiments is performed for Knee MRI reconstruction under different training losses (adversarial and pixel-wise) and unrolled recurrent network architectures. Our key observations indicate that: 1) adversarial losses introduce more uncertainty; and 2) recurrent unrolled nets reduce the prediction uncertainty and risk.

Spatial orthogonal attention generative adversarial network for MRI reconstruction

PURPOSE

Recent studies have witnessed that self-attention modules can better solve the vision understanding problems by capturing long-range dependencies. However, there are very few works designing a lightweight self-attention module to improve the quality of MRI reconstruction. Furthermore, it can be observed that several important self-attention modules (e.g., the non-local block) cause high computational complexity and need a huge number of GPU memory when the size of the input feature is large. The purpose of this study is to design a lightweight yet effective spatial orthogonal attention module (SOAM) to capture long-range dependencies, and develop a novel spatial orthogonal attention generative adversarial network, termed as SOGAN, to achieve more accurate MRI reconstruction.

METHODS

We first develop a lightweight SOAM, which can generate two small attention maps to effectively aggregate the long-range contextual information in vertical and horizontal directions, respectively. Then, we embed the proposed SOAMs into the concatenated convolutional autoencoders to form the generator of the proposed SOGAN.

RESULTS

The experimental results demonstrate that the proposed SOAMs improve the quality of the reconstructed MR images effectively by capturing long-range dependencies. Besides, compared with state-of-the-art deep learning-based CS-MRI methods, the proposed SOGAN reconstructs MR images more accurately, but with fewer model parameters.

CONCLUSIONS

The proposed SOAM is a lightweight yet effective self-attention module to capture long-range dependencies, thus, can improve the quality of MRI reconstruction to a large extent. Besides, with the help of SOAMs, the proposed SOGAN outperforms the state-of-the-art deep learning-based CS-MRI methods.

Deep learning for brain disorders: from data processing to disease treatment

In order to reach precision medicine and improve patients' quality of life, machine learning is increasingly used in medicine. Brain disorders are often complex and heterogeneous, and several modalities such as demographic, clinical, imaging, genetics and environmental data have been studied to improve their understanding. Deep learning, a subpart of machine learning, provides complex algorithms that can learn from such various data. It has become state of the art in numerous fields, including computer vision and natural language processing, and is also growingly applied in medicine. In this article, we review the use of deep learning for brain disorders. More specifically, we identify the main applications, the concerned disorders and the types of architectures and data used. Finally, we provide guidelines to bridge the gap between research studies and clinical routine.

A Deep Framework Assembling Principled Modules for CS-MRI: Unrolling Perspective, Convergence Behaviors, and Practical Modeling

Compressed Sensing Magnetic Resonance Imaging (CS-MRI) significantly accelerates MR acquisition at a sampling rate much lower than the Nyquist criterion. A major challenge for CS-MRI lies in solving the severely ill-posed inverse problem to reconstruct aliasing-free MR images from the sparse k -space data. Conventional methods typically optimize an energy function, producing restoration of high quality, but their iterative numerical solvers unavoidably bring extremely large time consumption. Recent deep techniques provide fast restoration by either learning direct prediction to final reconstruction or plugging learned modules into the energy optimizer. Nevertheless, these data-driven predictors cannot guarantee the reconstruction following principled constraints underlying the domain knowledge so that the reliability of their reconstruction process is questionable. In this paper, we propose a deep framework assembling principled modules for CS-MRI that fuses learning strategy with the iterative solver of a conventional reconstruction energy. This framework embeds an optimal condition checking mechanism, fostering efficient and reliable reconstruction. We also apply the framework to three practical tasks, i.e., complex-valued data reconstruction, parallel imaging and reconstruction with Rician noise. Extensive experiments on both benchmark and manufacturer-testing images demonstrate that the proposed method reliably converges to the optimal solution more efficiently and accurately than the state-of-the-art in various scenarios.

High-performance rapid MR parameter mapping using model-based deep adversarial learning

PURPOSE

To develop and evaluate a deep adversarial learning-based image reconstruction approach for rapid and efficient MR parameter mapping.

METHODS

The proposed method provides an image reconstruction framework by combining the end-to-end convolutional neural network (CNN) mapping, adversarial learning, and MR physical models. The CNN performs direct image-to-parameter mapping by transforming a series of undersampled images directly into MR parameter maps. Adversarial learning is used to improve image sharpness and enable better texture restoration during the image-to-parameter conversion. An additional pathway concerning the MR signal model is added between the estimated parameter maps and undersampled k-space data to ensure the data consistency during network training. The proposed framework was evaluated on T2 mapping of the brain and the knee at an acceleration rate R = 8 and was compared with other state-of-the-art reconstruction methods. Global and regional quantitative assessments were performed to demonstrate the reconstruction performance of the proposed method.

RESULTS

The proposed adversarial learning approach achieved accurate T2 mapping up to R = 8 in brain and knee joint image datasets. Compared to conventional reconstruction approaches that exploit image sparsity and low-rankness, the proposed method yielded lower errors and higher similarity to the reference and better image sharpness in the T2 estimation. The quantitative metrics were normalized root mean square error of 3.6% for brain and 7.3% for knee, structural similarity index of 85.1% for brain and 83.2% for knee, and tenengrad measures of 9.2% for brain and 10.1% for the knee. The adversarial approach also achieved better performance for maintaining greater image texture and sharpness in comparison to the CNN approach without adversarial learning.

CONCLUSION

The proposed framework by incorporating the efficient end-to-end CNN mapping, adversarial learning, and physical model enforced data consistency is a promising approach for rapid and efficient reconstruction of quantitative MR parameters.

Self-supervised learning of physics-guided reconstruction neural networks without fully sampled reference data

PURPOSE

To develop a strategy for training a physics-guided MRI reconstruction neural network without a database of fully sampled data sets.

METHODS

Self-supervised learning via data undersampling (SSDU) for physics-guided deep learning reconstruction partitions available measurements into two disjoint sets, one of which is used in the data consistency (DC) units in the unrolled network and the other is used to define the loss for training. The proposed training without fully sampled data is compared with fully supervised training with ground-truth data, as well as conventional compressed-sensing and parallel imaging methods using the publicly available fastMRI knee database. The same physics-guided neural network is used for both proposed SSDU and supervised training. The SSDU training is also applied to prospectively two-fold accelerated high-resolution brain data sets at different acceleration rates, and compared with parallel imaging.

RESULTS

Results on five different knee sequences at an acceleration rate of 4 shows that the proposed self-supervised approach performs closely with supervised learning, while significantly outperforming conventional compressed-sensing and parallel imaging, as characterized by quantitative metrics and a clinical reader study. The results on prospectively subsampled brain data sets, in which supervised learning cannot be used due to lack of ground-truth reference, show that the proposed self-supervised approach successfully performs reconstruction at high acceleration rates (4, 6, and 8). Image readings indicate improved visual reconstruction quality with the proposed approach compared with parallel imaging at acquisition acceleration.

CONCLUSION

The proposed SSDU approach allows training of physics-guided deep learning MRI reconstruction without fully sampled data, while achieving comparable results with supervised deep learning MRI trained on fully sampled data.

Parallel imaging with a combination of sensitivity encoding and generative adversarial networks

BACKGROUND

Magnetic resonance imaging (MRI) has the limitation of low imaging speed. Acceleration methods using under-sampled k-space data have been widely exploited to improve data acquisition without reducing the image quality. Sensitivity encoding (SENSE) is the most commonly used method for multi-channel imaging. However, SENSE has the drawback of severe g-factor artifacts when the under-sampling factor is high. This paper applies generative adversarial networks (GAN) to remove g-factor artifacts from SENSE reconstructions.

METHODS

Our method was evaluated on a public knee database containing 20 healthy participants. We compared our method with conventional GAN using zero-filled (ZF) images as input. Structural similarity (SSIM), peak signal to noise ratio (PSNR), and normalized mean square error (NMSE) were calculated for the assessment of image quality. A paired student's t-test was conducted to compare the image quality metrics between the different methods. Statistical significance was considered at P<0.01.

RESULTS

The proposed method outperformed SENSE, variational network (VN), and ZF + GAN methods in terms of SSIM (SENSE + GAN: 0.81±0.06, SENSE: 0.40±0.07, VN: 0.79±0.06, ZF + GAN: 0.77±0.06), PSNR (SENSE + GAN: 31.90±1.66, SENSE: 22.70±1.99, VN: 31.35±2.01, ZF + GAN: 29.95±1.59), and NMSE (×10-7) (SENSE + GAN: 0.95±0.34, SENSE: 4.81±1.33, VN: 0.97±0.30, ZF + GAN: 1.60±0.84) with an under-sampling factor of up to 6-fold.

CONCLUSIONS

This study demonstrated the feasibility of using GAN to improve the performance of SENSE reconstruction. The improvement of reconstruction is more obvious for higher under-sampling rates, which shows great potential for many clinical applications.

Multimodal MRI synthesis using unified generative adversarial networks

PURPOSE

Complementary information obtained from multiple contrasts of tissue facilitates physicians assessing, diagnosing and planning treatment of a variety of diseases. However, acquiring multiple contrasts magnetic resonance images (MRI) for every patient using multiple pulse sequences is time-consuming and expensive, where, medical image synthesis has been demonstrated as an effective alternative. The purpose of this study is to develop a unified framework for multimodal MR image synthesis.

METHODS

A unified generative adversarial network consisting of only a single generator and a single discriminator was developed to learn the mappings among images of four different modalities. The generator took an image and its modality label as inputs and learned to synthesize the image in the target modality, while the discriminator was trained to distinguish between real and synthesized images and classify them to their corresponding modalities. The network was trained and tested using multimodal brain MRI consisting of four different contrasts which are T1-weighted (T1), T1-weighted and contrast-enhanced (T1c), T2-weighted (T2), and fluid-attenuated inversion recovery (Flair). Quantitative assessments of our proposed method were made through computing normalized mean absolute error (NMAE), peak signal-to-noise ratio (PSNR), structural similarity index measurement (SSIM), visual information fidelity (VIF), and naturalness image quality evaluator (NIQE).

RESULTS

The proposed model was trained and tested on a cohort of 274 glioma patients with well-aligned multi-types of MRI scans. After the model was trained, tests were conducted by using each of T1, T1c, T2, Flair as a single input modality to generate its respective rest modalities. Our proposed method shows high accuracy and robustness for image synthesis with arbitrary MRI modality that is available in the database as input. For example, with T1 as input modality, the NMAEs for the generated T1c, T2, Flair respectively are 0.034 ± 0.005, 0.041 ± 0.006, and 0.041 ± 0.006, the PSNRs respectively are 32.353 ± 2.525 dB, 30.016 ± 2.577 dB, and 29.091 ± 2.795 dB, the SSIMs are 0.974 ± 0.059, 0.969 ± 0.059, and 0.959 ± 0.059, the VIF are 0.750 ± 0.087, 0.706 ± 0.097, and 0.654 ± 0.062, and NIQE are 1.396 ± 0.401, 1.511 ± 0.460, and 1.259 ± 0.358, respectively.

CONCLUSIONS

We proposed a novel multimodal MR image synthesis method based on a unified generative adversarial network. The network takes an image and its modality label as inputs and synthesizes multimodal images in a single forward pass. The results demonstrate that the proposed method is able to accurately synthesize multimodal MR images from a single MR image.

Deep Convolutional Encoder-Decoder algorithm for MRI brain reconstruction

Compressed Sensing Magnetic Resonance Imaging (CS-MRI) could be considered a challenged task since it could be designed as an efficient technique for fast MRI acquisition which could be highly beneficial for several clinical routines. In fact, it could grant better scan quality by reducing motion artifacts amount as well as the contrast washout effect. It offers also the possibility to reduce the exploration cost and the patient's anxiety. Recently, Deep Learning Neuronal Network (DL) has been suggested in order to reconstruct MRI scans with conserving the structural details and improving parallel imaging-based fast MRI. In this paper, we propose Deep Convolutional Encoder-Decoder architecture for CS-MRI reconstruction. Such architecture bridges the gap between the non-learning techniques, using data from only one image, and approaches using large training data. The proposed approach is based on autoencoder architecture divided into two parts: an encoder and a decoder. The encoder as well as the decoder has essentially three convolutional blocks. The proposed architecture has been evaluated through two databases: Hammersmith dataset (for the normal scans) and MICCAI 2018 (for pathological MRI). Moreover, we extend our model to cope with noisy pathological MRI scans. The normalized mean square error (NMSE), the peak-to-noise ratio (PSNR), and the structural similarity index (SSIM) have been adopted as evaluation metrics in order to evaluate the proposed architecture performance and to make a comparative study with the state-of-the-art reconstruction algorithms. The higher PSNR and SSIM values as well as the lowest NMSE values could attest that the proposed architecture offers better reconstruction and preserves textural image details. Furthermore, the running time is about 0.8 s, which is suitable for real-time processing. Such results could encourage the neurologist to adopt it in their clinical routines. Graphical abstract.

Dynamic MRI reconstruction with end-to-end motion-guided network

Temporal correlation in dynamic magnetic resonance imaging (MRI), such as cardiac MRI, is informative and important to understand motion mechanisms of body regions. Modeling such information into the MRI reconstruction process produces temporally coherent image sequence and reduces imaging artifacts and blurring. However, existing deep learning based approaches neglect motion information during the reconstruction procedure, while traditional motion-guided methods are hindered by heuristic parameter tuning and long inference time. We propose a novel dynamic MRI reconstruction approach called MODRN and an end-to-end improved version called MODRN(e2e), both of which enhance the reconstruction quality by infusing motion information into the modeling process with deep neural networks. The central idea is to decompose the motion-guided optimization problem of dynamic MRI reconstruction into three components: Dynamic Reconstruction Network, Motion Estimation and Motion Compensation. Extensive experiments have demonstrated the effectiveness of our proposed approach compared to other state-of-the-art approaches.

Improving Amide Proton Transfer-Weighted MRI Reconstruction Using T2-Weighted Images

Current protocol of Amide Proton Transfer-weighted (APTw) imaging commonly starts with the acquisition of high-resolution T2-weighted (T₂w) images followed by APTw imaging at particular geometry and locations (i.e. slice) determined by the acquired T₂w images. Although many advanced MRI reconstruction methods have been proposed to accelerate MRI, existing methods for APTw MRI lacks the capability of taking advantage of structural information in the acquired T₂w images for reconstruction. In this paper, we present a novel APTw image reconstruction framework that can accelerate APTw imaging by reconstructing APTw images directly from highly undersampled k-space data and corresponding T₂w image at the same location. The proposed framework starts with a novel sparse representation-based slice matching algorithm that aims to find the matched T₂w slice given only the undersampled APTw image. A Recurrent Feature Sharing Reconstruction network (RFS-Rec) is designed to utilize intermediate features extracted from the matched T₂w image by a Convolutional Recurrent Neural Network (CRNN), so that the missing structural information can be incorporated into the undersampled APT raw image thus effectively improving the image quality of the reconstructed APTw image. We evaluate the proposed method on two real datasets consisting of brain data from rats and humans. Extensive experiments demonstrate that the proposed RFS-Rec approach can outperform the state-of-the-art methods.

Spatial-Intensity Transform GANs for High Fidelity Medical Image-to-Image Translation

Despite recent progress in image-to-image translation, it remains challenging to apply such techniques to clinical quality medical images. We develop a novel parameterization of conditional generative adversarial networks that achieves high image fidelity when trained to transform MRIs conditioned on a patient's age and disease severity. The spatial-intensity transform generative adversarial network (SIT-GAN) constrains the generator to a smooth spatial transform composed with sparse intensity changes. This technique improves image quality and robustness to artifacts, and generalizes to different scanners. We demonstrate SIT-GAN on a large clinical image dataset of stroke patients, where it captures associations between ventricle expansion and aging, as well as between white matter hyperintensities and stroke severity. Additionally, SIT-GAN provides a disentangled view of the variation in shape and appearance across subjects.

J-MoDL: Joint Model-Based Deep Learning for Optimized Sampling and Reconstruction

Modern MRI schemes, which rely on compressed sensing or deep learning algorithms to recover MRI data from undersampled multichannel Fourier measurements, are widely used to reduce the scan time. The image quality of these approaches is heavily dependent on the sampling pattern. We introduce a continuous strategy to optimize the sampling pattern and the network parameters jointly. We use a multichannel forward model, consisting of a non-uniform Fourier transform with continuously defined sampling locations, to realize the data consistency block within a model-based deep learning image reconstruction scheme. This approach facilitates the joint and continuous optimization of the sampling pattern and the CNN parameters to improve image quality. We observe that the joint optimization of the sampling patterns and the reconstruction module significantly improves the performance of most deep learning reconstruction algorithms. The source code is available at https://github.com/hkaggarwal/J-MoDL.

Deep Learning for Ultrasound Localization Microscopy

By localizing microbubbles (MBs) in the vasculature, ultrasound localization microscopy (ULM) has recently been proposed, which greatly improves the spatial resolution of ultrasound (US) imaging and will be helpful for clinical diagnosis. Nevertheless, several challenges remain in fast ULM imaging. The main problems are that current localization methods used to implement fast ULM imaging, e.g., a previously reported localization method based on sparse recovery (CS-ULM), suffer from long data-processing time and exhaustive parameter tuning (optimization). To address these problems, in this paper, we propose a ULM method based on deep learning, which is achieved by using a modified sub-pixel convolutional neural network (CNN), termed as mSPCN-ULM. Simulations and in vivo experiments are performed to evaluate the performance of mSPCN-ULM. Simulation results show that even if under high-density condition (6.4 MBs/mm²), a high localization precision ( [Formula: see text] in the lateral direction and [Formula: see text] in the axial direction) and a high localization reliability (Jaccard index of 0.66) can be obtained by mSPCN-ULM, compared to CS-ULM. The in vivo experimental results indicate that with plane wave scan at a transmit center frequency of 15.625 MHz, microvessels with diameters of [Formula: see text] can be detected and adjacent microvessels with a distance of [Formula: see text] can be separated. Furthermore, when using GPU acceleration, the data-processing time of mSPCN-ULM can be shortened to ~6 sec/frame in the simulations and ~23 sec/frame in the in vivo experiments, which is 3-4 orders of magnitude faster than CS-ULM. Finally, once the network is trained, mSPCN-ULM does not need parameter tuning to implement ULM. As a result, mSPCN-ULM opens the door to implement ULM with fast data-processing speed, high imaging accuracy, short data-acquisition time, and high flexibility (robustness to parameters) characteristics.

Synthesize High-Quality Multi-Contrast Magnetic Resonance Imaging From Multi-Echo Acquisition Using Multi-Task Deep Generative Model

Multi-echo saturation recovery sequence can provide redundant information to synthesize multi-contrast magnetic resonance imaging. Traditional synthesis methods, such as GE's MAGiC platform, employ a model-fitting approach to generate parameter-weighted contrasts. However, models' over-simplification, as well as imperfections in the acquisition, can lead to undesirable reconstruction artifacts, especially in T2-FLAIR contrast. To improve the image quality, in this study, a multi-task deep learning model is developed to synthesize multi-contrast neuroimaging jointly using both signal relaxation relationships and spatial information. Compared with previous deep learning-based synthesis, the correlation between different destination contrast is utilized to enhance reconstruction quality. To improve model generalizability and evaluate clinical significance, the proposed model was trained and tested on a large multi-center dataset, including healthy subjects and patients with pathology. Results from both quantitative comparison and clinical reader study demonstrate that the multi-task formulation leads to more efficient and accurate contrast synthesis than previous methods.