Compressed Sensing MRI Reconstruction Using a Generative Adversarial Network With a Cyclic Loss

Residual RAKI: A hybrid linear and non-linear approach for scan-specific k-space deep learning

Parallel imaging is the most clinically used acceleration technique for magnetic resonance imaging (MRI) in part due to its easy inclusion into routine acquisitions. In k-space based parallel imaging reconstruction, sub-sampled k-space data are interpolated using linear convolutions. At high acceleration rates these methods have inherent noise amplification and reduced image quality. On the other hand, non-linear deep learning methods provide improved image quality at high acceleration, but the availability of training databases for different scans, as well as their interpretability hinder their adaptation. In this work, we present an extension of Robust Artificial-neural-networks for k-space Interpolation (RAKI), called residual-RAKI (rRAKI), which achieves scan-specific machine learning reconstruction using a hybrid linear and non-linear methodology. In rRAKI, non-linear CNNs are trained jointly with a linear convolution implemented via a skip connection. In effect, the linear part provides a baseline reconstruction, while the non-linear CNN that runs in parallel provides further reduction of artifacts and noise arising from the linear part. The explicit split between the linear and non-linear aspects of the reconstruction also help improve interpretability compared to purely non-linear methods. Experiments were conducted on the publicly available fastMRI datasets, as well as high-resolution anatomical imaging, comparing GRAPPA and its variants, compressed sensing, RAKI, Scan Specific Artifact Reduction in K-space (SPARK) and the proposed rRAKI. Additionally, highly-accelerated simultaneous multi-slice (SMS) functional MRI reconstructions were also performed, where the proposed rRAKI was compred to Read-out SENSE-GRAPPA and RAKI. Our results show that the proposed rRAKI method substantially improves the image quality compared to conventional parallel imaging, and offers sharper images compared to SPARK and ℓ₁-SPIRiT. Furthermore, rRAKI shows improved preservation of time-varying dynamics compared to both parallel imaging and RAKI in highly-accelerated SMS fMRI.

Deep Learning-Based 3D MRI Contrast-Enhanced Synthesis From A 2D Non-contrast T2Flair Sequence Deep Learning, MR images Synthesis

PURPOSE

Gadolinium-based contrast agents (GBCAs) have been successfully applied in magnetic resonance (MR) imaging to facilitate better lesion visualization. However, gadolinium deposition in the human brain raised widespread concerns recently. On the other hand, although high-resolution three-dimensional (3D) MR images are more desired for most existing medical image processing algorithms, their long scan duration and high acquiring costs make 2D MR images still much more common clinically. Therefore, developing alternative solutions for 3D contrast-enhanced MR images synthesis to replace GBCAs injection becomes an urgent requirement.

METHODS

This study proposed a deep learning framework that produces 3D isotropic full-contrast T2Flair images from 2D anisotropic non-contrast T2Flair image stacks. The super-resolution (SR) and contrast-enhanced (CE) synthesis tasks are completed in sequence by using an identical generative adversarial network (GAN) with the same techniques. To solve the problem that intra-modality datasets from different scanners have specific combinations of orientations, contrasts and resolutions, we conducted a region-based data augmentation technique on the fly during training to simulate various imaging protocols in the clinic. We further improved our network by introducing atrous spatial pyramid pooling, enhanced residual blocks and deep supervision for better quantitative and qualitative results.

RESULTS

Our proposed method achieved superior CE synthesized performance in quantitative metrics and perceptual evaluation. Detailedly, the PSNR, SSIM and AUC are 32.25 dB, 0.932 and 0.991 in the whole brain and 24.93 dB, 0.851 and 0.929 in tumor regions. The radiologists' evaluations confirmed that our proposed method has high confidence in the diagnosis. Analysis of the generalization ability showed that benefiting from the proposed data augmentation technique, our network can be applied to 'unseen' datasets with slight drops in quantitative and qualitative results.

CONCLUSION

Our work demonstrates the clinical potential of synthesizing diagnostic 3D isotropic CE brain MR images from a single 2D anisotropic non-contrast sequence. This article is protected by copyright. All rights reserved.

A feature-based convolutional neural network for reconstruction of interventional MRI

Real-time interventional MRI (I-MRI) could help to visualize the position of the interventional feature, thus improving patient outcomes in MR-guided neurosurgery. In particular, in deep brain stimulation, real-time visualization of the intervention procedure using I-MRI could improve the accuracy of the electrode placement. However, the requirements of a high undersampling rate and fast reconstruction speed for real-time imaging pose a great challenge for reconstruction of the interventional images. Based on recent advances in deep learning (DL), we proposed a feature-based convolutional neural network (FbCNN) for reconstructing interventional images from golden-angle radially sampled data. The method was composed of two stages: (a) reconstruction of the interventional feature and (b) feature refinement and postprocessing. With only five radially sampled spokes, the interventional feature was reconstructed with a cascade CNN. The final interventional image was constructed with a refined feature and a fully sampled reference image. With a comparison of traditional reconstruction techniques and recent DL-based methods, it was shown that only FbCNN could reconstruct the interventional feature and the final interventional image. With a reconstruction time of ~ 500 ms per frame and an acceleration factor of ~ 80, it was demonstrated that FbCNN had the potential for application in real-time I-MRI.

Machine Learning Applications of Surgical Imaging for the Diagnosis and Treatment of Spine Disorders: Current State of the Art

Recent developments in machine learning (ML) methods demonstrate unparalleled potential for application in the spine. The ability for ML to provide diagnostic faculty, produce novel insights from existing capabilities, and augment or accelerate elements of surgical planning and decision making at levels equivalent or superior to humans will tremendously benefit spine surgeons and patients alike. In this review, we aim to provide a clinically relevant outline of ML-based technology in the contexts of spinal deformity, degeneration, and trauma, as well as an overview of commercial-level and precommercial-level surgical assist systems and decisional support tools. Furthermore, we briefly discuss potential applications of generative networks before highlighting some of the limitations of ML applications. We conclude that ML in spine imaging represents a significant addition to the neurosurgeon's armamentarium-it has the capacity to directly address and manifest clinical needs and improve diagnostic and procedural quality and safety-but is yet subject to challenges that must be addressed before widespread implementation.

Suppressing motion artefacts in MRI using an Inception-ResNet network with motion simulation augmentation

The suppression of motion artefacts from MR images is a challenging task. The purpose of this paper was to develop a standalone novel technique to suppress motion artefacts in MR images using a data-driven deep learning approach. A simulation framework was developed to generate motion-corrupted images from motion-free images using randomly generated motion profiles. An Inception-ResNet deep learning network architecture was used as the encoder and was augmented with a stack of convolution and upsampling layers to form an encoder-decoder network. The network was trained on simulated motion-corrupted images to identify and suppress those artefacts attributable to motion. The network was validated on unseen simulated datasets and real-world experimental motion-corrupted in vivo brain datasets. The trained network was able to suppress the motion artefacts in the reconstructed images, and the mean structural similarity (SSIM) increased from 0.9058 to 0.9338. The network was also able to suppress the motion artefacts from the real-world experimental dataset, and the mean SSIM increased from 0.8671 to 0.9145. The motion correction of the experimental datasets demonstrated the effectiveness of the motion simulation generation process. The proposed method successfully removed motion artefacts and outperformed an iterative entropy minimization method in terms of the SSIM index and normalized root mean squared error, which were 5-10% better for the proposed method. In conclusion, a novel, data-driven motion correction technique has been developed that can suppress motion artefacts from motion-corrupted MR images. The proposed technique is a standalone, post-processing method that does not interfere with data acquisition or reconstruction parameters, thus making it suitable for routine clinical practice.

DIMENSION: Dynamic MR imaging with both k-space and spatial prior knowledge obtained via multi-supervised network training

Dynamic MR image reconstruction from incomplete k-space data has generated great research interest due to its capability in reducing scan time. Nevertheless, the reconstruction problem is still challenging due to its ill-posed nature. Most existing methods either suffer from long iterative reconstruction time or explore limited prior knowledge. This paper proposes a dynamic MR imaging method with both k-space and spatial prior knowledge integrated via multi-supervised network training, dubbed as DIMENSION. Specifically, the DIMENSION architecture consists of a frequential prior network for updating the k-space with its network prediction and a spatial prior network for capturing image structures and details. Furthermore, a multi-supervised network training technique is developed to constrain the frequency domain information and the spatial domain information. The comparisons with classical k-t FOCUSS, k-t SLR, L+S and the state-of-the-art CNN-based method on in vivo datasets show our method can achieve improved reconstruction results in shorter time.

SOUP-GAN: Super-Resolution MRI Using Generative Adversarial Networks

There is a growing demand for high-resolution (HR) medical images for both clinical and research applications. Image quality is inevitably traded off with acquisition time, which in turn impacts patient comfort, examination costs, dose, and motion-induced artifacts. For many image-based tasks, increasing the apparent spatial resolution in the perpendicular plane to produce multi-planar reformats or 3D images is commonly used. Single-image super-resolution (SR) is a promising technique to provide HR images based on deep learning to increase the resolution of a 2D image, but there are few reports on 3D SR. Further, perceptual loss is proposed in the literature to better capture the textural details and edges versus pixel-wise loss functions, by comparing the semantic distances in the high-dimensional feature space of a pre-trained 2D network (e.g., VGG). However, it is not clear how one should generalize it to 3D medical images, and the attendant implications are unclear. In this paper, we propose a framework called SOUP-GAN: Super-resolution Optimized Using Perceptual-tuned Generative Adversarial Network (GAN), in order to produce thinner slices (e.g., higher resolution in the ‘Z’ plane) with anti-aliasing and deblurring. The proposed method outperforms other conventional resolution-enhancement methods and previous SR work on medical images based on both qualitative and quantitative comparisons. Moreover, we examine the model in terms of its generalization for arbitrarily user-selected SR ratios and imaging modalities. Our model shows promise as a novel 3D SR interpolation technique, providing potential applications for both clinical and research applications.

DBGAN: A dual-branch generative adversarial network for undersampled MRI reconstruction

Compressed sensing magnetic resonance imaging (CS-MRI) greatly accelerates the acquisition process and yield considerable reconstructed images. Deep learning was introduced into CS-MRI to further speed up the reconstruction process and improve the image quality. Recently, generative adversarial network (GAN) using two-stage cascaded U-Net structure as generator has been proven to be effective in MRI reconstruction. However, previous cascaded structure was limited to few feature information propagation channels thus may lead to information missing. In this paper, we proposed a GAN-based model, DBGAN, for MRI reconstruction from undersampled k-space data. The model uses cross-stage skip connection (CSSC) between two end-to-end cascaded U-Net in our generator to widen the channels of feature propagation. To avoid discrepancy between training and inference, we replaced classical batch normalization (BN) with instance normalization (IN) . A stage loss is involved in the loss function to boost the training performance. In addition, a bilinear interpolation decoder branch is introduced in the generator to supplement the missing information of the deconvolution decoder. Tested under five variant patterns with four undersampling rates on different modality of MRI data, the quantitative results show that DBGAN model achieves mean improvements of 3.65 dB in peak signal-to-noise ratio (PSNR) and 0.016 in normalized mean square error (NMSE) compared with state-of-the-art GAN-based methods on T1-Weighted brain dataset from MICCAI 2013 grand challenge. The qualitative visual results show that our method can reconstruct considerable images on brain and knee MRI data from different modality. Furthermore, DBGAN is light and fast - the model parameters are fewer than half of state-of-the-art GAN-based methods and each 256 × 256 image is reconstructed in 60 milliseconds, which is suitable for real-time processing.

DualMMP-GAN: Dual-scale multi-modality perceptual generative adversarial network for medical image segmentation

Multi-modality magnetic resonance imaging (MRI) can reveal distinct patterns of tissue in the human body and is crucial to clinical diagnosis. But it still remains a challenge to obtain diverse and plausible multi-modality MR images due to expense, noise, and artifacts. For the same lesion, different modalities of MRI have big differences in context information, coarse location, and fine structure. In order to achieve better generation and segmentation performance, a dual-scale multi-modality perceptual generative adversarial network (DualMMP-GAN) is proposed based on cycle-consistent generative adversarial networks (CycleGAN). Dilated residual blocks are introduced to increase the receptive field, preserving structure and context information of images. A dual-scale discriminator is constructed. The generator is optimized by discriminating patches to represent lesions with different sizes. The perceptual consistency loss is introduced to learn the mapping between the generated and target modality at different semantic levels. Moreover, generative multi-modality segmentation (GMMS) combining given modalities with generated modalities is proposed for brain tumor segmentation. Experimental results show that the DualMMP-GAN outperforms the CycleGAN and some state-of-the-art methods in terms of PSNR, SSMI, and RMSE in most tasks. In addition, dice, sensitivity, specificity, and Hausdorff95 obtained from segmentation by GMMS are all higher than those from a single modality. The objective index obtained by the proposed methods are close to upper bounds obtained from real multiple modalities, indicating that GMMS can achieve similar effects as multi-modality. Overall, the proposed methods can serve as an effective method in clinical brain tumor diagnosis with promising application potential.

Accelerating susceptibility-weighted imaging with deep learning by complex-valued convolutional neural network (ComplexNet): validation in clinical brain imaging

OBJECTIVES

Susceptibility-weighted imaging (SWI) is crucial for the characterization of intracranial hemorrhage and mineralization, but has the drawback of long acquisition times. We aimed to propose a deep learning model to accelerate SWI, and evaluate the clinical feasibility of this approach.

METHODS

A complex-valued convolutional neural network (ComplexNet) was developed to reconstruct high-quality SWI from highly accelerated k-space data. ComplexNet can leverage the inherently complex-valued nature of SWI data and learn richer representations by using complex-valued network. SWI data were acquired from 117 participants who underwent clinical brain MRI examination between 2019 and 2021, including patients with tumor, stroke, hemorrhage, traumatic brain injury, etc. Reconstruction quality was evaluated using quantitative image metrics and image quality scores, including overall image quality, signal-to-noise ratio, sharpness, and artifacts.

RESULTS

The average reconstruction time of ComplexNet was 19 ms per section (1.33 s per participant). ComplexNet achieved significantly improved quantitative image metrics compared to a conventional compressed sensing method and a real-valued network with acceleration rates of 5 and 8 (p < 0.001). Meanwhile, there was no significant difference between fully sampled and ComplexNet approaches in terms of overall image quality and artifacts (p > 0.05) at both acceleration rates. Furthermore, ComplexNet showed comparable diagnostic performance to the fully sampled SWI for visualizing a wide range of pathology, including hemorrhage, cerebral microbleeds, and brain tumor.

CONCLUSIONS

ComplexNet can effectively accelerate SWI while providing superior performance in terms of overall image quality and visualization of pathology for routine clinical brain imaging.

KEY POINTS

• The complex-valued convolutional neural network (ComplexNet) allowed fast and high-quality reconstruction of highly accelerated SWI data, with an average reconstruction time of 19 ms per section. • ComplexNet achieved significantly improved quantitative image metrics compared to a conventional compressed sensing method and a real-valued network with acceleration rates of 5 and 8 (p < 0.001). • ComplexNet showed comparable diagnostic performance to the fully sampled SWI for visualizing a wide range of pathology, including hemorrhage, cerebral microbleeds, and brain tumor.

A Review of Deep Learning Methods for Compressed Sensing Image Reconstruction and Its Medical Applications

Compressed sensing (CS) and its medical applications are active areas of research. In this paper, we review recent works using deep learning method to solve CS problem for images or medical imaging reconstruction including computed tomography (CT), magnetic resonance imaging (MRI) and positron-emission tomography (PET). We propose a novel framework to unify traditional iterative algorithms and deep learning approaches. In short, we define two projection operators toward image prior and data consistency, respectively, and any reconstruction algorithm can be decomposed to the two parts. Though deep learning methods can be divided into several categories, they all satisfies the framework. We built the relationship between different reconstruction methods of deep learning, and connect them to traditional methods through the proposed framework. It also indicates that the key to solve CS problem and its medical applications is how to depict the image prior. Based on the framework, we analyze the current deep learning methods and point out some important directions of research in the future.

An Intelligent Music Production Technology Based on Generation Confrontation Mechanism

In recent years, with the development of deep neural network becoming more and more mature, especially after the proposal of generative confrontation mechanism, academia has made many achievements in the research of image, video and text generation. Therefore, scholars began to use similar attempts in the research of music generation. Therefore, based on the existing theoretical technology and research work, this paper studies music production, and then proposes an intelligent music production technology based on generation confrontation mechanism to enrich the research in the field of computer music generation. This paper takes the music generation method based on generation countermeasure mechanism as the research topic, and mainly studies the following: after studying the existing music generation model based on generation countermeasure network, a time structure model for maintaining music coherence is proposed. In music generation, avoid manual input and ensure the interdependence between tracks. At the same time, this paper studies and implements the generation method of discrete music events based on multi track, including multi track correlation model and discrete processing. The lakh MIDI data set is studied. On this basis, the lakh MIDI is pre-processed to obtain the LMD piano roll data set, which is used in the music generation experiment of MCT-GAN. When studying the multi track music generation based on generation countermeasure network, this paper studies and analyzes three models, and puts forward the multi track music generation method based on CT-GAN, which mainly improves the existing music generation model based on GAN. Finally, the generation results of MCT-GAN are compared with those of Muse-GAN, so as to reflect the improvement effect of MCT-GAN. Select 20 auditees to listen to the generated music and real music and distinguish them. Finally, analyze them according to the evaluation results. After evaluation, it is concluded that the research effect of multi track music generation based on CT-GAN is improved.

Investigating the limited performance of a deep-learning-based SPECT denoising approach: An observer-study-based characterization

Multiple objective assessment of image-quality (OAIQ)-based studies have reported that several deep-learning (DL)-based denoising methods show limited performance on signal-detection tasks. Our goal was to investigate the reasons for this limited performance. To achieve this goal, we conducted a task-based characterization of a DL-based denoising approach for individual signal properties. We conducted this study in the context of evaluating a DL-based approach for denoising single photon-emission computed tomography (SPECT) images. The training data consisted of signals of different sizes and shapes within a clustered-lumpy background, imaged with a 2D parallel-hole-collimator SPECT system. The projections were generated at normal and 20% low-count level, both of which were reconstructed using an ordered-subset-expectation-maximization (OSEM) algorithm. A convolutional neural network (CNN)-based denoiser was trained to process the low-count images. The performance of this CNN was characterized for five different signal sizes and four different signal-to-background ratio (SBRs) by designing each evaluation as a signal-known-exactly/background-known-statistically (SKE/BKS) signal-detection task. Performance on this task was evaluated using an anthropomorphic channelized Hotelling observer (CHO). As in previous studies, we observed that the DL-based denoising method did not improve performance on signal-detection tasks. Evaluation using the idea of observer-study-based characterization demonstrated that the DL-based denoising approach did not improve performance on the signal-detection task for any of the signal types. Overall, these results provide new insights on the performance of the DL-based denoising approach as a function of signal size and contrast. More generally, the observer study-based characterization provides a mechanism to evaluate the sensitivity of the method to specific object properties, and may be explored as analogous to characterizations such as modulation transfer function for linear systems. Finally, this work underscores the need for objective task-based evaluation of DL-based denoising approaches.

Studying osteoarthritis with artificial intelligence applied to magnetic resonance imaging

The 3D nature and soft-tissue contrast of MRI makes it an invaluable tool for osteoarthritis research, by facilitating the elucidation of disease pathogenesis and progression. The recent increasing employment of MRI has certainly been stimulated by major advances that are due to considerable investment in research, particularly related to artificial intelligence (AI). These AI-related advances are revolutionizing the use of MRI in clinical research by augmenting activities ranging from image acquisition to post-processing. Automation is key to reducing the long acquisition times of MRI, conducting large-scale longitudinal studies and quantitatively defining morphometric and other important clinical features of both soft and hard tissues in various anatomical joints. Deep learning methods have been used recently for multiple applications in the musculoskeletal field to improve understanding of osteoarthritis. Compared with labour-intensive human efforts, AI-based methods have advantages and potential in all stages of imaging, as well as post-processing steps, including aiding diagnosis and prognosis. However, AI-based methods also have limitations, including the arguably limited interpretability of AI models. Given that the AI community is highly invested in uncovering uncertainties associated with model predictions and improving their interpretability, we envision future clinical translation and progressive increase in the use of AI algorithms to support clinicians in optimizing patient care.

Edge-enhanced dual discriminator generative adversarial network for fast MRI with parallel imaging using multi-view information

In clinical medicine, magnetic resonance imaging (MRI) is one of the most important tools for diagnosis, triage, prognosis, and treatment planning. However, MRI suffers from an inherent slow data acquisition process because data is collected sequentially in k-space. In recent years, most MRI reconstruction methods proposed in the literature focus on holistic image reconstruction rather than enhancing the edge information. This work steps aside this general trend by elaborating on the enhancement of edge information. Specifically, we introduce a novel parallel imaging coupled dual discriminator generative adversarial network (PIDD-GAN) for fast multi-channel MRI reconstruction by incorporating multi-view information. The dual discriminator design aims to improve the edge information in MRI reconstruction. One discriminator is used for holistic image reconstruction, whereas the other one is responsible for enhancing edge information. An improved U-Net with local and global residual learning is proposed for the generator. Frequency channel attention blocks (FCA Blocks) are embedded in the generator for incorporating attention mechanisms. Content loss is introduced to train the generator for better reconstruction quality. We performed comprehensive experiments on Calgary-Campinas public brain MR dataset and compared our method with state-of-the-art MRI reconstruction methods. Ablation studies of residual learning were conducted on the MICCAI13 dataset to validate the proposed modules. Results show that our PIDD-GAN provides high-quality reconstructed MR images, with well-preserved edge information. The time of single-image reconstruction is below 5ms, which meets the demand of faster processing.

Inpainted Image Reconstruction Using an Extended Hopfield Neural Network Based Machine Learning System

This paper considers the use of a machine learning system for the reconstruction and recognition of distorted or damaged patterns, in particular, images of faces partially covered with masks. The most up-to-date image reconstruction structures are based on constrained optimization algorithms and suitable regularizers. In contrast with the above-mentioned image processing methods, the machine learning system presented in this paper employs the superposition of system vectors setting up asymptotic centers of attraction. The structure of the system is implemented using Hopfield-type neural network-based biorthogonal transformations. The reconstruction property gives rise to a superposition processor and reversible computations. Moreover, this paper's distorted image reconstruction sets up associative memories where images stored in memory are retrieved by distorted/inpainted key images.

Undersampled Multi-contrast MRI Reconstruction Based on Double-domain Generative Adversarial Network

Multi-contrast magnetic resonance imaging can provide comprehensive information for clinical diagnosis. However, multi-contrast imaging suffers from long acquisition time, which makes it inhibitive for daily clinical practice. Subsampling k-space is one of the main methods to speed up scan time. Missing k-space samples will lead to inevitable serious artifacts and noise. Considering the assumption that different contrast modalities share some mutual information, it may be possible to exploit this redundancy to accelerate multi-contrast imaging acquisition. Recently, generative adversarial network shows superior performance in image reconstruction and synthesis. Some studies based on k-space reconstruction also exhibit superior performance over conventional state-of-art method. In this study, we propose a cross-domain two-stage generative adversarial network for multi-contrast images reconstruction based on prior full-sampled contrast and undersampled information. The new approach integrates reconstruction and synthesis, which estimates and completes the missing k-space and then refines in image space. It takes one fully-sampled contrast modality data and highly undersampled data from several other modalities as input, and outputs high quality images for each contrast simultaneously. The network is trained and tested on a public brain dataset from healthy subjects. Quantitative comparisons against baseline clearly indicate that the proposed method can effectively reconstruct undersampled images. Even under high acceleration, the network still can recover texture details and reduce artifacts.

2D probabilistic undersampling pattern optimization for MR image reconstruction

With 3D magnetic resonance imaging (MRI), a tradeoff exists between higher image quality and shorter scan time. One way to solve this problem is to reconstruct high-quality MRI images from undersampled k-space. There have been many recent studies exploring effective k-space undersampling patterns and designing MRI reconstruction methods from undersampled k-space, which are two necessary steps. Most studies separately considered these two steps, although in theory, their performance is dependent on each other. In this study, we propose a joint optimization model, trained end-to-end, to simultaneously optimize the undersampling pattern in the Fourier domain and the reconstruction model in the image domain. A 2D probabilistic undersampling layer was designed to optimize the undersampling pattern and probability distribution in a differentiable manner. A 2D inverse Fourier transform layer was implemented to connect the Fourier domain and the image domain during the forward and back propagation. Finally, we discovered an optimized relationship between the probability distribution of the undersampling pattern and its corresponding sampling rate. Further testing was performed using 3D T1-weighted MR images of the brain from the MICCAI 2013 Grand Challenge on Multi-Atlas Labeling dataset and locally acquired brain 3D T1-weighted MR images of healthy volunteers and contrast-enhanced 3D T1-weighted MR images of high-grade glioma patients. The results showed that the recovered MR images using our 2D probabilistic undersampling pattern (with or without the reconstruction network) significantly outperformed those using the existing start-of-the-art undersampling strategies for both qualitative and quantitative comparison, suggesting the advantages and some extent of the generalization of our proposed method.

Recognition of CRISPR Off-Target Cleavage Sites with SeqGAN

Background:

The CRISPR system can quickly achieve the editing of different gene loci by changing a small sequence on a single guide RNA. But the off-target event limits the further development of the CRISPR system. How to improve the efficiency and specificity of this technology and minimize the risk of off-target have always been a challenge. For genome-wide CRISPR Off-Target Cleavage Sites (OTS) prediction, an important issue is data imbalance, that is, the number of true OTS identified is much less than that of all possible nucleotide mismatch loci.

Method:

In this work, based on the sequence-generating adversarial network (SeqGAN), positive offtarget sequences were generated to amplify the off-target gene locus OTS dataset of Cpf1. Then we trained the data by a deep Convolutional Neural Network (CNN) to obtain a predictor with stronger generalization ability and better performance.

Results:

In 10-fold cross-validation, the AUC value of the CNN classifier after SeqGAN balance was 0.941, which was higher than that of the original 0.863 and over-sampling 0.929. In independence testing, the AUC value of the CNN classifier after SeqGAN balance was 0.841, which was higher than that of the original 0.833 and over-sampling 0.836. The PR value was 0.722 after SeqGAN, which was also about higher 0.16 than the original data and higher about 0.03 than over-sampling.

Conclusion:

The sequence generation antagonistic network SeqGAN was firstly used to deal with data imbalance processing on CRISPR data. All the results showed that the SeqGAN can effectively generate positive data for CRISPR off-target sites.

POCS-Augmented CycleGAN for MR Image Reconstruction

Recent years have seen increased research interest in replacing the computationally intensive Magnetic resonance (MR) image reconstruction process with deep neural networks. We claim in this paper that the traditional image reconstruction methods and deep learning (DL) are mutually complementary and can be combined to achieve better image reconstruction quality. To test this hypothesis, a hybrid DL image reconstruction method was proposed by combining a state-of-the-art deep learning network, namely a generative adversarial network with cycle loss (CycleGAN), with a traditional data reconstruction algorithm: Projection Onto Convex Set (POCS). The output of the first iteration's training results of the CycleGAN was updated by POCS and used as the extra training data for the second training iteration of the CycleGAN. The method was validated using sub-sampled Magnetic resonance imaging data. Compared with other state-of-the-art, DL-based methods (e.g., U-Net, GAN, and RefineGAN) and a traditional method (compressed sensing), our method showed the best reconstruction results.

Compressed Sensing in Sodium Magnetic Resonance Imaging: Techniques, Applications, and Future Prospects

Sodium (²³ Na) yields the second strongest nuclear magnetic resonance (NMR) signal in biological tissues and plays a vital role in cell physiology. Sodium magnetic resonance imaging (MRI) can provide insights into cell integrity and tissue viability relative to pathologies without significant anatomical alternations, and thus it is considered to be a potential surrogate biomarker that provides complementary information for standard hydrogen (¹ H) MRI in a noninvasive and quantitative manner. However, sodium MRI suffers from a relatively low signal-to-noise ratio and long acquisition times due to its relatively low NMR sensitivity. Compressed sensing-based (CS-based) methods have been shown to accelerate sodium imaging and/or improve sodium image quality significantly. In this manuscript, the basic concepts of CS and how CS might be applied to improve sodium MRI are described, and the historical milestones of CS-based sodium MRI are briefly presented. Representative advanced techniques and evaluation methods are discussed in detail, followed by an expose of clinical applications in multiple anatomical regions and diseases as well as thoughts and suggestions on potential future research prospects of CS in sodium MRI. EVIDENCE LEVEL: 5 TECHNICAL EFFICACY: Stage 1.

Intelligent Generation Method of Innovative Structures Based on Topology Optimization and Deep Learning

Computer-aided design has been widely used in structural calculation and analysis, but there are still challenges in generating innovative structures intelligently. Aiming at this issue, a new method was proposed to realize the intelligent generation of innovative structures based on topology optimization and deep learning. Firstly, a large number of structural models obtained from topology optimization under different optimization parameters were extracted to produce the training set images, and the training set labels were defined as the corresponding load cases. Then, the boundary equilibrium generative adversarial networks (BEGAN) deep learning algorithm was applied to generate numerous innovative structures. Finally, the generated structures were evaluated by a series of evaluation indexes, including innovation, aesthetics, machinability, and mechanical performance. Combined with two engineering cases, the application process of the above method is described here in detail. Furthermore, the 3D reconstruction and additive manufacturing techniques were applied to manufacture the structural models. The research results showed that the proposed approach of structural generation based on topology optimization and deep learning is feasible, and can not only generate innovative structures but also optimize the material consumption and mechanical performance further.

Review and Prospect: Artificial Intelligence in Advanced Medical Imaging

Artificial intelligence (AI) as an emerging technology is gaining momentum in medical imaging. Recently, deep learning-based AI techniques have been actively investigated in medical imaging, and its potential applications range from data acquisition and image reconstruction to image analysis and understanding. In this review, we focus on the use of deep learning in image reconstruction for advanced medical imaging modalities including magnetic resonance imaging (MRI), computed tomography (CT), and positron emission tomography (PET). Particularly, recent deep learning-based methods for image reconstruction will be emphasized, in accordance with their methodology designs and performances in handling volumetric imaging data. It is expected that this review can help relevant researchers understand how to adapt AI for medical imaging and which advantages can be achieved with the assistance of AI.

MRI Knee Domain Translation for Unsupervised Segmentation By CycleGAN (data from Osteoarthritis initiative (OAI))

Accurate quantification of bone and cartilage features is the key to efficient management of knee osteoarthritis (OA). Bone and cartilage tissues can be accurately segmented from magnetic resonance imaging (MRI) data using supervised Deep Learning (DL) methods. DL training is commonly conducted using large datasets with expert-labeled annotations. DL models perform better if distributions of testing data (target domains) are close to those of training data (source domains). However, in practice, data distributions of images from different MRI scanners and sequences are different and DL models need to re-trained on each dataset separately. We propose a domain adaptation (DA) framework using the CycleGAN model for MRI translation that would aid in unsupervised MRI data segmentation. We have validated our pipeline on five scans from the Osteoarthritis Initiative (OAI) dataset. Using this pipeline, we translated TSE Fat Suppressed MRI sequences to pseudo-DESS images. An improved MaskRCNN (IMaskRCNN) instance segmentation network trained on DESS was used to segment cartilage and femoral head regions in TSE Fat Suppressed sequences. Segmentations of the I-MaskRCNN correlated well with approximated manual segmentation obtained from nearest DESS slices (DICE = 0.76) without the need for retraining. We anticipate this technique will aid in automatic unsupervised assessment of knee MRI using commonly acquired MRI sequences and save experts' time that would otherwise be required for manual segmentation.Clinical relevance- This technique paves the way to automatically convert one MRI sequence to its equivalent as if acquired by a different protocol or different magnet, facilitating robust, hardware-independent automated analysis. For example, routine clinically acquired knee MRI could be converted to high-resolution high-contrast images suitable for automated detection of cartilage defects.

When medical images meet generative adversarial network: recent development and research opportunities

Deep learning techniques have promoted the rise of artificial intelligence (AI) and performed well in computer vision. Medical image analysis is an important application of deep learning, which is expected to greatly reduce the workload of doctors, contributing to more sustainable health systems. However, most current AI methods for medical image analysis are based on supervised learning, which requires a lot of annotated data. The number of medical images available is usually small and the acquisition of medical image annotations is an expensive process. Generative adversarial network (GAN), an unsupervised method that has become very popular in recent years, can simulate the distribution of real data and reconstruct approximate real data. GAN opens some exciting new ways for medical image generation, expanding the number of medical images available for deep learning methods. Generated data can solve the problem of insufficient data or imbalanced data categories. Adversarial training is another contribution of GAN to medical imaging that has been applied to many tasks, such as classification, segmentation, or detection. This paper investigates the research status of GAN in medical images and analyzes several GAN methods commonly applied in this area. The study addresses GAN application for both medical image synthesis and adversarial learning for other medical image tasks. The open challenges and future research directions are also discussed.

Homotopic Gradients of Generative Density Priors for MR Image Reconstruction

Deep learning, particularly the generative model, has demonstrated tremendous potential to significantly speed up image reconstruction with reduced measurements recently. Rather than the existing generative models that often optimize the density priors, in this work, by taking advantage of the denoising score matching, homotopic gradients of generative density priors (HGGDP) are exploited for magnetic resonance imaging (MRI) reconstruction. More precisely, to tackle the low-dimensional manifold and low data density region issues in generative density prior, we estimate the target gradients in higher-dimensional space. We train a more powerful noise conditional score network by forming high-dimensional tensor as the network input at the training phase. More artificial noise is also injected in the embedding space. At the reconstruction stage, a homotopy method is employed to pursue the density prior, such as to boost the reconstruction performance. Experiment results implied the remarkable performance of HGGDP in terms of high reconstruction accuracy. Only 10% of the k-space data can still generate image of high quality as effectively as standard MRI reconstructions with the fully sampled data.

Learning Data Consistency and its Application to Dynamic MR Imaging

Magnetic resonance (MR) image reconstruction from undersampled k-space data can be formulated as a minimization problem involving data consistency and image prior. Existing deep learning (DL)-based methods for MR reconstruction employ deep networks to exploit the prior information and integrate the prior knowledge into the reconstruction under the explicit constraint of data consistency, without considering the real distribution of the noise. In this work, we propose a new DL-based approach termed Learned DC that implicitly learns the data consistency with deep networks, corresponding to the actual probability distribution of system noise. The data consistency term and the prior knowledge are both embedded in the weights of the networks, which provides an utterly implicit manner of learning reconstruction model. We evaluated the proposed approach with highly undersampled dynamic data, including the dynamic cardiac cine data with up to 24-fold acceleration and dynamic rectum data with the acceleration factor equal to the number of phases. Experimental results demonstrate the superior performance of the Learned DC both quantitatively and qualitatively than the state-of-the-art.

Blind Primed Supervised (BLIPS) Learning for MR Image Reconstruction

This paper examines a combined supervised-unsupervised framework involving dictionary-based blind learning and deep supervised learning for MR image reconstruction from under-sampled k-space data. A major focus of the work is to investigate the possible synergy of learned features in traditional shallow reconstruction using adaptive sparsity-based priors and deep prior-based reconstruction. Specifically, we propose a framework that uses an unrolled network to refine a blind dictionary learning-based reconstruction. We compare the proposed method with strictly supervised deep learning-based reconstruction approaches on several datasets of varying sizes and anatomies. We also compare the proposed method to alternative approaches for combining dictionary-based methods with supervised learning in MR image reconstruction. The improvements yielded by the proposed framework suggest that the blind dictionary-based approach preserves fine image details that the supervised approach can iteratively refine, suggesting that the features learned using the two methods are complementary.

Deep learning for neuroimaging-based diagnosis and rehabilitation of Autism Spectrum Disorder: A review

Accurate diagnosis of Autism Spectrum Disorder (ASD) followed by effective rehabilitation is essential for the management of this disorder. Artificial intelligence (AI) techniques can aid physicians to apply automatic diagnosis and rehabilitation procedures. AI techniques comprise traditional machine learning (ML) approaches and deep learning (DL) techniques. Conventional ML methods employ various feature extraction and classification techniques, but in DL, the process of feature extraction and classification is accomplished intelligently and integrally. DL methods for diagnosis of ASD have been focused on neuroimaging-based approaches. Neuroimaging techniques are non-invasive disease markers potentially useful for ASD diagnosis. Structural and functional neuroimaging techniques provide physicians substantial information about the structure (anatomy and structural connectivity) and function (activity and functional connectivity) of the brain. Due to the intricate structure and function of the brain, proposing optimum procedures for ASD diagnosis with neuroimaging data without exploiting powerful AI techniques like DL may be challenging. In this paper, studies conducted with the aid of DL networks to distinguish ASD are investigated. Rehabilitation tools provided for supporting ASD patients utilizing DL networks are also assessed. Finally, we will present important challenges in the automated detection and rehabilitation of ASD and propose some future works.

Deep Convolutional Neural Network for Compressive Sensing of Magnetic Resonance Images

Compressive Sensing for Magnetic Resonance Imaging (CS-MRI) aims to reconstruct Magnetic Resonance (MR) images from under-sampled raw data. There are two challenges to improve CS-MRI methods, i.e. designing an under-sampling algorithm to achieve optimal sampling, as well as designing fast and small deep neural networks to obtain reconstructed MR images with superior quality. To improve the reconstruction quality of MR images, we propose a novel deep convolutional neural network architecture for CS-MRI named MRCSNet. The MRCSNet consists of three sub-networks, a compressive sensing sampling sub-network, an initial reconstruction sub-network, and a refined reconstruction sub-network. Experimental results demonstrate that MRCSNet generates high-quality reconstructed MR images at various under-sampling ratios, and also meets the requirements of real-time CS-MRI applications. Compared to state-of-the-art CS-MRI approaches, MRCSNet offers a significant improvement in reconstruction accuracies, such as Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity (SSIM). Besides, it reduces the reconstruction error evaluated by the Normalized Root-Mean-Square Error (NRMSE). The source codes are available at https://github.com/TaihuLight/MRCSNet .

Generating synthetic CT from low-dose cone-beam CT by using generative adversarial networks for adaptive radiotherapy

OBJECTIVE

To develop high-quality synthetic CT (sCT) generation method from low-dose cone-beam CT (CBCT) images by using attention-guided generative adversarial networks (AGGAN) and apply these images to dose calculations in radiotherapy.

METHODS

The CBCT/planning CT images of 170 patients undergoing thoracic radiotherapy were used for training and testing. The CBCT images were scanned under a fast protocol with 50% less clinical projection frames compared with standard chest M20 protocol. Training with aligned paired images was performed using conditional adversarial networks (so-called pix2pix), and training with unpaired images was carried out with cycle-consistent adversarial networks (cycleGAN) and AGGAN, through which sCT images were generated. The image quality and Hounsfield unit (HU) value of the sCT images generated by the three neural networks were compared. The treatment plan was designed on CT and copied to sCT images to calculated dose distribution.

RESULTS

The image quality of sCT images by all the three methods are significantly improved compared with original CBCT images. The AGGAN achieves the best image quality in the testing patients with the smallest mean absolute error (MAE, 43.5 ± 6.69), largest structural similarity (SSIM, 93.7 ± 3.88) and peak signal-to-noise ratio (PSNR, 29.5 ± 2.36). The sCT images generated by all the three methods showed superior dose calculation accuracy with higher gamma passing rates compared with original CBCT image. The AGGAN offered the highest gamma passing rates (91.4 ± 3.26) under the strictest criteria of 1 mm/1% compared with other methods. In the phantom study, the sCT images generated by AGGAN demonstrated the best image quality and the highest dose calculation accuracy.

CONCLUSIONS

High-quality sCT images were generated from low-dose thoracic CBCT images by using the proposed AGGAN through unpaired CBCT and CT images. The dose distribution could be calculated accurately based on sCT images in radiotherapy.

Noise Reduction for SD-OCT Using a Structure-Preserving Domain Transfer Approach

Spectral-domain optical coherence tomography (SD-OCT) images inevitably suffer from multiplicative speckle noise caused by random interference. This study proposes an unsupervised domain adaptation approach for noise reduction by translating the SD-OCT to the corresponding high-quality enhanced depth imaging (EDI)-OCT. We propose a structure-persevered cycle-consistent generative adversarial network for unpaired image-to-image translation, which can be applied to imbalanced unpaired data, and can effectively preserve retinal details based on a structure-specific cross-domain description. It also imposes smoothness by penalizing the intensity variation of the low reflective region between consecutive slices. Our approach was tested on a local data set that consisted of 268 SD-OCT volumes and two public independent validation datasets including 20 SD-OCT volumes and 17 B-scans, respectively. Experimental results show that our method can effectively suppress noise and maintain the retinal structure, compared with other traditional approaches and deep learning methods in terms of qualitative and quantitative assessments. Our proposed method shows good performance for speckle noise reduction and can assist downstream tasks of OCT analysis.

Noninterpretive Uses of Artificial Intelligence in Radiology

We deem a computer to exhibit artificial intelligence (AI) when it performs a task that would normally require intelligent action by a human. Much of the recent excitement about AI in the medical literature has revolved around the ability of AI models to recognize anatomy and detect pathology on medical images, sometimes at the level of expert physicians. However, AI can also be used to solve a wide range of noninterpretive problems that are relevant to radiologists and their patients. This review summarizes some of the newer noninterpretive uses of AI in radiology.

FA-GAN: Fused attentive generative adversarial networks for MRI image super-resolution

High-resolution magnetic resonance images can provide fine-grained anatomical information, but acquiring such data requires a long scanning time. In this paper, a framework called the Fused Attentive Generative Adversarial Networks(FA-GAN) is proposed to generate the super- resolution MR image from low-resolution magnetic resonance images, which can reduce the scanning time effectively but with high resolution MR images. In the framework of the FA-GAN, the local fusion feature block, consisting of different three-pass networks by using different convolution kernels, is proposed to extract image features at different scales. And the global feature fusion module, including the channel attention module, the self-attention module, and the fusion operation, is designed to enhance the important features of the MR image. Moreover, the spectral normalization process is introduced to make the discriminator network stable. 40 sets of 3D magnetic resonance images (each set of images contains 256 slices) are used to train the network, and 10 sets of images are used to test the proposed method. The experimental results show that the PSNR and SSIM values of the super-resolution magnetic resonance image generated by the proposed FA-GAN method are higher than the state-of-the-art reconstruction methods.

Generating Virtual Short Tau Inversion Recovery (STIR) Images from T1- and T2-Weighted Images Using a Conditional Generative Adversarial Network in Spine Imaging

Short tau inversion recovery (STIR) sequences are frequently used in magnetic resonance imaging (MRI) of the spine. However, STIR sequences require a significant amount of scanning time. The purpose of the present study was to generate virtual STIR (vSTIR) images from non-contrast, non-fat-suppressed T1- and T2-weighted images using a conditional generative adversarial network (cGAN). The training dataset comprised 612 studies from 514 patients, and the validation dataset comprised 141 studies from 133 patients. For validation, 100 original STIR and respective vSTIR series were presented to six senior radiologists (blinded for the STIR type) in independent A/B-testing sessions. Additionally, for 141 real or vSTIR sequences, the testers were required to produce a structured report of 15 different findings. In the A/B-test, most testers could not reliably identify the real STIR (mean error of tester 1-6: 41%; 44%; 58%; 48%; 39%; 45%). In the evaluation of the structured reports, vSTIR was equivalent to real STIR in 13 of 15 categories. In the category of the number of STIR hyperintense vertebral bodies (p = 0.08) and in the diagnosis of bone metastases (p = 0.055), the vSTIR was only slightly insignificantly equivalent. By virtually generating STIR images of diagnostic quality from T1- and T2-weighted images using a cGAN, one can shorten examination times and increase throughput.

Systematic Analysis and Review of Magnetic Resonance Imaging (MRI) Reconstruction Techniques

BACKGROUND

Magnetic Resonance Imaging (MRI) plays an important role in the field of medical diagnostic imaging as it poses non-invasive acquisition and high soft-tissue contrast. However, a huge time is needed for the MRI scanning process that results in motion artifacts, degrades image quality, misinterprets the data, and may cause discomfort to the patient. Thus, the main goal of MRI research is to accelerate data acquisition processing without affecting the quality of the image.

INTRODUCTION

This paper presents a survey based on distinct conventional MRI reconstruction methodologies. In addition, a novel MRI reconstruction strategy is proposed based on weighted Compressive Sensing (CS), Penalty-aided minimization function, and Meta-heuristic optimization technique.

METHODS

An illustrative analysis is done concerning adapted methods, datasets used, execution tools, performance measures, and values of evaluation metrics. Moreover, the issues of existing methods and the research gaps considering conventional MRI reconstruction schemes are elaborated to obtain improved contribution for devising significant MRI reconstruction techniques.

RESULTS

The proposed method will reduce conventional aliasing artifact problems, may attain lower Mean Square Error (MSE), higher Peak Signal-to-Noise Ratio (PSNR), and Structural SIMilarity (SSIM) index.

CONCLUSION

The issues of existing methods and the research gaps considering conventional MRI reconstruction schemes are elaborated to devising an improved significant MRI reconstruction technique.

Domain knowledge augmentation of parallel MR image reconstruction using deep learning

A deep learning (DL) method for accelerated magnetic resonance (MR) imaging is presented that incorporates domain knowledge of parallel MR imaging to augment the DL networks for accurate and stable image reconstruction. The proposed DL method employs a novel loss function consisting of a combination of mean absolute error, structural similarity, and sobel edge loss. The DL model takes both original measurements and images reconstructed by the parallel imaging method as inputs to the network. The accuracy of the proposed method was evaluated using two anatomical regions and six MRI contrasts and was compared with state-of-the-art parallel imaging and deep learning methods. The proposed method significantly outperformed the other methods for all the six different contrasts in terms of structural similarity, peak signal to noise ratio, and normalized mean squared error. The out-of-sample performance of the proposed method was assessed for a truly "unseen" case in a volunteer scan. The method produced images without any artificial features, often occurring in the DL image reconstruction methods. A stability analysis was performed by adding perturbations to the input, which demonstrated that the proposed method is robust and stable with respect to small structural changes, and different undersampling ratios. Comprehensive validation on large datasets demonstrated that incorporation of domain knowledge sufficiently regularizes the DL based image reconstruction and produces accurate and stable image enhancement.

Atom selection strategy for signal compressed recovery based on sensing information entropy

In greedy pursuit algorithm, atom selection is commonly a concerned topic for signal compressed recovery. To improve the recovery performance, an optimal atom selection strategy without the prior information is proposed in this paper. The sensing information entropy is defined to prune the possible false atoms in the estimated support set. Fewer iterations are required in the proposed strategy and it can also be applied in the case with high sparsity level or low signal-noise-ratio. Compared with the existing representative algorithms, the superiority of the recovery error and probability is verified by the simulations. Furthermore, the proposed method is applied to recover the real random modulated signal. The results show that the recovered signal has greater consistence with the original input signal.

Temporally aware volumetric generative adversarial network-based MR image reconstruction with simultaneous respiratory motion compensation: Initial feasibility in 3D dynamic cine cardiac MRI

PURPOSE

Develop a novel three-dimensional (3D) generative adversarial network (GAN)-based technique for simultaneous image reconstruction and respiratory motion compensation of 4D MRI. Our goal was to enable high-acceleration factors 10.7X-15.8X, while maintaining robust and diagnostic image quality superior to state-of-the-art self-gating (SG) compressed sensing wavelet (CS-WV) reconstruction at lower acceleration factors 3.5X-7.9X.

METHODS

Our GAN was trained based on pixel-wise content loss functions, adversarial loss function, and a novel data-driven temporal aware loss function to maintain anatomical accuracy and temporal coherence. Besides image reconstruction, our network also performs respiratory motion compensation for free-breathing scans. A novel progressive growing-based strategy was adapted to make the training process possible for the proposed GAN-based structure. The proposed method was developed and thoroughly evaluated qualitatively and quantitatively based on 3D cardiac cine data from 42 patients.

RESULTS

Our proposed method achieved significantly better scores in general image quality and image artifacts at 10.7X-15.8X acceleration than the SG CS-WV approach at 3.5X-7.9X acceleration (4.53 ± 0.540 vs. 3.13 ± 0.681 for general image quality, 4.12 ± 0.429 vs. 2.97 ± 0.434 for image artifacts, P < .05 for both). No spurious anatomical structures were observed in our images. The proposed method enabled similar cardiac-function quantification as conventional SG CS-WV. The proposed method achieved faster central processing unit-based image reconstruction (6 s/cardiac phase) than the SG CS-WV (312 s/cardiac phase).

CONCLUSION

The proposed method showed promising potential for high-resolution (1 mm³ ) free-breathing 4D MR data acquisition with simultaneous respiratory motion compensation and fast reconstruction time.

Transfer learning enhanced generative adversarial networks for multi-channel MRI reconstruction

Deep learning based generative adversarial networks (GAN) can effectively perform image reconstruction with under-sampled MR data. In general, a large number of training samples are required to improve the reconstruction performance of a certain model. However, in real clinical applications, it is difficult to obtain tens of thousands of raw patient data to train the model since saving k-space data is not in the routine clinical flow. Therefore, enhancing the generalizability of a network based on small samples is urgently needed. In this study, three novel applications were explored based on parallel imaging combined with the GAN model (PI-GAN) and transfer learning. The model was pre-trained with public Calgary brain images and then fine-tuned for use in (1) patients with tumors in our center; (2) different anatomies, including knee and liver; (3) different k-space sampling masks with acceleration factors (AFs) of 2 and 6. As for the brain tumor dataset, the transfer learning results could remove the artifacts found in PI-GAN and yield smoother brain edges. The transfer learning results for the knee and liver were superior to those of the PI-GAN model trained with its own dataset using a smaller number of training cases. However, the learning procedure converged more slowly in the knee datasets compared to the learning in the brain tumor datasets. The reconstruction performance was improved by transfer learning both in the models with AFs of 2 and 6. Of these two models, the one with AF = 2 showed better results. The results also showed that transfer learning with the pre-trained model could solve the problem of inconsistency between the training and test datasets and facilitate generalization to unseen data.

Which GAN? A comparative study of generative adversarial network-based fast MRI reconstruction

Fast magnetic resonance imaging (MRI) is crucial for clinical applications that can alleviate motion artefacts and increase patient throughput. K-space undersampling is an obvious approach to accelerate MR acquisition. However, undersampling of k-space data can result in blurring and aliasing artefacts for the reconstructed images. Recently, several studies have been proposed to use deep learning-based data-driven models for MRI reconstruction and have obtained promising results. However, the comparison of these methods remains limited because the models have not been trained on the same datasets and the validation strategies may be different. The purpose of this work is to conduct a comparative study to investigate the generative adversarial network (GAN)-based models for MRI reconstruction. We reimplemented and benchmarked four widely used GAN-based architectures including DAGAN, ReconGAN, RefineGAN and KIGAN. These four frameworks were trained and tested on brain, knee and liver MRI images using twofold, fourfold and sixfold accelerations, respectively, with a random undersampling mask. Both quantitative evaluations and qualitative visualization have shown that the RefineGAN method has achieved superior performance in reconstruction with better accuracy and perceptual quality compared to other GAN-based methods. This article is part of the theme issue 'Synergistic tomographic image reconstruction: part 1'.

PhaseGAN: a deep-learning phase-retrieval approach for unpaired datasets

Phase retrieval approaches based on deep learning (DL) provide a framework to obtain phase information from an intensity hologram or diffraction pattern in a robust manner and in real-time. However, current DL architectures applied to the phase problem rely on i) paired datasets, i. e., they are only applicable when a satisfactory solution of the phase problem has been found, and ii) the fact that most of them ignore the physics of the imaging process. Here, we present PhaseGAN, a new DL approach based on Generative Adversarial Networks, which allows the use of unpaired datasets and includes the physics of image formation. The performance of our approach is enhanced by including the image formation physics and a novel Fourier loss function, providing phase reconstructions when conventional phase retrieval algorithms fail, such as ultra-fast experiments. Thus, PhaseGAN offers the opportunity to address the phase problem in real-time when no phase reconstructions but good simulations or data from other experiments are available.