PIC-GAN: A Parallel Imaging Coupled Generative Adversarial Network for Accelerated Multi-Channel MRI Reconstruction

Image Translation for Estimating Two-Dimensional Axial Amyloid-Beta PET From Structural MRI

BACKGROUND

Amyloid-beta and brain atrophy are hallmarks for Alzheimer's Disease that can be targeted with positron emission tomography (PET) and MRI, respectively. MRI is cheaper, less-invasive, and more available than PET. There is a known relationship between amyloid-beta and brain atrophy, meaning PET images could be inferred from MRI.

PURPOSE

To build an image translation model using a Conditional Generative Adversarial Network able to synthesize Amyloid-beta PET images from structural MRI.

STUDY TYPE

Retrospective.

POPULATION

Eight hundred eighty-two adults (348 males/534 females) with different stages of cognitive decline (control, mild cognitive impairment, moderate cognitive impairment, and severe cognitive impairment). Five hundred fifty-two subjects for model training and 331 for testing (80%:20%).

FIELD STRENGTH/SEQUENCE

3 T, T1-weighted structural (T1w).

ASSESSMENT

The testing cohort was used to evaluate the performance of the model using the Structural Similarity Index Measure (SSIM) and Peak Signal-to-Noise Ratio (PSNR), comparing the likeness of the overall synthetic PET images created from structural MRI with the overall true PET images. SSIM was computed in the overall image to include the luminance, contrast, and structural similarity components. Experienced observers reviewed the images for quality, performance and tried to determine if they could tell the difference between real and synthetic images.

STATISTICAL TESTS

Pixel wise Pearson correlation was significant, and had an R2 greater than 0.96 in example images. From blinded readings, a Pearson Chi-squared test showed that there was no significant difference between the real and synthetic images by the observers (P = 0.68).

RESULTS

A high degree of likeness across the evaluation set, which had a mean SSIM = 0.905 and PSNR = 2.685. The two observers were not able to determine the difference between the real and synthetic images, with accuracies of 54% and 46%, respectively.

CONCLUSION

Amyloid-beta PET images can be synthesized from structural MRI with a high degree of similarity to the real PET images.

EVIDENCE LEVEL

3 TECHNICAL EFFICACY: Stage 1.

Self-Supervised Deep Unrolled Reconstruction Using Regularization by Denoising

Deep learning methods have been successfully used in various computer vision tasks. Inspired by that success, deep learning has been explored in magnetic resonance imaging (MRI) reconstruction. In particular, integrating deep learning and model-based optimization methods has shown considerable advantages. However, a large amount of labeled training data is typically needed for high reconstruction quality, which is challenging for some MRI applications. In this paper, we propose a novel reconstruction method, named DURED-Net, that enables interpretable self-supervised learning for MR image reconstruction by combining a self-supervised denoising network and a plug-and-play method. We aim to boost the reconstruction performance of Noise2Noise in MR reconstruction by adding an explicit prior that utilizes imaging physics. Specifically, the leverage of a denoising network for MRI reconstruction is achieved using Regularization by Denoising (RED). Experiment results demonstrate that the proposed method requires a reduced amount of training data to achieve high reconstruction quality among the state-of-the-art approaches utilizing Noise2Noise.

Artificial intelligence generated content (AIGC) in medicine: A narrative review

Recently, artificial intelligence generated content (AIGC) has been receiving increased attention and is growing exponentially. AIGC is generated based on the intentional information extracted from human-provided instructions by generative artificial intelligence (AI) models. AIGC quickly and automatically generates large amounts of high-quality content. Currently, there is a shortage of medical resources and complex medical procedures in medicine. Due to its characteristics, AIGC can help alleviate these problems. As a result, the application of AIGC in medicine has gained increased attention in recent years. Therefore, this paper provides a comprehensive review on the recent state of studies involving AIGC in medicine. First, we present an overview of AIGC. Furthermore, based on recent studies, the application of AIGC in medicine is reviewed from two aspects: medical image processing and medical text generation. The basic generative AI models, tasks, target organs, datasets and contribution of studies are considered and summarized. Finally, we also discuss the limitations and challenges faced by AIGC and propose possible solutions with relevant studies. We hope this review can help readers understand the potential of AIGC in medicine and obtain some innovative ideas in this field.

Deep-learned generation of renal dual-energy CT from a single-energy scan

AIM

To investigate the role of the deep-learning (DL) method in the generation of dual-energy computed tomography (DECT) images from single-energy images for precise diagnosis of kidney stone type.

MATERIALS AND METHODS

DECT of 23 patients was acquired, and the stone types were investigated based on the DECT software suggestions. The data were divided into two paired groups:120 kVp input and 80 kVp target and 120 kVp input and 135 kVp targets, p2p-UNet-GAN was exploited to generate the different energy images based on the common CT protocols.

RESULTS

The images generated of the generative adversarial network (GAN) network were evaluated based on the SSIM, PSNR, and MSE metrics, and the values were estimated as 0.85-0.95, 28-32, and 0.85-0.89 respectively. The attenuation ratio of test patient images were estimated and compared with real patient reports. The network achieved high accuracy in stone region localisation and resulted in accurate stone type predictions.

CONCLUSION

This study presents a useful method based on the DL technique to reduce patient radiation dose and facilitate the prediction of urinary stone types using single-energy CT imaging.

AI-assisted compressed sensing and parallel imaging sequences for MRI of patients with nasopharyngeal carcinoma: comparison of their capabilities in terms of examination time and image quality

OBJECTIVE

To compare examination time and image quality between artificial intelligence (AI)-assisted compressed sensing (ACS) technique and parallel imaging (PI) technique in MRI of patients with nasopharyngeal carcinoma (NPC).

METHODS

Sixty-six patients with pathologically confirmed NPC underwent nasopharynx and neck examination using a 3.0-T MRI system. Transverse T2-weighted fast spin-echo (FSE) sequence, transverse T1-weighted FSE sequence, post-contrast transverse T1-weighted FSE sequence, and post-contrast coronal T1-weighted FSE were obtained by both ACS and PI techniques, respectively. The signal-to-noise ratio (SNR), contrast-to-noise ratio (CNR), and duration of scanning of both sets of images analyzed by ACS and PI techniques were compared. The images from the ACS and PI techniques were scored for lesion detection, margin sharpness of lesions, artifacts, and overall image quality using the 5-point Likert scale.

RESULTS

The examination time with ACS technique was significantly shorter than that with PI technique (p < 0.0001). The comparison of SNR and CNR showed that ACS technique was significantly superior with PI technique (p < 0.005). Qualitative image analysis showed that the scores of lesion detection, margin sharpness of lesions, artifacts, and overall image quality were higher in the ACS sequences than those in the PI sequences (p < 0.0001). Inter-observer agreement was evaluated for all qualitative indicators for each method, in which the results showed satisfactory-to-excellent agreement (p < 0.0001).

CONCLUSION

Compared with the PI technique, the ACS technique for MR examination of NPC can not only shorten scanning time but also improve image quality.

CLINICAL RELEVANCE STATEMENT

The artificial intelligence (AI)-assisted compressed sensing (ACS) technique shortens examination time for patients with nasopharyngeal carcinoma, while improving the image quality and examination success rate, which will benefit more patients.

KEY POINTS

• Compared with the parallel imaging (PI) technique, the artificial intelligence (AI)-assisted compressed sensing (ACS) technique not only reduced examination time, but also improved image quality. • Artificial intelligence (AI)-assisted compressed sensing (ACS) pulls the state-of-the-art deep learning technique into the reconstruction procedure and helps find an optimal balance of imaging speed and image quality.

Adaptive channel-modulated personalized federated learning for magnetic resonance image reconstruction

Magnetic resonance imaging (MRI) is extensively utilized in clinical practice for diagnostic purposes, owing to its non-invasive nature and remarkable ability to provide detailed characterization of soft tissues. However, its drawback lies in the prolonged scanning time. To accelerate MR imaging, how to reconstruct MR images from under-sampled data quickly and accurately has drawn intensive research interest; it, however, remains a challenging task. While some deep learning models have achieved promising performance in MRI reconstruction, these models usually require a substantial quantity of paired data for training, which proves challenging to gather and share owing to high scanning costs and data privacy concerns. Federated learning (FL) is a potential tool to alleviate these difficulties. It enables multiple clinical clients to collaboratively train a global model without compromising privacy. However, it is extremely challenging to fit a single model to diverse data distributions of different clients. Moreover, existing FL algorithms treat the features of each channel equally, lacking discriminative learning ability across feature channels, and hence hindering their representational capability. In this study, we propose a novel Adaptive Channel-Modulated Federal learning framework for personalized MRI reconstruction, dubbed as ACM-FedMRI. Specifically, considering each local client may focus on features in different channels, we first design a client-specific hypernetwork to guide the channel selection operation in order to optimize the extracted features. Additionally, we introduce a performance-based channel decoupling scheme, which dynamically separates the global model at the channel level to facilitate personalized adjustments based on the performance of individual clients. This approach eliminates the need for heuristic design of specific personalization layers. Extensive experiments on four datasets under two different settings show that our ACM-FedMRI achieves outstanding results compared to other cutting-edge federated learning techniques in the field of MRI reconstruction.

[Application of generative adversarial network in magnetic resonance image reconstruction]

Magnetic resonance imaging (MRI) is an important medical imaging method, whose major limitation is its long scan time due to the imaging mechanism, increasing patients' cost and waiting time for the examination. Currently, parallel imaging (PI) and compress sensing (CS) together with other reconstruction technologies have been proposed to accelerate image acquisition. However, the image quality of PI and CS depends on the image reconstruction algorithms, which is far from satisfying in respect to both the image quality and the reconstruction speed. In recent years, image reconstruction based on generative adversarial network (GAN) has become a research hotspot in the field of magnetic resonance imaging because of its excellent performance. In this review, we summarized the recent development of application of GAN in MRI reconstruction in both single- and multi-modality acceleration, hoping to provide a useful reference for interested researchers. In addition, we analyzed the characteristics and limitations of existing technologies and forecasted some development trends in this field.

Hierarchical Perception Adversarial Learning Framework for Compressed Sensing MRI

The long acquisition time has limited the accessibility of magnetic resonance imaging (MRI) because it leads to patient discomfort and motion artifacts. Although several MRI techniques have been proposed to reduce the acquisition time, compressed sensing in magnetic resonance imaging (CS-MRI) enables fast acquisition without compromising SNR and resolution. However, existing CS-MRI methods suffer from the challenge of aliasing artifacts. This challenge results in the noise-like textures and missing the fine details, thus leading to unsatisfactory reconstruction performance. To tackle this challenge, we propose a hierarchical perception adversarial learning framework (HP-ALF). HP-ALF can perceive the image information in the hierarchical mechanism: image-level perception and patch-level perception. The former can reduce the visual perception difference in the entire image, and thus achieve aliasing artifact removal. The latter can reduce this difference in the regions of the image, and thus recover fine details. Specifically, HP-ALF achieves the hierarchical mechanism by utilizing multilevel perspective discrimination. This discrimination can provide the information from two perspectives (overall and regional) for adversarial learning. It also utilizes a global and local coherent discriminator to provide structure information to the generator during training. In addition, HP-ALF contains a context-aware learning block to effectively exploit the slice information between individual images for better reconstruction performance. The experiments validated on three datasets demonstrate the effectiveness of HP-ALF and its superiority to the comparative methods.

Automated diagnosis of cardiovascular diseases from cardiac magnetic resonance imaging using deep learning models: A review

In recent years, cardiovascular diseases (CVDs) have become one of the leading causes of mortality globally. At early stages, CVDs appear with minor symptoms and progressively get worse. The majority of people experience symptoms such as exhaustion, shortness of breath, ankle swelling, fluid retention, and other symptoms when starting CVD. Coronary artery disease (CAD), arrhythmia, cardiomyopathy, congenital heart defect (CHD), mitral regurgitation, and angina are the most common CVDs. Clinical methods such as blood tests, electrocardiography (ECG) signals, and medical imaging are the most effective methods used for the detection of CVDs. Among the diagnostic methods, cardiac magnetic resonance imaging (CMRI) is increasingly used to diagnose, monitor the disease, plan treatment and predict CVDs. Coupled with all the advantages of CMR data, CVDs diagnosis is challenging for physicians as each scan has many slices of data, and the contrast of it might be low. To address these issues, deep learning (DL) techniques have been employed in the diagnosis of CVDs using CMR data, and much research is currently being conducted in this field. This review provides an overview of the studies performed in CVDs detection using CMR images and DL techniques. The introduction section examined CVDs types, diagnostic methods, and the most important medical imaging techniques. The following presents research to detect CVDs using CMR images and the most significant DL methods. Another section discussed the challenges in diagnosing CVDs from CMRI data. Next, the discussion section discusses the results of this review, and future work in CVDs diagnosis from CMR images and DL techniques are outlined. Finally, the most important findings of this study are presented in the conclusion section.

Medical image super-resolution reconstruction algorithms based on deep learning: A survey

BACKGROUND AND OBJECTIVE

With the high-resolution (HR) requirements of medical images in clinical practice, super-resolution (SR) reconstruction algorithms based on low-resolution (LR) medical images have become a research hotspot. This type of method can significantly improve image SR without improving hardware equipment, so it is of great significance to review it.

METHODS

Aiming at the unique SR reconstruction algorithms in the field of medical images, based on subdivided medical fields such as magnetic resonance (MR) images, computed tomography (CT) images, and ultrasound images. Firstly, we deeply analyzed the research progress of SR reconstruction algorithms, and summarized and compared the different types of algorithms. Secondly, we introduced the evaluation indicators corresponding to the SR reconstruction algorithms. Finally, we prospected the development trend of SR reconstruction technology in the medical field.

RESULTS

The medical image SR reconstruction technology based on deep learning can provide more abundant lesion information, relieve the expert's diagnosis pressure, and improve the diagnosis efficiency and accuracy.

CONCLUSION

The medical image SR reconstruction technology based on deep learning helps to improve the quality of medicine, provides help for the diagnosis of experts, and lays a solid foundation for the subsequent analysis and identification tasks of the computer, which is of great significance for improving the diagnosis efficiency of experts and realizing intelligent medical care.

Synthesis of Prospective Multiple Time Points F-18 FDG PET Images from a Single Scan Using a Supervised Generative Adversarial Network

The cumulative activity map estimation are essential tools for patient specific dosimetry with high accuracy, which is estimated using biokinetic models instead of patient dynamic data or the number of static PET scans, owing to economical and time-consuming points of view. In the era of deep learning applications in medicine, the pix-to-pix (p2 p) GAN neural networks play a significant role in image translation between imaging modalities. In this pilot study, we extended the p2 p GAN networks to generate PET images of patients at different times according to a 60 min scan time after the injection of F-18 FDG. In this regard, the study was conducted in two sections: phantom and patient studies. In the phantom study section, the SSIM, PSNR, and MSE metric results of the generated images varied from 0.98-0.99, 31-34 and 1-2 respectively and the fine-tuned Resnet-50 network classified the different timing images with high performance. In the patient study, these values varied from 0.88-0.93, 36-41 and 1.7-2.2, respectively and the classification network classified the generated images in the true group with high accuracy. The results of phantom studies showed high values of evaluation metrics owing to ideal image quality conditions. However, in the patient study, promising results were achieved which showed that the image quality and training data number affected the network performance. This study aims to assess the feasibility of p2 p GAN network application for different timing image generation.

Multi-channel GAN-based calibration-free diffusion-weighted liver imaging with simultaneous coil sensitivity estimation and reconstruction

Introduction

Diffusion-weighted imaging (DWI) with parallel reconstruction may suffer from a mismatch between the coil calibration scan and imaging scan due to motions, especially for abdominal imaging.

Methods

This study aimed to construct an iterative multichannel generative adversarial network (iMCGAN)-based framework for simultaneous sensitivity map estimation and calibration-free image reconstruction. The study included 106 healthy volunteers and 10 patients with tumors.

Results

The performance of iMCGAN was evaluated in healthy participants and patients and compared with the SAKE, ALOHA-net, and DeepcomplexMRI reconstructions. The peak signal-to-noise ratio (PSNR), structural similarity index measure (SSIM), root mean squared error (RMSE), and histograms of apparent diffusion coefficient (ADC) maps were calculated for assessing image qualities. The proposed iMCGAN outperformed the other methods in terms of the PSNR (iMCGAN: 41.82 ± 2.14; SAKE: 17.38 ± 1.78; ALOHA-net: 20.43 ± 2.11 and DeepcomplexMRI: 39.78 ± 2.78) for b = 800 DWI with an acceleration factor of 4. Besides, the ghosting artifacts in the SENSE due to the mismatch between the DW image and the sensitivity maps were avoided using the iMCGAN model.

Discussion

The current model iteratively refined the sensitivity maps and the reconstructed images without additional acquisitions. Thus, the quality of the reconstructed image was improved, and the aliasing artifact was alleviated when motions occurred during the imaging procedure.

Fast MRI Reconstruction: How Powerful Transformers Are?

Magnetic resonance imaging (MRI) is a widely used non-radiative and non-invasive method for clinical interro-gation of organ structures and metabolism, with an inherently long scanning time. Methods by k-space undersampling and deep learning based reconstruction have been popularised to accelerate the scanning process. This work focuses on investigating how powerful transformers are for fast MRI by exploiting and comparing different novel network architectures. In particular, a generative adversarial network (GAN) based Swin transformer (ST-GAN) was introduced for the fast MRI reconstruction. To further preserve the edge and texture information, edge enhanced GAN based Swin transformer (EES-GAN) and texture enhanced GAN based Swin transformer (TES-GAN) were also developed, where a dual-discriminator GAN structure was applied. We compared our proposed GAN based transformers, standalone Swin transformer and other convolutional neural networks based GAN model in terms of the evaluation metrics PSNR, SSIM and FID. We showed that transformers work well for the MRI reconstruction from different undersampling conditions. The utilisation of GAN's adversarial structure improves the quality of images reconstructed when undersampled for 30% or higher. The code is publicly available at https://github.comJayanglab/SwinGANMR.

Stabilizing deep tomographic reconstruction: Part A. Hybrid framework and experimental results

A recent PNAS paper reveals that several popular deep reconstruction networks are unstable. Specifically, three kinds of instabilities were reported: (1) strong image artefacts from tiny perturbations, (2) small features missed in a deeply reconstructed image, and (3) decreased imaging performance with increased input data. Here, we propose an analytic compressed iterative deep (ACID) framework to address this challenge. ACID synergizes a deep network trained on big data, kernel awareness from compressed sensing (CS)-inspired processing, and iterative refinement to minimize the data residual relative to real measurement. Our study demonstrates that the ACID reconstruction is accurate, is stable, and sheds light on the converging mechanism of the ACID iteration under a bounded relative error norm assumption. ACID not only stabilizes an unstable deep reconstruction network but also is resilient against adversarial attacks to the whole ACID workflow, being superior to classic sparsity-regularized reconstruction and eliminating the three kinds of instabilities.

A review and experimental evaluation of deep learning methods for MRI reconstruction

Following the success of deep learning in a wide range of applications, neural network-based machine-learning techniques have received significant interest for accelerating magnetic resonance imaging (MRI) acquisition and reconstruction strategies. A number of ideas inspired by deep learning techniques for computer vision and image processing have been successfully applied to nonlinear image reconstruction in the spirit of compressed sensing for accelerated MRI. Given the rapidly growing nature of the field, it is imperative to consolidate and summarize the large number of deep learning methods that have been reported in the literature, to obtain a better understanding of the field in general. This article provides an overview of the recent developments in neural-network based approaches that have been proposed specifically for improving parallel imaging. A general background and introduction to parallel MRI is also given from a classical view of k-space based reconstruction methods. Image domain based techniques that introduce improved regularizers are covered along with k-space based methods which focus on better interpolation strategies using neural networks. While the field is rapidly evolving with plenty of papers published each year, in this review, we attempt to cover broad categories of methods that have shown good performance on publicly available data sets. Limitations and open problems are also discussed and recent efforts for producing open data sets and benchmarks for the community are examined.

Edge-enhanced dual discriminator generative adversarial network for fast MRI with parallel imaging using multi-view information

In clinical medicine, magnetic resonance imaging (MRI) is one of the most important tools for diagnosis, triage, prognosis, and treatment planning. However, MRI suffers from an inherent slow data acquisition process because data is collected sequentially in k-space. In recent years, most MRI reconstruction methods proposed in the literature focus on holistic image reconstruction rather than enhancing the edge information. This work steps aside this general trend by elaborating on the enhancement of edge information. Specifically, we introduce a novel parallel imaging coupled dual discriminator generative adversarial network (PIDD-GAN) for fast multi-channel MRI reconstruction by incorporating multi-view information. The dual discriminator design aims to improve the edge information in MRI reconstruction. One discriminator is used for holistic image reconstruction, whereas the other one is responsible for enhancing edge information. An improved U-Net with local and global residual learning is proposed for the generator. Frequency channel attention blocks (FCA Blocks) are embedded in the generator for incorporating attention mechanisms. Content loss is introduced to train the generator for better reconstruction quality. We performed comprehensive experiments on Calgary-Campinas public brain MR dataset and compared our method with state-of-the-art MRI reconstruction methods. Ablation studies of residual learning were conducted on the MICCAI13 dataset to validate the proposed modules. Results show that our PIDD-GAN provides high-quality reconstructed MR images, with well-preserved edge information. The time of single-image reconstruction is below 5ms, which meets the demand of faster processing.

CryoETGAN: Cryo-Electron Tomography Image Synthesis via Unpaired Image Translation

Cryo-electron tomography (Cryo-ET) has been regarded as a revolution in structural biology and can reveal molecular sociology. Its unprecedented quality enables it to visualize cellular organelles and macromolecular complexes at nanometer resolution with native conformations. Motivated by developments in nanotechnology and machine learning, establishing machine learning approaches such as classification, detection and averaging for Cryo-ET image analysis has inspired broad interest. Yet, deep learning-based methods for biomedical imaging typically require large labeled datasets for good results, which can be a great challenge due to the expense of obtaining and labeling training data. To deal with this problem, we propose a generative model to simulate Cryo-ET images efficiently and reliably: CryoETGAN. This cycle-consistent and Wasserstein generative adversarial network (GAN) is able to generate images with an appearance similar to the original experimental data. Quantitative and visual grading results on generated images are provided to show that the results of our proposed method achieve better performance compared to the previous state-of-the-art simulation methods. Moreover, CryoETGAN is stable to train and capable of generating plausibly diverse image samples.

Updates in deep learning research in ophthalmology

Ophthalmology has been one of the early adopters of artificial intelligence (AI) within the medical field. Deep learning (DL), in particular, has garnered significant attention due to the availability of large amounts of data and digitized ocular images. Currently, AI in Ophthalmology is mainly focused on improving disease classification and supporting decision-making when treating ophthalmic diseases such as diabetic retinopathy, age-related macular degeneration (AMD), glaucoma and retinopathy of prematurity (ROP). However, most of the DL systems (DLSs) developed thus far remain in the research stage and only a handful are able to achieve clinical translation. This phenomenon is due to a combination of factors including concerns over security and privacy, poor generalizability, trust and explainability issues, unfavorable end-user perceptions and uncertain economic value. Overcoming this challenge would require a combination approach. Firstly, emerging techniques such as federated learning (FL), generative adversarial networks (GANs), autonomous AI and blockchain will be playing an increasingly critical role to enhance privacy, collaboration and DLS performance. Next, compliance to reporting and regulatory guidelines, such as CONSORT-AI and STARD-AI, will be required to in order to improve transparency, minimize abuse and ensure reproducibility. Thirdly, frameworks will be required to obtain patient consent, perform ethical assessment and evaluate end-user perception. Lastly, proper health economic assessment (HEA) must be performed to provide financial visibility during the early phases of DLS development. This is necessary to manage resources prudently and guide the development of DLS.

MAGnitude-Image-to-Complex K-space (MAGIC-K) Net: A Data Augmentation Network for Image Reconstruction

Deep learning has demonstrated superior performance in image reconstruction compared to most conventional iterative algorithms. However, their effectiveness and generalization capability are highly dependent on the sample size and diversity of the training data. Deep learning-based reconstruction requires multi-coil raw k-space data, which are not collected by routine scans. On the other hand, large amounts of magnitude images are readily available in hospitals. Hence, we proposed the MAGnitude Images to Complex K-space (MAGIC-K) Net to generate multi-coil k-space data from existing magnitude images and a limited number of required raw k-space data to facilitate the reconstruction. Compared to some basic data augmentation methods applying global intensity and displacement transformations to the source images, the MAGIC-K Net can generate more realistic intensity variations and displacements from pairs of anatomical Digital Imaging and Communications in Medicine (DICOM) images. The reconstruction performance was validated in 30 healthy volunteers and 6 patients with different types of tumors. The experimental results demonstrated that the high-resolution Diffusion Weighted Image (DWI) reconstruction benefited from the proposed augmentation method. The MAGIC-K Net enabled the deep learning network to reconstruct images with superior performance in both healthy and tumor patients, qualitatively and quantitatively.

Semi-supervised CycleGAN for domain transformation of chest CT images and its application to opacity classification of diffuse lung diseases

Purpose

The performance of deep learning may fluctuate depending on the imaging devices and settings. Although domain transformation such as CycleGAN for normalizing images is useful, CycleGAN does not use information on the disease classes. Therefore, we propose a semi-supervised CycleGAN with an additional classification loss to transform images suitable for the diagnosis. The method is evaluated by opacity classification of chest CT.

Methods

(1) CT images taken at two hospitals (source and target domains) are used. (2) A classifier is trained on the target domain. (3) Class labels are given to a small number of source domain images for semi-supervised learning. (4) The source domain images are transformed to the target domain. (5) A classification loss of the transformed images with class labels is calculated.

Results

The proposed method showed an F-measure of 0.727 in the domain transformation from hospital A to B, and 0.745 in that from hospital B to A, where significant differences are between the proposed method and the other three methods.

Conclusions

The proposed method not only transforms the appearance of the images but also retains the features being important to classify opacities, and shows the best precision, recall, and F-measure.

Simultaneous image reconstruction and lesion segmentation in accelerated MRI using multitasking learning

PURPOSE

Magnetic resonance imaging (MRI) serves as an important medical imaging modality for a variety of clinical applications. However, the problem of long imaging time limited its wide usage. In addition, prolonged scan time will cause discomfort to the patient, leading to severe image artifacts. On the other hand, manually lesion segmentation is time consuming. Algorithm-based automatic lesion segmentation is still challenging, especially for accelerated imaging with low quality.

METHODS

In this paper, we proposed a multitask learning-based method to perform image reconstruction and lesion segmentation simultaneously, called "RecSeg". Our hypothesis is that both tasks can benefit from the usage of the proposed combined model. In the experiment, we validated the proposed multitask model on MR k-space data with different acceleration factors (2×, 4×, and 6×). Two connected U-nets were used for the tasks of liver and renal image reconstruction and segmentation. A total of 50 healthy subjects and 100 patients with hepatocellular carcinoma were included for training and testing. For the segmentation part, we use healthy subjects to verify organ segmentation, and hepatocellular carcinoma patients to verify lesion segmentation. The organs and lesions were manually contoured by an experienced radiologist.

RESULTS

Experimental results show that the proposed RecSeg yielded the highest PSNR (RecSeg: 32.39 ± 1.64 vs. KSVD: 29.53 ± 2.74 and single U-net: 31.18 ± 1.68, respectively, p < 0.05) and highest structural similarity index measure (SSIM) (RecSeg: 0.93 ± 0.01 vs. KSVD: 0.88 ± 0.02 and single U-net: 0.90 ± 0.01, respectively, p < 0.05) under 6× acceleration. Moreover, in the task of lesion segmentation, it is proposed that RecSeg produced the highest Dice score (RecSeg: 0.86 ± 0.01 vs. KSVD: 0.82 ± 0.01 and single U-net: 0.84 ± 0.01, respectively, p < 0.05).

CONCLUSIONS

This study focused on the simultaneous reconstruction of medical images and the segmentation of organs and lesions. It is observed that the multitask learning-based method can improve performances of both image reconstruction and lesion segmentation.

Current status of deep learning in abdominal image reconstruction

Abdominal magnetic resonance imaging (MRI) and computed tomography (CT) are commonly used for disease screening, diagnosis, and treatment guidance. However, abdominal MRI has disadvantages including slow speed and vulnerability to motions, while CT suffers from problems of radiation. It has been reported that deep learning reconstruction can solve such problems while maintaining good image quality. Recently, deep learning-based image reconstruction has become a hot topic in the field of medical imaging. This study reviews the latest research on deep learning reconstruction in abdominal imaging, including the widely used convolutional neural network, generative adversarial network, and recurrent neural network.