Generative Adversarial Networks for Noise Reduction in Low-Dose CT

Development of a denoising convolutional neural network-based algorithm for metal artifact reduction in digital tomosynthesis for arthroplasty: A phantom study

The present study aimed to develop a denoising convolutional neural network metal artifact reduction hybrid reconstruction (DnCNN-MARHR) algorithm for decreasing metal objects in digital tomosynthesis (DT) for arthroplasty by using projection data. For metal artifact reduction (MAR), we implemented a DnCNN-MARHR algorithm based on a training network (mini-batch stochastic gradient descent algorithm with momentum) to estimate the residual reference (140 keV virtual monochromatic [VM]) and object (70 kV with metal artifacts) images. For this, we used projection data and subtracted the estimated residual images from the object images, involving hybrid and subjectively reconstructed image usage (back projection and maximum likelihood expectation maximization [MLEM]). The DnCNN-MARHR algorithm was compared with the dual-energy material decomposition reconstruction algorithm (DEMDRA), VM, MLEM, established and commonly used filtered back projection (FBP), and a simultaneous algebraic reconstruction technique-total variation (SART-TV) with MAR processing. MAR was compared using artifact index (AI) and texture analysis. Artifact spread functions (ASFs) for images that were out-of-plane and in-focus were evaluated using a prosthesis phantom. The overall performance of the DnCNN-MARHR algorithm was adequate with regard to the ASF, and the derived images showed better results, without being influenced by the metal type (AI was almost equal to the best value for the DEMDRA). In the ASF analysis, the DnCNN-MARHR algorithm generated better MAR compared with that obtained employing usual algorithms for reconstruction using MAR processing. In addition, comparison of the difference (mean square error) between DnCNN-MARHR and the conventional algorithm resulted in the smallest VM. The DnCNN-MARHR algorithm showed the best performance with regard to image homogeneity in the texture analysis. The proposed algorithm is particularly useful for reducing artifacts in the longitudinal direction, and it is not affected by tissue misclassification.

Efficient skin lesion segmentation using separable-Unet with stochastic weight averaging

BACKGROUND AND OBJECTIVE

Efficient segmentation of skin lesion in dermoscopy images can improve the classification accuracy of skin diseases, which provides a powerful approach for the dermatologists in examining pigmented skin lesions. However, the segmentation is challenging due to the low contrast of skin lesions from a captured image, fuzzy and indistinct lesion boundaries, huge variety of interclass variation of melanomas, the existence of artifacts, etc. In this work, an efficient and accurate melanoma region segmentation method is proposed for computer-aided diagnostic systems.

METHOD

A skin lesion segmentation (SLS) method based on the separable-Unet with stochastic weight averaging is proposed in this work. Specifically, the proposed Separable-Unet framework takes advantage of the separable convolutional block and U-Net architectures, which can extremely capture the context feature channel correlation and higher semantic feature information to enhance the pixel-level discriminative representation capability of fully convolutional networks (FCN). Further, considering that the over-fitting is a local optimum (or sub-optimum) problem, a scheme based on stochastic weight averaging is introduced, which can obtain much broader optimum and better generalization.

RESULTS

The proposed method is evaluated in three publicly available datasets. The experimental results showed that the proposed approach segmented the skin lesions with an average Dice coefficient of 93.03% and Jaccard index of 89.25% for the International Skin Imaging Collaboration (ISIC) 2016 Skin Lesion Challenge (SLC) dataset, 86.93% and 79.26% for the ISIC 2017 SLC, and 94.13% and 89.40% for the PH2 dataset, respectively. The proposed approach is compared with other state-of-the-art methods, and the results demonstrate that the proposed approach outperforms them for SLS on both melanoma and non-melanoma cases. Segmentation of a potential lesion with the proposed approach in a dermoscopy image requires less than 0.05 s of processing time, which is roughly 30 times faster than the second best method (regarding the value of Jaccard index) for the ISIC 2017 dataset with the same hardware configuration.

CONCLUSIONS

We concluded that using the separable convolutional block and U-Net architectures with stochastic weight averaging strategy could enable to obtain better pixel-level discriminative representation capability. Moreover, the considerably decreased computation time suggests that the proposed approach has potential for practical computer-aided diagnose systems, besides provides a segmentation for the specific analysis with improved segmentation performance.

A performance comparison of convolutional neural network-based image denoising methods: The effect of loss functions on low-dose CT images

PURPOSE

Convolutional neural network (CNN)-based image denoising techniques have shown promising results in low-dose CT denoising. However, CNN often introduces blurring in denoised images when trained with a widely used pixel-level loss function. Perceptual loss and adversarial loss have been proposed recently to further improve the image denoising performance. In this paper, we investigate the effect of different loss functions on image denoising performance using task-based image quality assessment methods for various signals and dose levels.

METHODS

We used a modified version of U-net that was effective at reducing the correlated noise in CT images. The loss functions used for comparison were two pixel-level losses (i.e., the mean-squared error and the mean absolute error), Visual Geometry Group network-based perceptual loss (VGG loss), adversarial loss used to train the Wasserstein generative adversarial network with gradient penalty (WGAN-GP), and their weighted summation. Each image denoising method was applied to reconstructed images and sinogram images independently and validated using the extended cardiac-torso (XCAT) simulation and Mayo Clinic datasets. In the XCAT simulation, we generated fan-beam CT datasets with four different dose levels (25%, 50%, 75%, and 100% of a normal-dose level) using 10 XCAT phantoms and inserted signals in a test set. The signals had two different shapes (spherical and spiculated), sizes (4 and 12 mm), and contrast levels (60 and 160 HU). To evaluate signal detectability, we used a detection task SNR (tSNR) calculated from a non-prewhitening model observer with an eye filter. We also measured the noise power spectrum (NPS) and modulation transfer function (MTF) to compare the noise and signal transfer properties.

RESULTS

Compared to CNNs without VGG loss, VGG-loss-based CNNs achieved a more similar tSNR to that of the normal-dose CT for all signals at different dose levels except for a small signal at the 25% dose level. For a low-contrast signal at 25% or 50% dose, adding other losses to the VGG loss showed more improved performance than only using VGG loss. The NPS shapes from VGG-loss-based CNN closely matched that of normal-dose CT images while CNN without VGG loss overly reduced the mid-high-frequency noise power at all dose levels. MTF also showed VGG-loss-based CNN with better-preserved high resolution for all dose and contrast levels. It is also observed that additional WGAN-GP loss helps improve the noise and signal transfer properties of VGG-loss-based CNN.

CONCLUSIONS

The evaluation results using tSNR, NPS, and MTF indicate that VGG-loss-based CNNs are more effective than those without VGG loss for natural denoising of low-dose images and WGAN-GP loss improves the denoising performance of VGG-loss-based CNNs, which corresponds with the qualitative evaluation.

Generative adversarial network in medical imaging: A review

Generative adversarial networks have gained a lot of attention in the computer vision community due to their capability of data generation without explicitly modelling the probability density function. The adversarial loss brought by the discriminator provides a clever way of incorporating unlabeled samples into training and imposing higher order consistency. This has proven to be useful in many cases, such as domain adaptation, data augmentation, and image-to-image translation. These properties have attracted researchers in the medical imaging community, and we have seen rapid adoption in many traditional and novel applications, such as image reconstruction, segmentation, detection, classification, and cross-modality synthesis. Based on our observations, this trend will continue and we therefore conducted a review of recent advances in medical imaging using the adversarial training scheme with the hope of benefiting researchers interested in this technique.

Gradient regularized convolutional neural networks for low-dose CT image enhancement

The potential risks of x-ray to patients have transferred the public's attention from normal dose CT (NDCT) to low-dose CT (LDCT). However, simply lowering the radiation dose of the CT system will significantly degrade the quality of CT images such as noise and artifacts, which compromises the diagnostic performance. Hence, various methods have been proposed to solve this problem over the past decades. Although these methods have achieved impressive results, they also suffer from a drawback of smoothing image details after denoising, which makes it difficult for clinical diagnosis and treatment. To address this issue, this paper introduces a novel gradient regularization method for LDCT enhancement. Rather than common methods which only consider the pixel-wise gray value loss in the reconstruction procedure, we also take the image gradient loss into consideration to preserve image details. By combining the gradient regularization method and the convolutional neural network (CNN) framework, a gradient regularized convolutional neural network (GRCNN) is proposed to enhance LDCT images which has achieved promising performance in our experiments both visually and quantitatively.

Applications of Deep Learning to Neuro-Imaging Techniques

Many clinical applications based on deep learning and pertaining to radiology have been proposed and studied in radiology for classification, risk assessment, segmentation tasks, diagnosis, prognosis, and even prediction of therapy responses. There are many other innovative applications of AI in various technical aspects of medical imaging, particularly applied to the acquisition of images, ranging from removing image artifacts, normalizing/harmonizing images, improving image quality, lowering radiation and contrast dose, and shortening the duration of imaging studies. This article will address this topic and will seek to present an overview of deep learning applied to neuroimaging techniques.

Artificial intelligence in cardiovascular imaging: state of the art and implications for the imaging cardiologist

Healthcare, conceivably more than any other area of human endeavour, has the greatest potential to be affected by artificial intelligence (AI). This potential has been shown by several reports that demonstrate equal or superhuman performance in medical tasks that aim to improve efficiency, diagnosis and prognosis. This review focuses on the state of the art of AI applications in cardiovascular imaging. It provides an overview of the current applications and studies performed, including the potential value, implications, limitations and future directions of AI in cardiovascular imaging.It is envisioned that AI will dramatically change the way doctors practise medicine. In the short term, it will assist physicians with easy tasks, such as automating measurements, making predictions based on big data, and putting clinical findings into an evidence-based context. In the long term, AI will not only assist doctors, it has the potential to significantly improve access to health and well-being data for patients and their caretakers. This empowers patients. From a physician's perspective, reliable AI assistance will be available to support clinical decision-making. Although cardiovascular studies implementing AI are increasing in number, the applications have only just started to penetrate contemporary clinical care.

Competitive performance of a modularized deep neural network compared to commercial algorithms for low-dose CT image reconstruction

Commercial iterative reconstruction techniques help to reduce CT radiation dose but altered image appearance and artifacts limit their adoptability and potential use. Deep learning has been investigated for low-dose CT (LDCT). Here we design a modularized neural network for LDCT and compared it with commercial iterative reconstruction methods from three leading CT vendors. While popular networks are trained for an end-to-end mapping, our network performs an end-to-process mapping so that intermediate denoised images are obtained with associated noise reduction directions towards a final denoised image. The learned workflow allows radiologists-in-the-loop to optimize the denoising depth in a task-specific fashion. Our network was trained with the Mayo LDCT Dataset, and tested on separate chest and abdominal CT exams from Massachusetts General Hospital. The best deep learning reconstructions were systematically compared to the best iterative reconstructions in a double-blinded reader study. This study confirms that our deep learning approach performed either favorably or comparably in terms of noise suppression and structural fidelity, and is much faster than the commercial iterative reconstruction algorithms.

Projection-domain scatter correction for cone beam computed tomography using a residual convolutional neural network

PURPOSE

Scatter is a major factor degrading the image quality of cone beam computed tomography (CBCT). Conventional scatter correction strategies require handcrafted analytical models with ad hoc assumptions, which often leads to less accurate scatter removal. This study aims to develop an effective scatter correction method using a residual convolutional neural network (CNN).

METHODS

A U-net based 25-layer CNN was constructed for CBCT scatter correction. The establishment of the model consists of three steps: model training, validation, and testing. For model training, a total of 1800 pairs of x-ray projection and the corresponding scatter-only distribution in nonanthropomorphic phantoms taken in full-fan scan were generated using Monte Carlo simulation of a CBCT scanner installed with a proton therapy system. An end-to-end CNN training was implemented with two major loss functions for 100 epochs with a mini-batch size of 10. Image rotations and flips were randomly applied to augment the training datasets during training. For validation, 200 projections of a digital head phantom were collected. The proposed CNN-based method was compared to a conventional projection-domain scatter correction method named fast adaptive scatter kernel superposition (fASKS) method using 360 projections of an anthropomorphic head phantom. Two different loss functions were applied for the same CNN to evaluate the impact of loss functions on the final results. Furthermore, the CNN model trained with full-fan projections was fine-tuned for scatter correction in half-fan scan by using transfer learning with additional 360 half-fan projection pairs of nonanthropomorphic phantoms. The tuned-CNN model for half-fan scan was compared with the fASKS method as well as the CNN-based method without the fine-tuning using additional lung phantom projections.

RESULTS

The CNN-based method provides projections with significantly reduced scatter and CBCT images with more accurate Hounsfield Units (HUs) than that of the fASKS-based method. Root mean squared error of the CNN-corrected projections was improved to 0.0862 compared to 0.278 for uncorrected projections or 0.117 for the fASKS-corrected projections. The CNN-corrected reconstruction provided better HU quantification, especially in regions near the air or bone interfaces. All four image quality measures, which include mean absolute error (MAE), mean squared error (MSE), peak signal-to-noise ratio (PSNR), and structural similarity (SSIM), indicated that the CNN-corrected images were significantly better than that of the fASKS-corrected images. Moreover, the proposed transfer learning technique made it possible for the CNN model trained with full-fan projections to be applicable to remove scatters in half-fan projections after fine-tuning with only a small number of additional half-fan training datasets. SSIM value of the tuned-CNN-corrected images was 0.9993 compared to 0.9984 for the non-tuned-CNN-corrected images or 0.9990 for the fASKS-corrected images. Finally, the CNN-based method is computationally efficient - the correction time for the 360 projections only took less than 5 s in the reported experiments on a PC (4.20 GHz Intel Core-i7 CPU) with a single NVIDIA GTX 1070 GPU.

CONCLUSIONS

The proposed deep learning-based method provides an effective tool for CBCT scatter correction and holds significant value for quantitative imaging and image-guided radiation therapy.

ADMM-based deep reconstruction for limited-angle CT

In the real applications of computed tomography (CT) imaging, the projection data of the scanned objects are usually acquired within a limited-angle range because of the limitation of the scanning condition. Under these circumstances, conventional analytical algorithms, such as filtered back-projection (FBP), do not work because the projection data are incomplete. The regularization method has proven to be effective for tomographic reconstruction from under-sampled measurements. To deal with the limited-angle CT reconstruction problem, the regularization method is commonly used, but it is difficult to find a generic regularization term and choose the regularization parameters. Moreover, in some cases, the quality of reconstructed images is less than satisfactory. To solve this problem, we developed an alternating direction method of multipliers (ADMM)-based deep reconstruction (ADMMBDR) algorithm for limited-angle CT. First, we used the ADMM algorithm to decompose a regularization reconstruction model. Then, we utilized a deep convolutional neural network (CNN) to replace a part of the ADMM algorithm to reduce artifacts and avoid the choice of the regularization term and the regularization parameter. Furthermore, we conducted some numerical experiments to evaluate the feasibility and the advantages of the proposed algorithm. The results showed that the proposed algorithm had a better performance than several state-of-the-art algorithms; with respect to structure preservation and artifact reduction.

Synthesizing Chest X-Ray Pathology for Training Deep Convolutional Neural Networks

Medical datasets are often highly imbalanced with over-representation of prevalent conditions and poor representation of rare medical conditions. Due to privacy concerns, it is challenging to aggregate large datasets between health care institutions. We propose synthesizing pathology in medical images as a means to overcome these challenges. We implement a deep convolutional generative adversarial network (DCGAN) to create synthesized chest X-rays based upon a modest sized labeled dataset. We used a combination of real and synthesized images to train deep convolutional neural networks (DCNNs) to detect pathology across five classes of chest X-rays. The comparative study of DCNNs trained with the combination of real and synthesized images showed that these networks can outperform similar networks trained solely with real images in pathology classification. This improved performance is largely attributable to the balancing of the dataset using DCGAN synthesized images, where classes that are lacking in example images are preferentially augmented.

Image reconstruction in cardiovascular CT: Part 2 - Iterative reconstruction; potential and pitfalls

The use of IR in CT previously has been prohibitively complicated and time consuming, however improvements in computer processing power now make it possible on almost all CT scanners. Due to its potential to allow scanning at lower doses, IR has received a lot of attention in the medical literature and has become a successful commercial product. Its use in cardiovascular CT has been driven in part due to concerns about radiation dose and image quality. This manuscript discusses the various vendor permutations of iterative reconstruction (IR) in detail and critically appraises the current clinical research available on the various IR techniques used in cardiovascular CT.

Preliminary results of DSA denoising based on a weighted low-rank approach using an advanced neurovascular replication system

PURPOSE

2D digital subtraction angiography (DSA) has become an important technique for interventional neuroradiology tasks, such as detection and subsequent treatment of aneurysms. In order to provide high-quality DSA images, usually undiluted contrast agent and a high X-ray dose are used. The iodinated contrast agent puts a burden on the patients' kidneys while the use of high-dose X-rays expose both patients and medical staff to a considerable amount of radiation. Unfortunately, reducing either the X-ray dose or the contrast agent concentration usually results in a sacrifice of image quality.

MATERIALS AND METHODS

To denoise a frame, the proposed spatiotemporal denoising method utilizes the low-rank nature of a spatially aligned temporal sequence where variation is introduced by the flow of contrast agent through a vessel tree of interest. That is, a constrained weighted rank-1 approximation of the stack comprising the frame to be denoised and its temporal neighbors is computed where the weights are used to prevent the contribution of non-similar pixels toward the low-rank approximation. The method has been evaluated using a vascular flow phantom emulating cranial arteries into which contrast agent can be manually injected (Vascular Simulations Replicator, Vascular Simulations, Stony Brook NY, USA). For the evaluation, image sequences acquired at different dose levels as well as different contrast agent concentrations have been used.

RESULTS

Qualitative and quantitative analyses have shown that with the proposed approach, the dose and the concentration of the contrast agent could both be reduced by about 75%, while maintaining the required image quality. Most importantly, it has been observed that the DSA images obtained using the proposed method have the closest resemblance to typical DSA images, i.e., they preserve the typical image characteristics best.

CONCLUSION

Using the proposed denoising approach, it is possible to improve the image quality of low-dose DSA images. This improvement could enable both a reduction in contrast agent and radiation dose when acquiring DSA images, thereby benefiting patients as well as clinicians. Since the resulting images are free from artifacts and as the inherent characteristics of the images are also preserved, the proposed method seems to be well suited for clinical images as well.

Deep Learning in the Biomedical Applications: Recent and Future Status

Deep neural networks represent, nowadays, the most effective machine learning technology in biomedical domain. In this domain, the different areas of interest concern the Omics (study of the genome—genomics—and proteins—transcriptomics, proteomics, and metabolomics), bioimaging (study of biological cell and tissue), medical imaging (study of the human organs by creating visual representations), BBMI (study of the brain and body machine interface) and public and medical health management (PmHM). This paper reviews the major deep learning concepts pertinent to such biomedical applications. Concise overviews are provided for the Omics and the BBMI. We end our analysis with a critical discussion, interpretation and relevant open challenges.

Automatic multiorgan segmentation in thorax CT images using U-net-GAN

PURPOSE

Accurate and timely organs-at-risk (OARs) segmentation is key to efficient and high-quality radiation therapy planning. The purpose of this work is to develop a deep learning-based method to automatically segment multiple thoracic OARs on chest computed tomography (CT) for radiotherapy treatment planning.

METHODS

We propose an adversarial training strategy to train deep neural networks for the segmentation of multiple organs on thoracic CT images. The proposed design of adversarial networks, called U-Net-generative adversarial network (U-Net-GAN), jointly trains a set of U-Nets as generators and fully convolutional networks (FCNs) as discriminators. Specifically, the generator, composed of U-Net, produces an image segmentation map of multiple organs by an end-to-end mapping learned from CT image to multiorgan-segmented OARs. The discriminator, structured as an FCN, discriminates between the ground truth and segmented OARs produced by the generator. The generator and discriminator compete against each other in an adversarial learning process to produce the optimal segmentation map of multiple organs. Our segmentation results were compared with manually segmented OARs (ground truth) for quantitative evaluations in geometric difference, as well as dosimetric performance by investigating the dose-volume histogram in 20 stereotactic body radiation therapy (SBRT) lung plans.

RESULTS

This segmentation technique was applied to delineate the left and right lungs, spinal cord, esophagus, and heart using 35 patients' chest CTs. The averaged dice similarity coefficient for the above five OARs are 0.97, 0.97, 0.90, 0.75, and 0.87, respectively. The mean surface distance of the five OARs obtained with proposed method ranges between 0.4 and 1.5 mm on average among all 35 patients. The mean dose differences on the 20 SBRT lung plans are -0.001 to 0.155 Gy for the five OARs.

CONCLUSION

We have investigated a novel deep learning-based approach with a GAN strategy to segment multiple OARs in the thorax using chest CT images and demonstrated its feasibility and reliability. This is a potentially valuable method for improving the efficiency of chest radiotherapy treatment planning.

Iterative PET Image Reconstruction Using Convolutional Neural Network Representation

PET image reconstruction is challenging due to the ill-poseness of the inverse problem and limited number of detected photons. Recently, the deep neural networks have been widely and successfully used in computer vision tasks and attracted growing interests in medical imaging. In this paper, we trained a deep residual convolutional neural network to improve PET image quality by using the existing inter-patient information. An innovative feature of the proposed method is that we embed the neural network in the iterative reconstruction framework for image representation, rather than using it as a post-processing tool. We formulate the objective function as a constrained optimization problem and solve it using the alternating direction method of multipliers algorithm. Both simulation data and hybrid real data are used to evaluate the proposed method. Quantification results show that our proposed iterative neural network method can outperform the neural network denoising and conventional penalized maximum likelihood methods.

PET Image Denoising Using a Deep Neural Network Through Fine Tuning

Positron emission tomography (PET) is a functional imaging modality widely used in clinical diagnosis. In this work, we trained a deep convolutional neural network (CNN) to improve PET image quality. Perceptual loss based on features derived from a pre-trained VGG network, instead of the conventional mean squared error, was employed as the training loss function to preserve image details. As the number of real patient data set for training is limited, we propose to pre-train the network using simulation data and fine-tune the last few layers of the network using real data sets. Results from simulation, real brain and lung data sets show that the proposed method is more effective in removing noise than the traditional Gaussian filtering method.

An analytical approach for the simulation of realistic low-dose fluoroscopic images

PURPOSE

The quality of X-ray images plays an important role in computer-assisted interventions. Although learning-based denoising techniques have been shown to be successful in improving the image quality, they often rely on pairs of associated low- and high-dose X-ray images that are usually not possible to acquire at different dose levels in a clinical scenario. Moreover, since data variation is an important requirement for learning-based methods, the use of phantom data alone may not be sufficient. A possibility to address this issue is a realistic simulation of low-dose images from their related high-dose counterparts.

METHOD

We introduce a novel noise simulation method based on an X-ray image formation model. The method makes use of the system parameters associated with low- and high-dose X-ray image acquisitions, such as system gain and electronic noise, to preserve the image noise characteristics of low-dose images.

RESULTS

We have compared several corresponding regions of the associated real and simulated low-dose images-obtained from two different imaging systems-visually as well as statistically, using a two-sample Kolmogorov-Smirnov test at 5% significance. In addition to being visually similar, the hypothesis that the corresponding regions-from 80 pairs of real and simulated low-dose regions-belonging to the same distribution has been accepted in 81.43% of the cases.

CONCLUSION

The results suggest that the simulated low-dose images obtained using the proposed method are almost indistinguishable from real low-dose images. Since extensive calibration procedures required in previous methods can be avoided using the proposed approach, it allows an easy adaptation to different X-ray imaging systems. This in turn leads to an increased diversity of the training data for potential learning-based methods.

[Super-resolution construction of intravascular ultrasound images using generative adversarial networks]

The low-resolution ultrasound images have poor visual effects. Herein we propose a method for generating clearer intravascular ultrasound images based on super-resolution reconstruction combined with generative adversarial networks. We used the generative adversarial networks to generate the images by a generator and to estimate the authenticity of the images by a discriminator. Specifically, the low-resolution image was passed through the sub-pixel convolution layer r²-feature channels to generate r²-feature maps in the same size, followed by realignment of the corresponding pixels in each feature map into r ×r sub-blocks, which corresponded to the sub-block in a high-resolution image; after amplification, an image with a r²-time resolution was generated. The generative adversarial networks can obtain a clearer image through continuous optimization. We compared the method (SRGAN) with other methods including Bicubic, super-resolution convolutional network (SRCNN) and efficient sub-pixel convolutional network (ESPCN), and the proposed method resulted in obvious improvements in the peak signal-to-noise ratio (PSNR) by 2.369 dB and in structural similarity index by 1.79% to enhance the diagnostic visual effects of intravascular ultrasound images.

Comparing Unet training with three different datasets to correct CBCT images for prostate radiotherapy dose calculations

Image intensity correction is crucial to enable cone beam computed tomography (CBCT) based radiotherapy dose calculations. This study evaluated three different deep learning based correction methods using a U-shaped convolutional neural network architecture (Unet) in terms of their photon and proton dose calculation accuracy. CT and CBCT imaging data of 42 prostate cancer patients were included. For target ground truth data generation, a CBCT correction method based on CT to CBCT deformable image registration (DIR) was used. The method yields a deformed CT called (i) virtual CT (vCT) which is used to generate (ii) corrected CBCT projections allowing the reconstruction of (iii) a final corrected CBCT image. The single Unet architecture was trained using these three different datasets: (Unet1) raw and corrected CBCT projections, (Unet2) raw CBCT and vCT image slices and (Unet3) raw and reference corrected CBCT image slices. Volumetric arc therapy (VMAT) and proton pencil beam scanning (PBS) single field uniform dose (SFUD) plans were optimized on the reference corrected image and recalculated on the obtained Unet-corrected CBCT images. The mean error (ME) and mean absolute error (MAE) for Unet1/2/3 were [Formula: see text] Hounsfield units (HU) and [Formula: see text] HU. The 1% dose difference pass rates were better than 98.4% for VMAT for 8 test patients not seen during training, with little difference between Unets. Gamma evaluation results were even better. For protons a gamma evaluation was employed to account for small range shifts, and [Formula: see text] mm pass rates for Unet1/2/3 were better than [Formula: see text] and 91%. A 3 mm range difference threshold was established. Only for Unet3 the 5th and 95th percentiles of the range difference distributions over all fields, test patients and dose profiles were within this threshold. A single Unet architecture was successfully trained using both CBCT projections and CBCT image slices. Since the results of the other Unets were poorer than Unet3, we conclude that training using corrected CBCT image slices as target data is optimal for PBS SFUD proton dose calculations, while for VMAT all Unets provided sufficient accuracy.

Improving the Quality of Synthetic FLAIR Images with Deep Learning Using a Conditional Generative Adversarial Network for Pixel-by-Pixel Image Translation

BACKGROUND AND PURPOSE

Synthetic FLAIR images are of lower quality than conventional FLAIR images. Here, we aimed to improve the synthetic FLAIR image quality using deep learning with pixel-by-pixel translation through conditional generative adversarial network training.

MATERIALS AND METHODS

Forty patients with MS were prospectively included and scanned (3T) to acquire synthetic MR imaging and conventional FLAIR images. Synthetic FLAIR images were created with the SyMRI software. Acquired data were divided into 30 training and 10 test datasets. A conditional generative adversarial network was trained to generate improved FLAIR images from raw synthetic MR imaging data using conventional FLAIR images as targets. The peak signal-to-noise ratio, normalized root mean square error, and the Dice index of MS lesion maps were calculated for synthetic and deep learning FLAIR images against conventional FLAIR images, respectively. Lesion conspicuity and the existence of artifacts were visually assessed.

RESULTS

The peak signal-to-noise ratio and normalized root mean square error were significantly higher and lower, respectively, in generated-versus-synthetic FLAIR images in aggregate intracranial tissues and all tissue segments (all P < .001). The Dice index of lesion maps and visual lesion conspicuity were comparable between generated and synthetic FLAIR images (P = 1 and .59, respectively). Generated FLAIR images showed fewer granular artifacts (P = .003) and swelling artifacts (in all cases) than synthetic FLAIR images.

CONCLUSIONS

Using deep learning, we improved the synthetic FLAIR image quality by generating FLAIR images that have contrast closer to that of conventional FLAIR images and fewer granular and swelling artifacts, while preserving the lesion contrast.

Deep Generative Adversarial Neural Networks for Compressive Sensing MRI

Undersampled magnetic resonance image (MRI) reconstruction is typically an ill-posed linear inverse task. The time and resource intensive computations require tradeoffs between accuracy and speed. In addition, state-of-the-art compressed sensing (CS) analytics are not cognizant of the image diagnostic quality. To address these challenges, we propose a novel CS framework that uses generative adversarial networks (GAN) to model the (low-dimensional) manifold of high-quality MR images. Leveraging a mixture of least-squares (LS) GANs and pixel-wise l1/l2 cost, a deep residual network with skip connections is trained as the generator that learns to remove the aliasing artifacts by projecting onto the image manifold. The LSGAN learns the texture details, while the l1/l2 cost suppresses high-frequency noise. A discriminator network, which is a multilayer convolutional neural network (CNN), plays the role of a perceptual cost that is then jointly trained based on high-quality MR images to score the quality of retrieved images. In the operational phase, an initial aliased estimate (e.g., simply obtained by zero-filling) is propagated into the trained generator to output the desired reconstruction. This demands a very low computational overhead. Extensive evaluations are performed on a large contrast-enhanced MR dataset of pediatric patients. Images rated by expert radiologists corroborate that GANCS retrieves higher quality images with improved fine texture details compared with conventional Wavelet-based and dictionary-learning-based CS schemes as well as with deep-learning-based schemes using pixel-wise training. In addition, it offers reconstruction times of under a few milliseconds, which are two orders of magnitude faster than the current state-of-the-art CS-MRI schemes.

Projection data smoothing through noise-level weighted total variation regularization for low-dose computed tomography

Reducing radiation dose while maintaining the quality of the reconstructed images is a major challenge in the computed tomography (CT) community. In light of the non-stationary Gaussian noise distribution, we developed a model that incorporates a noise-level weighted total variation (NWTV) regularization term for denoising the projection data. Contrary to the well-known edge-weighted total variation method, which aims for better edge preserving, the proposed NWTV tries to adapt the regularization with the spatially varying noise levels. Experiments on simulated data as well as the real imaging data suggest that the proposed NWTV regularization could achieve quite competitive results. For sinograms with sharp edges, the NWTV could do a better job at balancing noise reduction and edge preserving, such that noise is removed in a more uniform manner. Another conclusion from our experiments is that the well-recognized stair-casing artifacts of TV regularization play little role in the reconstructed images when the NWTV method is applied to low-dose CT imaging data.

Cycle-consistent adversarial denoising network for multiphase coronary CT angiography

PURPOSE

In multiphase coronary CT angiography (CTA), a series of CT images are taken at different levels of radiation dose during the examination. Although this reduces the total radiation dose, the image quality during the low-dose phases is significantly degraded. Recently, deep neural network approaches based on supervised learning technique have demonstrated impressive performance improvement over conventional model-based iterative methods for low-dose CT. However, matched low- and routine-dose CT image pairs are difficult to obtain in multiphase CT. To address this problem, we aim at developing a new deep learning framework.

METHOD

We propose an unsupervised learning technique that can remove the noise of the CT images in the low-dose phases by learning from the CT images in the routine dose phases. Although a supervised learning approach is not applicable due to the differences in the underlying heart structure in two phases, the images are closely related in two phases, so we propose a cycle-consistent adversarial denoising network to learn the mapping between the low- and high-dose cardiac phases.

RESULTS

Experimental results showed that the proposed method effectively reduces the noise in the low-dose CT image while preserving detailed texture and edge information. Moreover, thanks to the cyclic consistency and identity loss, the proposed network does not create any artificial features that are not present in the input images. Visual grading and quality evaluation also confirm that the proposed method provides significant improvement in diagnostic quality.

CONCLUSIONS

The proposed network can learn the image distributions from the routine-dose cardiac phases, which is a big advantage over the existing supervised learning networks that need exactly matched low- and routine-dose CT images. Considering the effectiveness and practicability of the proposed method, we believe that the proposed can be applied for many other CT acquisition protocols.

Unsupervised Reverse Domain Adaptation for Synthetic Medical Images via Adversarial Training

To realize the full potential of deep learning for medical imaging, large annotated datasets are required for training. Such datasets are difficult to acquire due to privacy issues, lack of experts available for annotation, underrepresentation of rare conditions, and poor standardization. The lack of annotated data has been addressed in conventional vision applications using synthetic images refined via unsupervised adversarial training to look like real images. However, this approach is difficult to extend to general medical imaging because of the complex and diverse set of features found in real human tissues. We propose a novel framework that uses a reverse flow, where adversarial training is used to make real medical images more like synthetic images, and clinically-relevant features are preserved via self-regularization. These domain-adapted synthetic-like images can then be accurately interpreted by networks trained on large datasets of synthetic medical images. We implement this approach on the notoriously difficult task of depth-estimation from monocular endoscopy which has a variety of applications in colonoscopy, robotic surgery, and invasive endoscopic procedures. We train a depth estimator on a large data set of synthetic images generated using an accurate forward model of an endoscope and an anatomically-realistic colon. Our analysis demonstrates that the structural similarity of endoscopy depth estimation in a real pig colon predicted from a network trained solely on synthetic data improved by 78.7% by using reverse domain adaptation.

Metal artifact reduction on cervical CT images by deep residual learning

Background

Cervical cancer is the fifth most common cancer among women, which is the third leading cause of cancer death in women worldwide. Brachytherapy is the most effective treatment for cervical cancer. For brachytherapy, computed tomography (CT) imaging is necessary since it conveys tissue density information which can be used for dose planning. However, the metal artifacts caused by brachytherapy applicators remain a challenge for the automatic processing of image data for image-guided procedures or accurate dose calculations. Therefore, developing an effective metal artifact reduction (MAR) algorithm in cervical CT images is of high demand.

Methods

A novel residual learning method based on convolutional neural network (RL-ARCNN) is proposed to reduce metal artifacts in cervical CT images. For MAR, a dataset is generated by simulating various metal artifacts in the first step, which will be applied to train the CNN. This dataset includes artifact-insert, artifact-free, and artifact-residual images. Numerous image patches are extracted from the dataset for training on deep residual learning artifact reduction based on CNN (RL-ARCNN). Afterwards, the trained model can be used for MAR on cervical CT images.

Results

The proposed method provides a good MAR result with a PSNR of 38.09 on the test set of simulated artifact images. The PSNR of residual learning (38.09) is higher than that of ordinary learning (37.79) which shows that CNN-based residual images achieve favorable artifact reduction. Moreover, for a 512 × 512 image, the average removal artifact time is less than 1 s.

Conclusions

The RL-ARCNN indicates that residual learning of CNN remarkably reduces metal artifacts and improves critical structure visualization and confidence of radiation oncologists in target delineation. Metal artifacts are eliminated efficiently free of sinogram data and complicated post-processing procedure.

Generative models: an upcoming innovation in musculoskeletal radiology? A preliminary test in spine imaging

Background

Deep learning is a ground-breaking technology that is revolutionising many research and industrial fields. Generative models are recently gaining interest. Here, we investigate their potential, namely conditional generative adversarial networks, in the field of magnetic resonance imaging (MRI) of the spine, by performing clinically relevant benchmark cases.

Methods

First, the enhancement of the resolution of T2-weighted (T2W) images (super-resolution) was tested. Then, automated image-to-image translation was tested in the following tasks: (1) from T1-weighted to T2W images of the lumbar spine and (2) vice versa; (3) from T2W to short time inversion-recovery (STIR) images; (4) from T2W to turbo inversion recovery magnitude (TIRM) images; (5) from sagittal standing x-ray projections to T2W images. Clinical and quantitative assessments of the outputs by means of image quality metrics were performed. The training of the models was performed on MRI and x-ray images from 989 patients.

Results

The performance of the models was generally positive and promising, but with several limitations. The number of disc protrusions or herniations showed good concordance (κ = 0.691) between native and super-resolution images. Moderate-to-excellent concordance was found when translating T2W to STIR and TIRM images (κ ≥ 0.842 regarding disc degeneration), while the agreement was poor when translating x-ray to T2W images.

Conclusions

Conditional generative adversarial networks are able to generate perceptually convincing synthetic images of the spine in super-resolution and image-to-image translation tasks. Taking into account the limitations of the study, deep learning-based generative methods showed the potential to be an upcoming innovation in musculoskeletal radiology.

The evolution of image reconstruction for CT-from filtered back projection to artificial intelligence

Abstract

The first CT scanners in the early 1970s already used iterative reconstruction algorithms; however, lack of computational power prevented their clinical use. In fact, it took until 2009 for the first iterative reconstruction algorithms to come commercially available and replace conventional filtered back projection. Since then, this technique has caused a true hype in the field of radiology. Within a few years, all major CT vendors introduced iterative reconstruction algorithms for clinical routine, which evolved rapidly into increasingly advanced reconstruction algorithms. The complexity of algorithms ranges from hybrid-, model-based to fully iterative algorithms. As a result, the number of scientific publications on this topic has skyrocketed over the last decade. But what exactly has this technology brought us so far? And what can we expect from future hardware as well as software developments, such as photon-counting CT and artificial intelligence? This paper will try answer those questions by taking a concise look at the overall evolution of CT image reconstruction and its clinical implementations. Subsequently, we will give a prospect towards future developments in this domain.

Key Points

• Advanced CT reconstruction methods are indispensable in the current clinical setting.

• IR is essential for photon-counting CT, phase-contrast CT, and dark-field CT.

• Artificial intelligence will potentially further increase the performance of reconstruction methods.

Robust navigation support in lowest dose image setting

PURPOSE

Clinical cardiac electrophysiology (EP) is concerned with diagnosis and treatment of cardiac arrhythmia describing abnormality or perturbation in the normal activation sequence of the myocardium. With the recent introduction of lowest dose X-ray imaging protocol for EP procedures, interventional image enhancement has gained crucial importance for the well-being of patients as well as medical staff.

METHODS

In this paper, we introduce a novel method to detect and track different EP catheter electrodes in lowest dose fluoroscopic sequences based on [Formula: see text]-sparse coding and online robust PCA (ORPCA). Besides being able to work on real lowest dose sequences, the underlying methodology achieves simultaneous detection and tracking of three main EP catheters used during ablation procedures.

RESULTS

We have validated our algorithm on 16 lowest dose fluoroscopic sequences acquired during real cardiac ablation procedures. In addition to expert labels for 2 sequences, we have employed a crowdsourcing strategy to obtain ground truth labels for the remaining 14 sequences. In order to validate the effect of different training data, we have employed a leave-one-out cross-validation scheme yielding an average detection rate of [Formula: see text].

CONCLUSION

Besides these promising quantitative results, our medical partners also expressed their high satisfaction. Being based on [Formula: see text]-sparse coding and online robust PCA (ORPCA), our method advances previous approaches by being able to detect and track electrodes attached to multiple different catheters.

Iterative quality enhancement via residual-artifact learning networks for low-dose CT

Radiation exposure and the associated risk of cancer for patients in computed tomography (CT) scans have been major clinical concerns. The radiation exposure can be reduced effectively via lowering the x-ray tube current (mA). However, this strategy may lead to excessive noise and streak artifacts in the conventional filtered back-projection reconstructed images. To address this issue, some deep convolutional neural network (ConvNet) based approaches have been developed for low-dose CT imaging inspired by the recent development of machine learning. Nevertheless, some of the image textures reconstructed by the ConvNet could be corrupted by the severe streaks, especially in ultra-low-dose cases, which could be close to prostheses and hamper diagnosis. Therefore, in this work, we propose an iterative residual-artifact learning ConvNet (IRLNet) approach to improve the reconstruction performance over the ConvNet based approaches. Specifically, the proposed IRLNet estimates the high-frequency details within the noise and then removes them iteratively; after eliminating severe streaks in the low-dose CT images, the residual low-frequency details can be processed through the conventional network. Moreover, the proposed IRLNet scheme can be extended for robust handling of quantitative dual energy CT/cerebral perfusion CT imaging, and statistical iterative reconstruction. Real patient data are used to evaluate the proposed IRLNet, and the experimental results demonstrate that the proposed IRLNet approach outperforms the previous ConvNet based approaches in reducing the image noise and streak artifacts efficiently at the same time as preserving edge details well, suggesting that the proposed IRLNet approach can be used to improve the CT image quality, especially in ultra-low-dose cases.

Sharpness-Aware Low-Dose CT Denoising Using Conditional Generative Adversarial Network

Low-dose computed tomography (LDCT) has offered tremendous benefits in radiation-restricted applications, but the quantum noise as resulted by the insufficient number of photons could potentially harm the diagnostic performance. Current image-based denoising methods tend to produce a blur effect on the final reconstructed results especially in high noise levels. In this paper, a deep learning-based approach was proposed to mitigate this problem. An adversarially trained network and a sharpness detection network were trained to guide the training process. Experiments on both simulated and real dataset show that the results of the proposed method have very small resolution loss and achieves better performance relative to state-of-the-art methods both quantitatively and visually.