Generative Adversarial Networks for Noise Reduction in Low-Dose CT

Simulation Study of Low-Dose Sparse-Sampling CT with Deep Learning-Based Reconstruction: Usefulness for Evaluation of Ovarian Cancer Metastasis

The usefulness of sparse-sampling CT with deep learning-based reconstruction for detection of metastasis of malignant ovarian tumors was evaluated. We obtained contrast-enhanced CT images (n = 141) of ovarian cancers from a public database, whose images were randomly divided into 71 training, 20 validation, and 50 test cases. Sparse-sampling CT images were calculated slice-by-slice by software simulation. Two deep-learning models for deep learning-based reconstruction were evaluated: Residual Encoder-Decoder Convolutional Neural Network (RED-CNN) and deeper U-net. For 50 test cases, we evaluated the peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) as quantitative measures. Two radiologists independently performed a qualitative evaluation for the following points: entire CT image quality; visibility of the iliac artery; and visibility of peritoneal dissemination, liver metastasis, and lymph node metastasis. Wilcoxon signed-rank test and McNemar test were used to compare image quality and metastasis detectability between the two models, respectively. The mean PSNR and SSIM performed better with deeper U-net over RED-CNN. For all items of the visual evaluation, deeper U-net scored significantly better than RED-CNN. The metastasis detectability with deeper U-net was more than 95%. Sparse-sampling CT with deep learning-based reconstruction proved useful in detecting metastasis of malignant ovarian tumors and might contribute to reducing overall CT-radiation exposure.

Generative adversarial network based regularized image reconstruction for PET

Positron emission tomography (PET) is an ill-posed inverse problem and suffers high noise due to limited number of detected events. Prior information can be used to improve the quality of reconstructed PET images. Deep neural networks have also been applied to regularized image reconstruction. One method is to use a pretrained denoising neural network to represent the PET image and to perform a constrained maximum likelihood estimation. In this work, we propose to use a generative adversarial network (GAN) to further improve the network performance. We also modify the objective function to include a data-matching term on the network input. Experimental studies using computer-based Monte Carlo simulations and real patient datasets demonstrate that the proposed method leads to noticeable improvements over the kernel-based and U-net-based regularization methods in terms of lesion contrast recovery versus background noise trade-offs.

Low-dose CT with deep learning regularization via proximal forward-backward splitting

Low-dose x-ray computed tomography (LDCT) is desirable for reduced patient dose. This work develops new image reconstruction methods with deep learning (DL) regularization for LDCT. Our methods are based on the unrolling of a proximal forward-backward splitting (PFBS) framework with data-driven image regularization via deep neural networks. In contrast to PFBS-IR, which utilizes standard data fidelity updates via an iterative reconstruction (IR) method, PFBS-AIR involves preconditioned data fidelity updates that fuse the analytical reconstruction (AR) and IR methods in a synergistic way, i.e. fused analytical and iterative reconstruction (AIR). The results suggest that the DL-regularized methods (PFBS-IR and PFBS-AIR) provide better reconstruction quality compared to conventional methods (AR or IR). In addition, owing to the AIR, PFBS-AIR noticeably outperformed PFBS-IR and another DL-based postprocessing method, FBPConvNet.

Artificial Intelligence and COVID-19: Deep Learning Approaches for Diagnosis and Treatment

COVID-19 outbreak has put the whole world in an unprecedented difficult situation bringing life around the world to a frightening halt and claiming thousands of lives. Due to COVID-19's spread in 212 countries and territories and increasing numbers of infected cases and death tolls mounting to 5,212,172 and 334,915 (as of May 22 2020), it remains a real threat to the public health system. This paper renders a response to combat the virus through Artificial Intelligence (AI). Some Deep Learning (DL) methods have been illustrated to reach this goal, including Generative Adversarial Networks (GANs), Extreme Learning Machine (ELM), and Long/Short Term Memory (LSTM). It delineates an integrated bioinformatics approach in which different aspects of information from a continuum of structured and unstructured data sources are put together to form the user-friendly platforms for physicians and researchers. The main advantage of these AI-based platforms is to accelerate the process of diagnosis and treatment of the COVID-19 disease. The most recent related publications and medical reports were investigated with the purpose of choosing inputs and targets of the network that could facilitate reaching a reliable Artificial Neural Network-based tool for challenges associated with COVID-19. Furthermore, there are some specific inputs for each platform, including various forms of the data, such as clinical data and medical imaging which can improve the performance of the introduced approaches toward the best responses in practical applications.

What scans we will read: imaging instrumentation trends in clinical oncology

Oncological diseases account for a significant portion of the burden on public healthcare systems with associated costs driven primarily by complex and long-lasting therapies. Through the visualization of patient-specific morphology and functional-molecular pathways, cancerous tissue can be detected and characterized non-invasively, so as to provide referring oncologists with essential information to support therapy management decisions. Following the onset of stand-alone anatomical and functional imaging, we witness a push towards integrating molecular image information through various methods, including anato-metabolic imaging (e.g., PET/CT), advanced MRI, optical or ultrasound imaging.This perspective paper highlights a number of key technological and methodological advances in imaging instrumentation related to anatomical, functional, molecular medicine and hybrid imaging, that is understood as the hardware-based combination of complementary anatomical and molecular imaging. These include novel detector technologies for ionizing radiation used in CT and nuclear medicine imaging, and novel system developments in MRI and optical as well as opto-acoustic imaging. We will also highlight new data processing methods for improved non-invasive tissue characterization. Following a general introduction to the role of imaging in oncology patient management we introduce imaging methods with well-defined clinical applications and potential for clinical translation. For each modality, we report first on the status quo and, then point to perceived technological and methodological advances in a subsequent status go section. Considering the breadth and dynamics of these developments, this perspective ends with a critical reflection on where the authors, with the majority of them being imaging experts with a background in physics and engineering, believe imaging methods will be in a few years from now.Overall, methodological and technological medical imaging advances are geared towards increased image contrast, the derivation of reproducible quantitative parameters, an increase in volume sensitivity and a reduction in overall examination time. To ensure full translation to the clinic, this progress in technologies and instrumentation is complemented by advances in relevant acquisition and image-processing protocols and improved data analysis. To this end, we should accept diagnostic images as "data", and - through the wider adoption of advanced analysis, including machine learning approaches and a "big data" concept - move to the next stage of non-invasive tumour phenotyping. The scans we will be reading in 10 years from now will likely be composed of highly diverse multi-dimensional data from multiple sources, which mandate the use of advanced and interactive visualization and analysis platforms powered by Artificial Intelligence (AI) for real-time data handling by cross-specialty clinical experts with a domain knowledge that will need to go beyond that of plain imaging.

Quadratic Autoencoder (Q-AE) for Low-Dose CT Denoising

Inspired by complexity and diversity of biological neurons, our group proposed quadratic neurons by replacing the inner product in current artificial neurons with a quadratic operation on input data, thereby enhancing the capability of an individual neuron. Along this direction, we are motivated to evaluate the power of quadratic neurons in popular network architectures, simulating human-like learning in the form of "quadratic-neuron-based deep learning". Our prior theoretical studies have shown important merits of quadratic neurons and networks in representation, efficiency, and interpretability. In this paper, we use quadratic neurons to construct an encoder-decoder structure, referred as the quadratic autoencoder, and apply it to low-dose CT denoising. The experimental results on the Mayo low-dose CT dataset demonstrate the utility and robustness of quadratic autoencoder in terms of image denoising and model efficiency. To our best knowledge, this is the first time that the deep learning approach is implemented with a new type of neurons and demonstrates a significant potential in the medical imaging field.

Deep learning improves contrast in low-fluence photoacoustic imaging

Low fluence illumination sources can facilitate clinical transition of photoacoustic imaging because they are rugged, portable, affordable, and safe. However, these sources also decrease image quality due to their low fluence. Here, we propose a denoising method using a multi-level wavelet-convolutional neural network to map low fluence illumination source images to its corresponding high fluence excitation map. Quantitative and qualitative results show a significant potential to remove the background noise and preserve the structures of target. Substantial improvements up to 2.20, 2.25, and 4.3-fold for PSNR, SSIM, and CNR metrics were observed, respectively. We also observed enhanced contrast (up to 1.76-fold) in an in vivo application using our proposed methods. We suggest that this tool can improve the value of such sources in photoacoustic imaging.

Updates in Vascular Computed Tomography

Computed tomography angiography (CTA) has become a mainstay for the imaging of vascular diseases, because of high accuracy, availability, and rapid turnaround time. High-quality CTA images can now be routinely obtained with high isotropic spatial resolution and temporal resolution. Advances in CTA have focused on improving the image quality, increasing the acquisition speed, eliminating artifacts, and reducing the doses of radiation and iodinated contrast media. Dual-energy computed tomography provides material composition capabilities that can be used for characterizing lesions, optimizing contrast, decreasing artifact, and reducing radiation dose. Deep learning techniques can be used for classification, segmentation, quantification, and image enhancement.

Artificial intelligence from A to Z: From neural network to legal framework

Artificial intelligence (AI) will continue to cause substantial changes within the field of radiology, and it will become increasingly important for clinicians to be familiar with several concepts behind AI algorithms in order to effectively guide their clinical implementation. This review aims to give medical professionals the basic information needed to understand AI development and research. The general concepts behind several AI algorithms, including their data requirements, training, and evaluation methods are explained. The potential legal implications of using AI algorithms in clinical practice are also discussed.

Constructing a tissue-specific texture prior by machine learning from previous full-dose scan for Bayesian reconstruction of current ultralow-dose CT images

Purpose: Bayesian theory provides a sound framework for ultralow-dose computed tomography (ULdCT) image reconstruction with two terms for modeling the data statistical property and incorporating a priori knowledge for the image that is to be reconstructed. We investigate the feasibility of using a machine learning (ML) strategy, particularly the convolutional neural network (CNN), to construct a tissue-specific texture prior from previous full-dose computed tomography. Approach: Our study constructs four tissue-specific texture priors, corresponding with lung, bone, fat, and muscle, and integrates the prior with the prelog shift Poisson (SP) data property for Bayesian reconstruction of ULdCT images. The Bayesian reconstruction was implemented by an algorithm called SP-CNN-T and compared with our previous Markov random field (MRF)-based tissue-specific texture prior algorithm called SP-MRF-T. Results: In addition to conventional quantitative measures, mean squared error and peak signal-to-noise ratio, structure similarity index, feature similarity, and texture Haralick features were used to measure the performance difference between SP-CNN-T and SP-MRF-T algorithms in terms of the structure and tissue texture preservation, demonstrating the feasibility and the potential of the investigated ML approach. Conclusions: Both training performance and image reconstruction results showed the feasibility of constructing CNN texture prior model and the potential of improving the structure preservation of the nodule comparing to our previous regional tissue-specific MRF texture prior model.

Preparing Medical Imaging Data for Machine Learning

Artificial intelligence (AI) continues to garner substantial interest in medical imaging. The potential applications are vast and include the entirety of the medical imaging life cycle from image creation to diagnosis to outcome prediction. The chief obstacles to development and clinical implementation of AI algorithms include availability of sufficiently large, curated, and representative training data that includes expert labeling (eg, annotations). Current supervised AI methods require a curation process for data to optimally train, validate, and test algorithms. Currently, most research groups and industry have limited data access based on small sample sizes from small geographic areas. In addition, the preparation of data is a costly and time-intensive process, the results of which are algorithms with limited utility and poor generalization. In this article, the authors describe fundamental steps for preparing medical imaging data in AI algorithm development, explain current limitations to data curation, and explore new approaches to address the problem of data availability.

Possibility of Deep Learning in Medical Imaging Focusing Improvement of Computed Tomography Image Quality

Deep learning (DL), part of a broader family of machine learning methods, is based on learning data representations rather than task-specific algorithms. Deep learning can be used to improve the image quality of clinical scans with image noise reduction. We review the ability of DL to reduce the image noise, present the advantages and disadvantages of computed tomography image reconstruction, and examine the potential value of new DL-based computed tomography image reconstruction.

Deep learning in fracture detection: a narrative review

Artificial intelligence (AI) is a general term that implies the use of a computer to model intelligent behavior with minimal human intervention. AI, particularly deep learning, has recently made substantial strides in perception tasks allowing machines to better represent and interpret complex data. Deep learning is a subset of AI represented by the combination of artificial neuron layers. In the last years, deep learning has gained great momentum. In the field of orthopaedics and traumatology, some studies have been done using deep learning to detect fractures in radiographs. Deep learning studies to detect and classify fractures on computed tomography (CT) scans are even more limited. In this narrative review, we provide a brief overview of deep learning technology: we (1) describe the ways in which deep learning until now has been applied to fracture detection on radiographs and CT examinations; (2) discuss what value deep learning offers to this field; and finally (3) comment on future directions of this technology.

Deep Learning for Cardiac Image Segmentation: A Review

Deep learning has become the most widely used approach for cardiac image segmentation in recent years. In this paper, we provide a review of over 100 cardiac image segmentation papers using deep learning, which covers common imaging modalities including magnetic resonance imaging (MRI), computed tomography (CT), and ultrasound and major anatomical structures of interest (ventricles, atria, and vessels). In addition, a summary of publicly available cardiac image datasets and code repositories are included to provide a base for encouraging reproducible research. Finally, we discuss the challenges and limitations with current deep learning-based approaches (scarcity of labels, model generalizability across different domains, interpretability) and suggest potential directions for future research.

VVBP-Tensor in the FBP Algorithm: Its Properties and Application in Low-Dose CT Reconstruction

For decades, commercial X-ray computed tomography (CT) scanners have been using the filtered backprojection (FBP) algorithm for image reconstruction. However, the desire for lower radiation doses has pushed the FBP algorithm to its limit. Previous studies have made significant efforts to improve the results of FBP through preprocessing the sinogram, modifying the ramp filter, or postprocessing the reconstructed images. In this paper, we focus on analyzing and processing the stacked view-by-view backprojections (named VVBP-Tensor) in the FBP algorithm. A key challenge for our analysis lies in the radial structures in each backprojection slice. To overcome this difficulty, a sorting operation was introduced to the VVBP-Tensor in its z direction (the direction of the projection views). The results show that, after sorting, the tensor contains structures that are similar to those of the object, and structures in different slices of the tensor are correlated. We then analyzed the properties of the VVBP-Tensor, including structural self-similarity, tensor sparsity, and noise statistics. Considering these properties, we have developed an algorithm using the tensor singular value decomposition (named VVBP-tSVD) to denoise the VVBP-Tensor for low-mAs CT imaging. Experiments were conducted using a physical phantom and clinical patient data with different mAs levels. The results demonstrate that the VVBP-tSVD is superior to all competing methods under different reconstruction schemes, including sinogram preprocessing, image postprocessing, and iterative reconstruction. We conclude that the VVBP-Tensor is a suitable processing target for improving the quality of FBP reconstruction, and the proposed VVBP-tSVD is an effective algorithm for noise reduction in low-mAs CT imaging. This preliminary work might provide a heuristic perspective for reviewing and rethinking the FBP algorithm.

TomoGAN: low-dose synchrotron x-ray tomography with generative adversarial networks: discussion

Synchrotron-based x-ray tomography is a noninvasive imaging technique that allows for reconstructing the internal structure of materials at high spatial resolutions from tens of micrometers to a few nanometers. In order to resolve sample features at smaller length scales, however, a higher radiation dose is required. Therefore, the limitation on the achievable resolution is set primarily by noise at these length scales. We present TomoGAN, a denoising technique based on generative adversarial networks, for improving the quality of reconstructed images for low-dose imaging conditions. We evaluate our approach in two photon-budget-limited experimental conditions: (1) sufficient number of low-dose projections (based on Nyquist sampling), and (2) insufficient or limited number of high-dose projections. In both cases, the angular sampling is assumed to be isotropic, and the photon budget throughout the experiment is fixed based on the maximum allowable radiation dose on the sample. Evaluation with both simulated and experimental datasets shows that our approach can significantly reduce noise in reconstructed images, improving the structural similarity score of simulation and experimental data from 0.18 to 0.9 and from 0.18 to 0.41, respectively. Furthermore, the quality of the reconstructed images with filtered back projection followed by our denoising approach exceeds that of reconstructions with the simultaneous iterative reconstruction technique, showing the computational superiority of our approach.

ADMM-CSNet: A Deep Learning Approach for Image Compressive Sensing

Compressive sensing (CS) is an effective technique for reconstructing image from a small amount of sampled data. It has been widely applied in medical imaging, remote sensing, image compression, etc. In this paper, we propose two versions of a novel deep learning architecture, dubbed as ADMM-CSNet, by combining the traditional model-based CS method and data-driven deep learning method for image reconstruction from sparsely sampled measurements. We first consider a generalized CS model for image reconstruction with undetermined regularizations in undetermined transform domains, and then two efficient solvers using Alternating Direction Method of Multipliers (ADMM) algorithm for optimizing the model are proposed. We further unroll and generalize the ADMM algorithm to be two deep architectures, in which all parameters of the CS model and the ADMM algorithm are discriminatively learned by end-to-end training. For both applications of fast CS complex-valued MR imaging and CS imaging of real-valued natural images, the proposed ADMM-CSNet achieved favorable reconstruction accuracy in fast computational speed compared with the traditional and the other deep learning methods.

MRI Cross-Modality Image-to-Image Translation

We present a cross-modality generation framework that learns to generate translated modalities from given modalities in MR images. Our proposed method performs Image Modality Translation (abbreviated as IMT) by means of a deep learning model that leverages conditional generative adversarial networks (cGANs). Our framework jointly exploits the low-level features (pixel-wise information) and high-level representations (e.g. brain tumors, brain structure like gray matter, etc.) between cross modalities which are important for resolving the challenging complexity in brain structures. Our framework can serve as an auxiliary method in medical use and has great application potential. Based on our proposed framework, we first propose a method for cross-modality registration by fusing the deformation fields to adopt the cross-modality information from translated modalities. Second, we propose an approach for MRI segmentation, translated multichannel segmentation (TMS), where given modalities, along with translated modalities, are segmented by fully convolutional networks (FCN) in a multichannel manner. Both of these two methods successfully adopt the cross-modality information to improve the performance without adding any extra data. Experiments demonstrate that our proposed framework advances the state-of-the-art on five brain MRI datasets. We also observe encouraging results in cross-modality registration and segmentation on some widely adopted brain datasets. Overall, our work can serve as an auxiliary method in medical use and be applied to various tasks in medical fields.

[Use of artificial intelligence for image reconstruction]

CLINICAL/METHODOLOGICAL PROBLEM

In the reconstruction of three-dimensional image data, artifacts that interfere with the appraisal often occur as a result of trying to minimize the dose or due to missing data. Used iterative reconstruction methods are time-consuming and have disadvantages.

STANDARD RADIOLOGICAL METHODS

These problems are known to occur in computed tomography (CT), cone beam CT, interventional imaging, magnetic resonance imaging (MRI) and nuclear medicine imaging (PET and SPECT).

METHODOLOGICAL INNOVATIONS

Using techniques based on the use of artificial intelligence (AI) in data analysis and data supplementation, a number of problems can be solved up to a certain extent.

PERFORMANCE

The performance of the methods varies greatly. Since the generated image data usually look very good using the AI-based methods presented here while their results depend strongly on the study design, reliable comparable quantitative statements on the performance are not yet available in broad terms.

EVALUATION

In principle, the methods of image reconstruction based on AI algorithms offer many possibilities for improving and optimizing three-dimensional image datasets. However, the validity strongly depends on the design of the respective study in the structure of the individual procedure. It is therefore essential to have a suitable test prior to use in clinical practice.

PRACTICAL RECOMMENDATIONS

Before the widespread use of AI-based reconstruction methods can be recommended, it is necessary to establish meaningful test procedures that can characterize the actual performance and applicability in terms of information content and a meaningful study design during the learning phase of the algorithms.

Deep Learning Based High-Resolution Reconstruction of Trabecular Bone Microstructures from Low-Resolution CT Scans using GAN-CIRCLE

Osteoporosis is a common age-related disease characterized by reduced bone density and increased fracture-risk. Microstructural quality of trabecular bone (Tb), commonly found at axial skeletal sites and at the end of long bones, is an important determinant of bone-strength and fracture-risk. High-resolution emerging CT scanners enable in vivo measurement of Tb microstructures at peripheral sites. However, resolution-dependence of microstructural measures and wide resolution-discrepancies among various CT scanners together with rapid upgrades in technology warrant data harmonization in CT-based cross-sectional and longitudinal bone studies. This paper presents a deep learning-based method for high-resolution reconstruction of Tb microstructures from low-resolution CT scans using GAN-CIRCLE. A network was developed and evaluated using post-registered ankle CT scans of nineteen volunteers on both low- and high-resolution CT scanners. 9,000 matching pairs of low- and high-resolution patches of size 64×64 were randomly harvested from ten volunteers for training and validation. Another 5,000 matching pairs of patches from nine other volunteers were used for evaluation. Quantitative comparison shows that predicted high-resolution scans have significantly improved structural similarity index (p < 0.01) with true high-resolution scans as compared to the same metric for low-resolution data. Different Tb microstructural measures such as thickness, spacing, and network area density are also computed from low- and predicted high-resolution images, and compared with the values derived from true high-resolution scans. Thickness and network area measures from predicted images showed higher agreement with true high-resolution CT (CCC = [0.95, 0.91]) derived values than the same measures from low-resolution images (CCC = [0.72, 0.88]).

GPU acceleration of liver enhancement for tumor segmentation

BACKGROUND AND OBJECTIVE

Medical image segmentation plays a vital role in medical image analysis. There are many algorithms developed for medical image segmentation which are based on edge or region characteristics. These are dependent on the quality of the image. The contrast of a CT or MRI image plays an important role in identifying region of interest i.e. lesion(s). In order to enhance the contrast of image, clinicians generally use manual histogram adjustment technique which is based on 1D histogram specification. This is time consuming and results in poor distribution of pixels over the image. Cross modality based contrast enhancement is 2D histogram specification technique. This is robust and provides a more uniform distribution of pixels over CT image by exploiting the inner structure information from MRI image. This helps in increasing the sensitivity and accuracy of lesion segmentation from enhanced CT image. The sequential implementation of cross modality based contrast enhancement is slow. Hence we propose GPU acceleration of cross modality based contrast enhancement for tumor segmentation.

METHODS

The aim of this study is fast parallel cross modality based contrast enhancement for CT liver images. This includes pairwise 2D histogram, histogram equalization and histogram matching. The sequential implementation of the cross modality based contrast enhancement is computationally expensive and hence time consuming. We propose persistence and grid-stride loop based fast parallel contrast enhancement for CT liver images. We use enhanced CT liver image for the lesion or tumor segmentation. We implement the fast parallel gradient based dynamic seeded region growing for lesion segmentation.

RESULTS

The proposed parallel approach is 104.416 ( ±  5.166) times faster compared to the sequential implementation and increases the sensitivity and specificity of tumor segmentation.

CONCLUSION

The cross modality approach is inspired by 2D histogram specification which incorporates spatial information existing in both guidance and input images for remapping the input image intensity values. The cross modality based liver contrast enhancement improves the quality of tumor segmentation.

Two stage residual CNN for texture denoising and structure enhancement on low dose CT image

BACKGROUND AND OBJECTIVE

X-ray computed tomography (CT) plays an important role in modern medical science. Human health problems caused by CT radiation have attracted the attention of the academic community widely. Reducing radiation dose results in a deterioration in image quality and further affects doctor's diagnosis. Therefore, this paper introduces a new denoise method for low dose CT (LDCT) images, called two stage residual convolutional neural network (TS-RCNN).

METHODS

There are two important parts with respect to our network. 1) The first stage RCNN is proposed for texture denoising via the stationary wavelet transform (SWT) and the perceptual loss. Specifically, SWT is performed on each normal dose CT (NDCT) image and generated four wavelet images are considered as the labels. 2) The second stage RCNN is established for structure enhancement via the average NDCT model on the basis of the first network's result. Finally, the denoised CT image is obtained via inverse SWT.

RESULTS

Our proposed TS-RCNN is trained on three groups of simulated LDCT images in 1123 images per group and evaluated on 129 simulated LDCT images for each group. Besides, to demonstrate the clinical application of TS-RCNN, we also test our method on the 2016 Low Dose CT Grand Challenge dataset. Quantitative results show that TS-RCNN almost achieves the best results in terms of MSE, SSIM and PSNR compared to other methods.

CONCLUSIONS

The experimental results and comparisons demonstrate that TS-RCNN not only preserves more texture information, but also enhances structural information on LDCT images.

Semi-Supervised Learning for Low-Dose CT Image Restoration with Hierarchical Deep Generative Adversarial Network (HD-GAN)

In the absence of duplicate high-dose CT data, it is challenging to restore high-quality images based on deep learning with only low-dose CT (LDCT) data. When different reconstruction algorithms and settings are adopted to prepare high-quality images, LDCT datasets for deep learning can be unpaired. To address this problem, we propose hierarchical deep generative adversarial networks (HD-GANs) for semi-supervised learning with the unpaired datasets. We first cluster each patient's CT images into multiple categories, and then collect the images in the same categories across different patients to build an imageset for denoising. Each imageset is fed into a generative adversarial network that consists of a denoising network and a following classification network. The denoising network efficiently reuses feature maps from the lower layers for end-to-end learning with full-size images. The classifier is trained to distinguish between the denoised images and the high-quality images. Evaluated with a clinical LDCT dataset, the proposed semi-supervised learning approach efficiently reduces the noise level of LDCT images without loss of information, thereby addressing the major shortcomings of IR such as computation time and anatomical inaccuracy.

Performance Evaluation of a Generative Adversarial Network for Deblurring Mobile-phone Cervical Images

Visual examination forms an integral part of cervical cancer screening. With the recent rise in smartphone-based health technologies, capturing cervical images using a smartphone camera for telemedicine and automated screening is gaining popularity. However, such images are highly prone to image corruption, typically out-of-focus target or camera shake blur. In this paper, we applied a generative adversarial network (GAN) to deblur mobile-phone cervical (MC) images, and we evaluate the deblur quality using various measures. Our evaluation process is three-fold: first, we calculate the peak signal to noise ratio (PSNR) and the structural similarity (SSIM) of a test dataset with ground truth availability. Next, we calculate the perception based image quality evaluator (PIQE) score of a test dataset without ground truth availability. Finally, we classify a dataset of blurred and the corresponding deblurred images into normal/abnormal MC images. The resulting change in classification accuracy was our final assessment. Our evaluation experiments show that deblurring of MC images can potentially improve the accuracy of both manual and automated cancerous lesion screening.

Low-dose CT Denoising Using Edge Detection Layer and Perceptual Loss

Low-dose CT imaging is a valid approach to reduce patients' exposure to X-ray radiation. However, reducing X-ray current increases noise and artifacts in the reconstructed CT images. Deep neural networks have been successfully employed to remove noise from low-dose CT images. This study proposes two novel techniques to boost the performance of a neural network with minimal change in the complexity. First, a non-trainable edge detection layer is proposed that extracts four edge maps from the input image. The layer improves quantitative metrics (PSNR and SSIM) and helps to predict a CT image with more precise boundaries. Next, a joint function of mean-square error and perceptual loss is employed to optimize the network. Using the perceptual loss helps to preserve structural detail; however, it adds check-board artifacts to the output. The proposed joint objective function takes advantage of the benefits offered by each loss. It improves the over-smoothing problem caused by mean-square error and the check-board artifacts caused by perceptual loss.

Deep Efficient End-to-end Reconstruction (DEER) Network for Few-view Breast CT Image Reconstruction

Breast CT provides image volumes with isotropic resolution in high contrast, enabling detection of small calcification (down to a few hundred microns in size) and subtle density differences. Since breast is sensitive to x-ray radiation, dose reduction of breast CT is an important topic, and for this purpose, few-view scanning is a main approach. In this article, we propose a Deep Efficient End-to-end Reconstruction (DEER) network for few-view breast CT image reconstruction. The major merits of our network include high dose efficiency, excellent image quality, and low model complexity. By the design, the proposed network can learn the reconstruction process with as few as O ( N ) parameters, where N is the side length of an image to be reconstructed, which represents orders of magnitude improvements relative to the state-of-the-art deep-learning-based reconstruction methods that map raw data to tomographic images directly. Also, validated on a cone-beam breast CT dataset prepared by Koning Corporation on a commercial scanner, our method demonstrates a competitive performance over the state-of-the-art reconstruction networks in terms of image quality. The source code of this paper is available at: https://github.com/HuidongXie/DEER.

Machine Learning in PET: from Photon Detection to Quantitative Image Reconstruction

Machine learning has found unique applications in nuclear medicine from photon detection to quantitative image reconstruction. While there have been impressive strides in detector development for time-of-flight positron emission tomography, most detectors still make use of simple signal processing methods to extract the time and position information from the detector signals. Now with the availability of fast waveform digitizers, machine learning techniques have been applied to estimate the position and arrival time of high-energy photons. In quantitative image reconstruction, machine learning has been used to estimate various corrections factors, including scattered events and attenuation images, as well as to reduce statistical noise in reconstructed images. Here machine learning either provides a faster alternative to an existing time-consuming computation, such as in the case of scatter estimation, or creates a data-driven approach to map an implicitly defined function, such as in the case of estimating the attenuation map for PET/MR scans. In this article, we will review the abovementioned applications of machine learning in nuclear medicine.

Deep Learning Reconstruction at CT: Phantom Study of the Image Characteristics

OBJECTIVES

Noise, commonly encountered on computed tomography (CT) images, can impact diagnostic accuracy. To reduce the image noise, we developed a deep-learning reconstruction (DLR) method that integrates deep convolutional neural networks into image reconstruction. In this phantom study, we compared the image noise characteristics, spatial resolution, and task-based detectability on DLR images and images reconstructed with other state-of-the art techniques.

METHODS

We scanned a phantom harboring cylindrical modules with different contrast on a 320-row detector CT scanner. Phantom images were reconstructed with filtered back projection, hybrid iterative reconstruction, model-based iterative reconstruction, and DLR. The standard deviation of the CT number and the noise power spectrum were calculated for noise characterization. The 10% modulation-transfer function (MTF) level was used to evaluate spatial resolution; task-based detectability was assessed using the model observer method.

RESULTS

On images reconstructed with DLR, the noise was lower than on images subjected to other reconstructions, especially at low radiation dose settings. Noise power spectrum measurements also showed that the noise amplitude was lower, especially for low-frequency components, on DLR images. Based on the MTF, spatial resolution was higher on model-based iterative reconstruction image than DLR image, however, for lower-contrast objects, the MTF on DLR images was comparable to images reconstructed with other methods. The machine observer study showed that at reduced radiation-dose settings, DLR yielded the best detectability.

CONCLUSION

On DLR images, the image noise was lower, and high-contrast spatial resolution and task-based detectability were better than on images reconstructed with other state-of-the art techniques. DLR also outperformed other methods with respect to task-based detectability.

Image Reconstruction: From Sparsity to Data-adaptive Methods and Machine Learning

The field of medical image reconstruction has seen roughly four types of methods. The first type tended to be analytical methods, such as filtered back-projection (FBP) for X-ray computed tomography (CT) and the inverse Fourier transform for magnetic resonance imaging (MRI), based on simple mathematical models for the imaging systems. These methods are typically fast, but have suboptimal properties such as poor resolution-noise trade-off for CT. A second type is iterative reconstruction methods based on more complete models for the imaging system physics and, where appropriate, models for the sensor statistics. These iterative methods improved image quality by reducing noise and artifacts. The FDA-approved methods among these have been based on relatively simple regularization models. A third type of methods has been designed to accommodate modified data acquisition methods, such as reduced sampling in MRI and CT to reduce scan time or radiation dose. These methods typically involve mathematical image models involving assumptions such as sparsity or low-rank. A fourth type of methods replaces mathematically designed models of signals and systems with data-driven or adaptive models inspired by the field of machine learning. This paper focuses on the two most recent trends in medical image reconstruction: methods based on sparsity or low-rank models, and data-driven methods based on machine learning techniques.

Artificial Intelligence in Cardiovascular Imaging

The number of cardiovascular imaging studies is growing exponentially, and so is the need to improve clinical workflow efficiency and avoid missed diagnoses. With the availability and use of large datasets, artificial intelligence (AI) has the potential to improve patient care at every stage of the imaging chain. Current literature indicates that in the short-term, AI has the capacity to reduce human error and save time in the clinical workflow through automated segmentation of cardiac structures. In the future, AI may expand the informational value of diagnostic images based on images alone or a combination of images and clinical variables, thus facilitating disease detection, prognosis, and decision making. This review describes the role of AI, specifically machine learning, in multimodality imaging, including echocardiography, nuclear imaging, computed tomography, and cardiac magnetic resonance, and highlights current uses of AI as well as potential challenges to its widespread implementation.

CT Super-Resolution GAN Constrained by the Identical, Residual, and Cycle Learning Ensemble (GAN-CIRCLE)

In this paper, we present a semi-supervised deep learning approach to accurately recover high-resolution (HR) CT images from low-resolution (LR) counterparts. Specifically, with the generative adversarial network (GAN) as the building block, we enforce the cycle-consistency in terms of the Wasserstein distance to establish a nonlinear end-to-end mapping from noisy LR input images to denoised and deblurred HR outputs. We also include the joint constraints in the loss function to facilitate structural preservation. In this process, we incorporate deep convolutional neural network (CNN), residual learning, and network in network techniques for feature extraction and restoration. In contrast to the current trend of increasing network depth and complexity to boost the imaging performance, we apply a parallel 1×1 CNN to compress the output of the hidden layer and optimize the number of layers and the number of filters for each convolutional layer. The quantitative and qualitative evaluative results demonstrate that our proposed model is accurate, efficient and robust for super-resolution (SR) image restoration from noisy LR input images. In particular, we validate our composite SR networks on three large-scale CT datasets, and obtain promising results as compared to the other state-of-the-art methods.

Deep Encoder-Decoder Adversarial Reconstruction(DEAR) Network for 3D CT from Few-View Data

X-ray computed tomography (CT) is widely used in clinical practice. The involved ionizingX-ray radiation, however, could increase cancer risk. Hence, the reduction of the radiation dosehas been an important topic in recent years. Few-view CT image reconstruction is one of the mainways to minimize radiation dose and potentially allow a stationary CT architecture. In this paper,we propose a deep encoder-decoder adversarial reconstruction (DEAR) network for 3D CT imagereconstruction from few-view data. Since the artifacts caused by few-view reconstruction appear in3D instead of 2D geometry, a 3D deep network has a great potential for improving the image qualityin a data driven fashion. More specifically, our proposed DEAR-3D network aims at reconstructing3D volume directly from clinical 3D spiral cone-beam image data. DEAR is validated on a publiclyavailable abdominal CT dataset prepared and authorized by Mayo Clinic. Compared with other2D deep learning methods, the proposed DEAR-3D network can utilize 3D information to producepromising reconstruction results.

Unpaired Low-Dose CT Denoising Network Based on Cycle-Consistent Generative Adversarial Network with Prior Image Information

The widespread application of X-ray computed tomography (CT) in clinical diagnosis has led to increasing public concern regarding excessive radiation dose administered to patients. However, reducing the radiation dose will inevitably cause server noise and affect radiologists' judgment and confidence. Hence, progressive low-dose CT (LDCT) image reconstruction methods must be developed to improve image quality. Over the past two years, deep learning-based approaches have shown impressive performance in noise reduction for LDCT images. Most existing deep learning-based approaches usually require the paired training dataset which the LDCT images correspond to the normal-dose CT (NDCT) images one-to-one, but the acquisition of well-paired datasets requires multiple scans, resulting the increase of radiation dose. Therefore, well-paired datasets are not readily available. To resolve this problem, this paper proposes an unpaired LDCT image denoising network based on cycle generative adversarial networks (CycleGAN) with prior image information which does not require a one-to-one training dataset. In this method, cyclic loss, an important trick in unpaired image-to-image translation, promises to map the distribution from LDCT to NDCT by using unpaired training data. Furthermore, to guarantee the accurate correspondence of the image content between the output and NDCT, the prior information obtained from the result preprocessed using the LDCT image is integrated into the network to supervise the generation of content. Given the map of distribution through the cyclic loss and the supervision of content through the prior image loss, our proposed method can not only reduce the image noise but also retain critical information. Real-data experiments were carried out to test the performance of the proposed method. The peak signal-to-noise ratio (PSNR) improves by more than 3 dB, and the structural similarity (SSIM) increases when compared with the original CycleGAN without prior information. The real LDCT data experiment demonstrates the superiority of the proposed method according to both visual inspection and quantitative evaluation.

Domain Progressive 3D Residual Convolution Network to Improve Low-Dose CT Imaging

The wide applications of X-ray computed tomography (CT) bring low-dose CT (LDCT) into a clinical prerequisite, but reducing the radiation exposure in CT often leads to significantly increased noise and artifacts, which might lower the judgment accuracy of radiologists. In this paper, we put forward a domain progressive 3D residual convolution network (DP-ResNet) for the LDCT imaging procedure that contains three stages: sinogram domain network (SD-net), filtered back projection (FBP), and image domain network (ID-net). Though both are based on the residual network structure, the SD-net and ID-net provide complementary effect on improving the final LDCT quality. The experimental results with both simulated and real projection data show that this domain progressive deep-learning network achieves significantly improved performance by combing the network processing in the two domains.

Artificial Intelligence in medical imaging practice: looking to the future

Artificial intelligence (AI) is heralded as the most disruptive technology to health services in the 21st century. Many commentary articles published in the general public and health domains recognise that medical imaging is at the forefront of these changes due to our large digital data footprint. Radiomics is transforming medical images into mineable high-dimensional data to optimise clinical decision-making; however, some would argue that AI could infiltrate workplaces with very few ethical checks and balances. In this commentary article, we describe how AI is beginning to change medical imaging services and the innovations that are on the horizon. We explore how AI and its various forms, including machine learning, will challenge the way medical imaging is delivered from workflow, image acquisition, image registration to interpretation. Diagnostic radiographers will need to learn to work alongside our 'virtual colleagues', and we argue that there are vital changes to entry and advanced curricula together with national professional capabilities to ensure machine-learning tools are used in the safest and most effective manner for our patients.

Longitudinal Prediction of Infant Diffusion MRI Data via Graph Convolutional Adversarial Networks

Missing data is a common problem in longitudinal studies due to subject dropouts and failed scans. We present a graph-based convolutional neural network to predict missing diffusion MRI data. In particular, we consider the relationships between sampling points in the spatial domain and the diffusion wave-vector domain to construct a graph. We then use a graph convolutional network to learn the non-linear mapping from available data to missing data. Our method harnesses a multi-scale residual architecture with adversarial learning for prediction with greater accuracy and perceptual quality. Experimental results show that our method is accurate and robust in the longitudinal prediction of infant brain diffusion MRI data.

Deep learning based noise reduction method for automatic 3D segmentation of the anterior of lamina cribrosa in optical coherence tomography volumetric scans

A deep-learning (DL) based noise reduction algorithm, in combination with a vessel shadow compensation method and a three-dimensional (3D) segmentation technique, has been developed to achieve, to the authors best knowledge, the first automatic segmentation of the anterior surface of the lamina cribrosa (LC) in volumetric ophthalmic optical coherence tomography (OCT) scans. The present DL-based OCT noise reduction algorithm was trained without the need of noise-free ground truth images by utilizing the latest development in deep learning of de-noising from single noisy images, and was demonstrated to be able to cover more locations in the retina and disease cases of different types to achieve high robustness. Compared with the original single OCT images, a 6.6 dB improvement in peak signal-to-noise ratio and a 0.65 improvement in the structural similarity index were achieved. The vessel shadow compensation method analyzes the energy profile in each A-line and automatically compensates the pixel intensity of locations underneath the detected blood vessel. Combining the noise reduction algorithm and the shadow compensation and contrast enhancement technique, medical experts were able to identify the anterior surface of the LC in 98.3% of the OCT images. The 3D segmentation algorithm employs a two-round procedure based on gradients information and information from neighboring images. An accuracy of 90.6% was achieved in a validation study involving 180 individual B-scans from 36 subjects, compared to 64.4% in raw images. This imaging and analysis strategy enables the first automatic complete view of the anterior LC surface, to the authors best knowledge, which may have the potentials in new LC parameters development for glaucoma diagnosis and management.

Learning from adversarial medical images for X-ray breast mass segmentation

BACKGROUND AND OBJECTIVE

Simulation of diverse lesions in images is proposed and applied to overcome the scarcity of labeled data, which has hindered the application of deep learning in medical imaging. However, most of current studies focus on generating samples with class labels for classification and detection rather than segmentation, because generating images with precise masks remains a challenge. Therefore, we aim to generate realistic medical images with precise masks for improving lesion segmentation in mammagrams.

METHODS

In this paper, we propose a new framework for improving X-ray breast mass segmentation performance aided by generated adversarial lesion images with precise masks. Firstly, we introduce a conditional generative adversarial network (cGAN) to learn the distribution of real mass images as well as a mapping between images and corresponding segmentation masks. Subsequently, a number of lesion images are generated from various binary input masks using the generator in the trained cGAN. Then the generated adversarial samples are concatenated with original samples to produce a dataset with increased diversity. Furthermore, we introduce an improved U-net and train it on the previous augmented dataset for breast mass segmentation.

RESULTS

To demonstrate the effectiveness of our proposed method, we conduct experiments on publicly available mammogram database of INbreast and a private database provided by Nanfang Hospital in China. Experimental results show that an improvement up to 7% in Jaccard index can be achieved over the same model trained on original real lesion images.

CONCLUSIONS

Our proposed method can be viewed as one of the first steps toward generating realistic X-ray breast mass images with masks for precise segmentation.

Learning to Reconstruct Computed Tomography Images Directly From Sinogram Data Under A Variety of Data Acquisition Conditions

Computed tomography (CT) is widely used in medical diagnosis and non-destructive detection. Image reconstruction in CT aims to accurately recover pixel values from measured line integrals, i.e., the summed pixel values along straight lines. Provided that the acquired data satisfy the data sufficiency condition as well as other conditions regarding the view angle sampling interval and the severity of transverse data truncation, researchers have discovered many solutions to accurately reconstruct the image. However, if these conditions are violated, accurate image reconstruction from line integrals remains an intellectual challenge. In this paper, a deep learning method with a common network architecture, termed iCT-Net, was developed and trained to accurately reconstruct images for previously solved and unsolved CT reconstruction problems with high quantitative accuracy. Particularly, accurate reconstructions were achieved for the case when the sparse view reconstruction problem (i.e., compressed sensing problem) is entangled with the classical interior tomographic problems.