3-D Convolutional Encoder-Decoder Network for Low-Dose CT via Transfer Learning From a 2-D Trained Network

Evaluation of Impact of Factors Affecting CT Radiation Dose for Optimizing Patient Dose Levels

The dose metrics and factors influencing radiation exposure for patients undergoing head, chest, and abdominal computed tomography (CT) scans were investigated for optimization of patient dose levels. The local diagnostic reference levels (DRLs) of adult CT scans performed in our hospital were established based on 28,147 consecutive examinations, including 5510 head scans, 9091 chest scans, and 13,526 abdominal scans. Among the six CT scanners used in our hospital, four of them are 64-slice multi-detector CT units (MDCT64), and the other two have detector slices higher than 64 (MDCTH). Multivariate analysis was conducted to evaluate the effects of body size, kVp, mAs, and pitch on volume CT dose index (CTDIvol). The local DRLs expressed in terms of the 75th percentile of CTDIvol for the head, chest, and abdominal scans performed on MDCT64 were 59.32, 9.24, and 10.64 mGy, respectively. The corresponding results for MDCTH were 57.90, 7.67, and 9.86 mGy. In regard to multivariate analysis, CTDIvol showed various dependence on the predictors investigated in this study. All regression relationships have coefficient of determination (R2) larger than 0.75, indicating a good fit to the data. Overall, the research results obtained through our workflow could facilitate the modification of CT imaging procedures once the local DRLs are unusually high compared to the national DRLs.

Reducing the Breast Cancer Risk and Radiation Dose of Radiography for Scoliosis in Children: A Phantom Study

Full-spinal radiographs (FRs) are often the first choice of imaging modality in the investigation of scoliosis. However, FRs are strongly related to breast cancer occurrence due to multiple large-field radiographic examinations taken during childhood and adolescence, which may increase the risk for breast cancer in adulthood among women with scoliosis. The purpose of this study was to consider various technical parameters to reduce the patient radiation dose of FRs for scoliosis. To evaluate breast surface doses (BSDs) in FRs, radio photoluminescence dosimeters were placed in contact with a child phantom. Using the PC-based Monte Carlo (PMC) program for calculating patient doses in medical X-ray examinations, the breast organ dose (BOD) and the effective dose were calculated by performing Monte Carlo simulations using mathematical phantom models. The BSDs in the posteroanterior (PA) view were 0.15-0.34-fold those in the anteroposterior (AP) view. The effective dose in the PA view was 0.4-0.61-fold that in the AP view. BSD measurements were almost equivalent to the BODs obtained using PMC at all exposure settings. During FRs, the PA view without an anti-scatter grid significantly reduced the breast dose compared to the AP view with an anti-scatter grid.

Multi-Contrast Super-Resolution MRI Through a Progressive Network

Magnetic resonance imaging (MRI) is widely used for screening, diagnosis, image-guided therapy, and scientific research. A significant advantage of MRI over other imaging modalities such as computed tomography (CT) and nuclear imaging is that it clearly shows soft tissues in multi-contrasts. Compared with other medical image super-resolution methods that are in a single contrast, multi-contrast super-resolution studies can synergize multiple contrast images to achieve better super-resolution results. In this paper, we propose a one-level non-progressive neural network for low up-sampling multi-contrast super-resolution and a two-level progressive network for high up-sampling multi-contrast super-resolution. The proposed networks integrate multi-contrast information in a high-level feature space and optimize the imaging performance by minimizing a composite loss function, which includes mean-squared-error, adversarial loss, perceptual loss, and textural loss. Our experimental results demonstrate that 1) the proposed networks can produce MRI super-resolution images with good image quality and outperform other multi-contrast super-resolution methods in terms of structural similarity and peak signal-to-noise ratio; 2) combining multi-contrast information in a high-level feature space leads to a significantly improved result than a combination in the low-level pixel space; and 3) the progressive network produces a better super-resolution image quality than the non-progressive network, even if the original low-resolution images were highly down-sampled.

Improving Diagnostic Accuracy in Low-Dose SPECT Myocardial Perfusion Imaging With Convolutional Denoising Networks

Lowering the administered dose in SPECT myocardial perfusion imaging (MPI) has become an important clinical problem. In this study we investigate the potential benefit of applying a deep learning (DL) approach for suppressing the elevated imaging noise in low-dose SPECT-MPI studies. We adopt a supervised learning approach to train a neural network by using image pairs obtained from full-dose (target) and low-dose (input) acquisitions of the same patients. In the experiments, we made use of acquisitions from 1,052 subjects and demonstrated the approach for two commonly used reconstruction methods in clinical SPECT-MPI: 1) filtered backprojection (FBP), and 2) ordered-subsets expectation-maximization (OSEM) with corrections for attenuation, scatter and resolution. We evaluated the DL output for the clinical task of perfusion-defect detection at a number of successively reduced dose levels (1/2, 1/4, 1/8, 1/16 of full dose). The results indicate that the proposed DL approach can achieve substantial noise reduction and lead to improvement in the diagnostic accuracy of low-dose data. In particular, at 1/2 dose, DL yielded an area-under-the-ROC-curve (AUC) of 0.799, which is nearly identical to the AUC = 0.801 obtained by OSEM at full-dose ( p -value = 0.73); similar results were also obtained for FBP reconstruction. Moreover, even at 1/8 dose, DL achieved AUC = 0.770 for OSEM, which is above the AUC = 0.755 obtained at full-dose by FBP. These results indicate that, compared to conventional reconstruction filtering, DL denoising can allow for additional dose reduction without sacrificing the diagnostic accuracy in SPECT-MPI.

The Promise of Clinical Decision Support Systems Targetting Low-Resource Settings

Low-resource clinical settings are plagued by low physician-to-patient ratios and a shortage of high-quality medical expertise and infrastructure. Together, these phenomena lead to over-burdened healthcare systems that under-serve the needs of the community. Alleviating this burden can be undertaken by the introduction of clinical decision support systems (CDSSs); systems that support stakeholders (ranging from physicians to patients) within the clinical setting in their day-to-day activities. Such systems, which have proven to be effective in the developed world, remain to be under-explored in low-resource settings. This review attempts to summarize the research focused on clinical decision support systems that either target stakeholders within low-resource clinical settings or diseases commonly found in such environments. When categorizing our findings according to disease applications, we find that CDSSs are predominantly focused on dealing with bacterial infections and maternal care, do not leverage deep learning, and have not been evaluated prospectively. Together, these highlight the need for increased research in this domain in order to impact a diverse set of medical conditions and ultimately improve patient outcomes.

Role of imaging in multiple myeloma

With rapid advancements in the diagnosis and treatment of multiple myeloma (MM), imaging has become instrumental in detection of intramedullary and extramedullary disease, providing prognostic information, and assessing therapeutic efficacy. Whole-body low dose computed tomography (WBLDCT) has emerged as the study of choice to detect osteolytic bone disease. Positron emission tomography/computed tomography (PET/CT) combines functional and morphologic information to identify MM disease activity and assess treatment response. Magnetic resonance imaging (MRI) has excellent soft-tissue contrast and is the modality of choice for bone marrow evaluation. This review focuses on the imaging modalities available for MM patient management, highlighting advantages, disadvantages, and applications of each.

Artificial intelligence in radiotherapy: a technological review

Radiation therapy (RT) is widely used to treat cancer. Technological advances in RT have occurred in the past 30 years. These advances, such as three-dimensional image guidance, intensity modulation, and robotics, created challenges and opportunities for the next breakthrough, in which artificial intelligence (AI) will possibly play important roles. AI will replace certain repetitive and labor-intensive tasks and improve the accuracy and consistency of others, particularly those with increased complexity because of technological advances. The improvement in efficiency and consistency is important to manage the increasing cancer patient burden to the society. Furthermore, AI may provide new functionalities that facilitate satisfactory RT. The functionalities include superior images for real-time intervention and adaptive and personalized RT. AI may effectively synthesize and analyze big data for such purposes. This review describes the RT workflow and identifies areas, including imaging, treatment planning, quality assurance, and outcome prediction, that benefit from AI. This review primarily focuses on deep-learning techniques, although conventional machine-learning techniques are also mentioned.

Image Deblurring Using Multi-Stream Bottom-Top-Bottom Attention Network and Global Information-Based Fusion and Reconstruction Network

Image deblurring has been a challenging ill-posed problem in computer vision. Gaussian blur is a common model for image and signal degradation. The deep learning-based deblurring methods have attracted much attention due to their advantages over the traditional methods relying on hand-designed features. However, the existing deep learning-based deblurring techniques still cannot perform well in restoring the fine details and reconstructing the sharp edges. To address this issue, we have designed an effective end-to-end deep learning-based non-blind image deblurring algorithm. In the proposed method, a multi-stream bottom-top-bottom attention network (MBANet) with the encoder-to-decoder structure is designed to integrate low-level cues and high-level semantic information, which can facilitate extracting image features more effectively and improve the computational efficiency of the network. Moreover, the MBANet adopts a coarse-to-fine multi-scale strategy to process the input images to improve image deblurring performance. Furthermore, the global information-based fusion and reconstruction network is proposed to fuse multi-scale output maps to improve the global spatial information and recurrently refine the output deblurred image. The experiments were done on the public GoPro dataset and the realistic and dynamic scenes (REDS) dataset to evaluate the effectiveness and robustness of the proposed method. The experimental results show that the proposed method generally outperforms some traditional deburring methods and deep learning-based state-of-the-art deblurring methods such as scale-recurrent network (SRN) and denoising prior driven deep neural network (DPDNN) in terms of such quantitative indexes as peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) and human vision.

SACNN: Self-Attention Convolutional Neural Network for Low-Dose CT Denoising With Self-Supervised Perceptual Loss Network

Computed tomography (CT) is a widely used screening and diagnostic tool that allows clinicians to obtain a high-resolution, volumetric image of internal structures in a non-invasive manner. Increasingly, efforts have been made to improve the image quality of low-dose CT (LDCT) to reduce the cumulative radiation exposure of patients undergoing routine screening exams. The resurgence of deep learning has yielded a new approach for noise reduction by training a deep multi-layer convolutional neural networks (CNN) to map the low-dose to normal-dose CT images. However, CNN-based methods heavily rely on convolutional kernels, which use fixed-size filters to process one local neighborhood within the receptive field at a time. As a result, they are not efficient at retrieving structural information across large regions. In this paper, we propose a novel 3D self-attention convolutional neural network for the LDCT denoising problem. Our 3D self-attention module leverages the 3D volume of CT images to capture a wide range of spatial information both within CT slices and between CT slices. With the help of the 3D self-attention module, CNNs are able to leverage pixels with stronger relationships regardless of their distance and achieve better denoising results. In addition, we propose a self-supervised learning scheme to train a domain-specific autoencoder as the perceptual loss function. We combine these two methods and demonstrate their effectiveness on both CNN-based neural networks and WGAN-based neural networks with comprehensive experiments. Tested on the AAPM-Mayo Clinic Low Dose CT Grand Challenge data set, our experiments demonstrate that self-attention (SA) module and autoencoder (AE) perceptual loss function can efficiently enhance traditional CNNs and can achieve comparable or better results than the state-of-the-art methods.

Quadratic Autoencoder (Q-AE) for Low-Dose CT Denoising

Inspired by complexity and diversity of biological neurons, our group proposed quadratic neurons by replacing the inner product in current artificial neurons with a quadratic operation on input data, thereby enhancing the capability of an individual neuron. Along this direction, we are motivated to evaluate the power of quadratic neurons in popular network architectures, simulating human-like learning in the form of "quadratic-neuron-based deep learning". Our prior theoretical studies have shown important merits of quadratic neurons and networks in representation, efficiency, and interpretability. In this paper, we use quadratic neurons to construct an encoder-decoder structure, referred as the quadratic autoencoder, and apply it to low-dose CT denoising. The experimental results on the Mayo low-dose CT dataset demonstrate the utility and robustness of quadratic autoencoder in terms of image denoising and model efficiency. To our best knowledge, this is the first time that the deep learning approach is implemented with a new type of neurons and demonstrates a significant potential in the medical imaging field.

Stepwise PathNet: a layer-by-layer knowledge-selection-based transfer learning algorithm

Some neural network can be trained by transfer learning, which uses a pre-trained neural network as the source task, for a small target task’s dataset. The performance of the transfer learning depends on the knowledge (i.e., layers) selected from the pre-trained network. At present, this knowledge is usually chosen by humans. The transfer learning method PathNet automatically selects pre-trained modules or adjustable modules in a modular neural network. However, PathNet requires modular neural networks as the pre-trained networks, therefore non-modular pre-trained neural networks are currently unavailable. Consequently, PathNet limits the versatility of the network structure. To address this limitation, we propose Stepwise PathNet, which regards the layers of a non-modular pre-trained neural network as the module in PathNet and selects the layers automatically through training. In an experimental validation of transfer learning from InceptionV3 pre-trained on the ImageNet dataset to networks trained on three other datasets (CIFAR-100, SVHN and Food-101), Stepwise PathNet was up to 8% and 10% more accurate than finely tuned and from-scratch approaches, respectively. Also, some of the selected layers were not supported by the layer functions assumed in PathNet.

Deep Volumetric Feature Encoding for Biomedical Images

Deep learning research has demonstrated the effectiveness of using pre-trained networks as feature encoders. The large majority of these networks are trained on 2D datasets with millions of samples and diverse classes of information. We demonstrate and evaluate approaches to transferring deep 2D feature spaces to 3D in order to take advantage of these and related resources in the biomedical domain. First, we show how VGG-19 activations can be mapped to a 3D variant of the network (VGG-19-3D). Second, using varied medical decathlon data, we provide a technique for training 3D networks to predict the encodings induced by 3D VGG-19. Lastly, we compare five different 3D networks (one of which is trained only on 3D MRI and another of which is not trained at all) across layers and patch sizes in terms of their ability to identify hippocampal landmark points in 3D MRI data that was not included in their training. We make observations about the performance, recommend different networks and layers and make them publicly available for further evaluation.

Deep Learning-based Inaccuracy Compensation in Reconstruction of High Resolution XCT Data

While X-ray computed tomography (XCT) is pushed further into the micro- and nanoscale, the limitations of various tool components and object motion become more apparent. For high-resolution XCT, it is necessary but practically difficult to align these tool components with sub-micron precision. The aim is to develop a novel reconstruction methodology that considers unavoidable misalignment and object motion during the data acquisition in order to obtain high-quality three-dimensional images and that is applicable for data recovery from incomplete datasets. A reconstruction software empowered by sophisticated correction modules that autonomously estimates and compensates artefacts using gradient descent and deep learning algorithms has been developed and applied. For motion estimation, a novel computer vision methodology coupled with a deep convolutional neural network approach provides estimates for the object motion by tracking features throughout the adjacent projections. The model is trained using the forward projections of simulated phantoms that consist of several simple geometrical features such as sphere, triangle and rectangular. The feature maps extracted by a neural network are used to detect and to classify features done by a support vector machine. For missing data recovery, a novel deep convolutional neural network is used to infer high-quality reconstruction data from incomplete sets of projections. The forward and back projections of simulated geometric shapes from a range of angular ranges are used to train the model. The model is able to learn the angular dependency based on a limited angle coverage and to propose a new set of projections to suppress artefacts. High-quality three-dimensional images demonstrate that it is possible to effectively suppress artefacts caused by thermomechanical instability of tool components and objects resulting in motion, by center of rotation misalignment and by inaccuracy in the detector position without additional computational efforts. Data recovery from incomplete sets of projections result in directly corrected projections instead of suppressing artefacts in the final reconstructed images. The proposed methodology has been proven and is demonstrated for a ball bearing sample. The reconstruction results are compared to prior corrections and benchmarked with a commercially available reconstruction software. Compared to conventional approaches in XCT imaging and data analysis, the proposed methodology for the generation of high-quality three-dimensional X-ray images is fully autonomous. The methodology presented here has been proven for high-resolution micro-XCT and nano-XCT, however, is applicable for all length scales.

Automatic mandibular canal detection using a deep convolutional neural network

The practicability of deep learning techniques has been demonstrated by their successful implementation in varied fields, including diagnostic imaging for clinicians. In accordance with the increasing demands in the healthcare industry, techniques for automatic prediction and detection are being widely researched. Particularly in dentistry, for various reasons, automated mandibular canal detection has become highly desirable. The positioning of the inferior alveolar nerve (IAN), which is one of the major structures in the mandible, is crucial to prevent nerve injury during surgical procedures. However, automatic segmentation using Cone beam computed tomography (CBCT) poses certain difficulties, such as the complex appearance of the human skull, limited number of datasets, unclear edges, and noisy images. Using work-in-progress automation software, experiments were conducted with models based on 2D SegNet, 2D and 3D U-Nets as preliminary research for a dental segmentation automation tool. The 2D U-Net with adjacent images demonstrates higher global accuracy of 0.82 than naïve U-Net variants. The 2D SegNet showed the second highest global accuracy of 0.96, and the 3D U-Net showed the best global accuracy of 0.99. The automated canal detection system through deep learning will contribute significantly to efficient treatment planning and to reducing patients’ discomfort by a dentist. This study will be a preliminary report and an opportunity to explore the application of deep learning to other dental fields.

Recent Advances of Bioresponsive Nano-Sized Contrast Agents for Ultra-High-Field Magnetic Resonance Imaging

The ultra-high-field magnetic resonance imaging (MRI) nowadays has been receiving enormous attention in both biomaterial research and clinical diagnosis. MRI contrast agents are generally comprising of T1-weighted and T2-weighted contrast agent types, where T1-weighted contrast agents show positive contrast enhancement with brighter images by decreasing the proton's longitudinal relaxation times and T2-weighted contrast agents show negative contrast enhancement with darker images by decreasing the proton's transverse relaxation times. To meet the incredible demand of MRI, ultra-high-field T2 MRI is gradually attracting the attention of research and medical needs owing to its high resolution and high accuracy for detection. It is anticipated that high field MRI contrast agents can achieve high performance in MRI imaging, where parameters of chemical composition, molecular structure and size of varied contrast agents show contrasted influence in each specific diagnostic test. This review firstly presents the recent advances of nanoparticle contrast agents for MRI. Moreover, multimodal molecular imaging with MRI for better monitoring is discussed during biological process. To fasten the process of developing better contrast agents, deep learning of artificial intelligent (AI) can be well-integrated into optimizing the crucial parameters of nanoparticle contrast agents and achieving high resolution MRI prior to the clinical applications. Finally, prospects and challenges are summarized.

Boundary-Weighted Domain Adaptive Neural Network for Prostate MR Image Segmentation

Accurate segmentation of the prostate from magnetic resonance (MR) images provides useful information for prostate cancer diagnosis and treatment. However, automated prostate segmentation from 3D MR images faces several challenges. The lack of clear edge between the prostate and other anatomical structures makes it challenging to accurately extract the boundaries. The complex background texture and large variation in size, shape and intensity distribution of the prostate itself make segmentation even further complicated. Recently, as deep learning, especially convolutional neural networks (CNNs), emerging as the best performed methods for medical image segmentation, the difficulty in obtaining large number of annotated medical images for training CNNs has become much more pronounced than ever. Since large-scale dataset is one of the critical components for the success of deep learning, lack of sufficient training data makes it difficult to fully train complex CNNs. To tackle the above challenges, in this paper, we propose a boundary-weighted domain adaptive neural network (BOWDA-Net). To make the network more sensitive to the boundaries during segmentation, a boundary-weighted segmentation loss is proposed. Furthermore, an advanced boundary-weighted transfer leaning approach is introduced to address the problem of small medical imaging datasets. We evaluate our proposed model on three different MR prostate datasets. The experimental results demonstrate that the proposed model is more sensitive to object boundaries and outperformed other state-of-the-art methods.

VVBP-Tensor in the FBP Algorithm: Its Properties and Application in Low-Dose CT Reconstruction

For decades, commercial X-ray computed tomography (CT) scanners have been using the filtered backprojection (FBP) algorithm for image reconstruction. However, the desire for lower radiation doses has pushed the FBP algorithm to its limit. Previous studies have made significant efforts to improve the results of FBP through preprocessing the sinogram, modifying the ramp filter, or postprocessing the reconstructed images. In this paper, we focus on analyzing and processing the stacked view-by-view backprojections (named VVBP-Tensor) in the FBP algorithm. A key challenge for our analysis lies in the radial structures in each backprojection slice. To overcome this difficulty, a sorting operation was introduced to the VVBP-Tensor in its z direction (the direction of the projection views). The results show that, after sorting, the tensor contains structures that are similar to those of the object, and structures in different slices of the tensor are correlated. We then analyzed the properties of the VVBP-Tensor, including structural self-similarity, tensor sparsity, and noise statistics. Considering these properties, we have developed an algorithm using the tensor singular value decomposition (named VVBP-tSVD) to denoise the VVBP-Tensor for low-mAs CT imaging. Experiments were conducted using a physical phantom and clinical patient data with different mAs levels. The results demonstrate that the VVBP-tSVD is superior to all competing methods under different reconstruction schemes, including sinogram preprocessing, image postprocessing, and iterative reconstruction. We conclude that the VVBP-Tensor is a suitable processing target for improving the quality of FBP reconstruction, and the proposed VVBP-tSVD is an effective algorithm for noise reduction in low-mAs CT imaging. This preliminary work might provide a heuristic perspective for reviewing and rethinking the FBP algorithm.

TomoGAN: low-dose synchrotron x-ray tomography with generative adversarial networks: discussion

Synchrotron-based x-ray tomography is a noninvasive imaging technique that allows for reconstructing the internal structure of materials at high spatial resolutions from tens of micrometers to a few nanometers. In order to resolve sample features at smaller length scales, however, a higher radiation dose is required. Therefore, the limitation on the achievable resolution is set primarily by noise at these length scales. We present TomoGAN, a denoising technique based on generative adversarial networks, for improving the quality of reconstructed images for low-dose imaging conditions. We evaluate our approach in two photon-budget-limited experimental conditions: (1) sufficient number of low-dose projections (based on Nyquist sampling), and (2) insufficient or limited number of high-dose projections. In both cases, the angular sampling is assumed to be isotropic, and the photon budget throughout the experiment is fixed based on the maximum allowable radiation dose on the sample. Evaluation with both simulated and experimental datasets shows that our approach can significantly reduce noise in reconstructed images, improving the structural similarity score of simulation and experimental data from 0.18 to 0.9 and from 0.18 to 0.41, respectively. Furthermore, the quality of the reconstructed images with filtered back projection followed by our denoising approach exceeds that of reconstructions with the simultaneous iterative reconstruction technique, showing the computational superiority of our approach.

High-Density Surface EMG-Based Gesture Recognition Using a 3D Convolutional Neural Network

High-density surface electromyography (HD-sEMG) and deep learning technology are becoming increasingly used in gesture recognition. Based on electrode grid data, information can be extracted in the form of images that are generated with instant values of multi-channel sEMG signals. In previous studies, image-based, two-dimensional convolutional neural networks (2D CNNs) have been applied in order to recognize patterns in the electrical activity of muscles from an instantaneous image. However, 2D CNNs with 2D kernels are unable to handle a sequence of images that carry information concerning how the instantaneous image evolves with time. This paper presents a 3D CNN with 3D kernels to capture both spatial and temporal structures from sequential sEMG images and investigates its performance on HD-sEMG-based gesture recognition in comparison to the 2D CNN. Extensive experiments were carried out on two benchmark datasets (i.e., CapgMyo DB-a and CSL-HDEMG). The results show that, where the same network architecture is used, 3D CNN can achieve a better performance than 2D CNN, especially for CSL-HDEMG, which contains the dynamic part of finger movement. For CapgMyo DB-a, the accuracy of 3D CNN was 1% higher than 2D CNN when the recognition window length was equal to 40 ms, and was 1.5% higher when equal to 150 ms. For CSL-HDEMG, the accuracies of 3D CNN were 15.3% and 18.6% higher than 2D CNN when the window length was equal to 40 ms and 150 ms, respectively. Furthermore, 3D CNN achieves a competitive performance in comparison to the baseline methods.

Deep Efficient End-to-end Reconstruction (DEER) Network for Few-view Breast CT Image Reconstruction

Breast CT provides image volumes with isotropic resolution in high contrast, enabling detection of small calcification (down to a few hundred microns in size) and subtle density differences. Since breast is sensitive to x-ray radiation, dose reduction of breast CT is an important topic, and for this purpose, few-view scanning is a main approach. In this article, we propose a Deep Efficient End-to-end Reconstruction (DEER) network for few-view breast CT image reconstruction. The major merits of our network include high dose efficiency, excellent image quality, and low model complexity. By the design, the proposed network can learn the reconstruction process with as few as O ( N ) parameters, where N is the side length of an image to be reconstructed, which represents orders of magnitude improvements relative to the state-of-the-art deep-learning-based reconstruction methods that map raw data to tomographic images directly. Also, validated on a cone-beam breast CT dataset prepared by Koning Corporation on a commercial scanner, our method demonstrates a competitive performance over the state-of-the-art reconstruction networks in terms of image quality. The source code of this paper is available at: https://github.com/HuidongXie/DEER.

CT Super-Resolution GAN Constrained by the Identical, Residual, and Cycle Learning Ensemble (GAN-CIRCLE)

In this paper, we present a semi-supervised deep learning approach to accurately recover high-resolution (HR) CT images from low-resolution (LR) counterparts. Specifically, with the generative adversarial network (GAN) as the building block, we enforce the cycle-consistency in terms of the Wasserstein distance to establish a nonlinear end-to-end mapping from noisy LR input images to denoised and deblurred HR outputs. We also include the joint constraints in the loss function to facilitate structural preservation. In this process, we incorporate deep convolutional neural network (CNN), residual learning, and network in network techniques for feature extraction and restoration. In contrast to the current trend of increasing network depth and complexity to boost the imaging performance, we apply a parallel 1×1 CNN to compress the output of the hidden layer and optimize the number of layers and the number of filters for each convolutional layer. The quantitative and qualitative evaluative results demonstrate that our proposed model is accurate, efficient and robust for super-resolution (SR) image restoration from noisy LR input images. In particular, we validate our composite SR networks on three large-scale CT datasets, and obtain promising results as compared to the other state-of-the-art methods.

Deep Encoder-Decoder Adversarial Reconstruction(DEAR) Network for 3D CT from Few-View Data

X-ray computed tomography (CT) is widely used in clinical practice. The involved ionizingX-ray radiation, however, could increase cancer risk. Hence, the reduction of the radiation dosehas been an important topic in recent years. Few-view CT image reconstruction is one of the mainways to minimize radiation dose and potentially allow a stationary CT architecture. In this paper,we propose a deep encoder-decoder adversarial reconstruction (DEAR) network for 3D CT imagereconstruction from few-view data. Since the artifacts caused by few-view reconstruction appear in3D instead of 2D geometry, a 3D deep network has a great potential for improving the image qualityin a data driven fashion. More specifically, our proposed DEAR-3D network aims at reconstructing3D volume directly from clinical 3D spiral cone-beam image data. DEAR is validated on a publiclyavailable abdominal CT dataset prepared and authorized by Mayo Clinic. Compared with other2D deep learning methods, the proposed DEAR-3D network can utilize 3D information to producepromising reconstruction results.

Mitigating inherent noise in Monte Carlo dose distributions using dilated U-Net

PURPOSE

Monte Carlo (MC) algorithms offer accurate modeling of dose calculation by simulating the transport and interactions of many particles through the patient geometry. However, given their random nature, the resulting dose distributions have statistical uncertainty (noise), which prevents making reliable clinical decisions. This issue is partly addressable using a huge number of simulated particles but is computationally expensive as it results in significantly greater computation times. Therefore, there is a trade-off between the computation time and the noise level in MC dose maps. In this work, we address the mitigation of noise inherent to MC dose distributions using dilated U-Net - an encoder-decoder-styled fully convolutional neural network, which allows fast and fully automated denoising of whole-volume dose maps.

METHODS

We use mean squared error (MSE) as loss function to train the model, where training is done in 2D and 2.5D settings by considering a number of adjacent slices. Our model is trained on proton therapy MC dose distributions of different tumor sites (brain, head and neck, liver, lungs, and prostate) acquired from 35 patients. We provide the network with input MC dose distributions simulated using 1 × 10 6 particles while keeping 1 × 10 9 particles as reference.

RESULTS

After training, our model successfully denoises new MC dose maps. On average (averaged over five patients with different tumor sites), our model recovers D 95 of 55.99 Gy from the noisy MC input of 49.51 Gy, whereas the low noise MC (reference) offers 56.03 Gy. We observed a significant reduction in average RMSE (thresholded >10% max ref) for reference vs denoised (1.25 Gy) than reference vs input (16.96 Gy) leading to an improvement in signal-to-noise ratio (ISNR) by 18.06 dB. Moreover, the inference time of our model for a dose distribution is less than 10 s vs 100 min (MC simulation using 1 × 10 9 particles).

CONCLUSIONS

We propose an end-to-end fully convolutional network that can denoise Monte Carlo dose distributions. The networks provide comparable qualitative and quantitative results as the MC dose distribution simulated with 1 × 10 9 particles, offering a significant reduction in computation time.

A dual-stream deep convolutional network for reducing metal streak artifacts in CT images

Machine learning and deep learning are rapidly finding applications in the medical imaging field. In this paper, we address the long-standing problem of metal artifacts in computed tomography (CT) images by training a dual-stream deep convolutional neural network for streak removal. While many metal artifact reduction methods exist, even state-of-the-art algorithms fall short in some clinical applications. Specifically, proton therapy planning requires high image quality with accurate tumor volumes to ensure treatment success. We explore a dual-stream deep network structure with residual learning to correct metal streak artifacts after a first-pass by a state-of-the-art interpolation-based algorithm, NMAR. We provide the network with a mask of the streaks in order to focus attention on those areas. Our experiments compare a mean squared error loss function with a perceptual loss function to emphasize preservation of image features and texture. Both visual and quantitative metrics are used to assess the resulting image quality for metal implant cases. Success may be due to the duality of information processing, with one network stream performing local structure correction, while the other stream provides an attention mechanism to destreak effectively. This study shows that image-domain deep learning can be highly effective for metal artifact reduction (MAR), and highlights the benefits and drawbacks of different loss functions for solving a major CT reconstruction challenge.

Automatic classification of lung nodule candidates based on a novel 3D convolution network and knowledge transferred from a 2D network

OBJECTIVE

In the automatic lung nodule detection system, the authenticity of a large number of nodule candidates needs to be judged, which is a classification task. However, the variable shapes and sizes of the lung nodules have posed a great challenge to the classification of candidates. To solve this problem, we propose a method for classifying nodule candidates through three-dimensional (3D) convolution neural network (ConvNet) model which is trained by transferring knowledge from a multiresolution two-dimensional (2D) ConvNet model.

METHODS

In this scheme, a novel 3D ConvNet model is preweighted with the weights of the trained 2D ConvNet model, and then the 3D ConvNet model is trained with 3D image volumes. In this way, the knowledge transfer method can make 3D network easier to converge and make full use of the spatial information of nodules with different sizes and shapes to improve the classification accuracy.

RESULTS

The experimental results on 551 065 pulmonary nodule candidates in the LUNA16 dataset show that our method gains a competitive average score in the false-positive reduction track in lung nodule detection, with the sensitivities of 0.619 and 0.642 at 0.125 and 0.25 FPs per scan, respectively.

CONCLUSIONS

The proposed method can maintain satisfactory classification accuracy even when the false-positive rate is extremely small in the face of nodules of different sizes and shapes. Moreover, as a transfer learning idea, the method to transfer knowledge from 2D ConvNet to 3D ConvNet is the first attempt to carry out full migration of parameters of various layers including convolution layers, full connection layers, and classifier between different dimensional models, which is more conducive to utilizing the existing 2D ConvNet resources and generalizing transfer learning schemes.

Learning to Reconstruct Computed Tomography Images Directly From Sinogram Data Under A Variety of Data Acquisition Conditions

Computed tomography (CT) is widely used in medical diagnosis and non-destructive detection. Image reconstruction in CT aims to accurately recover pixel values from measured line integrals, i.e., the summed pixel values along straight lines. Provided that the acquired data satisfy the data sufficiency condition as well as other conditions regarding the view angle sampling interval and the severity of transverse data truncation, researchers have discovered many solutions to accurately reconstruct the image. However, if these conditions are violated, accurate image reconstruction from line integrals remains an intellectual challenge. In this paper, a deep learning method with a common network architecture, termed iCT-Net, was developed and trained to accurately reconstruct images for previously solved and unsolved CT reconstruction problems with high quantitative accuracy. Particularly, accurate reconstructions were achieved for the case when the sparse view reconstruction problem (i.e., compressed sensing problem) is entangled with the classical interior tomographic problems.

A performance comparison of convolutional neural network-based image denoising methods: The effect of loss functions on low-dose CT images

PURPOSE

Convolutional neural network (CNN)-based image denoising techniques have shown promising results in low-dose CT denoising. However, CNN often introduces blurring in denoised images when trained with a widely used pixel-level loss function. Perceptual loss and adversarial loss have been proposed recently to further improve the image denoising performance. In this paper, we investigate the effect of different loss functions on image denoising performance using task-based image quality assessment methods for various signals and dose levels.

METHODS

We used a modified version of U-net that was effective at reducing the correlated noise in CT images. The loss functions used for comparison were two pixel-level losses (i.e., the mean-squared error and the mean absolute error), Visual Geometry Group network-based perceptual loss (VGG loss), adversarial loss used to train the Wasserstein generative adversarial network with gradient penalty (WGAN-GP), and their weighted summation. Each image denoising method was applied to reconstructed images and sinogram images independently and validated using the extended cardiac-torso (XCAT) simulation and Mayo Clinic datasets. In the XCAT simulation, we generated fan-beam CT datasets with four different dose levels (25%, 50%, 75%, and 100% of a normal-dose level) using 10 XCAT phantoms and inserted signals in a test set. The signals had two different shapes (spherical and spiculated), sizes (4 and 12 mm), and contrast levels (60 and 160 HU). To evaluate signal detectability, we used a detection task SNR (tSNR) calculated from a non-prewhitening model observer with an eye filter. We also measured the noise power spectrum (NPS) and modulation transfer function (MTF) to compare the noise and signal transfer properties.

RESULTS

Compared to CNNs without VGG loss, VGG-loss-based CNNs achieved a more similar tSNR to that of the normal-dose CT for all signals at different dose levels except for a small signal at the 25% dose level. For a low-contrast signal at 25% or 50% dose, adding other losses to the VGG loss showed more improved performance than only using VGG loss. The NPS shapes from VGG-loss-based CNN closely matched that of normal-dose CT images while CNN without VGG loss overly reduced the mid-high-frequency noise power at all dose levels. MTF also showed VGG-loss-based CNN with better-preserved high resolution for all dose and contrast levels. It is also observed that additional WGAN-GP loss helps improve the noise and signal transfer properties of VGG-loss-based CNN.

CONCLUSIONS

The evaluation results using tSNR, NPS, and MTF indicate that VGG-loss-based CNNs are more effective than those without VGG loss for natural denoising of low-dose images and WGAN-GP loss improves the denoising performance of VGG-loss-based CNNs, which corresponds with the qualitative evaluation.

Generative adversarial network in medical imaging: A review

Generative adversarial networks have gained a lot of attention in the computer vision community due to their capability of data generation without explicitly modelling the probability density function. The adversarial loss brought by the discriminator provides a clever way of incorporating unlabeled samples into training and imposing higher order consistency. This has proven to be useful in many cases, such as domain adaptation, data augmentation, and image-to-image translation. These properties have attracted researchers in the medical imaging community, and we have seen rapid adoption in many traditional and novel applications, such as image reconstruction, segmentation, detection, classification, and cross-modality synthesis. Based on our observations, this trend will continue and we therefore conducted a review of recent advances in medical imaging using the adversarial training scheme with the hope of benefiting researchers interested in this technique.

Accelerated Correction of Reflection Artifacts by Deep Neural Networks in Photo-Acoustic Tomography

Photo-Acoustic Tomography (PAT) is an emerging non-invasive hybrid modality driven by a constant yearning for superior imaging performance. The image quality, however, hinges on the acoustic reflection, which may compromise the diagnostic performance. To address this challenge, we propose to incorporate a deep neural network into conventional iterative algorithms to accelerate and improve the correction of reflection artifacts. Based on the simulated PAT dataset from computed tomography (CT) scans, this network-accelerated reconstruction approach is shown to outperform two state-of-the-art iterative algorithms in terms of the peak signal-to-noise ratio (PSNR) and the structural similarity (SSIM) in the presence of noise. The proposed network also demonstrates considerably higher computational efficiency than conventional iterative algorithms, which are time-consuming and cumbersome.

Competitive performance of a modularized deep neural network compared to commercial algorithms for low-dose CT image reconstruction

Commercial iterative reconstruction techniques help to reduce CT radiation dose but altered image appearance and artifacts limit their adoptability and potential use. Deep learning has been investigated for low-dose CT (LDCT). Here we design a modularized neural network for LDCT and compared it with commercial iterative reconstruction methods from three leading CT vendors. While popular networks are trained for an end-to-end mapping, our network performs an end-to-process mapping so that intermediate denoised images are obtained with associated noise reduction directions towards a final denoised image. The learned workflow allows radiologists-in-the-loop to optimize the denoising depth in a task-specific fashion. Our network was trained with the Mayo LDCT Dataset, and tested on separate chest and abdominal CT exams from Massachusetts General Hospital. The best deep learning reconstructions were systematically compared to the best iterative reconstructions in a double-blinded reader study. This study confirms that our deep learning approach performed either favorably or comparably in terms of noise suppression and structural fidelity, and is much faster than the commercial iterative reconstruction algorithms.

DeepPET: A deep encoder-decoder network for directly solving the PET image reconstruction inverse problem

The purpose of this research was to implement a deep learning network to overcome two of the major bottlenecks in improved image reconstruction for clinical positron emission tomography (PET). These are the lack of an automated means for the optimization of advanced image reconstruction algorithms, and the computational expense associated with these state-of-the art methods. We thus present a novel end-to-end PET image reconstruction technique, called DeepPET, based on a deep convolutional encoder-decoder network, which takes PET sinogram data as input and directly and quickly outputs high quality, quantitative PET images. Using simulated data derived from a whole-body digital phantom, we randomly sampled the configurable parameters to generate realistic images, which were each augmented to a total of more than 291,000 reference images. Realistic PET acquisitions of these images were simulated, resulting in noisy sinogram data, used for training, validation, and testing the DeepPET network. We demonstrated that DeepPET generates higher quality images compared to conventional techniques, in terms of relative root mean squared error (11%/53% lower than ordered subset expectation maximization (OSEM)/filtered back-projection (FBP), structural similarity index (1%/11% higher than OSEM/FBP), and peak signal-to-noise ratio (1.1/3.8 dB higher than OSEM/FBP). In addition, we show that DeepPET reconstructs images 108 and 3 times faster than OSEM and FBP, respectively. Finally, DeepPET was successfully applied to real clinical data. This study shows that an end-to-end encoder-decoder network can produce high quality PET images at a fraction of the time compared to conventional methods.

Structurally-sensitive Multi-scale Deep Neural Network for Low-Dose CT Denoising

Computed tomography (CT) is a popular medical imaging modality and enjoys wide clinical applications. At the same time, the x-ray radiation dose associated with CT scannings raises a public concern due to its potential risks to the patients. Over the past years, major efforts have been dedicated to the development of Low-Dose CT (LDCT) methods. However, the radiation dose reduction compromises the signal-to-noise ratio (SNR), leading to strong noise and artifacts that downgrade CT image quality. In this paper, we propose a novel 3D noise reduction method, called Structurally-sensitive Multi-scale Generative Adversarial Net (SMGAN), to improve the LDCT image quality. Specifically, we incorporate three-dimensional (3D) volumetric information to improve the image quality. Also, different loss functions for training denoising models are investigated. Experiments show that the proposed method can effectively preserve structural and textural information in reference to normal-dose CT (NDCT) images, and significantly suppress noise and artifacts. Qualitative visual assessments by three experienced radiologists demonstrate that the proposed method retrieves more information, and outperforms competing methods.