Framing U-Net via Deep Convolutional Framelets: Application to Sparse-View CT

Big in Japan: Regularizing Networks for Solving Inverse Problems

Deep learning and (deep) neural networks are emerging tools to address inverse problems and image reconstruction tasks. Despite outstanding performance, the mathematical analysis for solving inverse problems by neural networks is mostly missing. In this paper, we introduce and rigorously analyze families of deep regularizing neural networks (RegNets) of the formB α + N θ ( α ) B α , where B α is a classical regularization and the networkN θ ( α ) B α is trained to recover the missing partId X - B α not found by the classical regularization. We show that these regularizing networks yield a convergent regularization method for solving inverse problems. Additionally, we derive convergence rates (quantitative error estimates) assuming a sufficient decay of the associated distance function. We demonstrate that our results recover existing convergence and convergence rates results for filter-based regularization methods as well as the recently introduced null space network as special cases. Numerical results are presented for a tomographic sparse data problem, which clearly demonstrate that the proposed RegNets improve classical regularization as well as the null space network.

Learning to Reconstruct Computed Tomography Images Directly From Sinogram Data Under A Variety of Data Acquisition Conditions

Computed tomography (CT) is widely used in medical diagnosis and non-destructive detection. Image reconstruction in CT aims to accurately recover pixel values from measured line integrals, i.e., the summed pixel values along straight lines. Provided that the acquired data satisfy the data sufficiency condition as well as other conditions regarding the view angle sampling interval and the severity of transverse data truncation, researchers have discovered many solutions to accurately reconstruct the image. However, if these conditions are violated, accurate image reconstruction from line integrals remains an intellectual challenge. In this paper, a deep learning method with a common network architecture, termed iCT-Net, was developed and trained to accurately reconstruct images for previously solved and unsolved CT reconstruction problems with high quantitative accuracy. Particularly, accurate reconstructions were achieved for the case when the sparse view reconstruction problem (i.e., compressed sensing problem) is entangled with the classical interior tomographic problems.

A performance comparison of convolutional neural network-based image denoising methods: The effect of loss functions on low-dose CT images

PURPOSE

Convolutional neural network (CNN)-based image denoising techniques have shown promising results in low-dose CT denoising. However, CNN often introduces blurring in denoised images when trained with a widely used pixel-level loss function. Perceptual loss and adversarial loss have been proposed recently to further improve the image denoising performance. In this paper, we investigate the effect of different loss functions on image denoising performance using task-based image quality assessment methods for various signals and dose levels.

METHODS

We used a modified version of U-net that was effective at reducing the correlated noise in CT images. The loss functions used for comparison were two pixel-level losses (i.e., the mean-squared error and the mean absolute error), Visual Geometry Group network-based perceptual loss (VGG loss), adversarial loss used to train the Wasserstein generative adversarial network with gradient penalty (WGAN-GP), and their weighted summation. Each image denoising method was applied to reconstructed images and sinogram images independently and validated using the extended cardiac-torso (XCAT) simulation and Mayo Clinic datasets. In the XCAT simulation, we generated fan-beam CT datasets with four different dose levels (25%, 50%, 75%, and 100% of a normal-dose level) using 10 XCAT phantoms and inserted signals in a test set. The signals had two different shapes (spherical and spiculated), sizes (4 and 12 mm), and contrast levels (60 and 160 HU). To evaluate signal detectability, we used a detection task SNR (tSNR) calculated from a non-prewhitening model observer with an eye filter. We also measured the noise power spectrum (NPS) and modulation transfer function (MTF) to compare the noise and signal transfer properties.

RESULTS

Compared to CNNs without VGG loss, VGG-loss-based CNNs achieved a more similar tSNR to that of the normal-dose CT for all signals at different dose levels except for a small signal at the 25% dose level. For a low-contrast signal at 25% or 50% dose, adding other losses to the VGG loss showed more improved performance than only using VGG loss. The NPS shapes from VGG-loss-based CNN closely matched that of normal-dose CT images while CNN without VGG loss overly reduced the mid-high-frequency noise power at all dose levels. MTF also showed VGG-loss-based CNN with better-preserved high resolution for all dose and contrast levels. It is also observed that additional WGAN-GP loss helps improve the noise and signal transfer properties of VGG-loss-based CNN.

CONCLUSIONS

The evaluation results using tSNR, NPS, and MTF indicate that VGG-loss-based CNNs are more effective than those without VGG loss for natural denoising of low-dose images and WGAN-GP loss improves the denoising performance of VGG-loss-based CNNs, which corresponds with the qualitative evaluation.

Accurate Detection Method of Aviation Bearing Based on Local Characteristics

Aviation bearing assembled detection is the final barrier to quality and safety. Therefore, an accurate detection method of aviation bearing that is based on local characteristics is designed to solve the detection problem of mis-assembly and miss-assembly of balls in aviation bearing assembled. When considering the spatial limitation of aviation bearing assembled image acquisition, the dynamic distribution of balls and the interference of lubricating grease on the surface, a dynamic local ball segmentation model that is based on U-Net network with symmetrical structure is designed to achieve the accurate segmentation of the local ball region of aviation bearing. Subsequently, an incomplete circle fitting algorithm is designed based on the segmented local ball image and Hough transform principle. These two algorithms make the measurement error of aviation bearing ball size less than 100 μm. Using bearings validates the algorithm. The results show that the accuracy of dynamic local ball segmentation model that is based on U-Net network with symmetrical structure is over 99%. At the same time, on the basis of accurate segmentation in aviation bearing local ball, the designed Hough circle algorithm is used for circle detection. The experimental results show that the false detection rate of mis-assembly and miss-assembly of balls is less than 3%. Further, the goal of zero-missed detection of mis-assembly and miss-assembly of balls in aviation bearing is achieved. The accurate segmentation of aviation bearing local ball and the effective identification of mis-assembly and miss-assembly of balls are realized. This method can provide a theory for the improvement of mis-assembly and miss-assembly of balls detection in aviation bearing. Furthermore, it has high application value.

Development of a deep neural network for generating synthetic dual-energy chest x-ray images with single x-ray exposure

Dual-energy chest radiography (DECR) is a medical imaging technology that can improve diagnostic accuracy. This technique can decompose single-energy chest radiography (SECR) images into separate bone- and soft tissue-only images. This can, however, double the radiation exposure to the patient. To address this limitation, we developed an algorithm for the synthesis of DECR from a SECR through deep learning. To predict high resolution images, we developed a novel deep learning architecture by modifying a conventional U-net to take advantage of the high frequency-dominant information that propagates from the encoding part to the decoding part. In addition, we used the anticorrelated relationship (ACR) of DECR for improving the quality of the predicted images. For training data, 300 pairs of SECR and their corresponding DECR images were used. To test the trained model, 50 DECR images from Yonsei University Severance Hospital and 662 publicly accessible SECRs were used. To evaluate the performance of the proposed method, we compared DECR and predicted images using a structural similarity approach (SSIM). In addition, we quantitatively evaluated image quality calculating the modulation transfer function and coefficient of variation. The proposed model selectively predicted the bone- and soft tissue-only CR images from an SECR image. The strategy for improving the spatial resolution by ACR was effective. Quantitative evaluation showed that the proposed method with ACR showed relatively high SSIM (over 0.85). In addition, predicted images with the proposed ACR model achieved better image quality measures than those of U-net. In conclusion, the proposed method can obtain high-quality bone- and soft tissue-only CR images without the need for additional hardware for double x-ray exposures in clinical practice.

Cycle-consistent adversarial denoising network for multiphase coronary CT angiography

PURPOSE

In multiphase coronary CT angiography (CTA), a series of CT images are taken at different levels of radiation dose during the examination. Although this reduces the total radiation dose, the image quality during the low-dose phases is significantly degraded. Recently, deep neural network approaches based on supervised learning technique have demonstrated impressive performance improvement over conventional model-based iterative methods for low-dose CT. However, matched low- and routine-dose CT image pairs are difficult to obtain in multiphase CT. To address this problem, we aim at developing a new deep learning framework.

METHOD

We propose an unsupervised learning technique that can remove the noise of the CT images in the low-dose phases by learning from the CT images in the routine dose phases. Although a supervised learning approach is not applicable due to the differences in the underlying heart structure in two phases, the images are closely related in two phases, so we propose a cycle-consistent adversarial denoising network to learn the mapping between the low- and high-dose cardiac phases.

RESULTS

Experimental results showed that the proposed method effectively reduces the noise in the low-dose CT image while preserving detailed texture and edge information. Moreover, thanks to the cyclic consistency and identity loss, the proposed network does not create any artificial features that are not present in the input images. Visual grading and quality evaluation also confirm that the proposed method provides significant improvement in diagnostic quality.

CONCLUSIONS

The proposed network can learn the image distributions from the routine-dose cardiac phases, which is a big advantage over the existing supervised learning networks that need exactly matched low- and routine-dose CT images. Considering the effectiveness and practicability of the proposed method, we believe that the proposed can be applied for many other CT acquisition protocols.

High quality imaging from sparsely sampled computed tomography data with deep learning and wavelet transform in various domains

PURPOSE

Sparsely sampled computed tomography (CT) has been attracting attention as a technique that can reduce the high radiation dose of conventional CT. In general, iterative reconstruction techniques have been applied to sparsely sampled CT to realize high quality images. These methodologies require high computing power due to the modeling of the system and the trajectory of radiation rays. Therefore, the purpose of this study was to obtain high quality three-dimensional (3D) reconstructed images with deep learning under sparse sampling conditions.

METHODS

We used a deep learning model based on a fully convolutional network and a wavelet transform to predict high quality images. To reduce the spatial resolution loss of predicted images, we replaced the pooling layer with a wavelet transform. Three different domains were evaluated - the sinogram domain, the image domain, and the hybrid domain - to optimize a reconstruction technique based on deep learning. To train and develop a deep learning model, The Cancer Imaging Archive (TCIA) dataset was used.

RESULTS

Streak artifacts, which generally occur under sparse sampling conditions, were effectively removed from deep learning-based sparsely sampled reconstructed images. However, image characteristics of fine structures varied depending on the application of deep learning technologies. The use of deep learning techniques in the sinogram domain removed streak artifacts well, but some image noise remained. Likewise, when applying deep learning technology to the image domain, a blurring effect occurred. The proposed hybrid domain sparsely sampled reconstruction based on deep learning was able to restore images to a quality similar to fully sampled images. The structural similarity (SSIM) index values of sparsely sampled CT reconstruction based on deep learning technology were 0.85 or higher. Among the three domains studied, the hybrid domain techniques achieved the highest SSIM index values (0.9 or more).

CONCLUSION

We proposed a method of sparsely sampled CT reconstruction from a new perspective - unlike iterative reconstruction. In addition, we developed an optimal deep learning-based sparse sampling reconstruction technique by evaluating image quality with deep learning technologies.

Deep Residual Learning for Accelerated MRI Using Magnitude and Phase Networks

OBJECTIVE

Accelerated magnetic resonance (MR) image acquisition with compressed sensing (CS) and parallel imaging is a powerful method to reduce MR imaging scan time. However, many reconstruction algorithms have high computational costs. To address this, we investigate deep residual learning networks to remove aliasing artifacts from artifact corrupted images.

METHODS

The deep residual learning networks are composed of magnitude and phase networks that are separately trained. If both phase and magnitude information are available, the proposed algorithm can work as an iterative k-space interpolation algorithm using framelet representation. When only magnitude data are available, the proposed approach works as an image domain postprocessing algorithm.

RESULTS

Even with strong coherent aliasing artifacts, the proposed network successfully learned and removed the aliasing artifacts, whereas current parallel and CS reconstruction methods were unable to remove these artifacts.

CONCLUSION

Comparisons using single and multiple coil acquisition show that the proposed residual network provides good reconstruction results with orders of magnitude faster computational time than existing CS methods.

SIGNIFICANCE

The proposed deep learning framework may have a great potential for accelerated MR reconstruction by generating accurate results immediately.