Semi-supervised iterative adaptive network for low-dose CT sinogram recovery

Background.Concern has been expressed regarding the risk of carcinogenesis from medical computed tomography (CT) radiation. Lowering radiation in CT without appropriate modifications often leads to severe noise-induced artifacts in the images. The utilization of deep learning (DL) techniques has achieved promising reconstruction performance in low-dose CT (LDCT) imaging. However, most DL-based algorithms require the pre-collection of a large set of image pairs (low-dose/standard-dose) and the training of networks in an end-to-end supervised manner. Meanwhile, securing such a large volume of paired, well-registered training data in clinical practice is challenging. Moreover, these algorithms often overlook the potential to utilize the abundant information in a large collection of LDCT-only images/sinograms.Methods.In this paper, we introduce a semi-supervised iterative adaptive network (SIA-Net) for LDCT imaging, utilizing both labeled and unlabeled sinograms in a cohesive network framework, integrating supervised and unsupervised learning processes. Specifically, the supervised process captures critical features (i.e. noise distribution and tissue characteristics) latent in the paired sinograms, while the unsupervised process effectively learns these features in the unlabeled low-dose sinograms, employing a conventional weighted least-squares model with a regularization term. Furthermore, the SIA-Net method is designed to adaptively transfer the learned feature distribution from the supervised to the unsupervised process, thereby obtaining a high-fidelity sinogram through iterative adaptive learning. Finally, high-quality CT images can be reconstructed from the refined sinogram using the filtered back-projection algorithm.Results.Experimental results on two clinical datasets indicate that the proposed SIA-Net method achieves competitive performance in terms of noise reduction and structure preservation in LDCT imaging, when compared to traditional supervised learning methods.

Spectrum learning for super-resolution tomographic reconstruction

Objective. Computed Tomography (CT) has been widely used in industrial high-resolution non-destructive testing. However, it is difficult to obtain high-resolution images for large-scale objects due to their physical limitations. The objective is to develop an improved super-resolution technique that preserves small structures and details while efficiently capturing high-frequency information.Approach. The study proposes a new deep learning based method called spectrum learning (SPEAR) network for CT images super-resolution. This approach leverages both global information in the image domain and high-frequency information in the frequency domain. The SPEAR network is designed to reconstruct high-resolution images from low-resolution inputs by considering not only the main body of the images but also the small structures and other details. The symmetric property of the spectrum is exploited to reduce weight parameters in the frequency domain. Additionally, a spectrum loss is introduced to enforce the preservation of both high-frequency components and global information.Main results. The network is trained using pairs of low-resolution and high-resolution CT images, and it is fine-tuned using additional low-dose and normal-dose CT image pairs. The experimental results demonstrate that the proposed SPEAR network outperforms state-of-the-art networks in terms of image reconstruction quality. The approach successfully preserves high-frequency information and small structures, leading to better results compared to existing methods. The network's ability to generate high-resolution images from low-resolution inputs, even in cases of low-dose CT images, showcases its effectiveness in maintaining image quality.Significance. The proposed SPEAR network's ability to simultaneously capture global information and high-frequency details addresses the limitations of existing methods, resulting in more accurate and informative image reconstructions. This advancement can have substantial implications for various industries and medical diagnoses relying on accurate imaging.

Self-supervised dual-domain balanced dropblock-network for low-dose CT denoising

Objective.Self-supervised learning methods have been successfully applied for low-dose computed tomography (LDCT) denoising, with the advantage of not requiring labeled data. Conventional self-supervised methods operate only in the image domain, ignoring valuable priors in the sinogram domain. Recently proposed dual-domain methods address this limitation but encounter issues with blurring artifacts in the reconstructed image due to the inhomogeneous distribution of noise levels in low-dose sinograms.Approach.To tackle this challenge, this paper proposes SDBDNet, an end-to-end dual-domain self-supervised method for LDCT denoising. With the network designed based on the properties of inhomogeneous noise in low-dose sinograms and the principle of moderate sinogram-domain denoising, SDBDNet achieves effective denoising in dual domains without introducing blurring artifacts. Specifically, we split the sinogram into two subsets based on the positions of detector cells to generate paired training data with high similarity and independent noise. These sub-sinograms are then restored to their original size using 1D interpolation and learning-based correction. To achieve adaptive and moderate smoothing in the sinogram domain, we integrate Dropblock, a type of convolution layer with regularization, into SDBDNet, and set a weighted average between the denoised sinograms and their noisy counterparts, leading to a well-balanced dual-domain approach.Main results.Numerical experiments show that our method outperforms popular non-learning and self-supervised learning methods, demonstrating its effectiveness and superior performance.Significance.While introducing a novel high-performance dual-domain self-supervised LDCT denoising method, this paper also emphasizes and verifies the importance of appropriate sinogram-domain denoising in dual-domain methods, which might inspire future work.

Decomposition iteration strategy for low-dose CT denoising

In the medical field, computed tomography (CT) is a commonly used examination method, but the radiation generated increases the risk of illness in patients. Therefore, low-dose scanning schemes have attracted attention, in which noise reduction is essential. We propose a purposeful and interpretable decomposition iterative network (DISN) for low-dose CT denoising. This method aims to make the network design interpretable and improve the fidelity of details, rather than blindly designing or using deep CNN architecture. The experiment is trained and tested on multiple data sets. The results show that the DISN method can restore the low-dose CT image structure and improve the diagnostic performance when the image details are limited. Compared with other algorithms, DISN has better quantitative and visual performance, and has potential clinical application prospects.

A Deep Convolutional Gated Recurrent Unit for CT Image Reconstruction

Computed tomography (CT) is one of the most important medical imaging technologies in use today. Most commercial CT products use a technique known as the filtered backprojection (FBP) that is fast and can produce decent image quality when an X-ray dose is high. However, the FBP is not good enough on low-dose X-ray CT imaging because the CT image reconstruction problem becomes more stochastic. A more effective reconstruction technique proposed recently and implemented in a limited number of CT commercial products is an iterative reconstruction (IR). The IR technique is based on a Bayesian formulation of the CT image reconstruction problem with an explicit model of the CT scanning, including its stochastic nature, and a prior model that incorporates our knowledge about what a good CT image should look like. However, constructing such prior knowledge is more complicated than it seems. In this article, we propose a novel neural network for CT image reconstruction. The network is based on the IR formulation and constructed with a recurrent neural network (RNN). Specifically, we transform the gated recurrent unit (GRU) into a neural network performing CT image reconstruction. We call it "GRU reconstruction." This neural network conducts concurrent dual-domain learning. Many deep learning (DL)-based methods in medical imaging are single-domain learning, but dual-domain learning performs better because it learns from both the sinogram and the image domain. In addition, we propose backpropagation through stage (BPTS) as a new RNN backpropagation algorithm. It is similar to the backpropagation through time (BPTT) of an RNN; however, it is tailored for iterative optimization. Results from extensive experiments indicate that our proposed method outperforms conventional model-based methods, single-domain DL methods, and state-of-the-art DL techniques in terms of the root mean squared error (RMSE), the peak signal-to-noise ratio (PSNR), and the structure similarity (SSIM) and in terms of visual appearance.

Cross-Domain Unpaired Learning for Low-Dose CT Imaging

Supervised deep-learning techniques with paired training datasets have been widely studied for low-dose computed tomography (LDCT) imaging with excellent performance. However, the paired training datasets are usually difficult to obtain in clinical routine, which restricts the wide adoption of supervised deep-learning techniques in clinical practices. To address this issue, a general idea is to construct a pseudo paired training dataset based on the widely available unpaired data, after which, supervised deep-learning techniques can be adopted for improving the LDCT imaging performance by training on the pseudo paired training dataset. However, due to the complexity of noise properties in CT imaging, the LDCT data are difficult to generate in order to construct the pseudo paired training dataset. In this article, we propose a simple yet effective cross-domain unpaired learning framework for pseudo LDCT data generation and LDCT image reconstruction, which is denoted as CrossDuL. Specifically, a dedicated pseudo LDCT sinogram generative module is constructed based on a data-dependent noise model in the sinogram domain, and then instead of in the sinogram domain, a pseudo paired dataset is constructed in the image domain to train an LDCT image restoration module. To validate the effectiveness of the proposed framework, clinical datasets are adopted. Experimental results demonstrate that the CrossDuL framework can obtain promising LDCT imaging performance in both quantitative and qualitative measurements.

Multi-layer clustering-based residual sparsifying transform for low-dose CT image reconstruction

PURPOSE

The recently proposed sparsifying transform (ST) models incur low computational cost and have been applied to medical imaging. Meanwhile, deep models with nested network structure reveal great potential for learning features in different layers. In this study, we propose a network-structured ST learning approach for X-ray computed tomography (CT), which we refer to as multi-layer clustering-based residual sparsifying transform (MCST) learning. The proposed MCST scheme learns multiple different unitary transforms in each layer by dividing each layer's input into several classes. We apply the MCST model to low-dose CT (LDCT) reconstruction by deploying the learned MCST model into the regularizer in penalized weighted least squares (PWLS) reconstruction.

METHODS

The proposed MCST model combines a multi-layer sparse representation structure with multiple clusters for the features in each layer that are modeled by a rich collection of transforms. We train the MCST model in an unsupervised manner via a block coordinate descent (BCD) algorithm. Since our method is patch-based, the training can be performed with a limited set of images. For CT image reconstruction, we devise a novel algorithm called PWLS-MCST by integrating the pre-learned MCST signal model with PWLS optimization.

RESULTS

We conducted LDCT reconstruction experiments on XCAT phantom data, Numerical Mayo Clinical CT dataset and "LDCT image and projection dataset" (Clinical LDCT dataset). We trained the MCST model with two (or three) layers and with five clusters in each layer. The learned transforms in the same layer showed rich features while additional information is extracted from representation residuals. Our simulation results and clinical results demonstrate that PWLS-MCST achieves better image reconstruction quality than the conventional filtered back-projection (FBP) method and PWLS with edge-preserving (EP) regularizer. It also outperformed recent advanced methods like PWLS with a learned multi-layer residual sparsifying transform (MARS) prior and PWLS with a union of learned transforms (ULTRA), especially for displaying clear edges and preserving subtle details.

CONCLUSIONS

In this work, a multi-layer sparse signal model with a nested network structure is proposed. We refer this novel model as the MCST model that exploits multi-layer residual maps to sparsify the underlying image and clusters the inputs in each layer for accurate sparsification. We presented a new PWLS framework with a learned MCST regularizer for LDCT reconstruction. Experimental results show that the proposed PWLS-MCST provides clearer reconstructions than several baseline methods. The code for PWLS-MCST is released at https://github.com/Xikai97/PWLS-MCST.

Dual-domain fusion deep convolutional neural network for low-dose CT denoising

BACKGROUND

In view of the underlying health risks posed by X-ray radiation, the main goal of the present research is to achieve high-quality CT images at the same time as reducing x-ray radiation. In recent years, convolutional neural network (CNN) has shown excellent performance in removing low-dose CT noise. However, previous work mainly focused on deepening and feature extraction work on CNN without considering fusion of features from frequency domain and image domain.

OBJECTIVE

To address this issue, we propose to develop and test a new LDCT image denoising method based on a dual-domain fusion deep convolutional neural network (DFCNN).

METHODS

This method deals with two domains, namely, the DCT domain and the image domain. In the DCT domain, we design a new residual CBAM network to enhance the internal and external relations of different channels while reducing noise to promote richer image structure information. For the image domain, we propose a top-down multi-scale codec network as a denoising network to obtain more acceptable edges and textures while obtaining multi-scale information. Then, the feature images of the two domains are fused by a combination network.

RESULTS

The proposed method was validated on the Mayo dataset and the Piglet dataset. The denoising algorithm is optimal in both subjective and objective evaluation indexes as compared to other state-of-the-art methods reported in previous studies.

CONCLUSIONS

The study results demonstrate that by applying the new fusion model denoising, denoising results in both image domain and DCT domain are better than other models developed using features extracted in the single image domain.

Compound feature attention network with edge enhancement for low-dose CT denoising

BACKGROUND

Low-dose CT (LDCT) images usually contain serious noise and artifacts, which weaken the readability of the image.

OBJECTIVE

To solve this problem, we propose a compound feature attention network with edge enhancement for LDCT denoising (CFAN-Net), which consists of an edge-enhanced module and a proposed compound feature attention block (CFAB).

METHODS

The edge enhancement module extracts edge details with the trainable Sobel convolution. CFAB consists of an interactive feature learning module (IFLM), a multi-scale feature fusion module (MFFM), and a joint attention module (JAB), which removes noise from LDCT images in a coarse-to-fine manner. First, in IFLM, the noise is initially removed by cross-latitude interactive judgment learning. Second, in MFFM, multi-scale and pixel attention are integrated to explore fine noise removal. Finally, in JAB, we focus on key information, extract useful features, and improve the efficiency of network learning. To construct a high-quality image, we repeat the above operation by cascading CFAB.

RESULTS

By applying CFAN-Net to process the 2016 NIH AAPM-Mayo LDCT challenge test dataset, experiments show that the peak signal-to-noise ratio value is 33.9692 and the structural similarity value is 0.9198.

CONCLUSIONS

Compared with several existing LDCT denoising algorithms, CFAN-Net effectively preserves the texture of CT images while removing noise and artifacts.

Texture transformer super-resolution for low-dose computed tomography

Computed tomography (CT) is widely used to diagnose many diseases. Low-dose CT has been actively pursued to lower the ionizing radiation risk. A relatively smoother kernel is typically used in low-dose CT to suppress image noise, which may sacrifice spatial resolution. In this work, we propose a texture transformer network to simultaneously reduce image noise and improve spatial resolution in CT images. This network, referred to as Texture Transformer for Super Resolution (TTSR), is a reference-based deep-learning image super-resolution method built upon a generative adversarial network (GAN). The noisy low-resolution CT (LRCT) image and the routine-dose high-resolution (HRCT) image are severed as the query and key in a transformer, respectively. Image translation is optimized through deep neural network (DNN) texture extraction, correlation embedding, and attention-based texture transfer and synthesis to achieve joint feature learning between LRCT and HRCT images for super-resolution CT (SRCT) images. To evaluate SRCT performance, we use the data from both simulations of the XCAT phantom program and the real patient data. Peak signal-to-noise ratio (PSNR), structural similarity index measure (SSIM), and feature similarity (FSIM) index are used as quantitative metrics. For comparison of SRCT performance, the cubic spline interpolation, SRGAN (a GAN super-resolution with an additional content loss), and GAN-CIRCLE (a GAN super-resolution with cycle consistency) were used. Compared to the other two methods, TTSR can restore more details in SRCT images and achieve better PSNR, SSIM, and FSIM for both simulation and real-patient data. In addition, we show that TTSR can yield better image quality and demand much less computation time than high-resolution low-dose CT images denoised by block-matching and 3D filtering (BM3D) and GAN-CIRCLE. In summary, the proposed TTSR method based on texture transformer and attention mechanism provides an effective and efficient tool to improve spatial resolution and suppress noise of low-dose CT images.

Edge feature extraction-based dual CNN for LDCT denoising

In low-dose computed tomography (LDCT) denoising tasks, it is often difficult to balance edge/detail preservation and noise/artifact reduction. To solve this problem, we propose a dual convolutional neural network (CNN) based on edge feature extraction (Ed-DuCNN) for LDCT. Ed-DuCNN consists of two branches. One branch is the edge feature extraction subnet (Edge_Net) that can fully extract the edge details in the image. The other branch is the feature fusion subnet (Fusion_Net) that introduces an attention mechanism to fuse edge features and noisy image features. Specifically, first, shallow edge-specific detail features are extracted by trainable Sobel convolutional blocks and then are integrated into Edge_Net together with the LDCT images to obtain deep edge detail features. Finally, the input image, shallow edge detail, and deep edge detail features are fused in Fusion_Net to generate the final denoised image. The experimental results show that the proposed Ed-DuCNN can achieve competitive performance in terms of quantitative metrics and visual perceptual quality compared with that of state-of-the-art methods.

LRR-CED: low-resolution reconstruction-aware convolutional encoder–decoder network for direct sparse-view CT image reconstruction

Objective. Sparse-view computed tomography (CT) reconstruction has been at the forefront of research in medical imaging. Reducing the total x-ray radiation dose to the patient while preserving the reconstruction accuracy is a big challenge. The sparse-view approach is based on reducing the number of rotation angles, which leads to poor quality reconstructed images as it introduces several artifacts. These artifacts are more clearly visible in traditional reconstruction methods like the filtered-backprojection (FBP) algorithm. Approach. Over the years, several model-based iterative and more recently deep learning-based methods have been proposed to improve sparse-view CT reconstruction. Many deep learning-based methods improve FBP-reconstructed images as a post-processing step. In this work, we propose a direct deep learning-based reconstruction that exploits the information from low-dimensional scout images, to learn the projection-to-image mapping. This is done by concatenating FBP scout images at multiple resolutions in the decoder part of a convolutional encoder–decoder (CED). Main results. This approach is investigated on two different networks, based on Dense Blocks and U-Net to show that a direct mapping can be learned from a sinogram to an image. The results are compared to two post-processing deep learning methods (FBP-ConvNet and DD-Net) and an iterative method that uses a total variation (TV) regularization. Significance. This work presents a novel method that uses information from both sinogram and low-resolution scout images for sparse-view CT image reconstruction. We also generalize this idea by demonstrating results with two different neural networks. This work is in the direction of exploring deep learning across the various stages of the image reconstruction pipeline involving data correction, domain transfer and image improvement.

End-to-end deep learning for interior tomography with low-dose x-ray CT

Objective. There are several x-ray computed tomography (CT) scanning strategies used to reduce radiation dose, such as (1) sparse-view CT, (2) low-dose CT and (3) region-of-interest (ROI) CT (called interior tomography). To further reduce the dose, sparse-view and/or low-dose CT settings can be applied together with interior tomography. Interior tomography has various advantages in terms of reducing the number of detectors and decreasing the x-ray radiation dose. However, a large patient or a small field-of-view (FOV) detector can cause truncated projections, and then the reconstructed images suffer from severe cupping artifacts. In addition, although low-dose CT can reduce the radiation exposure dose, analytic reconstruction algorithms produce image noise. Recently, many researchers have utilized image-domain deep learning (DL) approaches to remove each artifact and demonstrated impressive performances, and the theory of deep convolutional framelets supports the reason for the performance improvement. Approach. In this paper, we found that it is difficult to solve coupled artifacts using an image-domain convolutional neural network (CNN) based on deep convolutional framelets. Significance. To address the coupled problem, we decouple it into two sub-problems: (i) image-domain noise reduction inside the truncated projection to solve low-dose CT problem and (ii) extrapolation of the projection outside the truncated projection to solve the ROI CT problem. The decoupled sub-problems are solved directly with a novel proposed end-to-end learning method using dual-domain CNNs. Main results. We demonstrate that the proposed method outperforms the conventional image-domain DL methods, and a projection-domain CNN shows better performance than the image-domain CNNs commonly used by many researchers.

Artifact and Detail Attention Generative Adversarial Networks for Low-Dose CT Denoising

Generative adversarial networks are being extensively studied for low-dose computed tomography denoising. However, due to the similar distribution of noise, artifacts, and high-frequency components of useful tissue images, it is difficult for existing generative adversarial network-based denoising networks to effectively separate the artifacts and noise in the low-dose computed tomography images. In addition, aggressive denoising may damage the edge and structural information of the computed tomography image and make the denoised image too smooth. To solve these problems, we propose a novel denoising network called artifact and detail attention generative adversarial network. First, a multi-channel generator is proposed. Based on the main feature extraction channel, an artifacts and noise attention channel and an edge feature attention channel are added to improve the denoising network's ability to pay attention to the noise and artifacts features and edge features of the image. Additionally, a new structure called multi-scale Res2Net discriminator is proposed, and the receptive field in the module is expanded by extracting the multi-scale features in the same scale of the image to improve the discriminative ability of discriminator. The loss functions are specially designed for each sub-channel of the denoising network corresponding to its function. Through the cooperation of multiple loss functions, the convergence speed, stability, and denoising effect of the network are accelerated, improved, and guaranteed, respectively. Experimental results show that the proposed denoising network can preserve the important information of the low-dose computed tomography image and achieve better denoising effect when compared to the state-of-the-art algorithms.

MAGIC: Manifold and Graph Integrative Convolutional Network for Low-Dose CT Reconstruction

Low-dose computed tomography (LDCT) scans, which can effectively alleviate the radiation problem, will degrade the imaging quality. In this paper, we propose a novel LDCT reconstruction network that unrolls the iterative scheme and performs in both image and manifold spaces. Because patch manifolds of medical images have low-dimensional structures, we can build graphs from the manifolds. Then, we simultaneously leverage the spatial convolution to extract the local pixel-level features from the images and incorporate the graph convolution to analyze the nonlocal topological features in manifold space. The experiments show that our proposed method outperforms both the quantitative and qualitative aspects of state-of-the-art methods. In addition, aided by a projection loss component, our proposed method also demonstrates superior performance for semi-supervised learning. The network can remove most noise while maintaining the details of only 10% (40 slices) of the training data labeled.

Low-dose CT reconstruction with Noise2Noise network and testing-time fine-tuning

PURPOSE

Deep learning-based image denoising and reconstruction methods demonstrated promising performance on low-dose CT imaging in recent years. However, most existing deep learning-based low-dose CT reconstruction methods require normal-dose images for training. Sometimes such clean images do not exist such as for dynamic CT imaging or very large patients. The purpose of this work is to develop a low-dose CT image reconstruction algorithm based on deep learning which does not need clean images for training.

METHODS

In this paper, we proposed a novel reconstruction algorithm where the image prior was expressed via the Noise2Noise network, whose weights were fine-tuned along with the image during the iterative reconstruction. The Noise2Noise network built a self-consistent loss by projection data splitting and mapping the corresponding filtered backprojection (FBP) results to each other with a deep neural network. Besides, the network weights are optimized along with the image to be reconstructed under an alternating optimization scheme. In the proposed method, no clean image is needed for network training and the testing-time fine-tuning leads to optimization for each reconstruction.

RESULTS

We used the 2016 Low-dose CT Challenge dataset to validate the feasibility of the proposed method. We compared its performance to several existing iterative reconstruction algorithms that do not need clean training data, including total variation, non-local mean, convolutional sparse coding, and Noise2Noise denoising. It was demonstrated that the proposed Noise2Noise reconstruction achieved better RMSE, SSIM and texture preservation compared to the other methods. The performance is also robust against the different noise levels, hyperparameters, and network structures used in the reconstruction. Furthermore, we also demonstrated that the proposed methods achieved competitive results without any pre-training of the network at all, that is, using randomly initialized network weights during testing. The proposed iterative reconstruction algorithm also has empirical convergence with and without network pre-training.

CONCLUSIONS

The proposed Noise2Noise reconstruction method can achieve promising image quality in low-dose CT image reconstruction. The method works both with and without pre-training, and only noisy data are required for pre-training.

Unified Supervised-Unsupervised (SUPER) Learning for X-Ray CT Image Reconstruction

Traditional model-based image reconstruction (MBIR) methods combine forward and noise models with simple object priors. Recent machine learning methods for image reconstruction typically involve supervised learning or unsupervised learning, both of which have their advantages and disadvantages. In this work, we propose a unified supervised-unsupervised (SUPER) learning framework for X-ray computed tomography (CT) image reconstruction. The proposed learning formulation combines both unsupervised learning-based priors (or even simple analytical priors) together with (supervised) deep network-based priors in a unified MBIR framework based on a fixed point iteration analysis. The proposed training algorithm is also an approximate scheme for a bilevel supervised training optimization problem, wherein the network-based regularizer in the lower-level MBIR problem is optimized using an upper-level reconstruction loss. The training problem is optimized by alternating between updating the network weights and iteratively updating the reconstructions based on those weights. We demonstrate the learned SUPER models' efficacy for low-dose CT image reconstruction, for which we use the NIH AAPM Mayo Clinic Low Dose CT Grand Challenge dataset for training and testing. In our experiments, we studied different combinations of supervised deep network priors and unsupervised learning-based or analytical priors. Both numerical and visual results show the superiority of the proposed unified SUPER methods over standalone supervised learning-based methods, iterative MBIR methods, and variations of SUPER obtained via ablation studies. We also show that the proposed algorithm converges rapidly in practice.

Downsampled Imaging Geometric Modeling for Accurate CT Reconstruction via Deep Learning

X-ray computed tomography (CT) is widely used clinically to diagnose a variety of diseases by reconstructing the tomographic images of a living subject using penetrating X-rays. For accurate CT image reconstruction, a precise imaging geometric model for the radiation attenuation process is usually required to solve the inversion problem of CT scanning, which encodes the subject into a set of intermediate representations in different angular positions. Here, we show that accurate CT image reconstruction can be subsequently achieved by downsampled imaging geometric modeling via deep-learning techniques. Specifically, we first propose a downsampled imaging geometric modeling approach for the data acquisition process and then incorporate it into a hierarchical neural network, which simultaneously combines both geometric modeling knowledge of the CT imaging system and prior knowledge gained from a data-driven training process for accurate CT image reconstruction. The proposed neural network is denoted as DSigNet, i.e., downsampled-imaging-geometry-based network for CT image reconstruction. We demonstrate the feasibility of the proposed DSigNet for accurate CT image reconstruction with clinical patient data. In addition to improving the CT image quality, the proposed DSigNet might help reduce the computational complexity and accelerate the reconstruction speed for modern CT imaging systems.

Multilayer residual sparsifying transform (MARS) model for low-dose CT image reconstruction

PURPOSE

Signal models based on sparse representations have received considerable attention in recent years. On the other hand, deep models consisting of a cascade of functional layers, commonly known as deep neural networks, have been highly successful for the task of object classification and have been recently introduced to image reconstruction. In this work, we develop a new image reconstruction approach based on a novel multilayer model learned in an unsupervised manner by combining both sparse representations and deep models. The proposed framework extends the classical sparsifying transform model for images to a Multilayer residual sparsifying transform (MARS) model, wherein the transform domain data are jointly sparsified over layers. We investigate the application of MARS models learned from limited regular-dose images for low-dose CT reconstruction using penalized weighted least squares (PWLS) optimization.

METHODS

We propose new formulations for multilayer transform learning and image reconstruction. We derive an efficient block coordinate descent algorithm to learn the transforms across layers, in an unsupervised manner from limited regular-dose images. The learned model is then incorporated into the low-dose image reconstruction phase.

RESULTS

Low-dose CT experimental results with both the XCAT phantom and Mayo Clinic data show that the MARS model outperforms conventional methods such as filtered back-projection and PWLS methods based on the edge-preserving (EP) regularizer in terms of two numerical metrics (RMSE and SSIM) and noise suppression. Compared with the single-layer learned transform (ST) model, the MARS model performs better in maintaining some subtle details.

CONCLUSIONS

This work presents a novel data-driven regularization framework for CT image reconstruction that exploits learned multilayer or cascaded residual sparsifying transforms. The image model is learned in an unsupervised manner from limited images. Our experimental results demonstrate the promising performance of the proposed multilayer scheme over single-layer learned sparsifying transforms. Learned MARS models also offer better image quality than typical nonadaptive PWLS methods.

Detecting failure modes in image reconstructions with interval neural network uncertainty

Purpose

The quantitative detection of failure modes is important for making deep neural networks reliable and usable at scale. We consider three examples for common failure modes in image reconstruction and demonstrate the potential of uncertainty quantification as a fine-grained alarm system.

Methods

We propose a deterministic, modular and lightweight approach called Interval Neural Network (INN) that produces fast and easy to interpret uncertainty scores for deep neural networks. Importantly, INNs can be constructed post hoc for already trained prediction networks. We compare it against state-of-the-art baseline methods (MCDrop, ProbOut).

Results

We demonstrate on controlled, synthetic inverse problems the capacity of INNs to capture uncertainty due to noise as well as directional error information. On a real-world inverse problem with human CT scans, we can show that INNs produce uncertainty scores which improve the detection of all considered failure modes compared to the baseline methods.

Conclusion

Interval Neural Networks offer a promising tool to expose weaknesses of deep image reconstruction models and ultimately make them more reliable. The fact that they can be applied post hoc to equip already trained deep neural network models with uncertainty scores makes them particularly interesting for deployment.

Current status of deep learning in abdominal image reconstruction

Abdominal magnetic resonance imaging (MRI) and computed tomography (CT) are commonly used for disease screening, diagnosis, and treatment guidance. However, abdominal MRI has disadvantages including slow speed and vulnerability to motions, while CT suffers from problems of radiation. It has been reported that deep learning reconstruction can solve such problems while maintaining good image quality. Recently, deep learning-based image reconstruction has become a hot topic in the field of medical imaging. This study reviews the latest research on deep learning reconstruction in abdominal imaging, including the widely used convolutional neural network, generative adversarial network, and recurrent neural network.

A Green Prospective for Learned Post-Processing in Sparse-View Tomographic Reconstruction

Deep Learning is developing interesting tools that are of great interest for inverse imaging applications. In this work, we consider a medical imaging reconstruction task from subsampled measurements, which is an active research field where Convolutional Neural Networks have already revealed their great potential. However, the commonly used architectures are very deep and, hence, prone to overfitting and unfeasible for clinical usages. Inspired by the ideas of the green AI literature, we propose a shallow neural network to perform efficient Learned Post-Processing on images roughly reconstructed by the filtered backprojection algorithm. The results show that the proposed inexpensive network computes images of comparable (or even higher) quality in about one-fourth of time and is more robust than the widely used and very deep ResUNet for tomographic reconstructions from sparse-view protocols.

Limited View Tomographic Reconstruction Using a Cascaded Residual Dense Spatial-Channel Attention Network With Projection Data Fidelity Layer

Limited view tomographic reconstruction aims to reconstruct a tomographic image from a limited number of projection views arising from sparse view or limited angle acquisitions that reduce radiation dose or shorten scanning time. However, such a reconstruction suffers from severe artifacts due to the incompleteness of sinogram. To derive quality reconstruction, previous methods use UNet-like neural architectures to directly predict the full view reconstruction from limited view data; but these methods leave the deep network architecture issue largely intact and cannot guarantee the consistency between the sinogram of the reconstructed image and the acquired sinogram, leading to a non-ideal reconstruction. In this work, we propose a cascaded residual dense spatial-channel attention network consisting of residual dense spatial-channel attention networks and projection data fidelity layers. We evaluate our methods on two datasets. Our experimental results on AAPM Low Dose CT Grand Challenge datasets demonstrate that our algorithm achieves a consistent and substantial improvement over the existing neural network methods on both limited angle reconstruction and sparse view reconstruction. In addition, our experimental results on Deep Lesion datasets demonstrate that our method is able to generate high-quality reconstruction for 8 major lesion types.

FISTA-Net: Learning a Fast Iterative Shrinkage Thresholding Network for Inverse Problems in Imaging

Inverse problems are essential to imaging applications. In this letter, we propose a model-based deep learning network, named FISTA-Net, by combining the merits of interpretability and generality of the model-based Fast Iterative Shrinkage/Thresholding Algorithm (FISTA) and strong regularization and tuning-free advantages of the data-driven neural network. By unfolding the FISTA into a deep network, the architecture of FISTA-Net consists of multiple gradient descent, proximal mapping, and momentum modules in cascade. Different from FISTA, the gradient matrix in FISTA-Net can be updated during iteration and a proximal operator network is developed for nonlinear thresholding which can be learned through end-to-end training. Key parameters of FISTA-Net including the gradient step size, thresholding value and momentum scalar are tuning-free and learned from training data rather than hand-crafted. We further impose positive and monotonous constraints on these parameters to ensure they converge properly. The experimental results, evaluated both visually and quantitatively, show that the FISTA-Net can optimize parameters for different imaging tasks, i.e. Electromagnetic Tomography (EMT) and X-ray Computational Tomography (X-ray CT). It outperforms the state-of-the-art model-based and deep learning methods and exhibits good generalization ability over other competitive learning-based approaches under different noise levels.

LoDoPaB-CT, a benchmark dataset for low-dose computed tomography reconstruction

Deep learning approaches for tomographic image reconstruction have become very effective and have been demonstrated to be competitive in the field. Comparing these approaches is a challenging task as they rely to a great extent on the data and setup used for training. With the Low-Dose Parallel Beam (LoDoPaB)-CT dataset, we provide a comprehensive, open-access database of computed tomography images and simulated low photon count measurements. It is suitable for training and comparing deep learning methods as well as classical reconstruction approaches. The dataset contains over 40000 scan slices from around 800 patients selected from the LIDC/IDRI database. The data selection and simulation setup are described in detail, and the generating script is publicly accessible. In addition, we provide a Python library for simplified access to the dataset and an online reconstruction challenge. Furthermore, the dataset can also be used for transfer learning as well as sparse and limited-angle reconstruction scenarios.

Measurement(s)	Low Dose Computed Tomography of the Chest • feature extraction objective
Technology Type(s)	digital curation • image processing technique
Sample Characteristic - Organism	Homo sapiens

Machine-accessible metadata file describing the reported data: 10.6084/m9.figshare.13526360

MetaInv-Net: Meta Inversion Network for Sparse View CT Image Reconstruction

X-ray Computed Tomography (CT) is widely used in clinical applications such as diagnosis and image-guided interventions. In this paper, we propose a new deep learning based model for CT image reconstruction with the backbone network architecture built by unrolling an iterative algorithm. However, unlike the existing strategy to include as many data-adaptive components in the unrolled dynamics model as possible, we find that it is enough to only learn the parts where traditional designs mostly rely on intuitions and experience. More specifically, we propose to learn an initializer for the conjugate gradient (CG) algorithm that involved in one of the subproblems of the backbone model. Other components, such as image priors and hyperparameters, are kept as the original design. Since a hypernetwork is introduced to inference on the initialization of the CG module, it makes the proposed model a certain meta-learning model. Therefore, we shall call the proposed model the meta-inversion network (MetaInv-Net). The proposed MetaInv-Net can be designed with much less trainable parameters while still preserves its superior image reconstruction performance than some state-of-the-art deep models in CT imaging. In simulated and real data experiments, MetaInv-Net performs very well and can be generalized beyond the training setting, i.e., to other scanning settings, noise levels, and data sets.

Radon Inversion via Deep Learning

The Radon transform is widely used in physical and life sciences, and one of its major applications is in medical X-ray computed tomography (CT), which is significantly important in disease screening and diagnosis. In this paper, we propose a novel reconstruction framework for Radon inversion with deep learning (DL) techniques. For simplicity, the proposed framework is denoted as iRadonMAP, i.e., inverse Radon transform approximation. Specifically, we construct an interpretable neural network that contains three dedicated components. The first component is a fully connected filtering (FCF) layer along the rotation angle direction in the sinogram domain, and the second one is a sinusoidal back-projection (SBP) layer, which back-projects the filtered sinogram data into the spatial domain. Next, a common network structure is added to further improve the overall performance. iRadonMAP is first pretrained on a large number of generic images from the ImageNet database and then fine-tuned with clinical patient data. The experimental results demonstrate the feasibility of the proposed iRadonMAP framework for Radon inversion.

Multi-Scale Learned Iterative Reconstruction

Model-based learned iterative reconstruction methods have recently been shown to outperform classical reconstruction algorithms. Applicability of these methods to large scale inverse problems is however limited by the available memory for training and extensive training times, the latter due to computationally expensive forward models. As a possible solution to these restrictions we propose a multi-scale learned iterative reconstruction scheme that computes iterates on discretisations of increasing resolution. This procedure does not only reduce memory requirements, it also considerably speeds up reconstruction and training times, but most importantly is scalable to large scale inverse problems with non-trivial forward operators, such as those that arise in many 3D tomographic applications. In particular, we propose a hybrid network that combines the multiscale iterative approach with a particularly expressive network architecture which in combination exhibits excellent scalability in 3D. Applicability of the algorithm is demonstrated for 3D cone beam computed tomography from real measurement data of an organic phantom. Additionally, we examine scalability and reconstruction quality in comparison to established learned reconstruction methods in two dimensions for low dose computed tomography on human phantoms.

CNN-Based Projected Gradient Descent for Consistent CT Image Reconstruction

We present a new image reconstruction method that replaces the projector in a projected gradient descent (PGD) with a convolutional neural network (CNN). Recently, CNNs trained as image-to-image regressors have been successfully used to solve inverse problems in imaging. However, unlike existing iterative image reconstruction algorithms, these CNN-based approaches usually lack a feedback mechanism to enforce that the reconstructed image is consistent with the measurements. We propose a relaxed version of PGD wherein gradient descent enforces measurement consistency, while a CNN recursively projects the solution closer to the space of desired reconstruction images. We show that this algorithm is guaranteed to converge and, under certain conditions, converges to a local minimum of a non-convex inverse problem. Finally, we propose a simple scheme to train the CNN to act like a projector. Our experiments on sparse-view computed-tomography reconstruction show an improvement over total variation-based regularization, dictionary learning, and a state-of-the-art deep learning-based direct reconstruction technique.

Penalized PET Reconstruction Using Deep Learning Prior and Local Linear Fitting

Motivated by the great potential of deep learning in medical imaging, we propose an iterative positron emission tomography reconstruction framework using a deep learning-based prior. We utilized the denoising convolutional neural network (DnCNN) method and trained the network using full-dose images as the ground truth and low dose images reconstructed from downsampled data by Poisson thinning as input. Since most published deep networks are trained at a predetermined noise level, the noise level disparity of training and testing data is a major problem for their applicability as a generalized prior. In particular, the noise level significantly changes in each iteration, which can potentially degrade the overall performance of iterative reconstruction. Due to insufficient existing studies, we conducted simulations and evaluated the degradation of performance at various noise conditions. Our findings indicated that DnCNN produces additional bias induced by the disparity of noise levels. To address this issue, we propose a local linear fitting function incorporated with the DnCNN prior to improve the image quality by preventing unwanted bias. We demonstrate that the resultant method is robust against noise level disparities despite the network being trained at a predetermined noise level. By means of bias and standard deviation studies via both simulations and clinical experiments, we show that the proposed method outperforms conventional methods based on total variation and non-local means penalties. We thereby confirm that the proposed method improves the reconstruction result both quantitatively and qualitatively.

Learned Primal-Dual Reconstruction

We propose the Learned Primal-Dual algorithm for tomographic reconstruction. The algorithm accounts for a (possibly non-linear) forward operator in a deep neural network by unrolling a proximal primal-dual optimization method, but where the proximal operators have been replaced with convolutional neural networks. The algorithm is trained end-to-end, working directly from raw measured data and it does not depend on any initial reconstruction such as filtered back-projection (FBP). We compare performance of the proposed method on low dose computed tomography reconstruction against FBP, total variation (TV), and deep learning based post-processing of FBP. For the Shepp-Logan phantom we obtain >6 dB peak signal to noise ratio improvement against all compared methods. For human phantoms the corresponding improvement is 6.6 dB over TV and 2.2 dB over learned post-processing along with a substantial improvement in the structural similarity index. Finally, our algorithm involves only ten forward-back-projection computations, making the method feasible for time critical clinical applications.