Learning to Reconstruct Computed Tomography Images Directly From Sinogram Data Under A Variety of Data Acquisition Conditions

End-to-End Deep Learning CT Image Reconstruction for Metal Artifact Reduction

Metal artifacts are common in CT-guided interventions due to the presence of metallic instruments. These artifacts often obscure clinically relevant structures, which can complicate the intervention. In this work, we present a deep learning CT reconstruction called iCTU-Net for the reduction of metal artifacts. The network emulates the filtering and back projection steps of the classical filtered back projection (FBP). A U-Net is used as post-processing to refine the back projected image. The reconstruction is trained end-to-end, i.e., the inputs of the iCTU-Net are sinograms and the outputs are reconstructed images. The network does not require a predefined back projection operator or the exact X-ray beam geometry. Supervised training is performed on simulated interventional data of the abdomen. For projection data exhibiting severe artifacts, the iCTU-Net achieved reconstructions with SSIM = 0.970±0.009 and PSNR = 40.7±1.6. The best reference method, an image based post-processing network, only achieved SSIM = 0.944±0.024 and PSNR = 39.8±1.9. Since the whole reconstruction process is learned, the network was able to fully utilize the raw data, which benefited from the removal of metal artifacts. The proposed method was the only studied method that could eliminate the metal streak artifacts.

Projection-to-image transform frame: a lightweight block reconstruction network (LBRN) for computed tomography

To solve the problem of learning based computed tomography (CT) reconstruction, several reconstruction networks were invented. However, applying neural network to tomographic reconstruction still remains challenging due to unacceptable memory space requirement. In this study, we presents a novel lightweight block reconstruction network (LBRN), which transforms the reconstruction operator into a deep neural network by unrolling the filter back-projection (FBP) method. Specifically, the proposed network contains two main modules, which, respectively, correspond to the filter and back-projection of FBP method. The first module of LBRN decouples the relationship of Radon transform between the reconstructed image and the projection data. Therefore, the following module, block back-projection module, can use the block reconstruction strategy. Due to each image block is only connected with part filtered projection data, the network structure is greatly simplified and the parameters of the whole network is dramatically reduced. Moreover, this approach is trained end-to-end, working directly from raw projection data and does not depend on any initial images. Five reconstruction experiments are conducted to evaluate the performance of the proposed LBRN: full angle, low-dose CT, region of interest (ROI), metal artifacts reduction and real data experiment. The results of the experiments show that the LBRN can be effectively introduced into the reconstruction process and has outstanding advantages in terms of different reconstruction problems.

MAGIC: Manifold and Graph Integrative Convolutional Network for Low-Dose CT Reconstruction

Low-dose computed tomography (LDCT) scans, which can effectively alleviate the radiation problem, will degrade the imaging quality. In this paper, we propose a novel LDCT reconstruction network that unrolls the iterative scheme and performs in both image and manifold spaces. Because patch manifolds of medical images have low-dimensional structures, we can build graphs from the manifolds. Then, we simultaneously leverage the spatial convolution to extract the local pixel-level features from the images and incorporate the graph convolution to analyze the nonlocal topological features in manifold space. The experiments show that our proposed method outperforms both the quantitative and qualitative aspects of state-of-the-art methods. In addition, aided by a projection loss component, our proposed method also demonstrates superior performance for semi-supervised learning. The network can remove most noise while maintaining the details of only 10% (40 slices) of the training data labeled.

CLEAR: Comprehensive Learning Enabled Adversarial Reconstruction for Subtle Structure Enhanced Low-Dose CT Imaging

X-ray computed tomography (CT) is of great clinical significance in medical practice because it can provide anatomical information about the human body without invasion, while its radiation risk has continued to attract public concerns. Reducing the radiation dose may induce noise and artifacts to the reconstructed images, which will interfere with the judgments of radiologists. Previous studies have confirmed that deep learning (DL) is promising for improving low-dose CT imaging. However, almost all the DL-based methods suffer from subtle structure degeneration and blurring effect after aggressive denoising, which has become the general challenging issue. This paper develops the Comprehensive Learning Enabled Adversarial Reconstruction (CLEAR) method to tackle the above problems. CLEAR achieves subtle structure enhanced low-dose CT imaging through a progressive improvement strategy. First, the generator established on the comprehensive domain can extract more features than the one built on degraded CT images and directly map raw projections to high-quality CT images, which is significantly different from the routine GAN practice. Second, a multi-level loss is assigned to the generator to push all the network components to be updated towards high-quality reconstruction, preserving the consistency between generated images and gold-standard images. Finally, following the WGAN-GP modality, CLEAR can migrate the real statistical properties to the generated images to alleviate over-smoothing. Qualitative and quantitative analyses have demonstrated the competitive performance of CLEAR in terms of noise suppression, structural fidelity and visual perception improvement.

CT Reconstruction With PDF: Parameter-Dependent Framework for Data From Multiple Geometries and Dose Levels

The current mainstream computed tomography (CT) reconstruction methods based on deep learning usually need to fix the scanning geometry and dose level, which significantly aggravates the training costs and requires more training data for real clinical applications. In this paper, we propose a parameter-dependent framework (PDF) that trains a reconstruction network with data originating from multiple alternative geometries and dose levels simultaneously. In the proposed PDF, the geometry and dose level are parameterized and fed into two multilayer perceptrons (MLPs). The outputs of the MLPs are used to modulate the feature maps of the CT reconstruction network, which condition the network outputs on different geometries and dose levels. The experiments show that our proposed method can obtain competitive performance compared to the original network trained with either specific or mixed geometry and dose level, which can efficiently save extra training costs for multiple geometries and dose levels.

Learning to Reconstruct CT Images From the VVBP-Tensor

Deep learning (DL) is bringing a big movement in the field of computed tomography (CT) imaging. In general, DL for CT imaging can be applied by processing the projection or the image data with trained deep neural networks (DNNs), unrolling the iterative reconstruction as a DNN for training, or training a well-designed DNN to directly reconstruct the image from the projection. In all of these applications, the whole or part of the DNNs work in the projection or image domain alone or in combination. In this study, instead of focusing on the projection or image, we train DNNs to reconstruct CT images from the view-by-view backprojection tensor (VVBP-Tensor). The VVBP-Tensor is the 3D data before summation in backprojection. It contains structures of the scanned object after applying a sorting operation. Unlike the image or projection that provides compressed information due to the integration/summation step in forward or back projection, the VVBP-Tensor provides lossless information for processing, allowing the trained DNNs to preserve fine details of the image. We develop a learning strategy by inputting slices of the VVBP-Tensor as feature maps and outputting the image. Such strategy can be viewed as a generalization of the summation step in conventional filtered backprojection reconstruction. Numerous experiments reveal that the proposed VVBP-Tensor domain learning framework obtains significant improvement over the image, projection, and hybrid projection-image domain learning frameworks. We hope the VVBP-Tensor domain learning framework could inspire algorithm development for DL-based CT imaging.

Data Extrapolation From Learned Prior Images for Truncation Correction in Computed Tomography

Data truncation is a common problem in computed tomography (CT). Truncation causes cupping artifacts inside the field-of-view (FOV) and anatomical structures missing outside the FOV. Deep learning has achieved impressive results in CT reconstruction from limited data. However, its robustness is still a concern for clinical applications. Although the image quality of learning-based compensation schemes may be inadequate for clinical diagnosis, they can provide prior information for more accurate extrapolation than conventional heuristic extrapolation methods. With extrapolated projection, a conventional image reconstruction algorithm can be applied to obtain a final reconstruction. In this work, a general plug-and-play (PnP) method for truncation correction is proposed based on this idea, where various deep learning methods and conventional reconstruction algorithms can be plugged in. Such a PnP method integrates data consistency for measured data and learned prior image information for truncated data. This shows to have better robustness and interpretability than deep learning only. To demonstrate the efficacy of the proposed PnP method, two state-of-the-art deep learning methods, FBPConvNet and Pix2pixGAN, are investigated for truncation correction in cone-beam CT in noise-free and noisy cases. Their robustness is evaluated by showing false negative and false positive lesion cases. With our proposed PnP method, false lesion structures are corrected for both deep learning methods. For FBPConvNet, the root-mean-square error (RMSE) inside the FOV can be improved from 92HU to around 30HU by PnP in the noisy case. Pix2pixGAN solely achieves better image quality than FBPConvNet solely for truncation correction in general. PnP further improves the RMSE inside the FOV from 42HU to around 27HU for Pix2pixGAN. The efficacy of PnP is also demonstrated on real clinical head data.

CycN-Net: A Convolutional Neural Network Specialized for 4D CBCT Images Refinement

Four-dimensional cone-beam computed tomography (4D CBCT) has been developed to provide a sequence of phase-resolved reconstructions in image-guided radiation therapy. However, 4D CBCT images are degraded by severe streaking artifacts and noise because the phase-resolved image is an extremely sparse-view CT procedure wherein a few under-sampled projections are used for the reconstruction of each phase. Aiming at improving the overall quality of 4D CBCT images, we proposed two CNN models, named N-Net and CycN-Net, respectively, by fully excavating the inherent property of 4D CBCT. To be specific, the proposed N-Net incorporates the prior image reconstructed from entire projection data based on U-Net to boost the image quality for each phase-resolved image. Based on N-Net, a temporal correlation among the phase-resolved images is also considered by the proposed CycN-Net. Extensive experiments on both XCAT simulation data and real patient 4D CBCT datasets were carried out to verify the feasibility of the proposed CNNs. Both networks can effectively suppress streaking artifacts and noise while restoring the distinct features simultaneously, compared with the existing CNN models and two state-of-the-art iterative algorithms. Moreover, the proposed method is robust in handling complicated tasks of various patient datasets and imaging devices, which implies its excellent generalization ability.

Deep Interactive Denoiser (DID) for X-Ray Computed Tomography

Low-dose computed tomography (LDCT) is desirable for both diagnostic imaging and image-guided interventions. Denoisers are widely used to improve the quality of LDCT. Deep learning (DL)-based denoisers have shown state-of-the-art performance and are becoming mainstream methods. However, there are two challenges to using DL-based denoisers: 1) a trained model typically does not generate different image candidates with different noise-resolution tradeoffs, which are sometimes needed for different clinical tasks; and 2) the model's generalizability might be an issue when the noise level in the testing images differs from that in the training dataset. To address these two challenges, in this work, we introduce a lightweight optimization process that can run on top of any existing DL-based denoiser during the testing phase to generate multiple image candidates with different noise-resolution tradeoffs suitable for different clinical tasks in real time. Consequently, our method allows users to interact with the denoiser to efficiently review various image candidates and quickly pick the desired one; thus, we termed this method deep interactive denoiser (DID). Experimental results demonstrated that DID can deliver multiple image candidates with different noise-resolution tradeoffs and shows great generalizability across various network architectures, as well as training and testing datasets with various noise levels.

Downsampled Imaging Geometric Modeling for Accurate CT Reconstruction via Deep Learning

X-ray computed tomography (CT) is widely used clinically to diagnose a variety of diseases by reconstructing the tomographic images of a living subject using penetrating X-rays. For accurate CT image reconstruction, a precise imaging geometric model for the radiation attenuation process is usually required to solve the inversion problem of CT scanning, which encodes the subject into a set of intermediate representations in different angular positions. Here, we show that accurate CT image reconstruction can be subsequently achieved by downsampled imaging geometric modeling via deep-learning techniques. Specifically, we first propose a downsampled imaging geometric modeling approach for the data acquisition process and then incorporate it into a hierarchical neural network, which simultaneously combines both geometric modeling knowledge of the CT imaging system and prior knowledge gained from a data-driven training process for accurate CT image reconstruction. The proposed neural network is denoted as DSigNet, i.e., downsampled-imaging-geometry-based network for CT image reconstruction. We demonstrate the feasibility of the proposed DSigNet for accurate CT image reconstruction with clinical patient data. In addition to improving the CT image quality, the proposed DSigNet might help reduce the computational complexity and accelerate the reconstruction speed for modern CT imaging systems.

Spatiotemporal structure-aware dictionary learning-based 4D CBCT reconstruction

PURPOSE

Four-dimensional cone-beam computed tomography (4D CBCT) is developed to reconstruct a sequence of phase-resolved images, which could assist in verifying the patient's position and offering information for cancer treatment planning. However, 4D CBCT images suffer from severe streaking artifacts and noise due to the extreme sparse-view CT reconstruction problem for each phase. As a result, it would cause inaccuracy of treatment estimation. The purpose of this paper was to develop a new 4D CBCT reconstruction method to generate a series of high spatiotemporal 4D CBCT images.

METHODS

Considering the advantage of (DL) on representing structural features and correlation between neighboring pixels effectively, we construct a novel DL-based method for the 4D CBCT reconstruction. In this study, both a motion-aware dictionary and a spatially structural 2D dictionary are trained for 4D CBCT by excavating the spatiotemporal correlation among ten phase-resolved images and the spatial information in each image, respectively. Specifically, two reconstruction models are produced in this study. The first one is the motion-aware dictionary learning-based 4D CBCT algorithm, called motion-aware DL based 4D CBCT (MaDL). The second one is the MaDL equipped with a prior knowledge constraint, called pMaDL. Qualitative and quantitative evaluations are performed using a 4D extended cardiac torso (XCAT) phantom, simulated patient data, and two sets of patient data sets. Several state-of-the-art 4D CBCT algorithms, such as the McKinnon-Bates (MKB) algorithm, prior image constrained compressed sensing (PICCS), and the high-quality initial image-guided 4D CBCT reconstruction method (HQI-4DCBCT) are applied for comparison to validate the performance of the proposed MaDL and prior constraint MaDL (pMaDL) pmadl reconstruction frameworks.

RESULTS

Experimental results validate that the proposed MaDL can output the reconstructions with few streaking artifacts but some structural information such as tumors and blood vessels, may still be missed. Meanwhile, the results of the proposed pMaDL demonstrate an improved spatiotemporal resolution of the reconstructed 4D CBCT images. In these improved 4D CBCT reconstructions, streaking artifacts are suppressed primarily and detailed structures are also restored. Regarding the XCAT phantom, quantitative evaluations indicate that an average of 58.70%, 45.25%, and 40.10% decrease in terms of root-mean-square error (RMSE) and an average of 2.10, 1.37, and 1.37 times in terms of structural similarity index (SSIM) are achieved by the proposed pMaDL method when compared with piccs, PICCS, MaDL(2D), and MaDL(2D), respectively. Moreover the proposed pMaDL achieves a comparable performance with HQI-4DCBCT algorithm in terms of RMSE and SSIM metrics. However, pMaDL has a better ability to suppress streaking artifacts than HQI-4DCBCT.

CONCLUSIONS

The proposed algorithm could reconstruct a set of 4D CBCT images with both high spatiotemporal resolution and detailed features preservation. Moreover the proposed pMaDL can effectively suppress the streaking artifacts in the resultant reconstructions, while achieving an overall improved spatiotemporal resolution by incorporating the motion-aware dictionary with a prior constraint into the proposed 4D CBCT iterative framework.

A neural network with encoded visible edge prior for limited-angle computed tomography reconstruction

PURPOSE

Limited-angle computed tomography is a challenging but important task in certain medical and industrial applications for nondestructive testing. The limited-angle reconstruction problem is highly ill-posed and conventional reconstruction algorithms would introduce heavy artifacts. Various models and methods have been proposed to improve the quality of reconstructions by introducing different priors regarding to the projection data or ideal images. However, the assumed priors might not be practically applicable to all limited-angle reconstruction problems. Convolutional neural network (CNN) exhibits great promise in the modeling of data coupling and has recently become an important technique in medical imaging applications. Although existing CNN methods have demonstrated promising results, their robustness is still a concern. In this paper, in light of the theory of visible and invisible boundaries, we propose an alternating edge-preserving diffusion and smoothing neural network (AEDSNN) for limited-angle reconstruction that builds the visible boundaries as priors into its structure. The proposed method generalizes the alternating edge-preserving diffusion and smoothing (AEDS) method for limited-angle reconstruction developed in the literature by replacing its regularization terms by CNNs, by which the piecewise constant assumption assumed by AEDS is effectively relaxed.

METHODS

The AEDSNN is derived by unrolling the AEDS algorithm. AEDSNN consists of several blocks, and each block corresponds to one iteration of the AEDS algorithm. In each iteration of the AEDS algorithm, three subproblems are sequentially solved. So, each block of AEDSNN possesses three main layers: data matching layer, x -direction regularization layer for visible edges diffusion, and y -direction regularization layer for artifacts suppressing. The data matching layer is implemented by conventional ordered-subset simultaneous algebraic reconstruction technique (OS-SART) reconstruction algorithm, while the two regularization layers are modeled by CNNs for more intelligent and better encoding of priors regarding to the reconstructed images. To further strength the visible edge prior, the attention mechanism and the pooling layers are incorporated into AEDSNN to facilitate the procedure of edge-preserving diffusion from visible edges.

RESULTS

We have evaluated the performance of AEDSNN by comparing it with popular algorithms for limited-angle reconstruction. Experiments on the medical dataset show that the proposed AEDSNN effectively breaks through the piecewise constant assumption usually assumed by conventional reconstruction algorithms, and works much better for piecewise smooth images with nonsharp edges. Experiments on the printed circuit board (PCB) dataset show that AEDSNN can better encode and utilize the visible edge prior, and its reconstructions are consistently better compared to the competing algorithms.

CONCLUSIONS

A deep-learning approach for limited-angle reconstruction is proposed in this paper, which significantly outperforms existing methods. The superiority of AEDSNN consists of three aspects. First, by the virtue of CNN, AEDSNN is free of parameter-tuning. This is a great facility compared to conventional reconstruction methods; Second, AEDSNN is quite fast. Conventional reconstruction methods usually need hundreds even thousands of iterations, while AEDSNN just needs three to five iterations (i.e., blocks); Third, the learned regularizer by AEDSNN enjoys a broader application capacity, which could work well with piecewise smooth images and surpass the piecewise constant assumption frequently assumed for computed tomography images.

Accurate and robust sparse-view angle CT image reconstruction using deep learning and prior image constrained compressed sensing (DL-PICCS)

BACKGROUND

Sparse-view CT image reconstruction problems encountered in dynamic CT acquisitions are technically challenging. Recently, many deep learning strategies have been proposed to reconstruct CT images from sparse-view angle acquisitions showing promising results. However, two fundamental problems with these deep learning reconstruction methods remain to be addressed: (1) limited reconstruction accuracy for individual patients and (2) limited generalizability for patient statistical cohorts.

PURPOSE

The purpose of this work is to address the previously mentioned challenges in current deep learning methods.

METHODS

A method that combines a deep learning strategy with prior image constrained compressed sensing (PICCS) was developed to address these two problems. In this method, the sparse-view CT data were reconstructed by the conventional filtered backprojection (FBP) method first, and then processed by the trained deep neural network to eliminate streaking artifacts. The outputs of the deep learning architecture were then used as the needed prior image in PICCS to reconstruct the image. If the noise level from the PICCS reconstruction is not satisfactory, another light duty deep neural network can then be used to reduce noise level. Both extensive numerical simulation data and human subject data have been used to quantitatively and qualitatively assess the performance of the proposed DL-PICCS method in terms of reconstruction accuracy and generalizability.

RESULTS

Extensive evaluation studies have demonstrated that: (1) quantitative reconstruction accuracy of DL-PICCS for individual patient is improved when it is compared with the deep learning methods and CS-based methods; (2) the false-positive lesion-like structures and false negative missing anatomical structures in the deep learning approaches can be effectively eliminated in the DL-PICCS reconstructed images; and (3) DL-PICCS enables a deep learning scheme to relax its working conditions to enhance its generalizability.

CONCLUSIONS

DL-PICCS offers a promising opportunity to achieve personalized reconstruction with improved reconstruction accuracy and enhanced generalizability.

Deep learning enabled ultra-fast-pitch acquisition in clinical X-ray computed tomography

OBJECTIVE

In X-raycomputed tomography (CT), many important clinical applications may benefit from a fast acquisition speed. The helical scan is the most widely used acquisition mode in clinical CT, where a fast helical pitch can improve the acquisition speed. However, on a typical single-source helical CT (SSCT) system, the helical pitch p typically cannot exceed 1.5; otherwise, reconstruction artifacts will result from data insufficiency. The purpose of this work is to develop a deep convolutional neural network (CNN) to correct for artifacts caused by an ultra-fast pitch, which can enable faster acquisition speed than what is currently achievable.

METHODS

A customized CNN (denoted as ultra-fast-pitch network (UFP-net)) was developed to restore the underlying anatomical structure from the artifact-corrupted post-reconstruction data acquired from SSCT with ultra-fast pitch (i.e., p ≥ 2). UFP-net employed residual learning to capture the features of image artifacts. UFP-net further deployed in-house-customized functional blocks with spatial-domain local operators and frequency-domain non-local operators, to explore multi-scale feature representation. Images of contrast-enhanced patient exams (n = 83) with routine pitch setting (i.e., p < 1) were retrospectively collected, which were used as training and testing datasets. This patient cohort involved CT exams over different scan ranges of anatomy (chest, abdomen, and pelvis) and CT systems (Siemens Definition, Definition Flash, Definition AS+, Siemens Healthcare, Inc.), and the corresponding base CT scanning protocols used consistent settings of major scan parameters (e.g., collimation and pitch). Forward projection of the original images was calculated to synthesize helical CT scans with one regular pitch setting (p = 1) and two ultra-fast-pitch setting (p = 2 and 3). All patient images were reconstructed using the standard filtered-back-projection (FBP) algorithm. A customized multi-stage training scheme was developed to incrementally optimize the parameters of UFP-net, using ultra-fast-pitch images as network inputs and regular pitch images as labels. Visual inspection was conducted to evaluate image quality. Structural similarity index (SSIM) and relative root-mean-square error (rRMSE) were used as quantitative quality metrics.

RESULTS

The UFP-net dramatically improved image quality over standard FBP at both ultra-fast-pitch settings. At p = 2, UFP-net yielded higher mean SSIM (> 0.98) with lower mean rRMSE (< 2.9%), compared to FBP (mean SSIM < 0.93; mean rRMSE > 9.1%). Quantitative metrics at p = 3: UFP-net-mean SSIM [0.86, 0.94] and mean rRMSE [5.0%, 8.2%]; FBP-mean SSIM [0.36, 0.61] and mean rRMSE [36.0%, 58.6%].

CONCLUSION

The proposed UFP-net has the potential to enable ultra-fast data acquisition in clinical CT without sacrificing image quality. This method has demonstrated reasonable generalizability over different body parts when the corresponding CT exams involved consistent base scan parameters.

Deep learning based synthetic-CT generation in radiotherapy and PET: A review

Recently,deep learning (DL)-based methods for the generation of synthetic computed tomography (sCT) have received significant research attention as an alternative to classical ones. We present here a systematic review of these methods by grouping them into three categories, according to their clinical applications: (i) to replace computed tomography in magnetic resonance (MR) based treatment planning, (ii) facilitate cone-beam computed tomography based image-guided adaptive radiotherapy, and (iii) derive attenuation maps for the correction of positron emission tomography. Appropriate database searching was performed on journal articles published between January 2014 and December 2020. The DL methods' key characteristics were extracted from each eligible study, and a comprehensive comparison among network architectures and metrics was reported. A detailed review of each category was given, highlighting essential contributions, identifying specific challenges, and summarizing the achievements. Lastly, the statistics of all the cited works from various aspects were analyzed, revealing the popularity and future trends and the potential of DL-based sCT generation. The current status of DL-based sCT generation was evaluated, assessing the clinical readiness of the presented methods.

A geometry-guided deep learning technique for CBCT reconstruction

Purpose.Although deep learning (DL) technique has been successfully used for computed tomography (CT) reconstruction, its implementation on cone-beam CT (CBCT) reconstruction is extremely challenging due to memory limitations. In this study, a novel DL technique is developed to resolve the memory issue, and its feasibility is demonstrated for CBCT reconstruction from sparsely sampled projection data.Methods.The novel geometry-guided deep learning (GDL) technique is composed of a GDL reconstruction module and a post-processing module. The GDL reconstruction module learns and performs projection-to-image domain transformation by replacing the traditional single fully connected layer with an array of small fully connected layers in the network architecture based on the projection geometry. The DL post-processing module further improves image quality after reconstruction. We demonstrated the feasibility and advantage of the model by comparing ground truth CBCT with CBCT images reconstructed using (1) GDL reconstruction module only, (2) GDL reconstruction module with DL post-processing module, (3) Feldkamp, Davis, and Kress (FDK) only, (4) FDK with DL post-processing module, (5) ray-tracing only, and (6) ray-tracing with DL post-processing module. The differences are quantified by peak-signal-to-noise ratio (PSNR), structural similarity index measure (SSIM), and root-mean-square error (RMSE).Results.CBCT images reconstructed with GDL show improvements in quantitative scores of PSNR, SSIM, and RMSE. Reconstruction time per image for all reconstruction methods are comparable. Compared to current DL methods using large fully connected layers, the estimated memory requirement using GDL is four orders of magnitude less, making DL CBCT reconstruction feasible.Conclusion.With much lower memory requirement compared to other existing networks, the GDL technique is demonstrated to be the first DL technique that can rapidly and accurately reconstruct CBCT images from sparsely sampled data.

Federated Compressed Learning Edge Computing Framework with Ensuring Data Privacy for PM2.5 Prediction in Smart City Sensing Applications

The sparse data in PM2.5 air quality monitoring systems is frequently happened on large-scale smart city sensing applications, which is collected via massive sensors. Moreover, it could be affected by inefficient node deployment, insufficient communication, and fragmented records, which is the main challenge of the high-resolution prediction system. In addition, data privacy in the existing centralized air quality prediction system cannot be ensured because the data which are mined from end sensory nodes constantly exposed to the network. Therefore, this paper proposes a novel edge computing framework, named Federated Compressed Learning (FCL), which provides efficient data generation while ensuring data privacy for PM2.5 predictions in the application of smart city sensing. The proposed scheme inherits the basic ideas of the compression technique, regional joint learning, and considers a secure data exchange. Thus, it could reduce the data quantity while preserving data privacy. This study would like to develop a green energy-based wireless sensing network system by using FCL edge computing framework. It is also one of key technologies of software and hardware co-design for reconfigurable and customized sensing devices application. Consequently, the prototypes are developed in order to validate the performances of the proposed framework. The results show that the data consumption is reduced by more than 95% with an error rate below 5%. Finally, the prediction results based on the FCL will generate slightly lower accuracy compared with centralized training. However, the data could be heavily compacted and securely transmitted in WSNs.

Quantitative Comparison of Deep Learning-Based Image Reconstruction Methods for Low-Dose and Sparse-Angle CT Applications

The reconstruction of computed tomography (CT) images is an active area of research. Following the rise of deep learning methods, many data-driven models have been proposed in recent years. In this work, we present the results of a data challenge that we organized, bringing together algorithm experts from different institutes to jointly work on quantitative evaluation of several data-driven methods on two large, public datasets during a ten day sprint. We focus on two applications of CT, namely, low-dose CT and sparse-angle CT. This enables us to fairly compare different methods using standardized settings. As a general result, we observe that the deep learning-based methods are able to improve the reconstruction quality metrics in both CT applications while the top performing methods show only minor differences in terms of peak signal-to-noise ratio (PSNR) and structural similarity (SSIM). We further discuss a number of other important criteria that should be taken into account when selecting a method, such as the availability of training data, the knowledge of the physical measurement model and the reconstruction speed.

Dual residual convolutional neural network (DRCNN) for low-dose CT imaging

The excessive radiation doses in the application of computed tomography (CT) technology pose a threat to the health of patients. However, applying a low radiation dose in CT can result in severe artifacts and noise in the captured images, thus affecting the diagnosis. Therefore, in this study, we investigate a dual residual convolution neural network (DRCNN) for low-dose CT (LDCT) imaging, whereby the CT images are reconstructed directly from the sinogram by integrating analytical domain transformations, thus reducing the loss of projection information. With this new framework, feature extraction is performed simultaneously on both the sinogram-domain sub-net and the image-domain sub-net, which utilize the residual shortcut networks and play a complementary role in suppressing the projection noise and reducing image error. This new DRCNN approach helps not only decrease the sinogram noise but also preserve significant structural information. The experimental results of simulated and real projection data demonstrate that our DRCNN achieve superior performance over other state-of-art methods in terms of visual inspection and quantitative metrics. For example, comparing with RED-CNN and DP-ResNet, the value of PSNR using our DRCNN is improved by nearly 3 dB and 1 dB, respectively.

A dual-domain deep learning-based reconstruction method for fully 3D sparse data helical CT

Helical CT has been widely used in clinical diagnosis. In this work, we focus on a new prototype of helical CT, equipped with sparsely spaced multidetector and multi-slit collimator (MSC) in the axis direction. This type of system can not only lower radiation dose, and suppress scattering by MSC, but also cuts down the manufacturing cost of the detector. The major problem to overcome with such a system, however, is that of insufficient data for reconstruction. Hence, we propose a deep learning-based function optimization method for this ill-posed inverse problem. By incorporating a Radon inverse operator, and disentangling each slice, we significantly simplify the complexity of our network for 3D reconstruction. The network is composed of three subnetworks. Firstly, a convolutional neural network (CNN) in the projection domain is constructed to estimate missing projection data, and to convert helical projection data to 2D fan-beam projection data. This is follwed by the deployment of an analytical linear operator to transfer the data from the projection domain to the image domain. Finally, an additional CNN in the image domain is added for further image refinement. These three steps work collectively, and can be trained end to end. The overall network is trained on a simulated CT dataset based on eight patients from the American Association of Physicists in Medicine (AAPM) Low Dose CT Grand Challenge. We evaluate the trained network on both simulated datasets and clinical datasets. Extensive experimental studies have yielded very encouraging results, based on both visual examination and quantitative evaluation. These results demonstrate the effectiveness of our method and its potential for clinical usage. The proposed method provides us with a new solution for a fully 3D ill-posed problem.

Fourier Properties of Symmetric-Geometry Computed Tomography and Its Linogram Reconstruction With Neural Network

In this work, we investigate the Fourier properties of a symmetric-geometry computed tomography (SGCT) with linearly distributed source and detector in a stationary configuration. A linkage between the 1D Fourier Transform of a weighted projection from SGCT and the 2D Fourier Transform of a deformed object is established in a simple mathematical form (i.e., the Fourier slice theorem for SGCT). Based on its Fourier slice theorem and its unique data sampling in the Fourier space, a Linogram-based Fourier reconstruction method is derived for SGCT. We demonstrate that the entire Linogram reconstruction process can be embedded as known operators into an end-to-end neural network. As a learning-based approach, the proposed Linogram-Net has capability of improving CT image quality for non-ideal imaging scenarios, a limited-angle SGCT for instance, through combining weights learning in the projection domain and loss minimization in the image domain. Numerical simulations and physical experiments on an SGCT prototype platform showed that our proposed Linogram-based method can achieve accurate reconstruction from a dual-SGCT scan and can greatly reduce computational complexity when compared with the filtered backprojection type reconstruction. The Linogram-Net achieved accurate reconstruction when projection data are complete and significantly suppressed image artifacts from a limited-angle SGCT scan mimicked by using a clinical CT dataset, with the average CT number error in the selected regions of interest reduced from 67.7 Hounsfield Units (HU) to 28.7 HU, and the average normalized mean square error of overall images reduced from 4.21e-3 to 2.65e-3.

Ultrasound transmission tomography image reconstruction with a fully convolutional neural network

Image reconstruction of ultrasound computed tomography based on the wave equation is able to show much more structural details than simpler ray-based image reconstruction methods. However, to invert the wave-based forward model is computationally demanding. To address this problem, we develop an efficient fully learned image reconstruction method based on a convolutional neural network. The image is reconstructed via one forward propagation of the network given input sensor data, which is much faster than the reconstruction using conventional iterative optimization methods. To transform the ultrasound measured data in the sensor domain into the reconstructed image in the image domain, we apply multiple down-scaling and up-scaling convolutional units to efficiently increase the number of hidden layers with a large receptive and projective field that can cover all elements in inputs and outputs, respectively. For dataset generation, a paraxial approximation forward model is used to simulate ultrasound measurement data. The neural network is trained with a dataset derived from natural images in ImageNet and tested with a dataset derived from medical images in OA-Breast Phantom dataset. Test results show the superior efficiency of the proposed neural network to other reconstruction algorithms including popular neural networks. When compared with conventional iterative optimization algorithms, our neural network can reconstruct a 110 × 86 image more than 20 times faster on a CPU and 1000 times faster on a GPU with comparable image quality and is also more robust to noise.

FBP-Net for direct reconstruction of dynamic PET images

Dynamic positron emission tomography (PET) imaging can provide information about metabolic changes over time, used for kinetic analysis and auxiliary diagnosis. Existing deep learning-based reconstruction methods have too many trainable parameters and poor generalization, and require mass data to train the neural network. However, obtaining large amounts of medical data is expensive and time-consuming. To reduce the need for data and improve the generalization of network, we combined the filtered back-projection (FBP) algorithm with neural network, and proposed FBP-Net which could directly reconstruct PET images from sinograms instead of post-processing the rough reconstruction images obtained by traditional methods. The FBP-Net contained two parts: the FBP part and the denoiser part. The FBP part adaptively learned the frequency filter to realize the transformation from the detector domain to the image domain, and normalized the coarse reconstruction images obtained. The denoiser part merged the information of all time frames to improve the quality of dynamic PET reconstruction images, especially the early time frames. The proposed FBP-Net was performed on simulation and real dataset, and the results were compared with the state-of-art U-net and DeepPET. The results showed that FBP-Net did not tend to overfit the training set and had a stronger generalization.

Full-Spectrum-Knowledge-Aware Tensor Model for Energy-Resolved CT Iterative Reconstruction

Energy-resolved computed tomography (ErCT) with a photon counting detector concurrently produces multiple CT images corresponding to different photon energy ranges. It has the potential to generate energy-dependent images with improved contrast-to-noise ratio and sufficient material-specific information. Since the number of detected photons in one energy bin in ErCT is smaller than that in conventional energy-integrating CT (EiCT), ErCT images are inherently more noisy than EiCT images, which leads to increased noise and bias in the subsequent material estimation. In this work, we first deeply analyze the intrinsic tensor properties of two-dimensional (2D) ErCT images acquired in different energy bins and then present a F ull- S pectrum-knowledge-aware Tensor analysis and processing (FSTensor) method for ErCT reconstruction to suppress noise-induced artifacts to obtain high-quality ErCT images and high-accuracy material images. The presented method is based on three considerations: (1) 2D ErCT images obtained in different energy bins can be treated as a 3-order tensor with three modes, i.e., width, height and energy bin, and a rich global correlation exists among the three modes, which can be characterized by tensor decomposition. (2) There is a locally piecewise smooth property in the 3-order ErCT images, and it can be captured by a tensor total variation regularization. (3) The images from the full spectrum are much better than the ErCT images with respect to noise variance and structural details and serve as external information to improve the reconstruction performance. We then develop an alternating direction method of multipliers algorithm to numerically solve the presented FSTensor method. We further utilize a genetic algorithm to tackle the parameter selection in ErCT reconstruction, instead of manually determining parameters. Simulation, preclinical and synthesized clinical ErCT results demonstrate that the presented FSTensor method leads to significant improvements over the filtered back-projection, robust principal component analysis, tensor-based dictionary learning and low-rank tensor decomposition with spatial-temporal total variation methods.

Artifact removal using a hybrid-domain convolutional neural network for limited-angle computed tomography imaging

The suppression of streak artifacts in computed tomography with a limited-angle configuration is challenging. Conventional analytical algorithms, such as filtered backprojection (FBP), are not successful due to incomplete projection data. Moreover, model-based iterative total variation algorithms effectively reduce small streaks but do not work well at eliminating large streaks. In contrast, FBP mapping networks and deep-learning-based postprocessing networks are outstanding at removing large streak artifacts; however, these methods perform processing in separate domains, and the advantages of multiple deep learning algorithms operating in different domains have not been simultaneously explored. In this paper, we present a hybrid-domain convolutional neural network (hdNet) for the reduction of streak artifacts in limited-angle computed tomography. The network consists of three components: the first component is a convolutional neural network operating in the sinogram domain, the second is a domain transformation operation, and the last is a convolutional neural network operating in the CT image domain. After training the network, we can obtain artifact-suppressed CT images directly from the sinogram domain. Verification results based on numerical, experimental and clinical data confirm that the proposed method can significantly reduce serious artifacts.

Radon Inversion via Deep Learning

The Radon transform is widely used in physical and life sciences, and one of its major applications is in medical X-ray computed tomography (CT), which is significantly important in disease screening and diagnosis. In this paper, we propose a novel reconstruction framework for Radon inversion with deep learning (DL) techniques. For simplicity, the proposed framework is denoted as iRadonMAP, i.e., inverse Radon transform approximation. Specifically, we construct an interpretable neural network that contains three dedicated components. The first component is a fully connected filtering (FCF) layer along the rotation angle direction in the sinogram domain, and the second one is a sinusoidal back-projection (SBP) layer, which back-projects the filtered sinogram data into the spatial domain. Next, a common network structure is added to further improve the overall performance. iRadonMAP is first pretrained on a large number of generic images from the ImageNet database and then fine-tuned with clinical patient data. The experimental results demonstrate the feasibility of the proposed iRadonMAP framework for Radon inversion.

Spectral CT imaging method based on blind separation of polychromatic projections with Poisson prior

The linear reconstruction of narrow-energy-width projections can suppress hardening artifacts in conventional computed tomography (CT). We develop a spectral CT blind separation algorithm for obtaining narrow-energy-width projections under a blind scenario where the incident spectra are unknown. The algorithm relies on an X-ray multispectral forward model. Based on the Poisson statistical properties of measurements, a constrained optimization problem is established and solved by a block coordinate descent algorithm that alternates between nonnegative matrix factorization and Gauss-Newton algorithm. Experiments indicate that the decomposed projections conform to the characteristics of narrow-energy-width projections. The new algorithm improves the accuracy of obtaining narrow-energy-width projections.

ADAPTIVE-NET: deep computed tomography reconstruction network with analytical domain transformation knowledge

Background

Recently, the paradigm of computed tomography (CT) reconstruction has shifted as the deep learning technique evolves. In this study, we proposed a new convolutional neural network (called ADAPTIVE-NET) to perform CT image reconstruction directly from a sinogram by integrating the analytical domain transformation knowledge.

Methods

In the proposed ADAPTIVE-NET, a specific network layer with constant weights was customized to transform the sinogram onto the CT image domain via analytical back-projection. With this new framework, feature extractions were performed simultaneously on both the sinogram domain and the CT image domain. The Mayo low dose CT (LDCT) data was used to validate the new network. In particular, the new network was compared with the previously proposed residual encoder-decoder (RED)-CNN network. For each network, the mean square error (MSE) loss with and without VGG-based perceptual loss was compared. Furthermore, to evaluate the image quality with certain metrics, the noise correlation was quantified via the noise power spectrum (NPS) on the reconstructed LDCT for each method.

Results

CT images that have clinically relevant dimensions of 512×512 can be easily reconstructed from a sinogram on a single graphics processing unit (GPU) with moderate memory size (e.g., 11 GB) by ADAPTIVE-NET. With the same MSE loss function, the new network is able to generate better results than the RED-CNN. Moreover, the new network is able to reconstruct natural looking CT images with enhanced image quality if jointly using the VGG loss.

Conclusions

The newly proposed end-to-end supervised ADAPTIVE-NET is able to reconstruct high-quality LDCT images directly from a sinogram.

Deep Efficient End-to-end Reconstruction (DEER) Network for Few-view Breast CT Image Reconstruction

Breast CT provides image volumes with isotropic resolution in high contrast, enabling detection of small calcification (down to a few hundred microns in size) and subtle density differences. Since breast is sensitive to x-ray radiation, dose reduction of breast CT is an important topic, and for this purpose, few-view scanning is a main approach. In this article, we propose a Deep Efficient End-to-end Reconstruction (DEER) network for few-view breast CT image reconstruction. The major merits of our network include high dose efficiency, excellent image quality, and low model complexity. By the design, the proposed network can learn the reconstruction process with as few as O ( N ) parameters, where N is the side length of an image to be reconstructed, which represents orders of magnitude improvements relative to the state-of-the-art deep-learning-based reconstruction methods that map raw data to tomographic images directly. Also, validated on a cone-beam breast CT dataset prepared by Koning Corporation on a commercial scanner, our method demonstrates a competitive performance over the state-of-the-art reconstruction networks in terms of image quality. The source code of this paper is available at: https://github.com/HuidongXie/DEER.

Deep Encoder-Decoder Adversarial Reconstruction(DEAR) Network for 3D CT from Few-View Data

X-ray computed tomography (CT) is widely used in clinical practice. The involved ionizingX-ray radiation, however, could increase cancer risk. Hence, the reduction of the radiation dosehas been an important topic in recent years. Few-view CT image reconstruction is one of the mainways to minimize radiation dose and potentially allow a stationary CT architecture. In this paper,we propose a deep encoder-decoder adversarial reconstruction (DEAR) network for 3D CT imagereconstruction from few-view data. Since the artifacts caused by few-view reconstruction appear in3D instead of 2D geometry, a 3D deep network has a great potential for improving the image qualityin a data driven fashion. More specifically, our proposed DEAR-3D network aims at reconstructing3D volume directly from clinical 3D spiral cone-beam image data. DEAR is validated on a publiclyavailable abdominal CT dataset prepared and authorized by Mayo Clinic. Compared with other2D deep learning methods, the proposed DEAR-3D network can utilize 3D information to producepromising reconstruction results.