Sam's Net: A Self-Augmented Multistage Deep-Learning Network for End-to-End Reconstruction of Limited Angle CT

Multi-stage deep learning artifact reduction for parallel-beam computed tomography

Computed tomography (CT) using synchrotron radiation is a powerful technique that, compared with laboratory CT techniques, boosts high spatial and temporal resolution while also providing access to a range of contrast-formation mechanisms. The acquired projection data are typically processed by a computational pipeline composed of multiple stages. Artifacts introduced during data acquisition can propagate through the pipeline and degrade image quality in the reconstructed images. Recently, deep learning has shown significant promise in enhancing image quality for images representing scientific data. This success has driven increasing adoption of deep learning techniques in CT imaging. Various approaches have been proposed to incorporate deep learning into computational pipelines, but each has limitations in addressing artifacts effectively and efficiently in synchrotron CT, either in properly addressing the specific artifacts or in computational efficiency. Recognizing these challenges, we introduce a novel method that incorporates separate deep learning models at each stage of the tomography pipeline - projection, sinogram and reconstruction - to address specific artifacts locally in a data-driven way. Our approach includes bypass connections that feed both the outputs from previous stages and raw data to subsequent stages, minimizing the risk of error propagation. Extensive evaluations on both simulated and real-world datasets illustrate that our approach effectively reduces artifacts and outperforms comparison methods.

Sinogram-characteristic-informed network for efficient restoration of low-dose SPECT projection data

BACKGROUND

Single Photon Emission Computed Tomography (SPECT) sinogram restoration for low-dose imaging is a critical challenge in medical imaging. Existing methods often overlook the characteristics of the sinograms, necessitating innovative approaches.

PURPOSE

In this study, we introduce the Sinogram-characteristic-informed network (SCI-Net) to address the restoration of low-dose SPECT sinograms. Our aim is to build and train a model based on the characteristics of sinograms, including continuity, periodicity, multi-scale properties of lines in sinograms, and others, to enhance the model's understanding of the restoration process.

METHODS

SCI-Net incorporates several novel mechanisms tailored to exploit the inherent characteristics of sinograms. We implement a channel attention module with a decay mechanism to leverage continuity across adjacent sinograms, while a position attention module captures global correlations within individual sinograms. Additionally, we propose a multi-stage progressive integration mechanism to balance local detail and overall structure. Multiple regularization terms, customized to sinogram image characteristics, are embedded into the loss function for model training.

RESULTS

The experimental evaluations are divided into two parts: simulation data evaluation and clinical evaluation. The simulation data evaluation is conducted on a dataset comprising ten organ types, generated by the SIMIND Monte Carlo program from extended cardiac-torso (XCAT) anatomical phantoms. The dataset includes a total of SPECT sinograms with low-dose as input data and normal-dose as ground truth, consisting of 3881 sinograms in the training dataset and 849 sinograms in the testing set. When comparing the restoration of low-dose sinograms to normal-dose references, SCI-Net effectively improves performance. Specifically, the peak signal-to-noise ratio (PSNR) and the structural similarity (SSIM) on sinograms increase from 15.72 to 34.66 (p < $\text{p}< $ 0.001) and 0.6297 to 0.9834 (p < $\text{p}<$ 0.001), respectively, and on reconstructed images, reconstructed by maximum likelihood-expectation maximization (ML-EM), the PSNR and the SSIM improve from 21.95 to 33.14 (p < $\text{p}<$ 0.001) and 0.9084 to 0.9866 (p < $\text{p}<$ 0.001), respectively. We compared SCI-Net with existing methods, including baseline models, traditional reconstruction algorithms, end-to-end methods, sinogram restoration methods, and image post-processing methods. The experimental results and visual examples demonstrate that SCI-Net surpasses these existing methods in SPECT sinogram restoration. The clinical evaluation is conducted on clinical data of low-dose SPECT sinograms for spleen, thyroid, skull, and bone. These SPECT projection data are obtained from Discovery NM/CT670 scans. We compare the reconstructed images from the SCI-Net restored sinograms, the reconstructed images from the original low-dose sinograms, and the reconstructed images using the built-in algorithm of the Discovery NM/CT670. The results show that our method effectively reduces the coefficient of variation (COV) in the regions of interest (ROI) of the reconstructed images, thereby enhancing the quality of the reconstructed images through SPECT sinogram restoration.

CONCLUSIONS

Our proposed SCI-Net exhibits promising performance in the restoration of low-dose SPECT projection data. In the SCI-Net, we have implemented three mechanisms based on distinct forms, which are advantageous for the model to more effectively leverage the characteristics of sinograms and achieve commendable restoration outcomes.

Wavelet-Inspired Multi-Channel Score-Based Model for Limited-Angle CT Reconstruction

Score-based generative model (SGM) has demonstrated great potential in the challenging limited-angle CT (LA-CT) reconstruction. SGM essentially models the probability density of the ground truth data and generates reconstruction results by sampling from it. Nevertheless, direct application of the existing SGM methods to LA-CT suffers multiple limitations. Firstly, the directional distribution of the artifacts attributing to the missing angles is ignored. Secondly, the different distribution properties of the artifacts in different frequency components have not been fully explored. These drawbacks would inevitably degrade the estimation of the probability density and the reconstruction results. After an in-depth analysis of these factors, this paper proposes a Wavelet-Inspired Score-based Model (WISM) for LA-CT reconstruction. Specifically, besides training a typical SGM with the original images, the proposed method additionally performs the wavelet transform and models the probability density in each wavelet component with an extra SGM. The wavelet components preserve the spatial correspondence with the original image while performing frequency decomposition, thereby keeping the directional property of the artifacts for further analysis. On the other hand, different wavelet components possess more specific contents of the original image in different frequency ranges, simplifying the probability density modeling by decomposing the overall density into component-wise ones. The resulting two SGMs in the image-domain and wavelet-domain are integrated into a unified sampling process under the guidance of the observation data, jointly generating high-quality and consistent LA-CT reconstructions. The experimental evaluation on various datasets consistently verifies the superior performance of the proposed method over the competing method.

Time-Reversion Fast-Sampling Score-Based Model for Limited-Angle CT Reconstruction

The score-based generative model (SGM) has received significant attention in the field of medical imaging, particularly in the context of limited-angle computed tomography (LACT). Traditional SGM approaches achieved robust reconstruction performance by incorporating a substantial number of sampling steps during the inference phase. However, these established SGM-based methods require large computational cost to reconstruct one case. The main challenge lies in achieving high-quality images with rapid sampling while preserving sharp edges and small features. In this study, we propose an innovative rapid-sampling strategy for SGM, which we have aptly named the time-reversion fast-sampling (TIFA) score-based model for LACT reconstruction. The entire sampling procedure adheres steadfastly to the principles of robust optimization theory and is firmly grounded in a comprehensive mathematical model. TIFA's rapid-sampling mechanism comprises several essential components, including jump sampling, time-reversion with re-sampling, and compressed sampling. In the initial jump sampling stage, multiple sampling steps are bypassed to expedite the attainment of preliminary results. Subsequently, during the time-reversion process, the initial results undergo controlled corruption by introducing small-scale noise. The re-sampling process then diligently refines the initially corrupted results. Finally, compressed sampling fine-tunes the refinement outcomes by imposing regularization term. Quantitative and qualitative assessments conducted on numerical simulations, real physical phantom, and clinical cardiac datasets, unequivocally demonstrate that TIFA method (using 200 steps) outperforms other state-of-the-art methods (using 2000 steps) from available [0°, 90°] and [0°, 60°]. Furthermore, experimental results underscore that our TIFA method continues to reconstruct high-quality images even with 10 steps. Our code at https://github.com/tianzhijiaoziA/TIFADiffusion.

Prior frequency guided diffusion model for limited angle (LA)-CBCT reconstruction

Objective.Cone-beam computed tomography (CBCT) is widely used in image-guided radiotherapy. Reconstructing CBCTs from limited-angle acquisitions (LA-CBCT) is highly desired for improved imaging efficiency, dose reduction, and better mechanical clearance. LA-CBCT reconstruction, however, suffers from severe under-sampling artifacts, making it a highly ill-posed inverse problem. Diffusion models can generate data/images by reversing a data-noising process through learned data distributions; and can be incorporated as a denoiser/regularizer in LA-CBCT reconstruction. In this study, we developed a diffusion model-based framework, prior frequency-guided diffusion model (PFGDM), for robust and structure-preserving LA-CBCT reconstruction.Approach.PFGDM uses a conditioned diffusion model as a regularizer for LA-CBCT reconstruction, and the condition is based on high-frequency information extracted from patient-specific prior CT scans which provides a strong anatomical prior for LA-CBCT reconstruction. Specifically, we developed two variants of PFGDM (PFGDM-A and PFGDM-B) with different conditioning schemes. PFGDM-A applies the high-frequency CT information condition until a pre-optimized iteration step, and drops it afterwards to enable both similar and differing CT/CBCT anatomies to be reconstructed. PFGDM-B, on the other hand, continuously applies the prior CT information condition in every reconstruction step, while with a decaying mechanism, to gradually phase out the reconstruction guidance from the prior CT scans. The two variants of PFGDM were tested and compared with current available LA-CBCT reconstruction solutions, via metrics including peak signal-to-noise ratio (PSNR) and structural similarity index measure (SSIM).Main results.PFGDM outperformed all traditional and diffusion model-based methods. The mean(s.d.) PSNR/SSIM were 27.97(3.10)/0.949(0.027), 26.63(2.79)/0.937(0.029), and 23.81(2.25)/0.896(0.036) for PFGDM-A, and 28.20(1.28)/0.954(0.011), 26.68(1.04)/0.941(0.014), and 23.72(1.19)/0.894(0.034) for PFGDM-B, based on 120°, 90°, and 30° orthogonal-view scan angles respectively. In contrast, the PSNR/SSIM was 19.61(2.47)/0.807(0.048) for 30° for DiffusionMBIR, a diffusion-based method without prior CT conditioning.Significance. PFGDM reconstructs high-quality LA-CBCTs under very-limited gantry angles, allowing faster and more flexible CBCT scans with dose reductions.

HEAL: High-Frequency Enhanced and Attention-Guided Learning Network for Sparse-View CT Reconstruction

X-ray computed tomography (CT) imaging technology has become an indispensable diagnostic tool in clinical examination. However, it poses a risk of ionizing radiation, making the reduction of radiation dose one of the current research hotspots in CT imaging. Sparse-view imaging, as one of the main methods for reducing radiation dose, has made significant progress in recent years. In particular, sparse-view reconstruction methods based on deep learning have shown promising results. Nevertheless, efficiently recovering image details under ultra-sparse conditions remains a challenge. To address this challenge, this paper proposes a high-frequency enhanced and attention-guided learning Network (HEAL). HEAL includes three optimization strategies to achieve detail enhancement: Firstly, we introduce a dual-domain progressive enhancement module, which leverages fidelity constraints within each domain and consistency constraints across domains to effectively narrow the solution space. Secondly, we incorporate both channel and spatial attention mechanisms to improve the network's feature-scaling process. Finally, we propose a high-frequency component enhancement regularization term that integrates residual learning with direction-weighted total variation, utilizing directional cues to effectively distinguish between noise and textures. The HEAL network is trained, validated and tested under different ultra-sparse configurations of 60 views and 30 views, demonstrating its advantages in reconstruction accuracy and detail enhancement.

Iterative reconstruction for limited-angle CT using implicit neural representation

Objective.Limited-angle computed tomography (CT) presents a challenge due to its ill-posed nature. In such scenarios, analytical reconstruction methods often exhibit severe artifacts. To tackle this inverse problem, several supervised deep learning-based approaches have been proposed. However, they are constrained by limitations such as generalization issue and the difficulty of acquiring a large amount of paired CT images.Approach.In this work, we propose an iterative neural reconstruction framework designed for limited-angle CT. By leveraging a coordinate-based neural representation, we formulate tomographic reconstruction as a convex optimization problem involving a deep neural network. We then employ differentiable projection layer to optimize this network by minimizing the discrepancy between the predicted and measured projection data. In addition, we introduce a prior-based weight initialization method to ensure the network starts optimization with an informed initial guess. This strategic initialization significantly improves the quality of iterative reconstruction by stabilizing the divergent behavior in ill-posed neural fields. Our method operates in a self-supervised manner, thereby eliminating the need for extensive data.Main results.The proposed method outperforms other iterative and learning-based methods. Experimental results on XCAT and Mayo Clinic datasets demonstrate the effectiveness of our approach in restoring anatomical features as well as structures. This finding was substantiated by visual inspections and quantitative evaluations using NRMSE, PSNR, and SSIM. Moreover, we conduct a comprehensive investigation into the divergent behavior of iterative neural reconstruction, thus revealing its suboptimal convergence when starting from scratch. In contrast, our method consistently produced accurate images by incorporating an initial estimate as informed initialization.Significance.This work showcases the feasibility to reconstruct high-fidelity CT images from limited-angle x-ray projections. The proposed methodology introduces a novel data-free approach to enhance medical imaging, holding promise across various clinical applications.

Hybrid-supervised deep learning for domain transfer 3D protoacoustic image reconstruction

Objective. Protoacoustic imaging showed great promise in providing real-time 3D dose verification of proton therapy. However, the limited acquisition angle in protoacoustic imaging induces severe artifacts, which impairs its accuracy for dose verification. In this study, we developed a hybrid-supervised deep learning method for protoacoustic imaging to address the limited view issue.Approach. We proposed a Recon-Enhance two-stage deep learning method. In the Recon-stage, a transformer-based network was developed to reconstruct initial pressure maps from raw acoustic signals. The network is trained in a hybrid-supervised approach, where it is first trained using supervision by the iteratively reconstructed pressure map and then fine-tuned using transfer learning and self-supervision based on the data fidelity constraint. In the enhance-stage, a 3D U-net is applied to further enhance the image quality with supervision from the ground truth pressure map. The final protoacoustic images are then converted to dose for proton verification.Main results. The results evaluated on a dataset of 126 prostate cancer patients achieved an average root mean squared errors (RMSE) of 0.0292, and an average structural similarity index measure (SSIM) of 0.9618, out-performing related start-of-the-art methods. Qualitative results also demonstrated that our approach addressed the limit-view issue with more details reconstructed. Dose verification achieved an average RMSE of 0.018, and an average SSIM of 0.9891. Gamma index evaluation demonstrated a high agreement (94.7% and 95.7% for 1%/3 mm and 1%/5 mm) between the predicted and the ground truth dose maps. Notably, the processing time was reduced to 6 s, demonstrating its feasibility for online 3D dose verification for prostate proton therapy.Significance. Our study achieved start-of-the-art performance in the challenging task of direct reconstruction from radiofrequency signals, demonstrating the great promise of PA imaging as a highly efficient and accurate tool forinvivo3D proton dose verification to minimize the range uncertainties of proton therapy to improve its precision and outcomes.

A model-based direct inversion network (MDIN) for dual spectral computed tomography

Objective. Dual spectral computed tomography (DSCT) is a very challenging problem in the field of imaging. Due to the nonlinearity of its mathematical model, the images reconstructed by the conventional CT usually suffer from the beam hardening artifacts. Additionally, several existing DSCT methods rely heavily on the information of the spectra, which is often not readily available in applications. To address this problem, in this study, we aim to develop a novel approach to improve the DSCT reconstruction performance.Approach. A model-based direct inversion network (MDIN) is proposed for DSCT, which can directly predict the basis material images from the collected polychromatic projections. The all operations are performed in the network, requiring neither the conventional algorithms nor the information of the spectra. It can be viewed as an approximation to the inverse procedure of DSCT imaging model. The MDIN is composed of projection pre-decomposition module (PD-module), domain transformation layer (DT-layer), and image post-decomposition module (ID-module). The PD-module first performs the pre-decomposition on the polychromatic projections that consists of a series of stacked one-dimensional convolution layers. The DT-layer is designed to obtain the preliminary decomposed results, which has the characteristics of sparsely connected and learnable parameters. And the ID-module uses a deep neural network to further decompose the reconstructed results of the DT-layer so as to achieve higher-quality basis material images.Main results. Numerical experiments demonstrate that the proposed MDIN has significant advantages in substance decomposition, artifact reduction and noise suppression compared to other methods in the DSCT reconstruction.Significance. The proposed method has a flexible applicability, which can be extended to other CT problems, such as multi-spectral CT and low dose CT.

PIDNET: Polar Transformation Based Implicit Disentanglement Network for Truncation Artifacts

The interior problem, a persistent ill-posed challenge in CT imaging, gives rise to truncation artifacts capable of distorting CT values, thereby significantly impacting clinical diagnoses. Traditional methods have long struggled to effectively solve this issue until the advent of supervised models built on deep neural networks. However, supervised models are constrained by the need for paired data, limiting their practical application. Therefore, we propose a simple and efficient unsupervised method based on the Cycle-GAN framework. Introducing an implicit disentanglement strategy, we aim to separate truncation artifacts from content information. The separated artifact features serve as complementary constraints and the source of generating simulated paired data to enhance the training of the sub-network dedicated to removing truncation artifacts. Additionally, we incorporate polar transformation and an innovative constraint tailored specifically for truncation artifact features, further contributing to the effectiveness of our approach. Experiments conducted on multiple datasets demonstrate that our unsupervised network outperforms the traditional Cycle-GAN model significantly. When compared to state-of-the-art supervised models trained on paired datasets, our model achieves comparable visual results and closely aligns with quantitative evaluation metrics.

SEA-Net: Structure-Enhanced Attention Network for Limited-Angle CBCT Reconstruction of Clinical Projection Data

This work aims to improve limited-angle (LA) cone beam computed tomography (CBCT) by developing deep learning (DL) methods for real clinical CBCT projection data, which is the first feasibility study of clinical-projection-data-based LA-CBCT, to the best of our knowledge. In radiation therapy (RT), CBCT is routinely used as the on-board imaging modality for patient setup. Compared to diagnostic CT, CBCT has a long acquisition time, e.g., 60 seconds for a full 360° rotation, which is subject to the motion artifact. Therefore, the LA-CBCT, if achievable, is of the great interest for the purpose of RT, for its proportionally reduced scanning time in addition to the radiation dose. However, LA-CBCT suffers from severe wedge artifacts and image distortions. Targeting at real clinical projection data, we have explored various DL methods such as image/data/hybrid-domain methods and finally developed a so-called Structure-Enhanced Attention Network (SEA-Net) method that has the best image quality from clinical projection data among the DL methods we have implemented. Specifically, the proposed SEA-Net employs a specialized structure enhancement sub-network to promote texture preservation. Based on the observation that the distribution of wedge artifacts in reconstruction images is non-uniform, the spatial attention module is utilized to emphasize the relevant regions while ignores the irrelevant ones, which leads to more accurate texture restoration.