Artifact and Detail Attention Generative Adversarial Networks for Low-Dose CT Denoising

Improved lung nodule segmentation with a squeeze excitation dilated attention based residual UNet

The diverse types and sizes, proximity to non-nodule structures, identical shape characteristics, and varying sizes of nodules make them challenging for segmentation methods. Although many efforts have been made in automatic lung nodule segmentation, most of them have not sufficiently addressed the challenges related to the type and size of nodules, such as juxta-pleural and juxta-vascular nodules. The current research introduces a Squeeze-Excitation Dilated Attention-based Residual U-Net (SEDARU-Net) with a robust intensity normalization technique to address the challenges related to different types and sizes of lung nodules and to achieve an improved lung nodule segmentation. After preprocessing the images with the intensity normalization method and extracting the Regions of Interest by YOLOv3, they are fed into the SEDARU-Net with dilated convolutions in the encoder part. Then, the extracted features are given to the decoder part, which involves transposed convolutions, Squeeze-Excitation Dilated Residual blocks, and skip connections equipped with an Attention Gate, to decode the feature maps and construct the segmentation mask. The proposed model was evaluated using the publicly available Lung Nodule Analysis 2016 (LUNA16) dataset, achieving a Dice Similarity Coefficient of 97.86%, IoU of 96.40%, sensitivity of 96.54%, and precision of 98.84%. Finally, it was shown that each added component to the U-Net's structure and the intensity normalization technique increased the Dice Similarity Coefficient by more than 2%. The proposed method suggests a potential clinical tool to address challenges related to the segmentation of lung nodules with different types located in the proximity of non-nodule structures.

Special Issue: Artificial Intelligence in Advanced Medical Imaging

Medical imaging is of great significance in modern medicine and is a crucial part of medical diagnosis [...].

A systematic review of deep learning-based denoising for low-dose computed tomography from a perceptual quality perspective

Low-dose computed tomography (LDCT) scans are essential in reducing radiation exposure but often suffer from significant image noise that can impair diagnostic accuracy. While deep learning approaches have enhanced LDCT denoising capabilities, the predominant reliance on objective metrics like PSNR and SSIM has resulted in over-smoothed images that lack critical detail. This paper explores advanced deep learning methods tailored specifically to improve perceptual quality in LDCT images, focusing on generating diagnostic-quality images preferred in clinical practice. We review and compare current methodologies, including perceptual loss functions and generative adversarial networks, addressing the significant limitations of current benchmarks and the subjective nature of perceptual quality evaluation. Through a systematic analysis, this study underscores the urgent need for developing methods that balance both perceptual and diagnostic quality, proposing new directions for future research in the field.

Cross-Domain Denoising for Low-Dose Multi-Frame Spiral Computed Tomography

Computed tomography (CT) has been used worldwide as a non-invasive test to assist in diagnosis. However, the ionizing nature of X-ray exposure raises concerns about potential health risks such as cancer. The desire for lower radiation doses has driven researchers to improve reconstruction quality. Although previous studies on low-dose computed tomography (LDCT) denoising have demonstrated the effectiveness of learning-based methods, most were developed on the simulated data. However, the real-world scenario differs significantly from the simulation domain, especially when using the multi-slice spiral scanner geometry. This paper proposes a two-stage method for the commercially available multi-slice spiral CT scanners that better exploits the complete reconstruction pipeline for LDCT denoising across different domains. Our approach makes good use of the high redundancy of multi-slice projections and the volumetric reconstructions while leveraging the over-smoothing issue in conventional cascaded frameworks caused by aggressive denoising. The dedicated design also provides a more explicit interpretation of the data flow. Extensive experiments on various datasets showed that the proposed method could remove up to 70% of noise without compromised spatial resolution, while subjective evaluations by two experienced radiologists further supported its superior performance against state-of-the-art methods in clinical practice. Code is available at https://github.com/YCL92/TMD-LDCT.

Computational modeling of tumor invasion from limited and diverse data in Glioblastoma

For diseases with high morbidity rates such as Glioblastoma Multiforme, the prognostic and treatment planning pipeline requires a comprehensive analysis of imaging, clinical, and molecular data. Many mutations have been shown to correlate strongly with the median survival rate and response to therapy of patients. Studies have demonstrated that these mutations manifest as specific visual biomarkers in tumor imaging modalities such as MRI. To minimize the number of invasive procedures on a patient and for the overall resource optimization for the prognostic and treatment planning process, the correlation of imaging and molecular features has garnered much interest. While the tumor mass is the most significant feature, the impacted tissue surrounding the tumor is also a significant biomarker contributing to the visual manifestation of mutations - which has not been studied as extensively. The pattern of tumor growth impacts the surrounding tissue accordingly, which is a reflection of tumor properties as well. Modeling how the tumor growth impacts the surrounding tissue can reveal important information about the patterns of tumor enhancement, which in turn has significant diagnostic and prognostic value. This paper presents the first work to automate the computational modeling of the impacted tissue surrounding the tumor using generative deep learning. The paper isolates and quantifies the impact of the Tumor Invasion (TI) on surrounding tissue based on change in mutation status, subsequently assessing its prognostic value. Furthermore, a TI Generative Adversarial Network (TI-GAN) is proposed to model the tumor invasion properties. Extensive qualitative and quantitative analyses, cross-dataset testing, and radiologist blind tests are carried out to demonstrate that TI-GAN can realistically model the tumor invasion under practical challenges of medical datasets such as limited data and high intra-class heterogeneity.

OA-GAN: organ-aware generative adversarial network for synthesizing contrast-enhanced medical images

Contrast-enhanced computed tomography (CE-CT) images are vital for clinical diagnosis of focal liver lesions (FLLs). However, the use of CE-CT images imposes a significant burden on patients due to the injection of contrast agents and extended shooting. Deep learning-based image synthesis models offer a promising solution that synthesizes CE-CT images from non-contrasted CT (NC-CT) images. Unlike natural images, medical image synthesis requires a specific focus on certain organs or localized regions to ensure accurate diagnosis. Determining how to effectively emphasize target organs poses a challenging issue in medical image synthesis. To solve this challenge, we present a novel CE-CT image synthesis model called, Organ-Aware Generative Adversarial Network (OA-GAN). The OA-GAN comprises an organ-aware (OA) network and a dual decoder-based generator. First, the OA network learns the most discriminative spatial features about the target organ (i.e. liver) by utilizing the ground truth organ mask as localization cues. Subsequently, NC-CT image and captured feature are fed into the dual decoder-based generator, which employs a local and global decoder network to simultaneously synthesize the organ and entire CECT image. Moreover, the semantic information extracted from the local decoder is transferred to the global decoder to facilitate better reconstruction of the organ in entire CE-CT image. The qualitative and quantitative evaluation on a CE-CT dataset demonstrates that the OA-GAN outperforms state-of-the-art approaches for synthesizing two types of CE-CT images such as arterial phase and portal venous phase. Additionally, subjective evaluations by expert radiologists and a deep learning-based FLLs classification also affirm that CE-CT images synthesized from the OA-GAN exhibit a remarkable resemblance to real CE-CT images.

A Review of deep learning methods for denoising of medical low-dose CT images

To prevent patients from being exposed to excess of radiation in CT imaging, the most common solution is to decrease the radiation dose by reducing the X-ray, and thus the quality of the resulting low-dose CT images (LDCT) is degraded, as evidenced by more noise and streaking artifacts. Therefore, it is important to maintain high quality CT image while effectively reducing radiation dose. In recent years, with the rapid development of deep learning technology, deep learning-based LDCT denoising methods have become quite popular because of their data-driven and high-performance features to achieve excellent denoising results. However, to our knowledge, no relevant article has so far comprehensively introduced and reviewed advanced deep learning denoising methods such as Transformer structures in LDCT denoising tasks. Therefore, based on the literatures related to LDCT image denoising published from year 2016-2023, and in particular from 2020 to 2023, this study presents a systematic survey of current situation, and challenges and future research directions in LDCT image denoising field. Four types of denoising networks are classified according to the network structure: CNN-based, Encoder-Decoder-based, GAN-based, and Transformer-based denoising networks, and each type of denoising network is described and summarized from the perspectives of structural features and denoising performances. Representative deep-learning denoising methods for LDCT are experimentally compared and analyzed. The study results show that CNN-based denoising methods capture image details efficiently through multi-level convolution operation, demonstrating superior denoising effects and adaptivity. Encoder-decoder networks with MSE loss, achieve outstanding results in objective metrics. GANs based methods, employing innovative generators and discriminators, obtain denoised images that exhibit perceptually a closeness to NDCT. Transformer-based methods have potential for improving denoising performances due to their powerful capability in capturing global information. Challenges and opportunities for deep learning based LDCT denoising are analyzed, and future directions are also presented.

CT image denoising methods for image quality improvement and radiation dose reduction

With the ever-increasing use of computed tomography (CT), concerns about its radiation dose have become a significant public issue. To address the need for radiation dose reduction, CT denoising methods have been widely investigated and applied in low-dose CT images. Numerous noise reduction algorithms have emerged, such as iterative reconstruction and most recently, deep learning (DL)-based approaches. Given the rapid advancements in Artificial Intelligence techniques, we recognize the need for a comprehensive review that emphasizes the most recently developed methods. Hence, we have performed a thorough analysis of existing literature to provide such a review. Beyond directly comparing the performance, we focus on pivotal aspects, including model training, validation, testing, generalizability, vulnerability, and evaluation methods. This review is expected to raise awareness of the various facets involved in CT image denoising and the specific challenges in developing DL-based models.

Joint denoising and interpolating network for low-dose cone-beam CT reconstruction under hybrid dose-reduction strategy

Cone-beam computed tomography (CBCT) is generally reconstructed with hundreds of two-dimensional X-Ray projections through the FDK algorithm, and its excessive ionizing radiation of X-Ray may impair patients' health. Two common dose-reduction strategies are to either lower the intensity of X-Ray, i.e., low-intensity CBCT, or reduce the number of projections, i.e., sparse-view CBCT. Existing efforts improve the low-dose CBCT images only under a single dose-reduction strategy. In this paper, we argue that applying the two strategies simultaneously can reduce dose in a gentle manner and avoid the extreme degradation of the projection data in a single dose-reduction strategy, especially under ultra-low-dose situations. Therefore, we develop a Joint Denoising and Interpolating Network (JDINet) in projection domain to improve the CBCT quality with the hybrid low-intensity and sparse-view projections. Specifically, JDINet mainly includes two important components, i.e., denoising module and interpolating module, to respectively suppress the noise caused by the low-intensity strategy and interpolate the missing projections caused by the sparse-view strategy. Because FDK actually utilizes the projection information after ramp-filtering, we develop a filtered structural similarity constraint to help JDINet focus on the reconstruction-required information. Afterward, we employ a Postprocessing Network (PostNet) in the reconstruction domain to refine the CBCT images that are reconstructed with denoised and interpolated projections. In general, a complete CBCT reconstruction framework is built with JDINet, FDK, and PostNet. Experiments demonstrate that our framework decreases RMSE by approximately 8 %, 15 %, and 17 %, respectively, on the 1/8, 1/16, and 1/32 dose data, compared to the latest methods. In conclusion, our learning-based framework can be deeply imbedded into the CBCT systems to promote the development of CBCT. Source code is available at https://github.com/LianyingChao/FusionLowDoseCBCT.

Learnable PM diffusion coefficients and reformative coordinate attention network for low dose CT denoising

Objective.Various deep learning methods have recently been used for low dose CT (LDCT) denoising. Aggressive denoising may destroy the edge and fine anatomical structures of CT images. Therefore a key issue in LDCT denoising tasks is the difficulty of balancing noise/artifact suppression and edge/structure preservation.Approach.We proposed an LDCT denoising network based on the encoder-decoder structure, namely the Learnable PM diffusion coefficient and efficient attention network (PMA-Net). First, using the powerful feature modeling capability of partial differential equations, we constructed a multiple learnable edge module to generate precise edge information, incorporating the anisotropic image processing idea of Perona-Malik (PM) model into the neural network. Second, a multiscale reformative coordinate attention module was designed to extract multiscale information. Non-overlapping dilated convolution capturing abundant contextual content was combined with coordinate attention which could embed the spatial location information of important features into the channel attention map. Finally, we imposed additional constraints on the edge information using edge-enhanced multiscale perceptual loss to avoid structure loss and over-smoothing.Main results.Experiments are conducted on simulated and real datasets. The quantitative and qualitative results show that the proposed method has better performance in suppressing noise/artifacts and preserving edges/structures.Significance.This work proposes a novel edge feature extraction method that unfolds partial differential equation into neural networks, which contributes to the interpretability and clinical application value of neural network.

Deep Learning-based Post Hoc CT Denoising for the Coronary Perivascular Fat Attenuation Index

RATIONALE AND OBJECTIVES

Coronary inflammation related to high-risk hemorrhagic plaques can be captured by the perivascular fat attenuation index (FAI) using coronary computed tomography angiography (CCTA). Since the FAI is susceptible to image noise, we believe deep learning (DL)-based post hoc noise reduction can improve diagnostic capability. We aimed to assess the diagnostic performance of the FAI in DL-based denoised high-fidelity CCTA images compared with coronary plaque magnetic resonance imaging (MRI) delivered high-intensity hemorrhagic plaques (HIPs).

MATERIALS AND METHODS

We retrospectively reviewed 43 patients who underwent CCTA and coronary plaque MRI. We generated high-fidelity CCTA images by denoising the standard CCTA images using a residual dense network that supervised the denoising task by averaging three cardiac phases with nonrigid registration. We measured the FAIs as the mean CT value of all voxels (range of -190 to -30 HU) located within a radial distance from the outer proximal right coronary artery wall. The diagnostic reference standard was defined as HIPs (high-risk hemorrhagic plaques) using MRI. The diagnostic performance of the FAI in the original and denoised images was assessed using receiver operating characteristic curves.

RESULTS

Of 43 patients, 13 had HIPs. The denoised CCTA improved the area under the curve (0.89 [95% confidence interval (CI) 0.78-0.99]) of the FAI compared with that in the original image (0.77 [95% CI, 0.62-0.91], p = 0.008). The optimal cutoff value for predicting HIPs in denoised CCTA was -69 HU with 0.85 (11/13) sensitivity, 0.79 (25/30) specificity, and 0.80 (36/43) accuracy.

CONCLUSION

DL-based denoised high-fidelity CCTA improved the AUC and specificity of the FAI for predicting HIPs.

A cascade-based dual-domain data correction network for sparse view CT image reconstruction

Computed tomography (CT) provides non-invasive anatomical structures of the human body and is also widely used for clinical diagnosis, but excessive ionizing radiation in X-rays can cause harm to the human body. Therefore, the researchers obtained sparse sinograms reconstructed sparse view CT images (SVCT) by reducing the amount of X-ray projection, thereby reducing the radiological effects caused by radiation. This paper proposes a cascade-based dual-domain data correction network (CDDCN), which can effectively combine the complementary information contained in the sinogram domain and the image domain to reconstruct high-quality CT images from sparse view sinograms. Specifically, several encoder-decoder subnets are cascaded in the sinogram domain to reconstruct artifact-free and noise-free CT images. In the encoder-decoder subnets, spatial-channel domain learning is designed to achieve efficient feature fusion through a group merging structure, providing continuous and elaborate pixel-level features and improving feature extraction efficiency. At the same time, to ensure that the original sinogram data collected can be retained, a sinogram data consistency layer is proposed to ensure the fidelity of the sinogram data. To further maintain the consistency between the reconstructed image and the reference image, a multi-level composite loss function is designed for regularization to compensate for excessive smoothing and distortion of the image caused by pixel loss and preserve image details and texture. Quantitative and qualitative analysis shows that CDDCN achieves competitive results in artifact removal, edge preservation, detail restoration, and visual improvement for sparsely sampled data under different views.

Self-supervised deep learning for joint 3D low-dose PET/CT image denoising

Deep learning (DL)-based denoising of low-dose positron emission tomography (LDPET) and low-dose computed tomography (LDCT) has been widely explored. However, previous methods have focused only on single modality denoising, neglecting the possibility of simultaneously denoising LDPET and LDCT using only one neural network, i.e., joint LDPET/LDCT denoising. Moreover, DL-based denoising methods generally require plenty of well-aligned LD-normal-dose (LD-ND) sample pairs, which can be difficult to obtain. To this end, we propose a self-supervised two-stage training framework named MAsk-then-Cycle (MAC), to achieve self-supervised joint LDPET/LDCT denoising. The first stage of MAC is masked autoencoder (MAE)-based pre-training and the second stage is self-supervised denoising training. Specifically, we propose a self-supervised denoising strategy named cycle self-recombination (CSR), which enables denoising without well-aligned sample pairs. Unlike other methods that treat noise as a homogeneous whole, CSR disentangles noise into signal-dependent and independent noises. This is more in line with the actual imaging process and allows for flexible recombination of noises and signals to generate new samples. These new samples contain implicit constraints that can improve the network's denoising ability. Based on these constraints, we design multiple loss functions to enable self-supervised training. Then we design a CSR-based denoising network to achieve joint 3D LDPET/LDCT denoising. Existing self-supervised methods generally lack pixel-level constraints on networks, which can easily lead to additional artifacts. Before denoising training, we perform MAE-based pre-training to indirectly impose pixel-level constraints on networks. Experiments on an LDPET/LDCT dataset demonstrate its superiority over existing methods. Our method is the first self-supervised joint LDPET/LDCT denoising method. It does not require any prior assumptions and is therefore more robust.

A novel denoising method for low-dose CT images based on transformer and CNN

Computed Tomography (CT) has become a mainstream imaging tool in medical diagnosis. However, the issue of increased cancer risk due to radiation exposure has raised public concern. Low-dose computed tomography (LDCT) technique is a CT scan with lower radiation dose than conventional scans. LDCT is used to make a diagnosis of lesions with the smallest dose of x-rays, and is currently mainly used for early lung cancer screening. However, LDCT has severe image noise, and these noises affect adversely the quality of medical images and thus the diagnosis of lesions. In this paper, we propose a novel LDCT image denoising method based on transformer combined with convolutional neural network (CNN). The encoder part of the network is based on CNN, which is mainly used to extract the image detail information. In the decoder part, we propose a dual-path transformer block (DPTB), which extracts the features of input of the skip connection and the features of input of the previous level through two paths respectively. DPTB can better restore the detail and structure information of the denoised image. In order to pay more attention to the key regions of the feature images extracted at the shallow level of the network, we also propose a multi-feature spatial attention block (MSAB) in the skip connection part. Experimental studies are conducted, and comparisons with the state-of-the-art networks are made, and the results demonstrate that the developed method can effectively remove the noise in CT images and improve the image quality in the evaluation metrics of peak signal to noise ratio (PSNR), structural similarity (SSIM), and root mean square error (RMSE) and is superior to the state-of-the-art models. Our method achieved 28.9720 of PSNR, 0.8595 of SSIM and 14.8657 of RMSE on the Mayo Clinic LDCT Grand Challenge dataset. For different noise level σ (15, 35, and 55) on the QIN_LUNG_CT dataset, our proposed also achieved better performances.

Compound feature attention network with edge enhancement for low-dose CT denoising

BACKGROUND

Low-dose CT (LDCT) images usually contain serious noise and artifacts, which weaken the readability of the image.

OBJECTIVE

To solve this problem, we propose a compound feature attention network with edge enhancement for LDCT denoising (CFAN-Net), which consists of an edge-enhanced module and a proposed compound feature attention block (CFAB).

METHODS

The edge enhancement module extracts edge details with the trainable Sobel convolution. CFAB consists of an interactive feature learning module (IFLM), a multi-scale feature fusion module (MFFM), and a joint attention module (JAB), which removes noise from LDCT images in a coarse-to-fine manner. First, in IFLM, the noise is initially removed by cross-latitude interactive judgment learning. Second, in MFFM, multi-scale and pixel attention are integrated to explore fine noise removal. Finally, in JAB, we focus on key information, extract useful features, and improve the efficiency of network learning. To construct a high-quality image, we repeat the above operation by cascading CFAB.

RESULTS

By applying CFAN-Net to process the 2016 NIH AAPM-Mayo LDCT challenge test dataset, experiments show that the peak signal-to-noise ratio value is 33.9692 and the structural similarity value is 0.9198.

CONCLUSIONS

Compared with several existing LDCT denoising algorithms, CFAN-Net effectively preserves the texture of CT images while removing noise and artifacts.

Enhancement based convolutional dictionary network with adaptive window for low-dose CT denoising

BACKGROUND

Recently, one promising approach to suppress noise/artifacts in low-dose CT (LDCT) images is the CNN-based approach, which learns the mapping function from LDCT to normal-dose CT (NDCT). However, most CNN-based methods are purely data-driven, thus lacking sufficient interpretability and often losing details.

OBJECTIVE

To solve this problem, we propose a deep convolutional dictionary learning method for LDCT denoising, in which a novel convolutional dictionary learning model with adaptive window (CDL-AW) is designed, and a corresponding enhancement-based convolutional dictionary learning network (called ECDAW-Net) is constructed to unfold the CDL-AW model iteratively using the proximal gradient descent technique.

METHODS

In detail, the adaptive window-constrained convolutional dictionary atom is proposed to alleviate spectrum leakage caused by data truncation during convolution. Furthermore, in the ECDAW-Net, a multi-scale edge extraction module that consists of LoG and Sobel convolution layers is proposed in the unfolding iteration, to supplement lost textures and details. Additionally, to further improve the detail retention ability, the ECDAW-Net is trained by the compound loss function of the pixel-level MSE loss and the proposed patch-level loss, which can assist to retain richer structural information.

RESULTS

Applying ECDAW-Net to the Mayo dataset, we obtained the highest peak signal-to-noise ratio (33.94) and sub-optimal structural similarity (0.92).

CONCLUSIONS

Compared with some state-of-art methods, the interpretable ECDAW-Net performs well in suppressing noise/artifacts and preserving textures of tissue.

Structure-preserved meta-learning uniting network for improving low-dose CT quality

Objective.Deep neural network (DNN) based methods have shown promising performances for low-dose computed tomography (LDCT) imaging. However, most of the DNN-based methods are trained on simulated labeled datasets, and the low-dose simulation algorithms are usually designed based on simple statistical models which deviate from the real clinical scenarios, which could lead to issues of overfitting, instability and poor robustness. To address these issues, in this work, we present a structure-preserved meta-learning uniting network (shorten as 'SMU-Net') to suppress noise-induced artifacts and preserve structure details in the unlabeled LDCT imaging task in real scenarios.Approach.Specifically, the presented SMU-Net contains two networks, i.e., teacher network and student network. The teacher network is trained on simulated labeled dataset and then helps the student network train with the unlabeled LDCT images via the meta-learning strategy. The student network is trained on real LDCT dataset with the pseudo-labels generated by the teacher network. Moreover, the student network adopts the Co-teaching strategy to improve the robustness of the presented SMU-Net.Main results.We validate the proposed SMU-Net method on three public datasets and one real low-dose dataset. The visual image results indicate that the proposed SMU-Net has superior performance on reducing noise-induced artifacts and preserving structure details. And the quantitative results exhibit that the presented SMU-Net method generally obtains the highest signal-to-noise ratio (PSNR), the highest structural similarity index measurement (SSIM), and the lowest root-mean-square error (RMSE) values or the lowest natural image quality evaluator (NIQE) scores.Significance.We propose a meta learning strategy to obtain high-quality CT images in the LDCT imaging task, which is designed to take advantage of unlabeled CT images to promote the reconstruction performance in the LDCT environments.

Improving image quality and lung nodule detection for low-dose chest CT by using generative adversarial network reconstruction

OBJECTIVES

To investigate the improvement of two denoising models with different learning targets (Dir and Res) of generative adversarial network (GAN) on image quality and lung nodule detectability in chest low-dose CT (LDCT).

METHODS

In training phase, by using LDCT images simulated from standard dose CT (SDCT) of 200 participants, Dir model was trained targeting SDCT images, while Res model targeting the residual between SDCT and LDCT images. In testing phase, a phantom and 95 chest LDCT, exclusively with training data, were included for evaluation of imaging quality and pulmonary nodules detectability.

RESULTS

For phantom images, structural similarity, peak signal-to-noise ratio of both Res and Dir models were higher than that of LDCT. Standard deviation of Res model was the lowest. For patient images, image noise and quality of both two models, were better than that of LDCT. Artifacts of Res model was less than that of LDCT. The diagnostic sensitivity of lung nodule by two readers for LDCT, Res and Dir model, were 72/77%, 79/83% and 72/79% respectively.

CONCLUSION

Two GAN denoising models, including Res and Dir trained with different targets, could effectively reduce image noise of chest LDCT. The image quality evaluation scoring and nodule detectability of Res denoising model was better than that of Dir denoising model and that of hybrid IR images.

ADVANCES IN KNOWLEDGE

The GAN-trained model, which learned the residual between SDCT and LDCT images, reduced image noise and increased the lung nodule detectability by radiologists on chest LDCT. This demonstrates the potential for clinical benefit.

A more effective CT synthesizer using transformers for cone-beam CT-guided adaptive radiotherapy

Purpose

The challenge of cone-beam computed tomography (CBCT) is its low image quality, which limits its application for adaptive radiotherapy (ART). Despite recent substantial improvement in CBCT imaging using the deep learning method, the image quality still needs to be improved for effective ART application. Spurred by the advantages of transformers, which employs multi-head attention mechanisms to capture long-range contextual relations between image pixels, we proposed a novel transformer-based network (called TransCBCT) to generate synthetic CT (sCT) from CBCT. This study aimed to further improve the accuracy and efficiency of ART.

Materials and methods

In this study, 91 patients diagnosed with prostate cancer were enrolled. We constructed a transformer-based hierarchical encoder–decoder structure with skip connection, called TransCBCT. The network also employed several convolutional layers to capture local context. The proposed TransCBCT was trained and validated on 6,144 paired CBCT/deformed CT images from 76 patients and tested on 1,026 paired images from 15 patients. The performance of the proposed TransCBCT was compared with a widely recognized style transferring deep learning method, the cycle-consistent adversarial network (CycleGAN). We evaluated the image quality and clinical value (application in auto-segmentation and dose calculation) for ART need.

Results

TransCBCT had superior performance in generating sCT from CBCT. The mean absolute error of TransCBCT was 28.8 ± 16.7 HU, compared to 66.5 ± 13.2 for raw CBCT, and 34.3 ± 17.3 for CycleGAN. It can preserve the structure of raw CBCT and reduce artifacts. When applied in auto-segmentation, the Dice similarity coefficients of bladder and rectum between auto-segmentation and oncologist manual contours were 0.92 and 0.84 for TransCBCT, respectively, compared to 0.90 and 0.83 for CycleGAN. When applied in dose calculation, the gamma passing rate (1%/1 mm criterion) was 97.5% ± 1.1% for TransCBCT, compared to 96.9% ± 1.8% for CycleGAN.

Conclusions

The proposed TransCBCT can effectively generate sCT for CBCT. It has the potential to improve radiotherapy accuracy.

Deep Learning-based Post Hoc CT Denoising for Myocardial Delayed Enhancement

Background To improve myocardial delayed enhancement (MDE) CT, a deep learning (DL)-based post hoc denoising method supervised with averaged MDE CT data was developed. Purpose To assess the image quality of denoised MDE CT images and evaluate their diagnostic performance by using late gadolinium enhancement (LGE) MRI as a reference. Materials and methods MDE CT data obtained by averaging three acquisitions with a single breath hold 5 minutes after the contrast material injection in patients from July 2020 to October 2021 were retrospectively reviewed. Preaveraged images obtained in 100 patients as inputs and averaged images as ground truths were used to supervise a residual dense network (RDN). The original single-shot image, standard averaged image, RDN-denoised original (DL_original) image, and RDN-denoised averaged (DL_ave) image of holdout cases were compared. In 40 patients, the CT value and image noise in the left ventricular cavity and myocardium were assessed. The segmental presence of MDE in the remaining 40 patients who underwent reference LGE MRI was evaluated. The sensitivity, specificity, and accuracy of each type of CT image and the improvement in accuracy achieved with the RDN were assessed using odds ratios (ORs) estimated with the generalized estimation equation. Results Overall, 180 patients (median age, 66 years [IQR, 53-74 years]; 107 men) were included. The RDN reduced image noise to 28% of the original level while maintaining equivalence in the CT values (P < .001 for all). The sensitivity, specificity, and accuracy of the original images were 77.9%, 84.4%, and 82.3%, of the averaged images were 89.7%, 87.9%, and 88.5%, of the DL_original images were 93.1%, 87.5%, and 89.3%, and of the DL_ave images were 95.1%, 93.1%, and 93.8%, respectively. DL_original images showed improved accuracy compared with the original images (OR, 1.8 [95% CI: 1.2, 2.9]; P = .011) and DL_ave images showed improved accuracy compared with the averaged images (OR, 2.0 [95% CI: 1.2, 3.5]; P = .009). Conclusion The proposed denoising network supervised with averaged CT images reduced image noise and improved the diagnostic performance for myocardial delayed enhancement CT. © RSNA, 2022 Online supplemental material is available for this article. See also the editorial by Vannier and Wang in this issue.

An architecture-level analysis on deep learning models for low-impact computations

Deep neural networks (DNNs) have made significant achievements in a wide variety of domains. For the deep learning tasks, multiple excellent hardware platforms provide efficient solutions, including graphics processing units (GPUs), central processing units (CPUs), field programmable gate arrays (FPGAs), and application-specific integrated circuit (ASIC). Nonetheless, CPUs outperform other solutions including GPUs in many cases for the inference workload of DNNs with the support of various techniques, such as the high-performance libraries being the basic building blocks for DNNs. Thus, CPUs have been a preferred choice for DNN inference applications, particularly in the low-latency demand scenarios. However, the DNN inference efficiency remains a critical issue, especially when low latency is required under conditions with limited hardware resources, such as embedded systems. At the same time, the hardware features have not been fully exploited for DNNs and there is much room for improvement. To this end, this paper conducts a series of experiments to make a thorough study for the inference workload of prominent state-of-the-art DNN architectures on a single-instruction-multiple-data (SIMD) CPU platform, as well as with widely applicable scopes for multiple hardware platforms. The study goes into depth in DNNs: the CPU kernel-instruction level performance characteristics of DNNs including branches, branch prediction misses, cache misses, etc, and the underlying convolutional computing mechanism at the SIMD level; The thorough layer-wise time consumption details with potential time-cost bottlenecks; And the exhaustive dynamic activation sparsity with exact details on the redundancy of DNNs. The research provides researchers with comprehensive and insightful details, as well as crucial target areas for optimising and improving the efficiency of DNNs at both the hardware and software levels.

A two-step method to improve image quality of CBCT with phantom-based supervised and patient-based unsupervised learning strategies

Objective.In this study, we aimed to develop deep learning framework to improve cone-beam computed tomography (CBCT) image quality for adaptive radiation therapy (ART) applications.Approach.Paired CBCT and planning CT images of 2 pelvic phantoms and 91 patients (15 patients for testing) diagnosed with prostate cancer were included in this study. First, well-matched images of rigid phantoms were used to train a U-net, which is the supervised learning strategy to reduce serious artifacts. Second, the phantom-trained U-net generated intermediate CT images from the patient CBCT images. Finally, a cycle-consistent generative adversarial network (CycleGAN) was trained with intermediate CT images and deformed planning CT images, which is the unsupervised learning strategy to learn the style of the patient images for further improvement. When testing or applying the trained model on patient CBCT images, the intermediate CT images were generated from the original CBCT image by U-net, and then the synthetic CT images were generated by the generator of CycleGAN with intermediate CT images as input. The performance was compared with conventional methods (U-net/CycleGAN alone trained with patient images) on the test set.Results.The proposed two-step method effectively improved the CBCT image quality to the level of CT scans. It outperformed conventional methods for region-of-interest contouring and HU calibration, which are important to ART applications. Compared with the U-net alone, it maintained the structure of CBCT. Compared with CycleGAN alone, our method improved the accuracy of CT number and effectively reduced the artifacts, making it more helpful for identifying the clinical target volume.Significance.This novel two-step method improves CBCT image quality by combining phantom-based supervised and patient-based unsupervised learning strategies. It has immense potential to be integrated into the ART workflow to improve radiotherapy accuracy.

Artifact-Assisted multi-level and multi-scale feature fusion attention network for low-dose CT denoising

BACKGROUND AND OBJECTIVE

Since low-dose computed tomography (LDCT) images typically have higher noise that may affect accuracy of disease diagnosis, the objective of this study is to develop and evaluate a new artifact-assisted feature fusion attention (AAFFA) network to extract and reduce image artifact and noise in LDCT images.

METHODS

In AAFFA network, a feature fusion attention block is constructed for local multi-scale artifact feature extraction and progressive fusion from coarse to fine. A multi-level fusion architecture based on skip connection and attention modules is also introduced for artifact feature extraction. Specifically, long-range skip connections are used to enhance and fuse artifact features with different depth levels. Then, the fused shallower features enter channel attention for better extraction of artifact features, and the fused deeper features are sent into pixel attention for focusing on the artifact pixel information. Besides, an artifact channel is designed to provide rich artifact features and guide the extraction of noise and artifact features. The AAPM LDCT Challenge dataset is used to train and test the network. The performance is evaluated by using both visual observation and quantitative metrics including peak signal-noise-ratio (PSNR), structural similarity index (SSIM) and visual information fidelity (VIF).

RESULTS

Using AAFFA network improves the averaged PSNR/SSIM/VIF values of AAPM LDCT images from 43.4961, 0.9595, 0.3926 to 48.2513, 0.9859, 0.4589, respectively.

CONCLUSIONS

The proposed AAFFA network is able to effectively reduce noise and artifacts while preserving object edges. Assessment of visual quality and quantitative index demonstrates the significant improvement compared with other image denoising methods.