Low-Dose CT Image Denoising Using a Generative Adversarial Network With Wasserstein Distance and Perceptual Loss

Deep learning based unpaired image-to-image translation applications for medical physics: a systematic review

Purpose. There is a growing number of publications on the application of unpaired image-to-image (I2I) translation in medical imaging. However, a systematic review covering the current state of this topic for medical physicists is lacking. The aim of this article is to provide a comprehensive review of current challenges and opportunities for medical physicists and engineers to apply I2I translation in practice.Methods and materials. The PubMed electronic database was searched using terms referring to unpaired (unsupervised), I2I translation, and medical imaging. This review has been reported in compliance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement. From each full-text article, we extracted information extracted regarding technical and clinical applications of methods, Transparent Reporting for Individual Prognosis Or Diagnosis (TRIPOD) study type, performance of algorithm and accessibility of source code and pre-trained models.Results. Among 461 unique records, 55 full-text articles were included in the review. The major technical applications described in the selected literature are segmentation (26 studies), unpaired domain adaptation (18 studies), and denoising (8 studies). In terms of clinical applications, unpaired I2I translation has been used for automatic contouring of regions of interest in MRI, CT, x-ray and ultrasound images, fast MRI or low dose CT imaging, CT or MRI only based radiotherapy planning, etc Only 5 studies validated their models using an independent test set and none were externally validated by independent researchers. Finally, 12 articles published their source code and only one study published their pre-trained models.Conclusion. I2I translation of medical images offers a range of valuable applications for medical physicists. However, the scarcity of external validation studies of I2I models and the shortage of publicly available pre-trained models limits the immediate applicability of the proposed methods in practice.

Utilization of an attentive map to preserve anatomical features for training convolutional neural-network-based low-dose CT denoiser

BACKGROUND

The purpose of a convolutional neural network (CNN)-based denoiser is to increase the diagnostic accuracy of low-dose computed tomography (LDCT) imaging. To increase diagnostic accuracy, there is a need for a method that reflects the features related to diagnosis during the denoising process.

PURPOSE

To provide a training strategy for LDCT denoisers that relies more on diagnostic task-related features to improve diagnostic accuracy.

METHODS

An attentive map derived from a lesion classifier (i.e., determining lesion-present or not) is created to represent the extent to which each pixel influences the decision by the lesion classifier. This is used as a weight to emphasize important parts of the image. The proposed training method consists of two steps. In the first one, the initial parameters of the CNN denoiser are trained using LDCT and normal-dose CT image pairs via supervised learning. In the second one, the learned parameters are readjusted using the attentive map to restore the fine details of the image.

RESULTS

Structural details and the contrast are better preserved in images generated by using the denoiser trained via the proposed method than in those generated by conventional denoisers. The proposed denoiser also yields higher lesion detectability and localization accuracy than conventional denoisers.

CONCLUSIONS

A denoiser trained using the proposed method preserves the small structures and the contrast in the denoised images better than without it. Specifically, using the attentive map improves the lesion detectability and localization accuracy of the denoiser.

A task-specific deep-learning-based denoising approach for myocardial perfusion SPECT

Deep-learning (DL)-based methods have shown significant promise in denoising myocardial perfusion SPECT images acquired at low dose. For clinical application of these methods, evaluation on clinical tasks is crucial. Typically, these methods are designed to minimize some fidelity-based criterion between the predicted denoised image and some reference normal-dose image. However, while promising, studies have shown that these methods may have limited impact on the performance of clinical tasks in SPECT. To address this issue, we use concepts from the literature on model observers and our understanding of the human visual system to propose a DL-based denoising approach designed to preserve observer-related information for detection tasks. The proposed method was objectively evaluated on the task of detecting perfusion defect in myocardial perfusion SPECT images using a retrospective study with anonymized clinical data. Our results demonstrate that the proposed method yields improved performance on this detection task compared to using low-dose images. The results show that by preserving task-specific information, DL may provide a mechanism to improve observer performance in low-dose myocardial perfusion SPECT.

Synthesizing MR Image Contrast Enhancement Using 3D High-Resolution ConvNets

OBJECTIVE

Gadolinium-based contrast agents (GBCAs) have been widely used to better visualize disease in brain magnetic resonance imaging (MRI). However, gadolinium deposition within the brain and body has raised safety concerns about the use of GBCAs. Therefore, the development of novel approaches that can decrease or even eliminate GBCA exposure while providing similar contrast information would be of significant use clinically.

METHODS

In this work, we present a deep learning based approach for contrast-enhanced T1 synthesis on brain tumor patients. A 3D high-resolution fully convolutional network (FCN), which maintains high resolution information through processing and aggregates multi-scale information in parallel, is designed to map pre-contrast MRI sequences to contrast-enhanced MRI sequences. Specifically, three pre-contrast MRI sequences, T1, T2 and apparent diffusion coefficient map (ADC), are utilized as inputs and the post-contrast T1 sequences are utilized as target output. To alleviate the data imbalance problem between normal tissues and the tumor regions, we introduce a local loss to improve the contribution of the tumor regions, which leads to better enhancement results on tumors.

RESULTS

Extensive quantitative and visual assessments are performed, with our proposed model achieving a PSNR of 28.24 dB in the brain and 21.2 dB in tumor regions.

CONCLUSION AND SIGNIFICANCE

Our results suggest the potential of substituting GBCAs with synthetic contrast images generated via deep learning.

Multi-scale cascaded networks for synthesis of mammogram to decrease intensity distortion and increase model-based perceptual similarity

PURPOSE

Synthetic digital mammogram (SDM) is a 2D image generated from digital breast tomosynthesis (DBT) and used as a substitute for a full-field digital mammogram (FFDM) to reduce the radiation dose for breast cancer screening. The previous deep learning-based method used FFDM images as the ground truth, and trained a single neural network to directly generate SDM images with similar appearances (e.g., intensity distribution, textures) to the FFDM images. However, the FFDM image has a different texture pattern from DBT. The difference in texture pattern might make the training of the neural network unstable and result in high-intensity distortion, which makes it hard to decrease intensity distortion and increase perceptual similarity (e.g., generate similar textures) at the same time. Clinically, radiologists want to have a 2D synthesized image that feels like an FFDM image in vision and preserves local structures such as both mass and microcalcifications (MCs) in DBT because radiologists have been trained on reading FFDM images for a long time, while local structures are important for diagnosis. In this study, we proposed to use a deep convolutional neural network to learn the transformation to generate SDM from DBT.

METHOD

To decrease intensity distortion and increase perceptual similarity, a multi-scale cascaded network (MSCN) is proposed to generate low-frequency structures (e.g., intensity distribution) and high-frequency structures (e.g., textures) separately. The MSCN consist of two cascaded sub-networks: the first sub-network is used to predict the low-frequency part of the FFDM image; the second sub-network is used to generate a full SDM image with textures similar to the FFDM image based on the prediction of the first sub-network. The mean-squared error (MSE) objective function is used to train the first sub-network, termed low-frequency network, to generate a low-frequency SDM image. The gradient-guided generative adversarial network's objective function is to train the second sub-network, termed high-frequency network, to generate a full SDM image with textures similar to the FFDM image.

RESULTS

1646 cases with FFDM and DBT were retrospectively collected from the Hologic Selenia system for training and validation dataset, and 145 cases with masses or MC clusters were independently collected from the Hologic Selenia system for testing dataset. For comparison, the baseline network has the same architecture as the high-frequency network and directly generates a full SDM image. Compared to the baseline method, the proposed MSCN improves the peak-to-noise ratio from 25.3 to 27.9 dB and improves the structural similarity from 0.703 to 0.724, and significantly increases the perceptual similarity.

CONCLUSIONS

The proposed method can stabilize the training and generate SDM images with lower intensity distortion and higher perceptual similarity.

Perfusion Maps Acquired From Dynamic Angiography MRI Using Deep Learning Approaches

BACKGROUND

A typical stroke MRI protocol includes perfusion-weighted imaging (PWI) and MR angiography (MRA), requiring a second dose of contrast agent. A deep learning method to acquire both PWI and MRA with single dose can resolve this issue.

PURPOSE

To acquire both PWI and MRA simultaneously using deep learning approaches.

STUDY TYPE

Retrospective.

SUBJECTS

A total of 60 patients (30-73 years old, 31 females) with ischemic symptoms due to occlusion or ≥50% stenosis (measured relative to proximal artery diameter) of the internal carotid artery, middle cerebral artery, or anterior cerebral artery. The 51/1/8 patient data were used as training/validation/test.

FIELD STRENGTH/SEQUENCE

A 3 T, time-resolved angiography with stochastic trajectory (contrast-enhanced MRA) and echo planar imaging (dynamic susceptibility contrast MRI, DSC-MRI).

ASSESSMENT

We investigated eight different U-Net architectures with different encoder/decoder sizes and with/without an adversarial network to generate perfusion maps from contrast-enhanced MRA. Relative cerebral blood volume (rCBV), relative cerebral blood flow (rCBF), mean transit time (MTT), and time-to-max (T_max ) were mapped from DSC-MRI and used as ground truth to train the networks and to generate the perfusion maps from the contrast-enhanced MRA input.

STATISTICAL TESTS

Normalized root mean square error, structural similarity (SSIM), peak signal-to-noise ratio (pSNR), DICE, and FID scores were calculated between the perfusion maps from DSC-MRI and contrast-enhanced MRA. One-tailed t-test was performed to check the significance of the improvements between networks. P values < 0.05 were considered significant.

RESULTS

The four perfusion maps were successfully extracted using the deep learning networks. U-net with multiple decoders and enhanced encoders showed the best performance (pSNR 24.7 ± 3.2 and SSIM 0.89 ± 0.08 for rCBV). DICE score in hypo-perfused area showed strong agreement between the generated perfusion maps and the ground truth (highest DICE: 0.95 ± 0.04).

DATA CONCLUSION

With the proposed approach, dynamic angiography MRI may provide vessel architecture and perfusion-relevant parameters simultaneously from a single scan.

EVIDENCE LEVEL

3 TECHNICAL EFFICACY: Stage 5.

STEDNet: Swin transformer-based encoder-decoder network for noise reduction in low-dose CT

BACKGROUND

Low-dose computed tomography (LDCT) can reduce the dose of X-ray radiation, making it increasingly significant for routine clinical diagnosis and treatment planning. However, the noise introduced by low-dose X-ray exposure degrades the quality of CT images, affecting the accuracy of clinical diagnosis. Purpose The noises, artifacts, and high-frequency components are similarly distributed in LDCT images. Transformer can capture global context information in an attentional manner to create distant dependencies on targets and extract more powerful features. In this paper, we reduce the impact of image errors on the ability to retain detailed information and improve the noise suppression performance by fully mining the distribution characteristics of image information.

METHODS

This paper proposed an LDCT noise and artifact suppressing network based on Swin Transformer. The network includes a noise extraction sub-network and a noise removal sub-network. The noise extraction and removal capability are improved using a coarse extraction network of high-frequency features based on full convolution. The noise removal sub-network improves the network's ability to extract relevant image features by using a Swin Transformer with a shift window as an encoder-decoder and skip connections for global feature fusion. Also, the perceptual field is extended by extracting multi-scale features of the images to recover the spatial resolution of the feature maps. The network uses a loss constraint with a combination of L1 and MS-SSIM to improve and ensure the stability and denoising effect of the network.

RESULTS

The denoising ability and clinical applicability of the methods were tested using clinical datasets. Compared with DnCNN, RED-CNN, CBDNet and TSCN, the STEDNet method shows a better denoising effect on RMSE and PSNR. The STEDNet method effectively removes image noise and preserves the image structure to the maximum extent, making the reconstructed image closest to the NDCT image. The subjective and objective analysis of several sets of experiments shows that the method in this paper can effectively maintain the structure, edges, and textures of the denoised images while having good noise suppression performance. In the real data evaluation, the RMSE of this method is reduced by 18.82%, 15.15%, 2.25%, and 1.10% on average compared with DnCNN, RED-CNN, CBDNet, and TSCNN, respectively. The average improvement of PSNR is 9.53%, 7.33%, 2.65%, and 3.69%, respectively.

CONCLUSIONS

This paper proposed a LDCT image denoising algorithm based on end-to-end training. The method in this paper can effectively improve the diagnostic performance of CT images by constraining the details of the images and restoring the LDCT image structure. The problem of increased noise and artifacts in CT images can be solved while maintaining the integrity of CT image tissue structure and pathological information. Compared with other algorithms, this method has better denoising effects both quantitatively and qualitatively.

Detector shifting and deep learning based ring artifact correction method for low-dose CT

BACKGROUND

In x-ray computed tomography (CT), the gain inconsistency of detector units leads to ring artifacts in the reconstructed images, seriously destroys the image structure, and is not conducive to image recognition. In addition, to reduce radiation dose and scanning time, especially photon counting CT, low-dose CT is required, so it is important to reduce the noise and suppress ring artifacts in low-dose CT images simultaneously.

PURPOSE

Deep learning is an effective method to suppress ring artifacts, but there are still residual artifacts in corrected images. And the feature recognition ability of the network for ring artifacts decreases due to the effect of noise in the low-dose CT images. In this paper, a method is proposed to achieve noise reduction and ring artifact removal simultaneously.

METHODS

To solve these problems, we propose a ring artifact correction method for low-dose CT based on detector shifting and deep learning in this paper. Firstly, at the CT scanning stage, the detector horizontally shifts randomly at each projection to alleviate the ring artifacts as front processing. Thus, the ring artifacts are transformed into dispersed noise in front processed images. Secondly, deep learning is used for dispersed noise and statistical noise reduction.

RESULTS

Both simulation and real data experiments are conducted to evaluate the proposed method. Compared to other methods, the results show that the proposed method in this paper has better effect on removing ring artifacts in the low-dose CT images. Specifically, the RMSEs and SSIMs of the two sets of simulated and experiment data are better compared to the raw images significantly.

CONCLUSIONS

The method proposed in this paper combines detector shifting and deep learning to remove ring artifacts and statistical noise simultaneously. The results show that the proposed method is able to get better performance.

An application of deep dual convolutional neural network for enhanced medical image denoising

This work investigates the medical image denoising (MID) application of the dual denoising network (DudeNet) model for chest X-ray (CXR). The DudeNet model comprises four components: a feature extraction block with a sparse mechanism, an enhancement block, a compression block, and a reconstruction block. The developed model uses residual learning to boost denoising performance and batch normalization to accelerate the training process. The name proposed for this model is dual convolutional medical image-enhanced denoising network (DCMIEDNet). The peak signal-to-noise ratio (PSNR) and structure similarity index measurement (SSIM) are used to assess the MID performance for five different additive white Gaussian noise (AWGN) levels of σ = 15, 25, 40, 50, and 60 in CXR images. Presented investigations revealed that the PSNR and SSIM offered by DCMIEDNet are better than several popular state-of-the-art models such as block matching and 3D filtering, denoising convolutional neural network, and feature-guided denoising convolutional neural network. In addition, it is also superior to the recently reported MID models like deep convolutional neural network with residual learning, real-valued medical image denoising network, and complex-valued medical image denoising network. Therefore, based on the presented experiments, it is concluded that applying the DudeNet methodology for DCMIEDNet promises to be quite helpful for physicians.

Systematic Review of Advanced AI Methods for Improving Healthcare Data Quality in Post COVID-19 Era

At the beginning of the COVID-19 pandemic, there was significant hype about the potential impact of artificial intelligence (AI) tools in combatting COVID-19 on diagnosis, prognosis, or surveillance. However, AI tools have not yet been widely successful. One of the key reason is the COVID-19 pandemic has demanded faster real-time development of AI-driven clinical and health support tools, including rapid data collection, algorithm development, validation, and deployment. However, there was not enough time for proper data quality control. Learning from the hard lessons in COVID-19, we summarize the important health data quality challenges during COVID-19 pandemic such as lack of data standardization, missing data, tabulation errors, and noise and artifact. Then we conduct a systematic investigation of computational methods that address these issues, including emerging novel advanced AI data quality control methods that achieve better data quality outcomes and, in some cases, simplify or automate the data cleaning process. We hope this article can assist healthcare community to improve health data quality going forward with novel AI development.

Domain-adaptive denoising network for low-dose CT via noise estimation and transfer learning

BACKGROUND

In recent years, low-dose computed tomography (LDCT) has played an important role in the diagnosis CT to reduce the potential adverse effects of X-ray radiation on patients, while maintaining the same diagnostic image quality.

PURPOSE

Deep learning (DL)-based methods have played an increasingly important role in the field of LDCT imaging. However, its performance is highly dependent on the consistency of feature distributions between training data and test data. Due to patient's breathing movements during data acquisition, the paired LDCT and normal dose CT images are difficult to obtain from realistic imaging scenarios. Moreover, LDCT images from simulation or clinical CT examination often have different feature distributions due to the pollution by different amounts and types of image noises. If a network model trained with a simulated dataset is used to directly test clinical patients' LDCT data, its denoising performance may be degraded. Based on this, we propose a novel domain-adaptive denoising network (DADN) via noise estimation and transfer learning to resolve the out-of-distribution problem in LDCT imaging.

METHODS

To overcome the previous adaptation issue, a novel network model consisting of a reconstruction network and a noise estimation network was designed. The noise estimation network based on a double branch structure is used for image noise extraction and adaptation. Meanwhile, the U-Net-based reconstruction network uses several spatially adaptive normalization modules to fuse multi-scale noise input. Moreover, to facilitate the adaptation of the proposed DADN network to new imaging scenarios, we set a two-stage network training plan. In the first stage, the public simulated dataset is used for training. In the second transfer training stage, we will continue to fine-tune the network model with a torso phantom dataset, while some parameters are frozen. The main reason using the two-stage training scheme is based on the fact that the feature distribution of image content from the public dataset is complex and diverse, whereas the feature distribution of noise pattern from the torso phantom dataset is closer to realistic imaging scenarios.

RESULTS

In an evaluation study, the trained DADN model is applied to both the public and clinical patient LDCT datasets. Through the comparison of visual inspection and quantitative results, it is shown that the proposed DADN network model can perform well in terms of noise and artifact suppression, while effectively preserving image contrast and details.

CONCLUSIONS

In this paper, we have proposed a new DL network to overcome the domain adaptation problem in LDCT image denoising. Moreover, the results demonstrate the feasibility and effectiveness of the application of our proposed DADN network model as a new DL-based LDCT image denoising method.

Compound feature attention network with edge enhancement for low-dose CT denoising

BACKGROUND

Low-dose CT (LDCT) images usually contain serious noise and artifacts, which weaken the readability of the image.

OBJECTIVE

To solve this problem, we propose a compound feature attention network with edge enhancement for LDCT denoising (CFAN-Net), which consists of an edge-enhanced module and a proposed compound feature attention block (CFAB).

METHODS

The edge enhancement module extracts edge details with the trainable Sobel convolution. CFAB consists of an interactive feature learning module (IFLM), a multi-scale feature fusion module (MFFM), and a joint attention module (JAB), which removes noise from LDCT images in a coarse-to-fine manner. First, in IFLM, the noise is initially removed by cross-latitude interactive judgment learning. Second, in MFFM, multi-scale and pixel attention are integrated to explore fine noise removal. Finally, in JAB, we focus on key information, extract useful features, and improve the efficiency of network learning. To construct a high-quality image, we repeat the above operation by cascading CFAB.

RESULTS

By applying CFAN-Net to process the 2016 NIH AAPM-Mayo LDCT challenge test dataset, experiments show that the peak signal-to-noise ratio value is 33.9692 and the structural similarity value is 0.9198.

CONCLUSIONS

Compared with several existing LDCT denoising algorithms, CFAN-Net effectively preserves the texture of CT images while removing noise and artifacts.

Artificial intelligence-based full aortic CT angiography imaging with ultra-low-dose contrast medium: a preliminary study

OBJECTIVES

To further reduce the contrast medium (CM) dose of full aortic CT angiography (ACTA) imaging using the augmented cycle-consistent adversarial framework (Au-CycleGAN) algorithm.

METHODS

We prospectively enrolled 150 consecutive patients with suspected aortic disease. All received ACTA scans of ultra-low-dose CM (ULDCM) protocol and low-dose CM (LDCM) protocol. These data were randomly assigned to the training datasets (n = 100) and the validation datasets (n = 50). The ULDCM images were reconstructed by the Au-CycleGAN algorithm. Then, the AI-based ULDCM images were compared with LDCM images in terms of image quality and diagnostic accuracy.

RESULTS

The mean image quality score of each location in the AI-based ULDCM group was higher than that in the ULDCM group but a little lower than that in the LDCM group (all p < 0.05). All AI-based ULDCM images met the diagnostic requirements (score ≥ 3). Except for the image noise, the AI-based ULDCM images had higher attenuation value than the ULDCM and LDCM images as well as higher SNR and CNR in all locations of the aorta analyzed (all p < 0.05). Similar results were also seen in obese patients (BMI > 25, all p < 0.05). Using the findings of LDCM images as the reference, the AI-based ULDCM images showed good diagnostic parameters and no significant differences in any of the analyzed aortic disease diagnoses (all K-values > 0.80, p < 0.05).

CONCLUSIONS

The required dose of CM for full ACTA imaging can be reduced to one-third of the CM dose of the LDCM protocol while maintaining image quality and diagnostic accuracy using the Au-CycleGAN algorithm.

KEY POINTS

• The required dose of contrast medium (CM) for full ACTA imaging can be reduced to one-third of the CM dose of the low-dose contrast medium (LDCM) protocol using the Au-CycleGAN algorithm. • Except for the image noise, the AI-based ultra-low-dose contrast medium (ULDCM) images had better quantitative image quality parameters than the ULDCM and LDCM images. • No significant diagnostic differences were noted between the AI-based ULDCM and LDCM images regarding all the analyzed aortic disease diagnoses.

Improved deep convolutional dictionary learning with no noise parameter for low-dose CT image processing

BACKGROUND

Low-Dose computed tomography (LDCT) reduces radiation damage to patients, however, the reconstructed images contain severe noise, which affects doctors' diagnosis of the disease. The convolutional dictionary learning has the advantage of the shift-invariant property. The deep convolutional dictionary learning algorithm (DCDicL) combines deep learning and convolutional dictionary learning, which has great suppression effects on Gaussian noise. However, applying DCDicL to LDCT images cannot get satisfactory results.

OBJECTIVE

To address this challenge, this study proposes and tests an improved deep convolutional dictionary learning algorithm for LDCT image processing and denoising.

METHODS

First, we use a modified DCDicL algorithm to improve the input network and make it do not need to input noise intensity parameter. Second, we use DenseNet121 to replace the shallow convolutional network to learn the prior on the convolutional dictionary, which can obtain more accurate convolutional dictionary. Last, in the loss function, we add MSSIM to enhance the detail retention ability of the model.

RESULTS

The experimental results on the Mayo dataset show that the proposed model obtained an average value of 35.2975 dB in PSNR, which is 0.2954 -1.0573 dB higher than the mainstream LDCT algorithm, indicating the excellent denoising performance.

CONCLUSION

The study demonstrates that the proposed new algorithm can effectively improve the quality of LDCT images acquired in the clinical practice.

PerceptFlow: Real-Time Ultrafast Doppler Image Enhancement Using Deep Convolutional Neural Network and Perceptual Loss

Ultrafast ultrasound is an emerging imaging modality derived from standard medical ultrasound. It allows for a high spatial resolution of 100 μm and a temporal resolution in the millisecond range with techniques such as ultrafast Doppler imaging. Ultrafast Doppler imaging has become a priceless tool for neuroscience, especially for visualizing functional vascular structures and navigating the brain in real time. Yet, the quality of a Doppler image strongly depends on experimental conditions and is easily subject to artifacts and deterioration, especially with transcranial imaging, which often comes at the cost of higher noise and lower sensitivity to small blood vessels. A common solution to better visualize brain vasculature is either accumulating more information, integrating the image over several seconds or using standard filter-based enhancement techniques, which often over-smooth the image, thus failing both to preserve sharp details and to improve our perception of the vasculature. In this study we propose combining the standard Doppler accumulation process with a real-time enhancement strategy, based on deep-learning techniques, using perceptual loss (PerceptFlow). With our perceptual approach, we bypass the need for long integration times to enhance Doppler images. We applied and evaluated our proposed method on transcranial Doppler images of mouse brains, outperforming state-of-the-art filters. We found that, in comparison to standard filters such as the Gaussian filter (GF) and block-matching and 3-D filtering (BM3D), PerceptFlow was capable of reducing background noise with a significant increase in contrast and contrast-to-noise ratio, as well as better preserving details without compromising spatial resolution.

A densely connected LDCT image denoising network based on dual-edge extraction and multi-scale attention under compound loss

BACKGROUND

Low dose computed tomography (LDCT) uses lower radiation dose, but the reconstructed images contain higher noise that can have negative impact in disease diagnosis. Although deep learning with the edge extraction operators reserves edge information well, only applying the edge extraction operators to input LDCT images does not yield overall satisfactory results.

OBJECTIVE

To improve LDCT images quality, this study proposes and tests a dual edge extraction multi-scale attention mechanism convolution neural network (DEMACNN) based on a compound loss.

METHODS

The network uses edge extraction operators to extract edge information from both the input images and the feature maps in the network, improving the utilization of the edge operators and retaining the images edge information. The feature enhancement block is constructed by fusing the attention mechanism and multi-scale module, enhancing effective information, while suppressing useless information. The residual learning method is used to learn the network, improving the performance of the network, and solving the problem of gradient disappearance. Except for the network structure, a compound loss function, which consists of the MSE loss, the proposed joint total variation loss, and the edge loss, is proposed to enhance the denoising ability of the network and reserve the edge of images.

RESULTS

Compared with other advanced methods (REDCNN, CT-former and EDCNN), the proposed new network achieves the best PSNR and SSIM values in LDCT images of the abdomen, which are 33.3486 and 0.9104, respectively. In addition, the new network also performs well on head and chest image data.

CONCLUSION

The experimental results demonstrate that the proposed new network structure and denoising algorithm not only effectively removes the noise in LDCT images, but also protects the edges and details of the images well.

Multislice input for 2D and 3D residual convolutional neural network noise reduction in CT

Purpose

Deep convolutional neural network (CNN)-based methods are increasingly used for reducing image noise in computed tomography (CT). Current attempts at CNN denoising are based on 2D or 3D CNN models with a single- or multiple-slice input. Our work aims to investigate if the multiple-slice input improves the denoising performance compared with the single-slice input and if a 3D network architecture is better than a 2D version at utilizing the multislice input.

Approach

Two categories of network architectures can be used for the multislice input. First, multislice images can be stacked channel-wise as the multichannel input to a 2D CNN model. Second, multislice images can be employed as the 3D volumetric input to a 3D CNN model, in which the 3D convolution layers are adopted. We make performance comparisons among 2D CNN models with one, three, and seven input slices and two versions of 3D CNN models with seven input slices and one or three output slices. Evaluation was performed on liver CT images using three quantitative metrics with full-dose images as reference. Visual assessment was made by an experienced radiologist.

Results

When the input channels of the 2D CNN model increases from one to three to seven, a trend of improved performance was observed. Comparing the three models with the seven-slice input, the 3D CNN model with a one-slice output outperforms the other models in terms of noise texture and homogeneity in liver parenchyma as well as subjective visualization of vessels.

Conclusions

We conclude the that multislice input is an effective strategy for improving performance for 2D deep CNN denoising models. The pure 3D CNN model tends to have a better performance than the other models in terms of continuity across axial slices, but the difference was not significant compared with the 2D CNN model with the same number of slices as the input.

TIME-Net: Transformer-Integrated Multi-Encoder Network for limited-angle artifact removal in dual-energy CBCT

Dual-energy cone-beam computed tomography (DE-CBCT) is a promising imaging technique with foreseeable clinical applications. DE-CBCT images acquired with two different spectra can provide material-specific information. Meanwhile, the anatomical consistency and energy-domain correlation result in significant information redundancy, which could be exploited to improve image quality. In this context, this paper develops the Transformer-Integrated Multi-Encoder Network (TIME-Net) for DE-CBCT to remove the limited-angle artifacts. TIME-Net comprises three encoders (image encoder, prior encoder, and transformer encoder), two decoders (low- and high-energy decoders), and one feature fusion module. Three encoders extract various features for image restoration. The feature fusion module compresses these features into more compact shared features and feeds them to the decoders. Two decoders perform differential learning for DE-CBCT images. By design, TIME-Net could obtain high-quality DE-CBCT images using two complementary quarter-scans, holding great potential to reduce radiation dose and shorten the acquisition time. Qualitative and quantitative analyses based on simulated data and real rat data have demonstrated the promising performance of TIME-Net in artifact removal, subtle structure restoration, and reconstruction accuracy preservation. Two clinical applications, virtual non-contrast (VNC) imaging and iodine quantification, have proved the potential utility of the DE-CBCT images provided by TIME-Net.

A region-adaptive non-local denoising algorithm for low-dose computed tomography images

Low-dose computed tomography (LDCT) can effectively reduce radiation exposure in patients. However, with such dose reductions, large increases in speckled noise and streak artifacts occur, resulting in seriously degraded reconstructed images. The non-local means (NLM) method has shown potential for improving the quality of LDCT images. In the NLM method, similar blocks are obtained using fixed directions over a fixed range. However, the denoising performance of this method is limited. In this paper, a region-adaptive NLM method is proposed for LDCT image denoising. In the proposed method, pixels are classified into different regions according to the edge information of the image. Based on the classification results, the adaptive searching window, block size and filter smoothing parameter could be modified in different regions. Furthermore, the candidate pixels in the searching window could be filtered based on the classification results. In addition, the filter parameter could be adjusted adaptively based on intuitionistic fuzzy divergence (IFD). The experimental results showed that the proposed method performed better in LDCT image denoising than several of the related denoising methods in terms of numerical results and visual quality.

X-ray CT image denoising with MINF: A modularized iterative network framework for data from multiple dose levels

In clinical applications, multi-dose scan protocols will cause the noise levels of computed tomography (CT) images to fluctuate widely. The popular low-dose CT (LDCT) denoising network outputs denoised images through an end-to-end mapping between an LDCT image and its corresponding ground truth. The limitation of this method is that the reduced noise level of the image may not meet the diagnostic needs of doctors. To establish a denoising model adapted to the multi-noise levels robustness, we proposed a novel and efficient modularized iterative network framework (MINF) to learn the feature of the original LDCT and the outputs of the previous modules, which can be reused in each following module. The proposed network can achieve the goal of gradual denoising, outputting clinical images with different denoising levels, and providing the reviewing physicians with increased confidence in their diagnosis. Moreover, a multi-scale convolutional neural network (MCNN) module is designed to extract as much feature information as possible during the network's training. Extensive experiments on public and private clinical datasets were carried out, and comparisons with several state-of-the-art methods show that the proposed method can achieve satisfactory results for noise suppression of LDCT images. In further comparisons with modularized adaptive processing neural network (MAP-NN), the proposed network shows superior step-by-step or gradual denoising performance. Considering the high quality of gradual denoising results, the proposed method can obtain satisfactory performance in terms of image contrast and detail protection as the level of denoising increases, which shows its potential to be suitable for a multi-dose levels denoising task.

Solving Inverse Problems With Deep Neural Networks - Robustness Included?

In the past five years, deep learning methods have become state-of-the-art in solving various inverse problems. Before such approaches can find application in safety-critical fields, a verification of their reliability appears mandatory. Recent works have pointed out instabilities of deep neural networks for several image reconstruction tasks. In analogy to adversarial attacks in classification, it was shown that slight distortions in the input domain may cause severe artifacts. The present article sheds new light on this concern, by conducting an extensive study of the robustness of deep-learning-based algorithms for solving underdetermined inverse problems. This covers compressed sensing with Gaussian measurements as well as image recovery from Fourier and Radon measurements, including a real-world scenario for magnetic resonance imaging (using the NYU-fastMRI dataset). Our main focus is on computing adversarial perturbations of the measurements that maximize the reconstruction error. A distinctive feature of our approach is the quantitative and qualitative comparison with total-variation minimization, which serves as a provably robust reference method. In contrast to previous findings, our results reveal that standard end-to-end network architectures are not only resilient against statistical noise, but also against adversarial perturbations. All considered networks are trained by common deep learning techniques, without sophisticated defense strategies.

Motion artifact removal in coronary CT angiography based on generative adversarial networks

OBJECTIVES

Coronary motion artifacts affect the diagnostic accuracy of coronary CT angiography (CCTA), especially in the mid right coronary artery (mRCA). The purpose is to correct CCTA motion artifacts of the mRCA using a GAN (generative adversarial network).

METHODS

We included 313 patients with CCTA scans, who had paired motion-affected and motion-free reference images at different R-R interval phases in the same cardiac cycle and included another 53 CCTA cases with invasive coronary angiography (ICA) comparison. Pix2pix, an image-to-image conversion GAN, was trained by the motion-affected and motion-free reference pairs to generate motion-free images from the motion-affected images. Peak signal-to-noise ratio (PSNR), structural similarity (SSIM), Dice similarity coefficient (DSC), and Hausdorff distance (HD) were calculated to evaluate the image quality of GAN-generated images.

RESULTS

At the image level, the median of PSNR, SSIM, DSC, and HD of GAN-generated images were 26.1 (interquartile: 24.4-27.5), 0.860 (0.830-0.882), 0.783 (0.714-0.825), and 4.47 (3.00-4.47), respectively, significantly better than the motion-affected images (p < 0.001). At the patient level, the image quality results were similar. GAN-generated images improved the motion artifact alleviation score (4 vs. 1, p < 0.001) and overall image quality score (4 vs. 1, p < 0.001) than those of the motion-affected images. In patients with ICA comparison, GAN-generated images achieved accuracy of 81%, 85%, and 70% in identifying no, < 50%, and ≥ 50% stenosis, respectively, higher than 66%, 72%, and 68% for the motion-affected images.

CONCLUSION

Generative adversarial network-generated CCTA images greatly improved the image quality and diagnostic accuracy compared to motion-affected images.

KEY POINTS

• A generative adversarial network greatly reduced motion artifacts in coronary CT angiography and improved image quality. • GAN-generated images improved diagnosis accuracy of identifying no, < 50%, and ≥ 50% stenosis.

Total-body low-dose CT image denoising using a prior knowledge transfer technique with a contrastive regularization mechanism

PURPOSE

Reducing the radiation exposure experienced by patients in total-body computed tomography (CT) imaging has attracted extensive attention in the medical imaging community. A low radiation dose may result in increased noise and artifacts that greatly affect the subsequent clinical diagnosis. To obtain high-quality total-body low-dose CT (LDCT) images, previous deep learning-based research works developed various network architectures. However, most of these methods only employ normal-dose CT (NDCT) images as ground truths to guide the training process of the constructed denoising network. As a result of this simple restriction, the reconstructed images tend to lose favorable image details and easily generate oversmoothed textures. This study explores how to better utilize the information contained in the feature spaces of NDCT images to guide the LDCT image reconstruction process and achieve high-quality results.

METHODS

We propose a novel intratask knowledge transfer (KT) method that leverages the knowledge distilled from NDCT images as an auxiliary component of the LDCT image reconstruction process. Our proposed architecture is named the teacher-student consistency network (TSC-Net), which consists of teacher and student networks with identical architectures. By employing the designed KT loss, the student network is encouraged to emulate the teacher network in the representation space and gain robust prior content. In addition, to further exploit the information contained in CT scans, a contrastive regularization mechanism (CRM) built upon contrastive learning is introduced. The CRM aims to minimize and maximize the L2 distances from the predicted CT images to the NDCT samples and to the LDCT samples in the latent space, respectively. Moreover, based on attention and the deformable convolution approach, we design a dynamic enhancement module (DEM) to improve the network capability to transform input information flows.

RESULTS

By conducting ablation studies, we prove the effectiveness of the proposed KT loss, CRM, and DEM. Extensive experimental results demonstrate that the TSC-Net outperforms the state-of-the-art methods in both quantitative and qualitative evaluations. Additionally, the excellent results obtained for clinical readings also prove that our proposed method can reconstruct high-quality CT images for clinical applications.

CONCLUSIONS

Based on the experimental results and clinical readings, the TSC-Net has better performance than other approaches. In our future work, we may explore the reconstruction of LDCT images by fusing the positron emission tomography (PET) and CT modalities to further improve the visual quality of the reconstructed CT images.

Computed Tomography slice interpolation in the longitudinal direction based on deep learning techniques: To reduce slice thickness or slice increment without dose increase

Large slice thickness or slice increment causes information insufficiency of Computed Tomography (CT) data in the longitudinal direction, which degrades the quality of CT-based diagnosis. Traditional approaches such as high-resolution computed tomography (HRCT) and linear interpolation can solve this problem. However, HRCT suffers from dose increase, and linear interpolation causes artifacts. In this study, we propose a deep-learning-based approach to reconstruct densely sliced CT from sparsely sliced CT data without any dose increase. The proposed method reconstructs CT images from neighboring slices using a U-net architecture. To prevent multiple reconstructed slices from influencing one another, we propose a parallel architecture in which multiple U-net architectures work independently. Moreover, for a specific organ (i.e., the liver), we propose a range-clip technique to improve reconstruction quality, which enhances the learning of CT values within this organ by enlarging the range of the training data. CT data from 130 patients were collected, with 80% used for training and the remaining 20% used for testing. Experiments showed that our parallel U-net architecture reduced the mean absolute error of CT values in the reconstructed slices by 22.05%, and also reduced the incidence of artifacts around the boundaries of target organs, compared with linear interpolation. Further improvements of 15.12%, 11.04%, 10.94%, and 10.63% were achieved for the liver, left kidney, right kidney, and stomach, respectively, using the proposed range-clip algorithm. Also, we compared the proposed architecture with original U-net method, and the experimental results demonstrated the superiority of our approach.

Structure-preserved meta-learning uniting network for improving low-dose CT quality

Objective.Deep neural network (DNN) based methods have shown promising performances for low-dose computed tomography (LDCT) imaging. However, most of the DNN-based methods are trained on simulated labeled datasets, and the low-dose simulation algorithms are usually designed based on simple statistical models which deviate from the real clinical scenarios, which could lead to issues of overfitting, instability and poor robustness. To address these issues, in this work, we present a structure-preserved meta-learning uniting network (shorten as 'SMU-Net') to suppress noise-induced artifacts and preserve structure details in the unlabeled LDCT imaging task in real scenarios.Approach.Specifically, the presented SMU-Net contains two networks, i.e., teacher network and student network. The teacher network is trained on simulated labeled dataset and then helps the student network train with the unlabeled LDCT images via the meta-learning strategy. The student network is trained on real LDCT dataset with the pseudo-labels generated by the teacher network. Moreover, the student network adopts the Co-teaching strategy to improve the robustness of the presented SMU-Net.Main results.We validate the proposed SMU-Net method on three public datasets and one real low-dose dataset. The visual image results indicate that the proposed SMU-Net has superior performance on reducing noise-induced artifacts and preserving structure details. And the quantitative results exhibit that the presented SMU-Net method generally obtains the highest signal-to-noise ratio (PSNR), the highest structural similarity index measurement (SSIM), and the lowest root-mean-square error (RMSE) values or the lowest natural image quality evaluator (NIQE) scores.Significance.We propose a meta learning strategy to obtain high-quality CT images in the LDCT imaging task, which is designed to take advantage of unlabeled CT images to promote the reconstruction performance in the LDCT environments.

Deep learning tomographic reconstruction through hierarchical decomposition of domain transforms

Deep learning (DL) has shown unprecedented performance for many image analysis and image enhancement tasks. Yet, solving large-scale inverse problems like tomographic reconstruction remains challenging for DL. These problems involve non-local and space-variant integral transforms between the input and output domains, for which no efficient neural network models are readily available. A prior attempt to solve tomographic reconstruction problems with supervised learning relied on a brute-force fully connected network and only allowed reconstruction with a 128⁴ system matrix size. This cannot practically scale to realistic data sizes such as 512⁴ and 512⁶ for three-dimensional datasets. Here we present a novel framework to solve such problems with DL by casting the original problem as a continuum of intermediate representations between the input and output domains. The original problem is broken down into a sequence of simpler transformations that can be well mapped onto an efficient hierarchical network architecture, with exponentially fewer parameters than a fully connected network would need. We applied the approach to computed tomography (CT) image reconstruction for a 512⁴ system matrix size. This work introduces a new kind of data-driven DL solver for full-size CT reconstruction without relying on the structure of direct (analytical) or iterative (numerical) inversion techniques. This work presents a feasibility demonstration of full-scale learnt reconstruction, whereas more developments will be needed to demonstrate superiority relative to traditional reconstruction approaches. The proposed approach is also extendable to other imaging problems such as emission and magnetic resonance reconstruction. More broadly, hierarchical DL opens the door to a new class of solvers for general inverse problems, which could potentially lead to improved signal-to-noise ratio, spatial resolution and computational efficiency in various areas.

Deep learning-based noise reduction for coronary CT angiography: using four-dimensional noise-reduction images as the ground truth

Background

To assess low-contrast areas such as plaque and coronary artery stenosis, coronary computed tomography angiography (CCTA) needs to provide images with lower noise without increasing radiation doses.

Purpose

To develop a deep learning-based noise-reduction method for CCTA using four-dimensional noise reduction (4DNR) as the ground truth for supervised learning.

Material and Methods

\We retrospectively collected 100 retrospective ECG-gated CCTAs. We created 4DNR images using non-rigid registration and weighted averaging three timeline CCTA volumetric data with intervals of 50 ms in the mid-diastolic phase. Our method set the original reconstructed image as the input and the 4DNR as the target image and obtained the noise-reduced image via residual learning. We evaluated the objective image quality of the original and deep learning-based noise-reduction (DLNR) images based on the image noise of the aorta and the contrast-to-noise ratio (CNR) of the coronary arteries. Further, a board-certified radiologist evaluated the blurring of several heart structures using a 5-point Likert scale subjectively and assigned a coronary artery disease reporting and data system (CAD-RADS) category independently.

Results

DLNR CCTAs showed 64.5% lower image noise ( P < 0.001) and achieved a 2.9 times higher CNR of coronary arteries than that in original images, without significant blurring in subjective comparison ( P > 0.1). The intra-observer agreement of CAD-RADS in the DLNR image was excellent (0.87, 95% confidence interval = 0.77–0.99) with original CCTAs.

Conclusion

Our DLNR method supervised by 4DNR significantly reduced the image noise of CCTAs without affecting the assessment of coronary stenosis.

Segmentation-guided Denoising Network for Low-dose CT Imaging

BACKGROUND

To reduce radiation exposure and improve diagnosis in low-dose computed tomography, several deep learning (DL)-based image denoising methods have been proposed to suppress noise and artifacts over the past few years. However, most of them seek an objective data distribution approximating the gold standard and neglect structural semantic preservation. Moreover, the numerical response in CT images presents substantial regional anatomical differences among tissues in terms of X-ray absorbency.

METHODS

In this paper, we introduce structural semantic information for low-dose CT imaging. First, the regional segmentation prior to low-dose CT can guide the denoising process. Second the structural semantical results can be considered as evaluation metrics on the estimated normal-dose CT images. Then, a semantic feature transform is engaged to combine the semantic and image features on a semantic fusion module. In addition, the structural semantic loss function is introduced to measure the segmentation difference.

RESULTS

Experiments are conducted on clinical abdomen data obtained from a clinical hospital, and the semantic labels consist of subcutaneous fat, muscle and visceral fat associated with body physical evaluation. Compared with other DL-based methods, the proposed method achieves better performance on quantitative metrics and better semantic evaluation.

CONCLUSIONS

The quantitative experimental results demonstrate the promising performance of the proposed methods in noise reduction and structural semantic preservation. While, the proposed method may suffer from several limitations on abnormalities, unknown noise and different manufacturers. In the future, the proposed method will be further explored, and wider applications in PET/CT and PET/MR will be sought.

RAD-UNet: a Residual, Attention-Based, Dense UNet for CT Sparse Reconstruction

To suppress the streak artifacts in images reconstructed from sparse-view projections in computed tomography (CT), a residual, attention-based, dense UNet (RAD-UNet) deep network is proposed to achieve accurate sparse reconstruction. The filtered back projection (FBP) algorithm is used to reconstruct the CT image with streak artifacts from sparse-view projections. Then, the image is processed by the RAD-UNet to suppress streak artifacts and obtain high-quality CT image. Those images with streak artifacts are used as the input of the RAD-UNet, and the output-label images are the corresponding high-quality images. Through training via the large-scale training data, the RAD-UNet can obtain the capability of suppressing streak artifacts. This network combines residual connection, attention mechanism, dense connection and perceptual loss. This network can improve the nonlinear fitting capability and the performance of suppressing streak artifacts. The experimental results show that the RAD-UNet can improve the reconstruction accuracy compared with three existing representative deep networks. It may not only suppress streak artifacts but also better preserve image details. The proposed networks may be readily applied to other image processing tasks including image denoising, image deblurring, and image super-resolution.

Learning low-dose CT degradation from unpaired data with flow-based model

BACKGROUND

There has been growing interest in low-dose computed tomography (LDCT) for reducing the X-ray radiation to patients. However, LDCT always suffers from complex noise in reconstructed images. Although deep learning-based methods have shown their strong performance in LDCT denoising, most of them require a large number of paired training data of normal-dose CT (NDCT) images and LDCT images, which are hard to acquire in the clinic. Lack of paired training data significantly undermines the practicability of supervised deep learning-based methods. To alleviate this problem, unsupervised or weakly supervised deep learning-based methods are required.

PURPOSE

We aimed to propose a method that achieves LDCT denoising without training pairs. Specifically, we first trained a neural network in a weakly supervised manner to simulate LDCT images from NDCT images. Then, simulated training pairs could be used for supervised deep denoising networks.

METHODS

We proposed a weakly supervised method to learn the degradation of LDCT from unpaired LDCT and NDCT images. Concretely, LDCT and normal-dose images were fed into one shared flow-based model and projected to the latent space. Then, the degradation between low-dose and normal-dose images was modeled in the latent space. Finally, the model was trained by minimizing the negative log-likelihood loss with no requirement of paired training data. After training, an NDCT image can be input to the trained flow-based model to generate the corresponding LDCT image. The simulated image pairs of NDCT and LDCT can be further used to train supervised denoising neural networks for test.

RESULTS

Our method achieved much better performance on LDCT image simulation compared with the most widely used image-to-image translation method, CycleGAN, according to the radial noise power spectrum. The simulated image pairs could be used for any supervised LDCT denoising neural networks. We validated the effectiveness of our generated image pairs on a classic convolutional neural network, REDCNN, and a novel transformer-based model, TransCT. Our method achieved mean peak signal-to-noise ratio (PSNR) of 24.43dB, mean structural similarity (SSIM) of 0.785 on an abdomen CT dataset, mean PSNR of 33.88dB, mean SSIM of 0.797 on a chest CT dataset, which outperformed several traditional CT denoising methods, the same network trained by CycleGAN-generated data, and a novel transfer learning method. Besides, our method was on par with the supervised networks in terms of visual effects.

CONCLUSION

We proposed a flow-based method to learn LDCT degradation from only unpaired training data. It achieved impressive performance on LDCT synthesis. Next, we could train neural networks with the generated paired data for LDCT denoising. The denoising results are better than traditional and weakly supervised methods, comparable to supervised deep learning methods.

Noise Characteristics Modeled Unsupervised Network for Robust CT Image Reconstruction

Deep learning (DL)-based methods show great potential in computed tomography (CT) imaging field. The DL-based reconstruction methods are usually evaluated on the training and testing datasets which are obtained from the same distribution, i.e., the same CT scan protocol (i.e., the region setting, kVp, mAs, etc.). In this work, we focus on analyzing the robustness of the DL-based methods against protocol-specific distribution shifts (i.e., the training and testing datasets are from different region settings, different kVp settings, or different mAs settings, respectively). The results show that the DL-based reconstruction methods are sensitive to the protocol-specific perturbations which can be attributed to the noise distribution shift between the training and testing datasets. Based on these findings, we presented a low-dose CT reconstruction method using an unsupervised strategy with the consideration of noise distribution to address the issue of protocol-specific perturbations. Specifically, unpaired sinogram data is enrolled into the network training, which represents unique information for specific imaging protocol, and a Gaussian mixture model (GMM) is introduced to characterize the noise distribution in CT images. It can be termed as GMM based unsupervised CT reconstruction network (GMM-unNet) method. Moreover, an expectation-maximization algorithm is designed to optimize the presented GMM-unNet method. Extensive experiments are performed on three datasets from different scan protocols, which demonstrate that the presented GMM-unNet method outperforms the competing methods both qualitatively and quantitatively.

DuDoUFNet: Dual-Domain Under-to-Fully-Complete Progressive Restoration Network for Simultaneous Metal Artifact Reduction and Low-Dose CT Reconstruction

To reduce the potential risk of radiation to the patient, low-dose computed tomography (LDCT) has been widely adopted in clinical practice for reconstructing cross-sectional images using sinograms with reduced x-ray flux. The LDCT image quality is often degraded by different levels of noise depending on the low-dose protocols. The image quality will be further degraded when the patient has metallic implants, where the image suffers from additional streak artifacts along with further amplified noise levels, thus affecting the medical diagnosis and other CT-related applications. Previous studies mainly focused either on denoising LDCT without considering metallic implants or full-dose CT metal artifact reduction (MAR). Directly applying previous LDCT or MAR approaches to the issue of simultaneous metal artifact reduction and low-dose CT (MARLD) may yield sub-optimal reconstruction results. In this work, we develop a dual-domain under-to-fully-complete progressive restoration network, called DuDoUFNet, for MARLD. Our DuDoUFNet aims to reconstruct images with substantially reduced noise and artifact by progressive sinogram to image domain restoration with a two-stage progressive restoration network design. Our experimental results demonstrate that our method can provide high-quality reconstruction, superior to previous LDCT and MAR methods under various low-dose and metal settings.

PET and MRI image fusion based on a dense convolutional network with dual attention

The fusion techniques of different modalities in medical images, e.g., Positron Emission Tomography (PET) and Magnetic Resonance Imaging (MRI), are increasingly significant in many clinical applications by integrating the complementary information from different medical images. In this paper, we propose a novel fusion model based on a dense convolutional network with dual attention (CSpA-DN) for PET and MRI images. In our framework, an encoder composed of the densely connected neural network is constructed to extract features from source images, and a decoder network is employed to generate the fused image from these features. Simultaneously, a dual-attention module is introduced in the encoder and decoder to further integrate local features along with their global dependencies adaptively. In the dual-attention module, a spatial attention block is leveraged to extract features of each point from encoder network by a weighted sum of feature information at all positions. Meanwhile, the interdependent correlation of all image features is aggregated via a module of channel attention. In addition, we design a specific loss function including image loss, structural loss, gradient loss and perception loss to preserve more structural and detail information and sharpen the edges of targets. Our approach facilitates the fused images to not only preserve abundant functional information from PET images but also retain rich detail structures of MRI images. Experimental results on publicly available datasets illustrate the superiorities of CSpA-DN model compared with state-of-the-art methods according to both qualitative observation and objective assessment.

Deep and Domain Transfer Learning Aided Photoacoustic Microscopy: Acoustic Resolution to Optical Resolution

Acoustic resolution photoacoustic micros- copy (AR-PAM) can achieve deeper imaging depth in biological tissue, with the sacrifice of imaging resolution compared with optical resolution photoacoustic microscopy (OR-PAM). Here we aim to enhance the AR-PAM image quality towards OR-PAM image, which specifically includes the enhancement of imaging resolution, restoration of micro-vasculatures, and reduction of artifacts. To address this issue, a network (MultiResU-Net) is first trained as generative model with simulated AR-OR image pairs, which are synthesized with physical transducer model. Moderate enhancement results can already be obtained when applying this model to in vivo AR imaging data. Nevertheless, the perceptual quality is unsatisfactory due to domain shift. Further, domain transfer learning technique under generative adversarial network (GAN) framework is proposed to drive the enhanced image's manifold towards that of real OR image. In this way, perceptually convincing AR to OR enhancement result is obtained, which can also be supported by quantitative analysis. Peak Signal to Noise Ratio (PSNR) and Structural Similarity Index (SSIM) values are significantly increased from 14.74 dB to 19.01 dB and from 0.1974 to 0.2937, respectively, validating the improvement of reconstruction correctness and overall perceptual quality. The proposed algorithm has also been validated across different imaging depths with experiments conducted in both shallow and deep tissue. The above AR to OR domain transfer learning with GAN (AODTL-GAN) framework has enabled the enhancement target with limited amount of matched in vivo AR-OR imaging data.

Machine learning in electron microscopy for advanced nanocharacterization: current developments, available tools and future outlook

In the last few years, electron microscopy has experienced a new methodological paradigm aimed to fix the bottlenecks and overcome the challenges of its analytical workflow. Machine learning and artificial intelligence are answering this call providing powerful resources towards automation, exploration, and development. In this review, we evaluate the state-of-the-art of machine learning applied to electron microscopy (and obliquely, to materials and nano-sciences). We start from the traditional imaging techniques to reach the newest higher-dimensionality ones, also covering the recent advances in spectroscopy and tomography. Additionally, the present review provides a practical guide for microscopists, and in general for material scientists, but not necessarily advanced machine learning practitioners, to straightforwardly apply the offered set of tools to their own research. To conclude, we explore the state-of-the-art of other disciplines with a broader experience in applying artificial intelligence methods to their research (e.g., high-energy physics, astronomy, Earth sciences, and even robotics, videogames, or marketing and finances), in order to narrow down the incoming future of electron microscopy, its challenges and outlook.

ERA-WGAT: Edge-enhanced residual autoencoder with a window-based graph attention convolutional network for low-dose CT denoising

Computed tomography (CT) has become a powerful tool for medical diagnosis. However, minimizing X-ray radiation risk for the patient poses significant challenges to obtain suitable low dose CT images. Although various low-dose CT methods using deep learning techniques have produced impressive results, convolutional neural network based methods focus more on local information and hence are very limited for non-local information extraction. This paper proposes ERA-WGAT, a residual autoencoder incorporating an edge enhancement module that performs convolution with eight types of learnable operators providing rich edge information and a window-based graph attention convolutional network that combines static and dynamic attention modules to explore non-local self-similarity. We use the compound loss function that combines MSE loss and multi-scale perceptual loss to mitigate the over-smoothing problem. Compared with current low-dose CT denoising methods, ERA-WGAT confirmed superior noise suppression and perceived image quality.

Synthetic Time of Flight Magnetic Resonance Angiography Generation Model Based on Cycle-Consistent Generative Adversarial Network Using PETRA-MRA in the Patients With Treated Intracranial Aneurysm

BACKGROUND

Pointwise encoding time reduction with radial acquisition (PETRA) magnetic resonance angiography (MRA) is useful for evaluating intracranial aneurysm recurrence, but the problem of severe background noise and low peripheral signal-to-noise ratio (SNR) remain. Deep learning could reduce noise using high- and low-quality images.

PURPOSE

To develop a cycle-consistent generative adversarial network (cycleGAN)-based deep learning model to generate synthetic TOF (synTOF) using PETRA.

STUDY TYPE

Retrospective.

POPULATION

A total of 377 patients (mean age: 60 ± 11; 293 females) with treated intracranial aneurysms who underwent both PETRA and TOF from October 2017 to January 2021. Data were randomly divided into training (49.9%, 188/377) and validation (50.1%, 189/377) groups.

FIELD STRENGTH/SEQUENCE

Ultra-short echo time and TOF-MRA on a 3-T MR system.

ASSESSMENT

For the cycleGAN model, the peak SNR (PSNR) and structural similarity (SSIM) were evaluated. Image quality was compared qualitatively (5-point Likert scale) and quantitatively (SNR). A multireader diagnostic optimality evaluation was performed with 17 radiologists (experience of 1-18 years).

STATISTICAL TESTS

Generalized estimating equation analysis, Friedman's test, McNemar test, and Spearman's rank correlation. P < 0.05 indicated statistical significance.

RESULTS

The PSNR and SSIM between synTOF and TOF were 17.51 [16.76; 18.31] dB and 0.71 ± 0.02. The median values of overall image quality, noise, sharpness, and vascular conspicuity were significantly higher for synTOF than for PETRA (4.00 [4.00; 5.00] vs. 4.00 [3.00; 4.00]; 5.00 [4.00; 5.00] vs. 3.00 [2.00; 4.00]; 4.00 [4.00; 4.00] vs. 4.00 [3.00; 4.00]; 3.00 [3.00; 4.00] vs. 3.00 [2.00; 3.00]). The SNRs of the middle cerebral arteries were the highest for synTOF (synTOF vs. TOF vs. PETRA; 63.67 [43.25; 105.00] vs. 52.42 [32.88; 74.67] vs. 21.05 [12.34; 37.88]). In the multireader evaluation, there was no significant difference in diagnostic optimality or preference between synTOF and TOF (19.00 [18.00; 19.00] vs. 20.00 [18.00; 20.00], P = 0.510; 8.00 [6.00; 11.00] vs. 11.00 [9.00, 14.00], P = 1.000).

DATA CONCLUSION

The cycleGAN-based deep learning model provided synTOF free from background artifact. The synTOF could be a versatile alternative to TOF in patients who have undergone PETRA for evaluating treated aneurysms.

EVIDENCE LEVEL

4 TECHNICAL EFFICACY: Stage 1.

Lung image segmentation based on DRD U-Net and combined WGAN with Deep Neural Network

PURPOSE

COVID-19 is a hot issue right now, and it's causing a huge number of infections in people, posing a grave threat to human life. Deep learning-based image diagnostic technology can effectively enhance the deficiencies of the current main detection method. This paper proposes a multi-classification model diagnosis based on segmentation and classification multi-task.

METHOD

In the segmentation task, the end-to-end DRD U-Net model is used to segment the lung lesions to improve the ability of feature reuse and target segmentation. In the classification task, the model combined with WGAN and Deep Neural Network classifier is used to effectively solve the problem of multi-classification of COVID-19 images with small samples, to achieve the goal of effectively distinguishing COVID-19 patients, other pneumonia patients, and normal subjects.

RESULTS

Experiments are carried out on common X-ray image and CT image data sets. The results display that in the segmentation task, the model is optimal in the key indicators of DSC and HD, and the error is increased by 0.33% and reduced by 3.57 mm compared with the original network U-Net. In the classification task, compared with SMOTE oversampling method, accuracy increased from 65.32% to 73.84%, F-measure increased from 67.65% to 74.65%, G-mean increased from 66.52% to 74.37%. At the same time, compared with other classical multi-task models, the results also have some advantages.

CONCLUSION

This study provides new possibilities for COVID-19 image diagnosis methods, improves the accuracy of diagnosis, and hopes to provide substantial help for COVID-19 diagnosis.

Artificial Intelligence (AI) for Lung Nodules, From the AJR Special Series on AI Applications

Interest in artificial intelligence (AI) applications for lung nodules continues to grow among radiologists, particularly with the expanding eligibility criteria and clinical utilization of lung cancer screening CT. AI has been heavily investigated for detecting and characterizing lung nodules and for guiding prognostic assessment. AI tools have also been used for image postprocessing (e.g., rib suppression on radiography or vessel suppression on CT) and for noninterpretive aspects of reporting and workflow, including management of nodule follow-up. Despite growing interest in and rapid development of AI tools and FDA approval of AI tools for pulmonary nodule evaluation, integration into clinical practice has been limited. Challenges to clinical adoption have included concerns about generalizability, regulatory issues, technical hurdles in implementation, and human skepticism. Further validation of AI tools for clinical use and demonstration of benefit in terms of patient-oriented outcomes also are needed. This article provides an overview of potential applications of AI tools in the imaging evaluation of lung nodules and discusses the challenges faced by practices interested in clinical implementation of such tools.

Improving reproducibility and performance of radiomics in low-dose CT using cycle GANs

BACKGROUND

As a means to extract biomarkers from medical imaging, radiomics has attracted increased attention from researchers. However, reproducibility and performance of radiomics in low-dose CT scans are still poor, mostly due to noise. Deep learning generative models can be used to denoise these images and in turn improve radiomics' reproducibility and performance. However, most generative models are trained on paired data, which can be difficult or impossible to collect.

PURPOSE

In this article, we investigate the possibility of denoising low-dose CTs using cycle generative adversarial networks (GANs) to improve radiomics reproducibility and performance based on unpaired datasets.

METHODS AND MATERIALS

Two cycle GANs were trained: (1) from paired data, by simulating low-dose CTs (i.e., introducing noise) from high-dose CTs and (2) from unpaired real low dose CTs. To accelerate convergence, during GAN training, a slice-paired training strategy was introduced. The trained GANs were applied to three scenarios: (1) improving radiomics reproducibility in simulated low-dose CT images and (2) same-day repeat low dose CTs (RIDER dataset), and (3) improving radiomics performance in survival prediction. Cycle GAN results were compared with a conditional GAN (CGAN) and an encoder-decoder network (EDN) trained on simulated paired data.

RESULTS

The cycle GAN trained on simulated data improved concordance correlation coefficients (CCC) of radiomic features from 0.87 (95%CI, [0.833,0.901]) to 0.93 (95%CI, [0.916,0.949]) on simulated noise CT and from 0.89 (95%CI, [0.881,0.914]) to 0.92 (95%CI, [0.908,0.937]) on the RIDER dataset, as well improving the area under the receiver operating characteristic curve (AUC) of survival prediction from 0.52 (95%CI, [0.511,0.538]) to 0.59 (95%CI, [0.578,0.602]). The cycle GAN trained on real data increased the CCCs of features in RIDER to 0.95 (95%CI, [0.933,0.961]) and the AUC of survival prediction to 0.58 (95%CI, [0.576,0.596]).

CONCLUSION

The results show that cycle GANs trained on both simulated and real data can improve radiomics' reproducibility and performance in low-dose CT and achieve similar results compared to CGANs and EDNs.

Deep-Learning-Based Fast Optical Coherence Tomography (OCT) Image Denoising for Smart Laser Osteotomy

Laser osteotomy promises precise cutting and minor bone tissue damage. We proposed Optical Coherence Tomography (OCT) to monitor the ablation process toward our smart laser osteotomy approach. The OCT image is helpful to identify tissue type and provide feedback for the ablation laser to avoid critical tissues such as bone marrow and nerve. Furthermore, in the implementation, the tissue classifier's accuracy is dependent on the quality of the OCT image. Therefore, image denoising plays an important role in having an accurate feedback system. A common OCT image denoising technique is the frame-averaging method. Inherent to this method is the need for multiple images, i.e., the more images used, the better the resulting image quality. However, this approach comes at the price of increased acquisition time and sensitivity to motion artifacts. To overcome these limitations, we applied a deep-learning denoising method capable of imitating the frame-averaging method. The resulting image had a similar image quality to the frame-averaging and was better than the classical digital filtering methods. We also evaluated if this method affects the tissue classifier model's accuracy that will provide feedback to the ablation laser. We found that image denoising significantly increased the accuracy of the tissue classifier. Furthermore, we observed that the classifier trained using the deep learning denoised images achieved similar accuracy to the classifier trained using frame-averaged images. The results suggest the possibility of using the deep learning method as a pre-processing step for real-time tissue classification in smart laser osteotomy.

Edge feature extraction-based dual CNN for LDCT denoising

In low-dose computed tomography (LDCT) denoising tasks, it is often difficult to balance edge/detail preservation and noise/artifact reduction. To solve this problem, we propose a dual convolutional neural network (CNN) based on edge feature extraction (Ed-DuCNN) for LDCT. Ed-DuCNN consists of two branches. One branch is the edge feature extraction subnet (Edge_Net) that can fully extract the edge details in the image. The other branch is the feature fusion subnet (Fusion_Net) that introduces an attention mechanism to fuse edge features and noisy image features. Specifically, first, shallow edge-specific detail features are extracted by trainable Sobel convolutional blocks and then are integrated into Edge_Net together with the LDCT images to obtain deep edge detail features. Finally, the input image, shallow edge detail, and deep edge detail features are fused in Fusion_Net to generate the final denoised image. The experimental results show that the proposed Ed-DuCNN can achieve competitive performance in terms of quantitative metrics and visual perceptual quality compared with that of state-of-the-art methods.

Motion correction in MR image for analysis of VSRAD using generative adversarial network

Voxel-based specific region analysis systems for Alzheimer's disease (VSRAD) are clinically used to measure the atrophied hippocampus captured by magnetic resonance imaging (MRI). However, motion artifacts during acquisition of images may distort the results of the analysis. This study aims to evaluate the usefulness of the Pix2Pix network in motion correction for the input image of VSRAD analysis. Seventy-three patients examined with MRI were distinguished into the training group (n = 51) and the test group (n = 22). To create artifact images, the k-space images were manipulated. Supervised deep learning was employed to obtain a Pix2Pix that generates motion-corrected images, with artifact images as the input data and original images as the reference data. The results of the VSRAD analysis (severity of voxel of interest (VOI) atrophy, the extent of gray matter (GM) atrophy, and extent of VOI atrophy) were recorded for artifact images and motion-corrected images, and were then compared with the original images. For comparison, the image quality of Pix2Pix generated motion-corrected image was also compared with that of U-Net. The Bland-Altman analysis showed that the mean of the limits of agreement was smaller for the motion-corrected images compared to the artifact images, suggesting successful motion correction by the Pix2Pix. The Spearman's rank correlation coefficients between original and motion-corrected images were almost perfect for all results (severity of VOI atrophy: 0.87-0.99, extent of GM atrophy: 0.88-00.98, extent of VOI atrophy: 0.90-1.00). Pix2Pix generated motion-corrected images that showed generally improved quantitative and qualitative image qualities compared with the U-net generated motion-corrected images. Our findings suggest that motion correction using Pix2Pix is a useful method for VSRAD analysis.

Wavelet subband-specific learning for low-dose computed tomography denoising

Deep neural networks have shown great improvements in low-dose computed tomography (CT) denoising. Early algorithms were primarily optimized to obtain an accurate image with low distortion between the denoised image and reference full-dose image at the cost of yielding an overly smoothed unrealistic CT image. Recent research has sought to preserve the fine details of denoised images with high perceptual quality, which has been accompanied by a decrease in objective quality due to a trade-off between perceptual quality and distortion. We pursue a network that can generate accurate and realistic CT images with high objective and perceptual quality within one network, achieving a better perception-distortion trade-off. To achieve this goal, we propose a stationary wavelet transform-assisted network employing the characteristics of high- and low-frequency domains of the wavelet transform and frequency subband-specific losses defined in the wavelet domain. We first introduce a stationary wavelet transform for the network training procedure. Then, we train the network using objective loss functions defined for high- and low-frequency domains to enhance the objective quality of the denoised CT image. With this network design, we train the network again after replacing the objective loss functions with perceptual loss functions in high- and low-frequency domains. As a result, we acquired denoised CT images with high perceptual quality using this strategy while minimizing the objective quality loss. We evaluated our algorithms on the phantom and clinical images, and the quantitative and qualitative results indicate that ours outperform the existing state-of-the-art algorithms in terms of objective and perceptual quality.