Low-Dose CT Image Denoising Using a Generative Adversarial Network With Wasserstein Distance and Perceptual Loss

Structure-preserving low-dose computed tomography image denoising using a deep residual adaptive global context attention network

Background

Low-dose computed tomography (LDCT) scans can effectively reduce the radiation damage to patients, but this is highly detrimental to CT image quality. Deep convolutional neural networks (CNNs) have shown their potential in improving LDCT image quality. However, the conventional CNN-based approaches rely fundamentally on the convolution operations, which are ineffective for modeling the correlations among nonlocal similar structures and the regionally distinct statistical properties in CT images. This modeling deficiency hampers the denoising performance for CT images derived in this manner.

Methods

In this paper, we propose an adaptive global context (AGC) modeling scheme to describe the nonlocal correlations and the regionally distinct statistics in CT images with negligible computation load. We further propose an AGC-based long-short residual encoder-decoder (AGC-LSRED) network for efficient LDCT image noise artifact-suppression tasks. Specifically, stacks of residual AGC attention blocks (RAGCBs) with long and short skip connections are constructed in the AGC-LSRED network, which allows valuable structural and positional information to be bypassed through these identity-based skip connections and thus eases the training of the deep denoising network. For training the AGC-LSRED network, we propose a compound loss that combines the L1 loss, adversarial loss, and self-supervised multi-scale perceptual loss.

Results

Quantitative and qualitative experimental studies were performed to verify and validate the effectiveness of the proposed method. The simulation experiments demonstrated the proposed method exhibits the best result in terms of noise suppression [root-mean-square error (RMSE) =9.02; peak signal-to-noise ratio (PSNR) =33.17] and fine structure preservation [structural similarity index (SSIM) =0.925] compared with other competitive CNN-based methods. The experiments on real data illustrated that the proposed method has advantages over other methods in terms of radiologists' subjective assessment scores (averaged scores =4.34).

Conclusions

With the use of the AGC modeling scheme to characterize the structural information in CT images and of residual AGC-attention blocks with long and short skip connections to ease the network training, the proposed AGC-LSRED method achieves satisfactory results in preserving fine anatomical structures and suppressing noise in LDCT images.

Clinical applications of artificial intelligence in radiology

The rapid growth of medical imaging has placed increasing demands on radiologists. In this scenario, artificial intelligence (AI) has become an attractive partner, one that may complement case interpretation and may aid in various non-interpretive aspects of the work in the radiological clinic. In this review, we discuss interpretative and non-interpretative uses of AI in the clinical practice, as well as report on the barriers to AI's adoption in the clinic. We show that AI currently has a modest to moderate penetration in the clinical practice, with many radiologists still being unconvinced of its value and the return on its investment. Moreover, we discuss the radiologists' liabilities regarding the AI decisions, and explain how we currently do not have regulation to guide the implementation of explainable AI or of self-learning algorithms.

Inflating 2D convolution weights for efficient generation of 3D medical images

BACKGROUND AND OBJECTIVE

The generation of three-dimensional (3D) medical images has great application potential since it takes into account the 3D anatomical structure. Two problems prevent effective training of a 3D medical generative model: (1) 3D medical images are expensive to acquire and annotate, resulting in an insufficient number of training images, and (2) a large number of parameters are involved in 3D convolution.

METHODS

We propose a novel GAN model called 3D Split&Shuffle-GAN. To address the 3D data scarcity issue, we first pre-train a two-dimensional (2D) GAN model using abundant image slices and inflate the 2D convolution weights to improve the initialization of the 3D GAN. Novel 3D network architectures are proposed for both the generator and discriminator of the GAN model to significantly reduce the number of parameters while maintaining the quality of image generation. Several weight inflation strategies and parameter-efficient 3D architectures are investigated.

RESULTS

Experiments on both heart (Stanford AIMI Coronary Calcium) and brain (Alzheimer's Disease Neuroimaging Initiative) datasets show that our method leads to improved 3D image generation quality (14.7 improvements on Frchet inception distance) with significantly fewer parameters (only 48.5% of the baseline method).

CONCLUSIONS

We built a parameter-efficient 3D medical image generation model. Due to the efficiency and effectiveness, it has the potential to generate high-quality 3D brain and heart images for real use cases.

Generative Adversarial Networks in Medicine: Important Considerations for this Emerging Innovation in Artificial Intelligence

The advent of artificial intelligence (AI) and machine learning (ML) has revolutionized the field of medicine. Although highly effective, the rapid expansion of this technology has created some anticipated and unanticipated bioethical considerations. With these powerful applications, there is a necessity for framework regulations to ensure equitable and safe deployment of technology. Generative Adversarial Networks (GANs) are emerging ML techniques that have immense applications in medical imaging due to their ability to produce synthetic medical images and aid in medical AI training. Producing accurate synthetic images with GANs can address current limitations in AI development for medical imaging and overcome current dataset type and size constraints. Offsetting these constraints can dramatically improve the development and implementation of AI medical imaging and restructure the practice of medicine. As observed with its other AI predecessors, considerations must be taken into place to help regulate its development for clinical use. In this paper, we discuss the legal, ethical, and technical challenges for future safe integration of this technology in the healthcare sector.

Unsupervised learning-based dual-domain method for low-dose CT denoising

Objective. Low-dose CT (LDCT) is an important research topic in the field of CT imaging because of its ability to reduce radiation damage in clinical diagnosis. In recent years, deep learning techniques have been widely applied in LDCT imaging and a large number of denoising methods have been proposed. However, One major challenge of supervised deep learning-based methods is the exactly geometric pairing of datasets with different doses. Therefore, the aim of this study is to develop an unsupervised learning-based LDCT imaging method to address the aforementioned challenges.Approach. In this paper, we propose an unsupervised learning-based dual-domain method for LDCT denoising, which consists of two stages: the first stage is projection domain denoising, in which the unsupervised learning method Noise2Self is applied to denoise the projection data with statistically independent and zero-mean noise. The second stage is an iterative enhancement approach, which combines the prior information obtained from the generative model with an iterative reconstruction algorithm to enhance the details of the reconstructed image.Main results. Experimental results show that our proposed method outperforms the comparison method in terms of denoising effect. Particularly, in terms of SSIM, the denoised results obtained using our method achieve the highest SSIM.Significance. In conclusion, our unsupervised learning-based method can be a promising alternative to the traditional supervised methods for LDCT imaging, especially when the availability of the labeled datasets is limited.

MLNAN: Multi-level noise-aware network for low-dose CT imaging implemented with constrained cycle Wasserstein generative adversarial networks

Low-dose CT techniques attempt to minimize the radiation exposure of patients by estimating the high-resolution normal-dose CT images to reduce the risk of radiation-induced cancer. In recent years, many deep learning methods have been proposed to solve this problem by building a mapping function between low-dose CT images and their high-dose counterparts. However, most of these methods ignore the effect of different radiation doses on the final CT images, which results in large differences in the intensity of the noise observable in CT images. What'more, the noise intensity of low-dose CT images exists significantly differences under different medical devices manufacturers. In this paper, we propose a multi-level noise-aware network (MLNAN) implemented with constrained cycle Wasserstein generative adversarial networks to recovery the low-dose CT images under uncertain noise levels. Particularly, the noise-level classification is predicted and reused as a prior pattern in generator networks. Moreover, the discriminator network introduces noise-level determination. Under two dose-reduction strategies, experiments to evaluate the performance of proposed method are conducted on two datasets, including the simulated clinical AAPM challenge datasets and commercial CT datasets from United Imaging Healthcare (UIH). The experimental results illustrate the effectiveness of our proposed method in terms of noise suppression and structural detail preservation compared with several other deep-learning based methods. Ablation studies validate the effectiveness of the individual components regarding the afforded performance improvement. Further research for practical clinical applications and other medical modalities is required in future works.

Recent progress in transformer-based medical image analysis

The transformer is primarily used in the field of natural language processing. Recently, it has been adopted and shows promise in the computer vision (CV) field. Medical image analysis (MIA), as a critical branch of CV, also greatly benefits from this state-of-the-art technique. In this review, we first recap the core component of the transformer, the attention mechanism, and the detailed structures of the transformer. After that, we depict the recent progress of the transformer in the field of MIA. We organize the applications in a sequence of different tasks, including classification, segmentation, captioning, registration, detection, enhancement, localization, and synthesis. The mainstream classification and segmentation tasks are further divided into eleven medical image modalities. A large number of experiments studied in this review illustrate that the transformer-based method outperforms existing methods through comparisons with multiple evaluation metrics. Finally, we discuss the open challenges and future opportunities in this field. This task-modality review with the latest contents, detailed information, and comprehensive comparison may greatly benefit the broad MIA community.

Self-adaption and texture generation: A hybrid loss function for low-dose CT denoising

BACKGROUND

Deep learning has been successfully applied to low-dose CT (LDCT) denoising. But the training of the model is very dependent on an appropriate loss function. Existing denoising models often use per-pixel loss, including mean abs error (MAE) and mean square error (MSE). This ignores the difference in denoising difficulty between different regions of the CT images and leads to the loss of large texture information in the generated image.

PURPOSE

In this paper, we propose a new hybrid loss function that adapts to the noise in different regions of CT images to balance the denoising difficulty and preserve texture details, thus acquiring CT images with high-quality diagnostic value using LDCT images, providing strong support for condition diagnosis.

METHODS

We propose a hybrid loss function consisting of weighted patch loss (WPLoss) and high-frequency information loss (HFLoss). To enhance the model's denoising ability of the local areas which are difficult to denoise, we improve the MAE to obtain WPLoss. After the generated image and the target image are divided into several patches, the loss weight of each patch is adaptively and dynamically adjusted according to its loss ratio. In addition, considering that texture details are contained in the high-frequency information of the image, we use HFLoss to calculate the difference between CT images in the high-frequency information part.

RESULTS

Our hybrid loss function improves the denoising performance of several models in the experiment, and obtains a higher peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM). Moreover, through visual inspection of the generated results of the comparison experiment, the proposed hybrid function can effectively suppress noise and retain image details.

CONCLUSIONS

We propose a hybrid loss function for LDCT image denoising, which has good interpretation properties and can improve the denoising performance of existing models. And the validation results of multiple models using different datasets show that it has good generalization ability. By using this loss function, high-quality CT images with low radiation are achieved, which can avoid the hazards caused by radiation and ensure the disease diagnosis for patients.

A novel denoising method for low-dose CT images based on transformer and CNN

Computed Tomography (CT) has become a mainstream imaging tool in medical diagnosis. However, the issue of increased cancer risk due to radiation exposure has raised public concern. Low-dose computed tomography (LDCT) technique is a CT scan with lower radiation dose than conventional scans. LDCT is used to make a diagnosis of lesions with the smallest dose of x-rays, and is currently mainly used for early lung cancer screening. However, LDCT has severe image noise, and these noises affect adversely the quality of medical images and thus the diagnosis of lesions. In this paper, we propose a novel LDCT image denoising method based on transformer combined with convolutional neural network (CNN). The encoder part of the network is based on CNN, which is mainly used to extract the image detail information. In the decoder part, we propose a dual-path transformer block (DPTB), which extracts the features of input of the skip connection and the features of input of the previous level through two paths respectively. DPTB can better restore the detail and structure information of the denoised image. In order to pay more attention to the key regions of the feature images extracted at the shallow level of the network, we also propose a multi-feature spatial attention block (MSAB) in the skip connection part. Experimental studies are conducted, and comparisons with the state-of-the-art networks are made, and the results demonstrate that the developed method can effectively remove the noise in CT images and improve the image quality in the evaluation metrics of peak signal to noise ratio (PSNR), structural similarity (SSIM), and root mean square error (RMSE) and is superior to the state-of-the-art models. Our method achieved 28.9720 of PSNR, 0.8595 of SSIM and 14.8657 of RMSE on the Mayo Clinic LDCT Grand Challenge dataset. For different noise level σ (15, 35, and 55) on the QIN_LUNG_CT dataset, our proposed also achieved better performances.

Direct estimation of regional lung volume change from paired and single CT images using residual regression neural network

BACKGROUND

Chest computed tomography (CT) enables characterization of pulmonary diseases by producing high-resolution and high-contrast images of the intricate lung structures. Deformable image registration is used to align chest CT scans at different lung volumes, yielding estimates of local tissue expansion and contraction.

PURPOSE

We investigated the utility of deep generative models for directly predicting local tissue volume change from lung CT images, bypassing computationally expensive iterative image registration and providing a method that can be utilized in scenarios where either one or two CT scans are available.

METHODS

A residual regression convolutional neural network, called Reg3DNet+, is proposed for directly regressing high-resolution images of local tissue volume change (i.e., Jacobian) from CT images. Image registration was performed between lung volumes at total lung capacity (TLC) and functional residual capacity (FRC) using a tissue mass- and structure-preserving registration algorithm. The Jacobian image was calculated from the registration-derived displacement field and used as the ground truth for local tissue volume change. Four separate Reg3DNet+ models were trained to predict Jacobian images using a multifactorial study design to compare the effects of network input (i.e., single image vs. paired images) and output space (i.e., FRC vs. TLC). The models were trained and evaluated on image datasets from the COPDGene study. Models were evaluated against the registration-derived Jacobian images using local, regional, and global evaluation metrics.

RESULTS

Statistical analysis revealed that both factors - network input and output space - were significant determinants for change in evaluation metrics. Paired-input models performed better than single-input models, and model performance was better in the output space of FRC rather than TLC. Mean structural similarity index for paired-input models was 0.959 and 0.956 for FRC and TLC output spaces, respectively, and for single-input models was 0.951 and 0.937. Global evaluation metrics demonstrated correlation between registration-derived Jacobian mean and predicted Jacobian mean: coefficient of determination (r2 ) for paired-input models was 0.974 and 0.938 for FRC and TLC output spaces, respectively, and for single-input models was 0.598 and 0.346. After correcting for effort, registration-derived lobar volume change was strongly correlated with the predicted lobar volume change: for paired-input models r2 was 0.899 for both FRC and TLC output spaces, and for single-input models r2 was 0.803 and 0.862, respectively.

CONCLUSIONS

Convolutional neural networks can be used to directly predict local tissue mechanics, eliminating the need for computationally expensive image registration. Networks that use paired CT images acquired at TLC and FRC allow for more accurate prediction of local tissue expansion compared to networks that use a single image. Networks that only require a single input image still show promising results, particularly after correcting for effort, and allow for local tissue expansion estimation in cases where multiple CT scans are not available. For single-input networks, the FRC image is more predictive of local tissue volume change compared to the TLC image.

Low-Dose CT Image Synthesis for Domain Adaptation Imaging Using a Generative Adversarial Network With Noise Encoding Transfer Learning

Deep learning (DL) based image processing methods have been successfully applied to low-dose x-ray images based on the assumption that the feature distribution of the training data is consistent with that of the test data. However, low-dose computed tomography (LDCT) images from different commercial scanners may contain different amounts and types of image noise, violating this assumption. Moreover, in the application of DL based image processing methods to LDCT, the feature distributions of LDCT images from simulation and clinical CT examination can be quite different. Therefore, the network models trained with simulated image data or LDCT images from one specific scanner may not work well for another CT scanner and image processing task. To solve such domain adaptation problem, in this study, a novel generative adversarial network (GAN) with noise encoding transfer learning (NETL), or GAN-NETL, is proposed to generate a paired dataset with a different noise style. Specifically, we proposed a method to perform noise encoding operator and incorporate it into the generator to extract a noise style. Meanwhile, with a transfer learning (TL) approach, the image noise encoding operator transformed the noise type of the source domain to that of the target domain for realistic noise generation. One public and two private datasets are used to evaluate the proposed method. Experiment results demonstrated the feasibility and effectiveness of our proposed GAN-NETL model in LDCT image synthesis. In addition, we conduct additional image denoising study using the synthesized clinical LDCT data, which verified the merit of the proposed synthesis in improving the performance of the DL based LDCT processing method.

Enhancing cryo-EM maps with 3D deep generative networks for assisting protein structure modeling

MOTIVATION

The tertiary structures of an increasing number of biological macromolecules have been determined using cryo-electron microscopy (cryo-EM). However, there are still many cases where the resolution is not high enough to model the molecular structures with standard computational tools. If the resolution obtained is near the empirical borderline (3-4.5 Å), improvement in the map quality facilitates structure modeling.

RESULTS

We report EM-GAN, a novel approach that modifies an input cryo-EM map to assist protein structure modeling. The method uses a 3D generative adversarial network (GAN) that has been trained on high- and low-resolution density maps to learn the density patterns, and modifies the input map to enhance its suitability for modeling. The method was tested extensively on a dataset of 65 EM maps in the resolution range of 3-6 Å and showed substantial improvements in structure modeling using popular protein structure modeling tools.

AVAILABILITY AND IMPLEMENTATION

https://github.com/kiharalab/EM-GAN, Google Colab: https://tinyurl.com/3ccxpttx.

Impact of loss functions on the performance of a deep neural network designed to restore low-dose digital mammography

Digital mammography is currently the most common imaging tool for breast cancer screening. Although the benefits of using digital mammography for cancer screening outweigh the risks associated with the x-ray exposure, the radiation dose must be kept as low as possible while maintaining the diagnostic utility of the generated images, thus minimizing patient risks. Many studies investigated the feasibility of dose reduction by restoring low-dose images using deep neural networks. In these cases, choosing the appropriate training database and loss function is crucial and impacts the quality of the results. In this work, we used a standard residual network (ResNet) to restore low-dose digital mammography images and evaluated the performance of several loss functions. For training purposes, we extracted 256,000 image patches from a dataset of 400 images of retrospective clinical mammography exams, where dose reduction factors of 75% and 50% were simulated to generate low and standard-dose pairs. We validated the network in a real scenario by using a physical anthropomorphic breast phantom to acquire real low-dose and standard full-dose images in a commercially available mammography system, which were then processed through our trained model. We benchmarked our results against an analytical restoration model for low-dose digital mammography. Objective assessment was performed through the signal-to-noise ratio (SNR) and the mean normalized squared error (MNSE), decomposed into residual noise and bias. Statistical tests revealed that the use of the perceptual loss (PL4) resulted in statistically significant differences when compared to all other loss functions. Additionally, images restored using the PL4 achieved the closest residual noise to the standard dose. On the other hand, perceptual loss PL3, structural similarity index (SSIM) and one of the adversarial losses achieved the lowest bias for both dose reduction factors. The source code of our deep neural network is available at https://github.com/WANG-AXIS/LdDMDenoising.

Degradation Adaption Local-to-Global Transformer for Low-Dose CT Image Denoising

Computer tomography (CT) has played an essential role in the field of medical diagnosis. Considering the potential risk of exposing patients to X-ray radiations, low-dose CT (LDCT) images have been widely applied in the medical imaging field. Since reducing the radiation dose may result in increased noise and artifacts, methods that can eliminate the noise and artifacts in the LDCT image have drawn increasing attentions and produced impressive results over the past decades. However, recent proposed methods mostly suffer from noise remaining, over-smoothing structures, or false lesions derived from noise. To tackle these issues, we propose a novel degradation adaption local-to-global transformer (DALG-Transformer) for restoring the LDCT image. Specifically, the DALG-Transformer is built on self-attention modules which excel at modeling long-range information between image patch sequences. Meanwhile, an unsupervised degradation representation learning scheme is first developed in medical image processing to learn abstract degradation representations of the LDCT images, which can distinguish various degradations in the representation space rather than the pixel space. Then, we introduce a degradation-aware modulated convolution and gated mechanism into the building modules (i.e., multi-head attention and feed-forward network) of each Transformer block, which can bring in the complementary strength of convolution operation to emphasize on the spatially local context. The experimental results show that the DALG-Transformer can provide superior performance in noise removal, structure preservation, and false lesions elimination compared with five existing representative deep networks. The proposed networks may be readily applied to other image processing tasks including image reconstruction, image deblurring, and image super-resolution.

Multi-Scale Feature Fusion Network for Low-Dose CT Denoising

Computed tomography (CT) is an imaging technique extensively used in medical treatment, but too much radiation dose in a CT scan will cause harm to the human body. Decreasing the dose of radiation will result in increased noise and artifacts in the reconstructed image, blurring the internal tissue and edge details. To get high-quality CT images, we present a multi-scale feature fusion network (MSFLNet) for low-dose CT (LDCT) denoising. In our MSFLNet, we combined multiple feature extraction modules, effective noise reduction modules, and fusion modules constructed using the attention mechanism to construct a horizontally connected multi-scale structure as the overall architecture of the network, which is used to construct different levels of feature maps at all scales. We innovatively define a composite loss function composed of pixel-level loss based on MS-SSIM-L1 and edge-based edge loss for LDCT denoising. In short, our approach learns a rich set of features that combine contextual information from multiple scales while maintaining the spatial details of denoised CT images. Our laboratory results indicate that compared with the existing methods, the peak signal-to-noise ratio (PSNR) value of CT images of the AAPM dataset processed by the new model is 33.6490, and the structural similarity (SSIM) value is 0.9174, which also achieves good results on the Piglet dataset with different doses. The results also show that the method removes noise and artifacts while effectively preserving CT images' architecture and grain information.

Dynamic controllable residual generative adversarial network for low-dose computed tomography imaging

Background

Computed tomography (CT) imaging technology has become an indispensable auxiliary method in medical diagnosis and treatment. In mitigating the radiation damage caused by X-rays, low-dose computed tomography (LDCT) scanning is becoming more widely applied. However, LDCT scanning reduces the signal-to-noise ratio of the projection, and the resulting images suffer from serious streak artifacts and spot noise. In particular, the intensity of noise and artifacts varies significantly across different body parts under a single low-dose protocol.

Methods

To improve the quality of different degraded LDCT images in a unified framework, we developed a generative adversarial learning framework with a dynamic controllable residual. First, the generator network consists of the basic subnetwork and the conditional subnetwork. Inspired by the dynamic control strategy, we designed the basic subnetwork to adopt a residual architecture, with the conditional subnetwork providing weights to control the residual intensity. Second, we chose the Visual Geometry Group Network-128 (VGG-128) as the discriminator to improve the noise artifact suppression and feature retention ability of the generator. Additionally, a hybrid loss function was specifically designed, including the mean square error (MSE) loss, structural similarity index metric (SSIM) loss, adversarial loss, and gradient penalty (GP) loss.

Results

The results obtained on two datasets show the competitive performance of the proposed framework, with a 3.22 dB peak signal-to-noise ratio (PSNR) margin, 0.03 SSIM margin, and 0.2 contrast-to-noise ratio margin on the Challenge data and a 1.0 dB PSNR margin and 0.01 SSIM margin on the real data.

Conclusions

Experimental results demonstrated the competitive performance of the proposed method in terms of noise decrease, structural retention, and visual impression improvement.

Area-Detector Computed Tomography for Pulmonary Functional Imaging

An area-detector CT (ADCT) has a 320-detector row and can obtain isotropic volume data without helical scanning within an area of nearly 160 mm. The actual-perfusion CT data within this area can, thus, be obtained by means of continuous dynamic scanning for the qualitative or quantitative evaluation of regional perfusion within nodules, lymph nodes, or tumors. Moreover, this system can obtain CT data with not only helical but also step-and-shoot or wide-volume scanning for body CT imaging. ADCT also has the potential to use dual-energy CT and subtraction CT to enable contrast-enhanced visualization by means of not only iodine but also xenon or krypton for functional evaluations. Therefore, systems using ADCT may be able to function as a pulmonary functional imaging tool. This review is intended to help the reader understand, with study results published during the last a few decades, the basic or clinical evidence about (1) newly applied reconstruction methods for radiation dose reduction for functional ADCT, (2) morphology-based pulmonary functional imaging, (3) pulmonary perfusion evaluation, (4) ventilation assessment, and (5) biomechanical evaluation.

A Deep Learning Approach for Rapid and Generalizable Denoising of Photon-Counting Micro-CT Images

Photon-counting CT (PCCT) is powerful for spectral imaging and material decomposition but produces noisy weighted filtered backprojection (wFBP) reconstructions. Although iterative reconstruction effectively denoises these images, it requires extensive computation time. To overcome this limitation, we propose a deep learning (DL) model, UnetU, which quickly estimates iterative reconstruction from wFBP. Utilizing a 2D U-net convolutional neural network (CNN) with a custom loss function and transformation of wFBP, UnetU promotes accurate material decomposition across various photon-counting detector (PCD) energy threshold settings. UnetU outperformed multi-energy non-local means (ME NLM) and a conventional denoising CNN called UnetwFBP in terms of root mean square error (RMSE) in test set reconstructions and their respective matrix inversion material decompositions. Qualitative results in reconstruction and material decomposition domains revealed that UnetU is the best approximation of iterative reconstruction. In reconstructions with varying undersampling factors from a high dose ex vivo scan, UnetU consistently gave higher structural similarity (SSIM) and peak signal-to-noise ratio (PSNR) to the fully sampled iterative reconstruction than ME NLM and UnetwFBP. This research demonstrates UnetU's potential as a fast (i.e., 15 times faster than iterative reconstruction) and generalizable approach for PCCT denoising, holding promise for advancing preclinical PCCT research.

Self-trained deep convolutional neural network for noise reduction in CT

Purpose

Supervised deep convolutional neural network (CNN)-based methods have been actively used in clinical CT to reduce image noise. The networks of these methods are typically trained using paired high- and low-quality data from a massive number of patients and/or phantom images. This training process is tedious, and the network trained under a given condition may not be generalizable to patient images acquired and reconstructed under different conditions. We propose a self-trained deep CNN (ST_CNN) method for noise reduction in CT that does not rely on pre-existing training datasets.

Approach

The ST_CNN training was accomplished using extensive data augmentation in the projection domain, and the inference was applied to the data itself. Specifically, multiple independent noise insertions were applied to the original patient projection data to generate multiple realizations of low-quality projection data. Then, rotation augmentation was adopted for both the original and low-quality projection data by applying the rotation angle directly on the projection data so that images were rotated at arbitrary angles without introducing additional bias. A large number of paired low- and high-quality images from the same patient were reconstructed and paired for training the ST_CNN model.

Results

No significant difference was found between the ST_CNN and conventional CNN models in terms of the peak signal-to-noise ratio and structural similarity index measure. The ST_CNN model outperformed the conventional CNN model in terms of noise texture and homogeneity in liver parenchyma as well as better subjective visualization of liver lesions. The ST_CNN may sacrifice the sharpness of vessels slightly compared to the conventional CNN model but without affecting the visibility of peripheral vessels or diagnosis of vascular pathology.

Conclusions

The proposed ST_CNN method trained from the data itself may achieve similar image quality in comparison with conventional deep CNN denoising methods pre-trained on external datasets.

Radiograph-comparable image synthesis for spine alignment analysis using deep learning with prospective clinical validation

Background

Adolescent idiopathic scoliosis (AIS) is the most common type of spinal disorder affecting children. Clinical screening and diagnosis require physical and radiographic examinations, which are either subjective or increase radiation exposure. We therefore developed and validated a radiation-free portable system and device utilising light-based depth sensing and deep learning technologies to analyse AIS by landmark detection and image synthesis.

Methods

Consecutive patients with AIS attending two local scoliosis clinics in Hong Kong between October 9, 2019, and May 21, 2022, were recruited. Patients were excluded if they had psychological and/or systematic neural disorders that could influence the compliance of the study and/or the mobility of the patients. For each participant, a Red Green Blue-Depth (RGBD) image of the nude back was collected using our in-house radiation-free device. Manually labelled landmarks and alignment parameters by our spine surgeons were considered as the ground truth (GT). Images from training and internal validation cohorts (n = 1936) were used to develop the deep learning models. The model was then prospectively validated on another cohort (n = 302) which was collected in Hong Kong and had the same demographic properties as the training cohort. We evaluated the prediction accuracy of the model on nude back landmark detection as well as the performance on radiograph-comparable image (RCI) synthesis. The obtained RCIs contain sufficient anatomical information that can quantify disease severities and curve types.

Findings

Our model had a consistently high accuracy in predicting the nude back anatomical landmarks with a less than 4-pixel error regarding the mean Euclidian and Manhattan distance. The synthesized RCI for AIS severity classification achieved a sensitivity and negative predictive value of over 0.909 and 0.933, and the performance for curve type classification was 0.974 and 0.908, with spine specialists' manual assessment results on real radiographs as GT. The estimated Cobb angle from synthesized RCIs had a strong correlation with the GT angles (R² = 0.984, p < 0.001).

Interpretation

The radiation-free medical device powered by depth sensing and deep learning techniques can provide instantaneous and harmless spine alignment analysis which has the potential for integration into routine screening for adolescents.

Funding

Innovation and Technology Fund (MRP/038/20X), Health Services Research Fund (HMRF) 08192266.

Need for objective task-based evaluation of deep learning-based denoising methods: A study in the context of myocardial perfusion SPECT

BACKGROUND

Artificial intelligence-based methods have generated substantial interest in nuclear medicine. An area of significant interest has been the use of deep-learning (DL)-based approaches for denoising images acquired with lower doses, shorter acquisition times, or both. Objective evaluation of these approaches is essential for clinical application.

PURPOSE

DL-based approaches for denoising nuclear-medicine images have typically been evaluated using fidelity-based figures of merit (FoMs) such as root mean squared error (RMSE) and structural similarity index measure (SSIM). However, these images are acquired for clinical tasks and thus should be evaluated based on their performance in these tasks. Our objectives were to: (1) investigate whether evaluation with these FoMs is consistent with objective clinical-task-based evaluation; (2) provide a theoretical analysis for determining the impact of denoising on signal-detection tasks; and (3) demonstrate the utility of virtual imaging trials (VITs) to evaluate DL-based methods.

METHODS

A VIT to evaluate a DL-based method for denoising myocardial perfusion SPECT (MPS) images was conducted. To conduct this evaluation study, we followed the recently published best practices for the evaluation of AI algorithms for nuclear medicine (the RELAINCE guidelines). An anthropomorphic patient population modeling clinically relevant variability was simulated. Projection data for this patient population at normal and low-dose count levels (20%, 15%, 10%, 5%) were generated using well-validated Monte Carlo-based simulations. The images were reconstructed using a 3-D ordered-subsets expectation maximization-based approach. Next, the low-dose images were denoised using a commonly used convolutional neural network-based approach. The impact of DL-based denoising was evaluated using both fidelity-based FoMs and area under the receiver operating characteristic curve (AUC), which quantified performance on the clinical task of detecting perfusion defects in MPS images as obtained using a model observer with anthropomorphic channels. We then provide a mathematical treatment to probe the impact of post-processing operations on signal-detection tasks and use this treatment to analyze the findings of this study.

RESULTS

Based on fidelity-based FoMs, denoising using the considered DL-based method led to significantly superior performance. However, based on ROC analysis, denoising did not improve, and in fact, often degraded detection-task performance. This discordance between fidelity-based FoMs and task-based evaluation was observed at all the low-dose levels and for different cardiac-defect types. Our theoretical analysis revealed that the major reason for this degraded performance was that the denoising method reduced the difference in the means of the reconstructed images and of the channel operator-extracted feature vectors between the defect-absent and defect-present cases.

CONCLUSIONS

The results show the discrepancy between the evaluation of DL-based methods with fidelity-based metrics versus the evaluation on clinical tasks. This motivates the need for objective task-based evaluation of DL-based denoising approaches. Further, this study shows how VITs provide a mechanism to conduct such evaluations computationally, in a time and resource-efficient setting, and avoid risks such as radiation dose to the patient. Finally, our theoretical treatment reveals insights into the reasons for the limited performance of the denoising approach and may be used to probe the effect of other post-processing operations on signal-detection tasks.

Generating synthetic mixed-type longitudinal electronic health records for artificial intelligent applications

The recent availability of electronic health records (EHRs) have provided enormous opportunities to develop artificial intelligence (AI) algorithms. However, patient privacy has become a major concern that limits data sharing across hospital settings and subsequently hinders the advances in AI. Synthetic data, which benefits from the development and proliferation of generative models, has served as a promising substitute for real patient EHR data. However, the current generative models are limited as they only generate single type of clinical data for a synthetic patient, i.e., either continuous-valued or discrete-valued. To mimic the nature of clinical decision-making which encompasses various data types/sources, in this study, we propose a generative adversarial network (GAN) entitled EHR-M-GAN that simultaneously synthesizes mixed-type timeseries EHR data. EHR-M-GAN is capable of capturing the multidimensional, heterogeneous, and correlated temporal dynamics in patient trajectories. We have validated EHR-M-GAN on three publicly-available intensive care unit databases with records from a total of 141,488 unique patients, and performed privacy risk evaluation of the proposed model. EHR-M-GAN has demonstrated its superiority over state-of-the-art benchmarks for synthesizing clinical timeseries with high fidelity, while addressing the limitations regarding data types and dimensionality in the current generative models. Notably, prediction models for outcomes of intensive care performed significantly better when training data was augmented with the addition of EHR-M-GAN-generated timeseries. EHR-M-GAN may have use in developing AI algorithms in resource-limited settings, lowering the barrier for data acquisition while preserving patient privacy.

Dual-scale similarity-guided cycle generative adversarial network for unsupervised low-dose CT denoising

Removing the noise in low-dose CT (LDCT) is crucial to improving the diagnostic quality. Previously, many supervised or unsupervised deep learning-based LDCT denoising algorithms have been proposed. Unsupervised LDCT denoising algorithms are more practical than supervised ones since they do not need paired samples. However, unsupervised LDCT denoising algorithms are rarely used clinically due to their unsatisfactory denoising ability. In unsupervised LDCT denoising, the lack of paired samples makes the direction of gradient descent full of uncertainty. On the contrary, paired samples used in supervised denoising allow the parameters of networks to have a clear direction of gradient descent. To bridge the gap in performance between unsupervised and supervised LDCT denoising, we propose dual-scale similarity-guided cycle generative adversarial network (DSC-GAN). DSC-GAN uses similarity-based pseudo-pairing to better accomplish unsupervised LDCT denoising. We design a Vision Transformer-based global similarity descriptor and a residual neural network-based local similarity descriptor for DSC-GAN to effectively describe the similarity between two samples. During training, pseudo-pairs, i.e., similar LDCT samples and normal-dose CT (NDCT) samples, dominate parameter updates. Thus, the training can achieve equivalent effect as training with paired samples. Experiments on two datasets demonstrate that DSC-GAN beats the state-of-the-art unsupervised algorithms and reaches a level close to supervised LDCT denoising algorithms.

Dual-domain fusion deep convolutional neural network for low-dose CT denoising

BACKGROUND

In view of the underlying health risks posed by X-ray radiation, the main goal of the present research is to achieve high-quality CT images at the same time as reducing x-ray radiation. In recent years, convolutional neural network (CNN) has shown excellent performance in removing low-dose CT noise. However, previous work mainly focused on deepening and feature extraction work on CNN without considering fusion of features from frequency domain and image domain.

OBJECTIVE

To address this issue, we propose to develop and test a new LDCT image denoising method based on a dual-domain fusion deep convolutional neural network (DFCNN).

METHODS

This method deals with two domains, namely, the DCT domain and the image domain. In the DCT domain, we design a new residual CBAM network to enhance the internal and external relations of different channels while reducing noise to promote richer image structure information. For the image domain, we propose a top-down multi-scale codec network as a denoising network to obtain more acceptable edges and textures while obtaining multi-scale information. Then, the feature images of the two domains are fused by a combination network.

RESULTS

The proposed method was validated on the Mayo dataset and the Piglet dataset. The denoising algorithm is optimal in both subjective and objective evaluation indexes as compared to other state-of-the-art methods reported in previous studies.

CONCLUSIONS

The study results demonstrate that by applying the new fusion model denoising, denoising results in both image domain and DCT domain are better than other models developed using features extracted in the single image domain.

Medical image super-resolution reconstruction algorithms based on deep learning: A survey

BACKGROUND AND OBJECTIVE

With the high-resolution (HR) requirements of medical images in clinical practice, super-resolution (SR) reconstruction algorithms based on low-resolution (LR) medical images have become a research hotspot. This type of method can significantly improve image SR without improving hardware equipment, so it is of great significance to review it.

METHODS

Aiming at the unique SR reconstruction algorithms in the field of medical images, based on subdivided medical fields such as magnetic resonance (MR) images, computed tomography (CT) images, and ultrasound images. Firstly, we deeply analyzed the research progress of SR reconstruction algorithms, and summarized and compared the different types of algorithms. Secondly, we introduced the evaluation indicators corresponding to the SR reconstruction algorithms. Finally, we prospected the development trend of SR reconstruction technology in the medical field.

RESULTS

The medical image SR reconstruction technology based on deep learning can provide more abundant lesion information, relieve the expert's diagnosis pressure, and improve the diagnosis efficiency and accuracy.

CONCLUSION

The medical image SR reconstruction technology based on deep learning helps to improve the quality of medicine, provides help for the diagnosis of experts, and lays a solid foundation for the subsequent analysis and identification tasks of the computer, which is of great significance for improving the diagnosis efficiency of experts and realizing intelligent medical care.

An optimized EBRSA-Bi LSTM model for highly undersampled rapid CT image reconstruction

COVID-19 has spread all over the world, causing serious panic around the globe. Chest computed tomography (CT) images are integral in confirming COVID positive patients. Several investigations were conducted to improve or maintain the image reconstruction quality for the sample image reconstruction. Deep learning (DL) methods have recently been proposed to achieve fast reconstruction, but many have focused on a single domain, such as the image domain of k-space. In this research, the highly under-sampled enhanced battle royale self-attention based bi-directional long short-term (EBRSA-bi LSTM) CT image reconstruction model is proposed to reconstruct the image from the under-sampled data. The research is adapted with two phases, namely, pre-processing and reconstruction. The extended cascaded filter (ECF) is proposed for image pre-processing and tends to suppress the noise and enhance the reconstruction accuracy. In the reconstruction model, the battle royale optimization (BrO) is intended to diminish the loss function of the reconstruction network model and weight updation. The proposed model is tested with two datasets, COVID-CT- and SARS-CoV-2 CT. The reconstruction accuracy of the proposed model with two datasets is 93.5 % and 97.7 %, respectively. Also, the image quality assessment parameters such as Peak-Signal to Noise Ratio (PSNR), Root Mean Square Error (RMSE) and Structural Similarity Index metric (SSIM) are evaluated, and it yields an outcome of (45 and 46 dB), (0.0026 and 0.0022) and (0.992, 0.996) with two datasets.

Generation of Whole-Body FDG Parametric Ki Images from Static PET Images Using Deep Learning

FDG parametric Ki images show great advantage over static SUV images, due to the higher contrast and better accuracy in tracer uptake rate estimation. In this study, we explored the feasibility of generating synthetic Ki images from static SUV ratio (SUVR) images using three configurations of U-Nets with different sets of input and output image patches, which were the U-Nets with single input and single output (SISO), multiple inputs and single output (MISO), and single input and multiple outputs (SIMO). SUVR images were generated by averaging three 5-min dynamic SUV frames starting at 60 minutes post-injection, and then normalized by the mean SUV values in the blood pool. The corresponding ground truth Ki images were derived using Patlak graphical analysis with input functions from measurement of arterial blood samples. Even though the synthetic Ki values were not quantitatively accurate compared with ground truth, the linear regression analysis of joint histograms in the voxels of body regions showed that the mean R2 values were higher between U-Net prediction and ground truth (0.596, 0.580, 0.576 in SISO, MISO and SIMO), than that between SUVR and ground truth Ki (0.571). In terms of similarity metrics, the synthetic Ki images were closer to the ground truth Ki images (mean SSIM = 0.729, 0.704, 0.704 in SISO, MISO and MISO) than the input SUVR images (mean SSIM = 0.691). Therefore, it is feasible to use deep learning networks to estimate surrogate map of parametric Ki images from static SUVR images.

De-Noising of Photoacoustic Microscopy Images by Attentive Generative Adversarial Network

As a hybrid imaging technology, photoacoustic microscopy (PAM) imaging suffers from noise due to the maximum permissible exposure of laser intensity, attenuation of ultrasound in the tissue, and the inherent noise of the transducer. De-noising is an image processing method to reduce noise, and PAM image quality can be recovered. However, previous de-noising techniques usually heavily rely on manually selected parameters, resulting in unsatisfactory and slow de-noising performance for different noisy images, which greatly hinders practical and clinical applications. In this work, we propose a deep learning-based method to remove noise from PAM images without manual selection of settings for different noisy images. An attention enhanced generative adversarial network is used to extract image features and adaptively remove various levels of Gaussian, Poisson, and Rayleigh noise. The proposed method is demonstrated on both synthetic and real datasets, including phantom (leaf veins) and in vivo (mouse ear blood vessels and zebrafish pigment) experiments. In the in vivo experiments using synthetic datasets, our method achieves the improvement of 6.53 dB and 0.26 in peak signal-to-noise ratio and structural similarity metrics, respectively. The results show that compared with previous PAM de-noising methods, our method exhibits good performance in recovering images qualitatively and quantitatively. In addition, the de-noising processing speed of 0.016 s is achieved for an image with 256×256 pixels, which has the potential for real-time applications. Our approach is effective and practical for the de-noising of PAM images.

Engineering an AI-based forward-reverse platform for the design of cross-ribosome binding sites of a transcription factor biosensor

A cross-ribosome binding site (cRBS) adjusts the dynamic range of transcription factor-based biosensors (TFBs) by controlling protein expression and folding. The rational design of a cRBS with desired TFB dynamic range remains an important issue in TFB forward and reverse engineering. Here, we report a novel artificial intelligence (AI)-based forward-reverse engineering platform for TFB dynamic range prediction and de novo cRBS design with selected TFB dynamic ranges. The platform demonstrated superior in processing unbalanced minority-class datasets and was guided by sequence characteristics from trained cRBSs. The platform identified correlations between cRBSs and dynamic ranges to mimic bidirectional design between these factors based on Wasserstein generative adversarial network (GAN) with a gradient penalty (GP) (WGAN-GP) and balancing GAN with GP (BAGAN-GP). For forward and reverse engineering, the predictive accuracy was up to 98% and 82%, respectively. Collectively, we generated an AI-based method for the rational design of TFBs with desired dynamic ranges.

A self-supervised guided knowledge distillation framework for unpaired low-dose CT image denoising

Low-dose computed tomography (LDCT) can significantly reduce the damage of X-ray to the human body, but the reduction of CT dose will produce images with severe noise and artifacts, which will affect the diagnosis of doctors. Recently, deep learning has attracted more and more attention from researchers. However, most of the denoising networks applied to deep learning-based LDCT imaging are supervised methods, which require paired data for network training. In a realistic imaging scenario, obtaining well-aligned image pairs is challenging due to the error in the table re-positioning and the patient's physiological movement during data acquisition. In contrast, the unpaired learning method can overcome the drawbacks of supervised learning, making it more feasible to collect unpaired training data in most real-world imaging applications. In this study, we develop a novel unpaired learning framework, Self-Supervised Guided Knowledge Distillation (SGKD), which enables the guidance of supervised learning using the results generated by self-supervised learning. The proposed SGKD scheme contains two stages of network training. First, we can achieve the LDCT image quality improvement by the designed self-supervised cycle network. Meanwhile, it can also produce two complementary training datasets from the unpaired LDCT and NDCT images. Second, a knowledge distillation strategy with the above two datasets is exploited to further improve the LDCT image denoising performance. To evaluate the effectiveness and feasibility of the proposed method, extensive experiments were performed on the simulated AAPM challenging and real-world clinical LDCT datasets. The qualitative and quantitative results show that the proposed SGKD achieves better performance in terms of noise suppression and detail preservation compared with some state-of-the-art network models.

Generative adversarial network with radiomic feature reproducibility analysis for computed tomography denoising

BACKGROUND

Most computed tomography (CT) denoising algorithms have been evaluated using image quality analysis (IQA) methods developed for natural image, which do not adequately capture the texture details in medical imaging. Radiomics is an emerging image analysis technique that extracts texture information to provide a more objective basis for medical imaging diagnostics, overcoming the subjective nature of traditional methods. By utilizing the difficulty of reproducing radiomics features under different imaging protocols, we can more accurately evaluate the performance of CT denoising algorithms.

METHOD

We introduced radiomic feature reproducibility analysis as an evaluation metric for a denoising algorithm. Also, we proposed a low-dose CT denoising method based on a generative adversarial network (GAN), which outperformed well-known CT denoising methods.

RESULTS

Although the proposed model produced excellent results visually, the traditional image assessment metrics such as peak signal-to-noise ratio and structural similarity failed to show distinctive performance differences between the proposed method and the conventional ones. However, radiomic feature reproducibility analysis provided a distinctive assessment of the CT denoising performance. Furthermore, radiomic feature reproducibility analysis allowed fine-tuning of the hyper-parameters of the GAN.

CONCLUSION

We demonstrated that the well-tuned GAN architecture outperforms the well-known CT denoising methods. Our study is the first to introduce radiomics reproducibility analysis as an evaluation metric for CT denoising. We look forward that the study may bridge the gap between traditional objective and subjective evaluations in the clinical medical imaging field.

Which Pixel to Annotate: A Label-Efficient Nuclei Segmentation Framework

Recently deep neural networks, which require a large amount of annotated samples, have been widely applied in nuclei instance segmentation of H&E stained pathology images. However, it is inefficient and unnecessary to label all pixels for a dataset of nuclei images which usually contain similar and redundant patterns. Although unsupervised and semi-supervised learning methods have been studied for nuclei segmentation, very few works have delved into the selective labeling of samples to reduce the workload of annotation. Thus, in this paper, we propose a novel full nuclei segmentation framework that chooses only a few image patches to be annotated, augments the training set from the selected samples, and achieves nuclei segmentation in a semi-supervised manner. In the proposed framework, we first develop a novel consistency-based patch selection method to determine which image patches are the most beneficial to the training. Then we introduce a conditional single-image GAN with a component-wise discriminator, to synthesize more training samples. Lastly, our proposed framework trains an existing segmentation model with the above augmented samples. The experimental results show that our proposed method could obtain the same-level performance as a fully-supervised baseline by annotating less than 5% pixels on some benchmarks.

Automated nonlinear registration of coronary PET to CT angiography using pseudo-CT generated from PET with generative adversarial networks

BACKGROUND

Coronary 18F-sodium-fluoride (18F-NaF) positron emission tomography (PET) showed promise in imaging coronary artery disease activity. Currently image processing remains subjective due to the need for manual registration of PET and computed tomography (CT) angiography data. We aimed to develop a novel fully automated method to register coronary 18F-NaF PET to CT angiography using pseudo-CT generated by generative adversarial networks (GAN).

METHODS

A total of 169 patients, 139 in the training and 30 in the testing sets were considered for generation of pseudo-CT from non-attenuation corrected (NAC) PET using GAN. Non-rigid registration was used to register pseudo-CT to CT angiography and the resulting transformation was used to align PET with CT angiography. We compared translations, maximal standard uptake value (SUVmax) and target to background ratio (TBRmax) at the location of plaques, obtained after observer and automated alignment.

RESULTS

Automatic end-to-end registration was performed for 30 patients with 88 coronary vessels and took 27.5 seconds per patient. Difference in displacement motion vectors between GAN-based and observer-based registration in the x-, y-, and z-directions was 0.8 ± 3.0, 0.7 ± 3.0, and 1.7 ± 3.9 mm, respectively. TBRmax had a coefficient of repeatability (CR) of 0.31, mean bias of 0.03 and narrow limits of agreement (LOA) (95% LOA: - 0.29 to 0.33). SUVmax had CR of 0.26, mean bias of 0 and narrow LOA (95% LOA: - 0.26 to 0.26).

CONCLUSION

Pseudo-CT generated by GAN are perfectly registered to PET can be used to facilitate quick and fully automated registration of PET and CT angiography.

Improving the diagnostic performance of computed tomography angiography for intracranial large arterial stenosis by a novel super-resolution algorithm based on multi-scale residual denoising generative adversarial network

BACKGROUND

Computed tomography angiography (CTA) is very popular because it is characterized by rapidity and accessibility. However, CTA is inferior to digital subtraction angiography (DSA) in the diagnosis of intracranial artery stenosis or occlusion. DSA is an invasive examination, so we optimized the quality of cephalic CTA images.

METHODS

We used 5000 CTA images to train multi-scale residual denoising generative adversarial network (MRDGAN). And then 71 CTA images with intracranial large arterial stenosis were treated by Super-Resolution based on Generative Adversarial Network (SRGAN), Enhanced Super-Resolution based on Generative Adversarial Network (ESRGAN) and post-trained MRDGAN, respectively. Peak signal-to-noise ratio (PSNR) and structural similarity index measurement (SSIM) of the SRGAN, ESRGAN, MRDGAN and original CTA images were measured respectively. The qualities of MRDGAN and original images were visually assessed using a 4-point scale. The diagnostic coherence of digital subtraction angiography (DSA) with MRDGAN and original images was analyzed.

RESULTS

The PSNR was significantly higher in the MRDGAN CTA images (35.96 ± 1.51) than in the original (31.51 ± 1.43), SRGAN (25.75 ± 1.18) and ESRGAN (30.36 ± 1.05) CTA images (all P < 0.001). The SSIM was significantly higher in the MRDGAN CTA images (0.95 ± 0.02) than in the SRGAN (0.88 ± 0.03) and ESRGAN (0.90 ± 0.02) CTA images (all P < 0.01). The visual assessment was significantly higher in the MRDGAN CTA images (3.52 ± 0.58) than in the original CTA images (2.39 ± 0.69) (P < 0.05). The diagnostic coherence between MRDGAN and DSA (κ = 0.89) was superior to that between original images and DSA (κ = 0.62).

CONCLUSION

Our MRDGAN can effectively optimize original CTA images and improve its clinical diagnostic value for intracranial large artery stenosis.

Phase Contrast Image Restoration by Formulating Its Imaging Principle and Reversing the Formulation With Deep Neural Networks

Phase contrast microscopy, as a noninvasive imaging technique, has been widely used to monitor the behavior of transparent cells without staining or altering them. Due to the optical principle of the specifically-designed microscope, phase contrast microscopy images contain artifacts such as halo and shade-off which hinder the cell segmentation and detection tasks. Some previous works developed simplified computational imaging models for phase contrast microscopes by linear approximations and convolutions. The approximated models do not exactly reflect the imaging principle of the phase contrast microscope and accordingly the image restoration by solving the corresponding deconvolution process is not perfect. In this paper, we revisit the optical principle of the phase contrast microscope to precisely formulate its imaging model without any approximation. Based on this model, we propose an image restoration procedure by reversing this imaging model with a deep neural network, instead of mathematically deriving the inverse operator of the model which is technically impossible. Extensive experiments are conducted to demonstrate the superiority of the newly derived phase contrast microscopy imaging model and the power of the deep neural network on modeling the inverse imaging procedure. Moreover, the restored images enable that high quality cell segmentation task can be easily achieved by simply thresholding methods. Implementations of this work are publicly available at https://github.com/LiangHann/Phase-Contrast-Microscopy-Image-Restoration.

New technique and application of truncated CBCT processing in adaptive radiotherapy for breast cancer

OBJECTIVE

A generative adversarial network (TCBCTNet) was proposed to generate synthetic computed tomography (sCT) from truncated low-dose cone-beam computed tomography (CBCT) and planning CT (pCT). The sCT was applied to the dose calculation of radiotherapy for patients with breast cancer.

METHODS

The low-dose CBCT and pCT images of 80 female thoracic patients were used for training. The CBCT, pCT, and replanning CT (rCT) images of 20 thoracic patients and 20 patients with breast cancer were used for testing. All patients were fixed in the same posture with a vacuum pad. The CBCT images were scanned under the Fast Chest M20 protocol with a 50% reduction in projection frames compared with the standard Chest M20 protocol. Rigid registration was performed between pCT and CBCT, and deformation registration was performed between rCT and CBCT. In the training stage of the TCBCTNet, truncated CBCT images obtained from complete CBCT images by simulation were used. The input of the CBCT→CT generator was truncated CBCT and pCT, and TCBCTNet was applied to patients with breast cancer after training. The accuracy of the sCT was evaluated by anatomy and dosimetry and compared with the generative adversarial network with UNet and ResNet as the generators (named as UnetGAN, ResGAN).

RESULTS

The three models could improve the image quality of CBCT and reduce the scattering artifacts while preserving the anatomical geometry of CBCT. For the chest test set, TCBCTNet achieved the best mean absolute error (MAE, 21.18±3.76 HU), better than 23.06±3.90 HU in UnetGAN and 22.47±3.57 HU in ResGAN. When applied to patients with breast cancer, TCBCTNet performance decreased, and MAE was 25.34±6.09 HU. Compared with rCT, sCT by TCBCTNet showed consistent dose distribution and subtle absolute dose differences between the target and the organ at risk. The 3D gamma pass rates were 98.98%±0.64% and 99.69%±0.22% at 2 mm/2% and 3 mm/3%, respectively. Ablation experiments confirmed that pCT and content loss played important roles in TCBCTNet.

CONCLUSIONS

High-quality sCT images could be synthesized from truncated low-dose CBCT and pCT by using the proposed TCBCTNet model. In addition, sCT could be used to accurately calculate the dose distribution for patients with breast cancer.

MLF-IOSC: Multi-Level Fusion Network With Independent Operation Search Cell for Low-Dose CT Denoising

Computed tomography (CT) is widely used in clinical medicine, and low-dose CT (LDCT) has become popular to reduce potential patient harm during CT acquisition. However, LDCT aggravates the problem of noise and artifacts in CT images, increasing diagnosis difficulty. Through deep learning, denoising CT images by artificial neural network has aroused great interest for medical imaging and has been hugely successful. We propose a framework to achieve excellent LDCT noise reduction using independent operation search cells, inspired by neural architecture search, and introduce the Laplacian to further improve image quality. Employing patch-based training, the proposed method can effectively eliminate CT image noise while retaining the original structures and details, hence significantly improving diagnosis efficiency and promoting LDCT clinical applications.

United multi-task learning for abdominal contrast-enhanced CT synthesis through joint deformable registration

Synthesizing abdominal contrast-enhanced computed tomography (CECT) images from non-enhanced CT (NECT) images is of great importance, in the delineation of radiotherapy target volumes, to reduce the risk of iodinated contrast agent and the registration error between NECT and CECT for transferring the delineations. NECT images contain structural information that can reflect the contrast difference between lesions and surrounding tissues. However, existing methods treat synthesis and registration as two separate tasks, which neglects the task collaborative and fails to address misalignment between images after the standard image pre-processing in training a CECT synthesis model. Thus, we propose an united multi-task learning (UMTL) for joint synthesis and deformable registration of abdominal CECT. Specifically, our UMTL is an end-to-end multi-task framework, which integrates a deformation field learning network for reducing the misalignment errors and a 3D generator for synthesizing CECT images. Furthermore, the learning of enhanced component images and the multi-loss function are adopted for enhancing the performance of synthetic CECT images. The proposed method is evaluated on two different resolution datasets and a separate test dataset from another center. The synthetic venous phase CECT images of the separate test dataset yield mean absolute error (MAE) of 32.78±7.27 HU, mean MAE of 24.15±5.12 HU on liver region, mean peak signal-to-noise rate (PSNR) of 27.59±2.45 dB, and mean structural similarity (SSIM) of 0.96±0.01. The Dice similarity coefficients of liver region between the true and synthetic venous phase CECT images are 0.96±0.05 (high-resolution) and 0.95±0.07 (low-resolution), respectively. The proposed method has great potential in aiding the delineation of radiotherapy target volumes.

EPRI sparse reconstruction method based on deep learning

Electron paramagnetic resonance imaging (EPRI) is an advanced tumor oxygen concentration imaging method. Now, the bottleneck problem of EPRI is that the scanning time is too long. Sparse reconstruction is an effective and fast imaging method, which means reconstructing images from sparse-view projections. However, the EPRI images sparsely reconstructed by the classic filtered back projection (FBP) algorithm often contain severe streak artifacts, which affect subsequent image processing. In this work, we propose a feature pyramid attention-based, residual, dense, deep convolutional network (FRD-Net) to suppress the streak artifacts in the FBP-reconstructed images. This network combines residual connection, attention mechanism, dense connections and introduces perceptual loss. The EPRI image with streak artifacts is used as the input of the network and the output-label is the corresponding high-quality image densely reconstructed by the FBP algorithm. After training, the FRD-Net gets the capability of suppressing streak artifacts. The real data reconstruction experiments show that the FRD-Net can better improve the sparse reconstruction accuracy, compared with three existing representative deep networks.

CTformer: convolution-free Token2Token dilated vision transformer for low-dose CT denoising

Objective. Low-dose computed tomography (LDCT) denoising is an important problem in CT research. Compared to the normal dose CT, LDCT images are subjected to severe noise and artifacts. Recently in many studies, vision transformers have shown superior feature representation ability over the convolutional neural networks (CNNs). However, unlike CNNs, the potential of vision transformers in LDCT denoising was little explored so far. Our paper aims to further explore the power of transformer for the LDCT denoising problem.Approach. In this paper, we propose a Convolution-free Token2Token Dilated Vision Transformer (CTformer) for LDCT denoising. The CTformer uses a more powerful token rearrangement to encompass local contextual information and thus avoids convolution. It also dilates and shifts feature maps to capture longer-range interaction. We interpret the CTformer by statically inspecting patterns of its internal attention maps and dynamically tracing the hierarchical attention flow with an explanatory graph. Furthermore, overlapped inference mechanism is employed to effectively eliminate the boundary artifacts that are common for encoder-decoder-based denoising models.Main results. Experimental results on Mayo dataset suggest that the CTformer outperforms the state-of-the-art denoising methods with a low computational overhead.Significance. The proposed model delivers excellent denoising performance on LDCT. Moreover, low computational cost and interpretability make the CTformer promising for clinical applications.

Training Generative Adversarial Networks via Stochastic Nash Games

Generative adversarial networks (GANs) are a class of generative models with two antagonistic neural networks: a generator and a discriminator. These two neural networks compete against each other through an adversarial process that can be modeled as a stochastic Nash equilibrium problem. Since the associated training process is challenging, it is fundamental to design reliable algorithms to compute an equilibrium. In this article, we propose a stochastic relaxed forward-backward (SRFB) algorithm for GANs, and we show convergence to an exact solution when an increasing number of data is available. We also show convergence of an averaged variant of the SRFB algorithm to a neighborhood of the solution when only a few samples are available. In both cases, convergence is guaranteed when the pseudogradient mapping of the game is monotone. This assumption is among the weakest known in the literature. Moreover, we apply our algorithm to the image generation problem.

Deep Learning Image Reconstruction for CT: Technical Principles and Clinical Prospects

Filtered back projection (FBP) has been the standard CT image reconstruction method for 4 decades. A simple, fast, and reliable technique, FBP has delivered high-quality images in several clinical applications. However, with faster and more advanced CT scanners, FBP has become increasingly obsolete. Higher image noise and more artifacts are especially noticeable in lower-dose CT imaging using FBP. This performance gap was partly addressed by model-based iterative reconstruction (MBIR). Yet, its "plastic" image appearance and long reconstruction times have limited widespread application. Hybrid iterative reconstruction partially addressed these limitations by blending FBP with MBIR and is currently the state-of-the-art reconstruction technique. In the past 5 years, deep learning reconstruction (DLR) techniques have become increasingly popular. DLR uses artificial intelligence to reconstruct high-quality images from lower-dose CT faster than MBIR. However, the performance of DLR algorithms relies on the quality of data used for model training. Higher-quality training data will become available with photon-counting CT scanners. At the same time, spectral data would greatly benefit from the computational abilities of DLR. This review presents an overview of the principles, technical approaches, and clinical applications of DLR, including metal artifact reduction algorithms. In addition, emerging applications and prospects are discussed.

Physics-/Model-Based and Data-Driven Methods for Low-Dose Computed Tomography: A survey

Since 2016, deep learning (DL) has advanced tomographic imaging with remarkable successes, especially in low-dose computed tomography (LDCT) imaging. Despite being driven by big data, the LDCT denoising and pure end-to-end reconstruction networks often suffer from the black box nature and major issues such as instabilities, which is a major barrier to apply deep learning methods in low-dose CT applications. An emerging trend is to integrate imaging physics and model into deep networks, enabling a hybridization of physics/model-based and data-driven elements. In this paper, we systematically review the physics/model-based data-driven methods for LDCT, summarize the loss functions and training strategies, evaluate the performance of different methods, and discuss relevant issues and future directions.

Transfer learning framework for low-dose CT reconstruction based on marginal distribution adaptation in multiscale

BACKGROUND

With the increasing use of computed tomography (CT) in clinical practice, limiting CT radiation exposure to reduce potential cancer risks has become one of the important directions of medical imaging research. As the dose decreases, the reconstructed CT image will be severely degraded by projection noise.

PURPOSE

As an important method of image processing, supervised deep learning has been widely used in the restoration of low-dose CT (LDCT) in recent years. However, the normal-dose CT (NDCT) corresponding to a specific LDCT (it is regarded as the label of the LDCT, which is necessary for supervised learning) is very difficult to obtain so that the application of supervised learning methods in LDCT reconstruction is limited. It is necessary to construct a unsupervised deep learning framework for LDCT reconstruction that does not depend on paired LDCT-NDCT datasets.

METHODS

We presented an unsupervised learning framework for the transferring from the identity mapping to the low-dose reconstruction task, called marginal distribution adaptation in multiscale (MDAM). For NDCTs as source domain data, MDAM is an identity map with two parts: firstly, it establishes a dimensionality reduction mapping, which can obtain the same feature distribution from NDCTs and LDCTs; and then NDCTs is retrieved by reconstructing the image overview and details from the low-dimensional features. For the purpose of the feature transfer between source domain and target domain (LDCTs), we introduce the multiscale feature extraction in the MDAM, and then eliminate differences in probability distributions of these multiscale features between NDCTs and LDCTs through wavelet decomposition and domain adaptation learning.

RESULTS

Image quality evaluation metrics and subjective quality scores show that, as an unsupervised method, the performance of the MDAM approaches or even surpasses some state-of-the-art supervised methods. Especially, MDAM has been favorably evaluated in terms of noise suppression, structural preservation, and lesion detection.

CONCLUSIONS

We demonstrated that, the MDAM framework can reconstruct corresponding NDCTs from LDCTs with high accuracy, and without relying on any labeles. Moreover, it is more suitable for clinical application compared with supervised learning methods.

Linearized Analysis of Noise and Resolution for DL-Based Image Generation

Deep-learning (DL) based CT image generation methods are often evaluated using RMSE and SSIM. By contrast, conventional model-based image reconstruction (MBIR) methods are often evaluated using image properties such as resolution, noise, bias. Calculating such image properties requires time consuming Monte Carlo (MC) simulations. For MBIR, linearized analysis using first order Taylor expansion has been developed to characterize noise and resolution without MC simulations. This inspired us to investigate if linearization can be applied to DL networks to enable efficient characterization of resolution and noise. We used FBPConvNet as an example DL network and performed extensive numerical evaluations, including both computer simulations and real CT data. Our results showed that network linearization works well under normal exposure settings. For such applications, linearization can characterize image noise and resolutions without running MC simulations. We provide with this work the computational tools to implement network linearization. The efficiency and ease of implementation of network linearization can hopefully popularize the physics-related image quality measures for DL applications. Our methodology is general; it allows flexible compositions of DL nonlinear modules and linear operators such as filtered-backprojection (FBP). For the latter, we develop a generic method for computing the covariance images that is needed for network linearization.

Ultrasound Volume Reconstruction From Freehand Scans Without Tracking

Transrectal ultrasound is commonly used for guiding prostate cancer biopsy, where 3D ultrasound volume reconstruction is often desired. Current methods for 3D reconstruction from freehand ultrasound scans require external tracking devices to provide spatial information of an ultrasound transducer. This paper presents a novel deep learning approach for sensorless ultrasound volume reconstruction, which efficiently exploits content correspondence between ultrasound frames to reconstruct 3D volumes without external tracking. The underlying deep learning model, deep contextual-contrastive network (DC 2-Net), utilizes self-attention to focus on the speckle-rich areas to estimate spatial movement and then minimizes a margin ranking loss for contrastive feature learning. A case-wise correlation loss over the entire input video helps further smooth the estimated trajectory. We train and validate DC 2-Net on two independent datasets, one containing 619 transrectal scans and the other having 100 transperineal scans. Our proposed approach attained superior performance compared with other methods, with a drift rate of 9.64 % and a prostate Dice of 0.89. The promising results demonstrate the capability of deep neural networks for universal ultrasound volume reconstruction from freehand 2D ultrasound scans without tracking information.

M3NAS: Multi-Scale and Multi-Level Memory-Efficient Neural Architecture Search for Low-Dose CT Denoising

Lowering the radiation dose in computed tomography (CT) can greatly reduce the potential risk to public health. However, the reconstructed images from dose-reduced CT or low-dose CT (LDCT) suffer from severe noise which compromises the subsequent diagnosis and analysis. Recently, convolutional neural networks have achieved promising results in removing noise from LDCT images. The network architectures that are used are either handcrafted or built on top of conventional networks such as ResNet and U-Net. Recent advances in neural network architecture search (NAS) have shown that the network architecture has a dramatic effect on the model performance. This indicates that current network architectures for LDCT may be suboptimal. Therefore, in this paper, we make the first attempt to apply NAS to LDCT and propose a multi-scale and multi-level memory-efficient NAS for LDCT denoising, termed M3NAS. On the one hand, the proposed M3NAS fuses features extracted by different scale cells to capture multi-scale image structural details. On the other hand, the proposed M3NAS can search a hybrid cell- and network-level structure for better performance. In addition, M3NAS can effectively reduce the number of model parameters and increase the speed of inference. Extensive experimental results on two different datasets demonstrate that the proposed M3NAS can achieve better performance and fewer parameters than several state-of-the-art methods. In addition, we also validate the effectiveness of the multi-scale and multi-level architecture for LDCT denoising, and present further analysis for different configurations of super-net.

TextureWGAN: texture preserving WGAN with multitask regularizer for computed tomography inverse problems

Purpose

This paper presents a deep learning (DL) based method called TextureWGAN. It is designed to preserve image texture while maintaining high pixel fidelity for computed tomography (CT) inverse problems. Over-smoothed images by postprocessing algorithms have been a well-known problem in the medical imaging industry. Therefore, our method tries to solve the over-smoothing problem without compromising pixel fidelity.

Approach

The TextureWGAN extends from Wasserstein GAN (WGAN). The WGAN can create an image that looks like a genuine image. This aspect of the WGAN helps preserve image texture. However, an output image from the WGAN is not correlated to the corresponding ground truth image. To solve this problem, we introduce the multitask regularizer (MTR) to the WGAN framework to make a generated image highly correlated to the corresponding ground truth image so that the TextureWGAN can achieve high-level pixel fidelity. The MTR is capable of using multiple objective functions. In this research, we adopt a mean squared error (MSE) loss to maintain pixel fidelity. We also use a perception loss to improve the look and feel of result images. Furthermore, the regularization parameters in the MTR are trained along with generator network weights to maximize the performance of the TextureWGAN generator.

Results

The proposed method was evaluated in CT image reconstruction applications in addition to super-resolution and image-denoising applications. We conducted extensive qualitative and quantitative evaluations. We used PSNR and SSIM for pixel fidelity analysis and the first-order and the second-order statistical texture analysis for image texture. The results show that the TextureWGAN is more effective in preserving image texture compared with other well-known methods such as the conventional CNN and nonlocal mean filter (NLM). In addition, we demonstrate that TextureWGAN can achieve competitive pixel fidelity performance compared with CNN and NLM. The CNN with MSE loss can attain high-level pixel fidelity, but it often damages image texture.

Conclusions

TextureWGAN can preserve image texture while maintaining pixel fidelity. The MTR is not only helpful to stabilize the TextureWGAN's generator training but also maximizes the generator performance.