Deep-learning-based cross-modality translation from Stokes image to bright-field contrast

Significance

Mueller matrix (MM) microscopy has proven to be a powerful tool for probing microstructural characteristics of biological samples down to subwavelength scale. However, in clinical practice, doctors usually rely on bright-field microscopy images of stained tissue slides to identify characteristic features of specific diseases and make accurate diagnosis. Cross-modality translation based on polarization imaging helps to improve the efficiency and stability in analyzing sample properties from different modalities for pathologists.

Aim

In this work, we propose a computational image translation technique based on deep learning to enable bright-field microscopy contrast using snapshot Stokes images of stained pathological tissue slides. Taking Stokes images as input instead of MM images allows the translated bright-field images to be unaffected by variations of light source and samples.

Approach

We adopted CycleGAN as the translation model to avoid requirements on co-registered image pairs in the training. This method can generate images that are equivalent to the bright-field images with different staining styles on the same region.

Results

Pathological slices of liver and breast tissues with hematoxylin and eosin staining and lung tissues with two types of immunohistochemistry staining, i.e., thyroid transcription factor-1 and Ki-67, were used to demonstrate the effectiveness of our method. The output results were evaluated by four image quality assessment methods.

Conclusions

By comparing the cross-modality translation performance with MM images, we found that the Stokes images, with the advantages of faster acquisition and independence from light intensity and image registration, can be well translated to bright-field images.

Enhanced CycleGAN Network with Adaptive Dark Channel Prior for Unpaired Single-Image Dehazing

Unpaired single-image dehazing has become a challenging research hotspot due to its wide application in modern transportation, remote sensing, and intelligent surveillance, among other applications. Recently, CycleGAN-based approaches have been popularly adopted in single-image dehazing as the foundations of unpaired unsupervised training. However, there are still deficiencies with these approaches, such as obvious artificial recovery traces and the distortion of image processing results. This paper proposes a novel enhanced CycleGAN network with an adaptive dark channel prior for unpaired single-image dehazing. First, a Wave-Vit semantic segmentation model is utilized to achieve the adaption of the dark channel prior (DCP) to accurately recover the transmittance and atmospheric light. Then, the scattering coefficient derived from both physical calculations and random sampling means is utilized to optimize the rehazing process. Bridged by the atmospheric scattering model, the dehazing/rehazing cycle branches are successfully combined to form an enhanced CycleGAN framework. Finally, experiments are conducted on reference/no-reference datasets. The proposed model achieved an SSIM of 94.9% and a PSNR of 26.95 on the SOTS-outdoor dataset and obtained an SSIM of 84.71% and a PSNR of 22.72 on the O-HAZE dataset. The proposed model significantly outperforms typical existing algorithms in both objective quantitative evaluation and subjective visual effect.

Image restoration of motion artifacts in cardiac arteries and vessels based on a generative adversarial network

BACKGROUND

When the heart rate of a patient exceeds the physical limits of a scanning device, even retrospective electrocardiography (ECG) gating technology cannot correct motion artifacts. The purpose of this study was to use deep learning methods to correct motion artifacts in coronary computed tomography angiography (CCTA) images acquired with retrospective ECG gating.

METHODS

To correct motion artifacts in CCTA images, we used a cycle Wasserstein generative adversarial network with a gradient penalty (WGAN-GP) to synthesize CCTA images without motion artifacts, and applied objective image indicators and clinical quantitative scores to evaluate the images. The objective image indicators included peak signal-to-noise ratio (PSNR), structural similarity (SSIM), and normalized mean square error (NMSE). For clinical quantitative scoring, we randomly selected 50 sets of images from the test data set as the scoring data set. We invited 2 radiologists from Zhongnan Hospital of Wuhan University to score the composite images.

RESULTS

In the test images, the PSNR, SSIM, NMSE and clinical quantitative score were 24.96±1.54, 0.769±0.055, 0.031±0.023, and 4.12±0.61, respectively. The images synthesized by cycle WGAN-GP performed better on objective image indicators and clinical quantitative scores than those synthesized by cycle least squares generative adversarial network (LSGAN), UNet, WGAN, and cycle WGAN.

CONCLUSIONS

Our proposed method can effectively correct the motion artifacts of coronary arteries in CCTA images and performs better than other methods. According to the performance of the clinical score, correction of images by this method does not affect the clinical diagnosis.

The synthesis of high-energy CT images from low-energy CT images using an improved cycle generative adversarial network

Background

The dose of radiation a patient receives when undergoing dual-energy computed tomography (CT) is of significant concern to the medical community, and balancing the tradeoffs between the level of radiation used and the quality of CT images is challenging. This paper proposes a method of synthesizing high-energy CT (HECT) images from low-energy CT (LECT) images using a neural network that achieves an alternative to HECT scanning by employing an LECT scan, which greatly reduces the radiation dose a patient receives.

Methods

In the training phase, the proposed structure cyclically generates HECT and LECT images to improve the accuracy of extracting edge and texture features. Specifically, we combine multiple connection methods with channel attention (CA) and pixel attention (PA) mechanisms to improve the network's mapping ability of image features. In the prediction phase, we use a model consisting of only the network component that synthesizes HECT images from LECT images.

Results

Our proposed method was conducted on clinical hip CT image data sets from Guizhou Provincial People's Hospital. In a comparison with other available methods [a generative adversarial network (GAN), a residual encoder-to-decoder network with a visual geometry group (VGG) pretrained model (RED-VGG), a Wasserstein GAN (WGAN), and CycleGAN] in terms of metrics of peak signal-to-noise ratio (PSNR), structural similarity index measure (SSIM), normalized mean square error (NMSE), and a visual effect evaluation, the proposed method was found to perform better on each of these evaluation criteria. Compared with the results produced by CycleGAN, the proposed method improved the PSNR by 2.44%, the SSIM by 1.71%, and the NMSE by 15.2%. Furthermore, the differences in the statistical indicators are statistically significant, proving the strength of the proposed method.

Conclusions

The proposed method synthesizes high-energy CT images from low-energy CT images, which significantly reduces both the cost of treatment and the radiation dose received by patients. Based on both image quality score metrics and visual effects comparisons, the results of the proposed method are superior to those obtained by other methods.

[Improvement in Image Quality of CBCT during Treatment by Cycle Generative Adversarial Network]

PURPOSE

Volumetric modulated arc therapy (VMAT) can acquire projection images during rotational irradiation, and cone-beam computed tomography (CBCT) images during VMAT delivery can be reconstructed. The poor quality of CBCT images prevents accurate recognition of organ position during the treatment. The purpose of this study was to improve the image quality of CBCT during the treatment by cycle generative adversarial network (CycleGAN).

METHOD

Twenty patients with clinically localized prostate cancer were treated with VMAT, and projection images for intra-treatment CBCT (iCBCT) were acquired. Synthesis of PCT (SynPCT) with improved image quality by CycleGAN requires only unpaired and unaligned iCBCT and planning CT (PCT) images for training. We performed visual and quantitative evaluation to compare iCBCT, SynPCT and PCT deformable image registration (DIR) to confirm the clinical usefulness.

RESULT

We demonstrated suitable CycleGAN networks and hyperparameters for SynPCT. The image quality of SynPCT improved visually and quantitatively while preserving anatomical structures of the original iCBCT. The undesirable deformation of PCT was reduced when SynPCT was used as its reference instead of iCBCT.

CONCLUSION

We have performed image synthesis with preservation of organ position by CycleGAN for iCBCT and confirmed the clinical usefulness.