A benchmarking protocol for SAR colorization: From regression to deep learning approaches

Synthetic aperture radar (SAR) images are widely used in remote sensing. Interpreting SAR images can be challenging due to their intrinsic speckle noise and grayscale nature. To address this issue, SAR colorization has emerged as a research direction to colorize gray scale SAR images while preserving the original spatial information and radiometric information. However, this research field is still in its early stages, and many limitations can be highlighted. In this paper, we propose a full research line for supervised learning-based approaches to SAR colorization. Our approach includes a protocol for generating synthetic color SAR images, several baselines, and an effective method based on the conditional generative adversarial network (cGAN) for SAR colorization. We also propose numerical assessment metrics for the problem at hand. To our knowledge, this is the first attempt to propose a research line for SAR colorization that includes a protocol, a benchmark, and a complete performance evaluation. Our extensive tests demonstrate the effectiveness of our proposed cGAN-based network for SAR colorization. The code is available at https://github.com/shenkqtx/SAR-Colorization-Benchmarking-Protocol.

Stain normalization using score-based diffusion model through stain separation and overlapped moving window patch strategies

Hematoxylin and eosin (H&E) staining is the gold standard modality for diagnosis in medicine. However, the dosage ratio of hematoxylin to eosin in H&E staining has not been standardized yet. Additionally, H&E stains fade out at various speeds. Therefore, the staining quality could differ among each image, and stain normalization is a critical preprocessing approach for training deep learning (DL) models, especially in long-term and/or multicenter digital pathology studies. However, conventional methods for stain normalization have some significant drawbacks, such as collapsing in the structure and/or texture of tissue. In addition, conventional methods must require a reference patch or slide. Meanwhile, DL-based methods have a risk of overfitting and/or grid artifacts. We developed a score-based diffusion model of colorization for stain normalization. However, mistransfer, in which the model confuses hematoxylin with eosin, can occur using a score-based diffusion model due to its high diversity nature. To overcome this mistransfer, we propose a stain separation method using sparse non-negative matrix factorization (SNMF), which can decompose pathology slide into Hematoxylin and Eosin to normalize each stain component. Furthermore, inpainting with overlapped moving window patches was used to prevent grid artifacts of whole slide image normalization. Our method can normalize the whole slide pathology images through this stain normalization pipeline with decent performance.

Colorizing Grayscale CT images of human lungs using deep learning methods

Image colorization refers to computer-aided rendering technology which transfers colors from a reference color image to grayscale images or video frames. Deep learning elevated notably in the field of image colorization in the past years. In this paper, we formulate image colorization methods relying on exemplar colorization and automatic colorization, respectively. For hybrid colorization, we select appropriate reference images to colorize the grayscale CT images. The colours of meat resemble those of human lungs, so the images of fresh pork, lamb, beef, and even rotten meat are collected as our dataset for model training. Three sets of training data consisting of meat images are analysed to extract the pixelar features for colorizing lung CT images by using an automatic approach. Pertaining to the results, we consider numerous methods (i.e., loss functions, visual analysis, PSNR, and SSIM) to evaluate the proposed deep learning models. Moreover, compared with other methods of colorizing lung CT images, the results of rendering the images by using deep learning methods are significantly genuine and promising. The metrics for measuring image similarity such as SSIM and PSNR have satisfactory performance, up to 0.55 and 28.0, respectively. Additionally, the methods may provide novel ideas for rendering grayscale X-ray images in airports, ferries, and railway stations.

Improving the fidelity of CT image colorization based on pseudo-intensity model and tumor metabolism enhancement

BACKGROUND

Subject to the principle of imaging, most medical images are gray-scale images. Human eyes are more sensitive to color images compared to gray-scale images. The state-of-the-art medical image colorization results are unnatural and unrealistic, especially in some organs, such as the lung field.

METHOD

We propose a CT image colorization network that consists of a pseudo-intensity model, tumor metabolic enhancement, and MemoPainter-cGAN colorization network. First, the distributions of both the density of CT images and the intensity of anatomical images are analyzed with the aim of building a pseudo-intensity model. Then, the PET images, which are sensitive to tumor metabolism, are used to highlight the tumor regions. Finally, the MemoPainter-cGAN is used to generate colorized anatomical images.

RESULTS

Our experiment verified that the mean structural similarity between the colorized images and the original color images is 0.995, which indicates that the colorized image maintains the features of the original images enormously. The average image information entropy is 6.62, which is 13.4% higher than that of the images before metabolism enhancement and colorization. It indicates that the image fidelity is significantly improved.

CONCLUSIONS

Our method can generate vivid and fresh anatomical images based on prior knowledge of tissue or organ intensity. The colorized PET/CT images with abundant anatomical knowledge and high sensitivity of metabolic information provide radiologists with access to a new modality that offers additional reference information.

A practical PET/CT data visualization method with dual-threshold PET colorization and image fusion

Multi-modal medical imaging has emerged as a general trend in clinical diagnosis and treatment planning. In recent years, great efforts have been made to investigate and develop dual-modality scanners, among which PET/CT is the most widespread one in clinical practice. In this paper, we propose a simple yet effective PET/CT data visualization method that can integrate these two modalities into composite data for better observation. The proposed method consists of three main steps. First, a PET data colorization approach is presented based on a dual-threshold scheme, which applies a pair of high and low thresholds to colorize the PET image. Then, to extract functional information from the PET image more adequately, unlike traditional blending fashion that directly uses the CT image as underlay, we merge the CT and the PET images with a Laplacian pyramid (LP)-based image fusion approach to generate the underlay. Finally, the visualization result is obtained by blending the fused image and the colorized PET image. Experiments are conducted on 5 sets of PET/CT scans that contain 200 paired slices in total. The ClearCanvas software and the method using the presented PET colorization approach but with the CT image as underlay are adopted for comparison. Experimental results demonstrate that the proposed method can achieve more promising performance in terms of both visual perception and quantitative assessment. The code of the proposed method has been made available online athttps://github.com/yuliu316316/Visualization.

Computer-assisted coloring and illuminating based on a region-tree structure

Colorization and illumination are key processes for creating animated cartoons. Computer assisted methods have been incorporated in animation/illustration systems to reduce the artists' workload. This paper presents a new method for illumination and colorization of 2D drawings based on a region- tree representation. Starting from a hand-drawn cartoon, the proposed method extracts geometric and topological information and builds a tree structure, ensuring independence among parts of the drawing, such as curves and regions. Based on this structure and its attributes, a colorization method that propagates through consecutive frames of animation is proposed, combined with an interpolation method that accurately computes a normal mapping for the illumination process. Different operators for curve and region attributes can be applied independently, obtaining different rendering effects.