Class-Aware Fully-Convolutional Gaussian and Poisson Denoising

Physics-informed deep neural network for image denoising

Image enhancement deep neural networks (DNN) can improve signal to noise ratio or resolution of optically collected visual information. The literature reports a variety of approaches with varying effectiveness. All these algorithms rely on arbitrary data (the pixels' count-rate) normalization, making their performance strngly affected by dataset or user-specific data pre-manipulation. We developed a DNN algorithm capable to enhance images signal-to-noise surpassing previous algorithms. Our model stems from the nature of the photon detection process which is characterized by an inherently Poissonian statistics. Our algorithm is thus driven by distance between probability functions instead than relying on the sole count-rate, producing high performance results especially in high-dynamic-range images. Moreover, it does not require any arbitrary image renormalization other than the transformation of the camera's count-rate into photon-number.

Disentangling Noise from Images: A Flow-Based Image Denoising Neural Network

The prevalent convolutional neural network (CNN)-based image denoising methods extract features of images to restore the clean ground truth, achieving high denoising accuracy. However, these methods may ignore the underlying distribution of clean images, inducing distortions or artifacts in denoising results. This paper proposes a new perspective to treat image denoising as a distribution learning and disentangling task. Since the noisy image distribution can be viewed as a joint distribution of clean images and noise, the denoised images can be obtained via manipulating the latent representations to the clean counterpart. This paper also provides a distribution-learning-based denoising framework. Following this framework, we present an invertible denoising network, FDN, without any assumptions on either clean or noise distributions, as well as a distribution disentanglement method. FDN learns the distribution of noisy images, which is different from the previous CNN-based discriminative mapping. Experimental results demonstrate FDN's capacity to remove synthetic additive white Gaussian noise (AWGN) on both category-specific and remote sensing images. Furthermore, the performance of FDN surpasses that of previously published methods in real image denoising with fewer parameters and faster speed.

Poisson Phase Retrieval in Very Low-count Regimes

This paper discusses phase retrieval algorithms for maximum likelihood (ML) estimation from measurements following independent Poisson distributions in very low-count regimes, e.g., 0.25 photon per pixel. To maximize the log-likelihood of the Poisson ML model, we propose a modified Wirtinger flow (WF) algorithm using a step size based on the observed Fisher information. This approach eliminates all parameter tuning except the number of iterations. We also propose a novel curvature for majorize-minimize (MM) algorithms with a quadratic majorizer. We show theoretically that our proposed curvature is sharper than the curvature derived from the supremum of the second derivative of the Poisson ML cost function. We compare the proposed algorithms (WF, MM) with existing optimization methods, including WF using other step-size schemes, quasi-Newton methods such as LBFGS and alternating direction method of multipliers (ADMM) algorithms, under a variety of experimental settings. Simulation experiments with a random Gaussian matrix, a canonical DFT matrix, a masked DFT matrix and an empirical transmission matrix demonstrate the following. 1) As expected, algorithms based on the Poisson ML model consistently produce higher quality reconstructions than algorithms derived from Gaussian noise ML models when applied to low-count data. Furthermore, incorporating regularizers, such as corner-rounded anisotropic total variation (TV) that exploit the assumed properties of the latent image, can further improve the reconstruction quality. 2) For unregularized cases, our proposed WF algorithm with Fisher information for step size converges faster (in terms of cost function and PSNR vs. time) than other WF methods, e.g., WF with empirical step size, backtracking line search, and optimal step size for the Gaussian noise model; it also converges faster than the LBFGS quasi-Newton method. 3) In regularized cases, our proposed WF algorithm converges faster than WF with backtracking line search, LBFGS, MM and ADMM.

End-to-End Convolutional Neural Network Model to Detect and Localize Myocardial Infarction Using 12-Lead ECG Images without Preprocessing

In recent years, many studies have proposed automatic detection and localization techniques for myocardial infarction (MI) using the 12-lead electrocardiogram (ECG). Most of them applied preprocessing to the ECG signals, e.g., noise removal, trend removal, beat segmentation, and feature selection, followed by model construction and classification based on machine-learning algorithms. The selection and implementation of preprocessing methods require specialized knowledge and experience to handle ECG data. In this paper, we propose an end-to-end convolutional neural network model that detects and localizes MI without such complicated multistep preprocessing. The proposed model executes comprehensive learning for the waveform features of unpreprocessed raw ECG images captured from 12-lead ECG signals. We evaluated the classification performance of the proposed model in two experimental settings: ten-fold cross-validation where ECG images were split randomly, and two-fold cross-validation where ECG images were split into one patient and the other patients. The experimental results demonstrate that the proposed model obtained MI detection accuracies of 99.82% and 93.93% and MI localization accuracies of 99.28% and 69.27% in the first and second settings, respectively. The performance of the proposed method is higher than or comparable to that of existing state-of-the-art methods. Thus, the proposed model is expected to be an effective MI diagnosis tool that can be used in intensive care units and as wearable technology.

DHNet: High-resolution and hierarchical network for cross-domain oct speckle noise reduction

PURPOSE

Optical coherence tomography (OCT) imaging uses the principle of Michelson interferometry to obtain high-resolution images by coherent superposing of multiple forward and backward scattered light waves with random phases. This process inevitably produces speckle noise that severely compromises visual quality of OCT images and degrades performances of subsequent image analysis tasks. In addition, datasets obtained by different OCT scanners have distribution shifts, making a speckle noise suppression model difficult to be generalized across multiple datasets. In order to solve the above issues, we propose a novel end-to-end denoising framework for OCT images collected by different scanners.

METHODS

The proposed model utilizes the high-resolution network (HRNet) as backbone for image restoration, which reconstructs high fidelity images by maintaining high-resolution representations throughout the entire learning process. To compensate distribution shifts among datasets collected by different scanners, we develop a hierarchical adversarial learning strategy for domain adaption. The proposed model is trained using datasets with clean ground truth produced by two commercial OCT scanners, and then applied to suppress speckle noise in OCT images collected by our recently developed OCT scanner, BV-1000 (China Bigvision Corporation). We name the proposed model as DHNet (Double-H-Net, High-resolution and Hierarchical Network).

RESULTS

We compare DHNet with state-of-the-art methods and experiment results show that DHNet improves signal-to-noise ratio (SNR) by a large margin of 18.137dB as compared to the best of our previous method. In addition, DHNet achieves a testing time of 25ms, which satisfies the real-time processing requirement for the BV-1000 scanner. We also conduct retinal layer segmentation experiment on OCT images before and after denoising and show that DHNet can also improve segmentation.

CONCLUSIONS

The proposed DHNet can compensate domain shifts between different datasets while significantly improve speckle noise suppression. The HRNet backbone is utilized to carry low- and high-resolution information to recover fidelity images. Domain adaptation is achieved by a hierarchical module through adversarial learning. In addition, DHNet achieved a testing time of 25ms, which satisfied the real-time processing requirement. This article is protected by copyright. All rights reserved.

Blind and Compact Denoising Network Based on Noise Order Learning

A lightweight blind image denoiser, called blind compact denoising network (BCDNet), is proposed in this paper to achieve excellent trade-offs between performance and network complexity. With only 330K parameters, the proposed BCDNet is composed of the compact denoising network (CDNet) and the guidance network (GNet). From a noisy image, GNet extracts a guidance feature, which encodes the severity of the noise. Then, using the guidance feature, CDNet filters the image adaptively according to the severity to remove the noise effectively. Moreover, by reducing the number of parameters without compromising the performance, CDNet achieves denoising not only effectively but also efficiently. Experimental results show that the proposed BCDNet yields state-of-the-art or competitive denoising performances on various datasets while requiring significantly fewer parameters.

Mobile Computational Photography: A Tour

The first mobile camera phone was sold only 20 years ago, when taking pictures with one's phone was an oddity, and sharing pictures online was unheard of. Today, the smartphone is more camera than phone. How did this happen? This transformation was enabled by advances in computational photography-the science and engineering of making great images from small-form-factor, mobile cameras. Modern algorithmic and computing advances, including machine learning, have changed the rules of photography, bringing to it new modes of capture, postprocessing, storage, and sharing. In this review, we give a brief history of mobile computational photography and describe some of the key technological components, including burst photography, noise reduction, and super-resolution. At each step, we can draw naive parallels to the human visual system.

Speckle Noise Reduction for OCT Images based on Image Style Transfer and Conditional GAN

Raw optical coherence tomography (OCT) images typically are of low quality because speckle noise blurs retinal structures, severely compromising visual quality and degrading performances of subsequent image analysis tasks. In our previous study, we have developed a Conditional Generative Adversarial Network (cGAN) for speckle noise removal in OCT images collected by several commercial OCT scanners, which we collectively refer to as scanner T. In this paper, we improve the cGAN model and apply it to our in-house OCT scanner (scanner B) for speckle noise suppression. The proposed model consists of two steps: 1) We train a Cycle-Consistent GAN (CycleGAN) to learn style transfer between two OCT image datasets collected by different scanners. The purpose of the CycleGAN is to leverage the ground truth dataset created in our previous study. 2) We train a mini-cGAN model based on the PatchGAN mechanism with the ground truth dataset to suppress speckle noise in OCT images. After training, we first apply the CycleGAN model to convert raw images collected by scanner B to match the style of the images from scanner T, and subsequently use the mini-cGAN model to suppress speckle noise in the style transferred images. We evaluate the proposed method on a dataset collected by scanner B. Experimental results show that the improved model outperforms our previous method and other state-of-the-art models in speckle noise removal, retinal structure preservation and contrast enhancement.

Ultra-Dense Denoising Network: Application to Cardiac Catheter-Based X-Ray Procedures

Reducing radiation dose in cardiac catheter-based X-ray procedures increases safety but also image noise and artifacts. Excessive noise and artifacts can compromise vital image information, which can affect clinical decision-making. Developing more effective X-ray denoising methodologies will be beneficial to both patients and healthcare professionals by allowing imaging at lower radiation dose without compromising image information. This paper proposes a framework based on a convolutional neural network (CNN), namely Ultra-Dense Denoising Network (UDDN), for low-dose X-ray image denoising. To promote feature extraction, we designed a novel residual block which establishes a solid correlation among multiple-path neural units via abundant cross connections in its representation enhancement section. Experiments on synthetic additive noise X-ray data show that the UDDN achieves statistically significant higher peak signal-to-noise ratio (PSNR) and structural similarity index measure (SSIM) than other comparative methods. We enhanced the clinical adaptability of our framework by training using normally-distributed noise and tested on clinical data taken from procedures at St. Thomas' hospital in London. The performance was assessed by using local SNR and by clinical voting using ten cardiologists. The results show that the UDDN outperforms the other comparative methods and is a promising solution to this challenging but clinically impactful task.

Deep learning on image denoising: An overview

Deep learning techniques have received much attention in the area of image denoising. However, there are substantial differences in the various types of deep learning methods dealing with image denoising. Specifically, discriminative learning based on deep learning can ably address the issue of Gaussian noise. Optimization models based on deep learning are effective in estimating the real noise. However, there has thus far been little related research to summarize the different deep learning techniques for image denoising. In this paper, we offer a comparative study of deep techniques in image denoising. We first classify the deep convolutional neural networks (CNNs) for additive white noisy images; the deep CNNs for real noisy images; the deep CNNs for blind denoising and the deep CNNs for hybrid noisy images, which represents the combination of noisy, blurred and low-resolution images. Then, we analyze the motivations and principles of the different types of deep learning methods. Next, we compare the state-of-the-art methods on public denoising datasets in terms of quantitative and qualitative analyses. Finally, we point out some potential challenges and directions of future research.

High-ISO Long-Exposure Image Denoising Based on Quantitative Blob Characterization

Blob detection and image denoising are fundamental, sometimes related tasks in computer vision. In this paper, we present a computational method to quantitatively measure blob characteristics using normalized unilateral second-order Gaussian kernels. This method suppresses non-blob structures while yielding a quantitative measurement of the position, prominence and scale of blobs, which can facilitate the tasks of blob reconstruction and blob reduction. Subsequently, we propose a denoising scheme to address high-ISO long-exposure noise, which sometimes spatially shows a blob appearance, employing a blob reduction procedure as a cheap preprocessing for conventional denoising methods. We apply the proposed denoising methods to real-world noisy images as well as standard images that are corrupted by real noise. The experimental results demonstrate the superiority of the proposed methods over state-of-the-art denoising methods.

Blind Universal Bayesian Image Denoising with Gaussian Noise Level Learning

Blind and universal image denoising consists of using a unique model that denoises images with any level of noise. It is especially practical as noise levels do not need to be known when the model is developed or at test time. We propose a theoretically-grounded blind and universal deep learning image denoiser for additive Gaussian noise removal. Our network is based on an optimal denoising solution, which we call fusion denoising. It is derived theoretically with a Gaussian image prior assumption. Synthetic experiments show our network's generalization strength to unseen additive noise levels. We also adapt the fusion denoising network architecture for image denoising on real images. Our approach improves real-world grayscale additive image denoising PSNR results for training noise levels and further on noise levels not seen during training. It also improves state-of-the-art color image denoising performance on every single noise level, by an average of 0.1dB, whether trained on or not.

Connecting Image Denoising and High-Level Vision Tasks via Deep Learning

Image denoising and high-level vision tasks are usually handled independently in the conventional practice of computer vision, and their connection is fragile. In this paper, we cope with the two jointly and explore the mutual influence between them with the focus on two questions, namely (1) how image denoising can help improving high-level vision tasks, and (2) how the semantic information from high-level vision tasks can be used to guide image denoising. First for image denoising we propose a convolutional neural network in which convolutions are conducted in various spatial resolutions via downsampling and upsampling operations in order to fuse and exploit contextual information on different scales. Second we propose a deep neural network solution that cascades two modules for image denoising and various high-level tasks, respectively, and use the joint loss for updating only the denoising network via backpropagation. We experimentally show that on one hand, the proposed denoiser has the generality to overcome the performance degradation of different high-level vision tasks. On the other hand, with the guidance of high-level vision information, the denoising network produces more visually appealing results. Extensive experiments demonstrate the benefit of exploiting image semantics simultaneously for image denoising and highlevel vision tasks via deep learning. The code is available online: https://github.com/Ding-Liu/DeepDenoising.

Photon-limited imaging through scattering medium based on deep learning

Imaging under ultra-weak light conditions is affected by Poisson noise heavily. The problem becomes worse if a scattering media is present in the optical path. Speckle patterns detected under ultra-weak light condition carry very little information which makes it difficult to reconstruct the image. Off-the-shelf methods are no longer available in this condition. In this paper, we experimentally demonstrate the use of a deep learning network to reconstruct images through scattering media under ultra-weak light illumination. The weak light limitation of this method is analyzed. The random Poisson detection under weak light condition obtains partial information of the object. Based on this property, we demonstrated better performance of our method by enlarging the training dataset with multiple detections of the speckle patterns. Our results demonstrate that our approach can reconstruct images through scattering media from close to 1 detected signal photon per pixel (PPP) per image.