Physics-Based Noise Modeling for Extreme Low-Light Photography

Noise Self-Regression: A New Learning Paradigm to Enhance Low-Light Images Without Task-Related Data

Deep learning-based low-light image enhancement (LLIE) is a task of leveraging deep neural networks to enhance the image illumination while keeping the image content unchanged. From the perspective of training data, existing methods complete the LLIE task driven by one of the following three data types: paired data, unpaired data and zero-reference data. Each type of these data-driven methods has its own advantages, e.g., zero-reference data-based methods have very low requirements on training data and can meet the human needs in many scenarios. In this paper, we leverage pure Gaussian noise to complete the LLIE task, which further reduces the requirements for training data in LLIE tasks and can be used as another alternative in practical use. Specifically, we propose Noise SElf-Regression (NoiSER) without access to any task-related data, simply learns a convolutional neural network equipped with an instance-normalization layer by taking a random noise image, for each pixel, as both input and output for each training pair, and then the low-light image is fed to the trained network for predicting the normal-light image. Technically, an intuitive explanation for its effectiveness is as follows: 1) the self-regression reconstructs the contrast between adjacent pixels of the input image, 2) the instance-normalization layer may naturally remediate the overall magnitude/lighting of the input image, and 3) the assumption for each pixel enforces the output image to follow the well-known gray-world hypothesis (Buchsbaum, 1980) when the image size is big enough. Compared to current state-of-the-art LLIE methods with access to different task-related data, NoiSER is highly competitive in enhancement quality, yet with a much smaller model size, and much lower training and inference cost. In addition, the experiments also demonstrate that NoiSER has great potential in overexposure suppression and joint processing with other restoration tasks.

LGIT: local-global interaction transformer for low-light image denoising

Transformer-based methods effectively capture global dependencies in images, demonstrating outstanding performance in multiple visual tasks. However, existing Transformers cannot effectively denoise large noisy images captured under low-light conditions owing to (1) the global self-attention mechanism causing high computational complexity in the spatial dimension owing to a quadratic increase in computation with the number of tokens; (2) the channel-wise self-attention computation unable to optimise the spatial correlations in images. We propose a local-global interaction Transformer (LGIT) that employs an adaptive strategy to select relevant patches for global interaction, achieving low computational complexity in global self-attention computation. A top-N patch cross-attention model (TPCA) is designed based on superpixel segmentation guidance. TPCA selects top-N patches most similar to the target image patch and applies cross attention to aggregate information from them into the target patch, effectively enhancing the utilisation of the image's nonlocal self-similarity. A mixed-scale dual-gated feedforward network (MDGFF) is introduced for the effective extraction of multiscale local correlations. TPCA and MDGFF were combined to construct a hierarchical encoder-decoder network, LGIT, to compute self-attention within and across patches at different scales. Extensive experiments using real-world image-denoising datasets demonstrated that LGIT outperformed state-of-the-art (SOTA) convolutional neural network (CNN) and Transformer-based methods in qualitative and quantitative results.

Low-frequency background estimation and noise separation from high-frequency for background and noise subtraction

In fluorescence microscopy, background blur and noise are two main factors preventing the achievement of high-signal-to-noise ratio (SNR) imaging. Background blur primarily emanates from inherent factors including the spontaneous fluorescence of biological samples and out-of-focus backgrounds, while noise encompasses Gaussian and Poisson noise components. To achieve background blur subtraction and denoising simultaneously, a pioneering algorithm based on low-frequency background estimation and noise separation from high-frequency (LBNH-BNS) is presented, which effectively disentangles noise from the desired signal. Furthermore, it seamlessly integrates low-frequency features derived from background blur estimation, leading to the effective elimination of noise and background blur in wide-field fluorescence images. In comparisons with other state-of-the-art background removal algorithms, LBNH-BNS demonstrates significant advantages in key quantitative metrics such as peak signal-to-noise ratio (PSNR) and manifests substantial visual enhancements. LBNH-BNS holds immense potential for advancing the overall performance and quality of wide-field fluorescence imaging techniques.