Adaptive Residual Networks for High-Quality Image Restoration

A Residual UNet Denoising Network Based on Multi-Scale Feature Extraction and Attention-Guided Filter

In order to obtain high-quality images, it is very important to remove noise effectively and retain image details reasonably. In this paper, we propose a residual UNet denoising network that adds the attention-guided filter and multi-scale feature extraction blocks. We design a multi-scale feature extraction block as the input block to expand the receiving domain and extract more useful features. We also develop the attention-guided filter block to hold the edge information. Further, we use the global residual network strategy to model residual noise instead of directly modeling clean images. Experimental results show our proposed network performs favorably against several state-of-the-art models. Our proposed model can not only suppress the noise more effectively, but also improve the sharpness of the image.

ARET-IQA: An Aspect-Ratio-Embedded Transformer for Image Quality Assessment

Image quality assessment (IQA) aims to automatically evaluate image perceptual quality by simulating the human visual system, which is an important research topic in the field of image processing and computer vision. Although existing deep-learning-based IQA models have achieved significant success, these IQA models usually require input images with a fixed size, which varies the perceptual quality of images. To this end, this paper proposes an aspect-ratio-embedded Transformer-based image quality assessment method, which can implant the adaptive aspect ratios of input images into the multihead self-attention module of the Swin Transformer. In this way, the proposed IQA model can not only relieve the variety of perceptual quality caused by size changes in input images but also leverage more global content correlations to infer image perceptual quality. Furthermore, to comprehensively capture the impact of low-level and high-level features on image quality, the proposed IQA model combines the output features of multistage Transformer blocks for jointly inferring image quality. Experimental results on multiple IQA databases show that the proposed IQA method is superior to state-of-the-art methods for assessing image technical and aesthetic quality.

Recovering Realistic Details for Magnification-Arbitrary Image Super-Resolution

The emergence of implicit neural representations (INR) has shown the potential to represent images in a continuous form by mapping pixel coordinates to RGB values. Recent work is capable of recovering arbitrary-resolution images from the continuous representations of the input low-resolution (LR) images. However, it can only super-resolve blurry images and lacks the ability to generate perceptual-pleasant details. In this paper, we propose implicit pixel flow (IPF) to model the coordinate dependency between the blurry INR distribution and the sharp real-world distribution. For each pixel near the blurry edges, IPF assigns offsets for the coordinates of the pixel so that the original RGB values can be replaced by the RGB values of a neighboring pixel which are more appropriate to form sharper edges. By modifying the relationship between the INR-domain coordinates and the image-domain pixels via IPF, we convert the original blurry INR distribution to a sharp one. Specifically, we adopt convolutional neural networks to extract continuous flow representations and employ multi-layer perceptrons to build the implicit function for calculating pixel flow. In addition, we propose a new double constraint module to search for more stable and optimal pixel flows during training. To the best of our knowledge, this is the first method to recover perceptually-pleasant details for magnification-arbitrary single image super-resolution. Experimental results on public benchmark datasets demonstrate that we successfully restore shape edges and satisfactory textures from continuous image representations.

Deep Learning Post-Filtering Using Multi-Head Attention and Multiresolution Feature Fusion for Image and Intra-Video Quality Enhancement

The paper proposes a novel post-filtering method based on convolutional neural networks (CNNs) for quality enhancement of RGB/grayscale images and video sequences. The lossy images are encoded using common image codecs, such as JPEG and JPEG2000. The video sequences are encoded using previous and ongoing video coding standards, high-efficiency video coding (HEVC) and versatile video coding (VVC), respectively. A novel deep neural network architecture is proposed to estimate fine refinement details for full-, half-, and quarter-patch resolutions. The proposed architecture is built using a set of efficient processing blocks designed based on the following concepts: (i) the multi-head attention mechanism for refining the feature maps, (ii) the weight sharing concept for reducing the network complexity, and (iii) novel block designs of layer structures for multiresolution feature fusion. The proposed method provides substantial performance improvements compared with both common image codecs and video coding standards. Experimental results on high-resolution images and standard video sequences show that the proposed post-filtering method provides average BD-rate savings of 31.44% over JPEG and 54.61% over HEVC (x265) for RGB images, Y-BD-rate savings of 26.21% over JPEG and 15.28% over VVC (VTM) for grayscale images, and 15.47% over HEVC and 14.66% over VVC for video sequences.

Cardiac Magnetic Resonance Images Superresolution via Multichannel Residual Attention Networks

The deep neural network has achieved good results in medical image superresolution. However, due to the medical equipment limitations and the complexity of the human body structure, it is difficult to reconstruct clear cardiac magnetic resonance (CMR) superresolution images. To reconstruct clearer CMR images, we propose a CMR image superresolution (SR) algorithm based on multichannel residual attention networks (MCRN), which uses the idea of residual learning to alleviate the difficulty of training and fully explore the feature information of the image and uses the back-projection learning mechanism to learn the interdependence between high-resolution images and low-resolution images. Furthermore, the MCRN model introduces an attention mechanism to dynamically allocate each feature map with different attention resources to discover more high-frequency information and learn the dependency between each channel of the feature map. Extensive benchmark evaluation shows that compared with state-of-the-art image SR methods, our MCRN algorithm not only improves the objective index significantly but also provides richer texture information for the reconstructed CMR images, and our MCRN algorithm is better than the Bicubic algorithm in evaluating the information entropy and average gradient of the reconstructed image quality.

MADNet: A Fast and Lightweight Network for Single-Image Super Resolution

Recently, deep convolutional neural networks (CNNs) have been successfully applied to the single-image super-resolution (SISR) task with great improvement in terms of both peak signal-to-noise ratio (PSNR) and structural similarity (SSIM). However, most of the existing CNN-based SR models require high computing power, which considerably limits their real-world applications. In addition, most CNN-based methods rarely explore the intermediate features that are helpful for final image recovery. To address these issues, in this article, we propose a dense lightweight network, called MADNet, for stronger multiscale feature expression and feature correlation learning. Specifically, a residual multiscale module with an attention mechanism (RMAM) is developed to enhance the informative multiscale feature representation ability. Furthermore, we present a dual residual-path block (DRPB) that utilizes the hierarchical features from original low-resolution images. To take advantage of the multilevel features, dense connections are employed among blocks. The comparative results demonstrate the superior performance of our MADNet model while employing considerably fewer multiadds and parameters.

Denoising Prior Driven Deep Neural Network for Image Restoration

Deep neural networks (DNNs) have shown very promising results for various image restoration (IR) tasks. However, the design of network architectures remains a major challenging for achieving further improvements. While most existing DNN-based methods solve the IR problems by directly mapping low quality images to desirable high-quality images, the observation models characterizing the image degradation processes have been largely ignored. In this paper, we first propose a denoising-based IR algorithm, whose iterative steps can be computed efficiently. Then, the iterative process is unfolded into a deep neural network, which is composed of multiple denoisers modules interleaved with back-projection (BP) modules that ensure the observation consistencies. A convolutional neural network (CNN) based denoiser that can exploit the multi-scale redundancies of natural images is proposed. As such, the proposed network not only exploits the powerful denoising ability of DNNs, but also leverages the prior of the observation model. Through end-to-end training, both the denoisers and the BP modules can be jointly optimized. Experimental results on several IR tasks, e.g., image denoisig, super-resolution and deblurring show that the proposed method can lead to very competitive and often state-of-the-art results on several IR tasks, including image denoising, deblurring, and super-resolution.

Linear Regression Supporting Vector Machine and Hybrid LOG Filter-Based Image Restoration

The image restoration (IR) technique is a part of image processing to improve the quality of an image that is affected by noise and blur. Thus, IR is required to attain a better quality of image. In this paper, IR is performed using linear regression-based support vector machine (LR-SVM). This LR-SVM has two steps: training and testing. The training and testing stages have a distinct windowing process for extracting blocks from the images. The LR-SVM is trained through a block-by-block training sequence. The extracted block-by-block values of images are used to enhance the classification process of IR. In training, the imperfections on the image are easily identified by setting the target vectors as the original images. Then, the noisy image is given at LR-SVM testing, based on the original image restored from the dictionary. Finally, the image block from the testing stage is enhanced using the hybrid Laplacian of Gaussian (HLOG) filter. The denoising of the HLOG filter provides enhanced results by using block-by-block values. This proposed approach is named as LR-SVM-HLOG. A dataset used in this LR-SVM-HLOG method is the Berkeley Segmentation Database. The performance of LR-SVM-HLOG was analyzed as peak signal-to-noise ratio (PSNR) and structural similarity index. The PSNR values of the house and pepper image (color image) are 40.82 and 36.56 dB, respectively, which are higher compared to the inter- and intra-block sparse estimation method and block matching and three-dimensional filtering for color images at 20% noise.

SCENet: Secondary Domain Intercorrelation Enhanced Network for Alleviating Compressed Poisson Noises

In real image coding systems, block-based coding is often applied on images contaminated by camera sensor noises such as Poisson noises, which cause complicated types of noises called compressed Poisson noises. Although many restoration methods have recently been proposed for compressed images, they do not provide satisfactory performance on the challenging compressed Poisson noises. This is mainly due to (i) inaccurate modeling regarding the image degradation, (ii) the signal-dependent noise property, and (iii) the lack of analysis on intercorrelation distortion. In this paper, we focused on the challenging issues in practical image coding systems and propose a compressed Poisson noise reduction scheme based on a secondary domain intercorrelation enhanced network. Specifically, we introduced a compressed Poisson noise corruption model and combined the secondary domain intercorrelation prior with a deep neural network especially designed for signal-dependent compression noise reduction. Experimental results showed that the proposed network is superior to the existing state-of-the-art restoration alternatives on classical images, the LIVE1 dataset, and the SIDD dataset.