Enhanced Non-Local Total Variation Model and Multi-Directional Feature Prediction Prior for Single Image Super Resolution

Image Interpolation with Regional Gradient Estimation

This paper proposes an image interpolation method with regional gradient estimation (GEI) to solve the problem of the nonlinear interpolation method not sufficiently considering non-edge pixels. First, the approach presented in this paper expanded on the edge diffusion idea used in CGI and proposed a regional gradient estimation strategy to improve the problem of gradient calculation in the CGI method. Next, the gradient value was used to determine whether a pixel was an edge pixel. Then, a 1D directional filter was employed to process edge pixels while interpolating non-edge pixels using a 2D directionless filter. Finally, we experimented with various representative interpolation methods for grayscale and color images, including the one presented in this paper, and compared them in terms of subjective results, objective criteria, and computational complexity. The experimental results showed that GEI performed better than the other methods in an experiment concerning the visual effect, objective criteria, and computational complexity.

Enhanced Edge Detection for 3D Crack Segmentation and Depth Measurement with Laser Data

With the usage of computer visual technology in civil engineering, pavement crack survey with imaging sensor gained wide attention in the past years. Unfortunately, it is still a challenge to achieve the satisfying results. This paper presents a pavement crack survey approach based on edge detection for laser data. At first, the LS-40 line-laser scanner is implemented to achieve 3D pavement surface data. With the advantage of the various depth information exhibited in the pavement data, an enhanced edge detection based on fractional differential is proposed for 3D crack segmentation. The proposed method could effectively enhance the crack boundary and maintain texture details, which can guarantee the high accuracy in crack segmentation. Moreover, a novel plane fitting method based on dynamic threshold is studied to calculate crack depth information. It can not only identify and remove invalid points effectively, but also accomplish plane calculation with errors in three directions. Experiments verify that the proposed approach can achieve the satisfying result and can work well in F-measure system.

Video Satellite Imagery Super-Resolution via Model-Based Deep Neural Networks

Video satellite imagery has become a hot research topic in Earth observation due to its ability to capture dynamic information. However, its high temporal resolution comes at the expense of spatial resolution. In recent years, deep learning (DL) based super-resolution (SR) methods have played an essential role to improve the spatial resolution of video satellite images. Instead of fully considering the degradation process, most existing DL-based methods attempt to learn the relationship between low-resolution (LR) satellite video frames and their corresponding high-resolution (HR) ones. In this paper, we propose model-based deep neural networks for video satellite imagery SR (VSSR). The VSSR is composed of three main modules: degradation estimation module, intermediate image generation module, and multi-frame feature fusion module. First, the blur kernel and noise level of LR video frames are flexibly estimated by the degradation estimation module. Second, an intermediate image generation module is proposed to iteratively solve two optimal subproblems and the outputs of this module are intermediate SR frames. Third, a three-dimensional (3D) feature fusion subnetwork is leveraged to fuse the features from multiple video frames. Different from previous video satellite SR methods, the proposed VSSR is a multi-frame-based method that can merge the advantages of both learning-based and model-based methods. Experiments on real-world Jilin-1 and OVS-1 video satellite images have been conducted and the SR results demonstrate that the proposed VSSR achieves superior visual effects and quantitative performance compared with the state-of-the-art methods.

Adversarial Multi-Path Residual Network for Image Super-Resolution

Recently, deep convolutional neural networks have demonstrated remarkable progresses on single image super-resolution (SR) problem. However, most of them use more deeper and wider networks to improve SR performance, which is not practical in real-world applications due to large complexity, high computation cost, and low efficiency. In addition, they cannot provide high perception quality and guarantee objective quality simultaneously. To address these limitations, we in this paper propose a novel Adversarial Multipath Residual Network (AMPRN), which can largely suppress the number of network parameters and achieve a higher SR performance compared with the state-of-the-art methods. More specifically, we propose a multi-path residual block (MPRB) for multi-path residual network (MPRN) with fewer network parameters, which can extract abundant local features by fully using features from different paths generated by channel slices. These hierarchical features from all the MPRBs are then jointly aggregated by global gradual feature fusion. Following MPRN, we construct an adversarial gradient network with a gradient loss to make the gradient distribution of the generated SR images and ground truth image closer. In this way, the generated SR images of our model can provide high perception quality and objective quality. Finally, several experimental results demonstrate that our AMPRN achieves better performance in comparison with fewer parameters than the state-of-the-art methods.

Reconstruction of porous media from extremely limited information using conditional generative adversarial networks

Porous media are ubiquitous in both nature and engineering applications. Therefore, their modeling and understanding is of vital importance. In contrast to direct acquisition of three-dimensional (3D) images of this type of medium, obtaining its subregion (s) such as 2D images or several small areas can be feasible. Therefore, reconstructing whole images from limited information is a primary technique in these types of cases. Given data in practice cannot generally be determined by users and may be incomplete or only partially informed, thus making existing reconstruction methods inaccurate or even ineffective. To overcome this shortcoming, in this study we propose a deep-learning-based framework for reconstructing full images from their much smaller subareas. In particular, conditional generative adversarial network is utilized to learn the mapping between the input (a partial image) and output (a full image). To ensure the reconstruction accuracy, two simple but effective objective functions are proposed and then coupled with the other two functions to jointly constrain the training procedure. Because of the inherent essence of this ill-posed problem, a Gaussian noise is introduced for producing reconstruction diversity, thus enabling the network to provide multiple candidate outputs. Our method is extensively tested on a variety of porous materials and validated by both visual inspection and quantitative comparison. It is shown to be accurate, stable, and even fast (∼0.08 s for a 128×128 image reconstruction). The proposed approach can be readily extended by, for example, incorporating user-defined conditional data and an arbitrary number of object functions into reconstruction, while being coupled with other reconstruction methods.