Semantic Face Hallucination: Super-Resolving Very Low-Resolution Face Images with Supplementary Attributes

Dual-Path Deep Fusion Network for Face Image Hallucination

Along with the performance improvement of deep-learning-based face hallucination methods, various face priors (facial shape, facial landmark heatmaps, or parsing maps) have been used to describe holistic and partial facial features, making the cost of generating super-resolved face images expensive and laborious. To deal with this problem, we present a simple yet effective dual-path deep fusion network (DPDFN) for face image super-resolution (SR) without requiring additional face prior, which learns the global facial shape and local facial components through two individual branches. The proposed DPDFN is composed of three components: a global memory subnetwork (GMN), a local reinforcement subnetwork (LRN), and a fusion and reconstruction module (FRM). In particular, GMN characterize the holistic facial shape by employing recurrent dense residual learning to excavate wide-range context across spatial series. Meanwhile, LRN is committed to learning local facial components, which focuses on the patch-wise mapping relations between low-resolution (LR) and high-resolution (HR) space on local regions rather than the entire image. Furthermore, by aggregating the global and local facial information from the preceding dual-path subnetworks, FRM can generate the corresponding high-quality face image. Experimental results of face hallucination on public face data sets and face recognition on real-world data sets (VGGface and SCFace) show the superiority both on visual effect and objective indicators over the previous state-of-the-art methods.

RelightGAN: Instance-level Generative Adversarial Network for Face Illumination Transfer

Face illumination perception and processing is a significantly difficult issue especially due to asymmetric shadings, local highlights, and local shadows. This study focuses on the face illumination transfer problem, which is to transfer the illumination style from a reference face image to a target face image while preserving other attributes. Such an instance-level transfer task is more challenging than the domain-level one that only considers the pre-defined lighting categories. To tackle this problem, we develop an instance-level conditional Generative Adversarial Networks (GAN). Specifically, face identifier is integrated into GAN learning, which enables an individual-specific low-level visual generation. Moreover, the illumination-inspired attention mechanism is conducted to allow GAN to well handle the local lighting effect. Our method requires neither lighting categorization, 3D information, nor strict face alignment, which are often employed by traditional methods. Experiments demonstrate that our method achieves significantly better results than previous methods.