PrivacyNet: Semi-Adversarial Networks for Multi-attribute Face Privacy

All Roads Lead to Rome: Achieving 3D Object Encryption Through 2D Image Encryption Methods

In this paper, we explore a new road for format-compatible 3D object encryption by proposing a novel mechanism of leveraging 2D image encryption methods. It alleviates the difficulty of designing 3D object encryption schemes coming from the intrinsic intricacy of the data structure, and implements the flexible and diverse 3D object encryption designs. First, turning complexity into simplicity, the vertex values, real numbers with continuous values, are converted into integers ranging from 0 to 255. The simplification result for a 3D object is a 2D numerical matrix. Second, six prototypes for three encryption patterns (permutation, diffusion, and permutation-diffusion) are designed as exemplifications to encrypt the 2D matrix. Third, the integer-valued elements in the encrypted numeric matrix are converted into real numbers complying with the syntax of the 3D object. In addition, some experiments are conducted to verify the effectiveness of the proposed mechanism.

Natural Adversarial Mask for Face Identity Protection in Physical World

Facial recognition (FR) technology offers convenience in our daily lives, but it also raises serious privacy issues due to unauthorized FR applications. To protect facial privacy, existing methods have proposed adversarial face examples that can fool FR systems. However, most of these methods work only in the digital domain and do not consider natural physical protections. In this paper, we present NatMask, a 3D-based method for creating natural and realistic adversarial face masks that can preserve facial identity in the physical world. Our method utilizes 3D face reconstruction and differentiable rendering to generate 2D face images with natural-looking facial masks. Moreover, we propose an identity-aware style injection (IASI) method to improve the naturalness and transferability of the mask texture. We evaluate our method on two face datasets to verify its effectiveness in protecting face identity against four state-of-the-art (SOTA) FR models and three commercial FR APIs in both digital and physical domains under black-box impersonation and dodging strategies. Experiments show that our method can generate adversarial masks with superior naturalness and physical realizability to safeguard face identity, outperforming SOTA methods by a large margin.

Reversible gender privacy enhancement via adversarial perturbations

The significant advancement in deep learning has made it feasible to extract gender from faces accurately. However, such unauthorized extraction would pose potential threats to individual privacy. Existing protection schemes for gender privacy have exhibited satisfactory performance. Nevertheless, they suffer from gender inference from gender-related attributes and fail to support the recovery of the original image. In this paper, we propose a novel gender privacy protection scheme that aims to enhance gender privacy while supporting reversibility. Firstly, our scheme utilizes continuously optimized adversarial perturbations to prevent gender recognition from unauthorized classifiers. Meanwhile, gender-related attributes are concealed for classifiers, which prevents the inference of gender from these attributes, thereby enhancing gender privacy. Moreover, an identity preservation constraint is added to maintain identity preservation. Secondly, reversibility is supported by a reversible image transformation, allowing the perturbations to be securely removed to losslessly recover the original face when required. Extensive experiments demonstrate the effectiveness of our scheme in gender privacy protection, identity preservation, and reversibility.

MaskFaceGAN: High-Resolution Face Editing With Masked GAN Latent Code Optimization

Face editing represents a popular research topic within the computer vision and image processing communities. While significant progress has been made recently in this area, existing solutions: (i) are still largely focused on low-resolution images, (ii) often generate editing results with visual artefacts, or (iii) lack fine-grained control over the editing procedure and alter multiple (entangled) attributes simultaneously, when trying to generate the desired facial semantics. In this paper, we aim to address these issues through a novel editing approach, called MaskFaceGAN that focuses on local attribute editing. The proposed approach is based on an optimization procedure that directly optimizes the latent code of a pre-trained (state-of-the-art) Generative Adversarial Network (i.e., StyleGAN2) with respect to several constraints that ensure: (i) preservation of relevant image content, (ii) generation of the targeted facial attributes, and (iii) spatially-selective treatment of local image regions. The constraints are enforced with the help of an (differentiable) attribute classifier and face parser that provide the necessary reference information for the optimization procedure. MaskFaceGAN is evaluated in extensive experiments on the FRGC, SiblingsDB-HQf, and XM2VTS datasets and in comparison with several state-of-the-art techniques from the literature. Our experimental results show that the proposed approach is able to edit face images with respect to several local facial attributes with unprecedented image quality and at high-resolutions ( 1024×1024 ), while exhibiting considerably less problems with attribute entanglement than competing solutions. The source code is publicly available from: https://github.com/MartinPernus/MaskFaceGAN.

StegEdge: Privacy protection of unknown sensitive attributes in edge intelligence via deception

Due to the limited capabilities of user devices, such as smart phones, and the Internet of Things (IoT), edge intelligence is being recognized as a promising paradigm to enable effective analysis of the data generated by these devices with complex artificial intelligence (AI) models, and it often entails either fully or partially offloading the computation of neural networks from user devices to edge computing servers. To protect users’ data privacy in the process, most existing researches assume that the private (sensitive) attributes of user data are known in advance when designing privacy-protection measures. This assumption is restrictive in real life, and thus limits the application of these methods. Inspired by the research in image steganography and cyber deception, in this paper, we propose StegEdge, a conceptually novel approach to this challenge. StegEdge takes as input the user-generated image and a randomly selected “cover” image that does not pose any privacy concern (e.g., downloaded from the Internet), and extracts the features such that the utility tasks can still be conducted by the edge computing servers, while potential adversaries seeking to reconstruct/recover the original user data or analyze sensitive attributes from the extracted features sent from users to the server, will largely acquire information of the cover image. Thus, users’ data privacy is protected via a form of deception. Empirical results conducted on the CelebA and ImageNet datasets show that, at the same level of accuracy for utility tasks, StegEdge reduces the adversaries’ accuracy of predicting sensitive attributes by up to 38% compared with other methods, while also defending against adversaries seeking to reconstruct user data from the extracted features.

Machine learning and AI-based approaches for bioactive ligand discovery and GPCR-ligand recognition

In the last decade, machine learning and artificial intelligence applications have received a significant boost in performance and attention in both academic research and industry. The success behind most of the recent state-of-the-art methods can be attributed to the latest developments in deep learning. When applied to various scientific domains that are concerned with the processing of non-tabular data, for example, image or text, deep learning has been shown to outperform not only conventional machine learning but also highly specialized tools developed by domain experts. This review aims to summarize AI-based research for GPCR bioactive ligand discovery with a particular focus on the most recent achievements and research trends. To make this article accessible to a broad audience of computational scientists, we provide instructive explanations of the underlying methodology, including overviews of the most commonly used deep learning architectures and feature representations of molecular data. We highlight the latest AI-based research that has led to the successful discovery of GPCR bioactive ligands. However, an equal focus of this review is on the discussion of machine learning-based technology that has been applied to ligand discovery in general and has the potential to pave the way for successful GPCR bioactive ligand discovery in the future. This review concludes with a brief outlook highlighting the recent research trends in deep learning, such as active learning and semi-supervised learning, which have great potential for advancing bioactive ligand discovery.