Perception and memory-based representations of facial emotions: Associations with personality functioning, affective states and recognition abilities

Attention and stimulus structure interact during ensemble encoding of facial expression

Face ensemble encoding involves synthesizing summary information from groups of faces, providing a mechanism to overcome limitations in visual working memory. Yet, research on the role of attention has revealed mixed findings. Also, the simultaneous processing of summary representations and individual faces within an ensemble remains largely unexplored. Here, across three experiments, participants viewed ensembles without a central face (n = 32), or with a central face, while attention was distributed across the entire ensemble (n = 38) or focused centrally (n = 38). Critically, the consistency of center and surround faces varied as a function of emotional valence (i.e., same versus opposite). Participants completed an expression similarity-rating task between an ensemble and a single face, which was used to recover, via image reconstruction, visual estimates of summary representations. Reconstructions were then assessed against central faces, surrounding faces, and their averages. We show that focused attention enhances central face representation and that consistency benefits the representation of both the center and of the surround. However, central faces outweigh the overall surround representation only when attention is focused on a center face inconsistent with its surround. These findings reveal a flexible relationship between attention and stimulus structure in ensemble perception.

Unraveling other-race face perception with GAN-based image reconstruction

The other-race effect (ORE) is the disadvantage of recognizing faces of another race than one's own. While its prevalence is behaviorally well documented, the representational basis of ORE remains unclear. This study employs StyleGAN2, a deep learning technique for generating photorealistic images to uncover face representations and to investigate ORE's representational basis. To this end, we collected pairwise visual similarity ratings with same- and other-race faces across East Asian and White participants exhibiting robust levels of ORE. Leveraging the significant overlap in representational similarity between the GAN's latent space and perceptual representations in human participants, we designed an image reconstruction approach aiming to reveal internal face representations from behavioral similarity data. This methodology yielded hyper-realistic depictions of face percepts, with reconstruction accuracy well above chance, as well as an accuracy advantage for same-race over other-race reconstructions, which mirrored ORE in both populations. Further, a comparison of reconstructions across participant race revealed a novel age bias, with other-race face reconstructions appearing younger than their same-race counterpart. Thus, our work proposes a new approach to exploiting the utility of GANs in image reconstruction and provides new avenues in the study of ORE.