The other-race effect in face learning: Using naturalistic images to investigate face ethnicity effects in a learning paradigm

Unraveling other-race face perception with GAN-based image reconstruction

The other-race effect (ORE) is the disadvantage of recognizing faces of another race than one's own. While its prevalence is behaviorally well documented, the representational basis of ORE remains unclear. This study employs StyleGAN2, a deep learning technique for generating photorealistic images to uncover face representations and to investigate ORE's representational basis. To this end, we collected pairwise visual similarity ratings with same- and other-race faces across East Asian and White participants exhibiting robust levels of ORE. Leveraging the significant overlap in representational similarity between the GAN's latent space and perceptual representations in human participants, we designed an image reconstruction approach aiming to reveal internal face representations from behavioral similarity data. This methodology yielded hyper-realistic depictions of face percepts, with reconstruction accuracy well above chance, as well as an accuracy advantage for same-race over other-race reconstructions, which mirrored ORE in both populations. Further, a comparison of reconstructions across participant race revealed a novel age bias, with other-race face reconstructions appearing younger than their same-race counterpart. Thus, our work proposes a new approach to exploiting the utility of GANs in image reconstruction and provides new avenues in the study of ORE.

Event-related brain potential correlates of the other-race effect: A review

People are better at remembering own-race relative to other-race faces. Here, we review event-related brain potential (ERP) correlates of this so-called other-'race' effect (ORE) by discussing three critical aspects that characterize the neural signature of this phenomenon. First, difficulties with other-race faces initially emerge during perceptual processing, which is indexed by an increased N170. Second, as evidenced by 'difference due to subsequent memory' effects, more effortful processing of other-race faces is needed for successful encoding into long-term memory. Third, ERP old/new effects reveal that a stronger engagement of processing resources is also required for successful retrieval of other-race faces from memory. The ERP evidence available to date thus suggests widespread ethnicity-related modulations during both perceptual and mnemonic processing stages. We further discuss how findings from the ORE compared with potentially related memory biases (e.g. other-gender or other-age effects) and how ERP findings inform the ongoing debate regarding the mechanisms underlying the ORE. Finally, we outline open questions and potential future directions with an emphasis on using multiple, ecologically more valid 'ambient' images for each face to assess the ORE in paradigms that capture identity rather than image recognition.

Naturalistic Face Learning in Infants and Adults

Everyday face recognition presents a difficult challenge because faces vary naturally in appearance as a result of changes in lighting, expression, viewing angle, and hairstyle. We know little about how humans develop the ability to learn faces despite natural facial variability. In the current study, we provide the first examination of attentional mechanisms underlying adults' and infants' learning of naturally varying faces. Adults (n = 48) and 6- to 12-month-old infants (n = 48) viewed videos of models reading a storybook; the facial appearance of these models was either high or low in variability. Participants then viewed the learned face paired with a novel face. Infants showed adultlike prioritization of face over nonface regions; both age groups fixated the face region more in the high- than low-variability condition. Overall, however, infants showed less ability to resist contextual distractions during learning, which potentially contributed to their lack of discrimination between the learned and novel faces. Mechanisms underlying face learning across natural variability are discussed.

Face Recognition by Humans and Machines: Three Fundamental Advances from Deep Learning

Deep learning models currently achieve human levels of performance on real-world face recognition tasks. We review scientific progress in understanding human face processing using computational approaches based on deep learning. This review is organized around three fundamental advances. First, deep networks trained for face identification generate a representation that retains structured information about the face (e.g., identity, demographics, appearance, social traits, expression) and the input image (e.g., viewpoint, illumination). This forces us to rethink the universe of possible solutions to the problem of inverse optics in vision. Second, deep learning models indicate that high-level visual representations of faces cannot be understood in terms of interpretable features. This has implications for understanding neural tuning and population coding in the high-level visual cortex. Third, learning in deep networks is a multistep process that forces theoretical consideration of diverse categories of learning that can overlap, accumulate over time, and interact. Diverse learning types are needed to model the development of human face processing skills, cross-race effects, and familiarity with individual faces.

Why the other-race effect matters: Poor recognition of other-race faces impacts everyday social interactions

What happens to everyday social interactions when other-race recognition fails? Here, we provide the first formal investigation of this question. We gave East Asian international students (N = 89) a questionnaire concerning their experiences of the other-race effect (ORE) in Australia, and a laboratory test of their objective other-race face recognition deficit using the Cambridge Face Memory Test (CFMT). As a 'perpetrator' of the ORE, participants reported that their problems telling apart Caucasian people contributed significantly to difficulties socializing with them. Moreover, the severity of this problem correlated with their ORE on the CFMT. As a 'victim' of the ORE, participants reported that Caucasians' problems telling them apart also contributed to difficulties socializing. Further, 81% of participants had been confused with other Asians by a Caucasian authority figure (e.g., university tutor, workplace boss), resulting in varying levels of upset/difficulty. When compared to previously established contributors to international students' high rates of social isolation, ORE-related problems were perceived as equally important as the language barrier and only moderately less important than cultural differences. We conclude that the real-world impact of the ORE extends beyond previously identified specialized settings (eyewitness testimony, security), to common everyday situations experienced by all humans.

Understanding face identification through within-person variability in appearance: Introduction to a virtual special issue

In the effort to determine the cognitive processes underlying the identification of faces, the dissimilarities between images of different people have long been studied. In contrast, the inherent variability between different images of the same face has either been treated as a nuisance variable that should be eliminated from psychological experiments or it has not been considered at all. Over the past decade, research efforts have increased substantially to demonstrate that this within-person variation is meaningful and can give insight into various processes of face identification, such as identity matching, face learning, and familiar face recognition. In this virtual special issue of the Quarterly Journal of Experimental Psychology, we explain the importance of within-person variability for face identification and bring together recent relevant articles published in the journal.

The Own-Race Bias for Face Recognition in a Multiracial Society

The own-race bias (ORB) is a reliable phenomenon across cultural and racial groups where unfamiliar faces from other races are usually remembered more poorly than own-race faces (Meissner and Brigham, 2001). By adopting a yes-no recognition paradigm, we found that ORB was pronounced across race groups (Malaysian-Malay, Malaysian-Chinese, Malaysian-Indian, and Western-Caucasian) when faces were presented with only internal features (Experiment 1), implying that growing up in a profoundly multiracial society does not necessarily eliminate ORB. Using a procedure identical to Experiment 1, we observed a significantly greater increment in recognition performance for other-race faces than for own-race faces when the external features (e.g. facial contour and hairline) were presented along with the internal features (Experiment 2)-this abolished ORB. Contrary to assumptions based on the contact hypothesis, participants' self-reported amount of interracial contact on a social contact questionnaire did not significantly predict the magnitude of ORB. Overall, our findings suggest that the level of exposure to other-race faces accounts for only a small part of ORB. In addition, the present results also support the notion that different neural mechanisms may be involved in processing own- and other-race faces, with internal features of own-race faces being processed more effectively, whereas external features dominate representations of other-race faces.

Learning own- and other-race facial identities: Testing implicit recognition with event-related brain potentials

Exposure to varying images of the same person can encourage the formation of a representation that is sufficiently robust to allow recognition of previously unseen images of this person. While behavioural work suggests that face identity learning is harder for other-race faces, the present experiment investigated the neural correlates underlying own- and other-race face learning. Participants sorted own- and other-race identities into separate identity clusters and were further familiarised with these identities in a matching task. Subsequently, we compared event-related brain potentials (ERPs) in an implicit recognition (butterfly detection) task for learnt and previously unseen identities. We observed better sorting and matching for own- than other-race identities, and behavioural learning effects were restricted to own-race identities. Similarly, the N170 ERP component showed clear learning effects for own-race faces only. The N250, a component more closely associated with face learning was more negative for learnt than novel identities. ERP findings thus suggests a processing advantage for own-race identities at an early perceptual level whereas later correlates of identity learning were unaffected by ethnicity. These results suggest learning advantages for own-race identities, which underscores the importance of perceptual expertise in the own-race bias.

Learning own- and other-race facial identities from natural variability

Exposure to multiple varying face images of the same person encourages the formation of identity representations which are sufficiently robust to allow subsequent recognition from new, never-before seen images. While recent studies suggest that identity information is initially harder to perceive in images of other- relative to own-race identities, it remains unclear whether these difficulties propagate to face learning, that is, to the formation of robust face representations. We report two experiments in which Caucasian and East Asian participants sorted multiple images of own- and other-race persons according to identity in an implicit learning task and subsequently either matched novel images of learnt and previously unseen faces for identity (Experiment 1) or made old/new decisions for new images of learnt and unfamiliar identities (Experiment 2). Caucasian participants demonstrated own-race advantages during sorting, matching, and old/new recognition, while corresponding effects were absent in East Asian participants with substantial other-race expertise. Surprisingly, East Asian participants showed enhanced learning for other-race identities during matching in Experiment 1, which may reflect their increased motivation to individuate other-race faces. Thus, our results highlight the importance of perceptual expertise for own- and other-race processing, but may also lend support to recent suggestions on how expertise and socio-cognitive factors can interact.

Becoming Familiar With a Newly Encountered Face: Evidence of an Own-Race Advantage

Adults' ability to match identity in images of unfamiliar faces is impaired for other- compared with own-race faces; their ability to match identity in images of familiar faces is independent of face race. Exposure to within-person variability in appearance plays a key role in face learning. Past research suggests that children need exposure to higher levels of variability than adults to learn a new face-a difference that has been attributed to experience. We predicted that adults' limited experience with other-race faces would result in their needing exposure to higher levels of variability when learning other- compared with own-race faces. We introduced adults to four new identities (two own-race; two other-race) in one of the three conditions: a single image, a low-variability video (filmed on 1 day), or a high-variability video (filmed across 3 days). Adults' ability to recognize new instances of learned identities improved in the low-variability condition for own-race faces but only in the high-variability condition for other-race faces. We discuss learning mechanisms that might drive this difference-a difference we attribute to experience.

The Own-Race Bias and the cerebral hemispheres

The Own-Race Bias (ORB) is the ability to better recognize and categorize a face when the depicted person belongs to the observer's ethnicity group. The relationship between the ORB and hemispheric asymmetries has been poorly explored, and the present study was aimed at investigating this relationship, as well as that between the ORB and the bias to better recognize own gender faces. Female and male Caucasian participants categorized the ethnicity of Caucasian and Asian female and male facial stimuli in a divided visual field paradigm. In a control experiment the same stimuli were presented centrally, confirming the ORB. Importantly, the lateralized presentation reversed the bias with higher accuracy and shorter response times in the categorization of Asian than Caucasian faces. This reversed bias was significant for female and male faces, and it was observed when stimuli were presented in the left but not in the right visual field, revealing the crucial role of the right hemisphere in face processing. These results shed new light on the hemispheric abilities in the categorization of facial features, and they are compared to previous evidence of cerebral asymmetries for facial age, gender and identity, both in healthy participants and in neurological patients.