Local features drive identity responses in macaque anterior face patches

Distortions of lip size bias perceived facial attractiveness

Perceiving faces as attractive or not guides decisions to approach or date a person and can sway opinions in recruiting and legal proceedings. However, the mechanisms underlying facial attractiveness are not fully understood. While popular models of face recognition emphasize holistic processing, individuals often attempt to enhance their own attractiveness in feature-centric ways (cosmetic surgery, make-up, injectables). Here, we use a local feature manipulation (lip expansion/contraction) and show that it alters the perceived attractiveness of male and female faces. Females showed peak preference for expanded lips when viewing female faces; males showed peak preference for contracted lips when viewing male faces. Distortions of lip size therefore mostly influence own-gender attractiveness ratings. Next, we tested whether visual adaptation to expanded or contracted lips would bias subsequent attractiveness judgements, and found peak attractiveness shifted towards the adapted lip size (e.g. expanded lips were preferred following exposure to expanded lips). Viewing faces with artificially altered lip size therefore powerfully influences attractiveness judgements. Outside the laboratory, cosmetic procedures to increase lip size are popular. Our findings indicate that (i) lip plumping will mostly appeal to women rather than men (who prefer thinner lips), and (ii) exposure to expanded lips renormalizes attractiveness to a larger baseline and may lead to lip dysmorphia.

Fast face-selective responses in prefrontal face patches of the macaque

Face processing models propose gradually more complex receptive field properties culminating in invariant representations in anterior inferotemporal cortex (aITC), leading to late socio-emotionally relevant encoding in pre- and orbitofrontal cortex (POC). Top-down facilitation models, however, predict that some lower-level POC neurons respond faster than aITC. To resolve this discrepancy, we recorded from 2,459 neurons in fMRI-defined POC and aITC face patches. POC patches are more heterogeneous, containing smaller fractions of face-selective neurons than aITC and a mixture of responses to faces and non-faces. In one POC patch, face responses are surprisingly fast, outpacing those in aITC. Moreover, its responses correlate inversely with stimulus spatial frequency. Hence, our extensive data, with a large diversity of POC neurons, support both models and suggest one POC face patch might be specialized in fast, low-level face processing, which may enable (partially) invariant face representations during subsequent processing stages in inferotemporal cortex (ITC).

Part-based processing, but not holistic processing, predicts individual differences in face recognition abilities

This study aimed to assess the roles of part-based and holistic processing for face processing ability (FPA). A psychophysical paradigm in which the efficiency at recognizing isolated or combined facial parts was used (N = 64), and holistic processing was defined as the perceptual integration from multiple parts. FPA and object processing ability were measured using a battery of tasks. A multiple linear regression including three predictors, namely perceptual integration, part-based efficiency, and object processing, explained 40 % of the variance in FPA. Most importantly, our results reveal a strong predictive relationship between part-based efficiency and FPA, a small predictive relationship between object processing ability and FPA, and no predictive relationship between perceptual integration and FPA. This result was obtained despite considerable variance in perceptual integration skills-with some participants exhibiting a highly efficient integration. These results indicate that part-based processing plays a pivotal role in FPA, whereas holistic processing does not.

Brain connectivity for constructing new face representations in typical adults versus a prosopagnosic patient

Will our brains get to know a new face better if we look at its external features first? Here we offer neurophysiological evidence of the relevance of external versus internal facial features for constructing new face representations, by contrasting successful face processing with a prototypical case of face agnosia. A woman with acquired prosopagnosia (E.C.) and 14 age-matched typical participants (7 women) were exposed to a face-feature matching task. External (E), internal (I) features, and whole target faces of unknown individuals (from an IdentiKit gallery) were displayed according to two different sequences: E →I→whole faces, or I→E→whole faces. Then, we studied the induced EEG activity using 'isolated effective coherence' to analyse the intracortical causal information flow among face-sensitive nodes. Initial presentation of external features (E before I), when compared to internal ones, triggered connections encompassing extensively the right-hemisphere face processing pathway [from posterior visual cortices for initial structural analysis, towards both intermediate (occipitotemporal) and high-level (prefrontal) relay stations], in which face-identity is thought to emerge progressively. Also, whereas exposure to internal features as second stimulus seemed to demand some sort of basic visual processing, external features triggered again more widespread and integrative connections. Connections for whole faces closing the E-I sequence resembled those for external features initiating the same sequence. Meanwhile, the predominant connections for whole faces completing the I-E sequence were more restricted to specific brain areas, with relevant prefrontal activity and a few connected nodes in right posterior regions, suggesting high attentional load plus initial and intermediate processing of face identity. Interestingly, the pattern of connections that distinguished typical participants from E.C. in the I-E sequence was the recruitment of left posterior visual regions, presumably underlying analytical subroutines for structural encoding of facial stimuli. These findings support that initial exposure to external features, followed by internal ones, provides the best visual cue to acquire new face configurations. Nevertheless, in case of face agnosia after right posterior damage, relying preferentially on internal features and left hemisphere specialized subroutines might be an alternative for cognitive training.

Face pareidolia minimally engages macaque face selective neurons

The macaque cerebral cortex contains concentrations of neurons that prefer faces over inanimate objects. Although these so-called face patches are thought to be specialized for the analysis of facial signals, their exact tuning properties remain unclear. For example, what happens when an object by chance resembles a face? Everyday objects can sometimes, through the accidental positioning of their internal components, appear as faces. This phenomenon is known as face pareidolia. Behavioral experiments have suggested that macaques, like humans, perceive illusory faces in such objects. However, it is an open question whether such stimuli would naturally stimulate neurons residing in cortical face patches. To address this question, we recorded single unit activity from four fMRI-defined face-selective regions: the anterior medial (AM), anterior fundus (AF), prefrontal orbital (PO), and perirhinal cortex (PRh) face patches. We compared neural responses elicited by images of real macaque faces, pareidolia-evoking objects, and matched control objects. Contrary to expectations, we found no evidence of a general preference for pareidolia-evoking objects over control objects. Although a subset of neurons exhibited stronger responses to pareidolia-evoking objects, the population responses to both categories of objects were similar, and collectively much less than to real macaque faces. These results suggest that neural responses in the four regions we tested are principally concerned with the analysis of realistic facial characteristics, whereas the special attention afforded to face-like pareidolia stimuli is supported by activity elsewhere in the brain.

Face cells encode object parts more than facial configuration of illusory faces

Humans perceive illusory faces in everyday objects with a face-like configuration, an illusion known as face pareidolia. Face-selective regions in humans and monkeys, believed to underlie face perception, have been shown to respond to face pareidolia images. Here, we investigated whether pareidolia selectivity in macaque inferotemporal cortex is explained by the face-like configuration that drives the human perception of illusory faces. We found that face cells responded selectively to pareidolia images. This selectivity did not correlate with human faceness ratings and did not require the face-like configuration. Instead, it was driven primarily by the "eye" parts of the illusory face, which are simply object parts when viewed in isolation. In contrast, human perceptual pareidolia relied primarily on the global configuration and could not be explained by "eye" parts. Our results indicate that face-cells encode local, generic features of illusory faces, in misalignment with human visual perception, which requires holistic information.

Neural Encoding of Bodies for Primate Social Perception

Primates, as social beings, have evolved complex brain mechanisms to navigate intricate social environments. This review explores the neural bases of body perception in both human and nonhuman primates, emphasizing the processing of social signals conveyed by body postures, movements, and interactions. Early studies identified selective neural responses to body stimuli in macaques, particularly within and ventral to the superior temporal sulcus (STS). These regions, known as body patches, represent visual features that are present in bodies but do not appear to be semantic body detectors. They provide information about posture and viewpoint of the body. Recent research using dynamic stimuli has expanded the understanding of the body-selective network, highlighting its complexity and the interplay between static and dynamic processing. In humans, body-selective areas such as the extrastriate body area (EBA) and fusiform body area (FBA) have been implicated in the perception of bodies and their interactions. Moreover, studies on social interactions reveal that regions in the human STS are also tuned to the perception of dyadic interactions, suggesting a specialized social lateral pathway. Computational work developed models of body recognition and social interaction, providing insights into the underlying neural mechanisms. Despite advances, significant gaps remain in understanding the neural mechanisms of body perception and social interaction. Overall, this review underscores the importance of integrating findings across species to comprehensively understand the neural foundations of body perception and the interaction between computational modeling and neural recording.

Neuronal Mechanisms Underlying Face Recognition in Non-human Primates

Humans and primates rely on visual face recognition for social interactions. Damage to specific brain areas causes prosopagnosia, a condition characterized by the inability to recognize familiar faces, indicating the presence of specialized brain areas for face processing. A breakthrough finding came from a non-human primate (NHP) study conducted in the early 2000s; it was the first to identify multiple face processing areas in the temporal lobe, termed face patches. Subsequent studies have demonstrated the unique role of each face patch in the structural analysis of faces. More recent studies have expanded these findings by exploring the role of face patch networks in social and memory functions and the importance of early face exposure in the development of the system. In this review, we discuss the neuronal mechanisms responsible for analyzing facial features, categorizing faces, and associating faces with memory and social contexts within both the cerebral cortex and subcortical areas. Use of NHPs in neuropsychological and neurophysiological studies can highlight the mechanistic aspects of the neuronal circuit underlying face recognition at both the single-neuron and whole-brain network levels.

Inactivation of face-selective neurons alters eye movements when free viewing faces

During free viewing, faces attract gaze and induce specific fixation patterns corresponding to the facial features. This suggests that neurons encoding the facial features are in the causal chain that steers the eyes. However, there is no physiological evidence to support a mechanistic link between face-encoding neurons in high-level visual areas and the oculomotor system. In this study, we targeted the middle face patches of the inferior temporal (IT) cortex in two macaque monkeys using an functional magnetic resonance imaging (fMRI) localizer. We then utilized muscimol microinjection to unilaterally suppress IT neural activity inside and outside the face patches and recorded eye movements while the animals free viewing natural scenes. Inactivation of the face-selective neurons altered the pattern of eye movements on faces: The monkeys found faces in the scene but neglected the eye contralateral to the inactivation hemisphere. These findings reveal the causal contribution of the high-level visual cortex in eye movements.

Multisensory interactions of face and vocal information during perception and memory in ventrolateral prefrontal cortex

The ventral frontal lobe is a critical node in the circuit that underlies communication, a multisensory process where sensory features of faces and vocalizations come together. The neural basis of face and vocal integration is a topic of great importance since the integration of multiple sensory signals is essential for the decisions that govern our social interactions. Investigations have shown that the macaque ventrolateral prefrontal cortex (VLPFC), a proposed homologue of the human inferior frontal gyrus, is involved in the processing, integration and remembering of audiovisual signals. Single neurons in VLPFC encode and integrate species-specific faces and corresponding vocalizations. During working memory, VLPFC neurons maintain face and vocal information online and exhibit selective activity for face and vocal stimuli. Population analyses indicate that identity, a critical feature of social stimuli, is encoded by VLPFC neurons and dictates the structure of dynamic population activity in the VLPFC during the perception of vocalizations and their corresponding facial expressions. These studies suggest that VLPFC may play a primary role in integrating face and vocal stimuli with contextual information, in order to support decision making during social communication. This article is part of the theme issue 'Decision and control processes in multisensory perception'.

When the whole is only the parts: non-holistic object parts predominate face-cell responses to illusory faces

Humans are inclined to perceive faces in everyday objects with a face-like configuration. This illusion, known as face pareidolia, is often attributed to a specialized network of 'face cells' in primates. We found that face cells in macaque inferotemporal cortex responded selectively to pareidolia images, but this selectivity did not require a holistic, face-like configuration, nor did it encode human faceness ratings. Instead, it was driven mostly by isolated object parts that are perceived as eyes only within a face-like context. These object parts lack usual characteristics of primate eyes, pointing to the role of lower-level features. Our results suggest that face-cell responses are dominated by local, generic features, unlike primate visual perception, which requires holistic information. These findings caution against interpreting neural activity through the lens of human perception. Doing so could impose human perceptual biases, like seeing faces where none exist, onto our understanding of neural activity.

Inactivation of face selective neurons alters eye movements when free viewing faces

During free viewing, faces attract gaze and induce specific fixation patterns corresponding to the facial features. This suggests that neurons encoding the facial features are in the causal chain that steers the eyes. However, there is no physiological evidence to support a mechanistic link between face encoding neurons in high-level visual areas and the oculomotor system. In this study, we targeted the middle face patches of inferior temporal (IT) cortex in two macaque monkeys using an fMRI localizer. We then utilized muscimol microinjection to unilaterally suppress IT neural activity inside and outside the face patches and recorded eye movements while the animals free viewing natural scenes. Inactivation of the face selective neurons altered the pattern of eye movements on faces: the monkeys found faces in the scene but neglected the eye contralateral to the inactivation hemisphere. These findings reveal the causal contribution of the high-level visual cortex in eye movements.

Significance

It has been shown, for more than half a century, that eye movements follow distinctive patterns when free viewing faces. This suggests causal involvement of the face-encoding visual neurons in the eye movements. However, the literature is scant of evidence for this possibility and has focused mostly on the link between low-level image saliency and eye movements. Here, for the first time, we bring causal evidence showing how face-selective neurons in inferior temporal cortex inform and steer eye movements when free viewing faces.

Representational geometry of incomplete faces in macaque face patches

The neural code of faces has been intensively studied in the macaque face patch system. Although the majority of previous studies used complete faces as stimuli, faces are often seen partially in daily life. Here, we investigated how face-selective cells represent two types of incomplete faces: face fragments and occluded faces, with the location of the fragment/occluder and the facial features systematically varied. Contrary to popular belief, we found that the preferred face regions identified with two stimulus types are dissociated in many face cells. This dissociation can be explained by the nonlinear integration of information from different face parts and is closely related to a curved representation of face completeness in the state space, which allows a clear discrimination between different stimulus types. Furthermore, identity-related facial features are represented in a subspace orthogonal to the nonlinear dimension of face completeness, supporting a condition-general code of facial identity.

Encoding of 3D physical dimensions by face-selective cortical neurons

Neurons throughout the primate inferior temporal (IT) cortex respond selectively to visual images of faces and other complex objects. The response magnitude of neurons to a given image often depends on the size at which the image is presented, usually on a flat display at a fixed distance. While such size sensitivity might simply reflect the angular subtense of retinal image stimulation in degrees, one unexplored possibility is that it tracks the real-world geometry of physical objects, such as their size and distance to the observer in centimeters. This distinction bears fundamentally on the nature of object representation in IT and on the scope of visual operations supported by the ventral visual pathway. To address this question, we assessed the response dependency of neurons in the macaque anterior fundus (AF) face patch to the angular versus physical size of faces. We employed a macaque avatar to stereoscopically render three-dimensional (3D) photorealistic faces at multiple sizes and distances, including a subset of size/distance combinations designed to cast the same size retinal image projection. We found that most AF neurons were modulated principally by the 3D physical size of the face rather than its two-dimensional (2D) angular size on the retina. Further, most neurons responded strongest to extremely large and small faces, rather than to those of normal size. Together, these findings reveal a graded encoding of physical size among face patch neurons, providing evidence that category-selective regions of the primate ventral visual pathway participate in a geometric analysis of real-world objects.