The neural code for "face cells" is not face-specific

Detection of Individual Differences Encoded in Sequential Variations of Elements in Zebra Finch Songs

Zebra finches sing individually unique songs and recognize conspecific songs and individual identities in songs. Their songs comprise several syllables/elements that share acoustic features within the species, with unique sequential arrangements. However, the neuronal mechanisms underlying the detection of individual differences and species specificity have yet to be elucidated. Herein, we examined the neuronal auditory responsiveness of neurons in the higher auditory area, the caudal nidopallium (NCM), to songs and their elements in male zebra finches to understand the mechanism for detecting species and individual identities in zebra finch songs. We found that various adult male zebra finch songs share acoustically similar song elements but differ in their sequential arrangement between individuals. The broader spiking (BS) neurons in the NCM detected only a small subset of zebra finch songs, whereas NCM BS neurons, as a neuronal ensemble, responded to all zebra finch songs. Notably, distinct combinations of BS neurons responded to each of the 18 presented songs in one bird. Subsets of NCM BS neurons were sensitive to sequential arrangements of species-specific elements, which dramatically increasing the capacity for song variation with a limited number of species-specific elements. The naive Bayes decoder analysis further showed that the response of sequence-sensitive BS neurons increased the accuracy of song stimulus predictions based on the response strength of neuronal ensembles. Our results suggest the neuronal mechanisms that NCM neurons as an ensemble decode the individual identities of songs, while each neuron detects a small subset of song elements and their sequential arrangement.

Face pareidolia minimally engages macaque face selective neurons

The macaque cerebral cortex contains concentrations of neurons that prefer faces over inanimate objects. Although these so-called face patches are thought to be specialized for the analysis of facial signals, their exact tuning properties remain unclear. For example, what happens when an object by chance resembles a face? Everyday objects can sometimes, through the accidental positioning of their internal components, appear as faces. This phenomenon is known as face pareidolia. Behavioral experiments have suggested that macaques, like humans, perceive illusory faces in such objects. However, it is an open question whether such stimuli would naturally stimulate neurons residing in cortical face patches. To address this question, we recorded single unit activity from four fMRI-defined face-selective regions: the anterior medial (AM), anterior fundus (AF), prefrontal orbital (PO), and perirhinal cortex (PRh) face patches. We compared neural responses elicited by images of real macaque faces, pareidolia-evoking objects, and matched control objects. Contrary to expectations, we found no evidence of a general preference for pareidolia-evoking objects over control objects. Although a subset of neurons exhibited stronger responses to pareidolia-evoking objects, the population responses to both categories of objects were similar, and collectively much less than to real macaque faces. These results suggest that neural responses in the four regions we tested are principally concerned with the analysis of realistic facial characteristics, whereas the special attention afforded to face-like pareidolia stimuli is supported by activity elsewhere in the brain.

Face cells encode object parts more than facial configuration of illusory faces

Humans perceive illusory faces in everyday objects with a face-like configuration, an illusion known as face pareidolia. Face-selective regions in humans and monkeys, believed to underlie face perception, have been shown to respond to face pareidolia images. Here, we investigated whether pareidolia selectivity in macaque inferotemporal cortex is explained by the face-like configuration that drives the human perception of illusory faces. We found that face cells responded selectively to pareidolia images. This selectivity did not correlate with human faceness ratings and did not require the face-like configuration. Instead, it was driven primarily by the "eye" parts of the illusory face, which are simply object parts when viewed in isolation. In contrast, human perceptual pareidolia relied primarily on the global configuration and could not be explained by "eye" parts. Our results indicate that face-cells encode local, generic features of illusory faces, in misalignment with human visual perception, which requires holistic information.

Innate face-selectivity in the brain of young domestic chicks

Shortly after birth, both naïve animals and newborn babies exhibit a spontaneous attraction to faces and face-like stimuli. While neurons selectively responding to faces have been found in the inferotemporal cortex of adult primates, face-selective domains in the brains of young monkeys seem to develop only later in life after exposure to faces. This has fueled a debate on the role of experience in the development of face-detector mechanisms, since face preferences are well documented in naïve animals, such as domestic chicks reared without exposure to faces. Here, we demonstrate that neurons in a higher-order processing brain area of one-week-old face-naïve domestic chicks selectively respond to a face-like configuration. Our single-cell recordings show that these neurons do not respond to alternative configurations or isolated facial features. Moreover, the population activity of face-selective neurons accurately encoded the face-like stimulus as a unique category. Thus, our findings show that face selectivity is present in the brains of very young animals without preexisting experience.

Neuronal Mechanisms Underlying Face Recognition in Non-human Primates

Humans and primates rely on visual face recognition for social interactions. Damage to specific brain areas causes prosopagnosia, a condition characterized by the inability to recognize familiar faces, indicating the presence of specialized brain areas for face processing. A breakthrough finding came from a non-human primate (NHP) study conducted in the early 2000s; it was the first to identify multiple face processing areas in the temporal lobe, termed face patches. Subsequent studies have demonstrated the unique role of each face patch in the structural analysis of faces. More recent studies have expanded these findings by exploring the role of face patch networks in social and memory functions and the importance of early face exposure in the development of the system. In this review, we discuss the neuronal mechanisms responsible for analyzing facial features, categorizing faces, and associating faces with memory and social contexts within both the cerebral cortex and subcortical areas. Use of NHPs in neuropsychological and neurophysiological studies can highlight the mechanistic aspects of the neuronal circuit underlying face recognition at both the single-neuron and whole-brain network levels.

Contrastive learning explains the emergence and function of visual category-selective regions

Modular and distributed coding theories of category selectivity along the human ventral visual stream have long existed in tension. Here, we present a reconciling framework-contrastive coding-based on a series of analyses relating category selectivity within biological and artificial neural networks. We discover that, in models trained with contrastive self-supervised objectives over a rich natural image diet, category-selective tuning naturally emerges for faces, bodies, scenes, and words. Further, lesions of these model units lead to selective, dissociable recognition deficits, highlighting their distinct functional roles in information processing. Finally, these pre-identified units can predict neural responses in all corresponding face-, scene-, body-, and word-selective regions of human visual cortex, under a highly constrained sparse positive encoding procedure. The success of this single model indicates that brain-like functional specialization can emerge without category-specific learning pressures, as the system learns to untangle rich image content. Contrastive coding, therefore, provides a unifying account of object category emergence and representation in the human brain.

Face detection mechanisms: Nature vs. nurture

For many animals, faces are a vitally important visual stimulus. Hence, it is not surprising that face perception has become a very popular research topic in neuroscience, with ca. 2000 papers published every year. As a result, significant progress has been made in understanding the intricate mechanisms underlying this phenomenon. However, the ontogeny of face perception, in particular the role of innate predispositions, remains largely unexplored at the neural level. Several influential studies in monkeys have suggested that seeing faces is necessary for the development of the face-selective brain domains. At the same time, behavioural experiments with newborn human babies and newly-hatched domestic chicks demonstrate that a spontaneous preference towards faces emerges early in life without pre-existing experience. Moreover, we were recently able to record face-selective neural responses in the brain of young, face-naïve chicks, thus demonstrating the existence of an innate face detection mechanism. In this review, we discuss these seemingly contradictory results and propose potential experimental approaches to resolve some of the open questions.

Decoding face recognition abilities in the human brain

Why are some individuals better at recognizing faces? Uncovering the neural mechanisms supporting face recognition ability has proven elusive. To tackle this challenge, we used a multimodal data-driven approach combining neuroimaging, computational modeling, and behavioral tests. We recorded the high-density electroencephalographic brain activity of individuals with extraordinary face recognition abilities-super-recognizers-and typical recognizers in response to diverse visual stimuli. Using multivariate pattern analyses, we decoded face recognition abilities from 1 s of brain activity with up to 80% accuracy. To better understand the mechanisms subtending this decoding, we compared representations in the brains of our participants with those in artificial neural network models of vision and semantics, as well as with those involved in human judgments of shape and meaning similarity. Compared to typical recognizers, we found stronger associations between early brain representations of super-recognizers and midlevel representations of vision models as well as shape similarity judgments. Moreover, we found stronger associations between late brain representations of super-recognizers and representations of the artificial semantic model as well as meaning similarity judgments. Overall, these results indicate that important individual variations in brain processing, including neural computations extending beyond purely visual processes, support differences in face recognition abilities. They provide the first empirical evidence for an association between semantic computations and face recognition abilities. We believe that such multimodal data-driven approaches will likely play a critical role in further revealing the complex nature of idiosyncratic face recognition in the human brain.

Disentangled deep generative models reveal coding principles of the human face processing network

Despite decades of research, much is still unknown about the computations carried out in the human face processing network. Recently, deep networks have been proposed as a computational account of human visual processing, but while they provide a good match to neural data throughout visual cortex, they lack interpretability. We introduce a method for interpreting brain activity using a new class of deep generative models, disentangled representation learning models, which learn a low-dimensional latent space that "disentangles" different semantically meaningful dimensions of faces, such as rotation, lighting, or hairstyle, in an unsupervised manner by enforcing statistical independence between dimensions. We find that the majority of our model's learned latent dimensions are interpretable by human raters. Further, these latent dimensions serve as a good encoding model for human fMRI data. We next investigate the representation of different latent dimensions across face-selective voxels. We find that low- and high-level face features are represented in posterior and anterior face-selective regions, respectively, corroborating prior models of human face recognition. Interestingly, though, we find identity-relevant and irrelevant face features across the face processing network. Finally, we provide new insight into the few "entangled" (uninterpretable) dimensions in our model by showing that they match responses in the ventral stream and carry information about facial identity. Disentangled face encoding models provide an exciting alternative to standard "black box" deep learning approaches for modeling and interpreting human brain data.

Rapid, concerted switching of the neural code in inferotemporal cortex

A fundamental paradigm in neuroscience is the concept of neural coding through tuning functions 1 . According to this idea, neurons encode stimuli through fixed mappings of stimulus features to firing rates. Here, we report that the tuning of visual neurons can rapidly and coherently change across a population to attend to a whole and its parts. We set out to investigate a longstanding debate concerning whether inferotemporal (IT) cortex uses a specialized code for representing specific types of objects or whether it uses a general code that applies to any object. We found that face cells in macaque IT cortex initially adopted a general code optimized for face detection. But following a rapid, concerted population event lasting < 20 ms, the neural code transformed into a face-specific one with two striking properties: (i) response gradients to principal detection-related dimensions reversed direction, and (ii) new tuning developed to multiple higher feature space dimensions supporting fine face discrimination. These dynamics were face specific and did not occur in response to objects. Overall, these results show that, for faces, face cells shift from detection to discrimination by switching from an object-general code to a face-specific code. More broadly, our results suggest a novel mechanism for neural representation: concerted, stimulus-dependent switching of the neural code used by a cortical area.

When the whole is only the parts: non-holistic object parts predominate face-cell responses to illusory faces

Humans are inclined to perceive faces in everyday objects with a face-like configuration. This illusion, known as face pareidolia, is often attributed to a specialized network of 'face cells' in primates. We found that face cells in macaque inferotemporal cortex responded selectively to pareidolia images, but this selectivity did not require a holistic, face-like configuration, nor did it encode human faceness ratings. Instead, it was driven mostly by isolated object parts that are perceived as eyes only within a face-like context. These object parts lack usual characteristics of primate eyes, pointing to the role of lower-level features. Our results suggest that face-cell responses are dominated by local, generic features, unlike primate visual perception, which requires holistic information. These findings caution against interpreting neural activity through the lens of human perception. Doing so could impose human perceptual biases, like seeing faces where none exist, onto our understanding of neural activity.

Intracranial Electroencephalography and Deep Neural Networks Reveal Shared Substrates for Representations of Face Identity and Expressions

According to a classical view of face perception (Bruce and Young, 1986; Haxby et al., 2000), face identity and facial expression recognition are performed by separate neural substrates (ventral and lateral temporal face-selective regions, respectively). However, recent studies challenge this view, showing that expression valence can also be decoded from ventral regions (Skerry and Saxe, 2014; Li et al., 2019), and identity from lateral regions (Anzellotti and Caramazza, 2017). These findings could be reconciled with the classical view if regions specialized for one task (either identity or expression) contain a small amount of information for the other task (that enables above-chance decoding). In this case, we would expect representations in lateral regions to be more similar to representations in deep convolutional neural networks (DCNNs) trained to recognize facial expression than to representations in DCNNs trained to recognize face identity (the converse should hold for ventral regions). We tested this hypothesis by analyzing neural responses to faces varying in identity and expression. Representational dissimilarity matrices (RDMs) computed from human intracranial recordings (n = 11 adults; 7 females) were compared with RDMs from DCNNs trained to label either identity or expression. We found that RDMs from DCNNs trained to recognize identity correlated with intracranial recordings more strongly in all regions tested-even in regions classically hypothesized to be specialized for expression. These results deviate from the classical view, suggesting that face-selective ventral and lateral regions contribute to the representation of both identity and expression.SIGNIFICANCE STATEMENT Previous work proposed that separate brain regions are specialized for the recognition of face identity and facial expression. However, identity and expression recognition mechanisms might share common brain regions instead. We tested these alternatives using deep neural networks and intracranial recordings from face-selective brain regions. Deep neural networks trained to recognize identity and networks trained to recognize expression learned representations that correlate with neural recordings. Identity-trained representations correlated with intracranial recordings more strongly in all regions tested, including regions hypothesized to be expression specialized in the classical hypothesis. These findings support the view that identity and expression recognition rely on common brain regions. This discovery may require reevaluation of the roles that the ventral and lateral neural pathways play in processing socially relevant stimuli.