Face Pareidolia in the Rhesus Monkey

Three key questions to move towards a theoretical framework of visuospatial perspective taking

What would a theory of visuospatial perspective taking (VSPT) look like? Here, ten researchers in the field, many with different theoretical viewpoints and empirical approaches, present their consensus on the three big questions we need to answer in order to bring this theory (or these theories) closer.

A behavioral advantage for the face pareidolia illusion in peripheral vision

Investigation of visual illusions helps us understand how we process visual information. For example, face pareidolia, the misperception of illusory faces in objects, could be used to understand how we process real faces. However, it remains unclear whether this illusion emerges from errors in face detection or from slower, cognitive processes. Here, our logic is straightforward; if examples of face pareidolia activate the mechanisms that rapidly detect faces in visual environments, then participants will look at objects more quickly when the objects also contain illusory faces. To test this hypothesis, we sampled continuous eye movements during a fast saccadic choice task-participants were required to select either faces or food items. During this task, pairs of stimuli were positioned close to the initial fixation point or further away, in the periphery. As expected, the participants were faster to look at face targets than food targets. Importantly, we also discovered an advantage for food items with illusory faces but, this advantage was limited to the peripheral condition. These findings are among the first to demonstrate that the face pareidolia illusion persists in the periphery and, thus, it is likely to be a consequence of erroneous face detection.

A limited visual search advantage for illusory faces

The human visual system is very sensitive to the presence of faces in the environment, so much so that it can produce the perception of illusory faces in everyday objects. Growing research suggests that illusory faces and real faces are processed by similar perceptual and neural mechanisms, but whether this similarity extends to visual attention is less clear. A visual search study showed that illusory faces have a search advantage over objects when the types of objects vary to match the objects in the illusory faces (e.g., chair, pepper, clock) (Keys et al., 2021). Here, we examine whether the search advantage for illusory faces over objects remains when compared against objects that belong to a single category (flowers). In three experiments, we compared visual search of illusory faces, real faces, variable objects, and uniform objects (flowers). Search for real faces was best compared with all other types of targets. In contrast, search for illusory faces was only better than search for variable objects, not uniform objects. This result shows a limited visual search advantage for illusory faces and suggests that illusory faces may not be processed like real faces in visual attention.

Inactivation of face-selective neurons alters eye movements when free viewing faces

During free viewing, faces attract gaze and induce specific fixation patterns corresponding to the facial features. This suggests that neurons encoding the facial features are in the causal chain that steers the eyes. However, there is no physiological evidence to support a mechanistic link between face-encoding neurons in high-level visual areas and the oculomotor system. In this study, we targeted the middle face patches of the inferior temporal (IT) cortex in two macaque monkeys using an functional magnetic resonance imaging (fMRI) localizer. We then utilized muscimol microinjection to unilaterally suppress IT neural activity inside and outside the face patches and recorded eye movements while the animals free viewing natural scenes. Inactivation of the face-selective neurons altered the pattern of eye movements on faces: The monkeys found faces in the scene but neglected the eye contralateral to the inactivation hemisphere. These findings reveal the causal contribution of the high-level visual cortex in eye movements.

When the whole is only the parts: non-holistic object parts predominate face-cell responses to illusory faces

Humans are inclined to perceive faces in everyday objects with a face-like configuration. This illusion, known as face pareidolia, is often attributed to a specialized network of 'face cells' in primates. We found that face cells in macaque inferotemporal cortex responded selectively to pareidolia images, but this selectivity did not require a holistic, face-like configuration, nor did it encode human faceness ratings. Instead, it was driven mostly by isolated object parts that are perceived as eyes only within a face-like context. These object parts lack usual characteristics of primate eyes, pointing to the role of lower-level features. Our results suggest that face-cell responses are dominated by local, generic features, unlike primate visual perception, which requires holistic information. These findings caution against interpreting neural activity through the lens of human perception. Doing so could impose human perceptual biases, like seeing faces where none exist, onto our understanding of neural activity.

Using Natural Scenes to Enhance our Understanding of the Cerebral Cortex's Role in Visual Search

Using natural scenes is an approach to studying the visual and eye movement systems approximating how these systems function in everyday life. This review examines the results from behavioral and neurophysiological studies using natural scene viewing in humans and monkeys. The use of natural scenes for the study of cerebral cortical activity is relatively new and presents challenges for data analysis. Methods and results from the use of natural scenes for the study of the visual and eye movement cortex are presented, with emphasis on new insights that this method provides enhancing what is known about these cortical regions from the use of conventional methods.

Preliminary evidence of an increased susceptibility to face pareidolia in postpartum women

As primates, we are hypersensitive to faces and face-like patterns in the visual environment, hence we often perceive illusory faces in otherwise inanimate objects, such as burnt pieces of toast and the surface of the moon. Although this phenomenon, known as face pareidolia, is a common experience, it is unknown whether our susceptibility to face pareidolia is static across our lifespan or what factors would cause it to change. Given the evidence that behaviour towards face stimuli is modulated by the neuropeptide oxytocin (OT), we reasoned that participants in stages of life associated with high levels of endogenous OT might be more susceptible to face pareidolia than participants in other stages of life. We tested this hypothesis by assessing pareidolia susceptibility in two groups of women; pregnant women (low endogenous OT) and postpartum women (high endogenous OT). We found evidence that postpartum women report seeing face pareidolia more easily than women who are currently pregnant. These data, collected online, suggest that our sensitivity to face-like patterns is not fixed and may change throughout adulthood, providing a crucial proof of concept that requires further research.

Hidden faces, altered perceptions: the impact of face masks on interpersonal perception

The pandemic has made wearing masks commonplace, prompting researchers to investigate their effects on interpersonal perception. Findings indicate masks obstruct face identification and expression recognition, with lower face cues being most affected. When judging attractiveness, masks can enhance the appeal of less attractive faces, but reduce the appeal of more attractive faces. Trust and speech perception outcomes are inconclusive. Future studies could focus on individual differences in how masks influence our perception of others.

Overwhelmed by the man in the moon? Pareidolic objects provoke increased amygdala activation in autism

An interesting feature of the primate face detection system results in the perception of illusory faces in objects, or pareidolia. These illusory faces do not per se contain social information, such as eye-gaze or specific identities, yet they activate the cortical brain face-processing network, possibly via the subcortical route, including the amygdala. In autism spectrum disorder (ASD), aversion to eye-contact is commonly reported, and so are alterations in face processing more generally, yet the underlying reasons are not clear. Here we show that in autistic participants (N=37), but not in non-autistic controls (N=34), pareidolic objects increase amygdala activation bilaterally (right amygdala peak: X = 26, Y = -6, Z = -16; left amygdala peak X = -24, Y = -6, Z = -20). In addition, illusory faces engage the face-processing cortical network significantly more in ASD than in controls. An early imbalance in the excitatory and inhibitory systems in autism, affecting typical brain maturation, may be at the basis of an overresponsive reaction to face configuration and to eye contact. Our data add to the evidence of an oversensitive subcortical face processing system in ASD.

A database of heterogeneous faces for studying naturalistic expressions

Facial expressions are thought to be complex visual signals, critical for communication between social agents. Most prior work aimed at understanding how facial expressions are recognized has relied on stimulus databases featuring posed facial expressions, designed to represent putative emotional categories (such as 'happy' and 'angry'). Here we use an alternative selection strategy to develop the Wild Faces Database (WFD); a set of one thousand images capturing a diverse range of ambient facial behaviors from outside of the laboratory. We characterized the perceived emotional content in these images using a standard categorization task in which participants were asked to classify the apparent facial expression in each image. In addition, participants were asked to indicate the intensity and genuineness of each expression. While modal scores indicate that the WFD captures a range of different emotional expressions, in comparing the WFD to images taken from other, more conventional databases, we found that participants responded more variably and less specifically to the wild-type faces, perhaps indicating that natural expressions are more multiplexed than a categorical model would predict. We argue that this variability can be employed to explore latent dimensions in our mental representation of facial expressions. Further, images in the WFD were rated as less intense and more genuine than images taken from other databases, suggesting a greater degree of authenticity among WFD images. The strong positive correlation between intensity and genuineness scores demonstrating that even the high arousal states captured in the WFD were perceived as authentic. Collectively, these findings highlight the potential utility of the WFD as a new resource for bridging the gap between the laboratory and real world in studies of expression recognition.

Pareidolic faces receive prioritized attention in the dot-probe task

Face pareidolia occurs when random or ambiguous inanimate objects are perceived as faces. While real faces automatically receive prioritized attention compared with nonface objects, it is unclear whether pareidolic faces similarly receive special attention. We hypothesized that, given the evolutionary importance of broadly detecting animacy, pareidolic faces may have enough faceness to activate a broad face template, triggering prioritized attention. To test this hypothesis, and to explore where along the faceness continuum pareidolic faces fall, we conducted a series of dot-probe experiments in which we paired pareidolic faces with other images directly competing for attention: objects, animal faces, and human faces. We found that pareidolic faces elicited more prioritized attention than objects, a process that was disrupted by inversion, suggesting this prioritized attention was unlikely to be driven by low-level features. However, unexpectedly, pareidolic faces received more privileged attention compared with animal faces and showed similar prioritized attention to human faces. This attentional efficiency may be due to pareidolic faces being perceived as not only face-like, but also as human-like, and having larger facial features-eyes and mouths-compared with real faces. Together, our findings suggest that pareidolic faces appear automatically attentionally privileged, similar to human faces. Our findings are consistent with the proposal of a highly sensitive broad face detection system that is activated by pareidolic faces, triggering false alarms (i.e., illusory faces), which, evolutionarily, are less detrimental relative to missing potentially relevant signals (e.g., conspecific or heterospecific threats). In sum, pareidolic faces appear "special" in attracting attention.

Children perceive illusory faces in objects as male more often than female

Face pareidolia is the experience of seeing illusory faces in inanimate objects. While children experience face pareidolia, it is unknown whether they perceive gender in illusory faces, as their face evaluation system is still developing in the first decade of life. In a sample of 412 children and adults from 4 to 80 years of age we found that like adults, children perceived many illusory faces in objects to have a gender and had a strong bias to see them as male rather than female, regardless of their own gender identification. These results provide evidence that the male bias for face pareidolia emerges early in life, even before the ability to discriminate gender from facial cues alone is fully developed. Further, the existence of a male bias in children suggests that any social context that elicits the cognitive bias to see faces as male has remained relatively consistent across generations.

Face pareidolia is enhanced by 40 Hz transcranial alternating current stimulation (tACS) of the face perception network

Pareidolia refers to the perception of ambiguous sensory patterns as carrying a specific meaning. In its most common form, pareidolia involves human-like facial features, where random objects or patterns are illusionary recognized as faces. The current study investigated the neurophysiological correlates of face pareidolia via transcranial alternating current stimulation (tACS). tACS was delivered at gamma (40 Hz) frequency over critical nodes of the "face perception" network (i.e., right lateral occipito-temporal and left prefrontal cortex) of 75 healthy participants while completing four face perception tasks ('Mooney test' for faces, 'Toast test', 'Noise pareidolia test', 'Pareidolia task') and an object perception task ('Mooney test' for objects). In this single-blind, sham-controlled between-subjects study, participants received 35 min of either Sham, Online, (40Hz-tACS_ON), or Offline (40Hz-tACS_PRE) stimulation. Results showed that face pareidolia was causally enhanced by 40Hz-tACS_PRE in the Mooney test for faces in which, as compared to sham, participants more often misperceived scrambled stimuli as faces. In addition, as compared to sham, participants receiving 40Hz-tACS_PRE showed similar reaction times (RTs) when perceiving illusory faces and correctly recognizing noise stimuli in the Toast test, thus not exhibiting hesitancy in identifying faces where there were none. Also, 40Hz-tACS_ON induced slower rejections of face pareidolia responses in the Noise pareidolia test. The current study indicates that 40 Hz tACS can enhance pareidolic illusions in healthy individuals and, thus, that high frequency (i.e., gamma band) oscillations are critical in forming coherent and meaningful visual perception.

Do chimpanzees see a face on Mars? A search for face pareidolia in chimpanzees

We sometimes perceive meaningful patterns or images in random arrangements of colors and shapes. This phenomenon is called pareidolia and has recently been studied intensively, especially face pareidolia. In contrast, there are few comparative-cognitive studies on face pareidolia with nonhuman primates. This study explored behavioral evidence for face pareidolia in chimpanzees using visual search and matching tasks. Faces are processed in a configural manner, and their perception and recognition are hampered by inversion and misalignment of top and bottom parts. We investigated whether the same effect occurs in a visual search for face-like objects. The results showed an effect of misalignment. On the other hand, consistent results were not obtained with the photographs of fruits. When only the top or bottom half of the face-like object was presented, chimpanzees showed better performance for the top-half condition, suggesting the importance of the eye area in face pareidolia. In the positive-control experiments, chimpanzees received the same experiment using human faces and human participants with face-like objects and fruits. As a result, chimpanzees showed an inefficient search for inverted and misaligned faces and humans for manipulated face-like objects. Finally, to examine the role of face awareness, we tested matching a human face to a face-like object in chimpanzees but obtained no substantial evidence that they saw the face-like object as a "face." Based on these results, we discussed the extents and limits of face pareidolia in chimpanzees.

Attention to Stimuli of Learned versus Innate Biological Value Relies on Separate Neural Systems

The neural bases of attention, a set of neural processes that promote behavioral selection, is a subject of intense investigation. In humans, rewarded cues influence attention, even when those cues are irrelevant to the current task. Because the amygdala plays a role in reward processing, and the activity of amygdala neurons has been linked to spatial attention, we reasoned that the amygdala may be essential for attending to rewarded images. To test this possibility, we used an attentional capture task, which provides a quantitative measure of attentional bias. Specifically, we compared reaction times (RTs) of adult male rhesus monkeys with bilateral amygdala lesions and unoperated controls as they made a saccade away from a high- or low-value rewarded image to a peripheral target. We predicted that: (1) RTs will be longer for high- compared with low-value images, revealing attentional capture by rewarded stimuli; and (2) relative to controls, monkeys with amygdala lesions would exhibit shorter RT for high-value images. For comparison, we assessed the same groups of monkeys for attentional capture by images of predators and conspecifics, categories thought to have innate biological value. In performing the attentional capture task, all monkeys were slowed more by high-value relative to low-value rewarded images. Contrary to our prediction, amygdala lesions failed to disrupt this effect. When presented with images of predators and conspecifics, however, monkeys with amygdala lesions showed significantly diminished attentional capture relative to controls. Thus, separate neural pathways are responsible for allocating attention to stimuli with learned versus innate value.SIGNIFICANCE STATEMENT Valuable objects attract attention. The amygdala is known to contribute to reward processing and the encoding of object reward value. We therefore examined whether the amygdala is necessary for allocating attention to rewarded objects. For comparison, we assessed the amygdala's contribution to attending to objects with innate biological value: predators and conspecifics. We found that the macaque amygdala is necessary for directing attention to images with innate biological value, but not for directing attention to recently learned reward-predictive images. These findings indicate that the amygdala makes selective contributions to attending to valuable objects. The data are relevant to mental health disorders, such as social anxiety disorders and small animal phobias, that arise from biased attention to select categories of objects.

Clutter Substantially Reduces Selectivity for Peripheral Faces in the Macaque Brain

According to a prominent view in neuroscience, visual stimuli are coded by discrete cortical networks that respond preferentially to specific categories, such as faces or objects. However, it remains unclear how these category-selective networks respond when viewing conditions are cluttered, i.e., when there is more than one stimulus in the visual field. Here, we asked three questions: (1) Does clutter reduce the response and selectivity for faces as a function of retinal location? (2) Is the preferential response to faces uniform across the visual field? And (3) Does the ventral visual pathway encode information about the location of cluttered faces? We used fMRI to measure the response of the face-selective network in awake, fixating macaques (two female, five male). Across a series of four experiments, we manipulated the presence and absence of clutter, as well as the location of the faces relative to the fovea. We found that clutter reduces the response to peripheral faces. When presented in isolation, without clutter, the selectivity for faces is fairly uniform across the visual field, but, when clutter is present, there is a marked decrease in the selectivity for peripheral faces. We also found no evidence of a contralateral visual field bias when faces were presented in clutter. Nonetheless, multivariate analyses revealed that the location of cluttered faces could be decoded from the multivoxel response of the face-selective network. Collectively, these findings demonstrate that clutter blunts the selectivity of the face-selective network to peripheral faces, although information about their retinal location is retained.SIGNIFICANCE STATEMENT Numerous studies that have measured brain activity in macaques have found visual regions that respond preferentially to faces. Although these regions are thought to be essential for social behavior, their responses have typically been measured while faces were presented in isolation, a situation atypical of the real world. How do these regions respond when faces are presented with other stimuli? We report that, when clutter is present, the preferential response to foveated faces is spared but preferential response to peripheral faces is reduced. Our results indicate that the presence of clutter changes the response of the face-selective network.

Pareidolia in a Built Environment as a Complex Phenomenological Ambiguous Stimuli

Pareidolia is a kind of misperception caused by meaningless, ambiguous stimuli perceived with meaning. Pareidolia in a built environment may trigger the emotions of residents, and the most frequently observed pareidolian images are human faces. Through a pilot experiment and an in-depth questionnaire survey, this research aims to compare built environmental pareidolian phenomena at different time points (6 a.m., 12 p.m., 2 a.m.) and to determine people’s sensitivity and reactions towards pareidolia in the built environment. Our findings indicate that the differences in stress level do not influence the sensitivity and reactions towards pareidolia in the built environment; however, age does, and the age of 40 seems to be a watershed. Females are more likely to identify pareidolian faces than males. Smokers, topers, and long-term medicine users are more sensitive to pareidolian images in the built environment. An unexpected finding is that most pareidolian images in built environments are much more easily detected in the early morning and at midnight but remain much less able to be perceived at midday. The results help architects better understand people’s reactions to pareidolia in the built environment, thus allowing them to decide whether to incorporate it appropriately or avoid it consciously in building design.

The cortical and subcortical correlates of face pareidolia in the macaque brain

Face detection is a foundational social skill for primates. This vital function is thought to be supported by specialized neural mechanisms; however, although several face-selective regions have been identified in both humans and nonhuman primates, there is no consensus about which region(s) are involved in face detection. Here, we used naturally occurring errors of face detection (i.e. objects with illusory facial features referred to as examples of 'face pareidolia') to identify regions of the macaque brain implicated in face detection. Using whole-brain functional magnetic resonance imaging to test awake rhesus macaques, we discovered that a subset of face-selective patches in the inferior temporal cortex, on the lower lateral edge of the superior temporal sulcus, and the amygdala respond more to objects with illusory facial features than matched non-face objects. Multivariate analyses of the data revealed differences in the representation of illusory faces across the functionally defined regions of interest. These differences suggest that the cortical and subcortical face-selective regions contribute uniquely to the detection of facial features. We conclude that face detection is supported by a multiplexed system in the primate brain.

Staring death in the face: chimpanzees' attention towards conspecific skulls and the implications of a face module guiding their behaviour

Chimpanzees exhibit a variety of behaviours surrounding their dead, although much less is known about how they respond towards conspecific skeletons. We tested chimpanzees' visual attention to images of conspecific and non-conspecific stimuli (cat/chimp/dog/rat), shown simultaneously in four corners of a screen in distinct orientations (frontal/diagonal/lateral) of either one of three types (faces/skulls/skull-shaped stones). Additionally, we compared their visual attention towards chimpanzee-only stimuli (faces/skulls/skull-shaped stones). Lastly, we tested their attention towards specific regions of chimpanzee skulls. We theorized that chimpanzee skulls retaining face-like features would be perceived similarly to chimpanzee faces and thus be subjected to similar biases. Overall, supporting our hypotheses, the chimpanzees preferred conspecific-related stimuli. The results showed that chimpanzees attended: (i) significantly longer towards conspecific skulls than other species skulls (particularly in forward-facing and to a lesser extent diagonal orientations); (ii) significantly longer towards conspecific faces than other species faces at forward-facing and diagonal orientations; (iii) longer towards chimpanzee faces compared with chimpanzee skulls and skull-shaped stones, and (iv) attended significantly longer to the teeth, similar to findings for elephants. We suggest that chimpanzee skulls retain relevant, face-like features that arguably activate a domain-specific face module in chimpanzees' brains, guiding their attention.

Illusory faces are more likely to be perceived as male than female

Face pareidolia is the phenomenon of perceiving illusory faces in inanimate objects. Here we show that illusory faces engage social perception beyond the detection of a face: they have a perceived age, gender, and emotional expression. Additionally, we report a striking bias in gender perception, with many more illusory faces perceived as male than female. As illusory faces do not have a biological sex, this bias is significant in revealing an asymmetry in our face evaluation system given minimal information. Our result demonstrates that the visual features that are sufficient for face detection are not generally sufficient for the perception of female. Instead, the perception of a nonhuman face as female requires additional features beyond that required for face detection.

Despite our fluency in reading human faces, sometimes we mistakenly perceive illusory faces in objects, a phenomenon known as face pareidolia. Although illusory faces share some neural mechanisms with real faces, it is unknown to what degree pareidolia engages higher-level social perception beyond the detection of a face. In a series of large-scale behavioral experiments (n_total = 3,815 adults), we found that illusory faces in inanimate objects are readily perceived to have a specific emotional expression, age, and gender. Most strikingly, we observed a strong bias to perceive illusory faces as male rather than female. This male bias could not be explained by preexisting semantic or visual gender associations with the objects, or by visual features in the images. Rather, this robust bias in the perception of gender for illusory faces reveals a cognitive bias arising from a broadly tuned face evaluation system in which minimally viable face percepts are more likely to be perceived as male.

Rapid neural categorization of facelike objects predicts the perceptual awareness of a face (face pareidolia)

The human brain rapidly and automatically categorizes faces vs. other visual objects. However, whether face-selective neural activity predicts the subjective experience of a face - perceptual awareness - is debated. To clarify this issue, here we use face pareidolia, i.e., the illusory perception of a face, as a proxy to relate the neural categorization of a variety of facelike objects to conscious face perception. In Experiment 1, scalp electroencephalogram (EEG) is recorded while pictures of human faces or facelike objects - in different stimulation sequences - are interleaved every second (i.e., at 1 Hz) in a rapid 6-Hz train of natural images of nonface objects. Participants do not perform any explicit face categorization task during stimulation, and report whether they perceived illusory faces post-stimulation. A robust categorization response to facelike objects is identified at 1 Hz and harmonics in the EEG frequency spectrum with a facelike occipito-temporal topography. Across all individuals, the facelike categorization response is of about 20% of the response to human faces, but more strongly right-lateralized. Critically, its amplitude is much larger in participants who report having perceived illusory faces. In Experiment 2, facelike or matched nonface objects from the same categories appear at 1 Hz in sequences of nonface objects presented at variable stimulation rates (60 Hz to 12 Hz) and participants explicitly report after each sequence whether they perceived illusory faces. The facelike categorization response already emerges at the shortest stimulus duration (i.e., 17 ms at 60 Hz) and predicts the behavioral report of conscious perception. Strikingly, neural facelike-selectivity emerges exclusively when participants report illusory faces. Collectively, these experiments characterize a neural signature of face pareidolia in the context of rapid categorization, supporting the view that face-selective brain activity reliably predicts the subjective experience of a face from a single glance at a variety of stimuli.

Objects that induce face pareidolia are prioritized by the visual system

The human visual system has evolved specialized neural mechanisms to rapidly detect faces. Its broad tuning for facial features is thought to underlie the illusory perception of faces in inanimate objects, a phenomenon called face pareidolia. Recent studies on face pareidolia suggest that the mechanisms underlying face processing, at least at the early stages of visual encoding, may treat objects that resemble faces as real faces; prioritizing their detection. In our study, we used breaking continuous flash suppression (b-CFS) to examine whether the human visual system prioritizes the detection of objects that induce face pareidolia over stimuli matched for object content. Similar to previous b-CFS results using real face stimuli, we found that participants detected the objects with pareidolia faces faster than object-matched control stimuli. Given that face pareidolia has been more frequently reported amongst individuals prone to hallucinations, we also explored whether this rapid prioritization is intact in individuals with schizophrenia, and found evidence suggesting that it was. Our findings suggest that face pareidolia engages a broadly tuned mechanism that facilitates rapid face detection. This may involve the proposed fast subcortical pathway that operates outside of visual awareness.

The involvement of monocular channels in the face pareidolia effect

Studies examining the neural mechanisms of face perception in humans have mainly focused on cortical networks of face-selective regions. However, subcortical regions are known to play a significant role in face perception as well. For instance, upon presenting pairs of faces sequentially to the same eye or to different eyes, superior performance is observed in the former condition. This superiority was explained by monocular, pre-striate processing of face stimuli. One of the intriguing face-related effects is the face pareidolia phenomenon, wherein observers perceive faces in inanimate objects. In this study, we examined whether face pareidolia involves similar low-level neural substrates to those that are involved in face perception. We presented participants with pairs of houses or face-like houses using a stereoscope to manipulate the information presented to each eye and asked them to determine whether the stimuli were similar or different. We managed to examine the contribution of monocular channels (mostly subcortical) in processing face-like stimuli. We hypothesized that besides their involvement in actual face perception, subcortical structures are engaged in face pareidolia as well. To test our hypothesis, we conducted three experiments to replicate and strengthen the reliability of our results and rule out alternative explanations. We demonstrated a perceptual benefit when presenting similar face-like houses to the same eye in comparison to their presentation to different eyes. This finding matches previous results found for images of real faces and indicates subcortical involvement not only in face perception but also in processing face-like objects.

Visual prototypes in the ventral stream are attuned to complexity and gaze behavior

Early theories of efficient coding suggested the visual system could compress the world by learning to represent features where information was concentrated, such as contours. This view was validated by the discovery that neurons in posterior visual cortex respond to edges and curvature. Still, it remains unclear what other information-rich features are encoded by neurons in more anterior cortical regions (e.g., inferotemporal cortex). Here, we use a generative deep neural network to synthesize images guided by neuronal responses from across the visuocortical hierarchy, using floating microelectrode arrays in areas V1, V4 and inferotemporal cortex of two macaque monkeys. We hypothesize these images ("prototypes") represent such predicted information-rich features. Prototypes vary across areas, show moderate complexity, and resemble salient visual attributes and semantic content of natural images, as indicated by the animals' gaze behavior. This suggests the code for object recognition represents compressed features of behavioral relevance, an underexplored aspect of efficient coding.

Single-Unit Recordings Reveal the Selectivity of a Human Face Area

The exquisite capacity of primates to detect and recognize faces is crucial for social interactions. Although disentangling the neural basis of human face recognition remains a key goal in neuroscience, direct evidence at the single-neuron level is limited. We recorded from face-selective neurons in human visual cortex in a region characterized by functional magnetic resonance imaging (fMRI) activations for faces compared with objects. The majority of visually responsive neurons in this fMRI activation showed strong selectivity at short latencies for faces compared with objects. Feature-scrambled faces and face-like objects could also drive these neurons, suggesting that this region is not tightly tuned to the visual attributes that typically define whole human faces. These single-cell recordings within the human face processing system provide vital experimental evidence linking previous imaging studies in humans and invasive studies in animal models.SIGNIFICANCE STATEMENT We present the first recordings of face-selective neurons in or near an fMRI-defined patch in human visual cortex. Our unbiased multielectrode array recordings (i.e., no selection of neurons based on a search strategy) confirmed the validity of the BOLD contrast (faces-objects) in humans, a finding with implications for all human imaging studies. By presenting faces, feature-scrambled faces, and face-pareidolia (perceiving faces in inanimate objects) stimuli, we demonstrate that neurons at this level of the visual hierarchy are broadly tuned to the features of a face, independent of spatial configuration and low-level visual attributes.

Social prediction modulates activity of macaque superior temporal cortex

The ability to attribute thoughts to others, also called theory of mind (TOM), has been extensively studied in humans; however, its evolutionary origins have been challenged. Computationally, the basis of TOM has been interpreted within the predictive coding framework and associated with activity in the temporoparietal junction (TPJ). Here, we revealed, using a nonlinguistic task and functional magnetic resonance imaging, that activity in a region of the macaque middle superior temporal cortex was specifically modulated by the predictability of social situations. As in human TPJ, this region could be distinguished from other temporal regions involved in face processing. Our result suggests the existence of a precursor for the TOM ability in the last common ancestor of human and Old World monkeys.

Reading Covered Faces

Covering faces with masks, due to mandatory pandemic safety regulations, we can no longer rely on the habitual daily-life information. This may be thought-provoking for healthy people, but particularly challenging for individuals with neuropsychiatric and neurodevelopmental conditions. Au fait research on reading covered faces reveals that: 1) wearing masks hampers facial affect recognition, though it leaves reliable inferring basic emotional expressions; 2) by buffering facial affect, masks lead to narrowing of emotional spectrum and dampen veridical evaluation of counterparts; 3) masks may affect perceived face attractiveness; 4) covered (either by masks or other veils) faces have a certain signal function introducing perceptual biases and prejudices; 5) reading covered faces is gender- and age-specific, being more challenging for males and more variable even in healthy aging; 6) the hampering effects of masks on social cognition occur over the globe; and 7) reading covered faces is likely to be supported by the large-scale assemblies of the neural circuits far beyond the social brain. Challenges and limitations of ongoing research and parallels to the Reading the Mind in the Eyes Test are assessed. Clarification of how masks affect face reading in the real world, where we deal with dynamic faces and have entrée to additional valuable social signals such as body language, as well as the specificity of neural networks underlying reading covered faces calls for further tailored research.

Social cognition in individuals born preterm

Faces hold a substantial value for effective social interactions and sharing. Covering faces with masks, due to COVID-19 regulations, may lead to difficulties in using social signals, in particular, in individuals with neurodevelopmental conditions. Daily-life social participation of individuals who were born preterm is of immense importance for their quality of life. Here we examined face tuning in individuals (aged 12.79 ± 1.89 years) who were born preterm and exhibited signs of periventricular leukomalacia (PVL), a dominant form of brain injury in preterm birth survivors. For assessing the face sensitivity in this population, we implemented a recently developed experimental tool, a set of Face-n-Food images bordering on the style of Giuseppe Arcimboldo. The key benefit of these images is that single components do not trigger face processing. Although a coarse face schema is thought to be hardwired in the brain, former preterms exhibit substantial shortages in the face tuning not only compared with typically developing controls but also with individuals with autistic spectrum disorders. The lack of correlations between the face sensitivity and other cognitive abilities indicates that these deficits are domain-specific. This underscores impact of preterm birth sequelae for social functioning at large. Comparison of the findings with data in individuals with other neurodevelopmental and neuropsychiatric conditions provides novel insights into the origins of deficient face processing.

A shared mechanism for facial expression in human faces and face pareidolia

Facial expressions are vital for social communication, yet the underlying mechanisms are still being discovered. Illusory faces perceived in objects (face pareidolia) are errors of face detection that share some neural mechanisms with human face processing. However, it is unknown whether expression in illusory faces engages the same mechanisms as human faces. Here, using a serial dependence paradigm, we investigated whether illusory and human faces share a common expression mechanism. First, we found that images of face pareidolia are reliably rated for expression, within and between observers, despite varying greatly in visual features. Second, they exhibit positive serial dependence for perceived facial expression, meaning an illusory face (happy or angry) is perceived as more similar in expression to the preceding one, just as seen for human faces. This suggests illusory and human faces engage similar mechanisms of temporal continuity. Third, we found robust cross-domain serial dependence of perceived expression between illusory and human faces when they were interleaved, with serial effects larger when illusory faces preceded human faces than the reverse. Together, the results support a shared mechanism for facial expression between human faces and illusory faces and suggest that expression processing is not tightly bound to human facial features.

Face Tuning in Depression

The latest COVID-19 pandemic reveals that unexpected changes elevate depression bringing people apart, but also calling for social sharing. Yet the impact of depression on social cognition and functioning is not well understood. Assessment of social cognition is crucial not only for a better understanding of major depressive disorder (MDD), but also for screening, intervention, and remediation. Here by applying a novel experimental tool, a Face-n-Food task comprising a set of images bordering on the Giuseppe Arcimboldo style, we assessed the face tuning in patients with MDD and person-by-person matched controls. The key benefit of these images is that single components do not trigger face processing. Contrary to common beliefs, the outcome indicates that individuals with depression express intact face responsiveness. Yet, while in depression face sensitivity is tied with perceptual organization, in typical development, it is knotted with social cognition capabilities. Face tuning in depression, therefore, may rely upon altered behavioral strategies and underwriting brain mechanisms. To exclude a possible camouflaging effect of female social skills, we examined gender impact. Neither in depression nor in typical individuals had females excelled in face tuning. The outcome sheds light on the origins of the face sensitivity and alterations in social functioning in depression and mental well-being at large. Aberrant social functioning in depression is likely to be the result of deeply-rooted maladaptive strategies rather than of poor sensitivity to social signals. This has implications for mental well-being under the current pandemic conditions.

Comparison of Scalp ERP to Faces in Macaques and Humans

Face recognition is an essential activity of social living, common to many primate species. Underlying processes in the brain have been investigated using various techniques and compared between species. Functional imaging studies have shown face-selective cortical regions and their degree of correspondence across species. However, the temporal dynamics of face processing, particularly processing speed, are likely different between them. Across sensory modalities activation of primary sensory cortices in macaque monkeys occurs at about 3/5 the latency of corresponding activation in humans, though this human simian difference may diminish or disappear in higher cortical regions. We recorded scalp event-related potentials (ERPs) to presentation of faces in macaques and estimated the peak latency of ERP components. Comparisons of latencies between macaques (112 ms) and humans (192 ms) suggested that the 3:5 ratio could be preserved in higher cognitive regions of face processing between those species.

View-tuned and view-invariant face encoding in IT cortex is explained by selected natural image fragments

Humans recognize individual faces regardless of variation in the facial view. The view-tuned face neurons in the inferior temporal (IT) cortex are regarded as the neural substrate for view-invariant face recognition. This study approximated visual features encoded by these neurons as combinations of local orientations and colors, originated from natural image fragments. The resultant features reproduced the preference of these neurons to particular facial views. We also found that faces of one identity were separable from the faces of other identities in a space where each axis represented one of these features. These results suggested that view-invariant face representation was established by combining view sensitive visual features. The face representation with these features suggested that, with respect to view-invariant face representation, the seemingly complex and deeply layered ventral visual pathway can be approximated via a shallow network, comprised of layers of low-level processing for local orientations and colors (V1/V2-level) and the layers which detect particular sets of low-level elements derived from natural image fragments (IT-level).

A visual search advantage for illusory faces in objects

Face detection is a priority of both the human and primate visual system. However, occasionally we misperceive faces in inanimate objects -- "face pareidolia". A key feature of these 'false positives' is that face perception occurs in the absence of visual features typical of real faces. Human faces are known to be located faster than objects in visual search. Here we used a visual search paradigm to test whether illusory faces share this advantage. Search times were faster for illusory faces than for matched objects amongst both matched (Experiment 1) and diverse (Experiment 2) distractors, however search times for real human faces were faster and more efficient than objects with or without an illusory face. Importantly, this result indicates that illusory faces are processed quickly enough by the human brain to confer a visual search advantage, suggesting the engagement of a broadly-tuned mechanism that facilitates rapid face detection in cluttered environments.

Do you see the “face”? Individual differences in face pareidolia

People tend to see faces from non-face objects or meaningless patterns. Such illusory face perception is called face pareidolia. Previous studies have revealed an interesting fact that there are huge individual differences in face pareidolia experience among the population. Here, we review previous findings on individual differences in face pareidolia experience from four categories: sex differences, developmental factors, personality traits and neurodevelopmental factors. We further discuss underlying cognitive or neural mechanisms to explain why some perceive the objects as faces while others do not. The individual differences in face pareidolia could not only offer scientific insights on how the brain works to process face information, but also suggest potential clinical applications.

Face pareidolia in the brain: Impact of gender and orientation

Research on face sensitivity is of particular relevance during the rapidly evolving Covid-19 pandemic leading to social isolation, but also calling for intact interaction and sharing. Humans possess high sensitivity even to a coarse face scheme, seeing faces in non-face images where real faces do not exist. The advantage of non-face images is that single components do not trigger face processing. Here by implementing a novel set of Face-n-Thing images, we examined (i) how face tuning alters with changing display orientation, and (ii) whether it is affected by observers’ gender. Young females and males were presented with a set of Face-n-Thing images either with canonical upright orientation or inverted 180° in the image plane. Face impression was substantially impeded by display inversion. Furthermore, whereas with upright display orientation, no gender differences were found, with inversion, Face-n-Thing images elicited face impression in females significantly more often. The outcome sheds light on the origins of the face inversion effect in general. Moreover, the findings open a way for examination of face sensitivity and underwriting brain networks in neuropsychiatric conditions related to the current pandemic (such as depression and anxiety), most of which are gender/sex-specific.

Individually distinctive features facilitate numerical discrimination of sets of objects in domestic chicks

Day-old domestic chicks approach the larger of two groups of identical objects, but in a 3 vs 4 comparison, their performance is random. Here we investigated whether adding individually distinctive features to each object would facilitate such discrimination. Chicks reared with 7 objects were presented with the operation 1 + 1 + 1 vs 1 + 1 + 1 + 1. When objects were all identical, chicks performed randomly, as expected (Experiment 1). In the remaining experiments, objects differed from one another due to additional features. Chicks succeeded when those features were differently oriented segments (Experiment 2) but failed when the features were arranged to depict individually different face-like displays (Experiment 3). Discrimination was restored if the face-like stimuli were presented upside-down, disrupting global processing (Experiment 4). Our results support the claim that numerical discrimination in 3 vs 4 comparison benefits from the presence of distinctive features that enhance object individuation due to individual processing. Interestingly, when the distinctive features are arranged into upright face-like displays, the process is susceptible to global over local interference due to configural processing. This study was aimed at assessing whether individual object processing affects numerical discrimination. We hypothesise that in humans similar strategies aimed at improving performance at the non-symbolic level may have positive effects on symbolic mathematical abilities.

What does a "face cell" want?'

In the 1970s Charlie Gross was among the first to identify neurons that respond selectively to faces, in the macaque inferior temporal (IT) cortex. This seminal finding has been followed by numerous studies quantifying the visual features that trigger a response from face cells in order to answer the question; what do face cells want? However, the connection between face-selective activity in IT cortex and visual perception remains only partially understood. Here we present fMRI results in the macaque showing that some face patches respond to illusory facial features in objects. We argue that to fully understand the functional role of face cells, we need to develop approaches that test the extent to which their response explains what we see.

Rapid and dynamic processing of face pareidolia in the human brain

The human brain is specialized for face processing, yet we sometimes perceive illusory faces in objects. It is unknown whether these natural errors of face detection originate from a rapid process based on visual features or from a slower, cognitive re-interpretation. Here we use a multifaceted approach to understand both the spatial distribution and temporal dynamics of illusory face representation in the brain by combining functional magnetic resonance imaging and magnetoencephalography neuroimaging data with model-based analysis. We find that the representation of illusory faces is confined to occipital-temporal face-selective visual cortex. The temporal dynamics reveal a striking evolution in how illusory faces are represented relative to human faces and matched objects. Illusory faces are initially represented more similarly to real faces than matched objects are, but within ~250 ms, the representation transforms, and they become equivalent to ordinary objects. This is consistent with the initial recruitment of a broadly-tuned face detection mechanism which privileges sensitivity over selectivity.

Human and monkey infant attention to dynamic social and nonsocial stimuli

The present study explored behavioral norms for infant social attention in typically developing human and nonhuman primate infants. We examined the normative development of attention to dynamic social and nonsocial stimuli longitudinally in macaques (Macaca mulatta) at 1, 3, and 5 months of age (N = 75) and humans at 2, 4, 6, 8, and 13 months of age (N = 69) using eye tracking. All infants viewed concurrently played silent videos-one social video and one nonsocial video. Both macaque and human infants were faster to look to the social than the nonsocial stimulus, and both species grew faster to orient to the social stimulus with age. Further, macaque infants' social attention increased linearly from 1 to 5 months. In contrast, human infants displayed a nonlinear pattern of social interest, with initially greater attention to the social stimulus, followed by a period of greater interest in the nonsocial stimulus, and then a rise in social interest from 6 to 13 months. Overall, human infants looked longer than macaque infants, suggesting humans have more sustained attention in the first year of life. These findings highlight potential species similarities and differences, and reflect a first step in establishing baseline patterns of early social attention development.

Face Pareidolia Recruits Mechanisms for Detecting Human Social Attention

Face pareidolia is the phenomenon of seeing facelike structures in everyday objects. Here, we tested the hypothesis that face pareidolia, rather than being limited to a cognitive or mnemonic association, reflects the activation of visual mechanisms that typically process human faces. We focused on sensory cues to social attention, which engage cell populations in temporal cortex that are susceptible to habituation effects. Repeated exposure to “pareidolia faces” that appear to have a specific direction of attention causes a systematic bias in the perception of where human faces are looking, indicating that overlapping sensory mechanisms are recruited when we view human faces and when we experience face pareidolia. These cross-adaptation effects are significantly reduced when pareidolia is abolished by removing facelike features from the objects. These results indicate that face pareidolia is essentially a perceptual phenomenon, occurring when sensory input is processed by visual mechanisms that have evolved to extract specific social content from human faces.

Face pareidolia in schizophrenia

BACKGROUND

Faces convey valuable daily life social signals. As in most psychiatric conditions, non-verbal social cognition or its components including face processing may be aberrant in schizophrenia (SZ). Social participation of individuals with SZ is vital for their quality of life, and remediation of social abilities in this population is of high relevance both for society and clinical care.

METHOD

Tuning to faces in non-face images such as shadows, grilled toasts, or ink blots is called face pareidolia. Humans possess high sensitivity to facial signals: even fetuses and infants are well tuned to coarse face cues. Here we assessed face tuning in individuals with SZ and person-by-person matched controls by using a new experimental tool, a set of food-plate images bordering on the Giuseppe Arcimboldo style. The key benefit of these images is that single components do not trigger face processing.

RESULTS AND CONCLUSIONS

The outcome indicates that individuals with SZ exhibit aberrant face tuning in face-like non-face images (χ²(1) = 17.44, p = 0.0001) that can hamper adaptive interaction with peers and social participation hindering, in turn, clinical remediation. Face response rate in SZ patients was related to the scores on the event arrangement task tapping social cognition (Pearson product-moment correlation, r = 0.602, p = 0.01) and on picture completion task assessing visual perceptual organization (Spearman's rho = 0.614, p = 0.009). Therefore, poor performance on the face tuning task is unlikely to be accounted for by deviant general cognitive abilities, but rather by impairments in perceptual integration and social cognition. Comparison of these findings with data in autism and other neuropsychiatric conditions provides novel insights on the origins of face tuning in SZ and triggers brain imaging research.

A Prototypical Template for Rapid Face Detection Is Embedded in the Monkey Superior Colliculus

Human babies respond preferentially to faces or face-like images. It has been proposed that an innate and rapid face detection system is present at birth before the cortical visual pathway is developed in many species, including primates. However, in primates, the visual area responsible for this process is yet to be unraveled. We hypothesized that the superior colliculus (SC) that receives direct and indirect retinal visual inputs may serve as an innate rapid face-detection system in primates. To test this hypothesis, we examined the responsiveness of monkey SC neurons to first-order information of faces required for face detection (basic spatial layout of facial features including eyes, nose, and mouth), by analyzing neuronal responses to line drawing images of: (1) face-like patterns with contours and properly placed facial features; (2) non-face patterns including face contours only; and (3) nonface random patterns with contours and randomly placed face features. Here, we show that SC neurons respond stronger and faster to upright and inverted face-like patterns compared to the responses to nonface patterns, regardless of contrast polarity and contour shapes. Furthermore, SC neurons with central receptive fields (RFs) were more selective to face-like patterns. In addition, the population activity of SC neurons with central RFs can discriminate face-like patterns from nonface patterns as early as 50 ms after the stimulus onset. Our results provide strong neurophysiological evidence for the involvement of the primate SC in face detection and suggest the existence of a broadly tuned template for face detection in the subcortical visual pathway.

Intranasal oxytocin selectively modulates the behavior of rhesus monkeys in an expression matching task

Although the neuropeptide oxytocin (OT) is thought to regulate prosocial behavior in mammals, there is considerable debate as to how intranasal OT influences primate behavior. The aim of this study was to determine whether intranasal OT has a general anxiolytic effect on the performance of rhesus monkeys tasked with matching face stimuli, or a more selective effect on their behavior towards aversive facial expressions. To this end, we developed an innovative delayed match-to-sample task where the exact same trials could be used to assess either a monkey’s ability to match facial expressions or facial identities. If OT has a general affect on behavior, then performance in both tasks should be altered by the administration of OT. We tested four male rhesus monkeys (Macaca mulatta) in both the expression and identity task after the intranasal administration of either OT or saline in a within-subjects design. We found that OT inhalation selectively reduced a selection bias against negatively valenced expressions. Based on the same visual input, performance in the identity task was also unaffected by OT. This dissociation provides evidence that intranasal OT affects primate behavior under very particular circumstances, rather than acting as a general anxiolytic, in a highly translatable nonhuman model, the rhesus monkey.

What Is a Face? Critical Features for Face Detection

Faces convey very rich information that is critical for intact social interaction. To extract this information efficiently, faces should be easily detected from a complex visual scene. Here, we asked which features are critical for face detection. To answer this question, we presented non-face objects that generate a strong percept of a face (i.e., Pareidolia). One group of participants rated the faceness of this set of inanimate images. A second group rated the presence of a set of 12 local and global facial features. Regression analysis revealed that only the eyes or mouth significantly contributed to faceness scores. We further showed that removing eyes or mouth, but not teeth or ears, significantly reduced faceness scores. These findings show that face detection depends on specific facial features, the eyes and the mouth. This minimal information leads to over-generalization that generates false face percepts but assures that real faces are not missed.

A behavioral face preference deficit in a monkey with an incomplete face patch system

Primates are experts in face perception and naturally show a preference for faces under free-viewing conditions. The primate ventral stream is characterized by a network of face patches that selectively responds to faces, but it remains uncertain how important such parcellation is for face perception. Here we investigated free-viewing behavior in a female monkey who naturally lacks fMRI-defined posterior and middle lateral face patches. We presented a series of content-rich images of scenes that included faces or other objects to that monkey during a free-viewing task and tested a group of 10 control monkeys on the same task for comparison. We found that, compared to controls, the monkey with missing face patches showed a marked reduction of face viewing preference that was most pronounced for the first few fixations. In addition, her gaze fixation patterns were substantially distinct from those of controls, especially for pictures with a face. These data demonstrate an association between the clustering of neurons in face selective patches and a behavioral bias for faces in natural images.

Experimental evidence that female rhesus macaques (Macaca mulatta) perceive variation in male facial masculinity

Among many primate species, face shape is sexually dimorphic, and male facial masculinity has been proposed to influence female mate choice and male-male competition by signalling competitive ability. However, whether conspecifics pay attention to facial masculinity has only been assessed in humans. In a study of free-ranging rhesus macaques, Macaca mulatta, we used a two-alternative look-time experiment to test whether females perceive male facial masculinity. We presented 107 females with pairs of images of male faces-one with a more masculine shape and one more feminine-and recorded their looking behaviour. Females looked at the masculine face longer than at the feminine face in more trials than predicted by chance. Although there was no overall difference in average look-time between masculine and feminine faces across all trials, females looked significantly longer at masculine faces in a subset of trials for which the within-pair difference in masculinity was most pronounced. Additionally, the proportion of time subjects looked toward the masculine face increased as the within-pair difference in masculinity increased. This study provides evidence that female macaques perceive variation in male facial shape, a necessary condition for intersexual selection to operate on such a trait. It also highlights the potential impact of perceptual thresholds on look-time experiments.