Neural Mechanisms for Accepting and Rejecting Artificial Social Partners in the Uncanny Valley

Chikamatsu, Mori, and the uncanny valley

In Japan, robotics projects like Geminoid, modeled after Hiroshi Ishiguro, exhibit a fascination with creating human doubles. Yet, warnings against this also thread through Japanese thought, from the Edo-period playwright Chikamatsu Monzaemon (1653-1724) to the robotics professor Mori Masahiro (1927-2025). Though centuries apart, they describe the same uncanny valley phenomenon-eerie, cold, repellent feelings that arise when confronting the imperfectly human. In an interview with Hozumi Ikan, translated here, Chikamatsu presents a theory of realism exemplified through puppet theater and kabuki. He divides realism into four zones: the unreal, conceptual realism, surface realism, and the real. The unreal lacks authenticity, surface realism lacks soul, and the real lacks expressiveness. For Chikamatsu, it is conceptual realism that captivates an audience. A play's unfolding events evoke empathy and emotion through their meaning for the characters. Similarly, Mori divides realism into four zones: industrial, humanoid, and android robots, and real people. Industrial robots evoke little affinity, and androids risk appearing eerie. Though real people evoke the most affinity, androids cannot become indistinguishable from them. For Mori, only humanoid robots evoke affinity without risking uncanniness. By exploring anthropomorphism, both Chikamatsu and Mori illuminate principles for designing robots that do not unsettle but delight.

Grasp and remember: the impact of human and robotic actions on object preference and memory

Goal contagion, the tendency to adopt others' goals, significantly impacts cognitive processes, which gains particular importance in the emerging field of human-robot interactions. The present study explored how observing human versus robotic actions affects preference and memory. Series of objects undergoing either human or robotic grasping actions together with static (no action) objects were presented, while participants indicated their preference for each object. After a short delay, their memory for grasped, static and new (unstudied) stimuli was tested. Human actions enhanced preference and subsequent recollection of objects, more than robotic actions. In the context of human action, static objects were also perceived as more familiar at recognition. The goal contagion's influence on memory was found to be independent from its impact on preference. These findings highlight the critical role of human interaction in eliciting the impact of goal contagion on cognitive evaluations, memory engagement and the creation of detailed associative memories.

Explorations in Creepiness: Tolerance for Ambiguity and Susceptibility to "Not Just Right Experiences" Predict the Ease of Getting "Creeped Out"

This study was designed to explore the role played by ambiguity in the experience of creepiness, as well as the relevance of personality traits for predicting individual differences in susceptibility to getting "creeped out," In an online study, a mixed sample of 278 college undergraduates and adults (60 males, 206 females, 12 nonbinary or chose not to report; Mean age = 31.43, range 18-68) recruited through social network platforms filled out scales measuring their tolerance for ambiguity and their susceptibility to having "Not Just Right Experiences." They then rated 25 images (12 normal, 13 prejudged to be creepy or confusing) on creepiness and several other adjective dimensions. The findings indicated that individuals who were less tolerant of ambiguity and those highly susceptible to not just right experiences perceived ambiguous or creepy persons, places, and objects to be more creepy, confusing and disturbing. Both measures were negatively related to time spent looking at confusing or creepy images, and females were generally more easily creeped out by creepy and confusing images than were males. The results support the conclusion that current models of creepiness are correct; the emotional experience of getting "creeped out" does indeed appear to be triggered by the need to resolve ambiguity.

The Neuroscience of Human and Artificial Intelligence Presence

Two decades of social neuroscience and neuroeconomics research illustrate the brain mechanisms that are engaged when people consider human beings, often in comparison to considering artificial intelligence (AI) as a nonhuman control. AI as an experimental control preserves agency and facilitates social interactions but lacks a human presence, providing insight into brain mechanisms that are engaged by human presence and the presence of AI. Here, I review this literature to determine how the brain instantiates human and AI presence across social perception and decision-making paradigms commonly used to realize a social context. People behave toward humans differently than they do toward AI. Moreover, brain regions more engaged by humans compared to AI extend beyond the social cognition brain network to all parts of the brain, and the brain sometimes is engaged more by AI than by humans. Finally, I discuss gaps in the literature, limitations in current neuroscience approaches, and how an understanding of the brain correlates of human and AI presence can inform social science in the wild.

A Neural Mechanism in the Human Orbitofrontal Cortex for Preferring High-Fat Foods Based on Oral Texture

Although overconsumption of high-fat foods is a major driver of weight gain, the neural mechanisms that link the oral sensory properties of dietary fat to reward valuation and eating behavior remain unclear. Here we combine novel food-engineering approaches with functional neuroimaging to show that the human orbitofrontal cortex (OFC) translates oral sensations evoked by high-fat foods into subjective economic valuations that guide eating behavior. Male and female volunteers sampled and evaluated nutrient-controlled liquid foods that varied in fat and sugar ("milkshakes"). During oral food processing, OFC activity encoded a specific oral-sensory parameter that mediated the influence of the foods' fat content on reward value: the coefficient of sliding friction. Specifically, OFC responses to foods in the mouth reflected the smooth, oily texture (i.e., mouthfeel) produced by fatty liquids on oral surfaces. Distinct activity patterns in OFC encoded the economic values associated with particular foods, which reflected the subjective integration of sliding friction with other food properties (sugar, fat, viscosity). Critically, neural sensitivity of OFC to oral texture predicted individuals' fat preferences in a naturalistic eating test: individuals whose OFC was more sensitive to fat-related oral texture consumed more fat during ad libitum eating. Our findings suggest that reward systems of the human brain sense dietary fat from oral sliding friction, a mechanical food parameter that likely governs our daily eating experiences by mediating interactions between foods and oral surfaces. These findings identify a specific role for the human OFC in evaluating oral food textures to mediate preference for high-fat foods.SIGNIFICANCE STATEMENT Fat and sugar enhance the reward value of food by imparting a sweet taste and rich mouthfeel but also contribute to overeating and obesity. Here we used a novel food-engineering approach to realistically quantify the physical-mechanical properties of high-fat liquid foods on oral surfaces and used functional neuroimaging while volunteers sampled these foods and placed monetary bids to consume them. We found that a specific area of the brain's reward system, the orbitofrontal cortex, detects the smooth texture of fatty foods in the mouth and links these sensory inputs to economic valuations that guide eating behavior. These findings can inform the design of low-calorie fat-replacement foods that mimic the impact of dietary fat on oral surfaces and neural reward systems.

Disentangling five dimensions of animacy in human brain and behaviour

Distinguishing animate from inanimate things is of great behavioural importance. Despite distinct brain and behavioural responses to animate and inanimate things, it remains unclear which object properties drive these responses. Here, we investigate the importance of five object dimensions related to animacy (“being alive”, “looking like an animal”, “having agency”, “having mobility”, and “being unpredictable”) in brain (fMRI, EEG) and behaviour (property and similarity judgements) of 19 participants. We used a stimulus set of 128 images, optimized by a genetic algorithm to disentangle these five dimensions. The five dimensions explained much variance in the similarity judgments. Each dimension explained significant variance in the brain representations (except, surprisingly, “being alive”), however, to a lesser extent than in behaviour. Different brain regions sensitive to animacy may represent distinct dimensions, either as accessible perceptual stepping stones toward detecting whether something is alive or because they are of behavioural importance in their own right.

Corneal reflections and skin contrast yield better memory of human and virtual faces

Virtual faces have been found to be rated less human-like and remembered worse than photographic images of humans. What it is in virtual faces that yields reduced memory has so far remained unclear. The current study investigated face memory in the context of virtual agent faces and human faces, real and manipulated, considering two factors of predicted influence, i.e., corneal reflections and skin contrast. Corneal reflections referred to the bright points in each eye that occur when the ambient light reflects from the surface of the cornea. Skin contrast referred to the degree to which skin surface is rough versus smooth. We conducted two memory experiments, one with high-quality virtual agent faces (Experiment 1) and the other with the photographs of human faces that were manipulated (Experiment 2). Experiment 1 showed better memory for virtual faces with increased corneal reflections and skin contrast (rougher rather than smoother skin). Experiment 2 replicated these findings, showing that removing the corneal reflections and smoothening the skin reduced memory recognition of manipulated faces, with a stronger effect exerted by the eyes than the skin. This study highlights specific features of the eyes and skin that can help explain memory discrepancies between real and virtual faces and in turn elucidates the factors that play a role in the cognitive processing of faces.

Causal Analysis of Activity in Social Brain Areas During Human-Agent Conversation

This article investigates the differences in cognitive and neural mechanisms between human-human and human-virtual agent interaction using a dataset recorded in an ecologically realistic environment. We use Convergent Cross Mapping (CCM) to investigate functional connectivity between pairs of regions involved in the framework of social cognitive neuroscience, namely the fusiform gyrus, superior temporal sulcus (STS), temporoparietal junction (TPJ), and the dorsolateral prefrontal cortex (DLPFC)-taken as prefrontal asymmetry. Our approach is a compromise between investigating local activation in specific regions and investigating connectivity networks that may form part of larger networks. In addition to concording with previous studies, our results suggest that the right TPJ is one of the most reliable areas for assessing processes occurring during human-virtual agent interactions, both in a static and dynamic sense.

A Meta-analysis of the Uncanny Valley's Independent and Dependent Variables

The uncanny valley (UV) effect is a negative affective reaction to human-looking artificial entities. It hinders comfortable, trust-based interactions with android robots and virtual characters. Despite extensive research, a consensus has not formed on its theoretical basis or methodologies. We conducted a meta-analysis to assess operationalizations of human likeness (independent variable) and the UV effect (dependent variable). Of 468 studies, 72 met the inclusion criteria. These studies employed 10 different stimulus creation techniques, 39 affect measures, and 14 indirect measures. Based on 247 effect sizes, a three-level meta-analysis model revealed the UV effect had a large effect size, Hedges’ g = 1.01 [0.80, 1.22]. A mixed-effects meta-regression model with creation technique as the moderator variable revealed face distortion produced the largest effect size, g = 1.46 [0.69, 2.24], followed by distinct entities, g = 1.20 [1.02, 1.38], realism render, g = 0.99 [0.62, 1.36], and morphing, g = 0.94 [0.64, 1.24]. Affective indices producing the largest effects were threatening, likable, aesthetics, familiarity , and eeriness , and indirect measures were dislike frequency, categorization reaction time, like frequency, avoidance , and viewing duration . This meta-analysis—the first on the UV effect—provides a methodological foundation and design principles for future research.

Measurement of Trust in Automation: A Narrative Review and Reference Guide

With the rise of automated and autonomous agents, research examining Trust in Automation (TiA) has attracted considerable attention over the last few decades. Trust is a rich and complex construct which has sparked a multitude of measures and approaches to study and understand it. This comprehensive narrative review addresses known methods that have been used to capture TiA. We examined measurements deployed in existing empirical works, categorized those measures into self-report, behavioral, and physiological indices, and examined them within the context of an existing model of trust. The resulting work provides a reference guide for researchers, providing a list of available TiA measurement methods along with the model-derived constructs that they capture including judgments of trustworthiness, trust attitudes, and trusting behaviors. The article concludes with recommendations on how to improve the current state of TiA measurement.

Robot faces elicit responses intermediate to human faces and objects at face-sensitive ERP components

Face recognition is supported by selective neural mechanisms that are sensitive to various aspects of facial appearance. These include event-related potential (ERP) components like the P100 and the N170 which exhibit different patterns of selectivity for various aspects of facial appearance. Examining the boundary between faces and non-faces using these responses is one way to develop a more robust understanding of the representation of faces in extrastriate cortex and determine what critical properties an image must possess to be considered face-like. Robot faces are a particularly interesting stimulus class to examine because they can differ markedly from human faces in terms of shape, surface properties, and the configuration of facial features, but are also interpreted as social agents in a range of settings. In the current study, we thus chose to investigate how ERP responses to robot faces may differ from the response to human faces and non-face objects. In two experiments, we examined how the P100 and N170 responded to human faces, robot faces, and non-face objects (clocks). In Experiment 1, we found that robot faces elicit intermediate responses from face-sensitive components relative to non-face objects (clocks) and both real human faces and artificial human faces (computer-generated faces and dolls). These results suggest that while human-like inanimate faces (CG faces and dolls) are processed much like real faces, robot faces are dissimilar enough to human faces to be processed differently. In Experiment 2 we found that the face inversion effect was only partly evident in robot faces. We conclude that robot faces are an intermediate stimulus class that offers insight into the perceptual and cognitive factors that affect how social agents are identified and categorized.

A Computer-Based Method for the Investigation of Human Behavior in the Iterative Chicken Game

The present study develops an artificial agent that plays the iterative chicken game based on a computational model that describes human behavior in competitive social interactions in terms of fairness. The computational model we adopted in this study, named as the self-concept fairness model, decides the agent's action according to the evaluation of fairness of both opponent and self. We implemented the artificial agent in a computer program with a set of parameters adjustable by researchers. These parameters allow researchers to determine the extent to which the agent behaves aggressively or cooperatively. To demonstrate the use of the proposed method for the investigation of human behavior, we performed an experiment in which human participants played the iterative chicken game against the artificial agent. Participants were divided into two groups, each being informed to play with either a person or the computer. The behavioral analysis results showed that the proposed method can induce changes in the behavioral pattern of human players by changing the agent's behavioral pattern. Also, we found that participants tended to be more sensitive to fairness when they played with a human opponent than with a computer opponent. These results support that the artificial agent developed in this study will be useful to investigate human behavior in competitive social interactions.

Creepy cats and strange high houses: Support for configural processing in testing predictions of nine uncanny valley theories

In 1970, Masahiro Mori proposed the uncanny valley (UV), a region in a human-likeness continuum where an entity risks eliciting a cold, eerie, repellent feeling. Recent studies have shown that this feeling can be elicited by entities modeled not only on humans but also nonhuman animals. The perceptual and cognitive mechanisms underlying the UV effect are not well understood, although many theories have been proposed to explain them. To test the predictions of nine classes of theories, a within-subjects experiment was conducted with 136 participants. The theories' predictions were compared with ratings of 10 classes of stimuli on eeriness and coldness indices. One type of theory, configural processing, predicted eight out of nine significant effects. Atypicality, in its extended form, in which the uncanny valley effect is amplified by the stimulus appearing more human, also predicted eight. Threat avoidance predicted seven; atypicality, perceptual mismatch, and mismatch+ predicted six; category+, novelty avoidance, mate selection, and psychopathy avoidance predicted five; and category uncertainty predicted three. Empathy's main prediction was not supported. Given that the number of significant effects predicted depends partly on our choice of hypotheses, a detailed consideration of each result is advised. We do, however, note the methodological value of examining many competing theories in the same experiment.

Face the Uncanny: The Effects of Doppelganger Talking Head Avatars on Affect-Based Trust Toward Artificial Intelligence Technology are Mediated by Uncanny Valley Perceptions

This experiment (N = 228) examined how exposure to a talking head doppelganger created by an artificial intelligence (AI) program influenced affect-based trust toward AIs. Using a 3 (talking head featuring the participant's or a stranger's face, audio-only condition) by 2 (pro-AI pitch and anti-AI pitch playback) design, we uncovered that exposure to a talking head featuring the participant's face instead of a stranger's face increased uncanny valley perceptions. Furthermore, uncanny valley perceptions mediated the link between exposure to a talking head with the participant's face on affect-based trust. Overall, exposure to a doppelganger talking head, who delivered a persuasive pitch, triggered discomfort on the participant whose features were sourced to craft a synthetic talking head, which in turn decreased affect-based trust attributed to AIs. This phenomenon is rooted in basic psychological mechanisms that underpin the uncanny valley hypothesis. Future studies may test for these findings across different platforms and also provide evidence regarding user mental processing.

The Human Takes It All: Humanlike Synthesized Voices Are Perceived as Less Eerie and More Likable. Evidence From a Subjective Ratings Study

Background: The increasing involvement of social robots in human lives raises the question as to how humans perceive social robots. Little is known about human perception of synthesized voices.

Aim: To investigate which synthesized voice parameters predict the speaker's eeriness and voice likability; to determine if individual listener characteristics (e.g., personality, attitude toward robots, age) influence synthesized voice evaluations; and to explore which paralinguistic features subjectively distinguish humans from robots/artificial agents.

Methods: 95 adults (62 females) listened to randomly presented audio-clips of three categories: synthesized (Watson, IBM), humanoid (robot Sophia, Hanson Robotics), and human voices (five clips/category). Voices were rated on intelligibility, prosody, trustworthiness, confidence, enthusiasm, pleasantness, human-likeness, likability, and naturalness. Speakers were rated on appeal, credibility, human-likeness, and eeriness. Participants' personality traits, attitudes to robots, and demographics were obtained.

Results: The human voice and human speaker characteristics received reliably higher scores on all dimensions except for eeriness. Synthesized voice ratings were positively related to participants' agreeableness and neuroticism. Females rated synthesized voices more positively on most dimensions. Surprisingly, interest in social robots and attitudes toward robots played almost no role in voice evaluation. Contrary to the expectations of an uncanny valley, when the ratings of human-likeness for both the voice and the speaker characteristics were higher, they seemed less eerie to the participants. Moreover, when the speaker's voice was more humanlike, it was more liked by the participants. This latter point was only applicable to one of the synthesized voices. Finally, pleasantness and trustworthiness of the synthesized voice predicted the likability of the speaker's voice. Qualitative content analysis identified intonation, sound, emotion, and imageability/embodiment as diagnostic features.

Discussion: Humans clearly prefer human voices, but manipulating diagnostic speech features might increase acceptance of synthesized voices and thereby support human-robot interaction. There is limited evidence that human-likeness of a voice is negatively linked to the perceived eeriness of the speaker.

Seeing the mind of robots: Harm augments mind perception but benevolent intentions reduce dehumanisation of artificial entities in visual vignettes

According to moral typecasting theory, good- and evil-doers (agents) interact with the recipients of their actions (patients) in a moral dyad. When this dyad is completed, mind attribution towards intentionally harmed liminal minds is enhanced. However, from a dehumanisation view, malevolent actions may instead result in a denial of humanness. To contrast both accounts, a visual vignette experiment (N = 253) depicted either malevolent or benevolent intentions towards robotic or human avatars. Additionally, we examined the role of harm-salience by showing patients as either harmed, or still unharmed. The results revealed significantly increased mind attribution towards visibly harmed patients, mediated by perceived pain and expressed empathy. Benevolent and malevolent intentions were evaluated respectively as morally right or wrong, but their impact on the patient was diminished for the robotic avatar. Contrary to dehumanisation predictions, our manipulation of intentions failed to affect mind perception. Nonetheless, benevolent intentions reduced dehumanisation of the patients. Moreover, when pain and empathy were statistically controlled, the effect of intentions on mind perception was mediated by dehumanisation. These findings suggest that perceived intentions might only be indirectly tied to mind perception, and that their role may be better understood when additionally accounting for empathy and dehumanisation.

Macaque Gaze Responses to the Primatar: A Virtual Macaque Head for Social Cognition Research

Following the expanding use and applications of virtual reality in everyday life, realistic virtual stimuli are of increasing interest in cognitive studies. They allow for control of features such as gaze, expression, appearance, and movement, which may help to overcome limitations of using photographs or video recordings to study social responses. In using virtual stimuli however, one must be careful to avoid the uncanny valley effect, where realistic stimuli can be perceived as eerie, and induce an aversion response. At the same time, it is important to establish whether responses to virtual stimuli mirror responses to depictions of a real conspecific. In the current study, we describe the development of a new virtual monkey head with realistic facial features for experiments with nonhuman primates, the "Primatar." As a first step toward validation, we assessed how monkeys respond to facial images of a prototype of this Primatar compared to images of real monkeys (RMs), and an unrealistic model. We also compared gaze responses between original images and scrambled as well as obfuscated versions of these images. We measured looking time to images in six freely moving long-tailed macaques (Macaca fascicularis) and gaze exploration behavior in three rhesus macaques (Macaca mulatta). Both groups showed more signs of overt attention to original images than scrambled or obfuscated images. In addition, we found no evidence for an uncanny valley effect; since for both groups, looking times did not differ between real, realistic, or unrealistic images. These results provide important data for further development of our Primatar for use in social cognition studies and more generally for cognitive research with virtual stimuli in nonhuman primates. Future research on the absence of an uncanny valley effect in macaques is needed, to elucidate the roots of this mechanism in humans.

A Naturalistic Dynamic Monkey Head Avatar Elicits Species-Typical Reactions and Overcomes the Uncanny Valley

Research on social perception in monkeys may benefit from standardized, controllable, and ethologically valid renditions of conspecifics offered by monkey avatars. However, previous work has cautioned that monkeys, like humans, show an adverse reaction toward realistic synthetic stimuli, known as the "uncanny valley" effect. We developed an improved naturalistic rhesus monkey face avatar capable of producing facial expressions (fear grin, lip smack and threat), animated by motion capture data of real monkeys. For validation, we additionally created decreasingly naturalistic avatar variants. Eight rhesus macaques were tested on the various videos and avoided looking at less naturalistic avatar variants, but not at the most naturalistic or the most unnaturalistic avatar, indicating an uncanny valley effect for the less naturalistic avatar versions. The avoidance was deepened by motion and accompanied by physiological arousal. Only the most naturalistic avatar evoked facial expressions comparable to those toward the real monkey videos. Hence, our findings demonstrate that the uncanny valley reaction in monkeys can be overcome by a highly naturalistic avatar.