The human likeness dimension of the "uncanny valley hypothesis": behavioral and functional MRI findings

The eyes as the exclamation mark of the face: exploring the relationship between eye size, intensity of female facial expressions and attractiveness in a range of emotions

Background-objective

The eyes play an important role in communicating emotions and shape the determination of other facial attributes. Here, we explored the relationship between eye size, perceived intensity and attractiveness of facial expressions.

Methods

A sample of N = 63 participants (men and women, aged 18-35) rated attractiveness and emotional intensity for images displaying emotionally expressive women's faces with digitally manipulated eye size (15% smaller, unchanged, or 15% larger).

Results

The analysis of perceived intensity showed an interaction parameter between eye size and gender. Female individuals reported differences when comparing unchanged and larger eyes; male participants showed differences across all eye size comparisons (smaller-unchanged, smaller-larger, unchanged-larger). Regarding perceived attractiveness, faces with smaller eyes registered lower mean scores than both unchanged and larger. The lowest intensity level was associated with neutral faces and the highest with fearful ones. Faces displaying happiness were perceived as the most attractive.

Conclusion

Larger eyes seem to make emotions more intense and attractive. We suggest that the more intense phenomenon serves an evolutive purpose, as it might encourage caretaking behavior.

Is implicit memory associated with the hippocampus?

According to the traditional memory-systems view, the hippocampus is critical during explicit (conscious) long-term memory, whereas other brain regions support implicit (nonconscious) memory. In the last two decades, some fMRI studies have reported hippocampal activity during implicit memory tasks. The aim of the present discussion paper was to identify whether any implicit memory fMRI studies have provided convincing evidence that the hippocampus is associated with nonconscious processes without being confounded by conscious processes. Experimental protocol and analysis parameters included the stimulus type(s), task(s), measures of subjective awareness, explicit memory accuracy, the relevant fMRI contrast(s) or analysis, and confound(s). A systematic review was conducted to identify implicit memory studies that reported fMRI activity in the hippocampus. After applying exclusion criteria, 13 articles remained for analysis. We found that there were no implicit memory fMRI studies where subjective awareness was absent, explicit memory performance was at chance, and there were no confounds that could have driven the observed hippocampal activity. The confounds included explicit memory (including false memory), imbalanced attentional states between conditions (yielding activation of the default-mode network), imbalanced stimuli between conditions, and differential novelty. As such, not a single fMRI study provided convincing evidence that implicit memory was associated with the hippocampus. Neuropsychological evidence was also considered, and implicit memory deficits were caused by factors known to disrupt brain regions beyond the hippocampus, such that the behavioral effects could not be attributed to this region. The present results indicate that implicit memory is not associated with the hippocampus.

A stimulus exposure of 50 ms elicits the uncanny valley effect

The uncanny valley (UV) effect captures the observation that artificial entities with near-human appearances tend to create feelings of eeriness. Researchers have proposed many hypotheses to explain the UV effect, but the visual processing mechanisms of the UV have yet to be fully understood. In the present study, we examined if the UV effect is as accessible in brief stimulus exposures compared to long stimulus exposures (Experiment 1). Forty-one participants, aged 21-31, rated each human-robot face presented for either a brief (50 ms) or long duration (3 s) in terms of attractiveness, eeriness, and humanness (UV indices) in a 7-point Likert scale. We found that brief and long exposures to stimuli generated a similar UV effect. This suggests that the UV effect is accessible at early visual processing. We then examined the effect of exposure duration on the categorisation of visual stimuli in Experiment 2. Thirty-three participants, aged 21-31, categorised faces as either human or robot in a two-alternative forced choice task. Their response accuracy and variance were recorded. We found that brief stimulus exposures generated significantly higher response variation and errors than the long exposure condition. This indicated that participants were more uncertain in categorising faces in the brief exposure condition due to insufficient time. Further comparisons between Experiment 1 and 2 revealed that the eeriest faces were not the hardest to categorise. Overall, these findings indicate (1) that both the UV effect and categorical uncertainty can be elicited through brief stimulus exposure, but (2) that categorical uncertainty is unlikely to cause the UV effect. These findings provide insights towards the perception of robotic faces and implications for the design of robots, androids, avatars, and artificial intelligence agents.

The Neuroscience of Human and Artificial Intelligence Presence

Two decades of social neuroscience and neuroeconomics research illustrate the brain mechanisms that are engaged when people consider human beings, often in comparison to considering artificial intelligence (AI) as a nonhuman control. AI as an experimental control preserves agency and facilitates social interactions but lacks a human presence, providing insight into brain mechanisms that are engaged by human presence and the presence of AI. Here, I review this literature to determine how the brain instantiates human and AI presence across social perception and decision-making paradigms commonly used to realize a social context. People behave toward humans differently than they do toward AI. Moreover, brain regions more engaged by humans compared to AI extend beyond the social cognition brain network to all parts of the brain, and the brain sometimes is engaged more by AI than by humans. Finally, I discuss gaps in the literature, limitations in current neuroscience approaches, and how an understanding of the brain correlates of human and AI presence can inform social science in the wild.

The inversion effect on the cubic humanness-uncanniness relation in humanlike agents

The uncanny valley describes the typically nonlinear relation between the esthetic appeal of artificial entities and their human likeness. The effect has been attributed to specialized (configural) processing that increases sensitivity to deviations from human norms. We investigate this effect in computer-generated, humanlike android and human faces using dynamic facial expressions. Angry and happy expressions with varying degrees of synchrony were presented upright and inverted and rated on their eeriness, strangeness, and human likeness. A sigmoidal function of human likeness and uncanniness ("uncanny slope") was found for upright expressions and a linear relation for inverted faces. While the function is not indicative of an uncanny valley, the results support the view that configural processing moderates the effect of human likeness on uncanniness and extend its role to dynamic facial expressions.

Audio–Visual Predictive Processing in the Perception of Humans and Robots

Recent work in cognitive science suggests that our expectations affect visual perception. With the rise of artificial agents in human life in the last few decades, one important question is whether our expectations about non-human agents such as humanoid robots affect how we perceive them. In the present study, we addressed this question in an audio–visual context. Participants reported whether a voice embedded in a noise belonged to a human or a robot. Prior to this judgment, they were presented with a human or a robot image that served as a cue and allowed them to form an expectation about the category of the voice that would follow. This cue was either congruent or incongruent with the category of the voice. Our results show that participants were faster and more accurate when the auditory target was preceded by a congruent cue than an incongruent cue. This was true regardless of the human-likeness of the robot. Overall, these results suggest that our expectations affect how we perceive non-human agents and shed light on future work in robot design.

From Freud to Android: Constructing a Scale of Uncanny Feelings

The uncanny valley is a topic for engineers, animators, and psychologists, yet uncanny emotions are without a clear definition. Across three studies, we developed an 8-item measure of unnerved feelings, finding that it was discriminable from other affective experiences. In Study 1, we conducted an exploratory factor analysis that yielded two factors; an unnerved factor, which connects to emotional reactions to the uncanny, and a disoriented factor, which connects to mental state changes more distally following uncanny experiences. Focusing on the unnerved measure, Study 2 tests the scale's convergent and discriminant validity, concluding that participants who watched an uncanny video were more unnerved than those who watched a disgusting, fearful, or a neutral video. In Study 3, we determined that our scale detects unnerved feelings created during early 2020 by the coronavirus pandemic; a distinct source of uncanniness. These studies contribute to the psychological and interdisciplinary understanding of this strange, eerie phenomenon of being confronted with what looms just beyond our understanding.

Corneal reflections and skin contrast yield better memory of human and virtual faces

Virtual faces have been found to be rated less human-like and remembered worse than photographic images of humans. What it is in virtual faces that yields reduced memory has so far remained unclear. The current study investigated face memory in the context of virtual agent faces and human faces, real and manipulated, considering two factors of predicted influence, i.e., corneal reflections and skin contrast. Corneal reflections referred to the bright points in each eye that occur when the ambient light reflects from the surface of the cornea. Skin contrast referred to the degree to which skin surface is rough versus smooth. We conducted two memory experiments, one with high-quality virtual agent faces (Experiment 1) and the other with the photographs of human faces that were manipulated (Experiment 2). Experiment 1 showed better memory for virtual faces with increased corneal reflections and skin contrast (rougher rather than smoother skin). Experiment 2 replicated these findings, showing that removing the corneal reflections and smoothening the skin reduced memory recognition of manipulated faces, with a stronger effect exerted by the eyes than the skin. This study highlights specific features of the eyes and skin that can help explain memory discrepancies between real and virtual faces and in turn elucidates the factors that play a role in the cognitive processing of faces.

Familiarity, orientation, and realism increase face uncanniness by sensitizing to facial distortions

The uncanny valley predicts aversive reactions toward near-humanlike entities. Greater uncanniness is elicited by distortions in realistic than unrealistic faces, possibly due to familiarity. Experiment 1 investigated how familiarity and inversion affect uncanniness of facial distortions and the ability to detect differences between the distorted variants of the same face (distortion sensitivity). Familiar or unfamiliar celebrity faces were incrementally distorted and presented either upright or inverted. Uncanniness ratings increased across the distortion levels, and were stronger for familiar and upright faces. Distortion sensitivity increased with increasing distortion difference levels, again stronger for familiar and upright faces. Experiment 2 investigated how face realism, familiarity, and face orientation interacted for the increase of uncanniness across distortions. Realism increased the increase of uncanniness across the distortion levels, further enhanced by upright orientation and familiarity. The findings show that familiarity, upright orientation, and high face realism increase the sensitivity of uncanniness, likely by increasing distortion sensitivity. Finally, a moderated linear function of face realism and deviation level could explain the uncanniness of stimuli better than a quadratic function. A re-interpretation of the uncanny valley as sensitivity toward deviations from familiarized patterns is discussed.

Pre-exposure to Ambiguous Faces Modulates Top-Down Control of Attentional Orienting to Counterpredictive Gaze Cues

Understanding and reacting to others' nonverbal social signals, such as changes in gaze direction (i.e., gaze cue), are essential for social interactions, as it is important for processes such as joint attention and mentalizing. Although attentional orienting in response to gaze cues has a strong reflexive component, accumulating evidence shows that it can be top-down controlled by context information regarding the signals' social relevance. For example, when a gazer is believed to be an entity "with a mind" (i.e., mind perception), people exert more top-down control on attention orienting. Although increasing an agent's physical human-likeness can enhance mind perception, it could have negative consequences on top-down control of social attention when a gazer's physical appearance is categorically ambiguous (i.e., difficult to categorize as human or nonhuman), as resolving this ambiguity would require using cognitive resources that otherwise could be used to top-down control attention orienting. To examine this question, we used mouse-tracking to explore if categorically ambiguous agents are associated with increased processing costs (Experiment 1), whether categorically ambiguous stimuli negatively impact top-down control of social attention (Experiment 2), and if resolving the conflict related to the agent's categorical ambiguity (using exposure) would restore top-down control to orient attention (Experiment 3). The findings suggest that categorically ambiguous stimuli are associated with cognitive conflict, which negatively impact the ability to exert top-down control on attentional orienting in a counterpredicitive gaze-cueing paradigm; this negative impact, however, is attenuated when being pre-exposed to the stimuli prior to the gaze-cueing task. Taken together, these findings suggest that manipulating physical human-likeness is a powerful way to affect mind perception in human-robot interaction (HRI) but has a diminishing returns effect on social attention when it is categorically ambiguous due to drainage of cognitive resources and impairment of top-down control.

Macaque Gaze Responses to the Primatar: A Virtual Macaque Head for Social Cognition Research

Following the expanding use and applications of virtual reality in everyday life, realistic virtual stimuli are of increasing interest in cognitive studies. They allow for control of features such as gaze, expression, appearance, and movement, which may help to overcome limitations of using photographs or video recordings to study social responses. In using virtual stimuli however, one must be careful to avoid the uncanny valley effect, where realistic stimuli can be perceived as eerie, and induce an aversion response. At the same time, it is important to establish whether responses to virtual stimuli mirror responses to depictions of a real conspecific. In the current study, we describe the development of a new virtual monkey head with realistic facial features for experiments with nonhuman primates, the "Primatar." As a first step toward validation, we assessed how monkeys respond to facial images of a prototype of this Primatar compared to images of real monkeys (RMs), and an unrealistic model. We also compared gaze responses between original images and scrambled as well as obfuscated versions of these images. We measured looking time to images in six freely moving long-tailed macaques (Macaca fascicularis) and gaze exploration behavior in three rhesus macaques (Macaca mulatta). Both groups showed more signs of overt attention to original images than scrambled or obfuscated images. In addition, we found no evidence for an uncanny valley effect; since for both groups, looking times did not differ between real, realistic, or unrealistic images. These results provide important data for further development of our Primatar for use in social cognition studies and more generally for cognitive research with virtual stimuli in nonhuman primates. Future research on the absence of an uncanny valley effect in macaques is needed, to elucidate the roots of this mechanism in humans.

The Security-Utility Trade-off for Iris Authentication and Eye Animation for Social Virtual Avatars

The gaze behavior of virtual avatars is critical to social presence and perceived eye contact during social interactions in Virtual Reality. Virtual Reality headsets are being designed with integrated eye tracking to enable compelling virtual social interactions. This paper shows that the near infra-red cameras used in eye tracking capture eye images that contain iris patterns of the user. Because iris patterns are a gold standard biometric, the current technology places the user's biometric identity at risk. Our first contribution is an optical defocus based hardware solution to remove the iris biometric from the stream of eye tracking images. We characterize the performance of this solution with different internal parameters. Our second contribution is a psychophysical experiment with a same-different task that investigates the sensitivity of users to a virtual avatar's eye movements when this solution is applied. By deriving detection threshold values, our findings provide a range of defocus parameters where the change in eye movements would go unnoticed in a conversational setting. Our third contribution is a perceptual study to determine the impact of defocus parameters on the perceived eye contact, attentiveness, naturalness, and truthfulness of the avatar. Thus, if a user wishes to protect their iris biometric, our approach provides a solution that balances biometric protection while preventing their conversation partner from perceiving a difference in the user's virtual avatar. This work is the first to develop secure eye tracking configurations for VR/AR/XR applications and motivates future work in the area.

Emotional appraisal processing of computer-generated facial expressions: an functional near-infrared spectroscopy study

OBJECTIVE

The current study aims to investigate whether computer generated (CG) expressions of emotion evoke similar emotional appraisal processing in the lateral orbitofrontal cortex (lOFC) compared to real human expressions, as well as how speech cues would influence the processing.

METHODS

Functional near-infrared spectroscopy was used to measure the neural activations in the prefrontal cortex during emotion recognition task. Thirty normal participants were asked to view videos of dynamic facial expressions and selected the emotions that were best matches with the expressions.

RESULTS

CG expressions evoked less activation in the lOFC comparing to real human expressions. Furthermore, speech cues increased the activation in the lOFC for CG expressions but not real expressions.

CONCLUSION

Comparing to real expressions, CG expressions evoked less appraisal processing related to motivational values although this disadvantage can be compensated to some extent by presenting the expressions with speech cues.

Dissociating affective and cognitive dimensions of uncertainty by altering regulatory focus

Cognitive uncertainty is evidenced across learning, memory, and decision-making tasks. Uncertainty has also been examined in studies of positive affect and preference by manipulating stimulus presentation frequency. Despite the extensive research in both of these areas, there has been little systematic study into the relationship between affective and cognitive uncertainty. Using a categorization task, the present study examined changes in cognitive and affective uncertainty by manipulating stimulus presentation frequency and processing focus (i.e., promotion v. prevention focus). Following training, participants categorized stimuli and provided ratings of both typicality and negative affect. Results indicated that cognitive uncertainty was influenced by a categorical representation of stimuli whereas affective uncertainty was also influenced by exemplar presentation frequency during training. We additionally found that when the training was framed in terms of the avoidance of errors (i.e., a prevention focus), categorization performance was affected across the stimulus continuum whereas affective ratings remained unchanged.

Large as being on top of the world and small as hitting the roof: a common magnitude representation for the comparison of emotions and numbers

Previous work on the direct speed-intensity association (SIA) on comparative judgement tasks involved spatially distributed responses over spatially distributed stimuli with high motivational significance like facial expressions of emotions. This raises the possibility that the inferred stimulus-driven regulation of lateralized motor reactivity described by SIA, which was against the one expected on the basis of a valence-specific lateral bias, was entirely due to attentional capture from motivational significance (beyond numerical cognition). In order to establish the relevance of numerical cognition on the regulation of attentional capture we ran two complementary experiments. These involved the same direct comparison task on stimulus pairs that were fully comparable in terms of their analog representation of intensity but with different representational domain and motivational significance: symbolic magnitudes with low motivational significance in experiment 1 vs. emotions with rather high motivational significance in experiment 2. The results reveal a general SIA and point to a general mechanism regulating comparative judgements. This is based on the way spatial attention is captured toward locations that contain the stimulus which is closest in term of relative intensity to the extremal values of the series, regardless from its representational domain being it symbolic or emotional.

Young Children's Indiscriminate Helping Behavior Toward a Humanoid Robot

Young children help others in a range of situations, relatively indiscriminate of the characteristics of those they help. Recent results have suggested that young children's helping behavior extends even to humanoid robots. However, it has been unclear how characteristics of robots would influence children's helping behavior. Considering previous findings suggesting that certain robot features influence adults' perception of and their behavior toward robots, the question arises of whether young children's behavior and perception would follow the same principles. The current study investigated whether two key characteristics of a humanoid robot (animate autonomy and friendly expressiveness) would affect children's instrumental helping behavior and their perception of the robot as an animate being. Eighty-two 3-year-old children participated in one of four experimental conditions manipulating a robot's ostensible animate autonomy (high/low) and friendly expressiveness (friendly/neutral). Helping was assessed in an out-of-reach task and animacy ratings were assessed in a post-test interview. Results suggested that both children's helping behavior, as well as their perception of the robot as animate, were unaffected by the robot's characteristics. The findings indicate that young children's helping behavior extends largely indiscriminately across two important characteristics. These results increase our understanding of the development of children's altruistic behavior and animate-inanimate distinctions. Our findings also raise important ethical questions for the field of child-robot interaction.

Neural Mechanisms for Accepting and Rejecting Artificial Social Partners in the Uncanny Valley

Artificial agents are becoming prevalent across human life domains. However, the neural mechanisms underlying human responses to these new, artificial social partners remain unclear. The uncanny valley (UV) hypothesis predicts that humans prefer anthropomorphic agents but reject them if they become too humanlike-the so-called UV reaction. Using fMRI, we investigated neural activity when subjects evaluated artificial agents and made decisions about them. Across two experimental tasks, the ventromedial prefrontal cortex (VMPFC) encoded an explicit representation of subjects' UV reactions. Specifically, VMPFC signaled the subjective likability of artificial agents as a nonlinear function of humanlikeness, with selective low likability for highly humanlike agents. In exploratory across-subject analyses, these effects explained individual differences in psychophysical evaluations and preference choices. Functionally connected areas encoded critical inputs for these signals: the temporoparietal junction encoded a linear humanlikeness continuum, whereas nonlinear representations of humanlikeness in dorsomedial prefrontal cortex (DMPFC) and fusiform gyrus emphasized a human-nonhuman distinction. Following principles of multisensory integration, multiplicative combination of these signals reconstructed VMPFC's valuation function. During decision making, separate signals in VMPFC and DMPFC encoded subjects' decision variable for choices involving humans or artificial agents, respectively. A distinct amygdala signal predicted rejection of artificial agents. Our data suggest that human reactions toward artificial agents are governed by a neural mechanism that generates a selective, nonlinear valuation in response to a specific feature combination (humanlikeness in nonhuman agents). Thus, a basic principle known from sensory coding-neural feature selectivity from linear-nonlinear transformation-may also underlie human responses to artificial social partners.SIGNIFICANCE STATEMENT Would you trust a robot to make decisions for you? Autonomous artificial agents are increasingly entering our lives, but how the human brain responds to these new artificial social partners remains unclear. The uncanny valley (UV) hypothesis-an influential psychological framework-captures the observation that human responses to artificial agents are nonlinear: we like increasingly anthropomorphic artificial agents, but feel uncomfortable if they become too humanlike. Here we investigated neural activity when humans evaluated artificial agents and made personal decisions about them. Our findings suggest a novel neurobiological conceptualization of human responses toward artificial agents: the UV reaction-a selective dislike of highly humanlike agents-is based on nonlinear value-coding in ventromedial prefrontal cortex, a key component of the brain's reward system.

Seeing minds in others: Mind perception modulates low-level social-cognitive performance and relates to ventromedial prefrontal structures

In social interactions, we rely on nonverbal cues like gaze direction to understand the behavior of others. How we react to these cues is affected by whether they are believed to originate from an entity with a mind, capable of having internal states (i.e., mind perception). While prior work has established a set of neural regions linked to social-cognitive processes like mind perception, the degree to which activation within this network relates to performance in subsequent social-cognitive tasks remains unclear. In the current study, participants performed a mind perception task (i.e., judging the likelihood that faces, varying in physical human-likeness, have internal states) while event-related fMRI was collected. Afterwards, participants performed a social attention task outside the scanner, during which they were cued by the gaze of the same faces that they previously judged within the mind perception task. Parametric analyses of the fMRI data revealed that activity within ventromedial prefrontal cortex (vmPFC) was related to both mind ratings inside the scanner and gaze-cueing performance outside the scanner. In addition, other social brain regions were related to gaze-cueing performance, including frontal areas like the left insula, dorsolateral prefrontal cortex, and inferior frontal gyrus, as well as temporal areas like the left temporo-parietal junction and bilateral temporal gyri. The findings suggest that functions subserved by the vmPFC are relevant to both mind perception and social attention, implicating a role of vmPFC in the top-down modulation of low-level social-cognitive processes.

Perceiving Animacy in Own-and Other-Species Faces

Though artificial faces of various kinds are rapidly becoming more and more life-like due to advances in graphics technology (Suwajanakorn et al., 2015; Booth et al., 2017), observers can typically distinguish real faces from artificial faces. In general, face recognition is tuned to experience such that expert-level processing is most evident for faces that we encounter frequently in our visual world, but the extent to which face animacy perception is also tuned to in-group vs. out-group categories remains an open question. In the current study, we chose to examine how the perception of animacy in human faces and dog faces was affected by face inversion and the duration of face images presented to adult observers. We hypothesized that the impact of these manipulations may differ as a function of species category, indicating that face animacy perception is tuned for in-group faces. Briefly, we found evidence of such a differential impact, suggesting either that distinct mechanisms are used to evaluate the "life" in a face for in-group and out-group faces, or that the efficiency of a common mechanism varies substantially as a function of visual expertise.

Evaluating the replicability of the uncanny valley effect

The uncanny valley (UV) effect refers to an eerie feeling of unfamiliarity people get while observing or interacting with robots that resemble humans almost but not quite perfectly. The effect is not well understood, and it is also unclear how well results from previous research on the UV can be replicated. In six studies, both in the laboratory and online (N = 1343), we attempted to replicate the UV effect with various stimuli used in previous research. In Studies 1 and 2 we failed to replicate the UV effect with CGI stimuli created using the so-called morphing technique (a robot image morphed into a human image, resulting in a supposedly creepy robot-human image). In Studies 3a and 3b we found a prominent UV effect using pre-evaluated, non-morphed and photorealistic robot pictures. Finally, in exploratory Studies 4a and 4b we found the UV effect using morphed and photorealistic human and robot pictures. Our results suggest that the UV effect is more robust when elicited by pre-validated or prima facie uncanny robot pictures than by non-photorealistic images generated using the morphing technique. We argue that photorealistic pictures are more suitable than less realistic CGI pictures as stimuli for research attempting to elicit the UV effect – however, our results do not invalidate any previous research on the UV effect using morphing techniques, but point to their domain of applicability and context sensitivity.

Those Virtual People all Look the Same to me: Computer-Rendered Faces Elicit a Higher False Alarm Rate Than Real Human Faces in a Recognition Memory Task

Virtual as compared with real human characters can elicit a sense of uneasiness in human observers, characterized by lack of familiarity and even feelings of eeriness (the “uncanny valley” hypothesis). Here we test the possibility that this alleged lack of familiarity is literal in the sense that people have lesser perceptual expertise in processing virtual as compared with real human faces. Sixty-four participants took part in a recognition memory study in which they first learned a set of faces and were then asked to recognize them in a testing session. We used real and virtual (computer-rendered) versions of the same faces, presented in either upright or inverted orientation. Real and virtual faces were matched for low-level visual features such as global luminosity and spatial frequency contents. Our results demonstrated a higher response bias toward responding “seen before” for virtual as compared with real faces, which was further explained by a higher false alarm rate for the former. This finding resembles a similar effect for recognizing human faces from other than one's own ethnic groups (the “other race effect”). Virtual faces received clearly higher subjective eeriness ratings than real faces. Our results did not provide evidence of poorer overall recognition memory or lesser inversion effect for virtual faces, however. The higher false alarm rate finding supports the notion that lesser perceptual expertise may contribute to the lack of subjective familiarity with virtual faces. We discuss alternative interpretations and provide suggestions for future research.

Uncanny valley as a window into predictive processing in the social brain

Uncanny valley refers to humans' negative reaction to almost-but-not-quite-human agents. Theoretical work proposes prediction violation as an explanation for uncanny valley but no empirical work has directly tested it. Here, we provide evidence that supports this theory using event-related brain potential recordings from the human scalp. Human subjects were presented images and videos of three agents as EEG was recorded: a real human, a mechanical robot, and a realistic robot in between. The real human and the mechanical robot had congruent appearance and motion whereas the realistic robot had incongruent appearance and motion. We hypothesize that the appearance of the agent would provide a context to predict her movement, and accordingly the perception of the realistic robot would elicit an N400 effect indicating the violation of predictions, whereas the human and the mechanical robot would not. Our data confirmed this hypothesis suggesting that uncanny valley could be explained by violation of one's predictions about human norms when encountered with realistic but artificial human forms. Importantly, our results implicate that the mechanisms underlying perception of other individuals in our environment are predictive in nature.

Behind the Robot's Smiles and Frowns: In Social Context, People Do Not Mirror Android's Expressions But React to Their Informational Value

Facial actions are key elements of non-verbal behavior. Perceivers' reactions to others' facial expressions often represent a match or mirroring (e.g., they smile to a smile). However, the information conveyed by an expression depends on context. Thus, when shown by an opponent, a smile conveys bad news and evokes frowning. The availability of anthropomorphic agents capable of facial actions raises the question of how people respond to such agents in social context. We explored this issue in a study where participants played a strategic game with or against a facially expressive android. Electromyography (EMG) recorded participants' reactions over zygomaticus muscle (smiling) and corrugator muscle (frowning). We found that participants' facial responses to android's expressions reflect their informational value, rather than a direct match. Overall, participants smiled more, and frowned less, when winning than losing. Critically, participants' responses to the game outcome were similar regardless of whether it was conveyed via the android's smile or frown. Furthermore, the outcome had greater impact on people's facial reactions when it was conveyed through android's face than a computer screen. These findings demonstrate that facial actions of artificial agents impact human facial responding. They also suggest a sophistication in human-robot communication that highlights the signaling value of facial expressions.

Subthalamic nucleus detects unnatural android movement

An android, i.e., a realistic humanoid robot with human-like capabilities, may induce an uncanny feeling in human observers. The uncanny feeling about an android has two main causes: its appearance and movement. The uncanny feeling about an android increases when its appearance is almost human-like but its movement is not fully natural or comparable to human movement. Even if an android has human-like flexible joints, its slightly jerky movements cause a human observer to detect subtle unnaturalness in them. However, the neural mechanism underlying the detection of unnatural movements remains unclear. We conducted an fMRI experiment to compare the observation of an android and the observation of a human on which the android is modelled, and we found differences in the activation pattern of the brain regions that are responsible for the production of smooth and natural movement. More specifically, we found that the visual observation of the android, compared with that of the human model, caused greater activation in the subthalamic nucleus (STN). When the android's slightly jerky movements are visually observed, the STN detects their subtle unnaturalness. This finding suggests that the detection of unnatural movements is attributed to an error signal resulting from a mismatch between a visual input and an internal model for smooth movement.

Avoidance of Novelty Contributes to the Uncanny Valley

A hypothesis suggests that objects with a high degree of visual similarity to real humans trigger negative impressions (i.e., the uncanny valley). Previous studies have suggested that difficulty in object categorization elicits negative emotional reactions to enable the avoidance of potential threats. The present study further investigated this categorization-difficulty hypothesis. In an experiment, observers categorized morphed images of photographs and human doll faces as "photograph" or "doll" and evaluated the perceived eeriness of the images. Additionally, we asked the observers to answer questionnaires on behavioral inhibition systems (BIS). The results indicated that individual differences in the BIS score were associated with enhanced eeriness in the objects with a specific human likeness. These findings suggest that the tendency to avoid a potentially threatening novel experience contributes to promoting the perceived eeriness of objects with some degree of visual similarity to real humans.

Robots As Intentional Agents: Using Neuroscientific Methods to Make Robots Appear More Social

Robots are increasingly envisaged as our future cohabitants. However, while considerable progress has been made in recent years in terms of their technological realization, the ability of robots to interact with humans in an intuitive and social way is still quite limited. An important challenge for social robotics is to determine how to design robots that can perceive the user's needs, feelings, and intentions, and adapt to users over a broad range of cognitive abilities. It is conceivable that if robots were able to adequately demonstrate these skills, humans would eventually accept them as social companions. We argue that the best way to achieve this is using a systematic experimental approach based on behavioral and physiological neuroscience methods such as motion/eye-tracking, electroencephalography, or functional near-infrared spectroscopy embedded in interactive human-robot paradigms. This approach requires understanding how humans interact with each other, how they perform tasks together and how they develop feelings of social connection over time, and using these insights to formulate design principles that make social robots attuned to the workings of the human brain. In this review, we put forward the argument that the likelihood of artificial agents being perceived as social companions can be increased by designing them in a way that they are perceived as intentional agents that activate areas in the human brain involved in social-cognitive processing. We first review literature related to social-cognitive processes and mechanisms involved in human-human interactions, and highlight the importance of perceiving others as intentional agents to activate these social brain areas. We then discuss how attribution of intentionality can positively affect human-robot interaction by (a) fostering feelings of social connection, empathy and prosociality, and by (b) enhancing performance on joint human-robot tasks. Lastly, we describe circumstances under which attribution of intentionality to robot agents might be disadvantageous, and discuss challenges associated with designing social robots that are inspired by neuroscientific principles.

Cognitive Conflict as Possible Origin of the Uncanny Valley

In social robotics, the term Uncanny Valley describes the phenomenon that linear increases in human-likeness of an agent do not entail an equally linear increase in favorable reactions towards that agent. Instead, a pronounced dip or ‘valley’ at around 70% human-likeness emerges. One currently popular view to explain this drop in favorable reactions is delivered by the Categorical Perception Hypothesis. It is suggested that categorization of agents with mixed human and non-human features is associated with additional cognitive costs and that these costs are the cause of the Uncanny Valley. However, the nature of the cognitive costs is still matter of debate. The current study explores whether the cognitive costs associated with stimulus categorization around the Uncanny Valley could be due to cognitive conflict as evoked by simultaneous activation of two categories. Using the mouse tracking technique, we show that cognitive conflict indeed peaks around the Uncanny Valley region of human-likeness. Our findings lay the foundation for investigating the effects of cognitive conflict on positive affect towards agents of around 70% human-likeness, possibly leading to the unraveling of the origins of the Uncanny Valley.

Human Perception of Animacy in Light of the Uncanny Valley Phenomenon

The uncanny valley hypothesis by Japanese roboticist Masahiro Mori posits a nonlinear relation between human replicas’ human likeness and the emotional responses they elicit. In three studies, we corroborated the uncanny valley hypothesis, using the uncanny phenomenon as a vehicle to shed a new light on human animacy perception. In Study 1, 62 participants rated emotional responses and human likeness of 89 artificial and human faces. In Study 2, another 62 participants conducted a visual looming task with the same 89 faces allowing for the measurement of perceived threat. Results support the uncanny valley hypothesis, suggesting that the uncanny feeling may serve a function to wary humans of the potential danger of entities crossing the animate–inanimate boundary. In Study 3, 36 participants sorted faces as either real or unreal as quickly as possible in a reaction time sorting task allowing for the measurement of categorical uncertainty associated with animacy perception. Faces associated with longer sorting reaction times were also those associated with the highest ratings of negative emotions, suggesting that categorical uncertainty in animacy detection is related to the uncanny feeling. Results are discussed in light of human animacy perception and new directions for future research are suggested.

Differential effects of face-realism and emotion on event-related brain potentials and their implications for the uncanny valley theory

Cartoon characters are omnipresent in popular media. While few studies have scientifically investigated their processing, in computer graphics, efforts are made to increase realism. Yet, close approximations of reality have been suggested to evoke sometimes a feeling of eeriness, the “uncanny valley” effect. Here, we used high-density electroencephalography to investigate brain responses to professionally stylized happy, angry, and neutral character faces. We employed six face-stylization levels varying from abstract to realistic and investigated the N170, early posterior negativity (EPN), and late positive potential (LPP) event-related components. The face-specific N170 showed a u-shaped modulation, with stronger reactions towards both most abstract and most realistic compared to medium-stylized faces. For abstract faces, N170 was generated more occipitally than for real faces, implying stronger reliance on structural processing. Although emotional faces elicited highest amplitudes on both N170 and EPN, on the N170 realism and expression interacted. Finally, LPP increased linearly with face realism, reflecting activity increase in visual and parietal cortex for more realistic faces. Results reveal differential effects of face stylization on distinct face processing stages and suggest a perceptual basis to the uncanny valley hypothesis. They are discussed in relation to face perception, media design, and computer graphics.

Circling Around the Uncanny Valley: Design Principles for Research Into the Relation Between Human Likeness and Eeriness

The uncanny valley effect (UVE) is a negative emotional response experienced when encountering entities that appear almost human. Research on the UVE typically investigates individual, or collections of, near human entities but may be prone to methodological circularity unless the properties that give rise to the emotional response are appropriately defined and quantified. In addition, many studies do not sufficiently control the variation in human likeness portrayed in stimulus images, meaning that the nature of stimuli that elicit the UVE is also not well defined or quantified. This article describes design criteria for UVE research to overcome the above problems by measuring three variables (human likeness, eeriness, and emotional response) and by using stimuli spanning the artificial to human continuum. These criteria allow results to be plotted and compared with the hypothesized uncanny valley curve and any effect observed can be quantified. The above criteria were applied to the methods used in a subset of existing UVE studies. Although many studies made use of some of the necessary measurements and controls, few used them all. The UVE is discussed in relation to this result and research methodology more broadly.

Of Mannequins and Men

Recent research has demonstrated that ascribing minds to humanlike stimuli is a product of both their perceptual similarity to human faces and whether they engaged configural face processing. We present the findings of two experiments in which we both manipulate the amount of humanlike features in faces (in a doll-to-human morph continuum) and manipulate perceivers’ ability to employ configural face processing (via face inversion) while measuring explicit ratings of mind ascription (Study 1) and the spontaneous activation of humanlike concepts (Study 2). In both studies, we find novel evidence that ascribing minds to entities is an interactive product of both having strong perceptual similarity to human faces and being processed using configural processing mechanisms typical of normal face perception. In short, ascribing mind to others is bounded jointly by the featural cues of the target and by processes employed by the perceiver.

The Neurological Traces of Look-Alike Avatars

We designed an observational study where participants (n = 17) were exposed to pictures and look-alike avatars pictures of themselves, a familiar friend or an unfamiliar person. By measuring participants’ brain activity with electroencephalography (EEG), we found face-recognition event related potentials (ERPs) in the visual cortex, around 200–250 ms, to be prominent for the different familiarity levels. A less positive component was found for self-recognized pictures (P200) than pictures of others, showing similar effects in both real faces and look-alike avatars. A rapid adaptation in the same component was found when comparing the neural processing of avatar faces vs. real faces, as if avatars in general were assimilated as real face representations over time. ERP results also showed that in the case of the self-avatar, the P200 component correlated with more complex conscious encodings of self-representation, i.e., the difference in voltage in the P200 between the self-avatar and the self-picture was reduced in participants that felt the avatar looked like them. This study is put into context within the literature of self-recognition and face recognition in the visual cortex. Additionally, the implications of these results on look-alike avatars are discussed both for future virtual reality (VR) and neuroscience studies.

The Uncanny Valley: Existence and Explanations

More than 40 years ago, Japanese roboticist Masahiro Mori (1970/2005) proposed the “uncanny valley” hypothesis, which predicted a nonlinear relation between robots’ perceived human likeness and their likability. Although some studies have corroborated this hypothesis and proposed explanations for its existence, the evidence on both fronts has been mixed and open to debate. We first review the literature to ascertain whether the uncanny valley exists. We then try to explain the uncanny phenomenon by reviewing hypotheses derived from diverse theoretical and methodological perspectives within psychology and allied fields, including evolutionary, social, cognitive, and psychodynamic approaches. Next, we provide an evaluation and critique of these studies by focusing on their methodological limitations, leading us to question the accepted definition of the uncanny valley. We examine the definitions of human likeness and likability, and propose a statistical test to preliminarily quantify their nonlinear relation. We argue that the uncanny valley hypothesis is ultimately an engineering problem that bears on the possibility of building androids that may some day become indistinguishable from humans. In closing, we propose a dehumanization hypothesis to explain the uncanny phenomenon.

What is the Value of Embedding Artificial Emotional Prosody in Human-Computer Interactions? Implications for Theory and Design in Psychological Science

In computerized technology, artificial speech is becoming increasingly important, and is already used in ATMs, online gaming and healthcare contexts. However, today’s artificial speech typically sounds monotonous, a main reason for this being the lack of meaningful prosody. One particularly important function of prosody is to convey different emotions. This is because successful encoding and decoding of emotions is vital for effective social cognition, which is increasingly recognized in human–computer interaction contexts. Current attempts to artificially synthesize emotional prosody are much improved relative to early attempts, but there remains much work to be done due to methodological problems, lack of agreed acoustic correlates, and lack of theoretical grounding. If the addition of synthetic emotional prosody is not of sufficient quality, it may risk alienating users instead of enhancing their experience. So the value of embedding emotion cues in artificial speech may ultimately depend on the quality of the synthetic emotional prosody. However, early evidence on reactions to synthesized non-verbal cues in the facial modality bodes well. Attempts to implement the recognition of emotional prosody into artificial applications and interfaces have perhaps been met with greater success, but the ultimate test of synthetic emotional prosody will be to critically compare how people react to synthetic emotional prosody vs. natural emotional prosody, at the behavioral, socio-cognitive and neural levels.

A Bayesian Model of the Uncanny Valley Effect for Explaining the Effects of Therapeutic Robots in Autism Spectrum Disorder

One of the core features of autism spectrum disorder (ASD) is impaired reciprocal social interaction, especially in processing emotional information. Social robots are used to encourage children with ASD to take the initiative and to interact with the robotic tools to stimulate emotional responses. However, the existing evidence is limited by poor trial designs. The purpose of this study was to provide computational evidence in support of robot-assisted therapy for children with ASD. We thus propose an emotional model of ASD that adapts a Bayesian model of the uncanny valley effect, which holds that a human-looking robot can provoke repulsion and sensations of eeriness. Based on the unique emotional responses of children with ASD to the robots, we postulate that ASD induces a unique emotional response curve, more like a cliff than a valley. Thus, we performed numerical simulations of robot-assisted therapy to evaluate its effects. The results showed that, although a stimulus fell into the uncanny valley in the typical condition, it was effective at avoiding the uncanny cliff in the ASD condition. Consequently, individuals with ASD may find it more comfortable, and may modify their emotional response, if the robots look like deformed humans, even if they appear "creepy" to typical individuals. Therefore, we suggest that our model explains the effects of robot-assisted therapy in children with ASD and that human-looking robots may have potential advantages for improving social interactions in ASD.

Real or Artificial? Intergroup Biases in Mind Perception in a Cross-Cultural Perspective

Recent research suggests that attributions of aliveness and mental capacities to faces are influenced by social group membership. In this article, we investigated group related biases in mind perception in participants from a Western and Eastern culture, employing faces of varying ethnic groups. In Experiment 1, Caucasian faces that ranged on a continuum from real to artificial were evaluated by participants in the UK (in-group) and in India (out-group) on animacy, abilities to plan and to feel pain, and having a mind. Human features were found to be assigned to a greater extent to faces when these belonged to in-group members, whereas out-group faces had to appear more realistic in order to be perceived as human. When participants in India evaluated South Asian (in-group) and Caucasian (out-group) faces in Experiment 2, the results closely mirrored those of the first experiment. For both studies, ratings of out-group faces were significantly predicted by participants’ levels of ethnocultural empathy. The findings highlight the role of intergroup processes (i.e., in-group favoritism, out-group dehumanization) in the perception of human and mental qualities and point to ethnocultural empathy as an important factor in responses to out-groups.

Arousal, valence, and the uncanny valley: psychophysiological and self-report findings

The main prediction of the Uncanny Valley Hypothesis (UVH) is that observation of humanlike characters that are difficult to distinguish from the human counterpart will evoke a state of negative affect. Well-established electrophysiological [late positive potential (LPP) and facial electromyography (EMG)] and self-report [Self-Assessment Manikin (SAM)] indices of valence and arousal, i.e., the primary orthogonal dimensions of affective experience, were used to test this prediction by examining affective experience in response to categorically ambiguous compared with unambiguous avatar and human faces (N = 30). LPP and EMG provided direct psychophysiological indices of affective state during passive observation and the SAM provided self-reported indices of affective state during explicit cognitive evaluation of static facial stimuli. The faces were drawn from well-controlled morph continua representing the UVH’ dimension of human likeness (DHL). The results provide no support for the notion that category ambiguity along the DHL is specifically associated with enhanced experience of negative affect. On the contrary, the LPP and SAM-based measures of arousal and valence indicated a general increase in negative affective state (i.e., enhanced arousal and negative valence) with greater morph distance from the human end of the DHL. A second sample (N = 30) produced the same finding, using an ad hoc self-rating scale of feelings of familiarity, i.e., an oft-used measure of affective experience along the UVH’ familiarity dimension. In conclusion, this multi-method approach using well-validated psychophysiological and self-rating indices of arousal and valence rejects – for passive observation and for explicit affective evaluation of static faces – the main prediction of the UVH.