How do we think machines think? An fMRI study of alleged competition with an artificial intelligence

Interpretable prediction of brain activity during conversations from multimodal behavioral signals

We present an analytical framework aimed at predicting the local brain activity in uncontrolled experimental conditions based on multimodal recordings of participants' behavior, and its application to a corpus of participants having conversations with another human or a conversational humanoid robot. The framework consists in extracting high-level features from the raw behavioral recordings and applying a dynamic prediction of binarized fMRI-recorded local brain activity using these behavioral features. The objective is to identify behavioral features required for this prediction, and their relative weights, depending on the brain area under investigation and the experimental condition. In order to validate our framework, we use a corpus of uncontrolled conversations of participants with a human or a robotic agent, focusing on brain regions involved in speech processing, and more generally in social interactions. The framework not only predicts local brain activity significantly better than random, it also quantifies the weights of behavioral features required for this prediction, depending on the brain area under investigation and on the nature of the conversational partner. In the left Superior Temporal Sulcus, perceived speech is the most important behavioral feature for predicting brain activity, regardless of the agent, while several features, which differ between the human and robot interlocutors, contribute to the prediction in regions involved in social cognition, such as the TemporoParietal Junction. This framework therefore allows us to study how multiple behavioral signals from different modalities are integrated in individual brain regions during complex social interactions.

The Neuroscience of Human and Artificial Intelligence Presence

Two decades of social neuroscience and neuroeconomics research illustrate the brain mechanisms that are engaged when people consider human beings, often in comparison to considering artificial intelligence (AI) as a nonhuman control. AI as an experimental control preserves agency and facilitates social interactions but lacks a human presence, providing insight into brain mechanisms that are engaged by human presence and the presence of AI. Here, I review this literature to determine how the brain instantiates human and AI presence across social perception and decision-making paradigms commonly used to realize a social context. People behave toward humans differently than they do toward AI. Moreover, brain regions more engaged by humans compared to AI extend beyond the social cognition brain network to all parts of the brain, and the brain sometimes is engaged more by AI than by humans. Finally, I discuss gaps in the literature, limitations in current neuroscience approaches, and how an understanding of the brain correlates of human and AI presence can inform social science in the wild.

Dyadic cooperation with human and artificial agents: Event-related potentials trace dynamic role taking during an interactive game

Humans are highly co-operative and thus cognitively, affectively, and motivationally tuned to pursue shared goals. Yet, cooperative tasks typically require people to constantly take and switch individual roles. Task relevance is dictated by these roles and thereby dynamically changing. Here, we designed a dyadic game to test whether the family of P3 components can trace this dynamic allocation of task relevance. We demonstrate that late positive event-related potential (ERP) modulations not only reflect predictable asymmetries between receiving and sending information but also differentiate whether the receiver's role is related to correct decision making or action monitoring. Furthermore, similar results were observed when playing the game with a computer, suggesting that experimental games may motivate humans to similarly cooperate with an artificial agent. Overall, late positive ERP waves provide a real-time measure of how role taking dynamically shapes the meaning and relevance of stimuli within collaborative contexts. Our results, therefore, shed light on how the processes of mutual coordination unfold during dyadic cooperation.

Theta synchronization as a neural marker of flexible (re-)use of socio-cognitive mechanisms for a new category of (artificial) interaction partners

Previous work shows that in some instances artificial agents, such as robots, can elicit higher-order socio-cognitive mechanisms, similar to those elicited by humans. This suggests that these socio-cognitive mechanisms, such as mentalizing processes, originally developed for interaction with other humans, might be flexibly (re-)used, or "hijacked", for approaching this new category of interaction partners (Wykowska, 2020). In this study, we set out to identify neural markers of such flexible reuse of socio-cognitive mechanisms. We focused on fronto-parietal theta synchronization, as it has been proposed to be a substrate of cognitive flexibility in general (Fries, 2005). We analyzed EEG data from two experiments (Bossi et al., 2020; Roselli et al., submitted), in which participants completed a test measuring their individual likelihood to adopt the intentional stance towards robots, the intentional stance (IST) test. Our results show that participants with higher scores on the IST, indicating that they had higher likelihood of adopting the intentional stance towards a robot, had a significantly higher theta synchronization value, relative to participants with lower scores on the IST. These results suggest that long-range synchronization in the theta band might be a marker socio-cognitive process that can be flexibly applied towards non-human agents, such as robots.

A social robot connected with chatGPT to improve cognitive functioning in ASD subjects

Neurodevelopmental Disorders (NDDs) represent a significant healthcare and economic burden for families and society. Technology, including AI and digital technologies, offers potential solutions for the assessment, monitoring, and treatment of NDDs. However, further research is needed to determine the effectiveness, feasibility, and acceptability of these technologies in NDDs, and to address the challenges associated with their implementation. In this work, we present the application of social robotics using a Pepper robot connected to the OpenAI system (Chat-GPT) for real-time dialogue initiation with the robot. After describing the general architecture of the system, we present two possible simulated interaction scenarios of a subject with Autism Spectrum Disorder in two different situations. Limitations and future implementations are also provided to provide an overview of the potential developments of interconnected systems that could greatly contribute to technological advancements for Neurodevelopmental Disorders (NDD).

Implicit Perception of Differences between NLP-Produced and Human-Produced Language in the Mentalizing Network

Natural language processing (NLP) is central to the communication with machines and among ourselves, and NLP research field has long sought to produce human-quality language. Identification of informative criteria for measuring NLP-produced language quality will support development of ever-better NLP tools. The authors hypothesize that mentalizing network neural activity may be used to distinguish NLP-produced language from human-produced language, even for cases where human judges cannot subjectively distinguish the language source. Using the social chatbots Google Meena in English and Microsoft XiaoIce in Chinese to generate NLP-produced language, behavioral tests which reveal that variance of personality perceived from chatbot chats is larger than for human chats are conducted, suggesting that chatbot language usage patterns are not stable. Using an identity rating task with functional magnetic resonance imaging, neuroimaging analyses which reveal distinct patterns of brain activity in the mentalizing network including the DMPFC and rTPJ in response to chatbot versus human chats that cannot be distinguished subjectively are conducted. This study illustrates a promising empirical basis for measuring the quality of NLP-produced language: adding a judge's implicit perception as an additional criterion.

A Massage Area Positioning Algorithm for Intelligent Massage System

A growing number of studies have been conducted over the past few years on the positioning of daily massage robots. However, most methods used for research have low interactivity, and a systematic method should be designed for accurate and intelligent positioning, thus compromising usability and user experience. In this study, a massage positioning algorithm with online learning capabilities is presented. The algorithm has the following main innovations: (1) autonomous massage localization can be achieved by gaining insights into natural human-machine interaction behavior and (2) online learning of user massage habits can be achieved by integrating recursive Bayesian ideas. As revealed by the experimental results, combining natural human-computer interaction and online learning with massage positioning is capable of helping people get rid of positioning aids, reducing their psychological and cognitive load, and achieving a more desirable positioning effect. Furthermore, the results of the analysis of user evaluations further verify the effectiveness of the algorithm.

Going Beyond the "Synthetic Method": New Paradigms Cross-Fertilizing Robotics and Cognitive Neuroscience

In so-called ethorobotics and robot-supported social cognitive neurosciences, robots are used as scientific tools to study animal behavior and cognition. Building on previous epistemological analyses of biorobotics, in this article it is argued that these two research fields, widely differing from one another in the kinds of robots involved and in the research questions addressed, share a common methodology, which significantly differs from the "synthetic method" that, until recently, dominated biorobotics. The methodological novelty of this strategy, the research opportunities that it opens, and the theoretical and technological challenges that it gives rise to, will be discussed with reference to the peculiarities of the two research fields. Some broad methodological issues related to the generalization of results concerning robot-animal interaction to theoretical conclusions on animal-animal interaction will be identified and discussed.

Mental State Attribution to Robots: A Systematic Review of Conceptions, Methods, and Findings

The topic of mental state attribution to robots has been approached by researchers from a variety of disciplines, including psychology, neuroscience, computer science, and philosophy. As a consequence, the empirical studies that have been conducted so far exhibit considerable diversity in terms of how the phenomenon is described and how it is approached from a theoretical and methodological standpoint. This literature review addresses the need for a shared scientific understanding of mental state attribution to robots by systematically and comprehensively collating conceptions, methods, and findings from 155 empirical studies across multiple disciplines. The findings of the review include that: (1) the terminology used to describe mental state attribution to robots is diverse but largely homogenous in usage; (2) the tendency to attribute mental states to robots is determined by factors such as the age and motivation of the human as well as the behavior, appearance, and identity of the robot; (3) there is a computer < robot < human pattern in the tendency to attribute mental states that appears to be moderated by the presence of socially interactive behavior; (4) there are conflicting findings in the empirical literature that stem from different sources of evidence, including self-report and non-verbal behavioral or neurological data. The review contributes toward more cumulative research on the topic and opens up for a transdisciplinary discussion about the nature of the phenomenon and what types of research methods are appropriate for investigation.

Mutual gaze with a robot affects human neural activity and delays decision-making processes

In most everyday life situations, the brain needs to engage not only in making decisions but also in anticipating and predicting the behavior of others. In such contexts, gaze can be highly informative about others’ intentions, goals, and upcoming decisions. Here, we investigated whether a humanoid robot’s gaze (mutual or averted) influences the way people strategically reason in a social decision-making context. Specifically, participants played a strategic game with the robot iCub while we measured their behavior and neural activity by means of electroencephalography (EEG). Participants were slower to respond when iCub established mutual gaze before their decision, relative to averted gaze. This was associated with a higher decision threshold in the drift diffusion model and accompanied by more synchronized EEG alpha activity. In addition, we found that participants reasoned about the robot’s actions in both conditions. However, those who mostly experienced the averted gaze were more likely to adopt a self-oriented strategy, and their neural activity showed higher sensitivity to outcomes. Together, these findings suggest that robot gaze acts as a strong social signal for humans, modulating response times, decision threshold, neural synchronization, as well as choice strategies and sensitivity to outcomes. This has strong implications for all contexts involving human-robot interaction, from robotics to clinical applications.

I Am Looking for Your Mind: Pupil Dilation Predicts Individual Differences in Sensitivity to Hints of Human-Likeness in Robot Behavior

The presence of artificial agents in our everyday lives is continuously increasing. Hence, the question of how human social cognition mechanisms are activated in interactions with artificial agents, such as humanoid robots, is frequently being asked. One interesting question is whether humans perceive humanoid robots as mere artifacts (interpreting their behavior with reference to their function, thereby adopting the design stance) or as intentional agents (interpreting their behavior with reference to mental states, thereby adopting the intentional stance). Due to their humanlike appearance, humanoid robots might be capable of evoking the intentional stance. On the other hand, the knowledge that humanoid robots are only artifacts should call for adopting the design stance. Thus, observing a humanoid robot might evoke a cognitive conflict between the natural tendency of adopting the intentional stance and the knowledge about the actual nature of robots, which should elicit the design stance. In the present study, we investigated the cognitive conflict hypothesis by measuring participants’ pupil dilation during the completion of the InStance Test. Prior to each pupillary recording, participants were instructed to observe the humanoid robot iCub behaving in two different ways (either machine-like or humanlike behavior). Results showed that pupil dilation and response time patterns were predictive of individual biases in the adoption of the intentional or design stance in the IST. These results may suggest individual differences in mental effort and cognitive flexibility in reading and interpreting the behavior of an artificial agent.

Processing and utilization of auditory action effects in individual and social tasks

The influence of action-effect integration on motor control and sensory processing is often investigated in arrangements featuring human-machine interactions. Such experiments focus on predictable sensory events produced through participants' interactions with simple response devices. Action-effect integration may, however, also occur when we interact with human partners. The current study examined the similarities and differences in perceptual and motor control processes related to generating sounds with or without the involvement of a human partner. We manipulated the complexity of the causal chain of events between the initial motor and the final sensory event. In the self-induced condition participants generated sounds directly by pressing a button, while in the interactive condition sounds resulted from a paired reaction-time task, that is, the final sound was generated indirectly, by relying on the contribution of the partner. Auditory event-related potentials (ERPs) and force application patterns were similar in the two conditions, suggesting that social action effects produced with the involvement of a second human agent in the causal sequence are processed, and utilized as action feedback in the same way as direct consequences of one's actions. The only reflection of a processing difference between the two conditions was a slow, posterior ERP waveform that started before the presentation of the auditory stimulus, which may reflect differences in stimulus expectancy or task difficulty.

Mind the Eyes: Artificial Agents' Eye Movements Modulate Attentional Engagement and Anthropomorphic Attribution

Artificial agents are on their way to interact with us daily. Thus, the design of embodied artificial agents that can easily cooperate with humans is crucial for their deployment in social scenarios. Endowing artificial agents with human-like behavior may boost individuals' engagement during the interaction. We tested this hypothesis in two screen-based experiments. In the first one, we compared attentional engagement displayed by participants while they observed the same set of behaviors displayed by an avatar of a humanoid robot and a human. In the second experiment, we assessed the individuals' tendency to attribute anthropomorphic traits towards the same agents displaying the same behaviors. The results of both experiments suggest that individuals need less effort to process and interpret an artificial agent's behavior when it closely resembles one of a human being. Our results support the idea that including subtle hints of human-likeness in artificial agents' behaviors would ease the communication between them and the human counterpart during interactive scenarios.

Confronting barriers to human-robot cooperation: balancing efficiency and risk in machine behavior

Many technical and psychological challenges make it difficult to design machines that effectively cooperate with people. To better understand these challenges, we conducted a series of studies investigating human-human, robot-robot, and human-robot cooperation in a strategically rich resource-sharing scenario, which required players to balance efficiency, fairness, and risk. In these studies, both human-human and robot-robot dyads typically learned efficient and risky cooperative solutions when they could communicate. In the absence of communication, robot dyads still often learned the same efficient solution, but human dyads achieved a less efficient (less risky) form of cooperation. This difference in how people and machines treat risk appeared to discourage human-robot cooperation, as human-robot dyads frequently failed to cooperate without communication. These results indicate that machine behavior should better align with human behavior, promoting efficiency while simultaneously considering human tendencies toward risk and fairness.

•

Experiments show that people learned risk-averse solutions without communication

•

With and without communication, robot pairs learned risky, but efficient, outcomes

•

Human-robot pairs often learned risky, but efficient, solutions with communication

•

Without communication, behavioral asymmetries inhibited human-robot cooperation

Making eye contact with a robot: Psychophysiological responses to eye contact with a human and with a humanoid robot

Previous research has shown that eye contact, in human-human interaction, elicits increased affective and attention related psychophysiological responses. In the present study, we investigated whether eye contact with a humanoid robot would elicit these responses. Participants were facing a humanoid robot (NAO) or a human partner, both physically present and looking at or away from the participant. The results showed that both in human-robot and human-human condition, eye contact versus averted gaze elicited greater skin conductance responses indexing autonomic arousal, greater facial zygomatic muscle responses (and smaller corrugator responses) associated with positive affect, and greater heart deceleration responses indexing attention allocation. With regard to the skin conductance and zygomatic responses, the human model's gaze direction had a greater effect on the responses as compared to the robot's gaze direction. In conclusion, eye contact elicits automatic affective and attentional reactions both when shared with a humanoid robot and with another human.

Social Robots to Test Flexibility of Human Social Cognition

As the field of social robotics has been dynamically growing and expanding over various areas of research and application, in which robots can be of assistance and companionship for humans, this paper offers a different perspective on a role that social robots can also play, namely the role of informing us about flexibility of human mechanisms of social cognition. The paper focuses on studies in which robots have been used as a new type of "stimuli" in psychological experiments to examine whether similar mechanisms of social cognition would be activated in interaction with a robot, as would be elicited in interaction with another human. Analysing studies in which a direct comparison has been made between a robot and a human agent, the paper examines whether for robot agents, the brain re-uses the same mechanisms that have been developed for interaction with other humans in terms of perception, action representation, attention and higher-order social cognition. Based on this analysis, the paper concludes that the human socio-cognitive mechanisms, in adult brains, are sufficiently flexible to be re-used for robotic agents, at least for those that have some level of resemblance to humans.

Neural mechanisms of deception in a social context: an fMRI replication study

Deception is a form of manipulation aimed at misleading another person by conveying false or truthful messages. Manipulative truthful statements could be considered as sophisticated deception and elicit an increased cognitive load. However, only one fMRI study reported its neural correlates. To provide independent evidence for sophisticated deception, we carried out an fMRI study replicating the experimental paradigm and Bayesian statistical approach utilized in that study. During the experiment, participants played a game against an opponent by sending deliberate deceptive or honest messages. Compared to truth-telling, deceptive intentions, regardless of how they were fulfilled, were associated with increased BOLD signals in the bilateral temporoparietal junction (TPJ), left precuneus, and right superior temporal sulcus (STS). The right TPJ participates in the attribution of mental states, acting in a social context, and moral behaviour. Moreover, the other revealed brain areas have been considered nodes in the theory of mind brain neural system. Therefore, the obtained results reflect an increased demand for socio‑cognitive processes associated with deceptive intentions. We replicated the original study showing the involvement of the right TPJ and expanded upon it by revealing the involvement of the left TPJ, left precuneus and right STS in actions with deceptive intentions.

Attributed social context and emotional content recruit frontal and limbic brain regions during virtual feedback processing

In communication, who is communicating can be just as important as what is said. However, sender identity in virtual communication is often inferred rather than perceived. Therefore, the present research investigates the brain structures activated by sender identity attributions and evaluative feedback processing during virtual communication. In a functional magnetic resonance imaging (fMRI) study, 32 participants were told that they would receive personality feedback, either sent from another human participant or from a randomly acting computer. In reality, both conditions contained random but counterbalanced feedback, automatically delivered by approving or denying negative, neutral, or positive adjectives. Although physically identical, feedback attributed to the "human" sender activated multiple regions within a "social brain" network, including the superior frontal, medial prefrontal, and orbitofrontal cortex, anterior and posterior parts of the cingulate cortex, and the bilateral insula. Regardless of attributed sender, positive feedback increased responses in the striatum and bilateral amygdalae, while negative compared to neutral feedback elicited stronger insula and somatosensory responses. These results reveal the recruitment of an extensive mentalizing and social brain network by mere sender attributions and the activation of brain structures related to reward and punishment by verbal feedback, demonstrating its embodied processing.

Re-imaging the intentional stance

The commonly used paradigm to investigate Dennet's 'intentional stance' compares neural activation when participants compete with a human versus a computer. This paradigm confounds whether the opponent is natural or artificial and whether it is intentional or an automaton. This functional magnetic resonance imaging study is, to our knowledge, the first to investigate the intentional stance by orthogonally varying perceptions of the opponents' intentionality (responding actively or passively according to a script) and embodiment (human or a computer). The mere perception of the opponent (whether human or computer) as intentional activated the mentalizing network: the temporoparietal junction (TPJ) bilaterally, right temporal pole, anterior paracingulate cortex (aPCC) and the precuneus. Interacting with humans versus computers induced activations in a more circumscribed right lateralized subnetwork within the mentalizing network, consisting of the TPJ and the aPCC, possibly reflective of the tendency to spontaneously attribute intentionality to humans. The interaction between intentionality (active versus passive) and opponent (human versus computer) recruited the left frontal pole, possibly in response to violations of the default intentional stance towards humans and computers. Employing an orthogonal design is important to adequately capture Dennett's conception of the intentional stance as a mentalizing strategy that can apply equally well to humans and other intentional agents.

Attending to Eliza: rapid brain responses reflect competence attribution in virtual social feedback processing

In the age of virtual communication, the source of a message is often inferred rather than perceived, raising the question of how sender attributions affect content processing. We investigated this issue in an evaluative feedback scenario. Participants were told that an expert psychotherapist, a layperson or a randomly acting computer was going to give them online positive, neutral or negative personality feedback while high-density EEG was recorded. Sender attribution affected processing rapidly, even though the feedback was on average identical. Event-related potentials revealed a linear increase with attributed expertise beginning 150 ms after disclosure and most pronounced for N1, P2 and early posterior negativity components. P3 and late positive potential amplitudes were increased for both human senders and for emotionally significant (positive or negative) feedback. Strikingly, feedback from a putative expert prompted large P3 responses, even for inherently neutral content. Source analysis localized early enhancements due to attributed sender expertise in frontal and somatosensory regions and later responses in the posterior cingulate and extended visual and parietal areas, supporting involvement of mentalizing, embodied processing and socially motivated attention. These findings reveal how attributed sender expertise rapidly alters feedback processing in virtual interaction and have implications for virtual therapy and online communication.

Brain stimulation to left prefrontal cortex modulates attentional orienting to gaze cues

In social interactions, we rely on non-verbal cues like gaze direction to understand the behaviour of others. How we react to these cues is determined by the degree to which we believe that they originate from an entity with a mind capable of having internal states and showing intentional behaviour, a process called mind perception. While prior work has established a set of neural regions linked to mind perception, research has just begun to examine how mind perception affects social-cognitive mechanisms like gaze processing on a neuronal level. In the current experiment, participants performed a social attention task (i.e. attentional orienting to gaze cues) with either a human or a robot agent (i.e. manipulation of mind perception) while transcranial direct current stimulation (tDCS) was applied to prefrontal and temporo-parietal brain areas. The results show that temporo-parietal stimulation did not modulate mechanisms of social attention, neither in response to the human nor in response to the robot agent, whereas prefrontal stimulation enhanced attentional orienting in response to human gaze cues and attenuated attentional orienting in response to robot gaze cues. The findings suggest that mind perception modulates low-level mechanisms of social cognition via prefrontal structures, and that a certain degree of mind perception is essential in order for prefrontal stimulation to affect mechanisms of social attention. This article is part of the theme issue 'From social brains to social robots: applying neurocognitive insights to human-robot interaction'.

Brain activity during reciprocal social interaction investigated using conversational robots as control condition

We present a novel functional magnetic resonance imaging paradigm for second-person neuroscience. The paradigm compares a human social interaction (human-human interaction, HHI) to an interaction with a conversational robot (human-robot interaction, HRI). The social interaction consists of 1 min blocks of live bidirectional discussion between the scanned participant and the human or robot agent. A final sample of 21 participants is included in the corpus comprising physiological (blood oxygen level-dependent, respiration and peripheral blood flow) and behavioural (recorded speech from all interlocutors, eye tracking from the scanned participant, face recording of the human and robot agents) data. Here, we present the first analysis of this corpus, contrasting neural activity between HHI and HRI. We hypothesized that independently of differences in behaviour between interactions with the human and robot agent, neural markers of mentalizing (temporoparietal junction (TPJ) and medial prefrontal cortex) and social motivation (hypothalamus and amygdala) would only be active in HHI. Results confirmed significantly increased response associated with HHI in the TPJ, hypothalamus and amygdala, but not in the medial prefrontal cortex. Future analysis of this corpus will include fine-grained characterization of verbal and non-verbal behaviours recorded during the interaction to investigate their neural correlates. This article is part of the theme issue 'From social brains to social robots: applying neurocognitive insights to human-robot interaction'.

Coin, telephone, and handcuffs: Neural correlates of social knowledge of inanimate objects

A variety of objects are made to serve social functions. The use of these objects has greatly enriched and expanded our social behaviors. How do our brains represent the social knowledge of inanimate objects such as coins, telephones, and handcuffs? According to a recent version of the grounded theory, social knowledge of inanimate objects might be grounded in the mentalizing network, as the social functions of inanimate objects are closely associated with the intentions of the people using them. However, there is also evidence that the mentalizing network may only get activated when a human/mental agent is detected. Using fMRI, we explored the neural correlates of social knowledge of inanimate objects by comparing the brain activation evoked by high-sociality object nouns (e.g., banknote) with that evoked by low-sociality object nouns (e.g., battery). The left anterior superior temporal sulcus, a classic part of the mentalizing network, showed higher activation for the high-sociality inanimate object nouns than for the low-sociality ones in the whole-brain analysis. Several other areas of the mentalizing network showed sensitivity to object sociality in small volume correction and/or region-of-interest analyses. Our findings indicate that social knowledge of inanimate objects is supported by brain areas in the mentalizing network.

Seeing minds in others: Mind perception modulates low-level social-cognitive performance and relates to ventromedial prefrontal structures

In social interactions, we rely on nonverbal cues like gaze direction to understand the behavior of others. How we react to these cues is affected by whether they are believed to originate from an entity with a mind, capable of having internal states (i.e., mind perception). While prior work has established a set of neural regions linked to social-cognitive processes like mind perception, the degree to which activation within this network relates to performance in subsequent social-cognitive tasks remains unclear. In the current study, participants performed a mind perception task (i.e., judging the likelihood that faces, varying in physical human-likeness, have internal states) while event-related fMRI was collected. Afterwards, participants performed a social attention task outside the scanner, during which they were cued by the gaze of the same faces that they previously judged within the mind perception task. Parametric analyses of the fMRI data revealed that activity within ventromedial prefrontal cortex (vmPFC) was related to both mind ratings inside the scanner and gaze-cueing performance outside the scanner. In addition, other social brain regions were related to gaze-cueing performance, including frontal areas like the left insula, dorsolateral prefrontal cortex, and inferior frontal gyrus, as well as temporal areas like the left temporo-parietal junction and bilateral temporal gyri. The findings suggest that functions subserved by the vmPFC are relevant to both mind perception and social attention, implicating a role of vmPFC in the top-down modulation of low-level social-cognitive processes.

National Stereotypes and Robots' Perception: The "Made in" Effect

In the near future, the human social environment worldwide might be populated by humanoid robots. The way we perceive these new social agents could depend on basic social psychological processes such as social categorization. Recent results indicate that humans can make use of social stereotypes when faced with robots based on their characterization as "male" or "female" and a perception of their group membership. However, the question of the application of nationality-based stereotypes to robots has not yet been studied. Given that humans attribute different levels of warmth and competence (the two universal dimensions of social perception) to individuals based in part on their nationality, we hypothesized that the way robots are perceived differs depending on their country of origin. In this study, participants had to evaluate four robots differing in their anthropomorphic shape. For each participant, these robots were presented as coming from one of four different countries selected for their level of perceived warmth and competence. Each robot was evaluated on their anthropomorphic and human traits. As expected, the country of origin's warmth and competence level biased the perception of robots in terms of the attribution of social and human traits. Our findings also indicated that these effects differed according to the extent to which the robots were anthropomorphically shaped. We discuss these results in relation to the way in which social constructs are applied to robots.

Do We Adopt the Intentional Stance Toward Humanoid Robots?

In daily social interactions, we need to be able to navigate efficiently through our social environment. According to Dennett (1971), explaining and predicting others' behavior with reference to mental states (adopting the intentional stance) allows efficient social interaction. Today we also routinely interact with artificial agents: from Apple's Siri to GPS navigation systems. In the near future, we might start casually interacting with robots. This paper addresses the question of whether adopting the intentional stance can also occur with respect to artificial agents. We propose a new tool to explore if people adopt the intentional stance toward an artificial agent (humanoid robot). The tool consists in a questionnaire that probes participants' stance by requiring them to choose the likelihood of an explanation (mentalistic vs. mechanistic) of a behavior of a robot iCub depicted in a naturalistic scenario (a sequence of photographs). The results of the first study conducted with this questionnaire showed that although the explanations were somewhat biased toward the mechanistic stance, a substantial number of mentalistic explanations were also given. This suggests that it is possible to induce adoption of the intentional stance toward artificial agents, at least in some contexts.

Intentional Mindset Toward Robots-Open Questions and Methodological Challenges

Natural and effective interaction with humanoid robots should involve social cognitive mechanisms of the human brain that normally facilitate social interaction between humans. Recent research has indicated that the presence and efficiency of these mechanisms in human-robot interaction (HRI) might be contingent on the adoption of a set of attitudes, mindsets, and beliefs concerning the robot's inner machinery. Current research is investigating the factors that influence these mindsets, and how they affect HRI. This review focuses on a specific mindset, namely the “intentional mindset” in which intentionality is attributed to another agent. More specifically, we focus on the concept of adopting the intentional stance toward robots, i.e., the tendency to predict and explain the robots' behavior with reference to mental states. We discuss the relationship between adoption of intentional stance and lower-level mechanisms of social cognition, and we provide a critical evaluation of research methods currently employed in this field, highlighting common pitfalls in the measurement of attitudes and mindsets.

Pupillary Responses to Robotic and Human Emotions: The Uncanny Valley and Media Equation Confirmed

Physiological responses during human–robots interaction are useful alternatives to subjective measures of uncanny feelings for nearly humanlike robots (uncanny valley) and comparable emotional responses between humans and robots (media equation). However, no studies have employed the easily accessible measure of pupillometry to confirm the uncanny valley and media equation hypotheses, evidence in favor of the existence of these hypotheses in interaction with emotional robots is scarce, and previous studies have not controlled for low level image statistics across robot appearances. We therefore recorded pupil size of 40 participants that viewed and rated pictures of robotic and human faces that expressed a variety of basic emotions. The robotic faces varied along the dimension of human likeness from cartoonish to humanlike. We strictly controlled for confounding factors by removing backgrounds, hair, and color, and by equalizing low level image statistics. After the presentation phase, participants indicated to what extent the robots appeared uncanny and humanlike, and whether they could imagine social interaction with the robots in real life situations. The results show that robots rated as nearly humanlike scored higher on uncanniness, scored lower on imagined social interaction, evoked weaker pupil dilations, and their emotional expressions were more difficult to recognize. Pupils dilated most strongly to negative expressions and the pattern of pupil responses across emotions was highly similar between robot and human stimuli. These results highlight the usefulness of pupillometry in emotion studies and robot design by confirming the uncanny valley and media equation hypotheses.

From automata to animate beings: the scope and limits of attributing socialness to artificial agents

Understanding the mechanisms and consequences of attributing socialness to artificial agents has important implications for how we can use technology to lead more productive and fulfilling lives. Here, we integrate recent findings on the factors that shape behavioral and brain mechanisms that support social interactions between humans and artificial agents. We review how visual features of an agent, as well as knowledge factors within the human observer, shape attributions across dimensions of socialness. We explore how anthropomorphism and dehumanization further influence how we perceive and interact with artificial agents. Based on these findings, we argue that the cognitive reconstruction within the human observer is likely to be far more crucial in shaping our interactions with artificial agents than previously thought, while the artificial agent's visual features are possibly of lesser importance. We combine these findings to provide an integrative theoretical account based on the "like me" hypothesis, and discuss the key role played by the Theory-of-Mind network, especially the temporal parietal junction, in the shift from mechanistic to social attributions. We conclude by highlighting outstanding questions on the impact of long-term interactions with artificial agents on the behavioral and brain mechanisms of attributing socialness to these agents.

Folk-Psychological Interpretation of Human vs. Humanoid Robot Behavior: Exploring the Intentional Stance toward Robots

People rely on shared folk-psychological theories when judging behavior. These theories guide people's social interactions and therefore need to be taken into consideration in the design of robots and other autonomous systems expected to interact socially with people. It is, however, not yet clear to what degree the mechanisms that underlie people's judgments of robot behavior overlap or differ from the case of human or animal behavior. To explore this issue, participants (N = 90) were exposed to images and verbal descriptions of eight different behaviors exhibited either by a person or a humanoid robot. Participants were asked to rate the intentionality, controllability and desirability of the behaviors, and to judge the plausibility of seven different types of explanations derived from a recently proposed psychological model of lay causal explanation of human behavior. Results indicate: substantially similar judgments of human and robot behavior, both in terms of (1a) ascriptions of intentionality/controllability/desirability and in terms of (1b) plausibility judgments of behavior explanations; (2a) high level of agreement in judgments of robot behavior - (2b) slightly lower but still largely similar to agreement over human behaviors; (3) systematic differences in judgments concerning the plausibility of goals and dispositions as explanations of human vs. humanoid behavior. Taken together, these results suggest that people's intentional stance toward the robot was in this case very similar to their stance toward the human.

Robots As Intentional Agents: Using Neuroscientific Methods to Make Robots Appear More Social

Robots are increasingly envisaged as our future cohabitants. However, while considerable progress has been made in recent years in terms of their technological realization, the ability of robots to interact with humans in an intuitive and social way is still quite limited. An important challenge for social robotics is to determine how to design robots that can perceive the user's needs, feelings, and intentions, and adapt to users over a broad range of cognitive abilities. It is conceivable that if robots were able to adequately demonstrate these skills, humans would eventually accept them as social companions. We argue that the best way to achieve this is using a systematic experimental approach based on behavioral and physiological neuroscience methods such as motion/eye-tracking, electroencephalography, or functional near-infrared spectroscopy embedded in interactive human-robot paradigms. This approach requires understanding how humans interact with each other, how they perform tasks together and how they develop feelings of social connection over time, and using these insights to formulate design principles that make social robots attuned to the workings of the human brain. In this review, we put forward the argument that the likelihood of artificial agents being perceived as social companions can be increased by designing them in a way that they are perceived as intentional agents that activate areas in the human brain involved in social-cognitive processing. We first review literature related to social-cognitive processes and mechanisms involved in human-human interactions, and highlight the importance of perceiving others as intentional agents to activate these social brain areas. We then discuss how attribution of intentionality can positively affect human-robot interaction by (a) fostering feelings of social connection, empathy and prosociality, and by (b) enhancing performance on joint human-robot tasks. Lastly, we describe circumstances under which attribution of intentionality to robot agents might be disadvantageous, and discuss challenges associated with designing social robots that are inspired by neuroscientific principles.

Rethinking Social Cognition in Light of Psychosis: Reciprocal Implications for Cognition and Psychopathology

The positive symptoms of psychosis largely involve the experience of illusory social actors and yet our current measures of social cognition, at best, only weakly predict their presence. We review evidence to suggest that the range of current approaches in social cognition is not sufficient to explain the fundamentally social nature of these experiences. We argue that social agent representation is an important organising principle for understanding social cognition and that alterations in social agent representation may be a factor in the formation of delusions and hallucination in psychosis. We evaluate the feasibility of this approach in light of clinical and non-clinical studies, developmental research, cognitive anthropology and comparative psychology. We conclude with recommendations for empirical testing of specific hypotheses and how studies of social cognition could more fully capture the extent of social reasoning and experience in both psychosis and more prosaic mental states.

Advice Taking from Humans and Machines: An fMRI and Effective Connectivity Study

With new technological advances, advice can come from different sources such as machines or humans, but how individuals respond to such advice and the neural correlates involved need to be better understood. We combined functional MRI and multivariate Granger causality analysis with an X-ray luggage-screening task to investigate the neural basis and corresponding effective connectivity involved with advice utilization from agents framed as experts. Participants were asked to accept or reject good or bad advice from a human or machine agent with low reliability (high false alarm rate). We showed that unreliable advice decreased performance overall and participants interacting with the human agent had a greater depreciation of advice utilization during bad advice compared to the machine agent. These differences in advice utilization can be perceivably due to reevaluation of expectations arising from association of dispositional credibility for each agent. We demonstrated that differences in advice utilization engaged brain regions that may be associated with evaluation of personal characteristics and traits (precuneus, posterior cingulate cortex, temporoparietal junction) and interoception (posterior insula). We found that the right posterior insula and left precuneus were the drivers of the advice utilization network that were reciprocally connected to each other and also projected to all other regions. Our behavioral and neuroimaging results have significant implications for society because of progressions in technology and increased interactions with machines.

The Body That Speaks: Recombining Bodies and Speech Sources in Unscripted Face-to-Face Communication

This article examines advances in research methods that enable experimental substitution of the speaking body in unscripted face-to-face communication. A taxonomy of six hybrid social agents is presented by combining three types of bodies (mechanical, virtual, and human) with either an artificial or human speech source. Our contribution is to introduce and explore the significance of two particular hybrids: (1) the cyranoid method that enables humans to converse face-to-face through the medium of another person's body, and (2) the echoborg method that enables artificial intelligence to converse face-to-face through the medium of a human body. These two methods are distinct in being able to parse the unique influence of the human body when combined with various speech sources. We also introduce a new framework for conceptualizing the body's role in communication, distinguishing three levels: self's perspective on the body, other's perspective on the body, and self's perspective of other's perspective on the body. Within each level the cyranoid and echoborg methodologies make important research questions tractable. By conceptualizing and synthesizing these methods, we outline a novel paradigm of research on the role of the body in unscripted face-to-face communication.

Believing androids - fMRI activation in the right temporo-parietal junction is modulated by ascribing intentions to non-human agents

Attributing mind to interaction partners has been shown to increase the social relevance we ascribe to others' actions and to modulate the amount of attention dedicated to them. However, it remains unclear how the relationship between higher-order mind attribution and lower-level attention processes is established in the brain. In this neuroimaging study, participants saw images of an anthropomorphic robot that moved its eyes left- or rightwards to signal the appearance of an upcoming stimulus in the same (valid cue) or opposite location (invalid cue). Independently, participants' beliefs about the intentionality underlying the observed eye movements were manipulated by describing the eye movements as under human control or preprogrammed. As expected, we observed a validity effect behaviorally and neurologically (increased response times and activation in the invalid vs. valid condition). More importantly, we observed that this effect was more pronounced for the condition in which the robot's behavior was believed to be controlled by a human, as opposed to be preprogrammed. This interaction effect between cue validity and belief was, however, only found at the neural level and was manifested as a significant increase of activation in bilateral anterior temporoparietal junction.

Embodied artificial agents for understanding human social cognition

In this paper, we propose that experimental protocols involving artificial agents, in particular the embodied humanoid robots, provide insightful information regarding social cognitive mechanisms in the human brain. Using artificial agents allows for manipulation and control of various parameters of behaviour, appearance and expressiveness in one of the interaction partners (the artificial agent), and for examining effect of these parameters on the other interaction partner (the human). At the same time, using artificial agents means introducing the presence of artificial, yet human-like, systems into the human social sphere. This allows for testing in a controlled, but ecologically valid, manner human fundamental mechanisms of social cognition both at the behavioural and at the neural level. This paper will review existing literature that reports studies in which artificial embodied agents have been used to study social cognition and will address the question of whether various mechanisms of social cognition (ranging from lower- to higher-order cognitive processes) are evoked by artificial agents to the same extent as by natural agents, humans in particular. Increasing the understanding of how behavioural and neural mechanisms of social cognition respond to artificial anthropomorphic agents provides empirical answers to the conundrum 'What is a social agent?'

In our own image? Emotional and neural processing differences when observing human-human vs human-robot interactions

Notwithstanding the significant role that human-robot interactions (HRI) will play in the near future, limited research has explored the neural correlates of feeling eerie in response to social robots. To address this empirical lacuna, the current investigation examined brain activity using functional magnetic resonance imaging while a group of participants (n = 26) viewed a series of human-human interactions (HHI) and HRI. Although brain sites constituting the mentalizing network were found to respond to both types of interactions, systematic neural variation across sites signaled diverging social-cognitive strategies during HHI and HRI processing. Specifically, HHI elicited increased activity in the left temporal-parietal junction indicative of situation-specific mental state attributions, whereas HRI recruited the precuneus and the ventromedial prefrontal cortex (VMPFC) suggestive of script-based social reasoning. Activity in the VMPFC also tracked feelings of eeriness towards HRI in a parametric manner, revealing a potential neural correlate for a phenomenon known as the uncanny valley. By demonstrating how understanding social interactions depends on the kind of agents involved, this study highlights pivotal sub-routes of impression formation and identifies prominent challenges in the use of humanoid robots.

Different impressions of other agents obtained through social interaction uniquely modulate dorsal and ventral pathway activities in the social human brain

Internal (neuronal) representations in the brain are modified by our experiences, and this phenomenon is not unique to sensory and motor systems. Here, we show that different impressions obtained through social interaction with a variety of agents uniquely modulate activity of dorsal and ventral pathways of the brain network that mediates human social behavior. We scanned brain activity with functional magnetic resonance imaging (fMRI) in 16 healthy volunteers when they performed a simple matching-pennies game with a human, human-like android, mechanical robot, interactive robot, and a computer. Before playing this game in the scanner, participants experienced social interactions with each opponent separately and scored their initial impressions using two questionnaires. We found that the participants perceived opponents in two mental dimensions: one represented "mind-holderness" in which participants attributed anthropomorphic impressions to some of the opponents that had mental functions, while the other dimension represented "mind-readerness" in which participants characterized opponents as intelligent. Interestingly, this "mind-readerness" dimension correlated to participants frequently changing their game tactic to prevent opponents from envisioning their strategy, and this was corroborated by increased entropy during the game. We also found that the two factors separately modulated activity in distinct social brain regions. Specifically, mind-holderness modulated activity in the dorsal aspect of the temporoparietal junction (TPJ) and medial prefrontal and posterior paracingulate cortices, while mind-readerness modulated activity in the ventral aspect of TPJ and the temporal pole. These results clearly demonstrate that activity in social brain networks is modulated through pre-scanning experiences of social interaction with a variety of agents. Furthermore, our findings elucidated the existence of two distinct functional networks in the social human brain. Social interaction with anthropomorphic or intelligent-looking agents may distinctly shape the internal representation of our social brain, which may in turn determine how we behave for various agents that we encounter in our society.

The neural underpinnings of an optimal exploitation of social information under uncertainty

Social information influences decision-making through an integration of information derived from individual experience with that derived from observing the actions of others. This raises the question as to which extent one should utilize social information. One strategy is to make use of uncertainty estimates, leading to a copy-when-uncertain strategy that weights information from individual and social sources based on their respective reliabilities. Here, we investigate this integration process by extending models of Bayes optimal integration of sensory information to a social decision context. We then use a key parameter of our behavioral model in conjunction with functional magnetic resonance imaging to identify the neural substrate that is specifically linked to the fidelity of this integration process. We show that individuals behave near Bayes optimal when integrating two distinct sources of social information but systematically deviate from Bayes optimal choice when integrating individual with social information. This systematic behavioral deviation from optimality is linked to activity of left inferior frontal gyrus. Thus, an ability to optimally exploit social information depends on processes that overcome an egocentric bias, and this regulatory role involves the left inferior prefrontal cortex. The findings provide a mechanistic explanation for observations wherein individuals neglect the benefits from exploiting social information.

Perceiving nonverbal behavior: neural correlates of processing movement fluency and contingency in dyadic interactions

Despite the fact that nonverbal dyadic social interactions are abundant in the environment, the neural mechanisms underlying their processing are not yet fully understood. Research in the field of social neuroscience has suggested that two neural networks appear to be involved in social understanding: (1) the action observation network (AON) and (2) the social neural network (SNN). The aim of this study was to determine the differential contributions of the AON and the SNN to the processing of nonverbal behavior as observed in dyadic social interactions. To this end, we used short computer animation sequences displaying dyadic social interactions between two virtual characters and systematically manipulated two key features of movement activity, which are known to influence the perception of meaning in nonverbal stimuli: (1) movement fluency and (2) contingency of movement patterns. A group of 21 male participants rated the "naturalness" of the observed scenes on a four-point scale while undergoing fMRI. Behavioral results showed that both fluency and contingency significantly influenced the "naturalness" experience of the presented animations. Neurally, the AON was preferentially engaged when processing contingent movement patterns, but did not discriminate between different degrees of movement fluency. In contrast, regions of the SNN were engaged more strongly when observing dyads with disturbed movement fluency. In conclusion, while the AON is involved in the general processing of contingent social actions, irrespective of their kinematic properties, the SNN is preferentially recruited when atypical kinematic properties prompt inferences about the agents' intentions.