The motor theory of social cognition: a critique

Observed reaching speed signals stimulus value and informs foraging

Optimal foraging requires agents to strike a balance between potential costs and rewards of interacting with stimuli in the environment. Research on human and animal foraging shows that the value an agent assigns to a stimulus is correlated with the speed of their reaching movement towards that stimulus (Shadmehr et al., 2019). Humans and other animals learn about the value of stimuli in their environment by observing others acting (Pyke, 1984; Boyd, Richerson & Henrich, 2011). Considering that humans are able to derive specific mental states such as intentions, emotions or confidence from specific movement parameters (Becchio et al., 2012), we aimed to investigate whether observers can use an actor's movement speed to: 1) infer the value of a foraging stimulus; and 2) use such cues to inform their own foraging behavior. The current study first replicated the effect of stimulus value on reaching movements in a novel foraging task (Exp. 1, N = 34). In three further experiments, we demonstrate that, depending on the speed by which an actor reaches for stimuli, observers infer the value of these stimuli (Exp. 2, N = 54), express foraging preferences (Exp. 3, N = 54), and invest time and effort to forage (Exp. 4, N = 105). This demonstrates that observers optimize their own explore-exploit decisions by inferring the value of a stimulus from the manner by which an actor approaches it, highlighting the fundamental role that action understanding plays in successful foraging.

Kinematics in context: Predicting other's action intentions entails the perception of affordances

Intention prediction is essential for successful social interaction, but traditional research focusing solely on movement kinematics often overlooks the array of action possibilities in natural settings. This study employs a mixed-methods approach to explore intention prediction, analysing free-text responses from participants who watched videos of an actor reaching for a cup, bottle, or spoon, each with a distinct intention. Each video included varied environmental contexts to suggest specific intentions (e.g., full cups for drinking, empty cups for clearing) or presented ambiguous contexts (e.g., half-full cups). We found that participants' intention predictions depended on the variety of action possibilities presented by both kinematics and context. Participants tended to identify the primary action possibility of the grasped item as the intended action when both kinematics and context supported its feasibility. Predictions diversified when kinematics or context suggested that the object's primary action was less likely. Our findings suggest that while intention predictions can sometimes be inaccurate, they align with the (most functional) action possibilities (i.e., affordances) indicated by the actor's movements within a given context.

Observers translate information about other agents' higher-order goals into expectations about their forthcoming action kinematics

Social perception relies on the ability to understand the higher-order goals that drive other people's behaviour. Under predictive coding views, this ability relies on a Bayesian-like hypothesis-testing mechanism, which translates prior higher-order information about another agent's goals into perceptual predictions of the actions with which these goals can be realised and tests these predictions against the actual behaviour. We tested this hypothesis in three preregistered experiments. Participants viewed an agent's hand next to two possible target objects (e.g., donut, hammer) and heard the agent state a higher-order goal, which could be fulfilled by one of the two objects (e.g., "I'm really hungry!"). The hand then reached towards the objects and disappeared at an unpredictable point mid-motion, and participants reported its last seen location. The results revealed the hypothesized integration of prior goals and observed hand trajectories. Reported hand disappearance points were predictively shifted towards the object with which the goal could be best realised. These biases were stronger when goal statements were explicitly processed (Experiment 1) than when passively heard (Experiment 2), more robust for more ambiguous reaches, and they could not be explained by attentional shifts towards the objects or participants' awareness of the experimental hypotheses. Moreover, similar biases were not elicited (Experiment 3) when the agent's statements referred to the same objects but did not specify them as action goals (e.g., "I'm really not hungry!"). These findings link action understanding to predictive/Bayesian mechanisms of social perception and Theory of Mind and provide the first evidence that prior knowledge about others' higher-level goals cascades to lower-level action expectations, which ultimately influence the visuospatial representation of others' behaviour.

Assessing the Recognition of Social Interactions Through Body Motion in the Routine Care of Patients with Post-Lingual Sensorineural Hearing Loss

Background: Body motion significantly contributes to understanding communicative and social interactions, especially when auditory information is impaired. The visual skills of people with hearing loss are often enhanced and compensate for some of the missing auditory information. In the present study, we investigated the recognition of social interactions by observing body motion in people with post-lingual sensorineural hearing loss (SNHL). Methods: In total, 38 participants with post-lingual SNHL and 38 matched normally hearing individuals (NHIs) were presented with point-light stimuli of two agents who were either engaged in a communicative interaction or acting independently. They were asked to classify the actions as communicative vs. independent and to select the correct action description. Results: No significant differences were found between the participants with SNHL and the NHIs when classifying the actions. However, the participants with SNHL showed significantly lower performance compared with the NHIs in the description task due to a higher tendency to misinterpret communicative stimuli. In addition, acquired SNHL was associated with a significantly higher number of errors, with a tendency to over-interpret independent stimuli as communicative and to misinterpret communicative actions. Conclusions: The findings of this study suggest a misinterpretation of visual understanding of social interactions in individuals with SNHL and over-interpretation of communicative intentions in SNHL acquired later in life.

Music, body, and machine: gesture-based synchronization in human-robot musical interaction

Musical performance relies on nonverbal cues for conveying information among musicians. Human musicians use bodily gestures to communicate their interpretation and intentions to their collaborators, from mood and expression to anticipatory cues regarding structure and tempo. Robotic Musicians can use their physical bodies in a similar way when interacting with fellow musicians. The paper presents a new theoretical framework to classify musical gestures and a study evaluating the effect of robotic gestures on synchronization between human musicians and Shimon - a robotic marimba player developed at Georgia Tech. Shimon utilizes head and arm movements to signify musical information such as expected notes, tempo, and beat. The study, in which piano players were asked to play along with Shimon, assessed the effectiveness of these gestures on human-robot synchronization. Subjects were evaluated for their ability to synchronize with unknown tempo changes as communicated by Shimon's ancillary and social gestures. The results demonstrate the significant contribution of non-instrumental gestures to human-robot synchronization, highlighting the importance of non-music-making gestures for anticipation and coordination in human-robot musical collaboration. Subjects also indicated more positive feelings when interacting with the robot's ancillary and social gestures, indicating the role of these gestures in supporting engaging and enjoyable musical experiences.

Who am I with my Lewy bodies? The insula as a core region of the self-concept networks

BACKGROUND

Dementia with Lewy bodies (DLB) is characterized by insular atrophy, which occurs at the early stage of the disease. Damage to the insula has been associated with disorders reflecting impairments of the most fundamental components of the self, such as anosognosia, which is a frequently reported symptom in patients with Lewy bodies (LB). The purpose of this study was to investigate modifications of the self-concept (SC), another component of the self, and to identify neuroanatomical correlates, in prodromal to mild DLB.

METHODS

Twenty patients with prodromal to mild DLB were selected to participate in this exploratory study along with 20 healthy control subjects matched in terms of age, gender, and level of education. The Twenty Statements Test (TST) was used to assess the SC. Behavioral performances were compared between LB patients and control subjects. Three-dimensional magnetic resonance images (MRI) were acquired for all participants and correlational analyses were performed using voxel-based morphometry (VBM) in whole brain and using a mask for the insula.

RESULTS

The behavioral results on the TST showed significantly impaired performances in LB patients in comparison with control subjects (p < .0001). Correlational analyses using VBM revealed positive correlations between the TST and grey matter volume within insular cortex, right supplementary motor area, bilateral inferior temporal gyri, right inferior frontal gyrus, and left lingual gyrus, using a threshold of p = .001 uncorrected, including total intracranial volume (TIV), age, and MMSE as nuisance covariates. Additionally, correlational analysis using a mask for the insula revealed positive correlation with grey matter volume within bilateral insular cortex, using a threshold of p = .005.

CONCLUSIONS

The behavioral results confirm the existence of SC impairments in LB patients from the prodromal stage of the disease, compared to matched healthy controls. As we expected, VBM analyses revealed involvement of the insula, among that of other brain regions, already known to be involved in other self-components. While this study is exploratory, our findings provide important insights regarding the involvement of the insula within the self, confirming the insula as a core region of the self-networks, including for high-order self-representations such as the SC.

Flexible Cultural Learning Through Action Coordination

The cultural transmission of technical know-how has proven vital to the success of our species. The broad diversity of learning contexts and social configurations, as well as the various kinds of coordinated interactions they involve, speaks to our capacity to flexibly adapt to and succeed in transmitting vital knowledge in various learning contexts. Although often recognized by ethnographers, the flexibility of cultural learning has so far received little attention in terms of cognitive mechanisms. We argue that a key feature of the flexibility of cultural learning is that both the models and learners recruit cognitive mechanisms of action coordination to modulate their behavior contingently on the behavior of their partner, generating a process of mutual adaptation supporting the successful transmission of technical skills in diverse and fluctuating learning environments. We propose that the study of cultural learning would benefit from the experimental methods, results, and insights of joint-action research and, complementarily, that the field of joint-action research could expand its scope by integrating a learning and cultural dimension. Bringing these two fields of research together promises to enrich our understanding of cultural learning, its contextual flexibility, and joint action coordination.

Action planning and execution cues influence economic partner choice

Prudently choosing who to interact with and who to avoid is an important ability to ensure that we benefit from a cooperative interaction. While the role of others' preferences, attributes, and values in partner choice have been established (Rossetti, Hilbe & Hauser, 2022), much less is known about whether the manner in which a potential partner plans and implements a decision provides helpful cues for partner choice. We used a partner choice paradigm in which participants chose who to interact with in the Prisoners' Dilemma. Before choosing a cooperation partner, participants were presented with information about the potential partners' decision-related actions in another round of the Prisoners' Dilemma. They received either information about the potential partners' planning during decision making (i.e., decision-time; Experiment 1) or action execution during decision implementation (i.e., movement directness; Experiment 2). Across both games, participants preferred to interact with those who planned actions quickly or executed actions with direct and smooth movements, indicating that they were cooperating confidently and without deliberation. This demonstrates that action cues present in either the planning or implementation of economic decisions influence partner choice. We discuss implications of this finding for human decision-making and perception-action coupling in action understanding.

The pain hidden in your hands: Facial expression of pain reduces the influence of goal-related information in action recognition

The involvement of the sensorimotor system in the perception of painful actions has been repeatedly demonstrated. Yet the cognitive processes corresponding to sensorimotor activations have not been identified. In particular, the respective role of higher-level and lower-level action representations such as goals and grips in the recognition of painful actions is not clear. Previous research has shown that in a neutral context, higher-level action representations (goals) are prioritized over lower-level action representations (grips) and guide action recognition. The present study evaluates to what extent the general priority given to goal-related information in the processing of visual actions can be modulated by a context of pain. We used the action violation paradigm developed by van Elk et al. (2008). In the present action tasks, participants had to judge whether the grip or the goal of object-directed actions displayed in photographs was correct or not. The actress in the photograph could show either a neutral facial expression or a facial expression of pain. In the control task, they had to judge whether the actress expressed pain. In the action tasks, goals influenced grip judgements more than grips influenced goal judgements overall, corroborating the priority given to goal-related information previously reported. Critically, the impact of irrelevant goal-related information on the identification of incorrect grips disappeared in the pain context. Moreover, judgements in the control task were similarly influenced by grip and goal-related information. Results suggest that a context of pain reduces the reliance on higher-level action for action judgments. Findings provide novel directions regarding the cognitive and brain mechanisms involved in action processing in painful situations and support pluralist views of action understanding.

Connectivity differences between inferior frontal gyrus and mentalizing network in autism as compared to developmental coordination disorder and non-autistic youth

Prior studies have compared neural connectivity during mentalizing tasks in autism (ASD) to non-autistic individuals and found reduced connectivity between the inferior frontal gyrus (IFG) and mentalizing regions. However, given that the IFG is involved in motor processing, and about 80% of autistic individuals have motor-related difficulties, it is necessary to explore if these differences are specific to ASD or instead similar across other developmental motor disorders, such as developmental coordination disorder (DCD). Participants (29 ASD, 20 DCD, 31 typically developing [TD]; ages 8-17) completed a mentalizing task in the fMRI scanner, where they were asked to think about why someone was performing an action. Results indicated that the ASD group, as compared to both TD and DCD groups, showed significant functional connectivity differences when mentalizing about other's actions. The left IFG seed revealed ASD connectivity differences with the: bilateral temporoparietal junction (TPJ), left insular cortex, and bilateral dorsolateral prefrontal cortex (DLPFC). Connectivity differences using the right IFG seed revealed ASD differences in the: left insula, and right DLPFC. These results indicate that connectivity differences between the IFG, mentalizing regions, emotion and motor processing regions are specific to ASD and not a result of potentially co-occurring motor differences.

An active inference model of hierarchical action understanding, learning and imitation

We advance a novel active inference model of the cognitive processing that underlies the acquisition of a hierarchical action repertoire and its use for observation, understanding and imitation. We illustrate the model in four simulations of a tennis learner who observes a teacher performing tennis shots, forms hierarchical representations of the observed actions, and imitates them. Our simulations show that the agent's oculomotor activity implements an active information sampling strategy that permits inferring the kinematic aspects of the observed movement, which lie at the lowest level of the action hierarchy. In turn, this low-level kinematic inference supports higher-level inferences about deeper aspects of the observed actions: proximal goals and intentions. Finally, the inferred action representations can steer imitative responses, but interfere with the execution of different actions. Our simulations show that hierarchical active inference provides a unified account of action observation, understanding, learning and imitation and helps explain the neurobiological underpinnings of visuomotor cognition, including the multiple routes for action understanding in the dorsal and ventral streams and mirror mechanisms.

Cognitive predictors of Social processing in congenital atypical development

According to current accounts of social cognition, the emergence of verbal and non-verbal components of social perception might rely on the acquisition of different cognitive abilities. These components might be differently sensitive to the pattern of neuropsychological impairments in congenital neurodevelopmental disorders. Here, we explored the association between social and non-social cognitive domains by administering subtests of the NEPSY-II battery to 92 patients with Intellectual and Developmental Disability (IDD). Regardless the level of intellectual functioning and presence of congenital brain malformations, results revealed that visuospatial skills predicted emotion recognition and verbal component of Theory of Mind, whereas imitation predicted the non-verbal one. Future interventions might focus on spatial and sensorimotor abilities to boost the development of social cognition in IDD.

Depth and direction effects in the prediction of static and shifted reaching goals from kinematics

The kinematic parameters of reach-to-grasp movements are modulated by action intentions. However, when an unexpected change in visual target goal during reaching execution occurs, it is still unknown whether the action intention changes with target goal modification and which is the temporal structure of the target goal prediction. We recorded the kinematics of the pointing finger and wrist during the execution of reaching movements in 23 naïve volunteers where the targets could be located at different directions and depths with respect to the body. During the movement execution, the targets could remain static for the entire duration of movement or shifted, with different timings, to another position. We performed temporal decoding of the final goals and of the intermediate trajectory from the past kinematics exploiting a recurrent neural network. We observed a progressive increase of the classification performance from the onset to the end of movement in both horizontal and sagittal dimensions, as well as in decoding shifted targets. The classification accuracy in decoding horizontal targets was higher than the classification accuracy of sagittal targets. These results are useful for establishing how human and artificial agents could take advantage from the observed kinematics to optimize their cooperation in three-dimensional space.

Expectations of efficient actions bias social perception: a pre-registered online replication

Humans take a teleological stance when observing others' actions, interpreting them as intentional and goal directed. In predictive processing accounts of social perception, this teleological stance would be mediated by a perceptual prediction of an ideal energy-efficient reference trajectory with which a rational actor would achieve their goals within the current environmental constraints. Hudson and colleagues (2018 Proc. R. Soc. B 285, 20180638. (doi:10.1098/rspb.2018.0638)) tested this hypothesis in a series of experiments in which participants reported the perceived disappearance points of hands reaching for objects. They found that these judgements were biased towards the expected efficient reference trajectories. Observed straight reaches were reported higher when an obstacle needed to be overcome than if the path was clear. By contrast, unnecessarily high reaches over empty space were perceptually flattened. Moreover, these perceptual biases increased the more the environmental constraints and expected action trajectories were explicitly processed. These findings provide an important advance to our understanding of the mechanisms underlying social perception. The current replication tests the robustness of these findings and whether they uphold in an online setting.

Systems Thinking in an era of climate change: Does cognitive neuroscience hold the key to improving environmental decision making? A perspective on Climate-Smart Agriculture

Systems Thinking (ST) can be defined as a mental construct that recognises patterns and connections in a particular complex system to make the "best decision" possible. In the field of sustainable agriculture and climate change, higher degrees of ST are assumed to be associated with more successful adaptation strategies under changing conditions, and "better" environmental decision making in a number of environmental and cultural settings. Future climate change scenarios highlight the negative effects on agricultural productivity worldwide, particularly in low-income countries (LICs) situated in the Global South. Alongside this, current measures of ST are limited by their reliance on recall, and are prone to possible measurement errors. Using Climate-Smart Agriculture (CSA), as an example case study, in this article we explore: (i) ST from a social science perspective; (ii) cognitive neuroscience tools that could be used to explore ST abilities in the context of LICs; (iii) an exploration of the possible correlates of systems thinking: observational learning, prospective thinking/memory and the theory of planned behaviour and (iv) a proposed theory of change highlighting the integration of social science frameworks and a cognitive neuroscience perspective. We find, recent advancements in the field of cognitive neuroscience such as Near-Infrared Spectroscopy (NIRS) provide exciting potential to explore previously hidden forms of cognition, especially in a low-income country/field setting; improving our understanding of environmental decision-making and the ability to more accurately test more complex hypotheses where access to laboratory studies is severely limited. We highlight that ST may correlate with other key aspects involved in environmental decision-making and posit motivating farmers via specific brain networks would: (a) enhance understanding of CSA practices (e.g., via the frontoparietal network extending from the dorsolateral prefrontal cortex (DLPFC) to the parietal cortex (PC) a control hub involved in ST and observational learning) such as tailoring training towards developing improved ST abilities among farmers and involving observational learning more explicitly and (b) motivate farmers to use such practices [e.g., via the network between the DLPFC and nucleus accumbens (NAc)] which mediates reward processing and motivation by focussing on a reward/emotion to engage farmers. Finally, our proposed interdisciplinary theory of change can be used as a starting point to encourage discussion and guide future research in this space.

When the timing is right: The link between temporal coupling in dyadic interactions and emotion recognition

Affective states can be understood as dynamic interpersonal processes developing over time and space. When we observe emotional interactions performed by other individuals, our visual system anticipates how the action will unfold. Thus, it has been proposed that the process of emotion perception is not only a simulative but also a predictive process - a phenomenon described as interpersonal predictive coding. The present study investigated whether the recognition of emotions from dyadic interactions depends on a fixed spatiotemporal coupling of the agents. We used an emotion recognition task to manipulate the actions of two interacting point-light figures by implementing different temporal offsets that delayed the onset of one of the agent's actions (+0 ms, +500 ms, +1000 ms or + 2000 ms). Participants had to determine both the subjective valence and the emotion category (happiness, anger, sadness, affection) of the interaction. Results showed that temporal decoupling had a critical effect on both emotion recognition and the subjective impression of valence intensity: Both measures decreased with increasing temporal offset. However, these effects were dependent on which emotion was displayed. Whereas affection and anger sequences were impacted by the temporal manipulation, happiness and sadness were not. To further investigate these effects, we conducted post-hoc exploratory analyses of interpersonal movement parameters. Our findings complement and extend previous evidence by showing that the complex, noncoincidental coordination of actions within dyadic interactions results in a meaningful movement pattern and might serve as a fundamental factor in both detecting and understanding complex actions during human interaction.

Representing linguistic communicative functions in the premotor cortex

Linguistic communication is often regarded as an action that serves a function to convey the speaker's goal to the addressee. Here, with an functional magnetic resonance imaging (fMRI) study and a lesion study, we demonstrated that communicative functions are represented in the human premotor cortex. Participants read scripts involving 2 interlocutors. Each script contained a critical sentence said by the speaker with a communicative function of either making a Promise, a Request, or a Reply to the addressee's query. With various preceding contexts, the critical sentences were supposed to induce neural activities associated with communicative functions rather than specific actions literally described by these sentences. The fMRI results showed that the premotor cortex contained more information, as revealed by multivariate analyses, on communicative functions and relevant interlocutors' attitudes than the perisylvian language regions. The lesion study results showed that, relative to healthy controls, the understanding of communicative functions was impaired in patients with lesions in the premotor cortex, whereas no reliable difference was observed between the healthy controls and patients with lesions in other brain regions. These findings convergently suggest the crucial role of the premotor cortex in representing the functions of linguistic communications, supporting that linguistic communication can be seen as an action.

Mirror neuron brain regions contribute to identifying actions, but not intentions

Previous studies have struggled to determine the relationship between mirror neuron brain regions and two distinct “action understanding” processes: identifying actions and identifying the intentions underlying those actions. This may be because the identification of intentions from others' actions requires an initial action identification process. Disruptive transcranial magnetic stimulation was administered to left inferior frontal gyrus (lIFG) during a novel cognitive task to determine which of these “action understanding” processes is subserved by mirror neuron brain regions. Participants identified either the actions performed by observed hand actions or the intentions underlying those actions. The extent to which intention identification was disrupted by lIFG (vs. control site) stimulation was dependent on the level of disruption to action identification. We subsequently performed functional magnetic resonance imaging during the same task. During action identification, responses were widespread within mirror neuron areas including lIFG and inferior parietal lobule. However, no independent responses were found in mirror neuron brain regions during intention identification. Instead, responses occurred in brain regions associated with two distinct mentalizing localizer tasks. This supports an account in which mirror neuron brain regions are involved in an initial action identification process, but the subsequent identification of intentions requires additional processing in mentalizing brain regions.

Social Perception Deficit as a Factor of Vulnerability to Psychosis: A Brief Proposal for a Definition

Disturbances in social cognition are a core feature of schizophrenia. While most research in the field has focused on emotion perception, social knowledge, theory of mind, and attribution styles, the domain of social perception has received little specific attention. In this paper, we suggest that this issue can be explained by the lack of a precise and unitary definition of the concept, this leads to the existence of different competing uses of the concept and their conflation with other domains of social cognition. Relying on resources coming from the ecological approach to psychology and the phenomenological tradition in psychiatry, we propose that the concept of Social Perception should be used to refer to low-level pre-reflective processes underlying the awareness of interpersonal interactions with and between others. Clinical data suggests that people with schizophrenia have problems perceiving social situations as opportunities for social engagement, so, in order to fulfil this explanatory need, we propose that the term should be used to capture this important-yet neglected-domain of social cognition. We conclude with the discussion of some future directions for research derived from our proposal.

Sensorimotor anticipation of others' actions in real-world and video settings: modulation by level of engagement?

Electroencephalography (EEG) studies investigating social cognition have used both video and real-world stimuli, often without a strong reasoning why one or the other was chosen. Video stimuli can be selected for practical reasons, while naturalistic real-world stimuli are ecologically valid. The current study investigated modulatory effects on EEG mu (8 - 13 Hz) suppression, directly prior to the onset - and during the course - of observed actions, related to real-world and video settings. Recordings were made over sensorimotor cortex and stimuli in both settings consisted of identical (un)predictable object-related grasping and placing actions. In both settings a very similar mu suppression was found during unfolding of the action, irrespective of predictability. However, mu suppression related to the anticipation of upcoming predictable actions was found exclusively in the real-world setting. Thus, even though the presentation setting does not seem to modulate mu suppression during action observation, it does affect the anticipation-related mu suppression. We discuss the possibility that this may be due to increased social engagement in real-world settings, which in particular affects anticipation. The findings emphasise the importance of using real-world stimuli to bring out the subtle, anticipatory, aspects related to action observation.

Effect of Social Context on Cognitive and Motor Behavior: A Systematic Review

Human cognitive and motor behavior is influenced by the social contexts. The aim of this systematic review is to investigate the impact of the social contexts on human behaviors. A systematic search of the literature was performed via Pub-Med/Medline, Web of sciences, Google scholar, Science direct, Springer-Link and EMBASE and 68 articles were selected. After applying all the inclusion and exclusion criteria, 16 articles were retained. The results show that the presence of other people and the social context influence motor behavior (i.e. movement duration, trajectory behavior, maximum speed) and cognitive behavior (reaction time). Studies have shown an improvement in performance in the presence of other people compared to the individual situation. However, other studies showed that the presence of other people led to deterioration in performance compared to the individual situation. The improvement of behavior is attributed to the social phenomenon of facilitation while the deterioration was explained by the conduct theory or the distraction conflict theory. These social phenomena of facilitation or inhibition could be related to the perception-action theory, which interferes with interaction with other. This, in turn, seems to be associated with neural circuits of mirror neurons and motor system.

Can the early visual processing of others' actions be related to social power and dominance?

Although goals often drive action understanding, this ability is also prone to important variability among individuals, which may have its origin in individual social characteristics. The present study aimed at evaluating the relationship between the tendency to prioritize goal information over grip information during early visual processing of action and several social dimensions. Visual processing of grip and goal information during action recognition was evaluated in 64 participants using the priming protocol developed by Decroix and Kalénine (Exp Brain Res 236(8):2411-2426, 2018). Object-directed action photographs were primed by photographs sharing the same goal and/or the same grip. The effects of goal and grip priming on action recognition were evaluated for different prime durations. The same participants further fulfilled questionnaires characterizing the way individuals deal with their social environment, namely their sense of social power, dominance, perspective taking, and construal level. At the group level, results confirmed greater goal than grip priming effects on action recognition for the shortest prime duration. Regression analyses between the pattern of response times in the action priming protocol and scores at the questionnaires further showed that the advantage of goal over grip priming was associated with higher sense of social power, and possibly to lower dominance. Overall, data confirm that observers tend to prioritize goal-related information when processing visual actions but further indicate that this tendency is sensitive to individual social characteristics. Results suggest that goal information may not always drive action understanding and point out the connection between low-level processing of observed actions and more general individual characteristics.

Emotion matters: Different psychophysiological responses to expressive and non-expressive full-body movements

We explore dance video clip stimuli as a means to test human observers' accuracy in detecting genuine emotional expressivity in full-body movements. Stimuli of every-day-type full-body expressions of emotions usually use culturally very recognizable actions (e.g. fist shaking for anger, etc). However, expressive dance movement stimuli can be created to contain fully abstract movements. The expressivity results from subtle variations in the body movements of the expressor, and emotions cannot be recognised by observers via particular actions (e.g. fist shaking, etc). Forty-one participants watched and rated 24 pairs of short dance videos -from a published normalised dance stimuli library- in randomised order (N = 48). Of each carefully matched pair, one version of the full-body movement sequence had been danced to be emotionally genuinely expressive (clip a), while the other version of the same sequence (clip b) had been danced -while technically correct- without any emotional expressivity. Participants rated (i) expressivity (to test their accuracy; block 1), and (ii) how much they liked each movement (an implicit measure to test their emotional response ("liking"); block 2). Participants rated clips that were intended to be expressive as more expressive (part 1: expressivity ratings), and liked those expressive clips more than the non-expressive clips (part 2: liking ratings). Besides, their galvanic skin response differed, depending on the category of clips they were watching (expressive vs. non-expressive), and this relationship was modulated by interceptive accuracy and arts experience. Results are discussed in relation to the Body Precision Hypothesis and the Hypothesis of Constructed Emotion.

A causal role for frontal cortico-cortical coordination in social action monitoring

Decision-making via monitoring others’ actions is a cornerstone of interpersonal exchanges. Although the ventral premotor cortex (PMv) and the medial prefrontal cortex (MPFC) are cortical nodes in social brain networks, the two areas are rarely concurrently active in neuroimaging, inviting the hypothesis that they are functionally independent. Here we show in macaques that the ability of the MPFC to monitor others’ actions depends on input from the PMv. We found that delta-band coherence between the two areas emerged during action execution and action observation. Information flow especially in the delta band increased from the PMv to the MPFC as the biological nature of observed actions increased. Furthermore, selective blockade of the PMv-to-MPFC pathway using a double viral vector infection technique impaired the processing of observed, but not executed, actions. These findings demonstrate that coordinated activity in the PMv-to-MPFC pathway has a causal role in social action monitoring.

Social interactions require monitoring others’ actions to optimally organise one’s own actions. Here, the authors show that the pathway from the ventral premotor cortex (PMv) to the medial prefrontal cortex (MPFC) is causally involved in monitoring observed, but not executed, actions.

Transient Disruption of the Inferior Parietal Lobule Impairs the Ability to Attribute Intention to Action

Although it is well established that fronto-parietal regions are active during action observation, whether they play a causal role in the ability to infer others' intentions from visual kinematics remains undetermined. In the experiments reported here, we combined offline continuous theta burst stimulation (cTBS) with computational modeling to reveal and causally probe single-trial computations in the inferior parietal lobule (IPL) and inferior frontal gyrus (IFG). Participants received cTBS over the left anterior IPL and the left IFG pars orbitalis in separate sessions before completing an intention discrimination task (discriminate intention of observed reach-to-grasp acts) or a kinematic discrimination task unrelated to intention (discriminate peak wrist height of the same acts). We targeted intention-sensitive regions whose fMRI activity, recorded when observing the same reach-to-grasp acts, could accurately discriminate intention. We found that transient disruption of activity of the left IPL, but not the IFG, impaired the observer's ability to attribute intention to action. Kinematic discrimination unrelated to intention, in contrast, was largely unaffected. Computational analyses of how encoding (mapping of intention to movement kinematics) and readout (mapping of kinematics to intention choices) intersect at the single-trial level revealed that IPL cTBS did not diminish the overall sensitivity of intention readout to movement kinematics. Rather, it selectively misaligned intention readout with respect to encoding, deteriorating mapping from informative kinematic features to intention choices. These results provide causal evidence of how the left anterior IPL computes mapping from kinematics to intentions.

Affordance matching predictively shapes the perceptual representation of others' ongoing actions

Predictive processing accounts of social perception argue that action observation is a predictive process, in which inferences about others' goals are tested against the perceptual input, inducing a subtle perceptual confirmation bias that distorts observed action kinematics toward the inferred goals. Here we test whether such biases are induced even when goals are not explicitly given but have to be derived from the unfolding action kinematics. In 2 experiments, participants briefly saw an actor reach ambiguously toward a large object and a small object, with either a whole-hand power grip or an index-finger and thumb precision grip. During its course, the hand suddenly disappeared, and participants reported its last seen position on a touch-screen. As predicted, judgments were consistently biased toward apparent action targets, such that power grips were perceived closer to large objects and precision grips closer to small objects, even if the reach kinematics were identical. Strikingly, these biases were independent of participants' explicit goal judgments. They were of equal size when action goals had to be explicitly derived in each trial (Experiment 1) or not (Experiment 2) and, across trials and across participants, explicit judgments and perceptual biases were uncorrelated. This provides evidence, for the first time, that people make online adjustments of observed actions based on the match between hand grip and object goals, distorting their perceptual representation toward implied goals. These distortions may not reflect high-level goal assumptions, but emerge from relatively low-level processing of kinematic features within the perceptual system. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

The combined effects of motor and social goals on the kinematics of object-directed motor action

Voluntary actions towards manipulable objects are usually performed with a particular motor goal (i.e., a task-specific object-target-effector interaction) and in a particular social context (i.e., who would benefit from these actions), but the mutual influence of these two constraints has not yet been properly studied. For this purpose, we asked participants to grasp an object and place it on either a small or large target in relation to Fitts’ law (motor goal). This first action prepared them for a second grasp-to-place action which was performed under temporal constraints, either by the participants themselves or by a confederate (social goal). Kinematic analysis of the first preparatory grasp-to-place action showed that, while deceleration time was impacted by the motor goal, peak velocity was influenced by the social goal. Movement duration and trajectory height were modulated by both goals, the effect of the social goal being attenuated by the effect of the motor goal. Overall, these results suggest that both motor and social constraints influence the characteristics of object-oriented actions, with effects that combine in a hierarchical way.

Neural dynamics of grip and goal integration during the processing of others' actions with objects: An ERP study

Recent behavioural evidence suggests that when processing others’ actions, motor acts and goal-related information both contribute to action recognition. Yet the neuronal mechanisms underlying the dynamic integration of the two action dimensions remain unclear. This study aims to elucidate the ERP components underlying the processing and integration of grip and goal-related information. The electrophysiological activity of 28 adults was recorded during the processing of object-directed action photographs (e.g., writing with pencil) containing either grip violations (e.g. upright pencil grasped with atypical-grip), goal violations (e.g., upside-down pencil grasped with typical-grip), both grip and goal violations (e.g., upside-down pencil grasped with atypical-grip), or no violations. Participants judged whether actions were overall typical or not according to object typical use. Brain activity was sensitive to the congruency between grip and goal information on the N400, reflecting the semantic integration between the two dimensions. On earlier components, brain activity was affected by grip and goal typicality independently. Critically, goal typicality but not grip typicality affected brain activity on the N300, supporting an earlier role of goal-related representations in action recognition. Findings provide new insights on the neural temporal dynamics of the integration of motor acts and goal-related information during the processing of others’ actions.

Perception of Discrete Emotions in Others: Evidence for Distinct Facial Mimicry Patterns

Covert facial mimicry involves subtle facial muscle activation in observers when they perceive the facial emotional expressions of others. It remains uncertain whether prototypical facial features in emotional expressions are being covertly mimicked and also whether covert facial mimicry involves distinct facial muscle activation patterns across muscles per emotion category, or simply distinguishes positive versus negative valence in observed facial emotions. To test whether covert facial mimicry is emotion-specific, we measured facial electromyography (EMG) from five muscle sites (corrugator supercilii, levator labii, frontalis lateralis, depressor anguli oris, zygomaticus major) whilst participants watched videos of people expressing 9 different basic and complex emotions and a neutral expression. This study builds upon previous research by including a greater number of facial muscle measures and emotional expressions. It is the first study to investigate activation patterns across muscles during facial mimicry and to provide evidence for distinct patterns of facial muscle activation when viewing individual emotion categories, suggesting that facial mimicry is emotion-specific, rather than just valence-based.

What first drives visual attention during the recognition of object-directed actions? The role of kinematics and goal information

The recognition of others' object-directed actions is known to involve the decoding of both the visual kinematics of the action and the action goal. Yet whether action recognition is first guided by the processing of visual kinematics or by a prediction about the goal of the actor remains debated. In order to provide experimental evidence to this issue, the present study aimed at investigating whether visual attention would be preferentially captured by visual kinematics or by action goal information when processing others' actions. In a visual search task, participants were asked to find correct actions (e.g., drinking from glass) among distractor actions. Distractors actions contained grip and/or goal violations and could therefore share the correct goal and/or the correct grip with the target. The time course of fixation proportion on each distractor action has been taken as an indicator of visual attention allocation. Results show that visual attention is first captured by the distractor action with similar goal. Then the withdrawal of visual attention from the action distractor with similar goal suggests a later attentional capture by the action distractor with similar grip. Overall, results are in line with predictive approaches of action understanding, which assume that observers first make a prediction about the actor's goal before verifying this prediction using the visual kinematics of the action.

To Watch is to Work: a Review of NeuroImaging Data on Tool Use Observation Network

Since the discovery of mirror neurons in the 1990s, many neuroimaging studies have tackled the issue of action observation with the aim of unravelling a putative homolog human system. However, these studies do not distinguish between non-tool-use versus tool-use actions, implying that a common brain network is systematically involved in the observation of any action. Here we provide evidence for a brain network dedicated to tool-use action observation, called the tool-use observation network, mostly situated in the left hemisphere, and distinct from the non-tool-use action observation network. Areas specific for tool-use action observation are the left cytoarchitectonic area PF within the left inferior parietal lobe and the left inferior frontal gyrus. The neural correlates associated with the observation of tool-use reported here offer new insights into the neurocognitive bases of action observation and tool use, as well as addressing more fundamental issues on the origins of specifically human phenomena such as cumulative technological evolution.

Understanding individual differences in theory of mind via representation of minds, not mental states

The human ability to make inferences about the minds of conspecifics is remarkable. The majority of work in this area focuses on mental state representation ('theory of mind'), but has had limited success in explaining individual differences in this ability, and is characterized by the lack of a theoretical framework that can account for the effect of variability in the population of minds to which individuals are exposed. We draw analogies between faces and minds as complex social stimuli, and suggest that theoretical and empirical progress on understanding the mechanisms underlying mind representation can be achieved by adopting a 'Mind-space' framework; that minds, like faces, are represented within a multidimensional psychological space. This Mind-space framework can accommodate the representation of whole cognitive systems, and may help to explain individual differences in the consistency and accuracy with which the mental states of others are inferred. Mind-space may also have relevance for understanding human development, intergroup relations, and the atypical social cognition seen in several clinical conditions.

Suppression of Facial Mimicry of Negative Facial Expressions in an Incongruent Context

Abstract. People react with Rapid Facial Reactions (RFRs) when presented with human facial emotional expressions. Recent studies show that RFRs are not always congruent with emotional cues. The processes underlying RFRs are still being debated. In our study described herein, we manipulate the context of perception and its influence on RFRs. We use a subliminal affective priming task with emotional labels. Facial electromyography (EMG) (frontalis, corrugator, zygomaticus, and depressor) was recorded while participants observed static facial expressions (joy, fear, anger, sadness, and neutral expression) preceded/not preceded by a subliminal word (JOY, FEAR, ANGER, SADNESS, or NEUTRAL). For the negative facial expressions, when the priming word was congruent with the facial expression, participants displayed congruent RFRs (mimicry). When the priming word was incongruent, we observed a suppression of mimicry. Happiness was not affected by the priming word. RFRs thus appear to be modulated by the context and type of emotion that is presented via facial expressions.

Identifying others' informative intentions from movement kinematics

Previous research has demonstrated that people can reliably distinguish between actions with different instrumental intentions on the basis of the kinematic signatures of these actions (Cavallo, Koul, Ansuini, Capozzi, & Becchio, 2016). It has also been demonstrated that different informative intentions result in distinct action kinematics (McEllin, Knoblich, & Sebanz, 2017). However, it is unknown whether people can discriminate between instrumental actions and actions performed with an informative intention, and between actions performed with different informative intentions, on the basis of kinematic cues produced in these actions. We addressed these questions using a visual discrimination paradigm in which participants were presented with point light animations of an actor playing a virtual xylophone. We systematically manipulated and amplified kinematic parameters that have been shown to reflect different informative intentions. We found that participants reliably used both spatial and temporal cues in order to discriminate between instrumental actions and actions performed with an informative intention, and between actions performed with different informative intentions. Our findings indicate that the informative cues produced in joint action and teaching go beyond serving a general informative purpose and can be used to infer specific informative intentions.

Social Cognition for Human-Robot Symbiosis-Challenges and Building Blocks

The next generation of robot companions or robot working partners will need to satisfy social requirements somehow similar to the famous laws of robotics envisaged by Isaac Asimov time ago (Asimov, 1942). The necessary technology has almost reached the required level, including sensors and actuators, but the cognitive organization is still in its infancy and is only partially supported by the current understanding of brain cognitive processes. The brain of symbiotic robots will certainly not be a “positronic” replica of the human brain: probably, the greatest part of it will be a set of interacting computational processes running in the cloud. In this article, we review the challenges that must be met in the design of a set of interacting computational processes as building blocks of a cognitive architecture that may give symbiotic capabilities to collaborative robots of the next decades: (1) an animated body-schema; (2) an imitation machinery; (3) a motor intentions machinery; (4) a set of physical interaction mechanisms; and (5) a shared memory system for incremental symbiotic development. We would like to stress that our approach is totally un-hierarchical: the five building blocks of the shared cognitive architecture are fully bi-directionally connected. For example, imitation and intentional processes require the “services” of the animated body schema which, on the other hand, can run its simulations if appropriately prompted by imitation and/or intention, with or without physical interaction. Successful experiences can leave a trace in the shared memory system and chunks of memory fragment may compete to participate to novel cooperative actions. And so on and so forth. At the heart of the system is lifelong training and learning but, different from the conventional learning paradigms in neural networks, where learning is somehow passively imposed by an external agent, in symbiotic robots there is an element of free choice of what is worth learning, driven by the interaction between the robot and the human partner. The proposed set of building blocks is certainly a rough approximation of what is needed by symbiotic robots but we believe it is a useful starting point for building a computational framework.

The Action Representation Elicited by Different Types of Drug-Related Cues in Heroin-Abstinent Individuals

Drug related cue-induced reactivity plays a significant role in maintaining drug use and relapse in addicted individuals. The activation of Dorsolateral striatum-Sensorimotor system (DLS-SM) has been suggested as an important route through which drug cues may induce automatic drug using behavior. The current study used fMRI to investigate the reactivity of heroin abstinent individuals to different types of cues, to clarify the characteristics of the cues that induce the activation of the sensorimotor area. Forty heroin-dependent abstinent individuals and 29 healthy subjects were recruited to perform the heroin cue-reactivity task during fMRI. The participants’ subjective craving and physical signs were evaluated before and after scanning. Whole-brain analysis showed that compared to drug use tool and drug cues, cues related to drug use action were more likely to activate posterior central gyrus, para-hippocampus, supra marginal gyrus, superior parietal lobule (SPL) and inferior parietal lobule (IPL). These areas are involved in motor preparation and output, indicating that the sensorimotor area is also an important neural basis of craving and automatic drug using behavior, and may mediate craving and drug seeking behavior. Our findings thus suggest that cues related to drug using action may induce automatic drug seeking behavior more than cues related only to the drug itself.

Action Observation Areas Represent Intentions From Subtle Kinematic Features

Mirror neurons have been proposed to underlie humans' ability to understand others' actions and intentions. Despite 2 decades of research, however, the exact computational and neuronal mechanisms implied in this ability remain unclear. In the current study, we investigated whether, in the absence of contextual cues, regions considered to be part of the human mirror neuron system represent intention from movement kinematics. A total of 21 participants observed reach-to-grasp movements, performed with either the intention to drink or to pour while undergoing functional magnetic resonance imaging. Multivoxel pattern analysis revealed successful decoding of intentions from distributed patterns of activity in a network of structures comprising the inferior parietal lobule, the superior parietal lobule, the inferior frontal gyrus, and the middle frontal gyrus. Consistent with the proposal that parietal regions play a key role in intention understanding, classifier weights were higher in the inferior parietal region. These results provide the first demonstration that putative mirror neuron regions represent subtle differences in movement kinematics to read the intention of an observed motor act.

Automatic attribution of social coordination information to chasing scenes: evidence from mu suppression

This study explored whether social coordination information that extends beyond individual goals is attributed to impoverished movements produced by simple geometric shapes. We manipulated coordination information by presenting two chasers and one common target performing coordinated or individual (i.e., uncoordinated) chases, and measured mu rhythms (electroencephalogram oscillations within the 8-13 Hz range at sensorimotor regions) related to understanding social interactions. We found that although the participants' task was completely unrelated to processing chasing motion, mu rhythms were more suppressed for coordinated chasing than in the control condition (backward replay for chasing motion), and this effect disappeared for uncoordinated chasing. Moreover, mu suppression increased with higher post-test ratings of social coordination but did not correlate with uncoordinated information. Such effects cannot be explained by general attentional involvement, as there was no difference in attention-related occipital alpha suppression across conditions. These findings are consistent with interpretations of processing coordinated actions, suggesting that our visual system can automatically attribute social coordination information to motion, at least in chasing scenes.

Predictive Mechanisms Are Not Involved the Same Way during Human-Human vs. Human-Machine Interactions: A Review

Nowadays, interactions with others do not only involve human peers but also automated systems. Many studies suggest that the motor predictive systems that are engaged during action execution are also involved during joint actions with peers and during other human generated action observation. Indeed, the comparator model hypothesis suggests that the comparison between a predicted state and an estimated real state enables motor control, and by a similar functioning, understanding and anticipating observed actions. Such a mechanism allows making predictions about an ongoing action, and is essential to action regulation, especially during joint actions with peers. Interestingly, the same comparison process has been shown to be involved in the construction of an individual's sense of agency, both for self-generated and observed other human generated actions. However, the implication of such predictive mechanisms during interactions with machines is not consensual, probably due to the high heterogeneousness of the automata used in the experimentations, from very simplistic devices to full humanoid robots. The discrepancies that are observed during human/machine interactions could arise from the absence of action/observation matching abilities when interacting with traditional low-level automata. Consistently, the difficulties to build a joint agency with this kind of machines could stem from the same problem. In this context, we aim to review the studies investigating predictive mechanisms during social interactions with humans and with automated artificial systems. We will start by presenting human data that show the involvement of predictions in action control and in the sense of agency during social interactions. Thereafter, we will confront this literature with data from the robotic field. Finally, we will address the upcoming issues in the field of robotics related to automated systems aimed at acting as collaborative agents.