Causative role of left aIPS in coding shared goals during human-avatar complementary joint actions

The Bodily Appearance of a Virtual Partner Affects the Activity of the Action Observation and Action Monitoring Systems in a Minimally Interactive Task

One pending question in social neuroscience is whether interpersonal interactions are processed differently by the brain depending on the bodily characteristics of the interactor, i.e., their physical appearance. To address this issue, we engaged participants in a minimally interactive task with an avatar either showing bodily features or not while recording their brain activity using electroencephalography (EEG) in order to investigate indices of action observation and action monitoring processing. Multivariate results showed that bodily compared with nonbodily appearance modulated parieto-occipital neural patterns throughout the entire duration of the observed movement and that, importantly, such patterns differ from the ones related to initial shape processing. Furthermore, among the electrocortical indices of action monitoring, only the early observational positivity (oPe) was responsive to the bodily appearance of the observed agent under the specific task requirement to predict the partner movement. Taken together, these findings broaden the understanding of how bodily appearance shapes the spatiotemporal processing of an interactor's movements. This holds particular relevance in our modern society, where human-artificial (virtual or robotic) agent interactions are rapidly becoming ubiquitous.

Action Segmentation in the Brain: The Role of Object-Action Associations

Motion information has been argued to be central to the subjective segmentation of observed actions. Concerning object-directed actions, object-associated action information might as well inform efficient action segmentation and prediction. The present study compared the segmentation and neural processing of object manipulations and equivalent dough ball manipulations to elucidate the effect of object-action associations. Behavioral data corroborated that objective relational changes in the form of (un-)touchings of objects, hand, and ground represent meaningful anchor points in subjective action segmentation rendering them objective marks of meaningful event boundaries. As expected, segmentation behavior became even more systematic for the weakly informative dough. fMRI data were modeled by critical subjective, and computer-vision-derived objective event boundaries. Whole-brain as well as planned ROI analyses showed that object information had significant effects on how the brain processes these boundaries. This was especially pronounced at untouchings, that is, events that announced the beginning of the upcoming action and might be the point where competing predictions are aligned with perceptual input to update the current action model. As expected, weak object-action associations at untouching events were accompanied by increased biological motion processing, whereas strong object-action associations came with an increased contextual associative information processing, as indicated by increased parahippocampal activity. Interestingly, anterior inferior parietal lobule activity increased for weak object-action associations at untouching events, presumably because of an unrestricted number of candidate actions for dough manipulation. Our findings offer new insights into the significance of objects for the segmentation of action.

Differential EBV protein-specific antibody response between responders and non-responders to EBVSTs immunotherapy

Epstein-Barr virus (EBV) is associated with a diverse range of lymphomas. EBV-specific T-cell (EBVST) immunotherapies have shown promise in safety and clinical effectiveness in treating EBV-associated lymphomas, but not all patients respond to treatment. To identify the set of EBV-directed antibody responses associated with clinical response in patients with EBV-associated lymphomas, we comprehensively characterized the immune response to the complete EBV proteome using a custom protein microarray in 56 EBV-associated lymphoma patients who were treated with EBVST infusions enrolled in Phase I clinical trials. Significant differences in antibody profiles between responders and non-responders emerged at 3 months post-EBVST infusion. Twenty-five IgG antibodies were present at significantly higher levels in non-responders compared to responders at 3 months post-EBVST infusion, and 10 of these IgG antibody associations remained after adjustment for sex, age, and cancer diagnosis type. Random forest prediction analysis further confirmed that these 10 antibodies were important for predicting clinical response. Differential IgG antibody responses were directed against LMP2A (four fragments), BGRF1/BDRF1 (two fragments), LMP1, BKRF2, BKRF4, and BALF5. Paired analyses using blood samples collected at both pre-infusion and 3 months post-EBVST infusion indicated an increase in the mean antibody level for six other anti-EBV antibodies (IgG: BGLF2, LF1, BGLF3; IgA: BGLF3, BALF2, BBLF2/3) in non-responders. Overall, our results indicate that EBV-directed antibodies can be biomarkers for predicting the clinical response of individuals with EBV-associated lymphomas treated with EBVST infusions.

Neuromodulation of the Left Inferior Frontal Cortex Affects Social Monitoring during Motor Interactions

Motor interactions require observing and monitoring a partner's performance as the interaction unfolds. Studies in monkeys suggest that this form of social monitoring might be mediated by the activity of the ventral premotor cortex (vPMc), a critical brain region in action observation and motor planning. Our previous fMRI studies in humans showed that the left vPMc is indeed recruited during social monitoring, but its causal role is unexplored. In three experiments, we applied online anodal or cathodal transcranial direct current stimulation over the left lateral frontal cortex during a music-like interactive task to test the hypothesis that neuromodulation of the left vPMc affects participants' performance when a partner violates the agent's expectations. Participants played short musical sequences together with a virtual partner by playing one note each in turn-taking. In 50% of the trials, the partner violated the participant's expectations by generating the correct note through an unexpected movement. During sham stimulation, the partner's unexpected behavior led to a slowdown in the participant's performance (observation-induced posterror slowing). A significant interaction with the stimulation type showed that cathodal and anodal transcranial direct current stimulation induced modulation of the observation-induced posterror slowing in opposite directions by reducing or enhancing it, respectively. Cathodal stimulation significantly reduced the effect compared to sham stimulation. No effect of neuromodulation was found when the partner behaved as expected or when the observed violation occurred within a context that was perceptually matched but noninteractive in nature. These results provide evidence for the critical causal role that the left vPMc might play in social monitoring during motor interactions, possibly through the interplay with other brain regions in the posterior medial frontal cortex.

Visuo-motor interference is modulated by task interactivity: A kinematic study

Extensive evidence shows that action observation can influence action execution, a phenomenon often referred to as visuo-motor interference. Little is known about whether this effect can be modulated by the type of interaction agents are involved in, as different studies show conflicting results. In the present study, we aimed at shedding light on this question by recording and analyzing the kinematic unfolding of reach-to-grasp movements performed in interactive and noninteractive settings. Using a machine learning approach, we investigated whether the extent of visuo-motor interference would be enhanced or reduced in two different joint action settings compared with a noninteractive one. Our results reveal that the detrimental effect of visuo-motor interference is reduced when the action performed by the partner is relevant to achieve a common goal, regardless of whether this goal requires to produce a concrete sensory outcome in the environment (joint outcome condition) or only a joint movement configuration (joint movement condition). These findings support the idea that during joint actions we form dyadic motor plans, in which both our own and our partner's actions are represented in predictive terms and in light of the common goal to be achieved. The formation of a dyadic motor plan might allow agents to shift from the automatic simulation of an observed action to the active prediction of the consequences of a partner's action. Overall, our results demonstrate the unavoidable impact of others' action on our motor behavior in social contexts, and how strongly this effect can be modulated by task interactivity.

Modulatory Effects of Communicative Gaze on Attentional Orienting Are Driven by Dorsomedial Prefrontal Cortex but Not Right Temporoparietal Junction

Communicative gaze (e.g., mutual or averted) has been shown to affect attentional orienting. However, no study to date has clearly separated the neural basis of the pure social component that modulates attentional orienting in response to communicative gaze from other processes that might be a combination of attentional and social effects. We used TMS to isolate the purely social effects of communicative gaze on attentional orienting. Participants completed a gaze-cueing task with a humanoid robot who engaged either in mutual or in averted gaze before shifting its gaze. Before the task, participants received either sham stimulation (baseline), stimulation of right TPJ (rTPJ), or dorsomedial prefrontal cortex (dmPFC). Results showed, as expected, that communicative gaze affected attentional orienting in baseline condition. This effect was not evident for rTPJ stimulation. Interestingly, stimulation to rTPJ also canceled out attentional orienting altogether. On the other hand, dmPFC stimulation eliminated the socially driven difference in attention orienting between the two gaze conditions while maintaining the basic general attentional orienting effect. Thus, our results allowed for separation of the pure social effect of communicative gaze on attentional orienting from other processes that are a combination of social and generic attentional components.

Early categorization of social affordances during the visual encoding of bodily stimuli

Interpersonal interactions rely on various communication channels, both verbal and non-verbal, through which information regarding one's intentions and emotions are perceived. Here, we investigated the neural correlates underlying the visual processing of hand postures conveying social affordances (i.e., hand-shaking), compared to control stimuli such as hands performing non-social actions (i.e., grasping) or showing no movement at all. Combining univariate and multivariate analysis on electroencephalography (EEG) data, our results indicate that occipito-temporal electrodes show early differential processing of stimuli conveying social information compared to non-social ones. First, the amplitude of the Early Posterior Negativity (EPN, an Event-Related Potential related to the perception of body parts) is modulated differently during the perception of social and non-social content carried by hands. Moreover, our multivariate classification analysis (MultiVariate Pattern Analysis - MVPA) expanded the univariate results by revealing early (<200 ms) categorization of social affordances over occipito-parietal sites. In conclusion, we provide new evidence suggesting that the encoding of socially relevant hand gestures is categorized in the early stages of visual processing.

The dopaminergic system supports flexible and rewarding dyadic motor interactive behaviour in Parkinson's Disease

Studies indicate that the dopaminergic system (DAS) supports individual flexible behaviour. While flexibility is quintessential to effective dyadic motor interactions, whether DAS mediates adaptations of one's own motor behaviour to that of a partner is not known. Here, we asked patients with Parkinson's Disease (PD) to synchronize their grasping movements with those of a virtual partner in conditions that did (Interactive) or did not (Cued) require to predict and adapt to its actions. PD performed the task during daily antiparkinsonian treatment ('On' condition) or after drug-withdrawal ('Off' condition). A group of healthy individuals also served as control group. In the Interactive condition, PDs performed better and found the interaction more enjoyable when in 'On' than in 'Off' condition. Crucially, PD performance in the 'On' condition did not differ from that of healthy controls. This pattern of results hints at the key role of the DAS in supporting the flexible adaptation of one's own actions to the partner's during motor interactions.

The performance monitoring system is attuned to others' actions during dyadic motor interactions

Interpersonal motor interactions require the simultaneous monitoring of one's own and one's partner's actions. To characterize how the action monitoring system tracks self and other behavior during synchronous interactions, we combined electroencephalography recordings and immersive virtual reality in two tasks where participants were asked to synchronize their actions with those of a virtual partner (VP). The two tasks differed in the features to be monitored: the Goal task required participants to predict and monitor the VP's reaching goal; the Spatial task required participants to predict and monitor the VP's reaching trajectory. In both tasks, the VP performed unexpected movement changes to which the participant needed to adapt. By extracting the neural activity locked to the detection of unexpected changes in the VP's action (other-monitoring) or to the participants' action-replanning (self-monitoring), we show that the monitoring system is more attuned to others' than to one's own actions. Additionally, distinctive neural responses to VP's unexpected goals and trajectory corrections were found: goal changes were reflected both in early fronto-central and later posterior neural responses while trajectory deviations were reflected only in later posterior responses. Altogether, our results indicate that the monitoring system adopts an inherent social mode to handle interpersonal motor interactions.

Distinct neural couplings to shared goal and action coordination in joint action: evidence based on fNIRS hyperscanning

Joint action is central to human nature, enabling individuals to coordinate in time and space to achieve a joint outcome. Such interaction typically involves two key elements: shared goal and action coordination. Yet, the substrates entrained to these two components in joint action remained unclear. In the current study, dyads performed two tasks involving both sharing goal and action coordination, i.e. complementary joint action and imitative joint action, a task only involving shared goal and a task only involving action coordination, while their brain activities were recorded by the functional near-infrared spectroscopy hyperscanning technique. The results showed that both complementary and imitative joint action (i.e. involving shared goal and action coordination) elicited better behavioral performance than the task only involving shared goal/action coordination. We observed that the interbrain synchronization (IBS) at the right inferior frontal cortex (IFC) entrained more to shared goal, while left-IFC IBS entrained more to action coordination. We also observed that the right-IFC IBS was greater during completing a complementary action than an imitative action. Our results suggest that IFC plays an important role in joint action, with distinct lateralization for the sub-components of joint action.

Specific subsystems of the inferior parietal lobule are associated with hand dysfunction following stroke: A cross-sectional resting-state fMRI study

AIM

The inferior parietal lobule (IPL) plays important roles in reaching and grasping during hand movements, but how reorganizations of IPL subsystems underlie the paretic hand remains unclear. We aimed to explore whether specific IPL subsystems were disrupted and associated with hand performance after chronic stroke.

METHODS

In this cross-sectional study, we recruited 65 patients who had chronic subcortical strokes and 40 healthy controls from China. Each participant underwent the Fugl-Meyer Assessment of Hand and Wrist and resting-state fMRI at baseline. We mainly explored the group differences in resting-state effective connectivity (EC) patterns for six IPL subregions in each hemisphere, and we correlated these EC patterns with paretic hand performance across the whole stroke group and stroke subgroups. Moreover, we used receiver operating characteristic curve analysis to distinguish the stroke subgroups with partially (PPH) and completely (CPH) paretic hands.

RESULTS

Stroke patients exhibited abnormal EC patterns with ipsilesional PFt and bilateral PGa, and five sensorimotor-parietal/two parietal-temporal subsystems were positively or negatively correlated with hand performance. Compared with CPH patients, PPH patients exhibited abnormal EC patterns with the contralesional PFop. The PPH patients had one motor-parietal subsystem, while the CPH patients had one sensorimotor-parietal and three parietal-occipital subsystems that were associated with hand performance. Notably, the EC strength from the contralesional PFop to the ipsilesional superior frontal gyrus could distinguish patients with PPH from patients with CPH.

CONCLUSIONS

The IPL subsystems manifest specific functional reorganization and are associated with hand dysfunction following chronic stroke.

When action prediction grows old: An fMRI study

Predicting the unfolding of others' actions (action prediction) is crucial for successfully navigating the social world and interacting efficiently. Age-related changes in this domain have remained largely unexplored, especially for predictions regarding simple gestures and independent of contextual information or motor expertise. Here, we evaluated whether healthy aging impacts the neurophysiological processes recruited to anticipate, from the observation of implied-motion postures, the correct conclusion of simple grasping and pointing actions. A color-discrimination task served as a control condition to assess the specificity of the age-related effects. Older adults showed reduced efficiency in performance that was yet not specific to the action prediction task. Nevertheless, fMRI results revealed task-specific age-related differences: while both groups showed stronger recruitment of the lateral occipito-temporal cortex bilaterally during the action prediction than the control task, the younger participants additionally showed a higher bilateral engagement of parietal regions. Importantly, in both groups, the recruitment of visuo-motor processes in the right posterior parietal cortex was a predictor of good performance. These results support the hypothesis of decreased involvement of sensorimotor processes in cognitive tasks when processing action- and body-related stimuli in healthy aging. These results have implications for social interaction, which requires the fast reading of others' gestures.

Mirror neurons 30 years later: implications and applications

Mirror neurons (MNs) were first described in a seminal paper in 1992 as a class of monkey premotor cells discharging during both action execution and observation. Despite their debated origin and function, recent studies in several species, from birds to humans, revealed that beyond MNs properly so called, a variety of cell types distributed among multiple motor, sensory, and emotional brain areas form a 'mirror mechanism' more complex and flexible than originally thought, which has an evolutionarily conserved role in social interaction. Here, we trace the current limits and envisage the future trends of this discovery, showing that it inspired translational research and the development of new neurorehabilitation approaches, and constitutes a point of no return in social and affective neuroscience.

Human adults prefer to cooperate even when it is costly

Joint actions are cooperative activities where humans coordinate their actions to achieve individual and shared goals. While the motivation to engage in joint action is clear when a goal cannot be achieved by individuals alone, we asked whether humans are motivated to act together even when acting together is not necessary and implies incurring additional costs compared to individual goal achievement. Using a utility-based empirical approach, we investigated the extent of humans' preference for joint action over individual action, when the instrumental costs of performing joint actions outweigh the benefits. The results of five experiments showed that human adults have a stable preference for joint action, even if individual action is more effective to achieve a certain goal. We propose that such preferences can be understood as ascribing additional reward value to performing actions together.

Midfrontal Theta tACS Facilitates Motor Coordination in Dyadic Human-Avatar Interactions

Synchronous interpersonal motor interactions require moment-to-moment prediction and proactive monitoring of the partner's actions. Neurophysiologically, this is highlighted by an enhancement of midfrontal theta (4-7 Hz) oscillations. In this study, we explored the causal role of midfrontal theta for interpersonal motor interactions using transcranial alternating current stimulation (tACS). We implemented a realistic human-avatar interaction task in immersive virtual reality where participants controlled a virtual arm and hand to press a button synchronously with a virtual partner. Participants completed the task while receiving EEG-informed theta (Experiment 1) or beta (control frequency, Experiment 2) tACS over the frontal midline, as well as sham stimulation as a control. Results showed that midfrontal theta tACS significantly improved behavioral performance (i.e., reduced interpersonal asynchrony) and participants' motor strategies (i.e., increased movement times and reduced RTs), whereas beta tACS had no effect on these measures. These results suggest that theta tACS over frontal areas facilitates action monitoring and motor abilities supporting interpersonal interactions.

Taking apart what brings us together: The role of action prediction, perspective-taking, and theory of mind in joint action

The ability to act together with others to achieve common goals is crucial in life, yet there is no full consensus on the underlying cognitive skills. While influential theoretical accounts suggest that interaction requires sophisticated insights into others' minds, alternative views propose that high-level social skills might not be necessary because interactions are grounded on sensorimotor predictive mechanisms. At present, empirical evidence is insufficient to decide between the two. This study addressed this issue and explored the association between performance at joint action tasks and cognitive abilities in three domains-action prediction, perspective-taking, and theory of mind-in healthy adults (N = 58). We found that, while perspective-taking played a role in reading the behaviour of others independently of the social context, action prediction abilities specifically influenced the agents' performance in an interactive task but not in a control (social but non-interactive) task. In our study, performance at a theory of mind test did not play any role, as confirmed by Bayesian analyses. The results suggest that, in adults, sensorimotor predictive mechanisms might play a significant and specific role in supporting interpersonal coordination during motor interactions. We discuss the implications of our findings for the contrasting theoretical views described earlier and propose a way they might be partly reconciled.

Inhibitory Theta Burst Stimulation Highlights the Role of Left aIPS and Right TPJ during Complementary and Imitative Human-Avatar Interactions in Cooperative and Competitive Scenarios

Competitive and cooperative interactions are based on anticipation or synchronization with the partner's actions. Both forms of interaction may either require performing imitative or complementary movements with respect to those performed by our partner. We explored how parietal regions involved in the control of imitative behavior (temporo-parietal junction, TPJ), goal coding and visuo-motor integration (anterior intraparietal sulcus, aIPS) contribute to the execution of imitative and complementary movements during cooperative and competitive interactions. To this aim, we delivered off-line non-invasive inhibitory brain stimulation to healthy individuals' left aIPS and right TPJ before they were asked to reach and grasp an object together with a virtual partner by either performing imitative or complementary interactions. In different blocks, participants were asked to compete or cooperate with the virtual partner that varied its behavior according to cooperative or competitive contexts. Left aIPS and right TPJ inhibition impaired individuals' performance (i.e., synchrony in cooperative task and anticipation in competition) during complementary and imitative interactions, respectively, in both cooperative and competitive contexts, indicating that aIPS and TPJ inhibition affects own-other action integration and action imitation (that are different in complementary vs imitative interactions) more than action synchronization or anticipation (that are different in cooperative vs competitive contexts).

Moving in synchrony with an avatar – presenting a novel and unbiased body sway synchronization paradigm

Moving in synchrony with one another is a fundamental mechanism that maintains human social bonds. Yet, not all individuals are equally likely to coordinate their behaviors with others. The degree of interpersonal coordination is greatly influenced by pre-existing characteristics of the interacting partners, like the cultural homogeneity of a group, shared goals, and the likability of the other person. Considering that most research questions necessitate an experimental set-up without such uncontrolled biases, we created a novel, unbiased paradigm: a human-avatar body sway synchronization paradigm. Participants’ body sway was measured by a force plate while being exposed to a medio-laterally moving avatar. Forty-nine participants were tested in a social condition (motionless vs. moving avatar) and a non-social control condition (motionless vs. moving column). The results revealed that participants increased their body sway on their medio-lateral axis while the avatar was moving. The participants did not increase their body sway in the non-social control condition, indicating that the participant’s movement was not simply caused by a basal motion perception process. The current study builds a methodological fundament that can help to reduce biases due to pre-existing rapport between interaction partners and serves as a valuable experimental paradigm for future synchrony studies.

Interpersonal Motor Interactions Shape Multisensory Representations of the Peripersonal Space

This perspective review focuses on the proposal that predictive multisensory integration occurring in one's peripersonal space (PPS) supports individuals' ability to efficiently interact with others, and that integrating sensorimotor signals from the interacting partners leads to the emergence of a shared representation of the PPS. To support this proposal, we first introduce the features of body and PPS representations that are relevant for interpersonal motor interactions. Then, we highlight the role of action planning and execution on the dynamic expansion of the PPS. We continue by presenting evidence of PPS modulations after tool use and review studies suggesting that PPS expansions may be accounted for by Bayesian sensory filtering through predictive coding. In the central section, we describe how this conceptual framework can be used to explain the mechanisms through which the PPS may be modulated by the actions of our interaction partner, in order to facilitate interpersonal coordination. Last, we discuss how this proposal may support recent evidence concerning PPS rigidity in Autism Spectrum Disorder (ASD) and its possible relationship with ASD individuals' difficulties during interpersonal coordination. Future studies will need to clarify the mechanisms and neural underpinning of these dynamic, interpersonal modulations of the PPS.

Mechanisms for mutual support in motor interactions

What is the key to successful interaction? Is it sufficient to represent a common goal, or does the way our partner achieves that goal count as well? How do we react when our partner misbehaves? We used a turn-taking music-like task requiring participants to play sequences of notes together with a partner, and we investigated how people adapt to a partner's error that violates their expectations. Errors consisted of either playing a wrong note of a sequence that the agents were playing together (thus preventing the achievement of the joint goal) or playing the expected note with an unexpected action. In both cases, we found post-error slowing and inaccuracy suggesting the participants' implicit tendency to correct the partner's error and produce the action that the partner should have done. We argue that these "joint" monitoring processes depend on the motor predictions made within a (dyadic) motor plan and may represent a basic mechanism for mutual support in motor interactions.

Dissociating cognitive, behavioral and physiological stress-related responses through dorsolateral prefrontal cortex inhibition

The left dorsolateral prefrontal cortex (dlPFC) has been implicated in the regulation of stress-related cognitive processes and physiological responses and is the principal target of noninvasive brain stimulation techniques applied to psychiatric conditions. However, existing studies are mostly correlational and causal evidence on the role of this region in mediating specific psychophysiological mechanisms underpinning stress-related responses are needed to make the application of such techniques more efficient. To fill this gap, this study used inhibitory continuous theta burst stimulation (cTBS) in healthy individuals to examine the extent to which activity of the left dlPFC is associated with cognitive (subjective focus on a tracking task), behavioral (reaction times and variability), and physiological responses (heart rate and its variability and cortisol level) following induction of perseverative cognition. Compared to sham and left ventral PreMotor area stimulation (as active control area), inhibition of left dlPFC determined sustained autonomic and neuroendocrine activation and increased the subjective perception of being task-focused, while not changing the behavioral and self-reported stress-related responses. Adopting a causative approach, we describe a role of left dlPFC in inhibitory control of the physiological stress-response associated to perseverative thinking.

The engaging nature of interactive gestures

The social interactions that we experience from early infancy often involve actions that are not strictly instrumental but engage the recipient by eliciting a (complementary) response. Interactive gestures may have privileged access to our perceptual and motor systems either because of their intrinsically engaging nature or as a result of extensive social learning. We compared these two hypotheses in a series of behavioral experiments by presenting individuals with interactive gestures that call for motor responses to complement the interaction ('hand shaking', 'requesting', 'high-five') and with communicative gestures that are equally socially relevant and salient, but do not strictly require a response from the recipient ('Ok', 'Thumbs up', 'Peace'). By means of a spatial compatibility task, we measured the interfering power of these task-irrelevant stimuli on the behavioral responses of individuals asked to respond to a target. Across three experiments, our results showed that the interactive gestures impact on response selection and reduce spatial compatibility effects as compared to the communicative (non-interactive) gestures. Importantly, this effect was independent of the activation of specific social scripts that may interfere with response selection. Overall, our results show that interactive gestures have privileged access to our perceptual and motor systems, possibly because they entail an automatic preparation to respond that involuntary engages the motor system of the observers. We discuss the implications from a developmental and neurophysiological point of view.

Developmental Differences in Cortical Activation During Action Observation, Action Execution and Interpersonal Synchrony: An fNIRS Study

Interpersonal synchrony (IPS) is an important everyday behavior influencing social cognitive development; however, few studies have investigated the developmental differences and underlying neural mechanisms of IPS. functional near-infrared spectroscopy (fNIRS) is a novel neuroimaging tool that allows the study of cortical activation in the presence of natural movements. Using fNIRS, we compared cortical activation patterns between children and adults during action observation, execution, and IPS. Seventeen school-age children and 15 adults completed a reach to cleanup task while we obtained cortical activation data from bilateral inferior frontal gyrus (IFG), superior temporal sulcus (STS), and inferior parietal lobes (IPL). Children showed lower spatial and temporal accuracy during IPS compared to adults (i.e., spatial synchrony scores (Mean ± SE) in children: 2.67 ± 0.08 and adults: 2.85 ± 0.06; temporal synchrony scores (Mean ± SE) in children: 2.74 ± 0.06 and adults: 2.88 ± 0.05). For both groups, the STS regions were more activated during action observation, while the IFG and STS were more activated during action execution and IPS. The IPS condition involved more right-sided activation compared to action execution suggesting that IPS is a higher-order process involving more bilateral cortical activation. In addition, adults showed more left lateralization compared to the children during movement conditions (execution and IPS); which indicated greater inhibition of ipsilateral cortices in the adults compared to children. These findings provide a neuroimaging framework to study imitation and IPS impairments in special populations such as children with Autism Spectrum Disorder.

Midline frontal and occipito-temporal activity during error monitoring in dyadic motor interactions

Discrepancies between sensory predictions and action outcome are at the base of error coding. However, these phenomena have mainly been studied focussing on individual performance. Here, we explored EEG responses to motor prediction errors during a human-avatar interaction and show that Theta/Alpha activity of the frontal error-monitoring system works in phase with activity of the occipito-temporal node of the action observation network. Our motor interaction paradigm required healthy individuals to synchronize their reach-to-grasp movements with those of a virtual partner in conditions that did (Interactive) or did not require (Cued) movement prediction and adaptation to the partner's actions. Crucially, in 30% of the trials the virtual partner suddenly and unpredictably changed its movement trajectory thereby violating the human participant's expectation. These changes elicited error-related neuromarkers (ERN/Pe - Theta/Alpha modulations) over fronto-central electrodes during the Interactive condition. Source localization and connectivity analyses showed that the frontal Theta/Alpha activity induced by violations of the expected interactive movements was in phase with occipito-temporal Theta/Alpha activity. These results expand current knowledge about the neural correlates of on-line interpersonal motor interactions linking the frontal error-monitoring system to visual, body motion-related, responses.

The role of cortical areas hMT/V5+ and TPJ on the magnitude of representational momentum and representational gravity: a transcranial magnetic stimulation study

The perceived vanishing location of a moving target is systematically displaced forward, in the direction of motion-representational momentum-, and downward, in the direction of gravity-representational gravity. Despite a wealth of research on the factors that modulate these phenomena, little is known regarding their neurophysiological substrates. The present experiment aims to explore which role is played by cortical areas hMT/V5+, linked to the processing of visual motion, and TPJ, thought to support the functioning of an internal model of gravity, in modulating both effects. Participants were required to perform a standard spatial localization task while the activity of the right hMT/V5+ or TPJ sites was selectively disrupted with an offline continuous theta-burst stimulation (cTBS) protocol, interspersed with control blocks with no stimulation. Eye movements were recorded during all spatial localizations. Results revealed an increase in representational gravity contingent on the disruption of the activity of hMT/V5+ and, conversely, some evidence suggested a bigger representational momentum when TPJ was stimulated. Furthermore, stimulation of hMT/V5+ led to a decreased ocular overshoot and to a time-dependent downward drift of gaze location. These outcomes suggest that a reciprocal balance between perceived kinematics and anticipated dynamics might modulate these spatial localization responses, compatible with a push-pull mechanism.

How Task Interactivity Shapes Action Observation

Action observation triggers imitation, a powerful mechanism permitting interpersonal coordination. Coordination, however, also occurs when the partners’ actions are nonimitative and physically incongruent. One influential theory postulates that this is achieved via top-down modulation of imitation exerted by prefrontal regions. Here, we rather argue that coordination depends on sharing a goal with the interacting partner: this shapes action observation, overriding involuntary imitation, through the predictive activity of the left ventral premotor cortex (lvPMc). During functional magnetic resonance imaging (fMRI), participants played music in turn with a virtual partner in interactive and noninteractive conditions requiring 50% of imitative/nonimitative responses. In a full-factorial design, both perceptual features and low-level motor requirements were kept constant throughout the experiment. Behaviorally, the interactive context minimized visuomotor interference due to the involuntary imitation of physically incongruent movements. This was paralleled by modulation of neural activity in the lvPMc, which was specifically recruited during the interactive task independently of the imitative/nonimitative nature of the social exchange. This lvPMc activity reflected the predictive decoding of the partner’s actions, as revealed by multivariate pattern analysis. This demonstrates that, during interactions, we process our partners’ behavior to prospectively infer their contribution to the shared goal achievement, generating motor predictions for cooperation beyond low-level imitation.

Interactor's body shape does not affect visuo-motor interference effects during motor coordination

The biological-tuning of the Action Observation Network is highly debated. A current open question relates to whether the morphological appearance (body shape) and/or the biological motion of the observed agent triggers action simulation processes. Motor simulation of the partner's action is critical for motor interactions, where two partners coordinate their actions in space and time. It supports interpersonal alignment and facilitates online coordination. However, motor simulation also leads to visuo-motor interference effects when people are required to coordinate with complementary actions, i.e. incongruent movements as compared to the observed ones. Movement kinematics of interactive partners allows us to capture their automatic tendency to simulate and imitate the partner's complementary movements. In an online reach-to-grasp task, we investigated whether visuo-motor interference effects, visible in the kinematics of complementary movements, are modulated by the visual presence of the interactor's body shape. We asked participants to interact with 1) a humanoid agent with a human-like body shape and with real human, biological, movement kinematics, or 2) a non-humanoid agent, which did not resemble the human body-shape but moved with the same real kinematics. Through the combination of inferential and Bayesian statistics, the results show no effect of interactor's body shape on visuo-motor interference in reaching and grasping kinematics during online motor coordination. We discuss the results and propose that the kinematics of the observed movements, during motor interactions, might be the key factor for visuo-motor interference to take place independently from the morphological appearance of the partner. This is particularly relevant in a technological society that constantly asks humans to interact with artificial agents.

Emergent Synergistic Grasp-Like Behavior in a Visuomotor Joint Action Task: Evidence for Internal Forward Models as Building Blocks of Human Interactions

Central to the mechanistic understanding of the human mind is to clarify how cognitive functions arise from simpler sensory and motor functions. A longstanding assumption is that forward models used by sensorimotor control to anticipate actions also serve to incorporate other people's actions and intentions, and give rise to sensorimotor interactions between people, and even abstract forms of interactions. That is, forward models could aid core aspects of human social cognition. To test whether forward models can be used to coordinate interactions, here we measured the movements of pairs of participants in a novel joint action task. For the task they collaborated to lift an object, each of them using fingers of one hand to push against the object from opposite sides, just like a single person would use two hands to grasp the object bimanually. Perturbations of the object were applied randomly as they are known to impact grasp-specific movement components in common grasping tasks. We found that co-actors quickly learned to make grasp-like movements with grasp components that showed coordination on average based on action observation of peak deviation and velocity of their partner's trajectories. Our data suggest that co-actors adopted pre-existing bimanual grasp programs for their own body to use forward models of their partner's effectors. This is consistent with the long-held assumption that human higher-order cognitive functions may take advantage of sensorimotor forward models to plan social behavior. New and Noteworthy: Taking an approach of sensorimotor neuroscience, our work provides evidence for a long-held belief that the coordination of physical as well as abstract interactions between people originates from certain sensorimotor control processes that form mental representations of people's bodies and actions, called forward models. With a new joint action paradigm and several new analysis approaches we show that, indeed, people coordinate each other's interactions based on forward models and mutual action observation.

Inhibition of left anterior intraparietal sulcus shows that mutual adjustment marks dyadic joint-actions in humans

Creating real-life dynamic contexts to study interactive behaviors is a fundamental challenge for the social neuroscience of interpersonal relations. Real synchronic interpersonal motor interactions involve online, inter-individual mutual adaptation (the ability to adapt one’s movements to those of another in order to achieve a shared goal). In order to study the contribution of the left anterior Intra Parietal Sulcus (aIPS) (i.e. a region supporting motor functions) to mutual adaptation, here, we combined a behavioral grasping task where pairs of participants synchronized their actions when performing mutually adaptive imitative and complementary movements, with the inhibition of activity of aIPS via non-invasive brain stimulation. This approach allowed us to investigate whether aIPS supports online complementary and imitative interactions. Behavioral results showed that inhibition of aIPS selectively impairs pair performance during complementary compared to imitative interactions. Notably, this effect depended on pairs’ mutual adaptation skills and was higher for pairs composed of participants who were less capable of adapting to each other. Thus, we provide the first causative evidence for a role of the left aIPS in supporting mutually adaptive interactions and show that the inhibition of the neural resources of one individual of a pair is compensated at the dyadic level.

Visuo-motor interference with a virtual partner is equally present in cooperative and competitive interactions

Automatic imitation of observed actions is thought to be a powerful mechanism, one that may mediate the reward value of interpersonal interactions, but that could also generate visuo-motor interference when interactions involve complementary movements. Since interpersonal coordination seems to be crucial both when cooperating and competing with others, the questions arises as to whether imitation-and thus visuo-motor interference-occurs in both scenarios. To address this issue, we asked human participants to engage in high- or low-interactive (Interactive or Cued condition, respectively), cooperative or competitive, joint reach-to-grasps with a virtual partner. More specifically, interactions occurred in: (i) a Cued condition, where participants simply adapted their movement timing to synchronize with (during cooperation) or anticipate (during competition) the virtual partner's grasp; (ii) an Interactive condition requiring the same adaptation, as well as a real-time selection of their action according to the virtual character's movement. To simulate a realistic human-human interaction, the virtual character would change its movement speed in consecutive trials according to participants' behaviour. Results demonstrate that visuo-motor interference-as indexed by movement kinematics (higher maximum wrist height during complementary compared to imitative power grips)-emerge in both cooperative and competitive motor interactions only when predictions about the partner's movements are needed to perform one's own action (interactive condition). These results support the idea that simulative imitation is heavily present when individuals need to match their behaviours closely.

Come together: human-avatar on-line interactions boost joint-action performance in apraxic patients

Limb apraxia (LA) is a high-order motor disorder linked to left-hemisphere damage. It is characterized by defective execution of purposeful actions upon delayed imitation, or verbal command when the actions are performed in isolated, non-naturalistic, conditions. Whether interpersonal interactions provide social affordances that activate neural resources different from those requested by individual action execution, which may improve LA performance, is unknown. To fill this gap, we measured interaction performance, behavioral and kinematic indexes of left-brain damaged patients with/without LA in a social reach-to-grasp task involving two different degrees of spatio-temporal interactivity with an avatar. We found that LA patients' impairment in coordinating with the virtual partner was abolished in highly interactive conditions (where patients selected their actions on-line based on the behavior of the virtual partner) with respect to low interactive conditions (where actions were selected beforehand based on abstract instructions). Voxel-based-Lesion-Symptom-Mapping indicated that impairments in low-interactive conditions were underpinned by lesions of premotor, motor and insular areas, and of the basal ganglia. Our approach expands current understanding of the behavioral and neural correlates of interactive motor performance by highlighting the important role of social affordances, and provides novel, potentially important, views on rehabilitation of higher-order motor cognition disorders.

Evidence for a dyadic motor plan in joint action

What mechanisms distinguish interactive from non-interactive actions? To answer this question we tested participants while they took turns playing music with a virtual partner: in the interactive joint action condition, the participants played a melody together with their partner by grasping (C note) or pressing (G note) a cube-shaped instrument, alternating in playing one note each. In the non-interactive control condition, players’ behavior was not guided by a shared melody, so that the partner’s actions and notes were irrelevant to the participant. In both conditions, the participant’s and partner’s actions were physically congruent (e.g., grasp-grasp) or incongruent (e.g., grasp-point), and the partner’s association between actions and notes was coherent with the participant’s or reversed. Performance in the non-interactive condition was only affected by physical incongruence, whereas joint action was only affected when the partner’s action-note associations were reversed. This shows that task interactivity shapes the sensorimotor coding of others’ behaviors, and that joint action is based on active prediction of the partner’s action effects rather than on passive action imitation. We suggest that such predictions are based on Dyadic Motor Plans that represent both the agent’s and the partner’s contributions to the interaction goal, like playing a melody together.

Mixed ether-based solvents provide a long cycle life with high rate capability to graphite anodes for Na-ion batteries

Energy storage devices using sodium ions have great potential in terms of their low cost, high power, and long life. In particular, there could be a significant cost reduction, because of the abundant resource of sodium. Recently, electro-chemical sodium storage performance has been demonstrated in graphite anodes using ether-based solvents. However, the high power and long life were not compatible in the same ether solution. Here, we demonstrated using mixed ether-based solvents that a long cycle life and high rate capability can coexist in the graphite anodes for sodium-ion batteries. The cycle performance indicated that the discharge capacity retention ratio after 10 000 cycles was more than 90% and the capability of 4 A g^-1/0.2 A g^-1 (40C/2C) was more than 75%. These results could advance the development of sodium storage devices for practical use.

Transitory Inhibition of the Left Anterior Intraparietal Sulcus Impairs Joint Actions: A Continuous Theta-Burst Stimulation Study

Although temporal coordination is a hallmark of motor interactions, joint action (JA) partners do not simply synchronize; rather, they dynamically adapt to each other to achieve a joint goal. We created a novel paradigm to tease apart the processes underlying synchronization and JA and tested the causal contribution of the left anterior intraparietal sulcus (aIPS) in these behaviors. Participants had to synchronize their congruent or incongruent movements with a virtual partner in two conditions: (i) being instructed on what specific action to perform, independently from what action the partner performed (synchronization), and (ii) being instructed to adapt online to the partner's action (JA). Offline noninvasive inhibitory brain stimulation (continuous theta-burst stimulation) over the left aIPS selectively modulated interpersonal synchrony in JA by boosting synchrony during congruent interactions and impairing it during incongruent ones, while leaving performance in the synchronization condition unaffected. These results suggest that the left aIPS plays a causal role in supporting online adaptation to a partner's action goal, whereas it is not necessarily engaged in social situations where the goal of the partner is irrelevant. This indicates that, during JAs, the integration of one's own and the partner's action goal is supported by aIPS.

How preschoolers and adults represent their joint action partner's behavior

We investigated the cognitive mechanisms underlying turn-taking joint action in 42-month-old children (Experiment 1) and adults (Experiment 2) using a behavioral task of dressing a virtual bear together. We aimed to investigate how participants represent a partners' behavior, i.e., in terms of specific action kinematics or of action effects. The bear was dressed by pressing a smaller and a bigger button. In the Action-response task, instructions asked participants to respond to the partner by pressing the same or opposite button; in the Action-effect task they had to respond to the partner's action effect by dressing the bear with the lacking part of the clothing, which in some cases implied pressing the same button and in other cases implied pressing the opposite button. In 50% of the trials, the partner's association between each button and the ensuing effect (dressing the bear with t-shirt or pants) was reversed, while it never changed for participants. Both children and adults showed no effect of physical congruency of actions, but showed impaired performance in the Action-effect task if their partner achieved her effect through a different action-effect association than their own. These results suggest that, when encoding their partner's actions, agents are influenced by action-effect associations that they learnt through their own experience. While interference led to overt errors in children, it caused longer reaction times in adults, suggesting that a flexible cognitive control (that is still in development in young children) is required to take on the partner's perspective.

Cortical Activation during Action Observation, Action Execution, and Interpersonal Synchrony in Adults: A functional Near-Infrared Spectroscopy (fNIRS) Study

Introduction: Humans engage in Interpersonal Synchrony (IPS) as they synchronize their own actions with that of a social partner over time. When humans engage in imitation/IPS behaviors, multiple regions in the frontal, temporal, and parietal cortices are activated including the putative Mirror Neuron Systems (Iacoboni, 2005; Buxbaum et al., 2014). In the present study, we compared fNIRS-based cortical activation patterns across three conditions of action observation (“Watch” partner), action execution (“Do” on your own), and IPS (move “Together”).

Methods: Fifteen typically developing adults completed a reach and cleanup task with the right arm while cortical activation was examined using a 24-channel, Hitachi fNIRS system. Each adult completed 8 trials across three conditions (Watch, Do, and Together). For each fNIRS channel, we obtained oxy hemoglobin (HbO₂) and deoxy hemoglobin (HHb) profiles. Spatial registration methods were applied to localize the cortical regions underneath each channel and to define six regions of interest (ROIs), right and left supero-anterior (SA or pre/post-central gyri), infero-posterior (IP or angular/supramarginal gyri), and infero-anterior (IA or superior/middle temporal gyri) regions.

Results: In terms of task-related differences, the majority of the ROIs were more active during Do and Together compared to Watch. Only the right/ipsilateral fronto-parietal and inferior parietal cortices had greater activation during Together compared to Do.

Conclusions: The similarities in cortical activation between action execution and IPS suggest that neural control of IPS is more similar to its execution than observational aspects. To be clear, the more complex the actions performed, the more difficult the IPS behaviors. Secondly, IPS behaviors required slightly more right-sided activation (vs. execution/observation) suggesting that IPS is a higher-order process involving more bilateral activation compared to its sub-components. These findings provide a neuroimaging framework to study imitation and IPS impairments in special populations such as infants at risk for and children with ASD.

Autistic traits affect interpersonal motor coordination by modulating strategic use of role-based behavior

Background

Despite the fact that deficits in social communication and interaction are at the core of Autism Spectrum Conditions (ASC), no study has yet tested individuals on a continuum from neurotypical development to autism in an on-line, cooperative, joint action task. In our study, we aimed to assess whether the degree of autistic traits affects participants' ability to modulate their motor behavior while interacting in a Joint Grasping task and according to their given role.

Methods

Sixteen pairs of adult participants played a cooperative social interactive game in which they had to synchronize their reach-to-grasp movements. Pairs were comprised of one ASC and one neurotypical with no cognitive disability. In alternate experimental blocks, one participant knew what action to perform (instructed role) while the other had to infer it from his/her partner’s action (adaptive role). When in the adaptive condition, participants were told to respond with an action that was either opposite or similar to their partner. Participants also played a non-social control game in which they had to synchronize with a non-biological stimulus.

Results

In the social interactive task, higher degree of autistic traits predicted less ability to modulate joint action according to one’s interactive role. In the non-social task, autistic traits did not predict differences in movement preparation and planning, thus ruling out the possibility that social interactive task results were due to basic motor or executive function difficulties. Furthermore, when participants played the non-social game, the higher their autistic traits, the more they were interfered by the non-biological stimulus.

Conclusions

Our study shows for the first time that high autistic traits predict a stereotypical interaction style when individuals are required to modulate their movements in order to coordinate with their partner according to their role in a joint action task. Specifically, the infrequent emergence of role-based motor behavior modulation during on-line motor cooperation in participants with high autistic traits sheds light on the numerous difficulties ASC have in nonverbal social interactions.

Electronic supplementary material

The online version of this article (doi:10.1186/s13229-017-0141-0) contains supplementary material, which is available to authorized users.

Dynamics of Social Interaction: Kinematic Analysis of a Joint Action

Non-verbal social interaction between humans requires accurate understanding of the others’ actions. The cognitivist approach suggests that successful interaction depends on the creation of a shared representation of the task, where the pairing of perceptive and motor systems of partners allows inclusion of the other’s goal into the overarching representation. Activity of the Mirror Neurons System (MNS) is thought to be a crucial mechanism linking two individuals during a joint action through action observation. The construction of a shared representation of an interaction (i.e., joint action) depends upon sensorimotor cognitive processes that modulate the ability to adapt in time and space. We attempted to detect individuals’ behavioral/kinematic change resulting in a global amelioration of performance for both subjects when a common representation of the action is built using a repetitive joint action. We asked pairs of subjects to carry out a simple task where one puts a base in the middle of a table and the other places a parallelepiped fitting into the base, the crucial manipulation being that participants switched roles during the experiment. We aimed to show that a full comprehension of a joint action is not an automatic process. We found that, before switching the interactional role, the participant initially placing the base orientated it in a way that led to an uncomfortable action for participants placing the parallelepiped. However, after switching roles, the action’s kinematics by the participant who places the base changed in order to facilitate the action of the other. More precisely, our data shows significant modulation of the base angle in order to ease the completion of the joint action, highlighting the fact that a shared knowledge of the complete action facilitates the generation of a common representation. This evidence suggests the ability to establish an efficient shared representation of a joint action benefits from physically taking our partner’s perspective because simply observing the actions of others may not be enough.

Examining ecological validity in social interaction: problems of visual fidelity, gaze, and social potential

Social interaction is an essential part of the human experience, and much work has been done to study it. However, several common approaches to examining social interactions in psychological research may inadvertently either unnaturally constrain the observed behaviour by causing it to deviate from naturalistic performance, or introduce unwanted sources of variance. In particular, these sources are the differences between naturalistic and experimental behaviour that occur from changes in visual fidelity (quality of the observed stimuli), gaze (whether it is controlled for in the stimuli), and social potential (potential for the stimuli to provide actual interaction). We expand on these possible sources of extraneous variance and why they may be important. We review the ways in which experimenters have developed novel designs to remove these sources of extraneous variance. New experimental designs using a 'two-person' approach are argued to be one of the most effective ways to develop more ecologically valid measures of social interaction, and we suggest that future work on social interaction should use these designs wherever possible.