Prediction in joint action: what, when, and where

When "more for others, less for self" leads to co-benefits: A tri-MRI dyad-hyperscanning study

Unselfishness is admired, especially when collaborations between groups of various scales are urgently needed. However, its neural mechanisms remain elusive. In a tri-MRI dyad-hyperscanning experiment involving 26 groups, each containing 4 participants as two rotating pairs in a coordination game, we sought to achieve reciprocity, or "winning in turn by the two interacting players," as the precursor to unselfishness. Due to its critical role in social processing, the right temporal-parietal junction (rTPJ) was the seed for both time domain (connectivity) and frequency domain (i.e., coherence) analyses. For the former, negative connectivity between the rTPJ and the mentalizing network areas (e.g., the right inferior parietal lobule, rIPL) was identified, and such connectivity was further negatively correlated with the individual's final gain, supporting our task design that "rewarded" the reciprocal participants. For the latter, cerebral coherences of the rTPJs emerged between the interacting pairs (i.e., within-group interacting pairs), and the coupling between the rTPJ and the right superior temporal gyrus (rSTG) between the players who were not interacting with each other (i.e., within-group noninteracting pairs). These coherences reinforce the hypotheses that the rTPJ-rTPJ coupling tracks the collaboration processes and the rTPJ-rSTG coupling for the emergence of decontextualized shared meaning. Our results underpin two social roles (inferring others' behavior and interpreting social outcomes) subserved by the rTPJ-related network and highlight its interaction with other-self/other-concerning brain areas in reaching co-benefits among unselfish players.

Me or we? Action-outcome learning in synchronous joint action

Goal-directed behaviour requires mental representations that encode instrumental relationships between actions and their outcomes. The present study investigated how people acquire representations of joint actions where co-actors perform synchronized action contributions to produce joint outcomes in the environment. Adapting an experimental procedure to assess individual action-outcome learning, we tested whether co-acting individuals link jointly produced action outcomes to individual-level features of their own action contributions or to group-level features of their joint action instead. In a learning phase, pairs of participants produced musical chords by synchronizing individual key press responses. In a subsequent test phase, the previously produced chords were presented as imperative stimuli requiring forced-choice responses by both pair members. Stimulus-response mappings were systematically manipulated to be either compatible or incompatible with the individual and joint action-outcome mappings of the preceding learning phase. Only joint but not individual compatibility was found to modulate participants' performance in the test phase. Yet, opposite to predictions of associative accounts of action-outcome learning, jointly incompatible mappings between learning and test phase resulted in better performance. We discuss a possible explanation of this finding, proposing that pairs' group-level learning experience modulated how participants encoded ambiguous task instructions in the test phase. Our findings inform current debates about mechanistic explanations of action-outcome learning effects and provide novel evidence that joint action is supported by dedicated mental representations encoding own and others' actions on a group level.

Co-perceiving: Bringing the social into perception

Humans and other animals possess the remarkable ability to effectively navigate a shared perceptual environment by discerning which objects and spaces are perceived by others and which remain private to themselves. Traditionally, this capacity has been encapsulated under the umbrella of joint attention or joint action. In this comprehensive review, we advocate for a broader and more mechanistic understanding of this phenomenon, termed co-perception. Co-perception encompasses the sensitivity to the perceptual engagement of others and the capability to differentiate between objects perceived privately and those perceived commonly with others. It represents a distinct concept from mere simultaneous individual perception. Moreover, discerning between private and common objects doesn't necessitate intricate mind-reading abilities or mutual coordination. The act of perceiving objects as either private or common provides a comprehensive account for social scenarios where individuals simply share the same context or may even engage in competition. This conceptual framework encourages a re-examination of classical paradigms that demonstrate social influences on perception. Furthermore, it suggests that the impacts of shared experiences extend beyond affective responses, also influencing perceptual processes. This article is categorized under: Psychology > Attention Philosophy > Foundations of Cognitive Science Philosophy > Psychological Capacities.

Impaired recognition of interactive intentions in adults with autism spectrum disorder not attributable to differences in visual attention or coordination via eye contact and joint attention

Altered nonverbal communication patterns especially with regard to gaze interactions are commonly reported for persons with autism spectrum disorder (ASD). In this study we investigate and differentiate for the first time the interplay of attention allocation, the establishment of shared focus (eye contact and joint attention) and the recognition of intentions in gaze interactions in adults with ASD compared to control persons. Participants interacted via gaze with a virtual character (VC), who they believed was controlled by another person. Participants were instructed to ascertain whether their partner was trying to interact with them. In fact, the VC was fully algorithm-controlled and showed either interactive or non-interactive gaze behavior. Participants with ASD were specifically impaired in ascertaining whether their partner was trying to interact with them or not as compared to participants without ASD whereas neither the allocation of attention nor the ability to establish a shared focus were affected. Thus, perception and production of gaze cues seem preserved while the evaluation of gaze cues appeared to be impaired. An additional exploratory analysis suggests that especially the interpretation of contingencies between the interactants' actions are altered in ASD and should be investigated more closely.

Self-Benefit and Others' Benefit in Cooperative Behavior in Shared Space

OBJECTIVE

The objective is to clarify the nature of cooperative moving behavior that realizes smooth traffic with others from the viewpoint of the trade-off between self-benefit and others' benefit in the shared space.

BACKGROUND

The shared space is not constrained by formal rules or behavioral norms, and is a potentially ambiguous situation where it is not clear who has priority. Therefore, the nature of cooperative behavior in the shared space is unclear.

METHOD

An experimental task was conducted to compare cooperative and nonurgent moving behavior regarding completion time (self-benefit), the amount of interruption (others' benefit), and the amount of operation (cognitive effort).

RESULTS

First, cooperative behavior benefits others. Second, although cooperative behavior decreases self-benefit compared to the baseline without any instructions, it can obtain relatively more self-benefit than nonurgent behavior without considering self-benefit. Third, cooperative behavior requires cognitive effort.

CONCLUSION

Cooperative behavior provides benefit to both oneself and others by spending cognitive effort in not interrupting others.

APPLICATION

If the nature of the cooperative behavior can be clarified, a cooperative module can be implemented into the algorithms of various mobilities.

Social-Motor Coordination Between Peers: Joint Action Developmental Trajectories in ASD and TD

Coordinating a physical movement in time and space with social and nonsocial partners to achieve a shared goal - "joint action" (JA) - characterizes many peer-engagement situations that pose challenges for individuals with autism spectrum disorder (ASD). This cross-sectional study examined development of JA capabilities comparing ASD versus typically developing (TD) groups in early childhood, preadolescence, and adolescence while performing mirroring and complementing JA tasks with social (peer) and nonsocial (computer) partners. Results indicated better motor coordination abilities on computerized tasks than in peer dyads, with larger peer-dyad deficits shown by the ASD group. Developmental growth in JA abilities emerged, but the ASD group lagged behind same-age peers with TD. Socio-motor interventions may offer new channels to facilitate peer engagement in ASD.

The role of dorsal premotor cortex in joint action inhibition

Behavioral interpersonal coordination requires smooth negotiation of actions in time and space (joint action-JA). Inhibitory control may play a role in fine-tuning appropriate coordinative responses. To date, little research has been conducted on motor inhibition during JA and on the modulatory influence that premotor areas might exert on inhibitory control. Here, we used an interactive task in which subjects were required to reach and open a bottle using one hand. The bottle was held and stabilized by a co-actor (JA) or by a mechanical holder (vice clamp, no-JA). We recorded two TMS-based indices of inhibition (short-interval intracortical inhibition-sICI; cortical silent period-cSP) during the reaching phase of the task. These reflect fast intracortical (GABAa-mediated) and slow corticospinal (GABAb-mediated) inhibition. Offline continuous theta burst stimulation (cTBS) was used to interfere with dorsal premotor cortex (PMd), ventral premotor cortex (PMv), and control site (vertex) before the execution of the task. Our results confirm a dissociation between fast and slow inhibition during JA coordination and provide evidence that premotor areas drive only slow inhibitory mechanisms, which in turn may reflect behavioral co-adaptation between trials. Exploratory analyses further suggest that PMd, more than PMv, is the key source of modulatory drive sculpting movements, according to the socio-interactive context.

Four fundamental dimensions underlie the perception of human actions

We evaluate the actions of other individuals based upon a variety of movements that reveal critical information to guide decision making and behavioural responses. These signals convey a range of information about the actor, including their goals, intentions and internal mental states. Although progress has been made to identify cortical regions involved in action processing, the organising principles underlying our representation of actions still remains unclear. In this paper we investigated the conceptual space that underlies action perception by assessing which qualities are fundamental to the perception of human actions. We recorded 240 different actions using motion-capture and used these data to animate a volumetric avatar that performed the different actions. 230 participants then viewed these actions and rated the extent to which each action demonstrated 23 different action characteristics (e.g., avoiding-approaching, pulling-pushing, weak-powerful). We analysed these data using Exploratory Factor Analysis to examine the latent factors underlying visual action perception. The best fitting model was a four-dimensional model with oblique rotation. We named the factors: friendly-unfriendly, formidable-feeble, planned-unplanned, and abduction-adduction. The first two factors of friendliness and formidableness explained approximately 22% of the variance each, compared to planned and abduction, which explained approximately 7-8% of the variance each; as such we interpret this representation of action space as having 2 + 2 dimensions. A closer examination of the first two factors suggests a similarity to the principal factors underlying our evaluation of facial traits and emotions, whilst the last two factors of planning and abduction appear unique to actions.

Verbal working memory capacity modulates semantic and phonological prediction in spoken comprehension

Mounting evidence suggests that people may use multiple cues to predict different levels of representation (e.g., semantic, syntactic, and phonological) during language comprehension. One question that has been less investigated is the relationship between general cognitive processing and the efficiency of prediction at various linguistic levels, such as semantic and phonological levels. To address this research gap, the present study investigated how working memory capacity (WMC) modulates different kinds of prediction behavior (i.e., semantic prediction and phonological prediction) in the visual world. Chinese speakers listened to the highly predictable sentences that contained a highly predictable target word, and viewed a visual display of objects. The visual display of objects contained a target object corresponding to the predictable word, a semantic or a phonological competitor that was semantically or phonologically related to the predictable word, and an unrelated object. We conducted a Chinese version of the reading span task to measure verbal WMC and grouped participants into high- and low-span groups. Participants showed semantic and phonological prediction with comparable size in both groups during language comprehension, with earlier semantic prediction in the high-span group, and a similar time course of phonological prediction in both groups. These results suggest that verbal working memory modulates predictive processing in language comprehension.

Flexible Cultural Learning Through Action Coordination

The cultural transmission of technical know-how has proven vital to the success of our species. The broad diversity of learning contexts and social configurations, as well as the various kinds of coordinated interactions they involve, speaks to our capacity to flexibly adapt to and succeed in transmitting vital knowledge in various learning contexts. Although often recognized by ethnographers, the flexibility of cultural learning has so far received little attention in terms of cognitive mechanisms. We argue that a key feature of the flexibility of cultural learning is that both the models and learners recruit cognitive mechanisms of action coordination to modulate their behavior contingently on the behavior of their partner, generating a process of mutual adaptation supporting the successful transmission of technical skills in diverse and fluctuating learning environments. We propose that the study of cultural learning would benefit from the experimental methods, results, and insights of joint-action research and, complementarily, that the field of joint-action research could expand its scope by integrating a learning and cultural dimension. Bringing these two fields of research together promises to enrich our understanding of cultural learning, its contextual flexibility, and joint action coordination.

Joint action with human and robotic co-actors: Self-other integration is immune to the perceived humanness of the interacting partner

When performing a joint action task, we automatically represent the action and/or task constraints of the co-actor with whom we are interacting. Current models suggest that, not only physical similarity, but also abstract, conceptual features shared between self and the interacting partner play a key role in the emergence of joint action effects. Across two experiments, we investigated the influence of the perceived humanness of a robotic agent on the extent to which we integrate the action of that agent into our own action/task representation, as indexed by the Joint Simon Effect (JSE). The presence (vs. absence) of a prior verbal interaction was used to manipulate robot's perceived humanness. In Experiment 1, using a within-participant design, we had participants perform the joint Go/No-go Simon task with two different robots. Before performing the joint task, one robot engaged in a verbal interaction with the participant and the other robot did not. In Experiment 2, we employed a between-participants design to contrast these two robot conditions as well as a human partner condition. In both experiments, a significant Simon effect emerged during joint action and its amplitude was not modulated by the humanness of the interacting partner. Experiment 2 further showed that the JSE obtained in robot conditions did not differ from that measured in the human partner condition. These findings contradict current theories of joint action mechanisms according to which perceived self-other similarity is a crucial determinant of self-other integration in shared task settings.

'United we stand, divided we fall': intertwining as evidence of joint actions in pea plants

In life, it is common for almost every kind of organism to interact with one another. In the human realm, such interactions are at the basis of joint actions, when two or more agents syntonize their actions to achieve a common goal. Shared intentionality is the theoretical construct referring to the suite of abilities that enable such coordinated and collaborative interactions. While shared intentionality has become an important concept in research on social cognition, there is controversy surrounding its evolutionary origins. An aspect still unexplored but promising to bring new insights into this open debate is the study of aneural organisms. To fill this gap, here we investigate whether climbing plants can act jointly to achieve a common goal, i.e. reaching the light. We examined Pisum Sativum plants growing intertwined when there is a need to climb but a potential support is not present in the environment. Three-dimensional kinematic analysis of their movement revealed a coordinated and complementary behaviour. They tend to coordinate their movement in time and space to achieve a joint climbing. By deliberately extending the context in which a joint action takes place, we pay tribute to the complex nature of this social phenomenon. The next challenge for the field of joint action is to generate a perspective that links coordination mechanisms to an evolutionary framework across taxa.

Granger Geweke Causality reveals information exchange during physical interaction is modulated by task difficulty

The haptic sense is an important mode of communication during physical interactions, and it is known to enable humans to estimate key features of their partner's behavior. It is proposed that such estimations are based upon the exchange of information mediated by the interaction forces, resulting in role distribution and coordination between partners. In the present study, we examined whether the information exchange is functionally modified to adapt to the task, or whether it is a fixed process, leaving the adaptation to individual's behaviors. We analyzed the forces during an empirical dyadic interaction task using Granger-Geweke causality analysis, which allowed us to quantify the causal influence of each individual's forces on their partner's. The dynamics of relative phase were also examined. We observed an increase of inter-partner influence with an increase in the spatial accuracy required by the task, demonstrating an adaptation of information flow to the task. This increase of exchange with the spatial accuracy constraint was accompanied by an increase of errors and of the variability of the relative phase between forces. The influence was dominated by participants in a specific role, showing a clear role division as well as task division between the dyad partners. Moreover, the influence occurred in the [2.15-7] Hz frequency band, demonstrating its importance as a frequency band of interest during cooperation involving haptic interaction. Several interpretations are introduced, ranging from sub-division of motion control to phase-amplitude coupling.

Embodied perspective-taking enhances interpersonal synchronization: A body-swap study

Humans exhibit a strong tendency to synchronize movements with each other, with visual perspective potentially influencing interpersonal synchronization. By manipulating the visual scenes of participants engaged in a joint finger-tapping task, we examined the effects of 1st person and 2nd person visual perspectives on their coordination dynamics. We hypothesized that perceiving the partner's movements from their 1st person perspective would enhance spontaneous interpersonal synchronization, potentially mediated by the embodiment of the partner's hand. We observed significant differences in attractor dynamics across visual perspectives. Specifically, participants in 1st person coupling were unable to maintain de-coupled trajectories as effectively as in 2nd person coupling. Our findings suggest that visual perspective influences coordination dynamics in dyadic interactions, engaging error-correction mechanisms in individual brains as they integrate the partner's hand into their body representation. Our results have the potential to inform the development of applications for motor training and rehabilitation.

The pain hidden in your hands: Facial expression of pain reduces the influence of goal-related information in action recognition

The involvement of the sensorimotor system in the perception of painful actions has been repeatedly demonstrated. Yet the cognitive processes corresponding to sensorimotor activations have not been identified. In particular, the respective role of higher-level and lower-level action representations such as goals and grips in the recognition of painful actions is not clear. Previous research has shown that in a neutral context, higher-level action representations (goals) are prioritized over lower-level action representations (grips) and guide action recognition. The present study evaluates to what extent the general priority given to goal-related information in the processing of visual actions can be modulated by a context of pain. We used the action violation paradigm developed by van Elk et al. (2008). In the present action tasks, participants had to judge whether the grip or the goal of object-directed actions displayed in photographs was correct or not. The actress in the photograph could show either a neutral facial expression or a facial expression of pain. In the control task, they had to judge whether the actress expressed pain. In the action tasks, goals influenced grip judgements more than grips influenced goal judgements overall, corroborating the priority given to goal-related information previously reported. Critically, the impact of irrelevant goal-related information on the identification of incorrect grips disappeared in the pain context. Moreover, judgements in the control task were similarly influenced by grip and goal-related information. Results suggest that a context of pain reduces the reliance on higher-level action for action judgments. Findings provide novel directions regarding the cognitive and brain mechanisms involved in action processing in painful situations and support pluralist views of action understanding.

The Construct Validity of the Childhood Joint Attention Rating Scale (C-JARS) in School-Aged Autistic Children

Preliminary evidence from the Childhood Joint Attention Rating Scale (C-JARS; Mundy et al., 2017) suggests symptoms related to diminished joint attention and the spontaneous sharing of experience with others can be assessed with a parent-report measure in children and adolescents with autism. This study was designed to expand on the previous study by examining the validity of both a Social Symptom (SS) and a Prosocial (PS) scale of the C-JARS in a study of school-aged autistic children (n  = 89) with and without co-occurring intellectual disability (ID), as well as an age matched neurotypical sample (n  = 62). Results indicated that both C-JARS scales were sensitive and specific with respect to identifying the diagnostic status of the children. In addition, the PS scale was sensitive to differences in cognitive abilities (IQ) and sex differences in the autism group. These results are consistent with the hypothesis that joint attention and spontaneous sharing of experience symptoms are not only characteristic of preschool children with autism but may also constitute a developmentally continuous dimension of the social phenotype of autism that can be measured in school-aged children.

Optimizing human-robot handovers: the impact of adaptive transport methods

Humans are increasingly coming into direct physical contact with robots in the context of object handovers. The technical development of robots is progressing so that handovers can be better adapted to humans. An important criterion for successful handovers between robots and humans is the predictability of the robot for the human. The better humans can anticipate the robot's actions, the better they can adapt to them and thus achieve smoother handovers. In the context of this work, it was investigated whether a highly adaptive transport method of the object, adapted to the human hand, leads to better handovers than a non-adaptive transport method with a predefined target position. To ensure robust handovers at high repetition rates, a Franka Panda robotic arm with a gripper equipped with an Intel RealSense camera and capacitive proximity sensors in the gripper was used. To investigate the handover behavior, a study was conducted with n = 40 subjects, each performing 40 handovers in four consecutive runs. The dependent variables examined are physical handover time, early handover intervention before the robot reaches its target position, and subjects' subjective ratings. The adaptive transport method does not result in significantly higher mean physical handover times than the non-adaptive transport method. The non-adaptive transport method does not lead to a significantly earlier handover intervention in the course of the runs than the adaptive transport method. Trust in the robot and the perception of safety are rated significantly lower for the adaptive transport method than for the non-adaptive transport method. The physical handover time decreases significantly for both transport methods within the first two runs. For both transport methods, the percentage of handovers with a physical handover time between 0.1 and 0.2 s increases sharply, while the percentage of handovers with a physical handover time of >0.5 s decreases sharply. The results can be explained by theories of motor learning. From the experience of this study, an increased understanding of motor learning and adaptation in the context of human-robot interaction can be of great benefit for further technical development in robotics and for the industrial use of robots.

An Integrated Model of Collaborative Skill Acquisition: Anticipation, Control Tuning, and Role Adoption

We studied collaborative skill acquisition in a dynamic setting with the game Co-op Space Fortress. While gaining expertise, the majority of subjects became increasingly consistent in the role they adopted without being able to communicate. Moreover, they acted in anticipation of the future task state. We constructed a collaborative skill acquisition model in the cognitive architecture ACT-R that reproduced subject skill acquisition trajectory. It modeled role adoption through reinforcement learning and predictive processes through motion extrapolation and learned relevant control parameters using both a reinforcement learning procedure and a new to ACT-R supervised learning procedure. This is the first integrated cognitive model of collaborative skill acquisition and, as such, gives us valuable insights into the multiple cognitive processes that are involved in learning to collaborate.

Strategic Task Decomposition in Joint Action

The core of human cooperation is people's ability to perform joint actions. Frequently, this requires effectively decomposing a joint task into individual subtasks, for example, when jointly shopping at the market to buy food. Surprisingly, little is known about how collaborators balance the costs of establishing a joint strategy for such decompositions and its expected benefits for a joint goal. We created a new online task that required pairs of randomly matched participants to jointly collect colored items. We then systematically varied the cognitive costs and benefits of applying a color-splitting strategy. The results showed that pairs adopted a color-splitting strategy more often when necessary to lower cognitive costs. However, once the strategy was jointly adopted, it continued to be used even when the cost-benefits changed. Our results provide first insights on how people decompose joint tasks into individual components and how decomposition strategies may evolve into conventions.

Videos posted on the internet provide evidence for joint rushing in naturalistic social interactions

When people engage in rhythmic joint actions, they unintentionally increase their tempo. However, this phenomenon of joint rushing has so far been investigated only under very specific and somewhat artificial conditions. Therefore, it remains unclear whether joint rushing generalizes to other instances of rhythmic joint action. In this study our aim was to investigate whether joint rushing can also be observed in a wider range of naturalistic rhythmic social interactions. To achieve this, we retrieved videos of a wide range of rhythmic interactions from an online video-sharing platform. The data suggest that joint rushing indeed can also be observed in more naturalistic social interactions. Furthermore, we provide evidence that group size matters for how tempo unfolds in social interactions with larger groups showing a stronger tempo increase than smaller groups. Comparing the data from naturalistic interactions with data collected in a lab study further showed that unintended tempo changes in social interactions are reduced in naturalistic interactions compared to interactions in a lab context. It is an open question which factors led to this reduction. One possibility is that humans might have come up with strategies to reduce the effects of joint rushing.

Dynamical biomarkers in teams and other multiagent systems

Effective team behavior in high-performance environments such as in sport and the military requires individual team members to efficiently perceive the unfolding task events, predict the actions and action intents of the other team members, and plan and execute their own actions to simultaneously accomplish individual and collective goals. To enhance team performance through effective cooperation, it is crucial to measure the situation awareness and dynamics of each team member and how they collectively impact the team's functioning. Further, to be practically useful for real-life settings, such measures must be easily obtainable from existing sensors. This paper presents several methodologies that can be used on positional and movement acceleration data of team members to quantify and/or predict team performance, assess situation awareness, and to help identify task-relevant information to support individual decision-making. Given the limited reporting of these methods within military cohorts, these methodologies are described using examples from team sports and teams training in virtual environments, with discussion as to how they can be applied to real-world military teams.

Proactive or reactive? Neural oscillatory insight into the leader-follower dynamics of early infant-caregiver interaction

We know that infants' ability to coordinate attention with others toward the end of the first year is fundamental to language acquisition and social cognition. Yet, we understand little about the neural and cognitive mechanisms driving infant attention in shared interaction: do infants play a proactive role in creating episodes of joint attention? Recording electroencephalography (EEG) from 12-mo-old infants while they engaged in table-top play with their caregiver, we examined the communicative behaviors and neural activity preceding and following infant- vs. adult-led joint attention. Infant-led episodes of joint attention appeared largely reactive: they were not associated with increased theta power, a neural marker of endogenously driven attention, and infants did not increase their ostensive signals before the initiation. Infants were, however, sensitive to whether their initiations were responded to. When caregivers joined their attentional focus, infants showed increased alpha suppression, a pattern of neural activity associated with predictive processing. Our results suggest that at 10 to 12 mo, infants are not routinely proactive in creating joint attention episodes yet. They do, however, anticipate behavioral contingency, a potentially foundational mechanism for the emergence of intentional communication.

A Novel Mirror Neuron Inspired Decision-Making Architecture for Human–Robot Interaction

Inspired by the role of mirror neurons and the importance of predictions in joint action, a novel decision-making structure is proposed, designed and tested for both individual and dyadic action. The structure comprises models representing individual decision policies, policy integration layer(s), and a negotiation layer. The latter is introduced to prevent and resolve conflicts among individuals through internal simulation rather than via explicit agent-agent communication. As the main modelling tool, Dynamic Neural Fields (DNFs) were chosen. Data was captured from human-human experiments with a decision-making task performed by either one or two participants. The task involves choosing and picking blocks one by one from seven wooden blocks to create an alpha/numeric character on a 7-segment. The task is designed to be as generic as possible. Recorded hand and blocks movements were used for developing DNF-based models by optimising parameters using a genetic algorithm. Results show that decision policies can be modelled and integrated with acceptable accuracy for individual performances. In the dyadic experiment, using only individual models without the negotiation layer, the model failed to resolve conflicts. However, with the implementation of a negotiation layer, this problem could be overcome. The proposed decision-making structure based on DNFs is developed and tested for a simple pick-and-place task. However, the main primitive underlying action of this task, pick-and-place, is indeed part of many more complex tasks people perform in their day-to-day life. Paired with the possibility to gradually evolve the architecture by adding new policies on demand, the architecture provides a general framework for modelling decision-making in joint action tasks.

Predicting and understanding human action decisions during skillful joint-action using supervised machine learning and explainable-AI

This study investigated the utility of supervised machine learning (SML) and explainable artificial intelligence (AI) techniques for modeling and understanding human decision-making during multiagent task performance. Long short-term memory (LSTM) networks were trained to predict the target selection decisions of expert and novice players completing a multiagent herding task. The results revealed that the trained LSTM models could not only accurately predict the target selection decisions of expert and novice players but that these predictions could be made at timescales that preceded a player's conscious intent. Importantly, the models were also expertise specific, in that models trained to predict the target selection decisions of experts could not accurately predict the target selection decisions of novices (and vice versa). To understand what differentiated expert and novice target selection decisions, we employed the explainable-AI technique, SHapley Additive explanation (SHAP), to identify what informational features (variables) most influenced modelpredictions. The SHAP analysis revealed that experts were more reliant on information about target direction of heading and the location of coherders (i.e., other players) compared to novices. The implications and assumptions underlying the use of SML and explainable-AI techniques for investigating and understanding human decision-making are discussed.

Cortico-cerebellar audio-motor regions coordinate self and other in musical joint action

Joint music performance requires flexible sensorimotor coordination between self and other. Cognitive and sensory parameters of joint action-such as shared knowledge or temporal (a)synchrony-influence this coordination by shifting the balance between self-other segregation and integration. To investigate the neural bases of these parameters and their interaction during joint action, we asked pianists to play on an MR-compatible piano, in duet with a partner outside of the scanner room. Motor knowledge of the partner's musical part and the temporal compatibility of the partner's action feedback were manipulated. First, we found stronger activity and functional connectivity within cortico-cerebellar audio-motor networks when pianists had practiced their partner's part before. This indicates that they simulated and anticipated the auditory feedback of the partner by virtue of an internal model. Second, we observed stronger cerebellar activity and reduced behavioral adaptation when pianists encountered subtle asynchronies between these model-based anticipations and the perceived sensory outcome of (familiar) partner actions, indicating a shift towards self-other segregation. These combined findings demonstrate that cortico-cerebellar audio-motor networks link motor knowledge and other-produced sounds depending on cognitive and sensory factors of the joint performance, and play a crucial role in balancing self-other integration and segregation.

Modulations of one's sense of agency during human-machine interactions: A behavioural study using a full humanoid robot

Although previous investigations reported a reduced sense of agency when individuals act with traditional machines, little is known about the mechanisms underpinning interactions with human-like automata. The aim of this study was twofold: (1) to investigate the effect of the machine's physical appearance on the individuals' sense of agency and (2) to explore the cognitive mechanisms underlying the individuals' sense of agency when they are engaged in a joint task. Twenty-eight participants performed a joint Simon task together with another human or an automated artificial system as a co-agent. The physical appearance of the automated artificial system was manipulated so that participants could cooperate either with a servomotor or a full humanoid robot during the joint task. Both participants' response times and temporal estimations of action-output delays (i.e., an implicit measure of agency) were collected. Results showed that participants' sense of agency for self- and other-generated actions sharply declined during interactions with the servomotor compared with the human-human interactions. Interestingly, participants' sense of agency for self- and other-generated actions was reinforced when participants interacted with the humanoid robot compared with the servomotor. These results are discussed further.

When Corticospinal Inhibition Favors an Efficient Motor Response

Many daily activities involve responding to the actions of other people. However, the functional relationship between the motor preparation and execution phases still needs to be clarified. With the combination of different and complementary experimental techniques (i.e., motor excitability measures, reaction times, electromyography, and dyadic 3-D kinematics), we investigated the behavioral and neurophysiological signatures characterizing different stages of a motor response in contexts calling for an interactive action. Participants were requested to perform an action (i.e., stirring coffee or lifting a coffee cup) following a co-experimenter's request gesture. Another condition, in which a non-interactive gesture was used, was also included. Greater corticospinal inhibition was found when participants prepared their motor response after observing an interactive request, compared to a non-interactive gesture. This, in turn, was associated with faster and more efficient action execution in kinematic terms (i.e., a social motor priming effect). Our results provide new insights on the inhibitory and facilitatory drives guiding social motor response generation. Altogether, the integration of behavioral and neurophysiological indexes allowed us to demonstrate that a more efficient action execution followed a greater corticospinal inhibition. These indexes provide a full picture of motor activity at both planning and execution stages.

Social Context and Tool Use Can Modulate Interpersonal Comfort Space

Recent research has investigated whether the representation of space around the body, in terms of reach-action (imagining of reaching another person) and comfort-social (tolerance of the other's proximity) spaces, may reflect a shared sensorimotor basis. Some studies exploiting motor plasticity induced by tool use have not observed sensorimotor identity (i.e., the same mechanisms that underlie, based on sensory information, the representation of proximal space in terms of action possibilities, goal-directed motor actions, and anticipation of the sensorimotor consequences), whereas evidence to the contrary has also emerged. Since the data are not fully convergent, here we wondered whether or not the combination of motor plasticity induced by tool use and the processing of the role of social context might reflect a similar modulation in both spaces. To this end, we conducted a randomized control trial with three groups of participants (N = 62) in which reaching and comfort distances were measured in Pre- and Post-tool-use sessions. The tool-use sessions were conducted under different conditions: (i) in the presence of a social stimulus (determining the social context) (Tool plus Mannequin group); (ii) without any stimulus (Only Tool group); (iii) in the presence of a box (Tool plus Object group) as a control condition. Results showed an extension of comfort distance in the Post-tool session of the Tool plus Mannequin group compared with the other conditions. Conversely, the reaching distance was larger after tool use than at the Pre-tool-use session, independently of the experimental conditions. Our findings suggest that motor plasticity impacts reaching and comfort spaces to different degrees; while reaching space is markedly sensitive to motor plasticity, comfort space needs qualification of social context information.

Expectations of efficient actions bias social perception: a pre-registered online replication

Humans take a teleological stance when observing others' actions, interpreting them as intentional and goal directed. In predictive processing accounts of social perception, this teleological stance would be mediated by a perceptual prediction of an ideal energy-efficient reference trajectory with which a rational actor would achieve their goals within the current environmental constraints. Hudson and colleagues (2018 Proc. R. Soc. B 285, 20180638. (doi:10.1098/rspb.2018.0638)) tested this hypothesis in a series of experiments in which participants reported the perceived disappearance points of hands reaching for objects. They found that these judgements were biased towards the expected efficient reference trajectories. Observed straight reaches were reported higher when an obstacle needed to be overcome than if the path was clear. By contrast, unnecessarily high reaches over empty space were perceptually flattened. Moreover, these perceptual biases increased the more the environmental constraints and expected action trajectories were explicitly processed. These findings provide an important advance to our understanding of the mechanisms underlying social perception. The current replication tests the robustness of these findings and whether they uphold in an online setting.

Understanding others' distal goals from proximal communicative actions

Many social interactions require individuals to coordinate their actions and to inform each other about their goals. Often these goals concern an immediate (i.e., proximal) action, as when people give each other a brief handshake, but they sometimes also refer to a future (i.e. distal) action, as when football players perform a passing sequence. The present study investigates whether observers can derive information about such distal goals by relying on kinematic modulations of an actor's instrumental actions. In Experiment 1 participants were presented with animations of a box being moved at different velocities towards an apparent endpoint. The distal goal, however, was for the object to be moved past this endpoint, to one of two occluded target locations. Participants then selected the location which they considered the likely distal goal of the action. As predicted, participants were able to detect differences in movement velocity and, based on these differences, systematically mapped the movements to the two distal goal locations. Adding a distal goal led to more variation in the way participants mapped the observed movements onto different target locations. The results of Experiments 2 and 3 indicated that this cannot be explained by difficulties in perceptual discrimination. Rather, the increased variability likely reflects differences in interpreting the underlying connection between proximal communicative actions and distal goals. The present findings extend previous research on sensorimotor communication by demonstrating that communicative action modulations are not restricted to predicting proximal goals but can also be used to infer more distal goals.

The role of reciprocity in dynamic interpersonal coordination of physiological rhythms

A central question in social cognition research is how people coordinate their bodily rhythms, and how important reciprocity of interaction is for interpersonal coordination. Previous research has primarily focused on interpersonal action coordination, which has been shown to be facilitated by mechanisms of prediction and mutual adaption. Recent research is beginning to show that people also coordinate their physiological rhythms (i.e., respiration, heart rhythms) when they engage in natural forms of social interaction, such as conversation, choir singing, and rituals. However, the mechanisms underlying interpersonal physiological synchronization remain obscure, and could provide insight into the dynamic mechanisms that underlie continuous and regulatory, rather than instrumental, joint actions. Using real-time biofeedback, we investigated whether people synchronize their respiration rhythms by forming a joint dynamical system through reciprocity of interaction, or by producing more predictable respiration rhythms. Our results show that people are more in-phase synchronized but less phase-locked when interacting bidirectionally versus unidirectionally (online), but there is no difference in synchronization during reciprocal interaction and when adapting unidirectionally (offline) to recordings of respiration signals that emerged during the reciprocal interaction. Moreover, the strength of synchronization is driven by the predictability of the respiration rhythms that emerge in the bidirectional interaction - specifically, the slowing of breathing rhythms and stability of breathing frequencies - rather than the online mutual adaptation itself. These results suggest that coordination is facilitated by the emergence of predictable breathing patterns, rather than reciprocity itself.

Smaller preferred interpersonal distance for joint versus parallel action

During social interaction, humans prefer to keep a certain distance between themselves and other individuals. This preferred 'interpersonal distance' (IPD) is known to be sensitive to social context, and in the present study we aimed to further investigate the extent to which IPD is affected by the specific type of social interaction. In particular, we focused on the contrast between joint actions, where two or more individuals coordinate their actions in space and time to achieve a shared goal, and parallel actions, where individuals act alongside each other but individually. We predicted that joint action would be associated with a smaller preferred IPD compared to parallel action. Additionally, given that this research took place in the midst of the COVID-19 pandemic, we aimed to assess whether IPD preferences are affected by individuals' concerns about infection in general, as well as COVID-19 in particular. We predicted that higher individual concerns would be associated with greater preferred IPD. To test these hypotheses, we asked participants to imagine different social scenarios (involving either joint or parallel actions alongside a stranger) and indicate, on a visual scale, their preferred IPD. The results of two experiments (n = 211, n = 212) showed that participants preferred a shorter distance when they imagined acting jointly compared to when they imagined acting in parallel. Moreover, participants who reported higher discomfort for potential pathogen contact and who were more aware of the COVID-19 context in which the study took place preferred a larger IPD in general. Our results provide further evidence that different types of social interaction shape IPD preference. We discuss potential reasons for this phenomenon and highlight remaining questions for future research.

Gender composition of pairs influences joint action effect

Research on joint action has demonstrated that individuals are sensitive to a coactor's attentional relation to jointly attend stimuli. It has also been suggested that some features are necessary to resolve the discrimination problem (i.e., self-own and other-own actions). In the present study, we aimed to test whether the gender composition of interacting pairs modulated the joint action effect. Same- (female-female or male-male) and mixed- (female-male) gender pairs performed a joint version of flanker tasks in Experiment 1 (90 participants, 50% males), while in Experiment 2 (154 participants, 50% males) Navon tasks were performed. In Experiment 1, a higher joint flanker effect in same-gender pairs than in mixed-gender pairs, and this joint effect was similar to the classical flanker effect reported by males and females in a classical procedure of the task (70 participants, 50% males). In Experiment 2, the same-gender pairs reported a joint Navon effect, which was reversed in mixed-gender pairs. In conclusion, our findings support how the gender composition of interacting pairs plays a role in joint attentional tasks.

The role of action inhibition for behavioral control in joint action

When two individuals share a task with a common goal, coordinating one's own and the other's actions is pivotal. Inhibition of one's own actions when it is the other's turn to act is assumed to play a crucial role in this process. For instance, in the joint Simon task, two individuals share a two-choice task such that one of them responds to one stimulus type and ignores the stimulus type to which the other responds. Because stimuli can either appear on one's own or on the other's side, stimulus location can conflict with stimulus identity, thus slowing response time. It has previously been shown that such conflict leads to a reduction of the detrimental effects of conflict on immediately upcoming trials both following own responses and even more so following the other's responses. This amplified trial-to-trial adjustment following the other's responses has been assumed to reflect the inhibition of own responses on the other's trials. The present study tested this hypothesis by comparing sequential trial-to-trial adjustments following correct responses and commission errors on which the inhibition of own responses has failed. As expected, adjustments were stronger following the other's correct responses than following own correct responses. Crucially, such amplification of sequential adjustment was not observed following own commission errors on the other's trials. This shows that amplification of sequential adjustments following the other's trials depend on successful inhibition of own responses on these trials and points to a crucial role of response inhibition for behavioral control in joint action.

Inter-individual coordination in walking chimpanzees

Humans, like many other animals, live in groups and coordinate actions with others in social settings.¹ Such interpersonal coordination may emerge unconsciously and when the goal is not the coordination of movements, as when falling into the same rhythm when walking together.² Although one of our closest living relatives, the chimpanzee (Pan troglodytes), shows the ability to succeed in complex joint action tasks where coordination is the goal,³ little is known about simpler forms of joint action. Here, we examine whether chimpanzees spontaneously synchronize their actions with conspecifics while walking together. We collected data on individual walking behavior of two groups of chimpanzees under semi-natural conditions. In addition, we assessed social relationships to investigate potential effects on the strength of coordination. When walking with a conspecific, individuals walked faster than when alone. The relative phase was symmetrically distributed around 0° with the highest frequencies around 0, indicating a tendency to coordinate actions. Further, coordination was stronger when walking with a partner compared with two individuals walking independently. Although the inter-limb entrainment was more pronounced between individuals of similar age as a proxy for height, it was not affected by the kinship or bonding status of the walkers or the behaviors they engaged in immediately after the walk. We conclude that chimpanzees adapt their individual behavior to temporally coordinate actions with others, which might provide a basis for engaging in other more complex forms of joint action. This spontaneous form of inter-individual coordination, often called entrainment, is thus shared with humans.

When the timing is right: The link between temporal coupling in dyadic interactions and emotion recognition

Affective states can be understood as dynamic interpersonal processes developing over time and space. When we observe emotional interactions performed by other individuals, our visual system anticipates how the action will unfold. Thus, it has been proposed that the process of emotion perception is not only a simulative but also a predictive process - a phenomenon described as interpersonal predictive coding. The present study investigated whether the recognition of emotions from dyadic interactions depends on a fixed spatiotemporal coupling of the agents. We used an emotion recognition task to manipulate the actions of two interacting point-light figures by implementing different temporal offsets that delayed the onset of one of the agent's actions (+0 ms, +500 ms, +1000 ms or + 2000 ms). Participants had to determine both the subjective valence and the emotion category (happiness, anger, sadness, affection) of the interaction. Results showed that temporal decoupling had a critical effect on both emotion recognition and the subjective impression of valence intensity: Both measures decreased with increasing temporal offset. However, these effects were dependent on which emotion was displayed. Whereas affection and anger sequences were impacted by the temporal manipulation, happiness and sadness were not. To further investigate these effects, we conducted post-hoc exploratory analyses of interpersonal movement parameters. Our findings complement and extend previous evidence by showing that the complex, noncoincidental coordination of actions within dyadic interactions results in a meaningful movement pattern and might serve as a fundamental factor in both detecting and understanding complex actions during human interaction.

Analysis of group behavior based on sharing heterogeneous roles in a triad using a coordinated drawing task

Humans often share roles and aim to achieve a group goal based on sociality, which is the tendency to spontaneously involve oneself with others. Cognitive science, psychology, and neuroscience studies suggest that in such planned coordination, adjusting one’s own actions based on other roles is crucial for high task performance. However, the mechanisms of complex and dynamically planned coordination, such as non-verbal group behavior with three or more members, remain to be fully investigated. This study introduced a coordinated drawing task in a triad, quantitatively analyzed non-verbal group behavior based on sharing heterogeneous roles, and investigated an important role. Participant triads engaged in the task repeatedly by operating reels to change thread tensions and moving a pen connected to the three threads to draw an equilateral triangle. Then, the three roles (pulling, relaxing, and adjusting) had to be shared. The pulling and relaxing roles served to move the pen as if an operator pulled it closer to the hand and to support the pen’s movement, respectively. However, these roles alone could not draw a side considering the task specification. The adjusting role needed to change the tension flexibly and maintain an overall balance. In the experiment, we measured the pen positions and tensions, and established statistical models to fit the analyzed data. The results estimated that the action in the adjusting role was related to the improved performance of faster drawing on a side. This role may moderately intervene in the actions by the other roles and fine-tune without disturbing the pen’s smooth movement while avoiding great pen deviation. Our findings may suggest the crucial role as a facilitator that handles resiliently in non-verbal coordinated behavior of a triad, and contribute to our understanding of social interactions.

Communicative capital: a key resource for human-machine shared agency and collaborative capacity

In this work, we present a perspective on the role machine intelligence can play in supporting human abilities. In particular, we consider research in rehabilitation technologies such as prosthetic devices, as this domain requires tight coupling between human and machine. Taking an agent-based view of such devices, we propose that human-machine collaborations have a capacity to perform tasks which is a result of the combined agency of the human and the machine. We introduce communicative capital as a resource developed by a human and a machine working together in ongoing interactions. Development of this resource enables the partnership to eventually perform tasks at a capacity greater than either individual could achieve alone. We then examine the benefits and challenges of increasing the agency of prostheses by surveying literature which demonstrates that building communicative resources enables more complex, task-directed interactions. The viewpoint developed in this article extends current thinking on how best to support the functional use of increasingly complex prostheses, and establishes insight toward creating more fruitful interactions between humans and supportive, assistive, and augmentative technologies.

Follow the sound of my violin: Granger causality reflects information flow in sound

Recent research into how musicians coordinate their expressive timing, phrasing, articulation, dynamics, and other stylistic characteristics during performances has highlighted the role of predictive processes, as musicians must anticipate how their partners will play in order to be together. Several studies have used information flow techniques such as Granger causality to show that upcoming movements of a musician can be predicted from immediate past movements of fellow musicians. Although musicians must move to play their instruments, a major goal of music making is to create a joint interpretation through the sounds they produce. Yet, information flow techniques have not been applied previously to examine the role that fellow musicians' sound output plays in these predictive processes and whether this changes as they learn to play together. In the present experiment, we asked professional violinists to play along with recordings of two folk pieces, each eight times in succession, and compared the amplitude envelopes of their performances with those of the recordings using Granger causality to measure information flow and cross-correlation to measure similarity and synchronization. In line with our hypotheses, our measure of information flow was higher from the recordings to the performances than vice versa, and decreased as the violinists became more familiar with the recordings over trials. This decline in information flow is consistent with a gradual shift from relying on auditory cues to predict the recording to relying on an internally-based (learned) model built through repetition. There was also evidence that violinists became more synchronized with the recordings over trials. These results shed light on the planning and learning processes involved in the aligning of expressive intentions in group music performance and lay the groundwork for the application of Granger causality to investigate information flow through sound in more complex musical interactions.

Gaze behavior in social interactions between beach volleyball players—An exploratory approach

Previous research has indicated that social interactions and gaze behavior analyses in a group setting could be essential tools in accomplishing group objectives. However, only a few studies have examined the impact of social interactions on group dynamics in team sports and their influence on team performance. This study aimed to investigate the effects of game performance pressure on the gaze behavior within social interactions between beach volleyball players during game-like situations. Therefore, 18 expert beach volleyball players conducted a high and a low game performance pressure condition while wearing an eye tracking system. The results indicate that higher game performance pressure leads to more and longer fixation on teammates’ faces. A higher need for communication without misunderstandings could explain this adaptation. The longer and more frequent look at the face could improve the receiving of verbal and non-verbal information of the teammate’s face. Further, players showed inter-individual strategies to cope with high game performance pressure regarding their gaze behavior, for example, increasing the number of fixations and the fixation duration on the teammate’s face. Thereby, this study opens a new avenue for research on social interaction and how it is influenced in/through sport.

Interacting humans use forces in specific frequencies to exchange information by touch

Object-mediated joint action is believed to be enabled by implicit information exchange between interacting individuals using subtle haptic signals within their interaction forces. The characteristics of these haptic signals have, however, remained unclear. Here we analyzed the interaction forces during an empirical dyadic interaction task using Granger–Geweke causality analysis, which allowed us to quantify the causal influence of each individual’s forces on their partner’s. We observed that the inter-partner influence was not the same at every frequency. Specifically, in the frequency band of [2.15–7] Hz, we observed inter-partner differences of causal influence that were invariant of the movement frequencies in the task and present only when information exchange was indispensable for task performance. Moreover, the inter-partner difference in this frequency band was observed to be correlated with the task performance by the dyad. Our results suggest that forces in the [2.15–7] Hz band constitute task related information exchange between individuals during physical interactions.

Facilitation of tactile processing during action observation of goal-directed reach and grasp movements

Our perception of sensory events can be altered by action, but less is known about how our perception can be altered by action observation. For example, our ability to detect tactile stimuli is reduced when our limb is moving, and task-relevance and movement speed can alter such tactile detectability. During action observation, however, the relationship between tactile processing and such modulating factors is not known. Thus, the current study sought to explore tactile processing at a task-relevant location during the observation of reaching and grasping movements performed at different speeds. Specifically, participants observed videos of an anonymous model performing movements at a slow (i.e., peak velocity [PV]: 155 mm/second), medium (i.e., PV: 547 mm/s), or fast speed (i.e., PV: 955 mm/s). To assess tactile processing, weak electrical stimuli of different amplitudes were presented to participants' right thumbs when the observed model was at their starting position and prior to any movement, or when the observed model's limb reached its PV. When observing slow movements, normalized perceptual thresholds were significantly lower/ better than for the pre-movement stimulation time. These data suggest that the movement speed can modulate tactile processing, even when observing a movement. Further, these findings provide seminal evidence for tactile facilitation at a task-relevant location during the observation of slow reaching and grasping movements (i.e., speeds associated with tactile exploration).

Reaction time coupling in a joint stimulus-response task: A matter of functional actions or likable agents?

Shaping one owns actions by observing others’ actions is driven by the deep-rooted mechanism of perception-action coupling. It typically occurs automatically, expressed as for example the unintentional synchronization of reaction times in interactive games. Theories on perception-action coupling highlight its benefits such as the joint coordination of actions to cooperatively perform tasks properly, the learning of novel actions from others, and the bonding with likable others. However, such functional aspects and how they shape perception-action coupling have never been compared quantitatively. Here we tested a total of hundred-fifteen participants that played a stimulus-response task while, in parallel, they observed videos of agents that played the exact same task several milliseconds in advance. We compared to what degree the reaction times of actions of agents, who varied their behavior in terms of functionality and likability in preceding prisoner dilemma games and quizzes, shape the reaction times of human test participants. To manipulate functionality and likability, we varied the predictability of cooperative behavior and correctness of actions of agents, respectively, resulting in likable (cooperative), dislikable (uncooperative), functional (correct actions), and dysfunctional (incorrect actions) agents. The results of three experiments showed that the participants’ reaction times correlated most with the reaction times of agents that expressed functional behavior. However, the likability of agents had no effects on reaction time correlations. These findings suggest that, at least in the current computer task, participants are more likely to adopt the timing of actions from people that perform correct actions than from people that they like.

The Components of Interpersonal Synchrony in the Typical Population and in Autism: A Conceptual Analysis

Interpersonal synchrony – the tendency for social partners to temporally co-ordinate their behaviour when interacting – is a ubiquitous feature of social interactions. Synchronous interactions play a key role in development, and promote social bonding and a range of pro-social behavioural outcomes across the lifespan. The process of achieving and maintaining interpersonal synchrony is highly complex, with inputs required from across perceptual, temporal, motor, and socio-cognitive domains. In this conceptual analysis, we synthesise evidence from across these domains to establish the key components underpinning successful non-verbal interpersonal synchrony, how such processes interact, and factors that may moderate their operation. We also consider emerging evidence that interpersonal synchrony is reduced in autistic populations. We use our account of the components contributing to interpersonal synchrony in the typical population to identify potential points of divergence in interpersonal synchrony in autism. The relationship between interpersonal synchrony and broader aspects of social communication in autism are also considered, together with implications for future research.

Mutual beta power modulation in dyadic entrainment

Across a broad spectrum of interactions, humans exhibit a prominent tendency to synchronize their movements with one another. Traditionally, this phenomenon has been explained from the perspectives of predictive coding or dynamical systems theory. While these theories diverge with respect to whether individuals hold internal models of each other, they both assume a predictive or anticipatory mechanism enabling rhythmic interactions. However, the neural bases underpinning interpersonal synchronization are still a subject under active investigation. Here we provide evidence that the brain relies on a common oscillatory mechanism to pace self-generated rhythmic movements and to track the movements produced by a partner. By performing dual-electroencephalography recordings during a joint finger-tapping task, we identified an oscillatory component in the beta range (∼ 20 Hz), which was significantly modulated by both self-generated and other-generated movement. In conditions where the partners perceived each other, we observed periodic fluctuations of beta power as a function of the reciprocal movement cycles. Crucially, this modulation occurred both in visually and in auditorily coupled conditions, and was accompanied by recurrent periods of dyadic synchronized behavior. Our results show that periodic beta power modulations may be a critical mechanism underlying interpersonal synchronization, possibly enabling mutual predictions between coupled individuals, leading to co-regulation of timing and overt mutual adaptation. Our findings thus provide a potential bridge between influential theories attempting to explain interpersonal coordination, and a concrete connection to its neurophysiological bases.

Artificial Partners to Understand Joint Action: Representing Others to Develop Effective Coordination

In the last years, artificial partners have been proposed as tools to study joint action, as they would allow to address joint behaviors in more controlled experimental conditions. Here we present an artificial partner architecture which is capable of integrating all the available information about its human counterpart and to develop efficient and natural forms of coordination. The model uses an extended state observer which combines prior information, motor commands and sensory observations to infer the partner's ongoing actions (partner model). Over trials, these estimates are gradually incorporated into action selection. Using a joint planar task in which the partners are required to perform reaching movements while mechanically coupled, we demonstrate that the artificial partner develops an internal representation of its human counterpart, whose accuracy depends on the degree of mechanical coupling and on the reliability of the sensory information. We also show that human-artificial dyads develop coordination strategies which closely resemble those observed in human-human dyads and can be interpreted as Nash equilibria. The proposed approach may provide insights for the understanding of the mechanisms underlying human-human interaction. Further, it may inform the development of novel neuro-rehabilitative solutions and more efficient human-machine interfaces.

Understanding Design Features of Music and Language: The Choric/Dialogic Distinction

Music and spoken language share certain characteristics: both consist of sequences of acoustic elements that are combinatorically combined, and these elements partition the same continuous acoustic dimensions (frequency, formant space and duration). However, the resulting categories differ sharply: scale tones and note durations of small integer ratios appear in music, while speech uses phonemes, lexical tone, and non-isochronous durations. Why did music and language diverge into the two systems we have today, differing in these specific features? We propose a framework based on information theory and a reverse-engineering perspective, suggesting that design features of music and language are a response to their differential deployment along three different continuous dimensions. These include the familiar propositional-aesthetic ('goal') and repetitive-novel ('novelty') dimensions, and a dialogic-choric ('interactivity') dimension that is our focus here. Specifically, we hypothesize that music exhibits specializations enhancing coherent production by several individuals concurrently-the 'choric' context. In contrast, language is specialized for exchange in tightly coordinated turn-taking-'dialogic' contexts. We examine the evidence for our framework, both from humans and non-human animals, and conclude that many proposed design features of music and language follow naturally from their use in distinct dialogic and choric communicative contexts. Furthermore, the hybrid nature of intermediate systems like poetry, chant, or solo lament follows from their deployment in the less typical interactive context.