Monitoring Individual and Joint Action Outcomes in Duet Music Performance

Me or we? Action-outcome learning in synchronous joint action

Goal-directed behaviour requires mental representations that encode instrumental relationships between actions and their outcomes. The present study investigated how people acquire representations of joint actions where co-actors perform synchronized action contributions to produce joint outcomes in the environment. Adapting an experimental procedure to assess individual action-outcome learning, we tested whether co-acting individuals link jointly produced action outcomes to individual-level features of their own action contributions or to group-level features of their joint action instead. In a learning phase, pairs of participants produced musical chords by synchronizing individual key press responses. In a subsequent test phase, the previously produced chords were presented as imperative stimuli requiring forced-choice responses by both pair members. Stimulus-response mappings were systematically manipulated to be either compatible or incompatible with the individual and joint action-outcome mappings of the preceding learning phase. Only joint but not individual compatibility was found to modulate participants' performance in the test phase. Yet, opposite to predictions of associative accounts of action-outcome learning, jointly incompatible mappings between learning and test phase resulted in better performance. We discuss a possible explanation of this finding, proposing that pairs' group-level learning experience modulated how participants encoded ambiguous task instructions in the test phase. Our findings inform current debates about mechanistic explanations of action-outcome learning effects and provide novel evidence that joint action is supported by dedicated mental representations encoding own and others' actions on a group level.

A practical guide to EEG hyperscanning in joint action research: from motivation to implementation

Developments in cognitive neuroscience have led to the emergence of hyperscanning, the simultaneous measurement of brain activity from multiple people. Hyperscanning is useful for investigating social cognition, including joint action, because of its ability to capture neural processes that occur within and between people as they coordinate actions toward a shared goal. Here, we provide a practical guide for researchers considering using hyperscanning to study joint action and seeking to avoid frequently raised concerns from hyperscanning skeptics. We focus specifically on Electroencephalography (EEG) hyperscanning, which is widely available and optimally suited for capturing fine-grained temporal dynamics of action coordination. Our guidelines cover questions that are likely to arise when planning a hyperscanning project, ranging from whether hyperscanning is appropriate for answering one's research questions to considerations for study design, dependent variable selection, data analysis and visualization. By following clear guidelines that facilitate careful consideration of the theoretical implications of research design choices and other methodological decisions, joint action researchers can mitigate interpretability issues and maximize the benefits of hyperscanning paradigms.

Synchrony Influences Estimates of Cooperation in a Public-Goods Game

Benefiting from a cooperative interaction requires people to estimate how cooperatively other members of a group will act so that they can calibrate their own behavior accordingly. We investigated whether the synchrony of a group's actions influences observers' estimates of cooperation. Participants (recruited through Prolific) watched animations of actors deciding how much to donate in a public-goods game and using a mouse to drag donations to a public pot. Participants then estimated how much was in the pot in total (as an index of how cooperative they thought the group members were). Experiment 1 (N = 136 adults) manipulated the synchrony between players' decision-making time, and Experiment 2 (N = 136 adults) manipulated the synchrony between players' decision-implementing movements. For both experiments, estimates of how much was in the pot were higher for synchronous than asynchronous groups, demonstrating that the temporal dynamics of an interaction contain signals of a group's level of cooperativity.

Better Together: 14-Month-Old Infants Expect Agents to Cooperate

Humans engage in cooperative activities from early on and the breadth of human cooperation is unparalleled. Human preference for cooperation might reflect cognitive and motivational mechanisms that drive engagement in cooperative activities. Here we investigate early indices of humans' cooperative abilities and test whether 14-month-old infants expect agents to prefer cooperative over individual goal achievement. Three groups of infants saw videos of agents facing a choice between two actions that led to identical rewards but differed in the individual costs. Our results show that, in line with prior research, infants expect agents to make instrumentally rational choices and prefer the less costly of two individual action alternatives. In contrast, when one of the action alternatives is cooperative, infants expect agents to choose cooperation over individual action, even though the cooperative action demands more effort from each agent to achieve the same outcome. Finally, we do not find evidence that infants expect agents to choose the less costly alternative when both options entail cooperative action. Combined, these results indicate an ontogenetically early expectation of cooperation, and raise interesting implications and questions regarding the nature of infants' representations of cooperative actions and their utility.

Flexible Cultural Learning Through Action Coordination

The cultural transmission of technical know-how has proven vital to the success of our species. The broad diversity of learning contexts and social configurations, as well as the various kinds of coordinated interactions they involve, speaks to our capacity to flexibly adapt to and succeed in transmitting vital knowledge in various learning contexts. Although often recognized by ethnographers, the flexibility of cultural learning has so far received little attention in terms of cognitive mechanisms. We argue that a key feature of the flexibility of cultural learning is that both the models and learners recruit cognitive mechanisms of action coordination to modulate their behavior contingently on the behavior of their partner, generating a process of mutual adaptation supporting the successful transmission of technical skills in diverse and fluctuating learning environments. We propose that the study of cultural learning would benefit from the experimental methods, results, and insights of joint-action research and, complementarily, that the field of joint-action research could expand its scope by integrating a learning and cultural dimension. Bringing these two fields of research together promises to enrich our understanding of cultural learning, its contextual flexibility, and joint action coordination.

Instructing somebody else to act: motor co-representations in the instructor

Instructions enable humans to perform novel tasks quickly. This is achieved by creating and activating the instruction representation for upcoming tasks, which can then modulate ongoing task behaviour in an almost 'reflexive' manner, an effect called instruction-based reflexivity. While most research has focused on understanding how verbal instructions are represented within the 'instructed' (i.e. the person receiving instructions), here we focus on how the instructor's (i.e. the person giving instructions) behaviour is affected through instructing. In a series of three experiments and one pooled analysis, we extended the classical instruction-based reflexivity paradigm to a novel social variant in which the instructions are given by an instructor (rather than visual computer-generated instructions). We found an instruction-based reflexivity effect for the instructor, that is, the instructor's task performance was better on congruent compared to incongruent trials (i.e. Experiments 1 and 2, pooled analysis). This suggests that the instructor represents the instructions of the instructed in an action-oriented format. However, this did not depend on the specific task of the instructed (i.e. Experiment 1), nor is it exclusively social (i.e. Experiment 3).

Neuromodulation of the Left Inferior Frontal Cortex Affects Social Monitoring during Motor Interactions

Motor interactions require observing and monitoring a partner's performance as the interaction unfolds. Studies in monkeys suggest that this form of social monitoring might be mediated by the activity of the ventral premotor cortex (vPMc), a critical brain region in action observation and motor planning. Our previous fMRI studies in humans showed that the left vPMc is indeed recruited during social monitoring, but its causal role is unexplored. In three experiments, we applied online anodal or cathodal transcranial direct current stimulation over the left lateral frontal cortex during a music-like interactive task to test the hypothesis that neuromodulation of the left vPMc affects participants' performance when a partner violates the agent's expectations. Participants played short musical sequences together with a virtual partner by playing one note each in turn-taking. In 50% of the trials, the partner violated the participant's expectations by generating the correct note through an unexpected movement. During sham stimulation, the partner's unexpected behavior led to a slowdown in the participant's performance (observation-induced posterror slowing). A significant interaction with the stimulation type showed that cathodal and anodal transcranial direct current stimulation induced modulation of the observation-induced posterror slowing in opposite directions by reducing or enhancing it, respectively. Cathodal stimulation significantly reduced the effect compared to sham stimulation. No effect of neuromodulation was found when the partner behaved as expected or when the observed violation occurred within a context that was perceptually matched but noninteractive in nature. These results provide evidence for the critical causal role that the left vPMc might play in social monitoring during motor interactions, possibly through the interplay with other brain regions in the posterior medial frontal cortex.

Music as a window into real-world communication

Communication has been studied extensively in the context of speech and language. While speech is tremendously effective at transferring ideas between people, music is another communicative mode that has a unique power to bring people together and transmit a rich tapestry of emotions, through joint music-making and listening in a variety of everyday contexts. Research has begun to examine the behavioral and neural correlates of the joint action required for successful musical interactions, but it has yet to fully account for the rich, dynamic, multimodal nature of musical communication. We review the current literature in this area and propose that naturalistic musical paradigms will open up new ways to study communication more broadly.

Slip or fallacy? Effects of error severity on own and observed pitch error processing in pianists

Errors elicit a negative, mediofrontal, event-related potential (ERP), for both own errors (error-related negativity; ERN) and observed errors (here referred to as observer mediofrontal negativity; oMN). It is unclear, however, if the action-monitoring system codes action valence as an all-or-nothing phenomenon or if the system differentiates between errors of different severity. We investigated this question by recording electroencephalography (EEG) data of pianists playing themselves (Experiment 1) or watching others playing (Experiment 2). Piano pieces designed to elicit large errors were used. While active participants' ERN amplitudes differed between small and large errors, observers' oMN amplitudes did not. The different pattern in the two groups of participants was confirmed in an exploratory analysis comparing ERN and oMN directly. We suspect that both prediction and action mismatches can be coded in action monitoring systems, depending on the task, and a need-to-adapt signal is sent whenever mismatches happen to indicate the magnitude of the needed adaptation.

A systematic review of handover actions in human dyads

Introduction

Handover actions are joint actions in which an object is passed from one actor to another. In order to carry out a smooth handover action, precise coordination of both actors' movements is of critical importance. This requires the synchronization of both the kinematics of the reaching movement and the grip forces of the two actors during the interaction. Psychologists, for example, may be interested in studying handover actions in order to identify the cognitive mechanisms underlying the interaction of two partners. In addition, robotic engineers may utilize insights from sensorimotor information processing in human handover as models for the design controllers in robots in hybrid (human-robot) interaction scenarios. To date, there is little knowledge transfer between researchers in different disciplines and no common framework or language for the study of handover actions.

Methods

For this reason, we systematically reviewed the literature on human-human handover actions in which at least one of the two types of behavioral data, kinematics or grip force, was measured.

Results

Nine relevant studies were identified. The different methodologies and results of the individual studies are here described and contextualized.

Discussion

Based on these results, a common framework is suggested that, provides a distinct and straightforward language and systematics for use in future studies. We suggest to term the actors as giver and receiver, as well as to subdivide the whole action into four phases: (1) Reach and grasp, (2) object transport, (3) object transfer, and (4) end of handover to comprehensively and clearly describe the handover action. The framework aims to foster the necessary exchange between different scientific disciplines to promote research on handover actions. Overall, the results support the assumption that givers adapt their executions according to the receiver's intentions, that the start of the release of the object is processed feedforward and that the release process is feedback-controlled in the transfer phase. We identified the action planning of the receiver as a research gap.

Not just in sync: Relations between partners' actions influence the sense of joint agency during joint action

When people perform joint actions together, they often experience a sense of joint agency ("we did that together"). The current study investigated whether relations between partners' actions within joint actions that require precise interpersonal synchrony influence joint agency, above and beyond the degree of synchrony partners achieve. We employed a mixed-methods approach that combined a quantitative experiment with a qualitative analysis of post-experiment interviews. Partners produced synchronized tone sequences that comprised either constant pitch sequences (simple temporal alignment between partners' actions) or musical duets (complex metrical and harmonic relations between partners' actions). Participants reported stronger joint agency for duets than constant pitches, when comparing trials with equally good synchronization. Post-experiment interviews revealed that joint agency was also influenced by participants' knowledge of the music and their perceptions of task performance, difficulty, and enjoyability. These findings further our understanding of joint agency for joint actions that require precise interpersonal synchrony.

Alpha oscillations related to self-other integration and distinction during live orchestral performance: A naturalistic case study

Ensemble music performance requires musicians to achieve precise interpersonal coordination while maintaining autonomous control over their own actions. To do so, musicians dynamically shift between integrating other performers' actions into their own action plans and maintaining a distinction between their own and others' actions. Research in laboratory settings has shown that this dynamic process of self-other integration and distinction is indexed by sensorimotor alpha oscillations. The purpose of the current descriptive case study was to examine oscillations related to self-other integration and distinction in a naturalistic performance context. We measured alpha activity from four violinists during a concert hall performance of a 60-musician orchestra. We selected a musical piece from the orchestra's repertoire and, before analyzing alpha activity, performed a score analysis to divide the piece into sections that were expected to strongly promote self-other integration and distinction. In line with previous laboratory findings, performers showed suppressed and enhanced alpha activity during musical sections that promoted self-other integration and distinction, respectively. The current study thus provides preliminary evidence that findings from carefully controlled laboratory experiments generalize to complex real-world performance. Its findings also suggest directions for future research and potential applications of interest to musicians, music educators, and music therapists.

The performance monitoring system is attuned to others' actions during dyadic motor interactions

Interpersonal motor interactions require the simultaneous monitoring of one's own and one's partner's actions. To characterize how the action monitoring system tracks self and other behavior during synchronous interactions, we combined electroencephalography recordings and immersive virtual reality in two tasks where participants were asked to synchronize their actions with those of a virtual partner (VP). The two tasks differed in the features to be monitored: the Goal task required participants to predict and monitor the VP's reaching goal; the Spatial task required participants to predict and monitor the VP's reaching trajectory. In both tasks, the VP performed unexpected movement changes to which the participant needed to adapt. By extracting the neural activity locked to the detection of unexpected changes in the VP's action (other-monitoring) or to the participants' action-replanning (self-monitoring), we show that the monitoring system is more attuned to others' than to one's own actions. Additionally, distinctive neural responses to VP's unexpected goals and trajectory corrections were found: goal changes were reflected both in early fronto-central and later posterior neural responses while trajectory deviations were reflected only in later posterior responses. Altogether, our results indicate that the monitoring system adopts an inherent social mode to handle interpersonal motor interactions.

Effects of subjective and objective task difficulties for feedback- related brain potentials in social situations: An electroencephalogram study

In this study, the relationship between two types of feedback task difficulties and feedback-related brain potentials, such as feedback-related negativity (FRN), reward positivity (RewP), and P300, was investigated in social situations where participants performed a task simultaneously by a pair. The electroencephalogram activity was measured while participants answered four-choice questions with their partners. Participants were informed about the general accuracy rate of the question (objective task difficulty) before responding to the questionnaire. The feedback outcome was definitely correct when the participants had the knowledge to answer the questions correctly. Therefore, the subjective task difficulty depended on the knowledge of the participant and differed from the objective task difficulty. In the task, the participants selected the choice they deemed correct. Before checking the answers, participants responded to the preceding question's subjective task difficulty. As one of the social factors, the task consisted of two types of conditions: one, in which one's response affected partner's reward, and another, in which it did not. The second social factor was the order of feedback outcomes; in our experiment, these outcomes were presented sequentially to pairs of participants. The effects of subjective and objective task difficulties and social factors on feedback-related brain potentials were comprehensively analyzed. The study showed that subjective task difficulty sensitively modulated the amplitude of gain-related P300, suggesting that it is sensitive to modulation in the allocation of attentional resources to own feedback outcome. The objective task difficulty sensitively modulated the amplitude of RewP after receiving the partner's incorrect feedback outcome. RewP was more sensitive to positive affective valence, such as feelings of superiority over the partner, than to task-dependent rewards received by the participants themselves. In contrast, FRN was more negative in the joint condition than in the individual condition, suggesting sensitivity to social responsibility felt by participants toward their partners.

Distinct neural couplings to shared goal and action coordination in joint action: evidence based on fNIRS hyperscanning

Joint action is central to human nature, enabling individuals to coordinate in time and space to achieve a joint outcome. Such interaction typically involves two key elements: shared goal and action coordination. Yet, the substrates entrained to these two components in joint action remained unclear. In the current study, dyads performed two tasks involving both sharing goal and action coordination, i.e. complementary joint action and imitative joint action, a task only involving shared goal and a task only involving action coordination, while their brain activities were recorded by the functional near-infrared spectroscopy hyperscanning technique. The results showed that both complementary and imitative joint action (i.e. involving shared goal and action coordination) elicited better behavioral performance than the task only involving shared goal/action coordination. We observed that the interbrain synchronization (IBS) at the right inferior frontal cortex (IFC) entrained more to shared goal, while left-IFC IBS entrained more to action coordination. We also observed that the right-IFC IBS was greater during completing a complementary action than an imitative action. Our results suggest that IFC plays an important role in joint action, with distinct lateralization for the sub-components of joint action.

What makes us act together? On the cognitive models supporting humans’ decisions for joint action

We face tasks every day that we can solve alone but decide to solve together with others. When do we choose to act together vs. alone? How long do we persist in working together when doing so is difficult? Do we prefer to act together when times are uncertain? An open question in joint action research is under what conditions humans prefer to act together or alone to achieve a certain goal, and whether their preference is based on a utility calculus that takes into account the costs and benefits associated with individual and joint action alternatives. Research on cooperation reveals that frequent engagement in joint activities provides high survival benefits, as it allows individuals to achieve goals together that are otherwise unavailable. Yet, survival advantage does not wholly explain the reasons for human cooperative behavior. In fact, humans are motivated to cooperate even when it is not necessary to achieve an outcome. Research in cognitive science suggests that navigating the potential costs of joint actions is a challenge for humans, and that joint actions might provide individuals with rewards that go beyond the achievement of instrumental goals. We here address the influence of key factors on the decision to engage in joint action, such as the coordination costs arising when acting together compared to alone and the social and instrumental rewards expected when acting together compared to alone. Addressing these questions will provide critical insight for the design of cognitive models of human decisions for cooperation.

Human adults prefer to cooperate even when it is costly

Joint actions are cooperative activities where humans coordinate their actions to achieve individual and shared goals. While the motivation to engage in joint action is clear when a goal cannot be achieved by individuals alone, we asked whether humans are motivated to act together even when acting together is not necessary and implies incurring additional costs compared to individual goal achievement. Using a utility-based empirical approach, we investigated the extent of humans' preference for joint action over individual action, when the instrumental costs of performing joint actions outweigh the benefits. The results of five experiments showed that human adults have a stable preference for joint action, even if individual action is more effective to achieve a certain goal. We propose that such preferences can be understood as ascribing additional reward value to performing actions together.

The sense of agency in joint action: An integrative review

When people perform joint actions together, their individual actions (e.g., moving one end of a heavy couch) must be coordinated to achieve a collective goal (e.g., moving the couch across the room). Joint actions pose unique challenges for understanding people's sense of agency, because each person engaged in the joint action can have a sense of agency not only at the individual level (a sense that "I moved my end of the couch" or "My partner moved their end of the couch"), but also at the collective level (a sense that "We moved the couch together"). This review surveys research that has examined people's sense of agency in joint action, including explicit judgments of agency, implicit measures of agency, and first-hand accounts of agency in real-world settings. The review provides a comprehensive summary of the factors that influence individual- and collective-level agency in joint action; reveals the progress that has been made toward understanding different forms of collective-level agency in joint action, including the sense that agency is shared among co-actors and the sense that co-actors are acting as a single unit; and synthesizes evidence concerning the relationships between different measures of implicit agency and individual- versus collective-level agency in joint action. The review concludes by highlighting numerous outstanding questions and promising avenues for future research.

A metastable attractor model of self-other integration (MEAMSO) in rhythmic synchronization

Human interaction is often accompanied by synchronized bodily rhythms. Such synchronization may emerge spontaneously as when a crowd's applause turns into a steady beat, be encouraged as in nursery rhymes, or be intentional as in the case of playing music together. The latter has been extensively studied using joint finger-tapping paradigms as a simplified version of rhythmic interpersonal synchronization. A key finding is that synchronization in such cases is multifaceted, with synchronized behaviour resting upon different synchronization strategies such as mutual adaptation, leading-following and leading-leading. However, there are multiple open questions regarding the mechanism behind these strategies and how they develop dynamically over time. Here, we propose a metastable attractor model of self-other integration (MEAMSO). This model conceptualizes dyadic rhythmic interpersonal synchronization as a process of integrating and segregating signals of self and other. Perceived sounds are continuously evaluated as either being attributed to self-produced or other-produced actions. The model entails a metastable system with two particular attractor states: one where an individual maintains two separate predictive models for self- and other-produced actions, and the other where these two predictive models integrate into one. The MEAMSO explains the three known synchronization strategies and makes testable predictions about the dynamics of interpersonal synchronization both in behaviour and the brain. This article is part of the theme issue 'Synchrony and rhythm interaction: from the brain to behavioural ecology'.

Coordinating With a Robot Partner Affects Neural Processing Related to Action Monitoring

Robots start to play a role in our social landscape, and they are progressively becoming responsive, both physically and socially. It begs the question of how humans react to and interact with robots in a coordinated manner and what the neural underpinnings of such behavior are. This exploratory study aims to understand the differences in human-human and human-robot interactions at a behavioral level and from a neurophysiological perspective. For this purpose, we adapted a collaborative dynamical paradigm from the literature. We asked 12 participants to hold two corners of a tablet while collaboratively guiding a ball around a circular track either with another participant or a robot. In irregular intervals, the ball was perturbed outward creating an artificial error in the behavior, which required corrective measures to return to the circular track again. Concurrently, we recorded electroencephalography (EEG). In the behavioral data, we found an increased velocity and positional error of the ball from the track in the human-human condition vs. human-robot condition. For the EEG data, we computed event-related potentials. We found a significant difference between human and robot partners driven by significant clusters at fronto-central electrodes. The amplitudes were stronger with a robot partner, suggesting a different neural processing. All in all, our exploratory study suggests that coordinating with robots affects action monitoring related processing. In the investigated paradigm, human participants treat errors during human-robot interaction differently from those made during interactions with other humans. These results can improve communication between humans and robot with the use of neural activity in real-time.

Not all errors are alike: modulation of error-related neural responses in musical joint action

During joint action, the sense of agency enables interaction partners to implement corrective and adaptive behaviour in response to performance errors. When agency becomes ambiguous (e.g. when action similarity encourages perceptual self-other overlap), confusion as to who produced what may disrupt this process. The current experiment investigated how ambiguity of agency affects behavioural and neural responses to errors in a joint action domain where self-other overlap is common: musical duos. Pairs of pianists performed piano pieces in synchrony, playing either the same pitches (ambiguous agency) or different pitches (unambiguous agency) while electroencephalography (EEG) was recorded for each individual. Behavioural and event-related potential results showed no effects of the agency manipulation but revealed differences in how distinct error types are processed. Self-produced 'wrong note' errors (substitutions) were left uncorrected, showed post-error slowing and elicited an error-related negativity (ERN) peaking before erroneous keystrokes (pre-ERN). In contrast, self-produced 'extra note' errors (additions) exhibited pre-error slowing, error and post-error speeding, were rapidly corrected and elicited the ERN. Other-produced errors evoked a feedback-related negativity but no behavioural effects. Overall findings shed light upon how the nervous system supports fluent interpersonal coordination in real-time joint action by employing distinct mechanisms to manage different types of errors.

The sound of silence: an EEG study of how musicians time pauses in individual and joint music performance

Pauses are an integral feature of social interaction. Conversation partners often pause between conversational turns, and musical co-performers often pause between musical phrases. How do humans coordinate the duration of pauses to ensure seamless interaction? A total of 40 trained pianists performed a simple melody containing fermatas (notated expressive pauses of unspecified duration) first alone (Solo) and then with a partner (Duet) while electroencephalography (EEG) was recorded. As predicted, Duet partners' tone onset synchrony was reduced for tones following pauses. Pauses were shorter in Duet relative to Solo performance, and synchrony of partners' Duet tone onsets was enhanced for tones following shorter pauses. EEG analysis revealed classic signatures of action preparation during pauses, namely decreases in the power of cortical beta oscillations (13-30 Hz, event-related desynchronization ERD). Beta ERD did not differ between pauses in Solo and Duet performance, but was enhanced for shorter relative to longer pauses, suggesting that reduced pause durations in Duet performance facilitated a neural state of enhanced action readiness. Together these findings provide novel insight into behavioural strategies by which musical partners resolve coordination challenges posed by expressive silence, and capture a clear neural signature of action planning during time-varying silences in natural music performance.

Simultaneous self-other integration and segregation support real-time interpersonal coordination in a musical joint action task

The ability to distinguish between an individual's own actions and those of another person is a requirement for successful joint action, particularly in domains such as group music making where precise interpersonal coordination ensures perceptual overlap in the effects of co-performers' actions. We tested the hypothesis that such coordination benefits from simultaneous integration and segregation of information about 'self' and 'other' in an experiment using a musical joint action paradigm. Sixteen pairs of individuals with little or no musical training performed a dyadic synchronization task on a pair of electronic music boxes. The relationship between pitches produced by paired participants (same vs. different) and the relationship between movement frequencies required to trigger synchronous tones (congruent vs. incongruent) were varied in a repeated measures design. The results indicate that interpersonal coordination was most accurate when sounds were different in pitch but movement frequency was congruent. Under other conditions, participants often drifted apart, resulting in poor coordination, especially with same sounds and incongruent movements across co-performers. These findings suggest that interpersonal coordination was facilitated when simultaneous self-other integration and segregation occurred across sensory modalities in an asymmetrical manner where pitch relations favoured segregation via auditory streaming while movement congruence favoured integration via visuo-motor coupling. Such self-other representational balance may enable co-performers to maintain autonomous control while attending, anticipating, and adapting to each other's timing when joint action requires precise temporal coordination.

Musical novices perform with equal accuracy when learning to drum alone or with a peer

The capacity of expert musicians to coordinate with each other when playing in ensembles or rehearsing has been widely investigated. However, little is known about the ability of novices to achieve satisfactory coordinated behaviour when making music together. We tested whether performance accuracy differs when novices play a newly learned drumming pattern with another musically untrained individual (duo group) or alone (solo group). A comparison between musical outcomes of the two groups revealed no significant differences concerning performative accuracy. An additional, exploratory examination of the degree of mutual influence between members of the duos suggested that they reciprocally affected each other when playing together. These findings indicate that a responsive auditory feedback involving surprises introduced by human errors could be part of pedagogical settings that employ repetition or imitation, thereby facilitating coordination among novices in a less prescribed fashion.

Coordination effort in joint action is reflected in pupil size

Humans often perform visual tasks together, and when doing so, they tend to devise division of labor strategies to share the load. Implementing such strategies, however, is effortful as co-actors need to coordinate their actions. We tested if pupil size - a physiological correlate of mental effort - can detect such a coordination effort in a multiple object tracking task (MOT). Participants performed the MOT task jointly with a computer partner and either devised a division of labor strategy (main experiment) or the labor division was already pre-determined (control experiment). We observed that pupil sizes increase relative to performing the MOT task alone in the main experiment while this is not the case in the control experiment. These findings suggest that pupil size can detect a rise in coordination effort, extending the view that pupil size indexes mental effort across a wide range of cognitively demanding tasks.

Understanding joint action: Current theoretical and empirical approaches

Joint actions are omnipresent, ranging from a handshake between two people to the coordination of groups of people playing in an orchestra. We are highly skilled at coordinating our actions with those of others to reach common goals and rely on this ability throughout our daily lives. What are the social, cognitive and neural processes underlying this ability? How do others around us influence our task representations? How does joint action influence interpersonal interactions? How do language and gesture support joint action? What differentiates joint action from individual action? This article forms an introductory editorial to the field of joint action. It accompanies contributions to the special issue entitled "Current Issues in Joint Action Research". The issue brings together conceptual and empirical approaches on different topics, ranging from lower-level issues such as the link between perception and joint action, to higher-level issues such as language as a form of joint action.

Progress in Joint-Action Research

Humans have a striking ability to coordinate their actions with each other to achieve joint goals. The tight interpersonal coordination that characterizes joint actions is achieved through processes that help with preparing for joint action as well as processes that are active while joint actions are being performed. To prepare for joint action, partners form representations of each other’s actions and tasks and the relation between them. This enables them to predict each other’s upcoming actions, which, in turn, facilitates coordination. While performing joint actions, partners’ coordination is maintained by (a) monitoring whether individual and joint outcomes correspond to what was planned, (b) predicting partners’ action parameters on the basis of familiarity with their individual actions, (c) communicating task-relevant information unknown to partners in an action-based fashion, and (d) relying on coupling of predictions through dense perceptual-information flow between coactors. The next challenge for the field of joint action is to generate an integrated perspective that links coordination mechanisms to normative, evolutionary, and communicative frameworks.

Emergent Shared Intentions Support Coordination During Collective Musical Improvisations

Human interactions are often improvised rather than scripted, which suggests that efficient coordination can emerge even when collective plans are largely underspecified. One possibility is that such forms of coordination primarily rely on mutual influences between interactive partners, and on perception-action couplings such as entrainment or mimicry. Yet some forms of improvised joint actions appear difficult to explain solely by appealing to these emergent mechanisms. Here, we focus on collective free improvisation, a form of highly unplanned creative practice where both agents' subjective reports and the complexity of their interactions suggest that shared intentions may sometimes emerge to support coordination during the course of the improvisation, even in the absence of verbal communication. In four experiments, we show that shared intentions spontaneously emerge during collective musical improvisations, and that they foster coordination on multiple levels, over and beyond the mere influence of shared information. We also show that musicians deploy communicative strategies to manifest and propagate their intentions within the group, and that this predicts better coordination. Overall, our results suggest that improvised and scripted joint actions are more continuous with one another than it first seems, and that they differ merely in the extent to which they rely on emergent or planned coordination mechanisms.

The motor-related brain activity that supports joint action: A review

Recent years have seen a rapid increase in research investigating the motor-related brain activity that supports joint action. This research has employed a variety of joint action tasks and an array of neuroimaging techniques, including fMRI, fNIRS, EEG, and TMS. In this review, we provide an overview of this research to delineate what is known about the motor-related brain activity that contributes to joint action and to highlight key questions for future research. Taken together, the surveyed research supports three major conclusions. First, the mere presence of a joint action context is sufficient to modulate motor activity elicited by observing others' actions. Second, joint action is supported by dissociable motor activity associated with a person's own actions, their partner's actions, and the joint action, and by between-brain coupling of motor-related oscillatory activity. Third, the structure of a joint action modulates the motor activity involved: Unique motor activity is associated with performing joint actions comprised of complementary actions and with holding the roles of leader and follower within a joint action. We conclude the review by highlighting overarching themes and key questions for future research.

Effects of trait empathy and expectation on the processing of observed actions

Recent evidence suggests that the processing of observed actions may reflect an action prediction error, with more pronounced mediofrontal negative event-related potentials (ERPs) for unexpected actions. This evidence comes from an application of a false-belief task, where unexpected correct responses elicited high ERP amplitudes. An alternative interpretation is that the ERP component reflects vicarious error processing, as objectively correct responses were errors from the observed person's perspective. In this study, we aimed to disentangle the two possibilities by adding the factor task difficulty, which varied expectations without affecting the definition of (vicarious) errors, and to explore the role of empathy in action observation. We found that the relationship between empathy and event-related potentials (ERPs) mirrored the relationship between empathy and behavioral expectancy measures. Only in the easy task condition did higher empathy lead to stronger expectancy of correct responses in the true-belief and of errors in the false-belief condition. A compatible pattern was found for an early ERP component (150-200 ms) after the observed response, with a larger negativity for error than correct responses in the true-belief and the reverse pattern in the false-belief condition, but only in highly empathic participants. We conclude that empathy facilitates the formation of expectations regarding the actions of others. These expectations then modulate the processing of observed actions, as indicated by the ERPs in the present study.

Dimensions of Musical Creativity

Current literature on creative cognition has developed rich conceptual landscapes dedicated to the analysis of both individual and collective forms of creativity. This work has favored the emergence of unifying theories on domain-general creative abilities in which the main experiential, behavioral, computational, and neural aspects involved in everyday creativity are examined and discussed. But while such accounts have gained important analytical leverage for describing the overall conditions and mechanisms through which creativity emerges and operates, they necessarily leave contextual forms of creativity less explored. Among the latter, musical practices have recently drawn the attention of scholars interested in its creative properties as well as in the creative potential of those who engage with them. In the present article, we compare previously posed theories of creativity in musical and non-musical domains to lay the basis of a conceptual framework that mitigates the tension between (i) individual and collective and (ii) domain-general and domain-specific perspectives on creativity. In doing so, we draw from a range of scholarship in music and enactive cognitive science, and propose that creative cognition may be best understood as a process of skillful organism-environment adaptation that one cultivates endlessly. With its focus on embodiment, plurality, and adaptiveness, our account points to a structured unity between living systems and their world, disclosing a variety of novel analytical resources for research and theory across different dimensions of (musical) creativity.

An approach to social flexibility: Congruency effects during spontaneous word-by-word interaction

Cognitive flexibility is the ability to switch between different concepts or to adapt goal-directed behavior in a changing environment. Although, cognitive research on this ability has long been focused on the individual mind, it is becoming increasingly clear that cognitive flexibility plays a central role in our social life. This is particularly evident in turn-taking in verbal conversation, where cognitive flexibility of the individual becomes part of social flexibility in the dyadic interaction. In this work, we introduce a model that reveals different parameters that explain how people flexibly handle unexpected events in verbal conversation. In order to study hypotheses derived from the model, we use a novel experimental approach in which thirty pairs of participants engaged in a word-by-word interaction by taking turns in generating sentences word by word. Similar to well established individual cognitive tasks, participants needed to adapt their behavior in order to respond to their co-actor’s last utterance. With our experimental approach we could manipulate the interaction between participants: Either both participants had to construct a sentence with a common target word (congruent condition) or with distinct target words (incongruent condition). We further studied the relation between the interactive Word-by-Word task measures and classical individual-centered, cognitive tasks, namely the Number-Letter task, the Stop-Signal task, and the GoNogo task. In the Word-by-Word task, we found that participants had faster response times in congruent compared to incongruent trials, which replicates the primary findings of standard cognitive tasks measuring cognitive flexibility. Further, we found a significant correlation between the performance in the Word-by-Word task and the Stop-Signal task indicating that participants with a high cognitive flexibility in the Word-by-Word task also showed high inhibition control.

Timing Markers of Interaction Quality During Semi-Hocket Singing

Music is believed to work as a bio-social tool enabling groups of people to establish joint action and group bonding experiences. However, little is known about the quality of the group members' interaction needed to bring about these effects. To investigate the role of interaction quality, and its effect on joint action and bonding experience, we asked dyads (two singers) to perform music in medieval "hocket" style, in order to engage their co-regulatory activity. The music contained three relative inter-onset-interval (IOI) classes: quarter note, dotted quarter note and eight note, marking time intervals between successive onsets (generated by both singers). We hypothesized that singers co-regulated their activity by minimizing prediction errors in view of stable IOI-classes. Prediction errors were measured using a dynamic Bayesian inference approach that allows us to identify three different types of error called fluctuation (micro-timing errors measured in milliseconds), narration (omission errors or misattribution of an IOI to a wrong IOI class), and collapse errors (macro-timing errors that cause the breakdown of a performance). These three types of errors were correlated with the singers' estimated quality of the performance and the experienced sense of joint agency. We let the singers perform either while moving or standing still, under the hypothesis that the moving condition would have reduced timing errors and increased We-agency as opposed to Shared-agency (the former portraying a condition in which the performers blend into one another, the latter portraying a joint, but distinct, control of the performance). The results show that estimated quality correlates with fluctuation and narration errors, while agency correlates (to a lesser degree) with narration errors. Somewhat unexpectedly, there was a minor effect of movement, and it was beneficial only for good performers. Joint agency resulted in a "shared," rather than a "we," sense of joint agency. The methodology and findings open up promising avenues for future research on social embodied music interaction.

Does musical interaction in a jazz duet modulate peripersonal space?

Researchers have widely studied peripersonal space (the space within reach) in the last 20 years with a focus on its plasticity following the use of tools and, more recently, social interactions. Ensemble music is a sophisticated joint action that is typically explored in its temporal rather than spatial dimensions, even within embodied approaches. We, therefore, devised a new paradigm in which two musicians could perform a jazz standard either in a cooperative (correct harmony) or uncooperative (incorrect harmony) condition, under the hypothesis that their peripersonal spaces are modulated by the interaction. We exploited a well-established audio-tactile integration task as a proxy for such a space. After the performances, we measured reaction times to tactile stimuli on the subjects' right hand and auditory stimuli delivered at two different distances, (next to the subject and next to the partner). Considering previous literature's evidence that integration of two different stimuli (e.g. a tactile and an auditory stimulus) is faster in near space compared to far space, we predicted that a cooperative interaction would have extended the peripersonal space of the musicians towards their partner, facilitating reaction times to bimodal stimuli in both spaces. Surprisingly, we obtained complementary results in terms of an increase of reaction times to tactile-auditory near stimuli, but only following the uncooperative condition. We interpret this finding as a suppression of the subject's peripersonal space or as a withdrawal from the uncooperative partner. Subjective reports and correlations between these reports and reaction times comply with that interpretation. Finally, we determined an overall better multisensory integration competence in musicians compared to non-musicians tested in the same task.

Adaptation to unstable coordination patterns in individual and joint actions

Previous research on interlimb coordination has shown that some coordination patterns are more stable than others, and function as attractors in the space of possible phase relations between different rhythmic movements. The canonical coordination patterns, i.e. the two most stable phase relations, are in-phase (0 degree) and anti-phase (180 degrees). Yet, musicians are able to perform other coordination patterns in intrapersonal as well as in interpersonal coordination with remarkable precision. This raises the question of how music experts manage to produce these unstable patterns of movement coordination. In the current study, we invited participants with at least five years of training on a musical instrument. We used an adaptation paradigm to address two factors that may facilitate producing unstable coordination patterns. First, we investigated adaptation in different coordination settings, to test the hypothesis that the lower coupling strength between individuals during joint performance makes it easier to achieve stability outside of the canonical patterns than the stronger coupling during individual bimanual performance. Second, we investigated whether adding to the structure of action effects may support achieving unstable coordination patterns, both intra- and inter-individually. The structure of action effects was strengthened by adding a melodic contour to the action effects, a measure that has been shown to improve the acquisition of bimanual coordination skills. Adaptation performance was measured both in terms of asynchrony and variability thereof. As predicted, we found that producing unstable patterns benefitted from the weaker coupling during joint performance. Surprisingly, the structure of action effects did not help with achieving unstable coordination patterns.

Music Making and Neuropsychological Aging: A Review

Aging is associated with a decline in social understanding and general cognition. Both are integral to wellbeing and rely on similar brain regions. Thus, as the population ages, there is a growing need for knowledge on the types of activities that maintain brain health in older adulthood. Active engagement in music making might be one such activity because it places a demand on brain networks tapping into multisensory integration, learning, reward, and cognition. It has been hypothesized that this demand may promote plasticity in the frontal and temporal lobes by taxing cognitive abilities and, hence, increase resistance to age-related neurodegeneration. We examine research relevant to this hypothesis and note that there is a lack of intervention studies with a well-matched control condition and random assignment. Thus, we discuss potential causal mechanisms underlying training-related neuropsychological changes, and provide suggestions for future research. It is argued that although music training might be a valuable tool for supporting healthy neuropsychological aging and mental wellbeing, well-controlled intervention studies are necessary to provide clear evidence.

How Task Interactivity Shapes Action Observation

Action observation triggers imitation, a powerful mechanism permitting interpersonal coordination. Coordination, however, also occurs when the partners’ actions are nonimitative and physically incongruent. One influential theory postulates that this is achieved via top-down modulation of imitation exerted by prefrontal regions. Here, we rather argue that coordination depends on sharing a goal with the interacting partner: this shapes action observation, overriding involuntary imitation, through the predictive activity of the left ventral premotor cortex (lvPMc). During functional magnetic resonance imaging (fMRI), participants played music in turn with a virtual partner in interactive and noninteractive conditions requiring 50% of imitative/nonimitative responses. In a full-factorial design, both perceptual features and low-level motor requirements were kept constant throughout the experiment. Behaviorally, the interactive context minimized visuomotor interference due to the involuntary imitation of physically incongruent movements. This was paralleled by modulation of neural activity in the lvPMc, which was specifically recruited during the interactive task independently of the imitative/nonimitative nature of the social exchange. This lvPMc activity reflected the predictive decoding of the partner’s actions, as revealed by multivariate pattern analysis. This demonstrates that, during interactions, we process our partners’ behavior to prospectively infer their contribution to the shared goal achievement, generating motor predictions for cooperation beyond low-level imitation.

Individual differences in musical training and executive functions: A latent variable approach

Learning and performing music draw on a host of cognitive abilities, and previous research has postulated that musicians might have advantages in related cognitive processes. One such aspect of cognition that may be related to musical training is executive functions (EFs), a set of top-down processes that regulate behavior and cognition according to task demands. Previous studies investigating the link between musical training and EFs have yielded mixed results and are difficult to compare. In part, this is because most studies have looked at only one specific cognitive process, and even studies looking at the same process have used different experimental tasks. Furthermore, most correlational studies have used different "musician" and "non-musician" categorizations for their comparisons, so generalizing the findings is difficult. The present study provides a more comprehensive assessment of how individual differences in musical training relate to latent measures of three separable aspects of EFs. We administered a well-validated EF battery containing multiple tasks tapping the EF components of inhibition, shifting, and working memory updating (Friedman et al. in Journal of Experimental Psychology: General, 137, 201-225, 2008), as well as a comprehensive, continuous measure of musical training and sophistication (Müllensiefen et al., in PLoS ONE, 9, e89642, 2014). Musical training correlated with some individual EF tasks involving inhibition and working memory updating, but not with individual tasks involving shifting. However, musical training only predicted the latent variable of working memory updating, but not the latent variables of inhibition or shifting after controlling for IQ, socioeconomic status, and handedness. Although these data are correlational, they nonetheless suggest that musical experience places particularly strong demands specifically on working memory updating processes.

Force asymmetry deteriorates complementary force production during joint action

The present study aimed to determine the effects of force asymmetry on interpersonal force production. This study consisted of an individual task executed by one participant at a time in a pair, and three joint tasks executed by two participants simultaneously under conditions of 1:1, 1:0.75, and 1:0.5. Two individuals produced discrete forces at the same time so that the sum of forces they produced was the target force in the joint task. Under the 1:1 condition, the target force was the sum of the maximum voluntary contraction (MVC) produced by the index finger of each participant × 0.1 (10% MVC). Under the 1:0.75 condition, the investigators manipulated the force produced by only one of the pair, for example, B, but not A. The feedback was also scaled as a result. The target force was the MVC of participant A + the MVC of participant B × 0.75 × 0.1. Similarly, the target force under the 1:0.5 condition was the MVC of participant A + the MVC of participant B × 0.5 × 0.1. The present study found that forces produced by pairs were negatively correlated and the correlation value was higher under the 1:1 condition than the 1:0.75 and 1:0.5 conditions. The absolute error was smaller under the 1:1 condition than the 1:0.5 condition. Complementary force production was attenuated and the error increased as differences between forces produced by two participants increased. Thus, asymmetry of forces produced by pairs deteriorated complementary force production and interpersonal performance.

Ears, heads, and eyes: When singers synchronise

We examined the relationship between endogenous rhythms, auditory and visual cues, and body movement in the temporal coordination of duet singers. Sixteen pairs of experienced vocalists sang a familiar melody in Solo and two Duet conditions. Vocalists sang together in Unison (simultaneously producing identical pitches) and Round Duet conditions (one vocalist, the Follower, producing pitches at an eight-tone delay from their partner, the Leader) while facing Inward (full visual cues) and Outward (reduced visual cues). Larger tempo differences in partners' spontaneous (temporally unconstrained) Solo performances were associated with larger asynchrony in Duet performances, consistent with coupling predictions for oscillators with similar natural frequencies. Vocalists were slightly but consistently more synchronous in Duets when facing their partner (Inward) than when facing Outward; Unison and Round performances were equally synchronous. The greater difficulty of Rounds production was evidenced in vocalists' slower performance rates and more variable head movements; Followers directed their head gaze away from their partner and used bobbing head movements to mark the musical beat. The strength of Followers' head movements corresponded to the amount of tone onset asynchrony with their partners, indicating a strong association between timing and movement under increased attentional and working memory demands in music performance.

The Social Situation Affects How We Process Feedback About Our Actions

Humans achieve their goals in joint action tasks either by cooperation or competition. In the present study, we investigated the neural processes underpinning error and monetary rewards processing in such cooperative and competitive situations. We used electroencephalography (EEG) and analyzed event-related potentials (ERPs) triggered by feedback in both social situations. 26 dyads performed a joint four-alternative forced choice (4AFC) visual task either cooperatively or competitively. At the end of each trial, participants received performance feedback about their individual and joint errors and accompanying monetary rewards. Furthermore, the outcome, i.e., resulting positive, negative, or neutral rewards, was dependent on the pay-off matrix, defining the social situation either as cooperative or competitive. We used linear mixed effects models to analyze the feedback-related-negativity (FRN) and used the Threshold-free cluster enhancement (TFCE) method to explore activations of all electrodes and times. We found main effects of the outcome and social situation, but no interaction at mid-line frontal electrodes. The FRN was more negative for losses than wins in both social situations. However, the FRN amplitudes differed between social situations. Moreover, we compared monetary with neutral outcomes in both social situations. Our exploratory TFCE analysis revealed that processing of feedback differs between cooperative and competitive situations at right temporo-parietal electrodes where the cooperative situation elicited more positive amplitudes. Further, the differences induced by the social situations were stronger in participants with higher scores on a perspective taking test. In sum, our results replicate previous studies about the FRN and extend them by comparing neurophysiological responses to positive and negative outcomes in a task that simultaneously engages two participants in competitive and cooperative situations.

Emergent Synergistic Grasp-Like Behavior in a Visuomotor Joint Action Task: Evidence for Internal Forward Models as Building Blocks of Human Interactions

Central to the mechanistic understanding of the human mind is to clarify how cognitive functions arise from simpler sensory and motor functions. A longstanding assumption is that forward models used by sensorimotor control to anticipate actions also serve to incorporate other people's actions and intentions, and give rise to sensorimotor interactions between people, and even abstract forms of interactions. That is, forward models could aid core aspects of human social cognition. To test whether forward models can be used to coordinate interactions, here we measured the movements of pairs of participants in a novel joint action task. For the task they collaborated to lift an object, each of them using fingers of one hand to push against the object from opposite sides, just like a single person would use two hands to grasp the object bimanually. Perturbations of the object were applied randomly as they are known to impact grasp-specific movement components in common grasping tasks. We found that co-actors quickly learned to make grasp-like movements with grasp components that showed coordination on average based on action observation of peak deviation and velocity of their partner's trajectories. Our data suggest that co-actors adopted pre-existing bimanual grasp programs for their own body to use forward models of their partner's effectors. This is consistent with the long-held assumption that human higher-order cognitive functions may take advantage of sensorimotor forward models to plan social behavior. New and Noteworthy: Taking an approach of sensorimotor neuroscience, our work provides evidence for a long-held belief that the coordination of physical as well as abstract interactions between people originates from certain sensorimotor control processes that form mental representations of people's bodies and actions, called forward models. With a new joint action paradigm and several new analysis approaches we show that, indeed, people coordinate each other's interactions based on forward models and mutual action observation.

fNIRS Responses in Professional Violinists While Playing Duets: Evidence for Distinct Leader and Follower Roles at the Brain Level

Music played in ensembles is a naturalistic model to study joint action and leader-follower relationships. Recently, the investigation of the brain underpinnings of joint musical actions has gained attention; however, the cerebral correlates underlying the roles of leader and follower in music performance remain elusive. The present study addressed this question by simultaneously measuring the hemodynamic correlates of functional neural activity elicited during naturalistic violin duet performance using fNIRS. Findings revealed distinct patterns of functional brain activation when musicians played the Violin 2 (follower) than the Violin 1 part (leader) in duets, both compared to solo performance. More specifically, results indicated that musicians playing the Violin 2 part had greater oxy-Hb activation in temporo-parietal (p = 0.02) and somatomotor (p = 0.04) regions during the duo condition in relation to the solo. On the other hand, there were no significant differences in the activation of these areas between duo/solo conditions during the execution of the Violin 1 part (p's > 0.05). These findings suggest that ensemble cohesion during a musical performance may impose particular demands when musicians play the follower position, especially in brain areas associated with the processing of dynamic social information and motor simulation. This study is the first to use fNIRS hyperscanning technology to simultaneously measure the brain activity of two musicians during naturalistic music ensemble performance, opening new avenues for the investigation of brain correlates underlying joint musical actions with multiple subjects in a naturalistic environment.

Eye gaze as a means of giving and seeking information during musical interaction

During skilled music ensemble performance, a multi-layered network of interaction processes allows musicians to negotiate common interpretations of ambiguously-notated music in real-time. This study investigated the conditions that encourage visual interaction during duo performance. Duos recorded performances of a new piece before and after a period of rehearsal. Mobile eye tracking and motion capture were used in combination to map uni- and bidirectional eye gaze patterns. Musicians watched each other more during temporally-unstable passages than during regularly-timed passages. They also watched each other more after rehearsal than before. Duo musicians may seek visual interaction with each other primarily, but not exclusively, when coordination is threatened by temporal instability. Visual interaction increases as musicians become familiar with the piece, suggesting that they visually monitor each other once a shared interpretation of the piece is established. Visual monitoring of co-performers' movements and attention may facilitate feelings of engagement and high-level creative collaboration.

Electrical Brain Responses Reveal Sequential Constraints on Planning during Music Performance

Elements in speech and music unfold sequentially over time. To produce sentences and melodies quickly and accurately, individuals must plan upcoming sequence events, as well as monitor outcomes via auditory feedback. We investigated the neural correlates of sequential planning and monitoring processes by manipulating auditory feedback during music performance. Pianists performed isochronous melodies from memory at an initially cued rate while their electroencephalogram was recorded. Pitch feedback was occasionally altered to match either an immediately upcoming Near-Future pitch (next sequence event) or a more distant Far-Future pitch (two events ahead of the current event). Near-Future, but not Far-Future altered feedback perturbed the timing of pianists' performances, suggesting greater interference of Near-Future sequential events with current planning processes. Near-Future feedback triggered a greater reduction in auditory sensory suppression (enhanced response) than Far-Future feedback, reflected in the P2 component elicited by the pitch event following the unexpected pitch change. Greater timing perturbations were associated with enhanced cortical sensory processing of the pitch event following the Near-Future altered feedback. Both types of feedback alterations elicited feedback-related negativity (FRN) and P3a potentials and amplified spectral power in the theta frequency range. These findings suggest similar constraints on producers' sequential planning to those reported in speech production.

Collaborative Musical Creativity: How Ensembles Coordinate Spontaneity

Music performance is inherently social. Most music is performed in groups, and even soloists are subject to influence from a (real or imagined) audience. It is also inherently creative. Performers are called upon to interpret notated music, improvise new musical material, adapt to unexpected playing conditions, and accommodate technical errors. The focus of this paper is how creativity is distributed across members of a music ensemble as they perform these tasks. Some aspects of ensemble performance have been investigated extensively in recent years as part of the broader literature on joint action (e.g., the processes underlying sensorimotor synchronization). Much of this research has been done under highly controlled conditions, using tasks that generate reliable results, but capture only a small part of ensemble performance as it occurs naturalistically. Still missing from this literature is an explanation of how ensemble musicians perform in conditions that require creative interpretation, improvisation, and/or adaptation: how do they coordinate the production of something new? Current theories of creativity endorse the idea that dynamic interaction between individuals, their actions, and their social and material environments underlies creative performance. This framework is much in line with the embodied music cognition paradigm and the dynamical systems perspective on ensemble coordination. This review begins by situating the concept of collaborative musical creativity in the context of embodiment. Progress that has been made toward identifying the mechanisms that underlie collaborative creativity in music performance is then assessed. The focus is on the possible role of musical imagination in facilitating performer flexibility, and on the forms of communication that are likely to support the coordination of creative musical output. Next, emergence and group flow–constructs that seem to characterize ensemble performance at its peak–are considered, and some of the conditions that may encourage periods of emergence or flow are identified. Finally, it is argued that further research is needed to (1) demystify the constructs of emergence and group flow, clarifying their effects on performer experience and listener response, (2) determine how constrained musical imagination is by perceptual experience and understand people's capacity to depart from familiar frameworks and imagine new sounds and sound structures, and (3) assess the technological developments that are supposed to facilitate or enhance musical creativity, and determine what effect they have on the processes underlying creative collaboration.

Performance monitoring of self and other in a turn-taking piano duet: A dual-EEG study

During joint action tasks, expectations for outcomes of one's own and other's actions are collectively monitored. Recent evidence suggests that trait empathy levels may also influence performance monitoring processes. The present study investigated how outcome expectation and empathy interact during a turn-taking piano duet task, using simultaneous electroencephalography (EEG) recording. During the performances, one note in each player's part was altered in pitch to elicit the feedback-related negativity (FRN) and subsequent P3 complex. Pianists memorized and performed pieces containing either a similar or dissimilar sequence as their partner. For additional blocks, pianists also played both sequence types with an audio-only computer partner. The FRN and P3a were larger in response to self than other, while P3b occurred only in response to self, suggesting greater online monitoring of self- compared to other-produced actions during turn-taking joint action. P3a was larger when pianists played a similar sequence as their partner. Finally, as trait empathy level increased, FRN in response to self decreased. This association was absent for FRN in response to other. This may reflect that highly-empathetic musicians during joint performance could use a strategy to suppress exclusive focus on self-monitoring.

Effect- and Performance-Based Auditory Feedback on Interpersonal Coordination

When two individuals interact in a collaborative task, such as carrying a sofa or a table, usually spatiotemporal coordination of individual motor behavior will emerge. In many cases, interpersonal coordination can arise independently of verbal communication, based on the observation of the partners' movements and/or the object's movements. In this study, we investigate how social coupling between two individuals can emerge in a collaborative task under different modes of perceptual information. A visual reference condition was compared with three different conditions with new types of additional auditory feedback provided in real time: effect-based auditory feedback, performance-based auditory feedback, and combined effect/performance-based auditory feedback. We have developed a new paradigm in which the actions of both participants continuously result in a seamlessly merged effect on an object simulated by a tablet computer application. Here, participants should temporally synchronize their movements with a 90° phase difference and precisely adjust the finger dynamics in order to keep the object (a ball) accurately rotating on a given circular trajectory on the tablet. Results demonstrate that interpersonal coordination in a joint task can be altered by different kinds of additional auditory information in various ways.