A practical guide to EEG hyperscanning in joint action research: from motivation to implementation

Developments in cognitive neuroscience have led to the emergence of hyperscanning, the simultaneous measurement of brain activity from multiple people. Hyperscanning is useful for investigating social cognition, including joint action, because of its ability to capture neural processes that occur within and between people as they coordinate actions toward a shared goal. Here, we provide a practical guide for researchers considering using hyperscanning to study joint action and seeking to avoid frequently raised concerns from hyperscanning skeptics. We focus specifically on Electroencephalography (EEG) hyperscanning, which is widely available and optimally suited for capturing fine-grained temporal dynamics of action coordination. Our guidelines cover questions that are likely to arise when planning a hyperscanning project, ranging from whether hyperscanning is appropriate for answering one's research questions to considerations for study design, dependent variable selection, data analysis and visualization. By following clear guidelines that facilitate careful consideration of the theoretical implications of research design choices and other methodological decisions, joint action researchers can mitigate interpretability issues and maximize the benefits of hyperscanning paradigms.

Motor-related cortical oscillations distinguish one's own from a partner's contributions to a joint action

The ability to distinguish between one's own and others' actions is a requirement for successful joint action. Such a distinction might be supported by dissociable motor activity underlying each partner's individual contributions to the joint action. However, little research has directly compared motor activity associated with one's own vs. others' actions during joint action. The current study investigated whether motor-related cortical oscillations distinguish between self- and partner-produced actions when partners take turns producing taps to meet a joint timing goal. Across two experiments, the degree of beta suppression differentiated one's own from a partner's actions, with more suppression occurring during one's own actions than during a partner's actions. Self-partner differences in mu suppression were also evident, particularly when partners produced actions in succession. Increased beta suppression was also observed during partners' actions when they were followed by one's own actions, suggesting that the coordination demands imposed by the joint action could affect the pattern of beta reactivity during a turn-taking joint action. Together, these findings demonstrate that dynamic patterns of motor activity underpin successful joint action and that periods of distinct motor activity are associated with one's own contributions to a joint action.

Not just in sync: Relations between partners' actions influence the sense of joint agency during joint action

When people perform joint actions together, they often experience a sense of joint agency ("we did that together"). The current study investigated whether relations between partners' actions within joint actions that require precise interpersonal synchrony influence joint agency, above and beyond the degree of synchrony partners achieve. We employed a mixed-methods approach that combined a quantitative experiment with a qualitative analysis of post-experiment interviews. Partners produced synchronized tone sequences that comprised either constant pitch sequences (simple temporal alignment between partners' actions) or musical duets (complex metrical and harmonic relations between partners' actions). Participants reported stronger joint agency for duets than constant pitches, when comparing trials with equally good synchronization. Post-experiment interviews revealed that joint agency was also influenced by participants' knowledge of the music and their perceptions of task performance, difficulty, and enjoyability. These findings further our understanding of joint agency for joint actions that require precise interpersonal synchrony.

The auditory P2 differentiates self- from partner-produced sounds during joint action: Contributions of self-specific attenuation and temporal orienting of attention

Sensory attenuation of the auditory P2 event-related potential (ERP) has been shown to differentiate the sensory consequences of one's own from others' action in joint action contexts. However, recent evidence suggests that when people coordinate joint actions over time, temporal orienting of attention might simultaneously contribute to enhancing the auditory P2. The current study employed a joint tapping task in which partners produced tone sequences together to examine whether temporal orienting influences auditory ERP amplitudes during the time window of self-other differentiation. Our findings demonstrate that the combined requirements of coordinating with a partner toward a joint goal and immediately adjusting to the partner's tone timing enhance P2 amplitudes elicited by the partner's tone onsets. Furthermore, our findings replicate prior evidence for self-specific sensory attenuation of the auditory P2 in joint action, and additionally demonstrate that it occurs regardless of the coordination requirements between partners. Together, these findings provide evidence that temporal orienting and sensory attenuation both modulate the auditory P2 during joint action and suggest that both processes play a role in facilitating precise interpersonal coordination between partners.

Alpha oscillations related to self-other integration and distinction during live orchestral performance: A naturalistic case study

Ensemble music performance requires musicians to achieve precise interpersonal coordination while maintaining autonomous control over their own actions. To do so, musicians dynamically shift between integrating other performers' actions into their own action plans and maintaining a distinction between their own and others' actions. Research in laboratory settings has shown that this dynamic process of self-other integration and distinction is indexed by sensorimotor alpha oscillations. The purpose of the current descriptive case study was to examine oscillations related to self-other integration and distinction in a naturalistic performance context. We measured alpha activity from four violinists during a concert hall performance of a 60-musician orchestra. We selected a musical piece from the orchestra's repertoire and, before analyzing alpha activity, performed a score analysis to divide the piece into sections that were expected to strongly promote self-other integration and distinction. In line with previous laboratory findings, performers showed suppressed and enhanced alpha activity during musical sections that promoted self-other integration and distinction, respectively. The current study thus provides preliminary evidence that findings from carefully controlled laboratory experiments generalize to complex real-world performance. Its findings also suggest directions for future research and potential applications of interest to musicians, music educators, and music therapists.

Sensory Attenuation of the Auditory P2 Differentiates Self- from Partner-Produced Sounds during Joint Action

Successful human interaction relies on people's ability to differentiate between the sensory consequences of their own and others' actions. Research in solo action contexts has identified sensory attenuation, that is, the selective perceptual or neural dampening of the sensory consequences of self-produced actions, as a potential marker of the distinction between self- and externally produced sensory consequences. However, very little research has examined whether sensory attenuation distinguishes self- from partner-produced sensory consequences in joint action contexts. The current study examined whether sensory attenuation of the auditory N1 or P2 ERPs distinguishes self- from partner-produced tones when pairs of people coordinate their actions to produce tone sequences that match a metronome pace. We did not find evidence of auditory N1 attenuation for either self- or partner-produced tones. Instead, the auditory P2 was attenuated for self-produced tones compared to partner-produced tones within the joint action. These findings indicate that self-specific attenuation of the auditory P2 differentiates the sensory consequences of one's own from others' actions during joint action. These findings also corroborate recent evidence that N1 attenuation may be driven by general rather than action-specific processes and support a recently proposed functional dissociation between auditory N1 and P2 attenuation.

The motor-related brain activity that supports joint action: A review

Recent years have seen a rapid increase in research investigating the motor-related brain activity that supports joint action. This research has employed a variety of joint action tasks and an array of neuroimaging techniques, including fMRI, fNIRS, EEG, and TMS. In this review, we provide an overview of this research to delineate what is known about the motor-related brain activity that contributes to joint action and to highlight key questions for future research. Taken together, the surveyed research supports three major conclusions. First, the mere presence of a joint action context is sufficient to modulate motor activity elicited by observing others' actions. Second, joint action is supported by dissociable motor activity associated with a person's own actions, their partner's actions, and the joint action, and by between-brain coupling of motor-related oscillatory activity. Third, the structure of a joint action modulates the motor activity involved: Unique motor activity is associated with performing joint actions comprised of complementary actions and with holding the roles of leader and follower within a joint action. We conclude the review by highlighting overarching themes and key questions for future research.

The sound of you and me: Novices represent shared goals in joint action

People performing joint actions coordinate their individual actions with each other to achieve a shared goal. The current study investigated the mental representations that are formed when people learn a new skill as part of a joint action. In a musical transfer-of-learning paradigm, piano novices first learned to perform simple melodies in the joint action context of coordinating with an accompanist to produce musical duets. Participants then performed their previously learned actions with two types of auditory feedback: while hearing either their individual action goal (the melody) or the shared action goal (the duet). As predicted, participants made more performance errors in the individual goal condition than in the shared goal condition. Further experimental manipulations indicated that this difference was not due to different coordination requirements in the two conditions or perceptual dissimilarities between learning and test. Together, these findings indicate that people form representations of shared goals in contexts that promote minimal representations, such as when learning a new action together with another person.

Monitoring Individual and Joint Action Outcomes in Duet Music Performance

We investigated whether people monitor the outcomes of their own and their partners' individual actions as well as the outcome of their combined actions when performing joint actions together. Pairs of pianists memorized both parts of a piano duet. Each pianist then performed one part while their partner performed the other; EEG was recorded from both. Auditory outcomes (pitches) associated with keystrokes produced by the pianists were occasionally altered in a way that either did or did not affect the joint auditory outcome (i.e., the harmony of a chord produced by the two pianists' combined pitches). Altered auditory outcomes elicited a feedback-related negativity whether they occurred in the pianist's own part or the partner's part, and whether they affected individual or joint action outcomes. Altered auditory outcomes also elicited a P300 whose amplitude was larger when the alteration affected the joint outcome compared with individual outcomes and when the alteration affected the pianist's own part compared with the partner's part. Thus, musicians engaged in joint actions monitor their own and their partner's actions as well as their combined action outcomes, while at the same time maintaining a distinction between their own and others' actions and between individual and joint outcomes.

Sensory attenuation for jointly produced action effects

Successful joint action often requires people to distinguish between their own and others’ contributions to a shared goal. One mechanism that is thought to underlie a self-other distinction is sensory attenuation, whereby the sensory consequences of one’s own actions are reduced compared to other sensory events. Previous research has shown that the auditory N1 event-related potential (ERP) response is reduced for self-generated compared to externally generated tones. The current study examined whether attenuation also occurs for jointly generated tones, which require two people to coordinate their actions to produce a single tone. ERP responses were measured when participants generated tones alone (tone onset immediately followed the participant’s button press) or with a partner (tone onset immediately followed the participant’s or the partner’s button press, whichever occurred second). N1 attenuation was smaller for jointly generated tones compared to self-generated tones. For jointly generated tones, greater delays between the participant’s and the partner’s button presses were associated with reduced attenuation; moreover, only trials in which there was no delay between the participant’s press and tone onset showed attenuation, whereas trials in which there were delays did not show attenuation. These findings indicate that people differentiate between their own and another person’s contributions to a joint action at the sensorimotor level, even when they must act together to produce a single, shared effect.

Temporal coordination and adaptation to rate change in music performance

People often coordinate their actions with sequences that exhibit temporal variability and unfold at multiple periodicities. We compared oscillator- and timekeeper-based accounts of temporal coordination by examining musicians' coordination of rhythmic musical sequences with a metronome that gradually changed rate at the end of a musical phrase (Experiment 1) or at the beginning of a phrase (Experiment 2). The rhythms contained events that occurred at the same periodic rate as the metronome and at half the period. Rate change consisted of a linear increase or decrease in intervals between metronome onsets. Musicians coordinated their performances better with a metronome that decreased than increased in tempo (as predicted by an oscillator model), at both beginnings and ends of musical phrases. Model performance was tested with an oscillator period or timekeeper interval set to the same period as the metronome (1:1 coordination) or half the metronome period (2:1 coordination). Only the oscillator model was able to predict musicians' coordination at both periods. These findings suggest that coordination is based on internal neural oscillations that entrain to external sequences.

Temporal Coordination between Performing Musicians

Many common behaviours require people to coordinate the timing of their actions with the timing of others' actions. We examined whether representations of musicians' actions are activated in coperformers with whom they must coordinate their actions in time and whether coperformers simulate each other's actions using their own motor systems during temporal coordination. Pianists performed right-hand melodies along with simple or complex left-hand accompaniments produced by themselves or by another pianist. Individual performers' preferred performance rates were measured in solo performance of the right-hand melody. The complexity of the left-hand accompaniment influenced the temporal grouping structure of the right-hand melody in the same way when it was performed by the self or by the duet partner, providing some support for the action corepresentation hypothesis. In contrast, accompaniment complexity had little influence on temporal coordination measures (asynchronies and cross-correlations between parts). Temporal coordination measures were influenced by a priori similarities between partners' preferred rates; partners who had similar preferred rates in solo performance were better synchronized and showed mutual adaptation to each other's timing during duet performances. These findings extend previous findings of action corepresentation and action simulation to a task that requires precise temporal coordination of independent yet simultaneous actions.

Sequential and biomechanical factors constrain timing and motion in tapping

The authors examined how timing accuracy in tapping sequences is influenced by sequential effects of preceding finger movements and biomechanical interdependencies among fingers. Skilled pianists tapped sequences at 3 rates; in each sequence, a finger whose motion was more or less independent of other fingers' motion was preceded by a finger to which it was more or less coupled. Less independent fingers and those preceded by a more coupled finger showed large timing errors and change in motion because of the preceding finger's motion. Motion change correlated with shorter intertap intervals and increased with rate. Thus, timing of sequence elements is not independent of the motion trajectories that individuals use to produce them. Neither motion nor its relation to timing is invariant across rates.

Cognitive and biomechanical influences in pianists’ finger tapping

Movement sequences such as typing or tapping display important interactions among finger movements arising from anticipatory motion (preparing for upcoming events) and coupling (non-independence among fingers). We examined pianists' finger tapping for the influence of cognitive chunking processes and biomechanical coupling constraints. In a synchronization-continuation task, pianists repeatedly tapped four-finger sequences that differed in terms of the chunks that formed subsequences and in the transitions among physically adjacent or non-adjacent fingers. Chunking influenced intertap intervals, regardless of the particular fingers tapped; the final tap of each chunk was lengthened and less variable relative to other taps. The particular fingers tapped influenced peak finger heights, consistency of motion, and velocity-acceleration patterns, regardless of chunking. Thus, cognitive constraints influenced timing, whereas biomechanical factors influenced motion trajectories. These findings provide an important caveat for study of anticipatory motion by documenting the influence of biomechanical coupling on motion trajectories.

Item method directed forgetting diminishes false memory

Directed forgetting may reduce DRM false memory illusion by interfering with meaning processing. Participants were presented with a list composed of six 10-word semantically associated sub-lists, and they were either (a) asked to remember all list items of (b) asked to remember all associates from sub-lists and to forget all associates from other sub-lists. All participants were requested to recall and recognise list items. Although directed forgetting effects have been previously reported only for true responses in the DRM paradigm with the list method, we also found directed forgetting for false responses with the item method. Such forgetting instructions reduced both verbatim and meaning processing, decreasing both the intrusion and the false alarm rate. These results are consistent with two-process explanations of DRM false memories, such as fuzzy-trace theory, and add to our understanding of false memory editing.

Omitting details from post-event information: are true and false memory affected in the same way?

Participants who witness an event and later receive post-event information that omits a critical scene are less likely to recall and to recognise that scene than are participants who receive no post-event information (Wright, Loftus, & Hall, 2001). The present study used the Deese-Roediger-McDermott (DRM) paradigm, in which participants study lists of semantic associates (e.g., hot, snow, warm, winter) that commonly elicit false memories of critical non-presented words (e.g., cold), to determine whether omitting information from a second presentation decreases memory for both presented and non-presented information. Participants were presented with a list of the semantic associates of six non-presented words. For half the participants, this list was presented a second time with the semantic associates of one of the non-presented words omitted. As expected, participants were less likely to recall and to recognise the presented words when they had been omitted from the second presentation. Omission also decreased the rate at which non-presented words were recalled, although false recognition of these words was not reduced. These results suggest that false recognition may be particularly difficult to attenuate and that post-event omission may be more detrimental to memory accuracy than previously thought.