Imaging the social brain: multi-subjects EEG recordings during the "Chicken's game"

Pseudo-mutual gazing enhances interbrain synchrony during remote joint attention tasking

INTRODUCTION

Mutual gaze enables people to share attention and increase engagement during social interactions through intentional and implicit messages. Although previous studies have explored gaze behaviors and neural mechanisms underlying in-person eye contact, the growing prevalence of remote communication has raised questions about how to establish mutual gaze remotely and how the brains of interacting individuals synchronize.

METHODS

To address these questions, we conducted a study using eye trackers to create a pseudo-mutual gaze channel that mirrors the gazes of each interacting dyad on their respective remote screens. To demonstrate fluctuations in coupling across brains, we incorporated electroencephalographic hyperscanning techniques to simultaneously record the brain activity of interacting dyads engaged in a joint attention task in player-observer, collaborative, and competitive modes.

RESULTS

Our results indicated that mutual gaze could improve the efficiency of joint attention activities among remote partners. Moreover, by employing the phase locking value, we could estimate interbrain synchrony (IBS) and observe low-frequency couplings in the frontal and temporal regions that varied based on the interaction mode. While dyadic gender composition significantly affected gaze patterns, it did not impact the IBS.

CONCLUSION

These results provide insight into the neurological mechanisms underlying remote interaction through the pseudo-mutual gaze channel and have significant implications for developing effective online communication environments.

Inter-Brain Synchrony Pattern Investigation on Triadic Board Game Play-Based Social Interaction: An fNIRS Study

Recent advances in functional neuroimaging techniques, including methodologies such as fNIRS, have enabled the evaluation of inter-brain synchrony (IBS) induced by interpersonal interactions. However, the social interactions assumed in existing dyadic hyperscanning studies do not sufficiently emulate polyadic social interactions in the real world. Therefore, we devised an experimental paradigm that incorporates the Korean folk board game "Yut-nori" to reproduce social interactions that emulate social activities in the real world. We recruited 72 participants aged 25.2 ± 3.9 years (mean ± standard deviation) and divided them into 24 triads to play Yut-nori, following the standard or modified rules. The participants either competed against an opponent (standard rule) or cooperated with an opponent (modified rule) to achieve a goal efficiently. Three different fNIRS devices were employed to record cortical hemodynamic activations in the prefrontal cortex both individually and simultaneously. Wavelet transform coherence (WTC) analyses were performed to assess prefrontal IBS within a frequency range of 0.05-0.2 Hz. Consequently, we observed that cooperative interactions increased prefrontal IBS across overall frequency bands of interest. In addition, we also found that different purposes for cooperation generated different spectral characteristics of IBS depending on the frequency bands. Moreover, IBS in the frontopolar cortex (FPC) reflected the influence of verbal interactions. The findings of our study suggest that future hyperscanning studies should consider polyadic social interactions to reveal the properties of IBS in real-world interactions.

Intra- and inter-brain synchrony oscillations underlying social adjustment

Humans naturally synchronize their behavior with other people. However, although it happens almost automatically, adjusting behavior and conformity to others is a complex phenomenon whose neural mechanisms are still yet to be understood entirely. The present experiment aimed to study the oscillatory synchronization mechanisms underlying automatic dyadic convergence in an EEG hyperscanning experiment. Thirty-six people performed a cooperative decision-making task where dyads had to guess the correct position of a point on a line. A reinforcement learning algorithm was used to model different aspects of the participants' behavior and their expectations of their peers. Intra- and inter-connectivity among electrode sites were assessed using inter-site phase clustering in three main frequency bands (theta, alpha, beta) using a two-level Bayesian mixed-effects modeling approach. The results showed two oscillatory synchronization dynamics related to attention and executive functions in alpha and reinforcement learning in theta. In addition, inter-brain synchrony was mainly driven by beta oscillations. This study contributes preliminary evidence on the phase-coherence mechanism underlying inter-personal behavioral adjustment.

Winners and losers in brain computer interface competitive gaming: Directional connectivity analysis

OBJECTIVE

to characterize the direction within and between brain connectivity in winning and losing players in a competitive brain-computer interface game.

APPROACH

ten dyads (26.9 ± 4.7 years old, eight females and 12 males) participated in the study. In a competitive game based on neurofeedback, they used their relative alpha (RA) band power from the electrode location Pz, to control a virtual seesaw. The players in each pair were separated into winners (W) and losers (L) based on their scores. Intrabrain connectivity was analyzed using multivariate Granger Causality (GC) and Directed Transfer Function, while interbrain connectivity was analyzed using bivariate GC.

RESULTS

linear regression analysis revealed a significant relationship (p<0.05) between RA and individual scores. During the game, W players maintained a higher RA than L players, although it was not higher than their baseline RA. The analysis of intrabrain GC indicated that both groups engaged in general social interactions, but only the W group succeeded in controlling their brain activity at Pz. Group L applied an inappropriate metal strategy, characterized by strong activity in the left frontal cortex, indicative of collaborative gaming. Interbrain GC showed a larger flow of information from the L to the W group, suggesting a higher capability of the W group to monitor the activity of their opponent.

SIGNIFICANCE

both innate neurological indices and gaming mental strategies contribute to game outcomes. Future studies should investigate whether there is a causal relationship between these two factors.

Is there collaboration specific neurophysiological activation during collaborative task activity? An analysis of brain responses using electroencephalography and hyperscanning

Collaboration between two individuals is thought to be associated with the synchrony of two different brain activities. Indeed, prefrontal cortical activation and alpha frequency band modulation has been widely reported, but little is known about interbrain synchrony (IBS) changes occurring during social interaction such as collaboration or competition. In this study, we assess the dynamic of IBS variation in order to provide novel insights into the frequency band modulation underlying collaboration. To address this question, we used electroencephalography (EEG) to simultaneously record the brain activity of two individuals playing a computer-based game facing four different conditions: collaboration, competition, single participation, and passive observation. The computer-based game consisted of a fast button response task. Using data recorded in sensor space, we calculated an IBS value for each frequency band using both wavelet coherence transform and phase-locking value and performed single-subject analysis to compare each condition. We found significant IBS in frontal electrodes only present during collaboration associated with alpha frequency band modulation. In addition, we observed significant IBS in the theta frequency band for both collaboration and competition conditions, along with a significant single-subject cortical activity. Competition is distinguishable through single-subject activity in several regions and frequency bands of the brain. Performance is correlated with single-subject frontal activation during collaboration in the alpha and beta frequency band.

Beyond physical entrainment: competitive and cooperative mental stances during identical joint-action tasks differently affect inter-subjective neural synchrony and judgments of agency

Little work has examined how mental stance alone, apart from physical entrainment, affects between-participant neural synchrony during joint social interaction. We report the first findings on how cooperative and competitive mental stances, even during identical visuomotor joint-action tasks, result in distinct neural oscillatory signatures in low beta and theta band between-participant phase synchrony. Two participants jointly controlled a cursor and were instructed to either compete or cooperate to move it to one of three targets. The visuomotor output was identical for both the compete and cooperate conditions because participants were privately given the same target for experimental trials. Cooperation enhanced theta band between-participant phase-locking value (PLV) midtrial at 1-2 seconds, reflecting activation of systems for social coordination to move the cursor in a shared direction. Competition enhanced low beta between-participant PLV, shifting from temporal to frontal regions, indicating that participants focused only on their target and later evaluated self-agency as winner or loser. This interpretation of the neural signature was corroborated by participants' greater post-trial ratings of the degree of control over the cursor during competition. Top-down cooperative and competitive mental stances shape perceptions of social context and affect interpersonal neural synchrony important for representation of self and others' actions.

Elevated Inter-Brain Coherence Between Subjects With Concordant Stances During Discussion of Social Issues

Social media platforms offer convenient, instantaneous social sharing on a mass scale with tremendous impact on public perceptions, opinions, and behavior. There is a need to understand why information spreads including the human motivations, cognitive processes, and neural dynamics of large-scale sharing. This study introduces a novel approach for investigating the effect social media messaging and in-person discussion has on the inter-brain dynamics within small groups of participants. The psychophysiological impact of information campaigns and narrative messaging within a closed social media environment was assessed using 24-channel wireless EEG. Data were acquired from three- or four-person groups while subjects debated contemporary social issues framed by four scenarios of varying controversy: (a) investing in ethical vs. unethical corporations, (b) selecting travel destination based on social awareness, (c) determining verdict in a murder trial and the punishment of life in prison or death penalty, and (d) decision to vaccinate. Pre-/post-scenario questionnaires assess the effects of the social media information. Inter-brain coherence between subject pairs on each social issue discussed by subjects was analyzed by concordance, agreement vs. disagreement, and by group unanimity, unanimous vs. not unanimous. Subject pairs that agreed on the social issues raised in the scenarios had significantly greater inter-brain coherence in gamma frequency range than disagreeing pairs over cortical regions known to be involved in social interactions. These effects were magnified when comparing groups where subject pairs were unanimous in their stance on the social issues for some but not all scenarios. While there was considerable overlap between scenarios in what EEG channels were significant, there was enough variability to indicate the possibility of scenario-specific effects on inter-brain coherence.

The Presence of Another Person Influences Oscillatory Cortical Dynamics During Dual Brain EEG Recording

Humans are innately social creatures and the social environment strongly influences brain development. As such, the human brain is primed for and sensitive to social information even in the absence of explicit task or instruction. In this study, we examined the influence of different levels of interpersonal proximity on resting state brain activity and its association with social cognition. We measured EEG in pairs of 13 typically developing (TD) adults seated in separate rooms, in the same room back-to-back, and in the same room facing each other. Interpersonal proximity modulated broadband EEG power from 4-55 Hz and individual differences in self-reported social cognition modulated these effects in the beta and gamma frequency bands. These findings provide novel insight into the influence of social environment on brain activity and its association with social cognition through dual-brain EEG recording and demonstrate the importance of using interactive methods to study the human brain.

Hyperscanning: A Valid Method to Study Neural Inter-brain Underpinnings of Social Interaction

Social interactions are a crucial part of human life. Understanding the neural underpinnings of social interactions is a challenging task that the hyperscanning method has been trying to tackle over the last two decades. Here, we review the existing literature and evaluate the current state of the hyperscanning method. We review the type of methods (fMRI, M/EEG, and fNIRS) that are used to measure brain activity from more than one participant simultaneously and weigh their pros and cons for hyperscanning. Further, we discuss different types of analyses that are used to estimate brain networks and synchronization. Lastly, we present results of hyperscanning studies in the context of different cognitive functions and their relations to social interactions. All in all, we aim to comprehensively present methods, analyses, and results from the last 20 years of hyperscanning research.

The Social Situation Affects How We Process Feedback About Our Actions

Humans achieve their goals in joint action tasks either by cooperation or competition. In the present study, we investigated the neural processes underpinning error and monetary rewards processing in such cooperative and competitive situations. We used electroencephalography (EEG) and analyzed event-related potentials (ERPs) triggered by feedback in both social situations. 26 dyads performed a joint four-alternative forced choice (4AFC) visual task either cooperatively or competitively. At the end of each trial, participants received performance feedback about their individual and joint errors and accompanying monetary rewards. Furthermore, the outcome, i.e., resulting positive, negative, or neutral rewards, was dependent on the pay-off matrix, defining the social situation either as cooperative or competitive. We used linear mixed effects models to analyze the feedback-related-negativity (FRN) and used the Threshold-free cluster enhancement (TFCE) method to explore activations of all electrodes and times. We found main effects of the outcome and social situation, but no interaction at mid-line frontal electrodes. The FRN was more negative for losses than wins in both social situations. However, the FRN amplitudes differed between social situations. Moreover, we compared monetary with neutral outcomes in both social situations. Our exploratory TFCE analysis revealed that processing of feedback differs between cooperative and competitive situations at right temporo-parietal electrodes where the cooperative situation elicited more positive amplitudes. Further, the differences induced by the social situations were stronger in participants with higher scores on a perspective taking test. In sum, our results replicate previous studies about the FRN and extend them by comparing neurophysiological responses to positive and negative outcomes in a task that simultaneously engages two participants in competitive and cooperative situations.

Modulation of reward in a live social context as revealed through interactive social neuroscience

Social neuroscience research investigating autism spectrum disorder (ASD) has yielded inconsistent findings, despite ASD being well-characterized by difficulties in social interaction and communication through behavioral observation. In particular, specific etiologies and functional and structural assays of the brain in autism have not been consistently identified. To date, most social neuroscience research has focused on a single person viewing static images. Research utilizing interactive social neuroscience featuring dual-brain recording offers great promise for the study of neurodevelopmental disabilities. Reward processing has been implicated in the pathology of ASD, yet mixed findings have brought uncertainty about the role reward processing deficits may play in ASD. The current study employed dual-brain EEG recording to examine reward processing during live interaction and its relation to autistic traits. Sixteen typically developing (TD) adults played a competitive treasure-hunt game against a computer and against a human partner. EEG results revealed enhanced neural sensitivity to reward outcome during live interaction with a human competitor. Further, individuals with higher levels of autistic traits demonstrated reduced sensitivity to reward outcome during live interaction. These findings provide novel insight into reward processing mechanisms associated with autistic traits, as well as support the necessary utility of interactive social neuroscience techniques to study developmental disorders.

Music Performance As an Experimental Approach to Hyperscanning Studies

Humans are fundamentally social and tend to create emergent organizations when interacting with each other; from dyads to families, small groups, large groups, societies, and civilizations. The study of the neuronal substrate of human social behavior is currently gaining momentum in the young field of social neuroscience. Hyperscanning is a neuroimaging technique by which we can study two or more brains simultaneously while participants interact with each other. The aim of this article is to discuss several factors that we deem important in designing hyperscanning experiments. We first review hyperscanning studies performed by means of electroencephalography (EEG) that have been relying on a continuous interaction paradigm. Then, we provide arguments for favoring ecological paradigms, for studying the emotional component of social interactions and for performing longitudinal studies, the last two aspects being largely neglected so far in the hyperscanning literature despite their paramount importance in social sciences. Based on these premises, we argue that music performance is a suitable experimental setting for hyperscanning and that for such studies EEG is an appropriate choice as neuroimaging modality.

Interpersonal brain synchronization in the right temporo-parietal junction during face-to-face economic exchange

In daily life, interpersonal interactions are influenced by uncertainty about other people's intentions. Face-to-face (FF) interaction reduces such uncertainty by providing external visible cues such as facial expression or body gestures and facilitates shared intentionality to promote belief of cooperative decisions and actual cooperative behaviors in interaction. However, so far little is known about interpersonal brain synchronization between two people engaged in naturally occurring FF interactions. In this study, we combined an adapted ultimatum game with functional near-infrared spectroscopy (fNIRS) hyperscanning to investigate how FF interaction impacts interpersonal brain synchronization during economic exchange. Pairs of strangers interacted repeatedly either FF or face-blocked (FB), while their activation was simultaneously measured in the right temporo-parietal junction (rTPJ) and the control region, right dorsolateral prefrontal cortex (rDLPFC). Behaviorally, FF interactions increased shared intentionality between strangers, leading more positive belief of cooperative decisions and more actual gains in the game. FNIRS results indicated increased interpersonal brain synchronizations during FF interactions in rTPJ (but not in rDLPFC) with greater shared intentionality between partners. These results highlighted the importance of rTPJ in collaborative social interactions during FF economic exchange and warrant future research that combines FF interactions with fNIRS hyperscanning to study social brain disorders such as autism.

An Internet-Based Real-Time Audiovisual Link for Dual MEG Recordings

HYPERSCANNING

Most neuroimaging studies of human social cognition have focused on brain activity of single subjects. More recently, "two-person neuroimaging" has been introduced, with simultaneous recordings of brain signals from two subjects involved in social interaction. These simultaneous "hyperscanning" recordings have already been carried out with a spectrum of neuroimaging modalities, such as functional magnetic resonance imaging (fMRI), electroencephalography (EEG), and functional near-infrared spectroscopy (fNIRS).

DUAL MEG SETUP

We have recently developed a setup for simultaneous magnetoencephalographic (MEG) recordings of two subjects that communicate in real time over an audio link between two geographically separated MEG laboratories. Here we present an extended version of the setup, where we have added a video connection and replaced the telephone-landline-based link with an Internet connection. Our setup enabled transmission of video and audio streams between the sites with a one-way communication latency of about 130 ms. Our software that allows reproducing the setup is publicly available.

VALIDATION

We demonstrate that the audiovisual Internet-based link can mediate real-time interaction between two subjects who try to mirror each others' hand movements that they can see via the video link. All the nine pairs were able to synchronize their behavior. In addition to the video, we captured the subjects' movements with accelerometers attached to their index fingers; we determined from these signals that the average synchronization accuracy was 215 ms. In one subject pair we demonstrate inter-subject coherence patterns of the MEG signals that peak over the sensorimotor areas contralateral to the hand used in the task.

Social neuroscience and hyperscanning techniques: past, present and future

This paper reviews the published literature on the hyperscanning methodologies using hemodynamic or neuro-electric modalities. In particular, we describe how different brain recording devices have been employed in different experimental paradigms to gain information about the subtle nature of human interactions. This review also included papers based on single-subject recordings in which a correlation was found between the activities of different (non-simultaneously recorded) participants in the experiment. The descriptions begin with the methodological issues related to the simultaneous measurements and the descriptions of the results generated by such approaches will follow. Finally, a discussion of the possible future uses of such new approaches to explore human social interactions will be presented.

On the interpretation of synchronization in EEG hyperscanning studies: a cautionary note

EEG Hyperscanning is a method for studying two or more individuals simultaneously with the objective of elucidating how co-variations in their neural activity (i.e., hyperconnectivity) are influenced by their behavioral and social interactions. The aim of this study was to compare the performance of different hyper-connectivity measures using (i) simulated data, where the degree of coupling could be systematically manipulated, and (ii) individually recorded human EEG combined into pseudo-pairs of participants where no hyper-connections could exist. With simulated data we found that each of the most widely used measures of hyperconnectivity were biased and detected hyper-connections where none existed. With pseudo-pairs of human data we found spurious hyper-connections that arose because there were genuine similarities between the EEG recorded from different people independently but under the same experimental conditions. Specifically, there were systematic differences between experimental conditions in terms of the rhythmicity of the EEG that were common across participants. As any imbalance between experimental conditions in terms of stimulus presentation or movement may affect the rhythmicity of the EEG, this problem could apply in many hyperscanning contexts. Furthermore, as these spurious hyper-connections reflected real similarities between the EEGs, they were not Type-1 errors that could be overcome by some appropriate statistical control. However, some measures that have not previously been used in hyperconnectivity studies, notably the circular correlation co-efficient (CCorr), were less susceptible to detecting spurious hyper-connections of this type. The reason for this advantage in performance is discussed and the use of the CCorr as an alternative measure of hyperconnectivity is advocated.

The two-brain approach: how can mutually interacting brains teach us something about social interaction?

Measuring brain activity simultaneously from two people interacting is intuitively appealing if one is interested in putative neural markers of social interaction. However, given the complex nature of interactions, it has proven difficult to carry out two-person brain imaging experiments in a methodologically feasible and conceptually relevant way. Only a small number of recent studies have put this into practice, using fMRI, EEG, or NIRS. Here, we review two main two-brain methodological approaches, each with two conceptual strategies. The first group has employed two-brain fMRI recordings, studying (1) turn-based interactions on the order of seconds, or (2) pseudo-interactive scenarios, where only one person is scanned at a time, investigating the flow of information between brains. The second group of studies has recorded dual EEG/NIRS from two people interacting, in (1) face-to-face turn-based interactions, investigating functional connectivity between theory-of-mind regions of interacting partners, or in (2) continuous mutual interactions on millisecond timescales, to measure coupling between the activity in one person's brain and the activity in the other's brain. We discuss the questions these approaches have addressed, and consider scenarios when simultaneous two-brain recordings are needed. Furthermore, we suggest that (1) quantification of inter-personal neural effects via measures of emergence, and (2) multivariate decoding models that generalize source-specific features of interaction, may provide novel tools to study brains in interaction. This may allow for a better understanding of social cognition as both representation and participation.

MEG dual scanning: a procedure to study real-time auditory interaction between two persons

Social interactions fill our everyday life and put strong demands on our brain function. However, the possibilities for studying the brain basis of social interaction are still technically limited, and even modern brain imaging studies of social cognition typically monitor just one participant at a time. We present here a method to connect and synchronize two faraway neuromagnetometers. With this method, two participants at two separate sites can interact with each other through a stable real-time audio connection with minimal delay and jitter. The magnetoencephalographic (MEG) and audio recordings of both laboratories are accurately synchronized for joint offline analysis. The concept can be extended to connecting multiple MEG devices around the world. As a proof of concept of the MEG-to-MEG link, we report the results of time-sensitive recordings of cortical evoked responses to sounds delivered at laboratories separated by 5 km.