Spontaneous dyadic behaviour predicts the emergence of interpersonal neural synchrony

Exploring the impact of manual and automatic EEG pre-processing methods on interpersonal neural synchrony measures in parent-infant hyperscanning studies

BACKGROUND

Electroencephalograph (EEG) hyperscanning allows studying Interpersonal Neural Synchrony (INS) between two or more individuals across different social conditions, including parent-infant interactions. Signal pre-processing is crucial to optimize computation of INS estimates; however, few attempts have been made at comparing the impact of different dyadic EEG data pre-processing methods on INS estimates.

NEW METHODS

EEG data collected on 31 mother-infant dyads (8-10 months) engaged in a Face-to-Face Still-Face Procedure were pre-processed with two versions of the same pipeline, the "automated" and the "manual". Cross-frequency PLV in the theta (3-5 Hz, 4-7 Hz) and alpha (6-9 Hz, 8-12 Hz) frequency bands were computed after automated and manual pre-processing and compared through Pearson's correlations and Repeated Measures ANOVAs.

RESULTS

PLVs computed in the theta, but not alpha, frequency band were significantly higher after automated pre-processing than after manual pre-processing. Moreover, the automated pipeline rejected a significantly lower percentage of ICs and epochs compared to the manual pipeline.

COMPARISON WITH EXISTING METHODS

While no direct comparison with existing dyadic EEG data pre-processing pipelines was made, this is the first study assessing the impact of different methodological decisions, particularly of the degree of pre-processing automatization, on cross-frequency PLV computed on a dataset of parent-infant dyads.

CONCLUSIONS

Non-directional phase-based INS indexes such as the PLV seem to be affected by the degree of automatization of the pre-processing pipeline. Future research should strive for standardization of dyadic EEG pre-processing methods.

Two-brain microstates: A novel hyperscanning-EEG method for quantifying task-driven inter-brain asymmetry

BACKGROUND

The neural mechanisms underlying real-time social interaction remain poorly understood. While hyperscanning has emerged as a popular method to better understand inter-brain mechanisms, inter-brain methods remain underdeveloped, and primarily focused on inter-brain synchronization (IBS).

NEW METHOD

We developed a novel approach employing two-brain EEG microstates, to investigate neural mechanisms during symmetric and asymmetric interactive tasks. Microstates are quasi-stable configurations of brain activity that have been proposed to represent basic building blocks for mental processing. Expanding the microstate methodology to dyads of interacting participants enables us to investigate quasi-stable moments of inter-brain synchronous and asymmetric activity.

RESULTS

Conventional microstates fitted to individuals were not related to the different interactive conditions. However, two-brain microstates were modulated in the observer-actor condition, compared to all other conditions where participants had more symmetric task demands, and the same trend was observed for the follower-leader condition. This indicates differences in resting state default-mode network activity during interactions with asymmetric tasks.

COMPARISON WITH EXISTING METHODS

Hyperscanning studies have primarily estimated IBS based on functional connectivity measures. However, localized connections are often hard to interpret on a larger scale when multiple connections across brains are found to be important. Two-brain microstates offer an alternative approach to evaluate neural activity from a large-scale global network perspective, by quantifying task-driven asymmetric neural states between interacting individuals.

CONCLUSIONS

We present a novel method using two-brain microstates, including open-source code, which expands the current hyperscanning-EEG methodology to measure and potentially identify both synchronous and asymmetric inter-brain states during real-time social interaction.

Interpersonal synchrony in dance/movement therapy: Neural underpinnings for individuals with dementia

Rising global levels of dementia including Alzheimer's disease call for the treatment of both cognitive and psychosocial deficits of this population. While there is no cure for dementia, the progression can be slowed, and symptoms eased. The positive effects of exercise and dance have been documented as has interpersonal synchrony. Dance/movement therapy uses kinesthetic empathy, attunement, and mirroring to communicate, synchronize, and connect with clients, salient for a population that often struggles with loneliness and isolation. Here I offer a perspective on how dance/movement therapy promotes the social functions and neural underpinning of interpersonal synchrony, possibly providing neuroprotection for this population.

Words as social tools (WAT): A reprise

The paper presents new evidence collected in the last five years supporting the Words As social Tools proposal on abstract concepts. We discuss findings revolving around three central tenets. First, we show that-like concrete concepts-also abstract concepts evoke sensorimotor experiences, even if to a lower extent, and that they are linked to inner experiences (e.g., interoceptive, proprioceptive, and metacognitive). Second, we present findings suggesting that linguistic and social interaction are crucial for acquiring and using abstract concepts. Specifically, rating and behavioral studies reveal that people tend to feel uncertain about the meaning of abstract concepts. On top of that, with abstract concepts, people rely more on others to ask for information, negotiate conceptual meaning, or outsource their knowledge. We propose that inner speech might contribute both to the monitoring process and the preparation to interact with others. Finally, we illustrate recent studies conducted in our lab highlighting abstract concepts variability across individuals (age, expertise), cultures, and languages.

Relational neuroscience: Insights from hyperscanning research

Humans are highly social, typically without this ability requiring noticeable efforts. Yet, such social fluency poses challenges both for the human brain to compute and for scientists to study. Over the last few decades, neuroscientific research of human sociality has witnessed a shift in focus from single-brain analysis to complex dynamics occurring across several brains, posing questions about what these dynamics mean and how they relate to multifaceted behavioural models. We propose the term 'Relational Neuroscience' to collate the interdisciplinary research field devoted to modelling the inter-brain dynamics subserving human connections, spanning from real-time joint experiences to long-term social bonds. Hyperscanning, i.e., simultaneously measuring brain activity from multiple individuals, has proven to be a highly promising technique to investigate inter-brain dynamics. Here, we discuss how hyperscanning can help investigate questions within the field of Relational Neuroscience, considering a variety of subfields, including cooperative interactions in dyads and groups, empathy, attachment and bonding, and developmental neuroscience. While presenting Relational Neuroscience in the light of hyperscanning, our discussion also takes into account behaviour, physiology and endocrinology to properly interpret inter-brain dynamics within social contexts. We consider the strengths but also the limitations and caveats of hyperscanning to answer questions about interacting people. The aim is to provide an integrative framework for future work to build better theories across a variety of contexts and research subfields to model human sociality.

2-Brain Regulation for Improved Neuroprotection during Early Development (2-BRAINED): a translational hyperscanning research project

Introduction

Very preterm (VPT) birth is a major risk condition for child development and parental wellbeing, mainly due to multiple sources of stress (e.g., separation and pain exposure) during the neonatal intensive care unit (NICU) stay. Early video-feedback (VF) interventions proved effective in promoting VOT infants' development and parental wellbeing. Electroencephalography (EEG) hyperscanning allows the assessment of brain-to-brain co-regulation during live interaction between infants and parents, offering promising insights into the mechanisms behind the interactive benefits of early VF interventions.

Goals

This study aimed to compare indices of brain-to-brain co-regulation between dyads of full-term (FT) and VPT infants interacting with their mothers and investigate the effect of an early post-discharge VF intervention on the brain-to-brain co-regulation indices of VPT dyads.

Methods and analysis

VPT and FT dyads will be enrolled at birth, and the former will be randomly allocated to one of two arms: VF intervention or care as usual. Short-term effectiveness will be assessed through ratings of mother-infant interaction videotaped before and after the VF intervention or care as usual. Mothers of VPT and FT infants will report on their mental state, parenting stress and bonding, and infant temperament and sensory profile at 3 and 6 months (corrected age, CA). At 9 months CA, all dyads will participate in a lab-based EEG-hyperscanning paradigm to assess brain-to-brain co-regulation through phase-locking value (PLV) and other explorative indices.

Ethics and dissemination

This study was funded by the Italian Ministry of Health and received approval by the Ethics Committee of Pavia (Italy) and participating hospitals. Research findings will be reported in scientific publications, presented at international conferences, and disseminated to the general public.

Study registration number

GR-2021-12375213 (Italian Ministry of Health registry).

How accurately can we estimate spontaneous body kinematics from video recordings? Effect of movement amplitude on OpenPose accuracy

Estimating how the human body moves in space and time-body kinematics-has important applications for industry, healthcare, and several research fields. Gold-standard methodologies capturing body kinematics are expensive and impractical for naturalistic recordings as they rely on infrared-reflective wearables and bulky instrumentation. To overcome these limitations, several algorithms have been developed to extract body kinematics from plain video recordings. This comes with a drop in accuracy, which however has not been clearly quantified. To fill this knowledge gap, we analysed a dataset comprising 46 human participants exhibiting spontaneous movements of varying amplitude. Body kinematics were estimated using OpenPose (video-based) and Vicon (infrared-based) motion capture systems simultaneously. OpenPose accuracy was assessed using Vicon estimates as ground truth. We report that OpenPose accuracy is overall moderate and varies substantially across participants and body parts. This is explained by variability in movement amplitude. OpenPose estimates are weak for low-amplitude movements. Conversely, large-amplitude movements (i.e., > ~ 10 cm) yield highly accurate estimates. The relationship between accuracy and movement amplitude is not linear (but mostly exponential or power) and relatively robust to camera-body distance. Together, these results dissect the limits of video-based motion capture and provide useful guidelines for future studies.

Emotions in multi-brain dynamics: A promising research frontier

Emotions drive and influence social interactions. Actions and reactions driven by emotions are dynamically modulated by continuous feedback loops between all interacting subjects. In this framework, interacting brains operate as an integrated system, with neural dynamics coevolving over time. Neuronal synchronization across brains has been observed in a range of species, including humans, monkeys, bats, and mice. This inter-neural synchrony (INS) has been proposed as a potential mechanism facilitating social interaction by enabling the functional integration of multiple brains. However, the role of emotions in modulating these processes remains underexplored and warrants further investigation. Here we provide a brief overview of studies on inter-neural synchrony in humans and other species, emphasizing the critical role that emotions might play in shaping multibrain dynamics.

Synchrony Across Brains

Second-person neuroscience focuses on studying the behavioral and neuronal mechanisms of real-time social interactions within single and across interacting brains. In this review article, we describe the developments that have been undertaken to study socially interactive phenomena and the behavioral and neurobiological processes that extend across interaction partners. More specifically, we focus on the role that synchrony across brains plays in enabling and facilitating social interaction and communication and in shaping social coordination and learning, and we consider how reduced synchrony across brains may constitute a core feature of psychopathology.

Hyperscanning: from inter-brain coupling to causality

In hyperscanning studies, participants perform a joint task while their brain activation is simultaneously recorded. Evidence of inter-brain coupling is examined, in these studies, as a predictor of behavioral change. While the field of hyperscanning has made significant strides in unraveling the associations between inter-brain coupling and changes in social interactions, drawing causal conclusions between brain and behavior remains challenging. This difficulty arises from factors like the inherently different timescales of behavioral responses and measured cerebral activity, as well as the predominant focus of existing methods on associations rather than causality. Specifically, a question remains as to whether inter-brain coupling between specific brain regions leads to changes in behavioral synchrony, or vice-versa. We propose two novel approaches to addressing this question. The first method involves using dyadic neurofeedback, wherein instances of inter-brain coupling are directly reinforced. Such a system could examine if continuous changes of inter-brain coupling are the result of deliberate mutual attempts to synchronize. The second method employs statistical approaches, including Granger causality and Structural Equation Modeling (SEM). Granger causality assesses the predictive influence of one time series on another, enabling the identification of directional neural interactions that drive behavior. SEM allows for detailed modeling of both direct and indirect effects of inter-brain coupling on behavior. We provide an example of data analysis with the SEM approach, discuss the advantages and limitations of each approach and posit that applying these approaches could provide significant insights into how inter-brain coupling supports crucial processes that occur in social interactions.

Capturing spontaneous interactivity: a multi-measure approach to analyzing the dynamics of interpersonal coordination in dance improvisation

Introduction

Interpersonal coordination is widely acknowledged as critical to relating with, connecting to, and understanding others, but the underlying mechanisms of this phenomenon are poorly understood. Dance-particularly improvised dance-offers a valuable paradigm for investigating the dynamics of interpersonal coordination due to its inherent ability to connect us. However, conventional approaches to studying coordination often fail to capture the co-creative spontaneity that is intrinsic to such interactions.

Methods

This study combined multiple measures of interpersonal coordination to detect moments of high coordination between two freely improvising dancers. We applied maximum correlation vectors, normalized Symbolic Transfer Entropy (NSTE), and surveys to analyze the time-varying dynamics of similarity in movement speeds, directed influence, and subjective perception of dancers engaged in an improvisation task.

Results

This multi-measure approach offers a means of capturing the interplay between different dimensions of interpersonal coordination.

Discussion

This approach may be used to understand the underlying mechanisms of co-creative social interactions in improvised dance and other forms of spontaneous interactivity.

Embodied hyperscanning for studying social interaction: A scoping review of simultaneous brain and body measurements

We systematically investigated the application of embodied hyperscanning methodologies in social neuroscience research. Hyperscanning enables the simultaneous recording of neurophysiological and physiological signals from multiple participants. We highlight the trend toward integrating Mobile Brain/Body Imaging (MoBI) within the 4E research framework, which emphasizes the interconnectedness of brain, body, and environment. Our analysis revealed a geographic concentration of studies in the Global North, calling for global collaboration and transcultural research to balance the field. The predominant use of Magneto/Electroencephalogram (M/EEG) in these studies suggests a traditional brain-centric perspective in social neuroscience. Future research directions should focus on integrating diverse techniques to capture the dynamic interplay between brain and body functions in real-world contexts. Our review also finds a preference for tasks involving natural settings. Nevertheless, the analysis in hyperscanning studies is often limited to physiological signal synchrony between participants. This suggests a need for more holistic and complex approaches that combine inter-corporeal synchrony with intra-individual measures. We believe that the future of the neuroscience of relationships lies in embracing the complexity of cognition, integrating diverse methods and theories to enrich our grasp of human social behavior in its natural contexts.

Revisiting the earliest hyperscanning study: power and functional connectivity in the alpha band may link brains far apart

This brief report revisits the earliest known hyperscanning study published in 1965, which examined simultaneous EEG recordings of identical twins separated by six meters. The original study's findings suggested that eye closure in one twin elicited alpha activity in the other, despite physical separation. Leveraging contemporary signal processing techniques, we reanalyzed the digitized data to validate their findings. Spectral analysis confirmed alpha activity in the twins' EEG signals, aligning with the original observations. Multitapering along with background noise subtraction also revealed the alpha activity in the unrelated subject, which could not be observed by visual inspection alone. Coherence analysis revealed significant alpha band synchrony between a twin and an unrelated subject, differing from the initial study's conclusions. Our findings indicate that even historical data can yield new insights when revisited with contemporary data analysis tools and highlight the potential for future large-scale studies using advanced techniques to explore nonlocal brain interactions.

Spontaneous movement synchrony as an exogenous source for interbrain synchronization in cooperative learning

Learning through cooperation with conspecifics-'cooperative learning'-is critical to cultural evolution and survival. Recent progress has established that interbrain synchronization (IBS) between individuals predicts success in cooperative learning. However, the likely sources of IBS during learning interactions remain poorly understood. To address this dearth of knowledge, we tested whether movement synchrony serves as an exogenous factor that drives IBS, taking an embodiment perspective. We formed dyads of individuals with varying levels of prior knowledge (high-high (HH), high-low (HL), low-low (LL) dyads) and instructed them to collaboratively analyse an ancient Chinese poem. During the task, we simultaneously recorded their brain activity using functional near-infrared spectroscopy and filmed the entire experiment to parse interpersonal movement synchrony using the computer-vision motion energy analysis. Interestingly, the homogeneous groups (HH and/or LL) exhibited stronger movement synchrony and IBS compared with the heterogeneous group. Importantly, mediation analysis revealed that spontaneous and synchronized body movements between individuals contribute to IBS, hence facilitating learning. This study therefore fills a critical gap in our understanding of how interpersonal transmission of information between individual brains, associated with behavioural entrainment, shapes social learning. This article is part of the theme issue 'Minds in movement: embodied cognition in the age of artificial intelligence'.

Meeting the multidimensional self: fostering selfhood at the interface of Creative Arts Therapies and neuroscience

Intriguing explorations at the intersection of the fields of neuroscience and psychology are driven by the quest to understand the neural underpinnings of "the self" and their psychotherapeutic implications. These translational efforts pertain to the unique Creative Arts Therapies (CATs) and the attributes and value of the self-related processes they offer. The self is considered as a multi-layered complex construct, comprising bodily and mental constituents, subjective-objective perspectives, spatial and temporal dimensions. Neuroscience research, mostly functional brain imaging, has proposed cogent models of the constitution, development and experience of the self, elucidating how the multiple dimensions of the self are supported by integrated hierarchical brain processes. The psychotherapeutic use of the art-forms, generating aesthetic experiences and creative processes, touch upon and connect the various layers of self-experience, nurturing the sense of self. The present conceptual analysis will describe and interweave the neural mechanisms and neural network configuration suggested to lie at the core of the ongoing self-experience, its deviations in psychopathology, and implications regarding the psychotherapeutic use of the arts. The well-established, parsimonious and neurobiologically plausible predictive processing account of brain-function will be discussed with regard to selfhood and consciousness. The epistemic affordance of the experiential CATs will further be portrayed, enabling and facilitating the creation of updated self-models of the body in the world. The neuropsychological impact of the relational therapeutic encounter will be delineated, acknowledging the intersubjective brain synchronization through communicative verbal and non-verbal means and aesthetic experiences. The recognition and assimilation of neuroscientific, phenomenological and clinical perspectives concerning the nested dimensionality of the self, ground the relational therapeutic process and the neuroplastic modulations that CATs have to offer on the premise of fostering, shaping and integrating selfhood.

Bringing together multimodal and multilevel approaches to study the emergence of social bonds between children and improve social AI

This protocol paper outlines an innovative multimodal and multilevel approach to studying the emergence and evolution of how children build social bonds with their peers, and its potential application to improving social artificial intelligence (AI). We detail a unique hyperscanning experimental framework utilizing functional near-infrared spectroscopy (fNIRS) to observe inter-brain synchrony in child dyads during collaborative tasks and social interactions. Our proposed longitudinal study spans middle childhood, aiming to capture the dynamic development of social connections and cognitive engagement in naturalistic settings. To do so we bring together four kinds of data: the multimodal conversational behaviors that dyads of children engage in, evidence of their state of interpersonal rapport, collaborative performance on educational tasks, and inter-brain synchrony. Preliminary pilot data provide foundational support for our approach, indicating promising directions for identifying neural patterns associated with productive social interactions. The planned research will explore the neural correlates of social bond formation, informing the creation of a virtual peer learning partner in the field of Social Neuroergonomics. This protocol promises significant contributions to understanding the neural basis of social connectivity in children, while also offering a blueprint for designing empathetic and effective social AI tools, particularly for educational contexts.

Post-interaction neuroplasticity of inter-brain networks underlies the development of social relationship

Inter-brain coupling has been increasingly recognized for its role in supporting connectedness during social communication. Here we investigate whether inter-brain coupling is plastic and persists beyond the offset of social interaction, facilitating the emergence of social closeness. Dyads were concurrently scanned using functional near infrared spectroscopy (fNIRS) while engaging in a task that involved movement synchronization. To assess post-interaction neuroplasticity, participants performed a baseline condition with no interaction before and after the interaction. The results reveal heightened inter-brain coupling in neural networks comprising the inferior frontal gyrus (IFG) and dorsomedial prefrontal cortex in the post-task compared to the pre-task baseline. Critically, the right IFG emerged as a highly connected hub, with post-task inter-brain coupling in this region predicting the levels of motivation to connect socially. We suggest that post-interactions inter-brain coupling may reflect consolidation of socially related cues, underscoring the role of inter-brain plasticity in fundamental aspects of relationship development.

Dynamic mutual predictions during social learning: A computational and interbrain model

During social interactions, we constantly learn about the thoughts, feelings, and personality traits of our interaction partners. Learning in social interactions is critical for bond formation and acquiring knowledge. Importantly, this type of learning is typically bi-directional, as both partners learn about each other simultaneously. Here we review the literature on social learning and propose a new computational and neural model characterizing mutual predictions that take place within and between interactions. According to our model, each partner in the interaction attempts to minimize the prediction error of the self and the interaction partner. In most cases, these inferential models become similar over time, thus enabling mutual understanding to develop. At the neural level, this type of social learning may be supported by interbrain plasticity, defined as a change in interbrain coupling over time in neural networks associated with social learning, among them the mentalizing network, the observation-execution system, and the hippocampus. The mutual prediction model constitutes a promising means of providing empirically verifiable accounts of how relationships develop over time.

Communicative signals during joint attention promote neural processes of infants and caregivers

Communicative signals such as eye contact increase infants' brain activation to visual stimuli and promote joint attention. Our study assessed whether communicative signals during joint attention enhance infant-caregiver dyads' neural responses to objects, and their neural synchrony. To track mutual attention processes, we applied rhythmic visual stimulation (RVS), presenting images of objects to 12-month-old infants and their mothers (n = 37 dyads), while we recorded dyads' brain activity (i.e., steady-state visual evoked potentials, SSVEPs) with electroencephalography (EEG) hyperscanning. Within dyads, mothers either communicatively showed the images to their infant or watched the images without communicative engagement. Communicative cues increased infants' and mothers' SSVEPs at central-occipital-parietal, and central electrode sites, respectively. Infants showed significantly more gaze behaviour to images during communicative engagement. Dyadic neural synchrony (SSVEP amplitude envelope correlations, AECs) was not modulated by communicative cues. Taken together, maternal communicative cues in joint attention increase infants' neural responses to objects, and shape mothers' own attention processes. We show that communicative cues enhance cortical visual processing, thus play an essential role in social learning. Future studies need to elucidate the effect of communicative cues on neural synchrony during joint attention. Finally, our study introduces RVS to study infant-caregiver neural dynamics in social contexts.

Mobile fNIRS for exploring inter-brain synchrony across generations and time

While still relatively rare, longitudinal hyperscanning studies are exceptionally valuable for documenting changes in inter-brain synchrony, which may in turn underpin how behaviors develop and evolve in social settings. The generalizability and ecological validity of this experimental approach hinges on the selected imaging technique being mobile-a requirement met by functional near-infrared spectroscopy (fNIRS). fNIRS has most frequently been used to examine the development of inter-brain synchrony and behavior in child-parent dyads. In this position paper, we contend that dedicating attention to longitudinal and intergenerational hyperscanning stands to benefit the fields of social and cognitive neuroscience more broadly. We argue that this approach is particularly relevant for understanding the neural mechanisms underpinning intergenerational social dynamics, and potentially for benchmarking progress in psychological and social interventions, many of which are situated in intergenerational contexts. In line with our position, we highlight areas of intergenerational research that stand to be enhanced by longitudinal hyperscanning with mobile devices, describe challenges that may arise from measuring across generations in the real world, and offer potential solutions.