Dynamics of Social Interaction: Kinematic Analysis of a Joint Action

Recognition of emotional body language from dyadic and monadic point-light displays in 5-year-old children and adults

Most child studies on emotion perception used faces and speech as emotion stimuli, but little is known about children's perception of emotions conveyed by body movements, that is, emotional body language (EBL). This study aimed to investigate whether processing advantages for positive emotions in children and negative emotions in adults found in studies on emotional face and term perception also occur in EBL perception. We also aimed to uncover which specific movement features of EBL contribute to emotion perception from interactive dyads compared with noninteractive monads in children and adults. We asked 5-year-old children and adults to categorize happy and angry point-light displays (PLDs), presented as pairs (dyads) and single actors (monads), in a button-press task. By applying representational similarity analyses, we determined intra- and interpersonal movement features of the PLDs and their relation to the participants' emotional categorizations. Results showed significantly higher recognition of happy PLDs in 5-year-olds and of angry PLDs in adults in monads but not in dyads. In both age groups, emotion recognition depended significantly on kinematic and postural movement features such as limb contraction and vertical movement in monads and dyads, whereas in dyads recognition also relied on interpersonal proximity measures such as interpersonal distance. Thus, EBL processing in monads seems to undergo a similar developmental shift from a positivity bias to a negativity bias, as was previously found for emotional faces and terms. Despite these age-specific processing biases, children and adults seem to use similar movement features in EBL processing.

Effects of the Interactive Features of Virtual Partner on Individual Exercise Level and Exercise Perception

BACKGROUND

We designed an exercise system in which the user is accompanied by a virtual partner (VP) and tested bodyweight squat performance with different interactive VP features to explore the comprehensive impact of these VP features on the individual's exercise level (EL) and exercise perception.

METHODS

This experiment used three interactive features of VP, including body movement (BM), eye gaze (EG), and sports performance (SP), as independent variables, and the exercise level (EL), subjective exercise enjoyment, attitude toward the team formed with the VP, and local muscle fatigue degree of the exerciser as observational indicators. We designed a 2 (with or without VP's BM) × 2 (with or without VP's EG) × 2 (with or without VP's SP) within-participants factorial experiment. A total of 40 college students were invited to complete 320 groups of experiments.

RESULTS

(1) Regarding EL, the main effects of BM and SP were significant (p < 0.001). The pairwise interaction effects of the three independent variables on EL were all significant (p < 0.05). (2) Regarding exercise perception, the main effects of BM (p < 0.001) and EG (p < 0.001) on subjective exercise enjoyment were significant. The main effect of BM on the attitude toward the sports team formed with the VP was significant (p < 0.001). The interaction effect of BM and SP on the attitude toward the sports team formed with the VP was significant (p < 0.001). (3) Regarding the degree of local muscle fatigue, the main effects of BM, EG, and SP and their interaction effects were not significant (p > 0.05).

CONCLUSION

BM and EG from the VP elevate EL and exercise perception during squat exercises, while the VP with SP inhibited the EL and harmed exercise perception. The conclusions of this study can provide references to guide the interactive design of VP-accompanied exercise systems.

Decoding spatiotemporal features of emotional body language in social interactions

How are emotions perceived through human body language in social interactions? This study used point-light displays of human interactions portraying emotional scenes (1) to examine quantitative intrapersonal kinematic and postural body configurations, (2) to calculate interaction-specific parameters of these interactions, and (3) to analyze how far both contribute to the perception of an emotion category (i.e. anger, sadness, happiness or affection) as well as to the perception of emotional valence. By using ANOVA and classification trees, we investigated emotion-specific differences in the calculated parameters. We further applied representational similarity analyses to determine how perceptual ratings relate to intra- and interpersonal features of the observed scene. Results showed that within an interaction, intrapersonal kinematic cues corresponded to emotion category ratings, whereas postural cues reflected valence ratings. Perception of emotion category was also driven by interpersonal orientation, proxemics, the time spent in the personal space of the counterpart, and the motion–energy balance between interacting people. Furthermore, motion–energy balance and orientation relate to valence ratings. Thus, features of emotional body language are connected with the emotional content of an observed scene and people make use of the observed emotionally expressive body language and interpersonal coordination to infer emotional content of interactions.

Coordination Dynamics: A Foundation for Understanding Social Behavior

Humans' interactions with each other or with socially competent machines exhibit lawful coordination patterns at multiple levels of description. According to Coordination Dynamics, such laws specify the flow of coordination states produced by functional synergies of elements (e.g., cells, body parts, brain areas, people…) that are temporarily organized as single, coherent units. These coordinative structures or synergies may be mathematically characterized as informationally coupled self-organizing dynamical systems (Coordination Dynamics). In this paper, we start from a simple foundation, an elemental model system for social interactions, whose behavior has been captured in the Haken-Kelso-Bunz (HKB) model. We follow a tried and tested scientific method that tightly interweaves experimental neurobehavioral studies and mathematical models. We use this method to further develop a body of empirical research that advances the theory toward more generalized forms. In concordance with this interdisciplinary spirit, the present paper is written both as an overview of relevant advances and as an introduction to its mathematical underpinnings. We demonstrate HKB's evolution in the context of social coordination along several directions, with its applicability growing to increasingly complex scenarios. In particular, we show that accommodating for symmetry breaking in intrinsic dynamics and coupling, multiscale generalization and adaptation are principal evolutions. We conclude that a general framework for social coordination dynamics is on the horizon, in which models support experiments with hypothesis generation and mechanistic insights.

The Human Dynamic Clamp Reveals the Fronto-Parietal Network Linking Real-Time Social Coordination and Cognition

How does the brain allow us to interact with others? Social neuroscience has already provided some answers to these questions but has tended to treat high-level, cognitive interpretations of social behavior separately from the sensorimotor mechanisms upon which they rely. The goal here is to identify the underlying neural processes and mechanisms linking sensorimotor coordination and intention attribution. We combine the human dynamic clamp, a novel paradigm for studyingrealistic social behavior, with high-resolution electroencephalography. The collection of humanness and intention attribution reports, kinematics, and neural data affords an opportunity to relate brain activity to the ongoing social behavior. Behavioral results demonstrate that sensorimotor coordination influences the judgments of cooperativeness and humanness. Analysis of brain dynamics reveals two distinct networks related to the integration of visuo-motor information from self and other which overlap over the right parietal region. Furthermore, judgment of humanness and cooperation of others modulate the functional connectivity between this right parietal hub and the prefrontal cortex. These results reveal how distributed neural dynamics integrates information from "low-level" sensorimotor mechanisms and "high-level" social cognition to support the realistic social behaviors that play out in real time during interactive scenarios.

Midline frontal and occipito-temporal activity during error monitoring in dyadic motor interactions

Discrepancies between sensory predictions and action outcome are at the base of error coding. However, these phenomena have mainly been studied focussing on individual performance. Here, we explored EEG responses to motor prediction errors during a human-avatar interaction and show that Theta/Alpha activity of the frontal error-monitoring system works in phase with activity of the occipito-temporal node of the action observation network. Our motor interaction paradigm required healthy individuals to synchronize their reach-to-grasp movements with those of a virtual partner in conditions that did (Interactive) or did not require (Cued) movement prediction and adaptation to the partner's actions. Crucially, in 30% of the trials the virtual partner suddenly and unpredictably changed its movement trajectory thereby violating the human participant's expectation. These changes elicited error-related neuromarkers (ERN/Pe - Theta/Alpha modulations) over fronto-central electrodes during the Interactive condition. Source localization and connectivity analyses showed that the frontal Theta/Alpha activity induced by violations of the expected interactive movements was in phase with occipito-temporal Theta/Alpha activity. These results expand current knowledge about the neural correlates of on-line interpersonal motor interactions linking the frontal error-monitoring system to visual, body motion-related, responses.

Role of the occipito-temporal theta rhythm in hand visual identification

Neuroimaging and EEG studies have shown that passive observation of the full body and of specific body parts is associated with 1) activity of an occipito-temporal region named the extrastriate body area (EBA), 2) amplitude modulations of a specific posterior event-related potential (ERP) component (N1/N190), and 3) a theta-band (4–7 Hz) synchronization recorded from occipito-temporal electrodes compatible with the location of EBA. To characterize the functional role of the occipito-temporal theta-band increase during the processing of body-part stimuli, we recorded EEG from healthy participants while they were engaged in an identification task (match-to-sample) of images of hands and nonbody control images (leaves). In addition to confirming that occipito-temporal electrodes show a larger N1 for hand images compared with control stimuli, cluster-based analysis revealed an occipito-temporal cluster showing an increased theta power when hands are presented (compared with leaves) and show that this theta increase is higher for identified hands compared with nonidentified ones while not being significantly different between not identified nonhand stimuli. Finally, single trial multivariate pattern analysis revealed that time-frequency modulation in the theta band is a better marker for classifying the identification of hand images than the ERP modulation. The present results support the notion that theta activity over the occipito-temporal cortex is an informative marker of hand visual processing and may reflect the activity of a network coding for stimulus identity.

NEW & NOTEWORTHY Hands provide crucial information regarding the identity of others, which is a key information for social processes. We recorded EEG activity of healthy participants during the visual identification of hand images. The combination of univariate and multivariate pattern analysis in time- and time-frequency domain highlights the functional role of theta (4–7 Hz) activity over visual areas during hand identification and emphasizes the robustness of this neuromarker in occipito-temporal visual processing dynamics.

Visuo-motor interference with a virtual partner is equally present in cooperative and competitive interactions

Automatic imitation of observed actions is thought to be a powerful mechanism, one that may mediate the reward value of interpersonal interactions, but that could also generate visuo-motor interference when interactions involve complementary movements. Since interpersonal coordination seems to be crucial both when cooperating and competing with others, the questions arises as to whether imitation-and thus visuo-motor interference-occurs in both scenarios. To address this issue, we asked human participants to engage in high- or low-interactive (Interactive or Cued condition, respectively), cooperative or competitive, joint reach-to-grasps with a virtual partner. More specifically, interactions occurred in: (i) a Cued condition, where participants simply adapted their movement timing to synchronize with (during cooperation) or anticipate (during competition) the virtual partner's grasp; (ii) an Interactive condition requiring the same adaptation, as well as a real-time selection of their action according to the virtual character's movement. To simulate a realistic human-human interaction, the virtual character would change its movement speed in consecutive trials according to participants' behaviour. Results demonstrate that visuo-motor interference-as indexed by movement kinematics (higher maximum wrist height during complementary compared to imitative power grips)-emerge in both cooperative and competitive motor interactions only when predictions about the partner's movements are needed to perform one's own action (interactive condition). These results support the idea that simulative imitation is heavily present when individuals need to match their behaviours closely.