The Body Action Coding System I: Muscle activations during the perception and expression of emotion

Multi-view emotional expressions dataset using 2D pose estimation

Human body expressions convey emotional shifts and intentions of action and, in some cases, are even more effective than other emotion models. Despite many datasets of body expressions incorporating motion capture available, there is a lack of more widely distributed datasets regarding naturalized body expressions based on the 2D video. In this paper, therefore, we report the multi-view emotional expressions dataset (MEED) using 2D pose estimation. Twenty-two actors presented six emotional (anger, disgust, fear, happiness, sadness, surprise) and neutral body movements from three viewpoints (left, front, right). A total of 4102 videos were captured. The MEED consists of the corresponding pose estimation results (i.e., 397,809 PNG files and 397,809 JSON files). The size of MEED exceeds 150 GB. We believe this dataset will benefit the research in various fields, including affective computing, human-computer interaction, social neuroscience, and psychiatry.

Construction and validation of the Dalian emotional movement open-source set (DEMOS)

Human body movements are important for emotion recognition and social communication and have received extensive attention from researchers. In this field, emotional biological motion stimuli, as depicted by point-light displays, are widely used. However, the number of stimuli in the existing material library is small, and there is a lack of standardized indicators, which subsequently limits experimental design and conduction. Therefore, based on our prior kinematic dataset, we constructed the Dalian Emotional Movement Open-source Set (DEMOS) using computational modeling. The DEMOS has three views (i.e., frontal 0°, left 45°, and left 90°) and in total comprises 2664 high-quality videos of emotional biological motion, each displaying happiness, sadness, anger, fear, disgust, and neutral. All stimuli were validated in terms of recognition accuracy, emotional intensity, and subjective movement. The objective movement for each expression was also calculated. The DEMOS can be downloaded for free from https://osf.io/83fst/ . To our knowledge, this is the largest multi-view emotional biological motion set based on the whole body. The DEMOS can be applied in many fields, including affective computing, social cognition, and psychiatry.

The McNorm library: creating and validating a new library of emotionally expressive whole body dance movements

The ability to exchange affective cues with others plays a key role in our ability to create and maintain meaningful social relationships. We express our emotions through a variety of socially salient cues, including facial expressions, the voice, and body movement. While significant advances have been made in our understanding of verbal and facial communication, to date, understanding of the role played by human body movement in our social interactions remains incomplete. To this end, here we describe the creation and validation of a new set of emotionally expressive whole-body dance movement stimuli, named the Motion Capture Norming (McNorm) Library, which was designed to reconcile a number of limitations associated with previous movement stimuli. This library comprises a series of point-light representations of a dancer's movements, which were performed to communicate to observers neutrality, happiness, sadness, anger, and fear. Based on results from two validation experiments, participants could reliably discriminate the intended emotion expressed in the clips in this stimulus set, with accuracy rates up to 60% (chance = 20%). We further explored the impact of dance experience and trait empathy on emotion recognition and found that neither significantly impacted emotion discrimination. As all materials for presenting and analysing this movement library are openly available, we hope this resource will aid other researchers in further exploration of affective communication expressed by human bodily movement.

Three- and six-year-old children are sensitive to natural body expressions of emotion: An event-related potential emotional priming study

Body movements provide a rich source of emotional information during social interactions. Although the ability to perceive biological motion cues related to those movements begins to develop during infancy, processing those cues to identify emotions likely continues to develop into childhood. Previous studies used posed or exaggerated body movements, which might not reflect the kind of body expressions children experience. The current study used an event-related potential (ERP) priming paradigm to investigate the development of emotion recognition from more naturalistic body movements. Point-light displays (PLDs) of male adult bodies expressing happy or angry emotional movements while narrating a story were used as prime stimuli, whereas audio recordings of the words "happy" and "angry" spoken with an emotionally neutral prosody were used as targets. We recorded the ERPs time-locked to the onset of the auditory target from 3- and 6-year-old children, and we compared amplitude and latency of the N300 and N400 responses between the two age groups in the different prime-target conditions. There was an overall effect of prime for the N300 amplitude, with more negative-going responses for happy PLDs compared with angry PLDs. There was also an interaction between prime and target for the N300 latency, suggesting that all children were sensitive to the emotional congruency between body movements and words. For the N400 component, there was only an interaction among age, prime, and target for latency, suggesting an age-dependent modulation of this component when prime and target did not match in emotional information. Overall, our results suggest that the emergence of more complex emotion processing of body expressions occurs around 6 years of age, but it is not fully developed at this point in ontogeny.

A computational neuroethology perspective on body and expression perception

Survival prompts organisms to prepare adaptive behavior in response to environmental and social threat. However, what are the specific features of the appearance of a conspecific that trigger such adaptive behaviors? For social species, the prime candidates for triggering defense systems are the visual features of the face and the body. We propose a novel approach for studying the ability of the brain to gather survival-relevant information from seeing conspecific body features. Specifically, we propose that behaviorally relevant information from bodies and body expressions is coded at the levels of midlevel features in the brain. These levels are relatively independent from higher-order cognitive and conscious perception of bodies and emotions. Instead, our approach is embedded in an ethological framework and mobilizes computational models for feature discovery.

Bridging the gap between emotion and joint action

Our daily human life is filled with a myriad of joint action moments, be it children playing, adults working together (i.e., team sports), or strangers navigating through a crowd. Joint action brings individuals (and embodiment of their emotions) together, in space and in time. Yet little is known about how individual emotions propagate through embodied presence in a group, and how joint action changes individual emotion. In fact, the multi-agent component is largely missing from neuroscience-based approaches to emotion, and reversely joint action research has not found a way yet to include emotion as one of the key parameters to model socio-motor interaction. In this review, we first identify the gap and then stockpile evidence showing strong entanglement between emotion and acting together from various branches of sciences. We propose an integrative approach to bridge the gap, highlight five research avenues to do so in behavioral neuroscience and digital sciences, and address some of the key challenges in the area faced by modern societies.

Happy Enough to Relax? How Positive and Negative Emotions Activate Different Muscular Regions in the Back - an Explorative Study

Embodiment theories have proposed a reciprocal relationship between emotional state and bodily reactions. Besides large body postures, recent studies have found emotions to affect rather subtle bodily expressions, such as slumped or upright sitting posture. This study investigated back muscle activity as an indication of an effect of positive and negative emotions on the sitting position. The electromyography (EMG) activity of six back muscles was recorded in 31 healthy subjects during exposure to positive and negative affective pictures. A resting period was used as a control condition. Increased muscle activity patterns in the back were found during the exposure to negative emotional stimuli, which was mainly measured in the lumbar and thorax regions. The positive emotion condition caused no elevated activity. The findings show that negative emotions lead to increased differential muscle activity in the back and thus corroborate those of previous research that emotion affects subtle bodily expressions.

Modulation of Response Times During Processing of Emotional Body Language

The investigation of how humans perceive and respond to emotional signals conveyed by the human body has been for a long time secondary compared with the investigation of facial expressions and emotional scenes recognition. The aims of this behavioral study were to assess the ability to process emotional body postures and to test whether motor response is mainly driven by the emotional content of the picture or if it is influenced by motor resonance. Emotional body postures and scenes (IAPS) divided into three clusters (fear, happiness, and neutral) were shown to 25 healthy subjects (13 males, mean age ± SD: 22.3 ± 1.8 years) in a three-alternative forced choice task. Subjects were asked to recognize the emotional content of the pictures by pressing one of three keys as fast as possible in order to estimate response times (RTs). The rating of valence and arousal was also performed. We found shorter RTs for fearful body postures as compared with happy and neutral postures. In contrast, no differences across emotional categories were found for the IAPS stimuli. Analysis on valence and arousal and the subsequent item analysis showed an excellent reliability of the two sets of images used in the experiment. Our results show that fearful body postures are rapidly recognized and processed, probably thanks to the automatic activation of a series of central nervous system structures orchestrating the defensive threat reactions, strengthening and supporting previous neurophysiological and behavioral findings in body language processing.

Computation-Based Feature Representation of Body Expressions in the Human Brain

Humans and other primate species are experts at recognizing body expressions. To understand the underlying perceptual mechanisms, we computed postural and kinematic features from affective whole-body movement videos and related them to brain processes. Using representational similarity and multivoxel pattern analyses, we showed systematic relations between computation-based body features and brain activity. Our results revealed that postural rather than kinematic features reflect the affective category of the body movements. The feature limb contraction showed a central contribution in fearful body expression perception, differentially represented in action observation, motor preparation, and affect coding regions, including the amygdala. The posterior superior temporal sulcus differentiated fearful from other affective categories using limb contraction rather than kinematics. The extrastriate body area and fusiform body area also showed greater tuning to postural features. The discovery of midlevel body feature encoding in the brain moves affective neuroscience beyond research on high-level emotion representations and provides insights in the perceptual features that possibly drive automatic emotion perception.

How Do We Recognize Emotion From Movement? Specific Motor Components Contribute to the Recognition of Each Emotion

Are there movement features that are recognized as expressing each basic emotion by most people, and what are they? In our previous study we identified sets of Laban movement components that, when moved, elicited the basic emotions of anger, sadness, fear, and happiness. Our current study aimed to investigate if movements composed from those sets would be recognized as expressing those emotions, regardless of any instruction to the mover to portray emotion. Our stimuli included 113 video-clips of five Certified Laban Movement Analysts (CMAs) moving combinations of two to four movement components from each set associated with only one emotion: happiness, sadness, fear, or anger. Each three second clip showed one CMA moving a single combination. The CMAs moved only the combination's required components. Sixty-two physically and mentally healthy men (n = 31) and women (n = 31), ages 19–48, watched the clips and rated the perceived emotion and its intensity. To confirm participants' ability to recognize emotions from movement and to compare our stimuli to existing validated emotional expression stimuli, participants rated 50 additional clips of bodily motor expressions of these same emotions validated by Atkinson et al. (2004). Results showed that for both stimuli types, all emotions were recognized far above chance level. Comparing recognition accuracy of the two clip types revealed better recognition of anger, fear, and neutral emotion from Atkinson's clips of actors expressing emotions, and similar levels of recognition accuracy for happiness and sadness. Further analysis was performed to determine the contribution of specific movement components to the recognition of the studied emotions. Our results indicated that these specific Laban motor components not only enhance feeling the associated emotions when moved, but also contribute to recognition of the associated emotions when being observed, even when the mover was not instructed to portray emotion, indicating that the presence of these movement components alone is sufficient for emotion recognition. This research-based knowledge regarding the relationship between Laban motor components and bodily emotional expressions can be used by dance-movement and drama therapists for better understanding of clients' emotional movements, for creating appropriate interventions, and for enhancing communication with other practitioners regarding bodily emotional expression.

Gaze patterns in viewing static and dynamic body expressions

Evidence for the importance of bodily cues for emotion recognition has grown over the last two decades. Despite this growing literature, it is underspecified how observers view whole bodies for body expression recognition. Here we investigate to which extent body-viewing is face- and context-specific when participants are categorizing whole body expressions in static (Experiment 1) and dynamic displays (Experiment 2). Eye-movement recordings showed that observers viewed the face exclusively when visible in dynamic displays, whereas viewing was distributed over head, torso and arms in static displays and in dynamic displays with faces not visible. The strong face bias in dynamic face-visible expressions suggests that viewing of the body responds flexibly to the informativeness of facial cues for emotion categorisation. However, when facial expressions are static or not visible, observers adopt a viewing strategy that includes all upper body regions. This viewing strategy is further influenced by subtle viewing biases directed towards emotion-specific body postures and movements to optimise recruitment of diagnostic information for emotion categorisation.

How Shall I Count the Ways? A Method for Quantifying the Qualitative Aspects of Unscripted Movement With Laban Movement Analysis

There is significant clinical evidence showing that creative and expressive movement processes involved in dance/movement therapy (DMT) enhance psycho-social well-being. Yet, because movement is a complex phenomenon, statistically validating which aspects of movement change during interventions or lead to significant positive therapeutic outcomes is challenging because movement has multiple, overlapping variables appearing in unique patterns in different individuals and situations. One factor contributing to the therapeutic effects of DMT is movement's effect on clients' emotional states. Our previous study identified sets of movement variables which, when executed, enhanced specific emotions. In this paper, we describe how we selected movement variables for statistical analysis in that study, using a multi-stage methodology to identify, reduce, code, and quantify the multitude of variables present in unscripted movement. We suggest a set of procedures for using Laban Movement Analysis (LMA)-described movement variables as research data. Our study used LMA, an internationally accepted comprehensive system for movement analysis, and a primary DMT clinical assessment tool for describing movement. We began with Davis's (1970) three-stepped protocol for analyzing movement patterns and identifying the most important variables: (1) We repeatedly observed video samples of validated (Atkinson et al., 2004) emotional expressions to identify prevalent movement variables, eliminating variables appearing minimally or absent. (2) We use the criteria repetition, frequency, duration and emphasis to eliminate additional variables. (3) For each emotion, we analyzed motor expression variations to discover how variables cluster: first, by observing ten movement samples of each emotion to identify variables common to all samples; second, by qualitative analysis of the two best-recognized samples to determine if phrasing, duration or relationship among variables was significant. We added three new steps to this protocol: (4) we created Motifs (LMA symbols) combining movement variables extracted in steps 1-3; (5) we asked participants in the pilot study to move these combinations and quantify their emotional experience. Based on the results of the pilot study, we eliminated more variables; (6) we quantified the remaining variables' prevalence in each Motif for statistical analysis that examined which variables enhanced each emotion. We posit that our method successfully quantified unscripted movement data for statistical analysis.

Innervation of the Long Head of the Triceps Brachii in Humans-A Fresh Look

The triceps brachii muscle occupies the posterior compartment of the arm in humans and has three heads. The lateral and medial heads originate from the humerus and the long head arises from the infraglenoid tubercle of the scapula. All heads form a common tendon that inserts onto the olecranon and the deep antebrachial fascia on each side of it. Each head receives its own motor branch, which all are thought to originate from the radial nerve. However, several studies reported that the motor branch of the long head of the triceps (LHT) arises from the axillary nerve or the posterior cord. Here, we dissected 27 triceps in 15 cadavers to analyze the innervation of the LHT and found only radial innervation, which contradicts those studies. We examined studies reporting that the motor branch to the LHT in humans does not arise from the radial nerve as well as studies of the triceps in primates. Occasional variations of the innervation of skeletal muscles are normal, but a change of principal motor innervation from radial to axillary nerve has important implications. This is because the axillary nerve is often involved during shoulder injuries. The precise identification of the prevalence of axillary versus radial innervation is therefore clinically relevant for surgery, nerve drafting, and occupational and physical therapy. We conclude that the primary motor branch to the LHT arises from the radial nerve but axillary/posterior cord innervations occur occasionally. We suggest the development of a standard methodology for further studies. Anat Rec, 301:473-483, 2018. © 2018 Wiley Periodicals, Inc.

Neural Responses to Consciously and Unconsciously Perceived Emotional Faces: A Spinal fMRI Study

Emotional stimuli modulate activity in brain areas related to attention, perception, and movement. Similar increases in neural activity have been detected in the spinal cord, suggesting that this understudied component of the central nervous system is an important part of our emotional responses. To date, previous studies of emotion-dependent spinal cord activity have utilized long presentations of complex emotional scenes. The current study differs from this research by (1) examining whether emotional faces will lead to enhanced spinal cord activity and (2) testing whether these stimuli require conscious perception to influence neural responses. Fifteen healthy undergraduate participants completed six spinal functional magnetic resonance imaging (fMRI) runs in which three one-minute blocks of fearful, angry, or neutral faces were interleaved with 40-s rest periods. In half of the runs, the faces were clearly visible while in the other half, the faces were displayed for only 17 ms. Spinal fMRI consisted of half-Fourier acquisition single-shot turbo spin-echo (HASTE) sequences targeting the cervical spinal cord. The results indicated that consciously perceived faces expressing anger elicited significantly more activity than fearful or neutral faces in ventral (motoric) regions of the cervical spinal cord. When stimuli were presented below the threshold of conscious awareness, neutral faces elicited significantly more activity than angry or fearful faces. Together, these data suggest that the emotional modulation of spinal cord activity is most impactful when the stimuli are consciously perceived and imply a potential threat toward the observer.

The Influence of Conscious and Unconscious Body Threat Expressions on Motor Evoked Potentials Studied With Continuous Flash Suppression

The observation of threatening expression in others is a strong cue for triggering an action response. One method of capturing such action responses is by measuring the amplitude of motor evoked potentials (MEPs) elicited with single pulse TMS over the primary motor cortex. Indeed, it has been shown that viewing whole body expressions of threat modulate the size of MEP amplitude. Furthermore, emotional cues have been shown to act on certain brain areas even outside of conscious awareness. In the current study, we explored if the influence of viewing whole body expressions of threat extends to stimuli that are presented outside of conscious awareness in healthy participants. To accomplish this, we combined the measurement of MEPs with a continuous flash suppression task. In experiment 1, participants were presented with images of neutral bodies, fearful bodies, or objects that were either perceived consciously or unconsciously, while single pulses of TMS were applied at different times after stimulus onset (200, 500, or 700 ms). In experiment 2 stimuli consisted of neutral bodies, angry bodies or objects, and pulses were applied at either 200 or 400 ms post stimulus onset. In experiment 1, there was a general effect of the time of stimulation, but no condition specific effects were evident. In experiment 2 there were no significant main effects, nor any significant interactions. Future studies need to look into earlier effects of MEP modulation by emotion body stimuli, specifically when presented outside of conscious awareness, as well as an exploration of other outcome measures such as intracortical facilitation.

Bodily Communication of Emotion: Evidence for Extrafacial Behavioral Expressions and Available Coding Systems

Although scientists dating back to Darwin have noted the importance of the body in communicating emotion, current research on emotion communication tends to emphasize the face. In this article we review the evidence for bodily expressions of emotions—that is, the handful of emotions that are displayed and recognized from certain bodily behaviors (i.e., pride, joy, sadness, shame, embarrassment, anger, fear, and disgust). We also review the previously developed coding systems available for identifying emotions from bodily behaviors. Although no extant coding system provides an exhaustive list of bodily behaviors known to communicate a panoply of emotions, our review provides the foundation for developing such a system.

Coverage of Emotion Recognition for Common Wearable Biosensors

The present research proposes a novel emotion recognition framework for the computer prediction of human emotions using common wearable biosensors. Emotional perception promotes specific patterns of biological responses in the human body, and this can be sensed and used to predict emotions using only biomedical measurements. Based on theoretical and empirical psychophysiological research, the foundation of autonomic specificity facilitates the establishment of a strong background for recognising human emotions using machine learning on physiological patterning. However, a systematic way of choosing the physiological data covering the elicited emotional responses for recognising the target emotions is not obvious. The current study demonstrates through experimental measurements the coverage of emotion recognition using common off-the-shelf wearable biosensors based on the synchronisation between audiovisual stimuli and the corresponding physiological responses. The work forms the basis of validating the hypothesis for emotional state recognition in the literature and presents coverage of the use of common wearable biosensors coupled with a novel preprocessing algorithm to demonstrate the practical prediction of the emotional states of wearers.

Physiological stress response patterns during a blood donation

BACKGROUND

Donating blood is associated with increased psychological stress. This study investigates whether a blood donation induces physiological stress and if response patterns differ by gender, donation experience and non-acute stress.

STUDY DESIGN AND METHODS

In 372 donors, physiological stress [blood pressure, pulse rate, pulse rate variability (PRV)] was measured at seven moments during routine donation. PRV was assessed using time domain [root mean square of successive differences (RMSSD)] and frequency domain [high frequency (HF) and low frequency (LF) power] parameters. Non-acute stress was assessed by questionnaire. Shape and significance of time course patterns were assessed by fitting multilevel models for each stress measure and comparing men and women, first-time and experienced donors, and donors with high and low levels of non-acute stress.

RESULTS

Significant response patterns were found for all stress measures, where levels of systolic blood pressure (F(1,1315) = 24·2, P < 0·001), RMSSD (F(1,1315) = 24·2, P < 0·001), LF (F(1,1627) = 14·1, P < 0·001) and HF (F(1,1624) = 34·0, P < 0·001) increased towards needle insertion and then decreased to values lower than when arriving at the donation centre. Diastolic blood pressure (F(1,1326) = 50·9, P < 0·001) increased and pulse rate (F(1,1393) = 507·4, P < 0·001) showed a U-shaped curve. Significant group effects were found, that is, higher systolic blood pressure/pulse rate in women; higher pulse rate in first-time donors; higher RMSSD at arrival and from screening until leaving in first-time donors; and higher LF and HF in first-time donors.

CONCLUSION

This study shows an increase in physiological stress related to needle insertion, followed by a decrease when leaving the donation centre. Some group effects were also found.

Affective vocalizations influence body ownership as measured in the rubber hand illusion

Emotional signals, like threatening sounds, automatically ready the perceiver to prepare an appropriate defense behavior. Conjecturing that this would manifest itself in extending the safety zone around the body we used the rubber hand illusion (RHI) to test this prediction. The RHI is a perceptual illusion in which body ownership is manipulated by synchronously stroking a rubber hand and real hand occluded from view. Many factors, both internal and external, have been shown to influence the strength of the illusion, yet the effect of emotion perception on body ownership remains unexplored. We predicted that listening to affective vocalizations would influence how strongly participants experience the RHI. In the first experiment four groups were tested that listened either to affective sounds (angry or happy vocalizations), non-vocal sounds or no sound while undergoing synchronous or asynchronous stroking of the real and rubber hand. In a second experiment three groups were tested comparing angry or neutral vocalizations and no sound condition. There was a significantly larger drift towards the rubber hand in the emotion versus the no emotion conditions. We interpret these results in the framework that the spatial increase in the RHI indicates that under threat the body has the capacity to extend its safety zone.

Face-body integration of intense emotional expressions of victory and defeat

Human facial expressions can be recognized rapidly and effortlessly. However, for intense emotions from real life, positive and negative facial expressions are difficult to discriminate and the judgment of facial expressions is biased towards simultaneously perceived body expressions. This study employed event-related potentials (ERPs) to investigate the neural dynamics involved in the integration of emotional signals from facial and body expressions of victory and defeat. Emotional expressions of professional players were used to create pictures of face-body compounds, with either matched or mismatched emotional expressions in faces and bodies. Behavioral results showed that congruent emotional information of face and body facilitated the recognition of facial expressions. ERP data revealed larger P1 amplitudes for incongruent compared to congruent stimuli. Also, a main effect of body valence on the P1 was observed, with enhanced amplitudes for the stimuli with losing compared to winning bodies. The main effect of body expression was also observed in N170 and N2, with winning bodies producing larger N170/N2 amplitudes. In the later stage, a significant interaction of congruence by body valence was found on the P3 component. Winning bodies elicited lager P3 amplitudes than losing bodies did when face and body conveyed congruent emotional signals. Beyond the knowledge based on prototypical facial and body expressions, the results of this study facilitate us to understand the complexity of emotion evaluation and categorization out of laboratory.

Deficits in Response Inhibition in Patients with Attention-Deficit/Hyperactivity Disorder: The Impaired Self-Protection System Hypothesis

Problems in inhibitory control are regarded in Psychology as a key problem associated with attention-deficit/hyperactivity disorder (ADHD). They, however, might not be primary deficits, but instead a consequence of inattention. At least two components have been identified and dissociated in studies in regards to inhibitory control: interference suppression, responsible for controlling interference by resisting irrelevant or misleading information, and response inhibition, referring to withholding a response or overriding an ongoing behavior. Poor error awareness and self-monitoring undermine an individual's ability to inhibit inadequate responses and change course of action. In non-social contexts, an individual depends on his own cognition to regulate his mistakes. In social contexts, however, there are many social cues that should help that individual to perceive his mistakes and inhibit inadequate responses. The processes involved in perceiving and interpreting those social cues are arguably part of a self-protection system (SPS). Individuals with ADHD not only present impulsive behaviors in social contexts, but also have difficulty perceiving their inadequate responses and overriding ongoing actions toward more appropriate ones. In this paper, we discuss that those difficulties are arguably a consequence of an impaired SPS, due to visual attention deficits and subsequent failure in perceiving and recognizing accurately negative emotions in facial expressions, especially anger. We discuss evidence that children with ADHD exhibit problems in a series of components involved in the activation of that system and advocate that the inability to identify the anger expressed by others, and thus, not experiencing the fear response that should follow, is, ultimately, what prevents them from inhibiting the ongoing inappropriate behavior, since a potential threat is not registered. Getting involved in high-risk situations, such as reckless driving, could also be a consequence of not registering a threat and thus, not experiencing fear.

The Automatic Detection of Chronic Pain-Related Expression: Requirements, Challenges and the Multimodal EmoPain Dataset

Pain-related emotions are a major barrier to effective self rehabilitation in chronic pain. Automated coaching systems capable of detecting these emotions are a potential solution. This paper lays the foundation for the development of such systems by making three contributions. First, through literature reviews, an overview of how pain is expressed in chronic pain and the motivation for detecting it in physical rehabilitation is provided. Second, a fully labelled multimodal dataset (named 'EmoPain') containing high resolution multiple-view face videos, head mounted and room audio signals, full body 3D motion capture and electromyographic signals from back muscles is supplied. Natural unconstrained pain related facial expressions and body movement behaviours were elicited from people with chronic pain carrying out physical exercises. Both instructed and non-instructed exercises were considered to reflect traditional scenarios of physiotherapist directed therapy and home-based self-directed therapy. Two sets of labels were assigned: level of pain from facial expressions annotated by eight raters and the occurrence of six pain-related body behaviours segmented by four experts. Third, through exploratory experiments grounded in the data, the factors and challenges in the automated recognition of such expressions and behaviour are described, the paper concludes by discussing potential avenues in the context of these findings also highlighting differences for the two exercise scenarios addressed.

Synchrony in Psychotherapy: A Review and an Integrative Framework for the Therapeutic Alliance

During psychotherapy, patient and therapist tend to spontaneously synchronize their vocal pitch, bodily movements, and even their physiological processes. In the present article, we consider how this pervasive phenomenon may shed new light on the therapeutic relationship- or alliance- and its role within psychotherapy. We first review clinical research on the alliance and the multidisciplinary area of interpersonal synchrony. We then integrate both literatures in the Interpersonal Synchrony (In-Sync) model of psychotherapy. According to the model, the alliance is grounded in the coupling of patient and therapist's brains. Because brains do not interact directly, movement synchrony may help to establish inter-brain coupling. Inter-brain coupling may provide patient and therapist with access to another's internal states, which facilitates common understanding and emotional sharing. Over time, these interpersonal exchanges may improve patients' emotion-regulatory capacities and related therapeutic outcomes. We discuss the empirical assessment of interpersonal synchrony and review preliminary research on synchrony in psychotherapy. Finally, we summarize our main conclusions and consider the broader implications of viewing psychotherapy as the product of two interacting brains.

Clear signals or mixed messages: inter-individual emotion congruency modulates brain activity underlying affective body perception

The neural basis of emotion perception has mostly been investigated with single face or body stimuli. However, in daily life one may also encounter affective expressions by groups, e.g. an angry mob or an exhilarated concert crowd. In what way is brain activity modulated when several individuals express similar rather than different emotions? We investigated this question using an experimental design in which we presented two stimuli simultaneously, with same or different emotional expressions. We hypothesized that, in the case of two same-emotion stimuli, brain activity would be enhanced, while in the case of two different emotions, one emotion would interfere with the effect of the other. The results showed that the simultaneous perception of different affective body expressions leads to a deactivation of the amygdala and a reduction of cortical activity. It was revealed that the processing of fearful bodies, compared with different-emotion bodies, relied more strongly on saliency and action triggering regions in inferior parietal lobe and insula, while happy bodies drove the occipito-temporal cortex more strongly. We showed that this design could be used to uncover important differences between brain networks underlying fearful and happy emotions. The enhancement of brain activity for unambiguous affective signals expressed by several people simultaneously supports adaptive behaviour in critical situations.

Emotion Regulation through Movement: Unique Sets of Movement Characteristics are Associated with and Enhance Basic Emotions

We have recently demonstrated that motor execution, observation, and imagery of movements expressing certain emotions can enhance corresponding affective states and therefore could be used for emotion regulation. But which specific movement(s) should one use in order to enhance each emotion? This study aimed to identify, using Laban Movement Analysis (LMA), the Laban motor elements (motor characteristics) that characterize movements whose execution enhances each of the basic emotions: anger, fear, happiness, and sadness. LMA provides a system of symbols describing its motor elements, which gives a written instruction (motif) for the execution of a movement or movement-sequence over time. Six senior LMA experts analyzed a validated set of video clips showing whole body dynamic expressions of anger, fear, happiness and sadness, and identified the motor elements that were common to (appeared in) all clips expressing the same emotion. For each emotion, we created motifs of different combinations of the motor elements common to all clips of the same emotion. Eighty subjects from around the world read and moved those motifs, to identify the emotion evoked when moving each motif and to rate the intensity of the evoked emotion. All subjects together moved and rated 1241 motifs, which were produced from 29 different motor elements. Using logistic regression, we found a set of motor elements associated with each emotion which, when moved, predicted the feeling of that emotion. Each emotion was predicted by a unique set of motor elements and each motor element predicted only one emotion. Knowledge of which specific motor elements enhance specific emotions can enable emotional self-regulation through adding some desired motor qualities to one's personal everyday movements (rather than mimicking others' specific movements) and through decreasing motor behaviors which include elements that enhance negative emotions.

An exploration of familial associations of two movement pattern-derived subgroups of chronic disabling low back pain; a cross-sectional cohort study

BACKGROUND

Altered movement patterns with pain have been demonstrated in children, adolescents and adults with chronic disabling low back pain (CDLBP). A previously developed classification system has identified different subgroups including active extension and multidirectional patterns in patients with CDLBP. While familial associations have been identified for certain spinal postures in standing, it is unknown whether a familial relationship might exist between movement pattern-derived subgroups in families with CDLBP.

OBJECTIVES

This study explored whether familial associations in movement pattern-derived subgroups within and between members of families with CDLBP existed.

DESIGN

Cross-sectional cohort study.

METHOD

33 parents and 28 children with CDLBP were classified into two subgroups based on clinical analysis of video footage of postures and functional movements, combined with aggravating factors obtained from Oswestry Disability Questionnaire. Prevalence of subgroups within family members was determined, associations between parent and child's subgroup membership was evaluated using Fisher's exact test, and spearman's correlation coefficient was used to determine the strength of association between familial dyads.

RESULTS

The majority of parents were classified as active extenders, sons predominately multidirectional and daughters were evenly distributed between the two subgroups. No significant association was found when comparing subgroups in nine parent-child relationships.

CONCLUSIONS

The exploration of a small cohort of family dyads in this study demonstrated that children's movement pattern-derived subgroups could not be explained by their parents' subgroup membership. These results cannot be generalised to the CLBP population due to this study's small sample. Larger sample studies are needed to further elucidate this issue.

Is it the real deal? Perception of virtual characters versus humans: an affective cognitive neuroscience perspective

Recent developments in neuroimaging research support the increased use of naturalistic stimulus material such as film, avatars, or androids. These stimuli allow for a better understanding of how the brain processes information in complex situations while maintaining experimental control. While avatars and androids are well suited to study human cognition, they should not be equated to human stimuli. For example, the uncanny valley hypothesis theorizes that artificial agents with high human-likeness may evoke feelings of eeriness in the human observer. Here we review if, when, and how the perception of human-like avatars and androids differs from the perception of humans and consider how this influences their utilization as stimulus material in social and affective neuroimaging studies. First, we discuss how the appearance of virtual characters affects perception. When stimuli are morphed across categories from non-human to human, the most ambiguous stimuli, rather than the most human-like stimuli, show prolonged classification times and increased eeriness. Human-like to human stimuli show a positive linear relationship with familiarity. Secondly, we show that expressions of emotions in human-like avatars can be perceived similarly to human emotions, with corresponding behavioral, physiological and neuronal activations, with exception of physical dissimilarities. Subsequently, we consider if and when one perceives differences in action representation by artificial agents versus humans. Motor resonance and predictive coding models may account for empirical findings, such as an interference effect on action for observed human-like, natural moving characters. However, the expansion of these models to explain more complex behavior, such as empathy, still needs to be investigated in more detail. Finally, we broaden our outlook to social interaction, where virtual reality stimuli can be utilized to imitate complex social situations.

Automatic emotion recognition based on body movement analysis: a survey

Humans are emotional beings, and their feelings influence how they perform and interact with computers. One of the most expressive modalities for humans is body posture and movement, which researchers have recently started exploiting for emotion recognition. This survey describes emerging techniques and modalities related to emotion recognition based on body movement, as well as recent advances in automatic emotion recognition. It also describes application areas and notation systems and explains the importance of movement segmentation. It then discusses unsolved problems and provides promising directions for future research. The Web extra (a PDF file) contains tables with additional information related to the article.

The Body Action Coding System II: muscle activations during the perception and expression of emotion

Research into the expression and perception of emotions has mostly focused on facial expressions. Recently, body postures have become increasingly important in research, but knowledge on muscle activity during the perception or expression of emotion is lacking. The current study continues the development of a Body Action Coding System (BACS), which was initiated in a previous study, and described the involvement of muscles in the neck, shoulders and arms during expression of fear and anger. The current study expands the BACS by assessing the activity patterns of three additional muscles. Surface electromyography of muscles in the neck (upper trapezius descendens), forearms (extensor carpi ulnaris), lower back (erector spinae longissimus) and calves (peroneus longus) were measured during active expression and passive viewing of fearful and angry body expressions. The muscles in the forearm were strongly active for anger expression and to a lesser extent for fear expression. In contrast, muscles in the calves were recruited slightly more for fearful expressions. It was also found that muscles automatically responded to the perception of emotion, without any overt movement. The observer's forearms responded to the perception of fear, while the muscles used for leaning backwards were activated when faced with an angry adversary. Lastly, the calf responded immediately when a fearful person was seen, but responded slower to anger. There is increasing interest in developing systems that are able to create or recognize emotional body language for the development of avatars, robots, and online environments. To that end, multiple coding systems have been developed that can either interpret or create bodily expressions based on static postures, motion capture data or videos. However, the BACS is the first coding system based on muscle activity.

Emotion categorization of body expressions in narrative scenarios

Humans can recognize emotions expressed through body motion with high accuracy even when the stimuli are impoverished. However, most of the research on body motion has relied on exaggerated displays of emotions. In this paper we present two experiments where we investigated whether emotional body expressions could be recognized when they were recorded during natural narration. Our actors were free to use their entire body, face, and voice to express emotions, but our resulting visual stimuli used only the upper body motion trajectories in the form of animated stick figures. Observers were asked to perform an emotion recognition task on short motion sequences using a large and balanced set of emotions (amusement, joy, pride, relief, surprise, anger, disgust, fear, sadness, shame, and neutral). Even with only upper body motion available, our results show recognition accuracy significantly above chance level and high consistency rates among observers. In our first experiment, that used more classic emotion induction setup, all emotions were well recognized. In the second study that employed narrations, four basic emotion categories (joy, anger, fear, and sadness), three non-basic emotion categories (amusement, pride, and shame) and the "neutral" category were recognized above chance. Interestingly, especially in the second experiment, observers showed a bias toward anger when recognizing the motion sequences for emotions. We discovered that similarities between motion sequences across the emotions along such properties as mean motion speed, number of peaks in the motion trajectory and mean motion span can explain a large percent of the variation in observers' responses. Overall, our results show that upper body motion is informative for emotion recognition in narrative scenarios.