The recognition of 18 facial-bodily expressions across nine cultures

Dissociable neural networks for processing fearful bodily expressions at different spatial frequencies

The human brain processes visual input across various spatial frequency (SF) ranges to extract emotional cues. Prior studies have extensively explored SF processing in facial expressions, yielding partly conflicting results. However, bodily expressions, which provide complementary emotional and survival-relevant cues, remain unexplored. We investigated the neural mechanisms underlying the processing of low (LSF), high (HSF), and broad spatial frequency (BSF) components in fearful versus neutral bodily postures. Using functional Magnetic Resonance Imaging, we examined brain activity in 20 participants viewing SF-filtered images of bodily expressions in a semi-passive task. A multivariate "searchlight" analysis based on Multi-Voxel Pattern Analysis was employed to decode the non-linear activation patterns associated with each SF band. Our findings reveal that SF processing engages distinct neural networks in response to fearful bodily expressions. BSF stimuli activated a widespread network, including the amygdala, pulvinar, frontal, and temporal cortices. These findings suggest a general threat-detection system integrating information across all SFs. HSF stimuli engaged cortical regions associated with detailed emotional evaluation and motor planning, such as the orbitofrontal cortex, anterior cingulate cortex, and premotor areas, suggesting that processing fine-grained fear cues involves computationally demanding networks related to emotional resonance and action preparation. In contrast, LSF stimuli primarily activated motor-preparatory regions linked to rapid, action-oriented responses, highlighting the brain prioritization of quick readiness to low-detail threats. Notably, the amygdala showed no SF selectivity, supporting its role as a generalized "relevance detector" in emotional processing. The present study demonstrates that the brain flexibly adapts its SF processing strategy based on the visual details available in fearful bodily expressions, underscoring the complexity and adaptability of emotional processing from bodily signals.

Blocking lower facial features reduces emotion identification accuracy in static faces and full body dynamic expressions

During COVID, much of the world wore masks covering their lower faces to prevent the spread of disease. These masks cover lower facial features, but how vital are these lower facial features to the recognition of facial expressions of emotion? Going beyond the Ekman 6 emotions, in Study 1 (N = 372), we used a multilevel logistic regression to examine how artificially rendered masks influence emotion recognition from static photos of facial muscle configurations for many commonly experienced positive and negative emotions. On average, masks reduced emotion recognition accuracy by 17% percent for negative emotions and 23% for positive emotions. In Study 2 (N = 338), we asked whether these results generalised to multimodal full-body expressions of emotions, accompanied by vocal expressions. Participants viewed videos from a previously validated set, where the lower facial features were blurred from the nose down. Here, though the decreases in emotion recognition were noticeably less pronounced, highlighting the power of multimodal information, we did see important decreases for certain specific emotions and for positive emotions overall. Results are discussed in the context of the social and emotional consequences of compromised emotion recognition, as well as the unique facial features which accompany certain emotions.

Emotion specificity, coherence, and cultural variation in conceptualizations of positive emotions: a study of body sensations and emotion recognition

The present study examines the association between people's interoceptive representation of physical sensations and the recognition of vocal and facial expressions of emotion. We used body maps to study the granularity of the interoceptive conceptualisation of 11 positive emotions (amusement, awe, compassion, contentment, desire, love, joy, interest, pride, relief, and triumph) and a new emotion recognition test (Emotion Expression Understanding Test) to assess the ability to recognise emotions from vocal and facial behaviour. Overall, we found evidence for distinct interoceptive conceptualizations of 11 positive emotions across Asian American, European American, and Latino/a American cultures, as well as the reliable identification of emotion in facial and vocal expressions. Central to new theorising about emotion-related representation, the granularity of physical sensations did not covary with emotion recognition accuracy, suggesting that two kinds of emotion conceptualisation processes might be distinct.

Commonalities and variations in emotion representation across modalities and brain regions

Humans express emotions through various modalities such as facial expressions and natural language. However, the relationships between emotions expressed through different modalities and their correlations with neural activities remain uncertain. Here, we aimed to unveil some of these uncertainties by investigating the similarity of emotion representations across modalities and brain regions. First, we represented various emotion categories as multi-dimensional vectors derived from visual (face), linguistic, and visio-linguistic data, and used representational similarity analysis to compare these modalities. Second, we examined the linear transferability of emotion representation from other modalities to the visual modality. Third, we compared the representational structure derived in the first step with those from brain activities across 360 regions. Our findings revealed that emotion representations share commonalities across modalities with modality-type dependent variations, and they can be linearly mapped from other modalities to the visual modality. Additionally, emotion representations in uni-modalities showed relatively higher similarity with specific brain regions, while multi-modal emotion representation was most similar to representations across the entire brain region. These findings suggest that emotional experiences are represented differently across various brain regions with varying degrees of similarity to different modality types, and that they may be multi-modally conveyable in visual and linguistic domains.

Emotional concepts shape the perceptual representation of body expressions

Emotion perception interacts with how we think and speak, including our concept of emotions. Body expression is an important way of emotion communication, but it is unknown whether and how its perception is modulated by conceptual knowledge. In this study, we employed representational similarity analysis and conducted three experiments combining semantic similarity, mouse-tracking task, and one-back behavioral task with electroencephalography and functional magnetic resonance imaging techniques, the results of which show that conceptual knowledge predicted the perceptual representation of body expressions. Further, this prediction effect occurred at approximately 170 ms post-stimulus. The neural encoding of body expressions in the fusiform gyrus and lingual gyrus was impacted by emotion concept knowledge. Taken together, our results indicate that conceptual knowledge of emotion categories shapes the configural representation of body expressions in the ventral visual cortex, which offers compelling evidence for the constructed emotion theory.

The bright side of words: Norms for 9000 Spanish words in seven discrete positive emotions

In recent years, assumptions about the existence of a single construct of happiness that accounts for all positive emotions have been questioned. Instead, several discrete positive emotions with their own neurobiological and psychological mechanisms have been proposed. Of note, the effects of positive emotions on language processing are not yet properly understood. Here we provide a database for a large set of 9000 Spanish words scored by 3437 participants in the positive emotions of awe, contentment, amusement, excitement, serenity, relief, and pleasure. We also report significant correlations between discrete positive emotions and several affective (e.g., valence, arousal, happiness, negative discrete emotions) and lexico-semantic (e.g., frequency of use, familiarity, concreteness, age of acquisition) characteristics of words. Finally, we analyze differences between words conveying a single emotion ("pure" emotion words) and those denoting more than one emotion ("mixed" emotion words). This study will provide researchers a rich source of information to do research that contributes to expanding the current knowledge on the role of positive emotions in language. The norms are available at https://doi.org/10.6084/m9.figshare.21533571.v2.

How emotion is experienced and expressed in multiple cultures: a large-scale experiment across North America, Europe, and Japan

Core to understanding emotion are subjective experiences and their expression in facial behavior. Past studies have largely focused on six emotions and prototypical facial poses, reflecting limitations in scale and narrow assumptions about the variety of emotions and their patterns of expression. We examine 45,231 facial reactions to 2,185 evocative videos, largely in North America, Europe, and Japan, collecting participants' self-reported experiences in English or Japanese and manual and automated annotations of facial movement. Guided by Semantic Space Theory, we uncover 21 dimensions of emotion in the self-reported experiences of participants in Japan, the United States, and Western Europe, and considerable cross-cultural similarities in experience. Facial expressions predict at least 12 dimensions of experience, despite massive individual differences in experience. We find considerable cross-cultural convergence in the facial actions involved in the expression of emotion, and culture-specific display tendencies-many facial movements differ in intensity in Japan compared to the U.S./Canada and Europe but represent similar experiences. These results quantitatively detail that people in dramatically different cultures experience and express emotion in a high-dimensional, categorical, and similar but complex fashion.

Increased body movement equals better performance? Not always! Musical style determines motion degree perceived as optimal in music performance

Musicians' body behaviour has a preponderant role in audience perception. We investigated how performers' motion is perceived depending on the musical style and musical expertise. To further explore the effect of visual input, stimuli were presented in audio-only, audio-visual and visual-only conditions. We used motion and audio recordings of expert saxophone players playing two contrasting excerpts (positively and negatively valenced). For each excerpt, stimuli represented five motion degrees with increasing quantity of motion (QoM) and distinct predominant gestures. In the experiment (online and in-person), 384 participants rated performance recordings for expressiveness, professionalism and overall quality. Results revealed that, for the positively valenced excerpt, ratings increased as a function of QoM, whilst for the negatively valenced, the recording with predominant flap motion was favoured. Musicianship did not have a significant effect in motion perception. Concerning multisensory integration, both musicians and non-musicians presented visual dominance in the positively valenced excerpt, whereas in the negatively valenced, musicians shifted to auditory dominance. Our findings demonstrate that musical style not only determines the way observers perceive musicians' movement as adequate, but also that it can promote changes in multisensory integration.

Impact of the War in Ukraine on the Ability of Children to Recognize Basic Emotions

Objectives

This study assessed emotion recognition skills in school-age children in wartime conditions in Ukraine.

Methods

An online survey based on the concept of basic emotions was administrated to a sample of 419 schoolchildren from Ukraine and a control group of 310 schoolchildren from the Czech Republic, aged 8 to 12.

Results

There is no difference in judging the intensity of anger and fear by Ukrainian children, compared with the control group. There is no evidence that the emotions of anger, fear, and sadness were better recognized in the Ukrainian group. Children from Ukraine were better at recognizing positive emotions than Czech children.

Conclusion

Increased risks of threats and wartime experience do not impair the accuracy of identification of emotions like fear or the assessment of intensity of basic emotions by children who experience war in Ukraine. Still, it is important to continue studying the long-term consequences of military conflicts in order to deepen the understanding of their impact on human mental functioning.

Deep learning reveals what facial expressions mean to people in different cultures

Cross-cultural studies of the meaning of facial expressions have largely focused on judgments of small sets of stereotypical images by small numbers of people. Here, we used large-scale data collection and machine learning to map what facial expressions convey in six countries. Using a mimicry paradigm, 5,833 participants formed facial expressions found in 4,659 naturalistic images, resulting in 423,193 participant-generated facial expressions. In their own language, participants also rated each expression in terms of 48 emotions and mental states. A deep neural network tasked with predicting the culture-specific meanings people attributed to facial movements while ignoring physical appearance and context discovered 28 distinct dimensions of facial expression, with 21 dimensions showing strong evidence of universality and the remainder showing varying degrees of cultural specificity. These results capture the underlying dimensions of the meanings of facial expressions within and across cultures in unprecedented detail.

Embracing the Emotion in Emotional Intelligence Measurement: Insights from Emotion Theory and Research

Emotional intelligence (EI) has gained significant popularity as a scientific construct over the past three decades, yet its conceptualization and measurement still face limitations. Applied EI research often overlooks its components, treating it as a global characteristic, and there are few widely used performance-based tests for assessing ability EI. The present paper proposes avenues for advancing ability EI measurement by connecting the main EI components to models and theories from the emotion science literature and related fields. For emotion understanding and emotion recognition, we discuss the implications of basic emotion theory, dimensional models, and appraisal models of emotion for creating stimuli, scenarios, and response options. For the regulation and management of one's own and others' emotions, we discuss how the process model of emotion regulation and its extensions to interpersonal processes can inform the creation of situational judgment items. In addition, we emphasize the importance of incorporating context, cross-cultural variability, and attentional and motivational factors into future models and measures of ability EI. We hope this article will foster exchange among scholars in the fields of ability EI, basic emotion science, social cognition, and emotion regulation, leading to an enhanced understanding of the individual differences in successful emotional functioning and communication.

The Meso-Expression Test (MET): A Novel Assessment of Emotion Perception

Emotion perception is a primary facet of Emotional Intelligence (EI) and the underpinning of interpersonal communication. In this study, we examined meso-expressions-the everyday, moderate-intensity emotions communicated through the face, voice, and body. We theoretically distinguished meso-expressions from other well-known emotion research paradigms (i.e., macro-expression and micro-expressions). In Study 1, we demonstrated that people can reliably discriminate between meso-expressions, and we created a corpus of 914 unique video displays of meso-expressions across a race- and gender-diverse set of expressors. In Study 2, we developed a novel video-based assessment of emotion perception ability: The Meso-Expression Test (MET). In this study, we found that the MET is psychometrically valid and demonstrated measurement equivalence across Asian, Black, Hispanic, and White perceiver groups and across men and women. In Study 3, we examined the construct validity of the MET and showed that it converged with other well-known measures of emotion perception and diverged from cognitive ability. Finally, in Study 4, we showed that the MET is positively related to important psychosocial outcomes, including social well-being, social connectedness, and empathic concern and is negatively related to alexithymia, stress, depression, anxiety, and adverse social interactions. We conclude with a discussion focused on the implications of our findings for EI ability research and the practical applications of the MET.

Owners' Beliefs regarding the Emotional Capabilities of Their Dogs and Cats

The correct interpretation of an animal's emotional state is crucial for successful human-animal interaction. When studying dog and cat emotional expressions, a key source of information is the pet owner, given the extensive interactions they have had with their pets. In this online survey we asked 438 owners whether their dogs and/or cats could express 22 different primary and secondary emotions, and to indicate the behavioral cues they relied upon to identify those expressed emotions. Overall, more emotions were reported in dogs compared to cats, both from owners that owned just one species and those that owned both. Although owners reported a comparable set of sources of behavioral cues (e.g., body posture, facial expression, and head posture) for dogs and cats in expressing the same emotion, distinct combinations tended to be associated with specific emotions in both cats and dogs. Furthermore, the number of emotions reported by dog owners was positively correlated with their personal experience with dogs but negatively correlated with their professional experience. The number of emotions reported in cats was higher in cat-only households compared to those that also owned dogs. These results provide a fertile ground for further empirical investigation of the emotional expressions of dogs and cats, aimed at validating specific emotions in these species.

Deep learning reveals what vocal bursts express in different cultures

Human social life is rich with sighs, chuckles, shrieks and other emotional vocalizations, called 'vocal bursts'. Nevertheless, the meaning of vocal bursts across cultures is only beginning to be understood. Here, we combined large-scale experimental data collection with deep learning to reveal the shared and culture-specific meanings of vocal bursts. A total of n = 4,031 participants in China, India, South Africa, the USA and Venezuela mimicked vocal bursts drawn from 2,756 seed recordings. Participants also judged the emotional meaning of each vocal burst. A deep neural network tasked with predicting the culture-specific meanings people attributed to vocal bursts while disregarding context and speaker identity discovered 24 acoustic dimensions, or kinds, of vocal expression with distinct emotion-related meanings. The meanings attributed to these complex vocal modulations were 79% preserved across the five countries and three languages. These results reveal the underlying dimensions of human emotional vocalization in remarkable detail.

Pride: The Emotional Foundation of Social Rank Attainment

Pride is a self-conscious emotion, comprised of two distinct facets known as authentic and hubristic pride, and associated with a cross-culturally recognized nonverbal expression. Authentic pride involves feelings of accomplishment and confidence and promotes prosocial behaviors, whereas hubristic pride involves feelings of arrogance and conceit and promotes antisociality. Each facet of pride, we argue, contributes to a distinct means of attaining social rank: Authentic pride seems to promote prestige-a rank based on earned respect-whereas hubristic pride seems to promote dominance-a rank based on aggression and coercion. Both prestige and dominance are effective routes to power and influence in human groups, so both facets of pride are likely to be functional adaptations. Overall, the reviewed research suggests that pride is likely to be a human universal, critical for social relationships and rank attainment across human societies.

Automatic facial coding predicts self-report of emotion, advertisement and brand effects elicited by video commercials

Introduction

Consumers' emotional responses are the prime target for marketing commercials. Facial expressions provide information about a person's emotional state and technological advances have enabled machines to automatically decode them.

Method

With automatic facial coding we investigated the relationships between facial movements (i.e., action unit activity) and self-report of commercials advertisement emotion, advertisement and brand effects. Therefore, we recorded and analyzed the facial responses of 219 participants while they watched a broad array of video commercials.

Results

Facial expressions significantly predicted self-report of emotion as well as advertisement and brand effects. Interestingly, facial expressions had incremental value beyond self-report of emotion in the prediction of advertisement and brand effects. Hence, automatic facial coding appears to be useful as a non-verbal quantification of advertisement effects beyond self-report.

Discussion

This is the first study to measure a broad spectrum of automatically scored facial responses to video commercials. Automatic facial coding is a promising non-invasive and non-verbal method to measure emotional responses in marketing.

Searching, Navigating, and Recommending Movies through Emotions: A Scoping Review

Movies offer viewers a broad range of emotional experiences, providing entertainment, and meaning. Following the PRISMA-ScR guidelines, we reviewed the literature on digital systems designed to help users search and browse movie libraries and offer recommendations based on emotional content. Our search yielded 83 eligible documents (published between 2000 and 2021). We identified 22 case studies, 34 empirical studies, 26 proof of concept, and one theoretical paper. User transactions (e.g., ratings, tags) were the preferred source of information. The documents examined approached emotions from both a categorical ( $n=35$ ) and dimensional ( $n=18$ ) perspectives, and nine documents offer a combination of both approaches. Although there are several authors mentioned, the references used are frequently dated, and 12 documents do not mention author or model used. We identified 61 words related to emotion or affect. Documents presented on average 1.36 positive terms and 2.64 negative terms. Sentiment analysis ( $n=31$ ) is frequently used for emotion identification, followed by subjective evaluations ( $n=15$ ), movie low-level audio and visual features (n = 11), and face recognition technologies ( $n=8$ ). We discuss limitations and offer a brief review of current emotion models and research.

Only the good cry: Investigating the relationship between crying proneness and moral judgments and behavior