Human and machine recognition of dynamic and static facial expressions: prototypicality, ambiguity, and complexity

Wireless high-resolution surface facial electromyography mask for discrimination of standardized facial expressions in healthy adults

Wired high resolution surface electromyography (sEMG) using gelled electrodes is a standard method for psycho-physiological, neurological and medical research. Despite its widespread use electrode placement is elaborative, time-consuming, and the overall experimental setting is prone to mechanical artifacts and thus offers little flexibility. Wireless and easy-to-apply technologies would facilitate more accessible examination in a realistic setting. To address this, a novel smart skin technology consisting of wireless dry 16-electrodes was tested. The soft electrode arrays were attached to the right hemiface of 37 healthy adult participants (60% female; 20 to 57 years). The participants performed three runs of a standard set of different facial expression exercises. Linear mixed-effects models utilizing the sEMG amplitudes as outcome measure were used to evaluate differences between the facial movement tasks and runs (separately for every task). The smart electrodes showed specific activation patterns for each of the exercises. 82% of the exercises could be differentiated from each other with very high precision when using the average muscle action of all electrodes. The effects were consistent during the 3 runs. Thus, it appears that wireless high-resolution sEMG analysis with smart skin technology successfully discriminates standard facial expressions in research and clinical settings.

Understanding Naturalistic Facial Expressions with Deep Learning and Multimodal Large Language Models

This paper provides a comprehensive overview of affective computing systems for facial expression recognition (FER) research in naturalistic contexts. The first section presents an updated account of user-friendly FER toolboxes incorporating state-of-the-art deep learning models and elaborates on their neural architectures, datasets, and performances across domains. These sophisticated FER toolboxes can robustly address a variety of challenges encountered in the wild such as variations in illumination and head pose, which may otherwise impact recognition accuracy. The second section of this paper discusses multimodal large language models (MLLMs) and their potential applications in affective science. MLLMs exhibit human-level capabilities for FER and enable the quantification of various contextual variables to provide context-aware emotion inferences. These advancements have the potential to revolutionize current methodological approaches for studying the contextual influences on emotions, leading to the development of contextualized emotion models.