Sociality of facial expressions in immersive virtual environments: A facial EMG study

Topographic mapping of the sensorimotor qualities of empathic reactivity: A psychophysiological study in people with spinal cord injuries

The experience of empathy for pain is underpinned by sensorimotor and affective dimensions which, although interconnected, are at least in part behaviorally and neurally distinct. Spinal cord injuries (SCI) induce a massive, below-lesion level, sensorimotor body-brain disconnection. This condition may make it possible to test whether sensorimotor deprivation alters specific dimensions of empathic reactivity to observed pain. To explore this issue, we asked SCI people with paraplegia and healthy controls to observe videos of painful or neutral stimuli administered to a hand (intact) or a foot (deafferented). The stimuli were displayed by means of a virtual reality set-up and seen from a first person (1PP) or third person (3PP) visual perspective. A number of measures were recorded ranging from explicit behaviors like explicit verbal reports on the videos, to implicit measures of muscular activity (like EMG from the corrugator and zygomatic muscles that may represent a proxy of sensorimotor empathy) and of autonomic reactivity (like the electrodermal response and Respiratory Sinus Arrhythmia that may represent a general proxy of affective empathy). While no across group differences in explicit verbal reports about the pain stimuli were found, SCI people exhibited reduced facial muscle reactivity to the stimuli applied to the foot (but not the hand) seen from the 1PP. Tellingly, the corrugator activity correlated with SCI participants' neuropathic pain. There were no across group differences in autonomic reactivity suggesting that SCI lesions may affect sensorimotor dimensions connected to empathy for pain.

Virtual environments to study emotional responses to clinical communication: A scoping review

OBJECTIVE

This scoping review explores the potential for virtual environments (VE) to evaluate emotional outcomes in clinical communication research. Authors representing multiple disciplines use review results to propose potential research opportunities and considerations.

METHODS

We utilized a structured framework for scoping reviews. We searched four literature databases for relevant articles. We applied multidisciplinary perspectives to synthesize relevant potential opportunities for emotion-focused communications research using VE.

RESULTS

Twenty-one articles met inclusion criteria. They applied different methodological approaches, including a range of VE technologies and diverse emotional outcome measures, such as psychophysiological arousal, emotional valence, or empathy. Major research topics included use of virtual reality to provoke and measure emotional responses, train clinicians in communication skills, and increase clinician empathy.

CONCLUSION

Researchers may leverage VE technologies to ethically and systematically examine how characteristics of clinical interactions, environments, and communication impact emotional reactions and responses among patients and clinicians. Variability exists in how VE technologies are employed and reported in published literature, and this may limit the internal and external validity of the research. However, virtual reality can provide a low-cost, low-risk, experimentally controlled, and ecologically valid approach for studying clinician-patient communication.

PRACTICE IMPLICATIONS

Future research should leverage psychophysiological measures to further examine emotional responses during clinical communication scenarios and clearly report virtual environment characteristics to support evaluation of study conclusions, study replicability, and meta-analyses.

Emotion Recognition in Immersive Virtual Reality: From Statistics to Affective Computing

Emotions play a critical role in our daily lives, so the understanding and recognition of emotional responses is crucial for human research. Affective computing research has mostly used non-immersive two-dimensional (2D) images or videos to elicit emotional states. However, immersive virtual reality, which allows researchers to simulate environments in controlled laboratory conditions with high levels of sense of presence and interactivity, is becoming more popular in emotion research. Moreover, its synergy with implicit measurements and machine-learning techniques has the potential to impact transversely in many research areas, opening new opportunities for the scientific community. This paper presents a systematic review of the emotion recognition research undertaken with physiological and behavioural measures using head-mounted displays as elicitation devices. The results highlight the evolution of the field, give a clear perspective using aggregated analysis, reveal the current open issues and provide guidelines for future research.

Does Social Content Influence the Subjective Evaluation of Affective Pictures?

This study explored the effect of the perceived social content of affective pictures on the subjective evaluation of affective valence and arousal. For this purpose, we established three categories of social content (pictures without people, with one person and with two or more people). A sample of 161 subjects rated 200 pictures varying in affective valence (unpleasant, neutral, and pleasant), arousal and social content. Results of two-factor analysis of variance, F(4, 157) = 71.7, p < .001, ηp2 = .31, showed that perceived social content influenced the ratings of affective valence, specially for unpleasant pictures, with the greatest social content (two or more people) leading subjects to rate unpleasant pictures with the lowest ratings (all pairwise comparisons' p < .001). Regarding arousal, F(4, 157) = 64.0, p < .001, ηp2 = .29), the higher the social content, the higher the arousal ratings, but only for pleasant (all pairwise comparisons' p < .007) and unpleasant (all pairwise comparisons' p < .001) pictures. Overall, this study demonstrated an effect of the perceived social content on the subjective evaluation of affective valence and arousal of emotional stimuli.

An unfamiliar social presence reduces facial disgust responses to food stimuli

Consumers' emotional responses complement sensory and hedonic ratings in the prediction of food choice and consumption behaviour. The challenge with the measurement of consumption emotions is that emotions are highly context dependent. For emotion evaluations to bring greater insight to food research and development, it is essential that the influence of contextual variables on emotion are quantified. The present study contributes to the discussion with an investigation of the effect of an unfamiliar social presence on affective facial responses to visual food stimuli. Seventy participants (52 female and 18 male) viewed food images of varying acceptability either alone, or in the presence of the researcher. Subjective liking ratings were measured using a labelled affective magnitude scale, and facial muscle activity from zygomaticus major (contracted during smiling), corrugator supercilii (contracted during frowning) and levator labii superioris (contracted during nose wrinkling) were measured with an EMG recording system. Controlling for individual differences in facial expressivity and food image acceptability using linear mixed models, it was found that social context did not predict smiling or frowning muscle activity. Social context did predict the intensity of muscle activity indicative of a disgust response, with participants in the observed condition exhibiting less levator activity than participants in the alone condition. Regardless of social context, each muscle was found to have a relationship with subjective liking, with the direction of effects as expected. The results indicate that emotional stimuli and social context both influence food-evoked facial expression and provides support for the utility of facial EMG in measuring food-evoked emotion.

Characteristics of facial muscle activity during voluntary facial expressions: Imaging analysis of facial expressions based on myogenic potential data

Purpose

Facial expressions are formed by the coordination of facial muscles and reflect changes in emotion. Nurses observe facial expressions as way of understanding patients. This study conducted basic research using facial myogenic potential topography to visually determine changes in the location and strength of facial muscle activity associated with voluntary facial expression to examine relationships with facial expressions.

Methods

Participants comprised 18 healthy adults (6 men, 12 women; mean age, 24.3 ± 4.3 years). Facial myogenic potentials were measured from 19 electrodes arranged concentrically on the face, and topographic analysis was conducted. Using potential changes and topograms, the muscle activity associated with nonvoluntary facial expression and voluntary facial expressions of happiness and disgust were classified according to the characteristics of expressions. To classify homogeneous groups among the reaction of disgust, hierarchical cluster analysis was utilized.

Results

One characteristic of the facial expression of happiness was activity in areas including the greater zygomatic muscle. With the facial expression of disgust, characteristic changes were seen in areas including the corrugator supercilii. Cluster analysis of the expression of disgust showed four homogeneous subgroups.

Conclusion

With facial myogenic potential topography, facial expressions can be evaluated objectively without being influenced by face shape or countenance. Color changes in topograms showed subtle changes in expressions that could not be supplemented with statistical processing alone, and these were useful in identifying individuality. Topography is thus expected to be utilized to supplement basic knowledge of facial expressions for a better understanding of patients.

With facial myogenic potential topography, facial expressions can be evaluated objectively without being influenced by face shape or countenance. Cluster analysis of the expression of disgust showed four homogeneous subgroups. Color changes in topograms showed subtle changes in expressions that could not be supplemented with statistical processing alone, and these were useful in identifying individuality.