Deficits in the Mimicry of Facial Expressions in Parkinson's Disease

Effects of facial biofeedback on hypomimia, emotion recognition, and affect in Parkinson's disease

OBJECTIVES

Facial expressions are a core component of emotions and nonverbal social communication. Therefore, hypomimia as secondary symptom of Parkinson's disease (PD) has adverse effects like social impairment, stigmatization, under-diagnosis and under-treatment of depression, and a generally lower quality of life. Beside unspecific dopaminergic treatment, specific treatment options for hypomimia in PD are rarely investigated. This quasi-randomized controlled trial evaluated the short-term effects of facial electromyogram (EMG) based biofeedback to enhance facial expression and emotion recognition as nonverbal social communication skills in PD patients. Furthermore effects on affect are examined.

METHOD

A sample of 34 in-patients with PD were allocated either to facial EMG-biofeedback as experimental group or non-facial exercises as control group. Facial expression during posing of emotions (measured via EMG), facial emotion recognition, and positive and negative affect were assessed before and after treatment. Stronger improvements were expected in the EMG-biofeedback in comparison to the control group.

RESULTS

The facial EMG-biofeedback group showed significantly greater improvements in overall facial expression, and especially for happiness and disgust. Also, overall facial emotion recognition abilities improved significantly stronger in the experimental group. Positive affect was significantly increased in both groups with no significant differences between them, while negative affect did not change within both groups.

CONCLUSIONS

The study provides promising evidence for facial EMG-biofeedback as a tool to improve facial expression and emotion recognition in PD. Embodiment theories are discussed as working mechanism.

Neurocognitive Impairment and Social Cognition in Parkinson's Disease Patients

In addition to motor symptoms, neurocognitive impairment (NCI) affects patients with prodromal Parkinson's disease (PD). NCI in PD ranges from subjective cognitive complaints to dementia. The purpose of this review is to present the available evidence of NCI in PD and highlight the heterogeneity of NCI phenotypes as well as the range of factors that contribute to NCI onset and progression. A review of publications related to NCI in PD up to March 2023 was performed using PubMed/Medline. There is an interconnection between the neurocognitive and motor symptoms of the disease, suggesting a common underlying pathophysiology as well as an interconnection between NCI and non-motor symptoms, such as mood disorders, which may contribute to confounding NCI. Motor and non-motor symptom evaluation could be used prognostically for NCI onset and progression in combination with imaging, laboratory, and genetic data. Additionally, the implications of NCI on the social cognition of afflicted patients warrant its prompt management. The etiology of NCI onset and its progression in PD is multifactorial and its effects are equally grave as the motor effects. This review highlights the importance of the prompt identification of subjective cognitive complaints in PD patients and NCI management.

Facial expressions and identities recognition in Parkinson disease

Parkinson's Disease (PD) is associated with motor and non-motor symptoms. Among the latter are deficits in matching, identification, and recognition of emotional facial expressions. On one hand, this deficit has been attributed to a dysfunction in emotion processing. Another explanation (which does not exclude the former) links this deficit with reduced facial expressiveness in these patients, which prevents them from properly understanding or embodying emotions. To disentangle the specific contribution of emotion comprehension and that of facial expression processing in PD's observed deficit with emotions we performed two experiments on non-emotional facial expressions. In Experiment 1, a group of PD patients and a group of Healthy Controls (HC) underwent a task of non-emotional expression recognition in faces of different identity and a task of identity recognition in faces with different expression. No differences were observed between the two groups in accuracies. In Experiment 2, PD patients and Healthy Controls underwent a task where they had to recognize the identity of faces encoded through a non-emotional facial expression, through a rigid head movement, or as neutral. Again, no group differences were observed. In none of the two experiments hypomimia scores had a specific effect on expression processing. We conclude that in PD patients the observed impairment with emotional expressions is likely due to a specific deficit for emotions to a greater extent than for facial expressivity processing.

The Story behind the Mask: A Narrative Review on Hypomimia in Parkinson's Disease

Facial movements are crucial for social and emotional interaction and well-being. Reduced facial expressions (i.e., hypomimia) is a common feature in patients with Parkinson's disease (PD) and previous studies linked this manifestation to both motor symptoms of the disease and altered emotion recognition and processing. Nevertheless, research on facial motor impairment in PD has been rather scarce and only a limited number of clinical evaluation tools are available, often suffering from poor validation processes and high inter- and intra-rater variability. In recent years, the availability of technology-enhanced quantification methods of facial movements, such as automated video analysis and machine learning application, led to increasing interest in studying hypomimia in PD. In this narrative review, we summarize the current knowledge on pathophysiological hypotheses at the basis of hypomimia in PD, with particular focus on the association between reduced facial expressions and emotional processing and analyze the current evaluation tools and management strategies for this symptom, as well as future research perspectives.

I spy with my little eye: The detection of changes in emotional faces and the influence of facial feedback in Parkinson disease

BACKGROUND AND PURPOSE

Parkinson disease (PD) is a progressive neurodegenerative disorder that affects the motor system but also involves deficits in emotional processing such as facial emotion recognition. In healthy participants, it has been shown that facial mimicry, the automatic imitation of perceived facial expressions, facilitates the interpretation of the emotional states of our counterpart. In PD patients, recent studies revealed reduced facial mimicry and consequently reduced facial feedback, suggesting that this reduction might contribute to the prominent emotion recognition deficits found in PD.

METHODS

We investigated the influence of facial mimicry on facial emotion recognition. Twenty PD patients and 20 healthy controls (HCs) underwent a classical facial mimicry manipulation (holding a pen with the lips, teeth, or nondominant hand) while performing an emotional change detection task with faces.

RESULTS

As expected, emotion recognition was significantly influenced by facial mimicry manipulation in HCs, further supporting the hypothesis of facial feedback and the related theory of embodied simulation. Importantly, patients with PD, generally and independent from the facial mimicry manipulation, were impaired in their ability to detected emotion changes. Our data further show that PD patients' facial emotional recognition abilities are completely unaffected by mimicry manipulation, suggesting that PD patients cannot profit from an artificial modulation of the already impaired facial feedback.

CONCLUSIONS

These findings suggest that it is not the hypomimia and the absence of facial feedback per se, but a disruption of the facial feedback loop, that leads to the prominent emotion recognition deficit in PD patients.

Music-based interventions in community settings: Navigating the tension between rigor and ecological validity

Empirical research of community-based music interventions has advanced to investigate the individual, social, and educational implications of arts-for-wellbeing practices. Here, we present the motivations, aims, hypotheses, and implications of this complex field of inquiry. We describe examples of recent large-scale investigations to reflect on the major methodological challenges. Community-based music interventions strike a balance between the empirical rigor of clinical trials and the demands of ecological validity. We argue that this balance should be viewed as an asset rather than a mere pragmatic compromise. We also offer some perspectives on best-practice models for effectively engaging in this type of work.

Facial muscle movements in patients with Parkinson's disease undergoing phonation tests

Purpose

Parkinson's disease (PD) is a serious neurodegenerative disease affecting the elderly. In general, the locomotion deficit, which seriously affects the daily life of patients with PD, usually occurs at a later stage. The mask face symptom meanwhile progressively worsens. However, facial muscle disorders and changes involved in the freezing mask are unclear.

Method

In this study, we recruited 35 patients with PD and 26 age- and sex-balanced controls to undergo phonation tests, while the built-in camera on the laptop recorded their facial expressions during the whole pronunciation process. Furthermore, FaceReader (version 7.0; Noldus Information Technology, Wageningen, Netherlands) was used to analyze changes in PD facial landmark movement and region movement.

Results

The two-tailed Student's t-test showed that the changes in facial landmark movement among 49 landmarks were significantly lower in patients with PD than in the control group (P < 0.05). The data on facial region movement revealed that the eyes and upper lip of patients with PD differed significantly from those in the control group.

Conclusion

Patients with PD had defects in facial landmark movement and regional movement when producing a single syllable, double syllable, and multiple syllables, which may be related to reduced facial expressions in patients with PD.

Hypomimia May Influence the Facial Emotion Recognition Ability in Patients with Parkinson's Disease

BACKGROUND

Hypomimia is a clinical feature of Parkinson's disease (PD). Based on the embodied simulation theory, the impairment of facial mimicry may worsen facial emotion recognition; however, the empirical results are inconclusive.

OBJECTIVE

We aimed to explore the worsening of emotion recognition by hypomimia. We further explored the relationship between the hypomimia, emotion recognition, and social functioning.

METHODS

A total of 114 participants were recruited. The patients with PD and normal controls (NCs) were matched for demographic characteristics. All the participants completed the Mini-Mental State Examination and the Chinese Multi-modalities Emotion Recognition Test. In addition to the above tests, the patients were assessed with the Movement Disorder Society-Unified Parkinson's Disease Rating Scale and Parkinson's Disease Social Functioning Scale (PDSFS).

RESULTS

Patients with PD with hypomimia had worse recognition of disgust than NCs (p = 0.018). The severity of hypomimia was predictive of the recognition of disgust (β= -0.275, p = 0.028). Facial emotion recognition was predictive of the PDSFS score of PD patients (β= 0.433, p = 0.001). We also found that recognizing disgust could mediate the relationship between hypomimia and the PDSFS score (β= 0.264, p = 0.045).

CONCLUSION

Patients with hypomimia had the worst disgust facial recognition. Hypomimia may affect the social function of PD patients, which is related to recognizing the expression of disgust. Emotion recognition training may improve the social function of patients with PD.

Audio and visual speech emotion activate the left pre-supplementary motor area

Sensorimotor brain areas have been implicated in the recognition of emotion expressed on the face and through nonverbal vocalizations. However, no previous study has assessed whether sensorimotor cortices are recruited during the perception of emotion in speech-a signal that includes both audio (speech sounds) and visual (facial speech movements) components. To address this gap in the literature, we recruited 24 participants to listen to speech clips produced in a way that was either happy, sad, or neutral in expression. These stimuli also were presented in one of three modalities: audio-only (hearing the voice but not seeing the face), video-only (seeing the face but not hearing the voice), or audiovisual. Brain activity was recorded using electroencephalography, subjected to independent component analysis, and source-localized. We found that the left presupplementary motor area was more active in response to happy and sad stimuli than neutral stimuli, as indexed by greater mu event-related desynchronization. This effect did not differ by the sensory modality of the stimuli. Activity levels in other sensorimotor brain areas did not differ by emotion, although they were greatest in response to visual-only and audiovisual stimuli. One possible explanation for the pre-SMA result is that this brain area may actively support speech emotion recognition by using our extensive experience expressing emotion to generate sensory predictions that in turn guide our perception.

Quantitative Evaluation of Hypomimia in Parkinson's Disease: A Face Tracking Approach

Parkinson’s disease (PD) is a neurological disorder that mainly affects the motor system. Among other symptoms, hypomimia is considered one of the clinical hallmarks of the disease. Despite its great impact on patients’ quality of life, it remains still under-investigated. The aim of this work is to provide a quantitative index for hypomimia that can distinguish pathological and healthy subjects and that can be used in the classification of emotions. A face tracking algorithm was implemented based on the Facial Action Coding System. A new easy-to-interpret metric (face mobility index, FMI) was defined considering distances between pairs of geometric features and a classification based on this metric was proposed. Comparison was also provided between healthy controls and PD patients. Results of the study suggest that this index can quantify the degree of impairment in PD and can be used in the classification of emotions. Statistically significant differences were observed for all emotions when distances were taken into account, and for happiness and anger when FMI was considered. The best classification results were obtained with Random Forest and kNN according to the AUC metric.

Simulated proximity enhances perceptual and physiological responses to emotional facial expressions

Physical proximity is important in social interactions. Here, we assessed whether simulated physical proximity modulates the perceived intensity of facial emotional expressions and their associated physiological signatures during observation or imitation of these expressions. Forty-four healthy volunteers rated intensities of dynamic angry or happy facial expressions, presented at two simulated locations, proximal (0.5 m) and distant (3 m) from the participants. We tested whether simulated physical proximity affected the spontaneous (in the observation task) and voluntary (in the imitation task) physiological responses (activity of the corrugator supercilii face muscle and pupil diameter) as well as subsequent ratings of emotional intensity. Angry expressions provoked relative activation of the corrugator supercilii muscle and pupil dilation, whereas happy expressions induced a decrease in corrugator supercilii muscle activity. In proximal condition, these responses were enhanced during both observation and imitation of the facial expressions, and were accompanied by an increase in subsequent affective ratings. In addition, individual variations in condition related EMG activation during imitation of angry expressions predicted increase in subsequent emotional ratings. In sum, our results reveal novel insights about the impact of physical proximity in the perception of emotional expressions, with early proximity-induced enhancements of physiological responses followed by an increased intensity rating of facial emotional expressions.

Effects of Increasing the Overall Level or Fitting Hearing Aids on Emotional Responses to Sounds

Adults with hearing loss demonstrate a reduced range of emotional responses to nonspeech sounds compared to their peers with normal hearing. The purpose of this study was to evaluate two possible strategies for addressing the effects of hearing loss on emotional responses: (a) increasing overall level and (b) hearing aid use (with and without nonlinear frequency compression, NFC). Twenty-three adults (mean age  =  65.5 years) with mild-to-severe sensorineural hearing loss and 17 adults (mean age  =  56.2 years) with normal hearing participated. All adults provided ratings of valence and arousal without hearing aids in response to nonspeech sounds presented at a moderate and at a high level. Adults with hearing loss also provided ratings while using individually fitted study hearing aids with two settings (NFC-OFF or NFC-ON). Hearing loss and hearing aid use impacted ratings of valence but not arousal. Listeners with hearing loss rated pleasant sounds as less pleasant than their peers, confirming findings in the extant literature. For both groups, increasing the overall level resulted in lower ratings of valence. For listeners with hearing loss, the use of hearing aids (NFC-OFF) also resulted in lower ratings of valence but to a lesser extent than increasing the overall level. Activating NFC resulted in ratings that were similar to ratings without hearing aids (with a moderate presentation level) but did not improve ratings to match those from the listeners with normal hearing. These findings suggest that current interventions do not ameliorate the effects of hearing loss on emotional responses to sound.

Measurement of the facial nerve thickness and its correlation with freezing phenomenon and hypomimia in Parkinson's disease

OBJECTIVE

Hypomimia is one of the diagnostic features in Parkinson's disease, and freezing blocks may also occur with the degenerative process. We investigated that the thickness of the cranial facial nerve that innervates facial expressions, and the relationship between bradymimia and freezing phenomena in these patients.

METHODS

We included 70 patients and healthy participants in this cross-sectional study. Clinical characteristics, modified Hoehn and Yahr (mHY) stages, Unified Parkinson's Disease Rating Scale (UPDRS) scores and Freezing of Gait Questionnaire (FOG-Q) scores of Parkinson's patients were recorded. FOG-Q was scored between 1 and 4 points. The thickness of the facial nerve and its neighborhoods of numerically equal groups were measured radiologically in the internal acoustic channel (IAC) with magnetic resonance imaging of temporal lobe. Right and left facial nerve thicknesses were compared.

RESULTS

The right facial nerve thicknesses of the patient and control groups were measured as 0.97 ± 0.12 mm and 1.20 ± 0.10 mm at the proximal level and 0.71 (0.69-0.81) mm and 1.21 (1.13-1.24) mm at the distal level, respectively (P < 0.001). Notably, the facial nerve was more atrophied on the right than on the left (P < 0.001). Facial nerve to IAC ratio (%) was significantly lower and cerebrospinal fluid thickness distance (%) measurement was significantly higher (P < 0.001). Also, the FOG-Q and facial nerve to IAC ratio were negatively correlated (P = 0.049, rho = -0.335).

CONCLUSIONS

Our study provides new information about the facial nerve and its neighborhoods and clinical relationships in individuals with PD. In studies investigating hypomimia and FOG in Parkinson's, neuroimaging of the facial nerve can also be used. These results need to be proven with more comprehensive studies.

Autonomic factors do not underlie the elevated self-disgust levels in Parkinson's disease

INTRODUCTION

Parkinson's disease (PD) is manifested along with non-motor symptoms such as impairments in basic emotion regulation, recognition and expression. Yet, self-conscious emotion (SCEs) such as self-disgust, guilt and shame are under-investigated. Our previous research indicated that Parkinson patients have elevated levels of self-reported and induced self-disgust. However, the cause of that elevation-whether lower level biophysiological factors, or higher level cognitive factors, is unknown.

METHODS

To explore the former, we analysed Skin Conductance Response (SCR, measuring sympathetic activity) amplitude and high frequency Heart Rate Variability (HRV, measuring parasympathetic activity) across two emotion induction paradigms, one involving narrations of personal experiences of self-disgust, shame and guilt, and one targeting self-disgust selectively via images of the self. Both paradigms had a neutral condition.

RESULTS

Photo paradigm elicited significant changes in physiological responses in patients relative to controls-higher percentages of HRV in the high frequency range but lower SCR amplitudes, with patients to present lower responses compared to controls. In the narration paradigm, only guilt condition elicited significant SCR differences between groups.

CONCLUSIONS

Consequently, lower level biophysiological factors are unlikely to cause elevated self-disgust levels in Parkinson's disease, which by implication suggests that higher level cognitive factors may be responsible.

Artificial intelligence in neurodegenerative diseases: A review of available tools with a focus on machine learning techniques

Neurodegenerative diseases have shown an increasing incidence in the older population in recent years. A significant amount of research has been conducted to characterize these diseases. Computational methods, and particularly machine learning techniques, are now very useful tools in helping and improving the diagnosis as well as the disease monitoring process. In this paper, we provide an in-depth review on existing computational approaches used in the whole neurodegenerative spectrum, namely for Alzheimer's, Parkinson's, and Huntington's Diseases, Amyotrophic Lateral Sclerosis, and Multiple System Atrophy. We propose a taxonomy of the specific clinical features, and of the existing computational methods. We provide a detailed analysis of the various modalities and decision systems employed for each disease. We identify and present the sleep disorders which are present in various diseases and which represent an important asset for onset detection. We overview the existing data set resources and evaluation metrics. Finally, we identify current remaining open challenges and discuss future perspectives.

Automated Computer Vision Assessment of Hypomimia in Parkinson Disease: Proof-of-Principle Pilot Study

BACKGROUND

Facial expressions require the complex coordination of 43 different facial muscles. Parkinson disease (PD) affects facial musculature leading to "hypomimia" or "masked facies."

OBJECTIVE

We aimed to determine whether modern computer vision techniques can be applied to detect masked facies and quantify drug states in PD.

METHODS

We trained a convolutional neural network on images extracted from videos of 107 self-identified people with PD, along with 1595 videos of controls, in order to detect PD hypomimia cues. This trained model was applied to clinical interviews of 35 PD patients in their on and off drug motor states, and seven journalist interviews of the actor Alan Alda obtained before and after he was diagnosed with PD.

RESULTS

The algorithm achieved a test set area under the receiver operating characteristic curve of 0.71 on 54 subjects to detect PD hypomimia, compared to a value of 0.75 for trained neurologists using the United Parkinson Disease Rating Scale-III Facial Expression score. Additionally, the model accuracy to classify the on and off drug states in the clinical samples was 63% (22/35), in contrast to an accuracy of 46% (16/35) when using clinical rater scores. Finally, each of Alan Alda's seven interviews were successfully classified as occurring before (versus after) his diagnosis, with 100% accuracy (7/7).

CONCLUSIONS

This proof-of-principle pilot study demonstrated that computer vision holds promise as a valuable tool for PD hypomimia and for monitoring a patient's motor state in an objective and noninvasive way, particularly given the increasing importance of telemedicine.

Social Symptoms of Parkinson's Disease

Parkinson's disease (PD) is typically well recognized by its characteristic motor symptoms (e.g., bradykinesia, rigidity, and tremor). The cognitive symptoms of PD are increasingly being acknowledged by clinicians and researchers alike. However, PD also involves a host of emotional and communicative changes which can cause major disruptions to social functioning. These incude problems producing emotional facial expressions (i.e., facial masking) and emotional speech (i.e., dysarthria), as well as difficulties recognizing the verbal and nonverbal emotional cues of others. These social symptoms of PD can result in severe negative social consequences, including stigma, dehumanization, and loneliness, which might affect quality of life to an even greater extent than more well-recognized motor or cognitive symptoms. It is, therefore, imperative that researchers and clinicans become aware of these potential social symptoms and their negative effects, in order to properly investigate and manage the socioemotional aspects of PD. This narrative review provides an examination of the current research surrounding some of the most common social symptoms of PD and their related social consequences and argues that proactively and adequately addressing these issues might improve disease outcomes.

The neuroscience of sadness: A multidisciplinary synthesis and collaborative review

Sadness is typically characterized by raised inner eyebrows, lowered corners of the mouth, reduced walking speed, and slumped posture. Ancient subcortical circuitry provides a neuroanatomical foundation, extending from dorsal periaqueductal grey to subgenual anterior cingulate, the latter of which is now a treatment target in disorders of sadness. Electrophysiological studies further emphasize a role for reduced left relative to right frontal asymmetry in sadness, underpinning interest in the transcranial stimulation of left dorsolateral prefrontal cortex as an antidepressant target. Neuroimaging studies - including meta-analyses - indicate that sadness is associated with reduced cortical activation, which may contribute to reduced parasympathetic inhibitory control over medullary cardioacceleratory circuits. Reduced cardiac control may - in part - contribute to epidemiological reports of reduced life expectancy in affective disorders, effects equivalent to heavy smoking. We suggest that the field may be moving toward a theoretical consensus, in which different models relating to basic emotion theory and psychological constructionism may be considered as complementary, working at different levels of the phylogenetic hierarchy.

Impairments in face discrimination and emotion recognition are related to aging and cognitive dysfunctions in Parkinson's disease with dementia

Patients with Parkinson’s disease (PD) suffer from motor and non-motor symptoms; 40% would develop dementia (PD-D). Impaired face and emotion processing in PD has been reported; however, the deficits of face processing in PD-D remain unclear. We investigated three essential aspects of face processing capacity in PD-D, and the associations between cognitive, neuropsychiatric assessments and task performances. Twenty-four PD-D patients (mean age: 74.0 ± 5.55) and eighteen age-matched healthy controls (HC) (mean age: 71.0 ± 6.20) received three computerized tasks, morphing-face discrimination, dynamic facial emotion recognition, and expression imitation. Compared to HC, PD-D patients had lower sensitivity (d’) and greater neural internal noises in discriminating faces; responded slower and had difficulties with negative emotions; imitated some expressions but with lower strength. Correlation analyses revealed that patients with advancing age, slow mentation, and poor cognition (but not motor symptoms) showed stronger deterioration in face perception. Importantly, these correlations were absent in the age-matched HC. The present study is among the first few examined face processing in patients with PD-D, and found consistent deficits correlated with advancing age and slow mentation. We propose that face discrimination task could be included as a potential test for the early detection of dementia in PD.

[Effects of facial rehabilitation exercise on the mood, facial expressions, and facial muscle activities in patients with Parkinson's disease]

AIM

Facial expressions are often impaired in patients with Parkinson's disease (PD). Few studies have examined the effects of head and neck rehabilitation in patients with PD using a facial expression analysis. In the present study, to further elucidate the effects of facial rehabilitation exercise in patients with PD, a three-dimensional facial expression analysis with FaceReader™ and surface electromyography (EMG) were performed in order to assess the facial expressions and muscle activities, respectively. The effects of such exercises on the mood and mental health were also evaluated.

METHOD

Twenty-one patients with PD (63.3±12.1 years) participated in the present study and were randomly assigned to an intervention group and non-intervention group. Facial rehabilitation exercise was performed for 60 minutes once a week for 12 weeks in the intervention group. GHQ-12, the facial expression analysis with FaceReader™, surface EMG, and the VAS scale for mood changes were used to evaluate the effects of the program. The results from both groups were compared.

RESULTS

The results from eight patients in the intervention group and five in the non-intervention group were analyzed. FaceReader™ revealed a higher "Happy" index and lower "Sad" index in the intervention group than in the non-intervention group, and a significant interaction "Happy" index by an analysis of variance was noted between the two groups. EMG also showed increases in the activity of facial muscles in the intervention group. The subjects' mood improved after each facial rehabilitation exercise session.

CONCLUSION

The results of the present study suggest that the facial rehabilitation exercise affected the mood, facial expression, and facial muscle activities in patients with PD and indicate that the expression analysis with the FaceReader™ and surface EMG are useful for evaluating the effects of facial rehabilitation exercise.

Facial expression recognition for monitoring neurological disorders based on convolutional neural network

Facial expressions are a significant part of non-verbal communication. Recognizing facial expressions of people with neurological disorders is essential because these people may have lost a significant amount of their verbal communication ability. Such an assessment requires time consuming examination involving medical personnel, which can be quite challenging and expensive. Automated facial expression recognition systems that are low-cost and noninvasive can help experts detect neurological disorders. In this study, an automated facial expression recognition system is developed using a novel deep learning approach. The architecture consists of four-stage networks. The first, second and third networks segment the facial components which are essential for facial expression recognition. Owing to the three networks, an iconize facial image is obtained. The fourth network classifies facial expressions using raw facial images and iconize facial images. This four-stage method combines holistic facial information with local part-based features to achieve more robust facial expression recognition. Preliminary experimental results achieved 94.44% accuracy for facial expression recognition on RaFD database. The proposed system produced 5% improvement than the facial expression recognition system by using raw images. This study presents a quantitative, objective and non-invasive facial expression recognition system to help in the monitoring and diagnosis of neurological disorders influencing facial expressions.

Out of Focus: Facial Feedback Manipulation Modulates Automatic Processing of Unattended Emotional Faces

Facial expressions provide information about an individual's intentions and emotions and are thus an important medium for nonverbal communication. Theories of embodied cognition assume that facial mimicry and resulting facial feedback plays an important role in the perception of facial emotional expressions. Although behavioral and electrophysiological studies have confirmed the influence of facial feedback on the perception of facial emotional expressions, the influence of facial feedback on the automatic processing of such stimuli is largely unexplored. The automatic processing of unattended facial expressions can be investigated by visual expression-related MMN. The expression-related MMN reflects a differential ERP of automatic detection of emotional changes elicited by rarely presented facial expressions (deviants) among frequently presented facial expressions (standards). In this study, we investigated the impact of facial feedback on the automatic processing of facial expressions. For this purpose, participants (n = 19) performed a centrally presented visual detection task while neutral (standard), happy, and sad faces (deviants) were presented peripherally. During the task, facial feedback was manipulated by different pen holding conditions (holding the pen with teeth, lips, or nondominant hand). Our results indicate that automatic processing of facial expressions is influenced and thus dependent on the own facial feedback.

Objective assessment of blinking and facial expressions in Parkinson's disease using a vertical electro-oculogram and facial surface electromyography

OBJECTIVE

Hypomimia is a common and early symptom of Parkinson's disease (PD), which reduces the ability of PD patients to manifest emotions. Currently, it is visually evaluated by the neurologist during neurological examinations for PD diagnosis, as described in task 3.2 of the Movement Disorder Society-Unified Parkinson's Disease Rating Scale (MDS-UPDRS). Since such an evaluation is semi-quantitative and affected by inter-variability, this paper aims to measure the physiological parameters related to eye blink and facial expressions extracted from a vertical electro-oculogram (VEOG) and facial surface electromyography (fsEMG) to differentiate PD patients from healthy control subjects (HCs).

APPROACH

The spontaneous eye blink rate-minute (sEBR), its maximum amplitude (BMP), and facial cutaneous muscle activity were measured in 24 PD patients and 24 HCs while the subjects looked at a visual-tester composed of three main parts: static vision, dynamic vision and reading silently. Specificity and sensitivity for each parameter were calculated.

MAIN RESULTS

The VEOG and the fsEMG allowed the identification of some parameters related to eye blink and facial expressions (i.e. sEBR, BMP, frontal and peribuccal muscular activities), being able to distinguish between PD patients and HCs with high sensitivity and specificity.

SIGNIFICANCE

The demonstration that the combination of parameters related to eye blink and facial expressions can discriminate (with high accuracy) between PD patients versus HCs, thus resulting in a useful tool to support the neurologist in objective assessment of hypomimia for improving PD diagnosis.

Voluntary and spontaneous facial mimicry toward other's emotional expression in patients with Parkinson's disease

A "masked face", that is, decreased facial expression is considered as one of the cardinal symptoms among individuals with Parkinson's disease (PD). Both spontaneous and voluntary mimicry toward others' emotional expressions is essential for both social communication and emotional sharing with others. Despite many studies showing impairments in facial movements in PD in general, it is still unclear whether voluntary, spontaneous, or both types of mimicry are affected and how the impairments affect the patients' quality of life. We investigated to verify whether impairments in facial movements happen for spontaneous as well as for voluntary expressions by quantitatively comparing muscle activations using surface electromyography. Dynamic facial expressions of Neutral, Anger, Joy, and Sad were presented during recordings in corrugator and zygomatic areas. In the spontaneous condition, participants were instructed to simply watch clips, whereas in the voluntary condition they were instructed to actively mimic the stimuli. We found that PD patients showed decreased mimicry in both spontaneous and voluntary conditions compared to a matched control group, although movement patterns in each emotion were similar in the two groups. Moreover, whereas the decrease in mimicry correlated with the decrease not in a health-related quality of life index (PDQ), it did so in a more subjective measurement of general quality of life index (SWB). The correlation between facial mimicry and subjective well-being index suggests that the 'masked face' symptom deteriorates patients' quality of life in a complex way affecting social and psychological aspects, which in turn may be linked to the increased depression risk among individuals with PD.

Problems with Facial Mimicry Might Contribute to Emotion Recognition Impairment in Parkinson's Disease

Difficulty with emotion recognition is increasingly being recognized as a symptom of Parkinson's disease. Most research into this area contends that progressive cognitive decline accompanying the disease is to be blamed. However, facial mimicry (i.e., the involuntary congruent activation of facial expression muscles upon viewing a particular facial expression) might also play a role and has been relatively understudied in this clinical population. In healthy participants, facial mimicry has been shown to improve recognition of observed emotions, a phenomenon described by embodied simulation theory. Due to motor disturbances, Parkinson's disease patients frequently show reduced emotional expressiveness, which translates into reduced mimicry. Therefore, it is likely that facial mimicry problems in Parkinson's disease contribute at least partly to the emotional recognition deficits that these patients experience and might greatly influence their social cognition abilities and quality of life. The present review aims to highlight the need for further inquiry into the motor mechanisms behind emotional recognition in Parkinson's disease by synthesizing behavioural, physiological, and neuroanatomical evidence.

Assessing Normal Smiling Function Through 3D-3D Surfaces Registration: An Innovative Method for the Assessment of Facial Mimicry

Smiling has a great importance, both from a functional and an aesthetic point of view. The introduction of modern 3D acquisition and elaboration methods may provide additional help in the evaluation of facial mimicry. This study aims at proposing an innovative method to assess facial and labial movements in different types of smiles. Twenty healthy subjects (10 males, 10 females, mean age 27.5 years, SD 4.5 years), were recorded through a stereophotogrammetric system in neutral position and in three types of smiles: Mona-Lisa smile, canine smile, full-denture smile. All the 3D smiling models were superimposed on the corresponding neutral one and point-to-point root mean square (RMS) differences were computed. Labial surface areas in rest position and during each smile were calculated as well, together with the percentage modification in different types of smile. RMS values (facial and labial models), labial surface areas and percentage modifications were compared through ANCOVA tests to verify possible statistically significant differences according to sex and type of smile (p < 0.05). Intercanthal labial width was considered a covariate. RMS values progressively increased from Mona-Lisa to full-denture smile; statistically significant differences were found among all types of smiles, both for facial and labial models (p < 0.05), while no statistically significant sex and sex × smile interactions were found (p > 0.05). Labial surface and percentage of modification showed statistically significant differences according to both sex and type of smile (p < 0.05). The study provides a novel contribution to the field of sexual dimorphism in smiling. Level of Evidence V This journal requires that authors assign a level of evidence to each article. For a full description of these evidence-based medicine ratings, please refer to the Table of Contents or the online Instructions to Authors www.springer.com/00266 .

Facial emotion recognition in Parkinson's disease: A review and new hypotheses

Parkinson's disease is a neurodegenerative disorder classically characterized by motor symptoms. Among them, hypomimia affects facial expressiveness and social communication and has a highly negative impact on patients' and relatives' quality of life. Patients also frequently experience nonmotor symptoms, including emotional-processing impairments, leading to difficulty in recognizing emotions from faces. Aside from its theoretical importance, understanding the disruption of facial emotion recognition in PD is crucial for improving quality of life for both patients and caregivers, as this impairment is associated with heightened interpersonal difficulties. However, studies assessing abilities in recognizing facial emotions in PD still report contradictory outcomes. The origins of this inconsistency are unclear, and several questions (regarding the role of dopamine replacement therapy or the possible consequences of hypomimia) remain unanswered. We therefore undertook a fresh review of relevant articles focusing on facial emotion recognition in PD to deepen current understanding of this nonmotor feature, exploring multiple significant potential confounding factors, both clinical and methodological, and discussing probable pathophysiological mechanisms. This led us to examine recent proposals about the role of basal ganglia-based circuits in emotion and to consider the involvement of facial mimicry in this deficit from the perspective of embodied simulation theory. We believe our findings will inform clinical practice and increase fundamental knowledge, particularly in relation to potential embodied emotion impairment in PD. © 2018 The Authors. Movement Disorders published by Wiley Periodicals, Inc. on behalf of International Parkinson and Movement Disorder Society.

Neural Correlates of Facial Mimicry: Simultaneous Measurements of EMG and BOLD Responses during Perception of Dynamic Compared to Static Facial Expressions

Facial mimicry (FM) is an automatic response to imitate the facial expressions of others. However, neural correlates of the phenomenon are as yet not well established. We investigated this issue using simultaneously recorded EMG and BOLD signals during perception of dynamic and static emotional facial expressions of happiness and anger. During display presentations, BOLD signals and zygomaticus major (ZM), corrugator supercilii (CS) and orbicularis oculi (OO) EMG responses were recorded simultaneously from 46 healthy individuals. Subjects reacted spontaneously to happy facial expressions with increased EMG activity in ZM and OO muscles and decreased CS activity, which was interpreted as FM. Facial muscle responses correlated with BOLD activity in regions associated with motor simulation of facial expressions [i.e., inferior frontal gyrus, a classical Mirror Neuron System (MNS)]. Further, we also found correlations for regions associated with emotional processing (i.e., insula, part of the extended MNS). It is concluded that FM involves both motor and emotional brain structures, especially during perception of natural emotional expressions.

Hearing, Emotion, Amplification, Research, and Training Workshop: Current Understanding of Hearing Loss and Emotion Perception and Priorities for Future Research

The question of how hearing loss and hearing rehabilitation affect patients' momentary emotional experiences is one that has received little attention but has considerable potential to affect patients' psychosocial function. This article is a product from the Hearing, Emotion, Amplification, Research, and Training workshop, which was convened to develop a consensus document describing research on emotion perception relevant for hearing research. This article outlines conceptual frameworks for the investigation of emotion in hearing research; available subjective, objective, neurophysiologic, and peripheral physiologic data acquisition research methods; the effects of age and hearing loss on emotion perception; potential rehabilitation strategies; priorities for future research; and implications for clinical audiologic rehabilitation. More broadly, this article aims to increase awareness about emotion perception research in audiology and to stimulate additional research on the topic.

Impaired Emotional Mirroring in Parkinson's Disease-A Study on Brain Activation during Processing of Facial Expressions

Background

Affective dysfunctions are common in patients with Parkinson’s disease, but the underlying neurobiological deviations have rarely been examined. Parkinson’s disease is characterized by a loss of dopamine neurons in the substantia nigra resulting in impairment of motor and non-motor basal ganglia-cortical loops. Concerning emotional deficits, some studies provide evidence for altered brain processing in limbic- and lateral-orbitofrontal gating loops. In a second line of evidence, human premotor and inferior parietal homologs of mirror neuron areas were involved in processing and understanding of emotional facial expressions. We examined deviations in brain activation during processing of facial expressions in patients and related these to emotion recognition accuracy.

Methods

13 patients and 13 healthy controls underwent an emotion recognition task and a functional magnetic resonance imaging (fMRI) measurement. In the Emotion Hexagon test, participants were presented with blends of two emotions and had to indicate which emotion best described the presented picture. Blended pictures with three levels of difficulty were included. During fMRI scanning, participants observed video clips depicting emotional, non-emotional, and neutral facial expressions or were asked to produce these facial expressions themselves.

Results

Patients performed slightly worse in the emotion recognition task, but only when judging the most ambiguous facial expressions. Both groups activated inferior frontal and anterior inferior parietal homologs of mirror neuron areas during observation and execution of the emotional facial expressions. During observation, responses in the pars opercularis of the right inferior frontal gyrus, in the bilateral inferior parietal lobule and in the bilateral supplementary motor cortex were decreased in patients. Furthermore, in patients, activation of the right anterior inferior parietal lobule was positively related to accuracy in the emotion recognition task.

Conclusion

Our data provide evidence for a contribution of human homologs of monkey mirror areas to the emotion recognition deficit in Parkinson’s disease.

Altered emotional recognition and expression in patients with Parkinson's disease

BACKGROUND

Parkinson's disease (PD) patients exhibit deficits in emotional recognition and expression abilities, including emotional faces and voices. The aim of this study was to explore emotional processing in pre-deep brain stimulation (pre-DBS) PD patients using two sensory modalities (visual and auditory).

METHODS

Fifteen PD patients who needed DBS surgery and 15 healthy, age- and gender-matched controls were recruited as participants. All participants were assessed by the Karolinska Directed Emotional Faces database 50 Faces Recognition test. Vocal recognition was evaluated by the Montreal Affective Voices database 50 Voices Recognition test. For emotional facial expression, the participants were asked to imitate five basic emotions (neutral, happiness, anger, fear, and sadness). The subjects were required to express nonverbal vocalizations of the five basic emotions. Fifteen Chinese native speakers were recruited as decoders. We recorded the accuracy of the responses, reaction time, and confidence level.

RESULTS

For emotional recognition and expression, the PD group scored lower on both facial and vocal emotional processing than did the healthy control group. There were significant differences between the two groups in both reaction time and confidence level. A significant relationship was also found between emotional recognition and emotional expression when considering all participants between the two groups together.

CONCLUSION

The PD group exhibited poorer performance on both the recognition and expression tasks. Facial emotion deficits and vocal emotion abnormalities were associated with each other. In addition, our data allow us to speculate that emotional recognition and expression may share a common system.

Altered Kinematics of Facial Emotion Expression and Emotion Recognition Deficits Are Unrelated in Parkinson's Disease

BACKGROUND

Altered emotional processing, including reduced emotion facial expression and defective emotion recognition, has been reported in patients with Parkinson's disease (PD). However, few studies have objectively investigated facial expression abnormalities in PD using neurophysiological techniques. It is not known whether altered facial expression and recognition in PD are related.

OBJECTIVE

To investigate possible deficits in facial emotion expression and emotion recognition and their relationship, if any, in patients with PD.

METHODS

Eighteen patients with PD and 16 healthy controls were enrolled in this study. Facial expressions of emotion were recorded using a 3D optoelectronic system and analyzed using the facial action coding system. Possible deficits in emotion recognition were assessed using the Ekman test. Participants were assessed in one experimental session. Possible relationship between the kinematic variables of facial emotion expression, the Ekman test scores, and clinical and demographic data in patients were evaluated using the Spearman's test and multiple regression analysis.

RESULTS

The facial expression of all six basic emotions had slower velocity and lower amplitude in patients in comparison to healthy controls (all Ps < 0.05). Patients also yielded worse Ekman global score and disgust, sadness, and fear sub-scores than healthy controls (all Ps < 0.001). Altered facial expression kinematics and emotion recognition deficits were unrelated in patients (all Ps > 0.05). Finally, no relationship emerged between kinematic variables of facial emotion expression, the Ekman test scores, and clinical and demographic data in patients (all Ps > 0.05).

CONCLUSION

The results in this study provide further evidence of altered emotional processing in PD. The lack of any correlation between altered facial emotion expression kinematics and emotion recognition deficits in patients suggests that these abnormalities are mediated by separate pathophysiological mechanisms.