Prosody recognition and audiovisual emotion matching in schizophrenia: the contribution of cognition and psychopathology

Automatic recognition of schizophrenia from facial videos using 3D convolutional neural network

Schizophrenia affects patients and their families and society because of chronic impairments in cognition, behavior, and emotion. However, its clinical diagnosis mainly depends on the clinicians' knowledge of the patients' symptoms. Other auxiliary diagnostic methods such as MRI and EEG are cumbersome and time-consuming. Recently, the convolutional neural network (CNN) has been applied to the auxiliary diagnosis of psychiatry. Hence, in this study, a method based on deep learning and facial videos is proposed for the rapid detection of schizophrenia. Herein, 125 videos from 125 schizophrenic patients and 75 videos from 75 healthy controls based on emotional stimulation tasks were obtained. The video preprocessing included the experiment clips extraction, face detection, facial region cropping, resizing to 500 × 500 pixel size, and uniform sampling of 100 frames. The preprocessed facial videos were used to train the Resnet18_3D. We utilized ten-fold cross-validation, and held-out testing set to evaluate the model with the accuracy, the precision, the sensitivity, the specificity, the balanced accuracy, and the AUC. The Resnet18_3D trained on Film_order achieved the best performance with accuracy, sensitivity, specificity, balanced accuracy, and AUC of 89.00%, 96.80%, 76.00%, 86.40% and 0.9397. The neural network model indeed recognizes healthy controls and schizophrenic patients through the changes in the area of the face. The results show that facial video under emotional stimulation can be used to classify schizophrenic patients and help clinicians with diagnosis in the clinical environment. Among the different types of stimuli, the video stimuli with fixed emotional order showed the best classification performance.

Changes in emotion processing and social cognition with auditory versus visual neuroscience-informed cognitive training in individuals with schizophrenia

BACKGROUND

Neuroscience-informed cognitive training has been used to remediate cognitive deficits in schizophrenia, but their effect on emotion processing and social cognition deficits, which may involve auditory and visual impairments, remain relatively unknown. In this study, we compared the efficacy of auditory versus visual neuroscience-informed cognitive training on emotion processing and social cognition in individuals with schizophrenia.

METHODS

In this randomised, double-blind clinical trial, 79 participants with chronic schizophrenia performed 40-hours auditory or visual dynamically equivalent computerised cognitive training. We assessed emotion processing and social cognition using Emotion Recognition, Affective Go-NoGo, Mayer-Salovey-Caruso Emotional-Intelligence, Theory of mind, and Hinting tests before and after 20 h and 40 h of training.

RESULTS

After training, participants from both groups decreased their reaction time for facial emotion recognition (p = 3 × 10-6, d = 0.9). This was more remarkable for the auditory group when analysing individual emotions. Both groups also reduced omissions in the affective go-no go (p = 0.01, d = 0.6), which was also attributed, post hoc, to the auditory group. Trends for improvement were observed in theory of mind (p = 0.06, d = 0.6) for both groups. Improvement in emotion processing was associated with improvement in reasoning and problem solving and global cognition and improvement in theory of mind was associated with improvement in attention and global cognition.

CONCLUSIONS

Both the auditory and the visual neuroscience-informed cognitive training were efficacious at improving emotion processing and social cognition in individuals with schizophrenia, although improvement was more remarkable for the auditory training group. These improvements were related to cognitive - but not symptom - improvement.

Voices 2: Improving Prosodic Recognition in Schizophrenia With an Online Rehabilitation Program

Introduction: Emotion recognition of voices may play an important role in interpersonal communication and patients with schizophrenia present alterations in this regard. Several on-line rehabilitation tools have been developed for treatment in this area. Voices is an on-line prosodic recognition program consisting of identifying different emotional tones in neutral phrases, in different sessions of gradually increasing difficulty. This training tool has previously reported benefits, and a new version has been created called Voices 2. The main aim of this study is to test the capacity of the Voices 2 program to improve emotion recognition through prosody for adults with schizophrenia. Secondly, it seeks to observe durability effects 1 month after intervention.

Method: A randomized, single-blind, multicenter clinical trial was conducted with 44 outpatients diagnosed with schizophrenia or schizoaffective disorder. The intervention group (also called Voices) was treated with Voices 2, whereas the control group was treated with auditory training that was not related to emotions. Sociodemographic and clinical data, clinical state (PANSS), Intelligence Quotient and prosodic recognition (RMV-SV) were measured at baseline. After intervention, RMV-SV and PANSS were assessed. One month later, the RMV-SV measure was repeated.

Results: The control group (n = 19) and the Voices group (n = 22) did not differ on χ², t or U tests in sociodemographic, clinical and psychometric variables at baseline or post-intervention (all p-values > 0.05). In the Voices group, statistically significant differences were observed in the RMV-SV scale applied post-intervention vs. that applied pre-intervention (Z = 2.47, p = 0.013). Similar results were observed in the 1-month follow-up RMV-SV vs. the pre-intervention RMV-SV (Z = 1.97, p = 0.049). PANSS scale was also assessed with no significant differences between pre vs. post measures in both groups. Lastly, Voices 2 was rated relatively higher, based on its ease of understanding, entertainment value, usefulness and the appropriateness of use of its emotional glossary.

Discussion: Improvements were observed in prosodic recognition following intervention with Voices 2 in the Voices group. Although these results are similar to other clinical trial rehabilitation programs, specific research on the matter remains scarce. Certain aspects, such as the durability of effects or adherence should be thoroughly studied and clarified.

Clinical Trial Registration: [https://doi.org/10.17605/OSF.IO/G95C4].

Prosodic deficits and interpersonal difficulties in patients with schizophrenia

The present study examines the use of receptive emotional and linguistic prosody in patients with schizophrenia; particularly, its aim was to evaluate the type and number of errors made when comprehending the emotions and modes implied by meaningless utterances. Seventy-eight participants were enrolled to the study, i.e. two groups (patients with schizophrenia and healthy controls) consisting of 39 subjects. The severity of illness was evaluated with the Positive and Negative Syndrome Scale; comprehension of emotional and linguistic prosody was assessed by the subtests of the Polish Version of the Right Hemisphere Language Battery. Neither emotional nor linguistic prosody comprehension both correlated with schizophrenia symptoms. The study group experienced more difficulties in distinguishing between happiness and anger, and were more likely to misunderstand imperative utterances, confusing them with interrogative or affirmative ones. Such impairments are significant as they may affect the ability to form and sustain relationships with other people, achieve success in the work environment, and integrate in the community. They may also be a trait mark of the illness independent of psychotic symptoms. Further research is needed to translate this knowledge into meaningful and therapeutic interventions to improve quality of life, both for affected individuals and for their communication partners.

Paliperidone Compared with Haloperidol on the Theory of Mind Tasks in Schizophrenia: A Pilot Trial

PURPOSE

Theory of mind (ToM) is an important part of social cognitive function and is associated with medial prefrontal cortical (mPFC) activity. This study aimed to evaluate the efficacy of paliperidone in improving ToM task performance in patients with schizophrenia compared with haloperidol.

PATIENTS AND METHODS

This study was a single-center, single-blinded (assessor), parallel-group randomized clinical trial of patients with schizophrenia randomized to paliperidone or haloperidol. ToM was assessed at weeks 0, 8, 12, and 16 using the first-order belief, higher-order belief, faux-pas, and Reading the Mind in the Eyes tests. The primary outcome was the change in the ToM performance scores from baseline to after 16 weeks of treatment.

RESULTS

The participants received paliperidone (n = 29) or haloperidol (n = 31). For the first-order belief task, there were no between-group differences (P > 0.05) but time differences in both groups (P < 0.05). For the higher-order belief task, there were no between-group differences (P > 0.05), but there were time differences in both groups (P < 0.05) and a time×group interaction in the paliperidone group only (P < 0.05). For the faux-pas task, there was a difference between groups at week 16 (P < 0.05), and the improvement in time was significant for the paliperidone group only (P < 0.05). For the Reading the Mind in the Eyes task, there was an improvement over time for the paliperidone group only (P < 0.05). Safety was manageable in both groups.

CONCLUSION

Paliperidone treatment might be more effective than haloperidol in improving ToM task performance in schizophrenia.

TRIAL REGISTRATION

chictr.org.cn_identifier ChiCTR-IPR-15007635.

Effects of an Audiovisual Emotion Perception Training for Schizophrenia: A Preliminary Study

Individuals with schizophrenia show a reduced ability to integrate facial and vocal information in emotion perception. Although emotion perception has been a target for treatment, no study has yet examined the effect of multimodal training on emotion perception in schizophrenia. In the present study, we developed an audiovisual emotion perception training and test in which a voice and a face were simultaneously presented, and subjects were asked to judge whether the emotions of the voice and the face matched. The voices were either angry or happy, and the faces were morphed on a continuum ranging from angry to happy. Sixteen patients with schizophrenia participated in six training sessions and three test sessions (i.e., pre-training, post-training, and generalization). Eighteen healthy controls participated only in pre-training test session. Prior to training, the patients with schizophrenia performed significantly worse than did the controls in the recognition of anger; however, following the training, the patients showed a significant improvement in recognizing anger, which was maintained and generalized to a new set of stimuli. The patients also improved the recognition of happiness following the training, but this effect was not maintained or generalized. These results provide preliminary evidence that a multimodal, audiovisual training may yield improvements in anger perception for patients with schizophrenia.

Multisensory Integration of Emotion in Schizophrenic Patients

Multisensory integration (MSI) of emotion has been increasingly recognized as an essential element of schizophrenic patients' impairments, leading to the breakdown of their interpersonal functioning. The present review provides an updated synopsis of schizophrenics' MSI abilities in emotion processing by examining relevant behavioral and neurological research. Existing behavioral studies have adopted well-established experimental paradigms to investigate how participants understand multisensory emotion stimuli, and interpret their reciprocal interactions. Yet it remains controversial with regard to congruence-induced facilitation effects, modality dominance effects, and generalized vs specific impairment hypotheses. Such inconsistencies are likely due to differences and variations in experimental manipulations, participants' clinical symptomatology, and cognitive abilities. Recent electrophysiological and neuroimaging research has revealed aberrant indices in event-related potential (ERP) and brain activation patterns, further suggesting impaired temporal processing and dysfunctional brain regions, connectivity and circuities at different stages of MSI in emotion processing. The limitations of existing studies and implications for future MSI work are discussed in light of research designs and techniques, study samples and stimuli, and clinical applications.

Speech Prosody as a Bridge Between Psychopathology and Linguistics: The Case of the Schizophrenia Spectrum

Patients with schizophrenia spectrum disorders experience severe difficulties in interpersonal communication, as described by traditional psychopathology and current research on social cognition. From a linguistic perspective, pragmatic abilities are crucial for successful communication. Empirical studies have shown that these abilities are significantly impaired in this group of patients. Prosody, the tone of voice with which words and sentences are pronounced, is one of the most important carriers of pragmatic meaning and can serve a range of functions from linguistic to emotional ones. Most of the existing literature on prosody of patients with schizophrenia spectrum disorders focuses on the expression of emotion, generally showing significant impairments. By contrast, the use of non-emotional prosody in these patients is scarcely investigated. In this paper, we first present a linguistic model to classify prosodic functions. Second, we discuss existing studies on the use of non-emotional prosody in these patients, providing an overview of the state of the art. Third, we delineate possible future lines of research in this field, also taking into account some classical psychopathological assumptions, for both diagnostic and therapeutic purposes.

Dissociated features of social cognition altered in mouse models of schizophrenia: Focus on social dominance and acoustic communication

Social and communication impairments are common features of psychiatric disorders. Animal models of schizophrenia display various social deficits due to difference in tests, mouse strains and drugs. Moreover, communication deficits have not been studied. Our objectives were to assess and compare three major features of social cognition in different mouse models of schizophrenia: interest for a social stimulus, organization and acceptance of social contact, and acoustic communication to question whether mouse models for schizophrenia with social dysfunction also exhibit vocal communication defects. To achieve these aims we treated acutely C57BL/6J mice either with MK-801 or ketamine and tested WT and microtubule-associated protein 6 -MAP6- KO mice in two complementary social tasks: the 3-chamber test which measures social motivation and the social interaction task -SIT- which relies on prefrontal cortex activity and measures the ability to organize and respond to a real interaction, and which promotes ultrasonic vocalizations. Our results reveal that schizophrenia models have intact interest for a social stimulus in the 3-chamber test. However, thanks to principal component analyses of social interaction data, we demonstrate that social motivation and the ability to act socially rely on distinct mechanisms in revealing a decrease in dominance and communication in pharmacological schizophrenia models along with social withdraw, classically observed in schizophrenia, in MK-801 model. In this latter model, some social parameters can be significantly improved by aripiprazole, an atypical antipsychotic. Our social protocol, combined with fine-tuned analysis, is expected to provide an innovative framework for testing future treatments in preclinical models. This article is part of the Special Issue entitled 'The neuropharmacology of social behavior: from bench to bedside'.

Impaired facial and vocal emotion decoding in schizophrenia is underpinned by basic perceptivo-motor deficits

INTRODUCTION

Emotional decoding impairments have been largely demonstrated in schizophrenia for facial and prosodic stimuli, when presented separately. Nevertheless, the exploration of crossmodal integration has been far less considered, despite its omnipresence in daily social interactions. Moreover, the role played by basic visuo-motor impairments in unimodal and crossmodal decoding remains unexplored.

METHODS

Thirty-two patients were compared with 32 matched controls in an emotional decoding task including unimodal (visual and auditory) and crossmodal (congruent and incongruent) conditions. A control perceptive task was also conducted to take potential low-level perceptual deficits into account.

RESULTS

Schizoprenic patients presented lower performance and higher reaction times for both unimodal tasks (visual and auditory) and crossmodal conditions. Moreover, reaction times for the visuo-perceptive task were also significantly longer for patients compared to controls.

CONCLUSIONS

The consistency of the results across unimodal and crossmodal tasks suggests a globalised emotional impairment in schizophrenia, independent of the sensorial modality and crossmodal nature of the stimuli. Centrally, given the results in the visuo-perceptive task, the impairments observed for emotional recognition appears at least partly explained by primary cognitive deficits, namely reduced processing speed.

A Theoretical and Clinical Perspective on Social Relatedness and the Patient With Serious Mental Illness

BACKGROUND

A therapeutic relationship forms the basis of care of patients in psychiatric mental health nursing. However, individuals with schizophrenia have difficulty participating in these relationships. Recent research in the area of social cognitive psychology offers that deficits in this area affect the flow of perceiving and relating in interpersonal relationships. This literature has not been applied to nursing.

OBJECTIVE

The objective of this article is to review the theories underpinning therapeutic relationships from a nursing and psychological perspective, including the newest research from social cognitive psychology.

DESIGN

The article presents a literature review of the theories of nursing, psychology, and social cognitive science. Two patient case studies are used as examples of application of the theories.

RESULTS

This article incorporates new knowledge about the components of social cognition to inform nurses as they build therapeutic relationships with patients with chronic and persistent mental illnesses.

CONCLUSIONS

The science of social cognitive psychology offers nursing a new perspective on the evolving therapeutic nurse-patient relationship with patients with chronic and persistent mental illnesses. It has implications for clinicians, educators, and nurse scientists.

Integration of verbal and nonverbal emotional signals in patients with schizophrenia: Decreased nonverbal dominance

In day-to-day social interaction, emotions are usually expressed by verbal (e.g. spoken words) and nonverbal signals (e.g. facial expressions, prosody). In case of conflicting signals nonverbal signals are perceived as being the more reliable source of information. Deficits in interpreting nonverbal signals - as described for patients with schizophrenic disorders - might interfere with the ability to integrate verbal and nonverbal social cues into a meaningful whole. The aim of this study was to examine how schizophrenic disorders influence the integration of verbal and nonverbal signals. For this purpose short video sequences were presented to 21 patients with schizophrenia and 21 healthy controls. Each sequence showed an actor speaking a short sentence with independently varying emotional connotations at the verbal and the nonverbal level. The participants rated the valence of the speaker's emotional state on a four-point scale (from very negative to very positive). The relative impact of nonverbal cues as compared to verbal cues on these ratings was evaluated. Both groups base their decisions primarily on nonverbal information. However, this effect is significantly less prominent in the patient group. Patients tend to base their decisions less on nonverbal signals and more on verbal information than healthy controls.

Effects of cue modality and emotional category on recognition of nonverbal emotional signals in schizophrenia

BACKGROUND

Impaired interpretation of nonverbal emotional cues in patients with schizophrenia has been reported in several studies and a clinical relevance of these deficits for social functioning has been assumed. However, it is unclear to what extent the impairments depend on specific emotions or specific channels of nonverbal communication.

METHODS

Here, the effect of cue modality and emotional categories on accuracy of emotion recognition was evaluated in 21 patients with schizophrenia and compared to a healthy control group (n = 21). To this end, dynamic stimuli comprising speakers of both genders in three different sensory modalities (auditory, visual and audiovisual) and five emotional categories (happy, alluring, neutral, angry and disgusted) were used.

RESULTS

Patients with schizophrenia were found to be impaired in emotion recognition in comparison to the control group across all stimuli. Considering specific emotions more severe deficits were revealed in the recognition of alluring stimuli and less severe deficits in the recognition of disgusted stimuli as compared to all other emotions. Regarding cue modality the extent of the impairment in emotional recognition did not significantly differ between auditory and visual cues across all emotional categories. However, patients with schizophrenia showed significantly more severe disturbances for vocal as compared to facial cues when sexual interest is expressed (alluring stimuli), whereas more severe disturbances for facial as compared to vocal cues were observed when happiness or anger is expressed.

CONCLUSION

Our results confirmed that perceptual impairments can be observed for vocal as well as facial cues conveying various social and emotional connotations. The observed differences in severity of impairments with most severe deficits for alluring expressions might be related to specific difficulties in recognizing the complex social emotional information of interpersonal intentions as compared to "basic" emotional states. Therefore, future studies evaluating perception of nonverbal cues should consider a broader range of social and emotional signals beyond basic emotions including attitudes and interpersonal intentions. Identifying specific domains of social perception particularly prone for misunderstandings in patients with schizophrenia might allow for a refinement of interventions aiming at improving social functioning.

Facial emotion perception in schizophrenia: Does sex matter?

AIM: To review the literature on sex differences in facial emotion perception (FEP) across the schizophrenia spectrum.

METHODS: We conducted a systematic review of empirical articles that were included in five separate meta-analyses of FEP across the schizophrenia spectrum, including meta-analyses that predominantly examined adults with chronic schizophrenia, people with early (onset prior to age 18) or recent-onset (experiencing their first or second psychotic episode or illness duration less than 2 years) schizophrenia, and unaffected first-degree relatives of people with schizophrenia. We also examined articles written in English (from November 2011 through June 2015) that were not included in the aforementioned meta-analyses through a literature search in the PubMed database. All relevant articles were accessed in full text. We examined all studies to determine the sample sizes, diagnostic characteristics, demographic information, methodologies, results, and whether each individual study reported on sex differences. The results from the meta-analyses themselves as well as the individual studies are reported in tables and text.

RESULTS: We retrieved 134 articles included in five separate meta-analyses and the PubMed database that examined FEP across the schizophrenia spectrum. Of these articles, 38 examined sex differences in FEP. Thirty of these studies did not find sex differences in FEP in either chronically ill adults with schizophrenia, early-onset or recently diagnosed people with schizophrenia, or first-degree relatives of people with schizophrenia. Of the eight studies that found sex differences in FEP, three found that chronically ill women outperformed men, one study found that girls with early-onset schizophrenia outperformed boys, and two studies found that women (including first-degree relatives, adults with schizophrenia, and the healthy control group) outperformed men on FEP tasks. In total, six of the eight studies that examined sex differences in FEP found that women outperformed men across the schizophrenia spectrum.

CONCLUSION: Evidence to date suggests few sex differences in FEP in schizophrenia; both men and women across the schizophrenia spectrum have deficits in FEP.

Factors contributing to social cognition impairment in borderline personality disorder and schizophrenia

Social cognition (SC) deficits have been described both in patients with schizophrenia and borderline personality disorder (BPD). However, while the former tend towards simplistic mental state attributions (undermentalizing), the latter are more likely to make overly complex mental state inferences (overmentalizing). Performance on complex SC tasks has been shown to correlate with neurocognitive ability, emotion perception, a history of trauma, and overconfidence in errors. However, it is unclear how these factors relate to different aspects of SC deficits. Aim of the present study was to examine the pathways of SC impairment by investigating performance profiles and their predictors comparatively in BPD and schizophrenia. Participants were 44 patients with BPD, 36 patients with schizophrenia, and 38 healthy controls. Undermentalizing and overmentalizing were assessed with an ecologically valid SC task. Patients with BPD exhibited increased overmentalizing, whereas patients with schizophrenia showed a more extensive deficit pattern, their main error type being undermentalizing. Overconfidence in errors was the most important predictor for overmentalizing, while undermentalizing depended mainly on verbal memory and emotion perception. Thus, BPD und schizophrenia exhibited different SC impairment patterns, and different types of SC errors were predicted by different factors. These findings have implications for the optimization of treatment approaches.

Social cognition in schizophrenia

Individuals with schizophrenia exhibit impaired social cognition, which manifests as difficulties in identifying emotions, feeing connected to others, inferring people's thoughts and reacting emotionally to others. These social cognitive impairments interfere with social connections and are strong determinants of the degree of impaired daily functioning in such individuals. Here, we review recent findings from the fields of social cognition and social neuroscience and identify the social processes that are impaired in schizophrenia. We also consider empathy as an example of a complex social cognitive function that integrates several social processes and is impaired in schizophrenia. This information may guide interventions to improve social cognition in patients with this disorder.

Can you hear what I feel? A validated prosodic set of angry, happy, and neutral Italian pseudowords

This contribution aims to establish a set of validated vocal Italian pseudowords that convey three emotional tones (angry, happy, and neutral) for prosodic emotional processing research. We elaborated the materials by following a series of specific steps. First, we tested the valence of a set of written pseudowords generated by specific software. Two Italian actors (male and female) then recorded the resulting subset of linguistically legal and neutral pseudowords in three emotional tones. Finally, on the basis of the results of independent ratings of emotional intensity, we selected a set of 30 audio stimuli expressed in each of the three different emotions. Acoustic analyses indicated that the prosodic indexes of fundamental frequency, vocal intensity, and speech rate anchored individual perceptions of the emotions expressed. Finally, the acoustic profile of the set of emotional stimuli confirmed previous findings. The happy tone stimuli showed high f0 values, high intensity, high pitch variability, and a faster speech rate. The angry tone stimuli were also characterized by high f0 and intensity, but by relatively smaller pitch variability and a lower speech rate. This last profile echoes the description of "cold anger." This new set of prosodic emotion stimuli will constitute a useful resource for future research that requires emotional prosody materials. It could be used both for Italian and for cross-language studies.

Mirroring the self: testing neurophysiological correlates of disturbed self-experience in schizophrenia spectrum

BACKGROUND

Self-disorders (SDs) have been described as a core schizophrenia spectrum vulnerability phenotype, both in classic and contemporary psychopathological literature. However, such a core phenotype has not yet been investigated adopting a trans-domain approach that combines the phenomenological and the neurophysiological levels of analysis. The aim of this study is to investigate the relation between SDs and subtle, schizophrenia-specific impairments of emotional resonance that are supposed to reflect abnormalities in the mirror neurons mechanism. Specifically, we tested whether electromyographic response to emotional stimuli (i.e. a proxy for subtle changes in facial mimicry and related motor resonance mechanisms) would predict the occurrence of anomalous subjective experiences (i.e. SDs).

SAMPLING AND METHODS

Eighteen schizophrenia spectrum (SzSp) patients underwent a comprehensive psychopathological examination and were contextually tested with a multimodal paradigm, recording facial electromyographic activity of muscles in response to positive and negative emotional stimuli. Experiential anomalies were explored with the Bonn Scale for the Assessment of Basic Symptoms (BSABS) and then condensed into rational subscales mapping SzSp anomalous self-experiences.

RESULTS

SzSp patients showed an imbalance in emotional motor resonance with a selective bias toward negative stimuli, as well as a multisensory integration impairment. Multiple regression analysis showed that electromyographic facial reactions in response to negative stimuli presented in auditory modality specifically and strongly correlated with SD subscore.

CONCLUSIONS

The study confirms the potential of SDs as target phenotype for neurobiological research and encourages research into disturbed motor/emotional resonance as possible body-level correlate of disturbed subjective experiences in SzSp.

Relationship between amygdala volume and emotion recognition in adolescents at ultra-high risk for psychosis

Amygdala volume has been proposed as a neural risk biomarker for psychotic illness, but findings in the ultra-high risk for psychosis (UHR) population have been somewhat inconsistent, which may be related to underlying social cognitive abilities. The current study investigated whether amygdala volumes were related to emotion-recognition impairments in UHR individuals, and whether volumes differed by sex. Secondary aims were to assess whether (a) emotion-recognition performance was associated with interhemispheric amygdala volume asymmetry and (b) amgydala volume and volume asymmetry acted as a mediator between emotion-recognition and outcome measures. The amygdala was manually delineated from magnetic resonance images for 39 UHR individuals who had also completed facial and prosody emotion-recognition tasks. Partial correlations were conducted to examine associations between amydgala volume/asymmetry and recognition of negative emotions. Mediation analyses were conducted using regression and bootstrapping techniques. Amygdala volume was positively correlated with sadness emotion recognition, in particular prosody, for females only. Left amygdala volume mediated the effect of sadness recognition on depressive symptoms, negative symptoms, overall psychopathology, and global functioning in females. Findings suggest a complex relationship between emotion recognition, the structure of the amygdala and illness outcome, where recognition of sadness appears to be the precipitator of this relationship in UHR females. Further research is needed to determine illness specificity and to confirm our sex- and emotion-specific results.

Perception of emotion-related conflict in human communications: what are the effects of schizophrenia?

Our ability to make sense of emotional cues is of paramount importance for understanding state of mind and communicative intent. However, emotional cues often conflict with each other; this presents a significant challenge for people with schizophrenia. We conducted a theoretical review to determine the extent and types of impaired processing of emotion-related conflict in schizophrenia; we evaluated the relationship with medication and symptoms, and considered possible mediatory mechanisms. The literature established that people with schizophrenia demonstrated impaired function: (i) when passively exposed to emotion cues whilst performing an unrelated task, (ii) when selectively attending to one source of emotion cues whilst trying to ignore interference from another source, and (iii) when trying to resolve conflicting emotion cues and judge meta-communicative intent. These deficits showed associations with both negative and positive symptoms. There was limited evidence for antipsychotic medications attenuating impaired emotion perception when there are conflicting cues, with further direct research needed. Impaired attentional control and context processing may underlie some of the observed impairments. Neuroanatomical correlates are likely to involve interhemispheric transfer via the corpus callosum, limbic regions such as the amygdala, and possibly dorsolateral prefrontal and anterior cingulate cortex through their role in conflict processing.

Neurocognition and symptoms identify links between facial recognition and emotion processing in schizophrenia: meta-analytic findings

BACKGROUND

In schizophrenia patients, one of the most commonly studied deficits of social cognition is emotion processing (EP), which has documented links to facial recognition (FR). But, how are deficits in facial recognition linked to emotion processing deficits? Can neurocognitive and symptom correlates of FR and EP help differentiate the unique contribution of FR to the domain of social cognition?

METHODS

A meta-analysis of 102 studies (combined n=4826) in schizophrenia patients was conducted to determine the magnitude and pattern of relationships between facial recognition, emotion processing, neurocognition, and type of symptom.

RESULTS

Meta-analytic results indicated that facial recognition and emotion processing are strongly interrelated (r=.51). In addition, the relationship between FR and EP through voice prosody (r=.58) is as strong as the relationship between FR and EP based on facial stimuli (r=.53). Further, the relationship between emotion recognition, neurocognition, and symptoms is independent of the emotion processing modality - facial stimuli and voice prosody.

DISCUSSION

The association between FR and EP that occurs through voice prosody suggests that FR is a fundamental cognitive process. The observed links between FR and EP might be due to bottom-up associations between neurocognition and EP, and not simply because most emotion recognition tasks use visual facial stimuli. In addition, links with symptoms, especially negative symptoms and disorganization, suggest possible symptom mechanisms that contribute to FR and EP deficits.

Facial reactions in response to dynamic emotional stimuli in different modalities in patients suffering from schizophrenia: a behavioral and EMG study

Emotional facial expression is an important low-level mechanism contributing to the experience of empathy, thereby lying at the core of social interaction. Schizophrenia is associated with pervasive social cognitive impairments, including emotional processing of facial expressions. In this study we test a novel paradigm in order to investigate the evaluation of the emotional content of perceived emotions presented through dynamic expressive stimuli, facial mimicry evoked by the same stimuli, and their functional relation. Fifteen healthy controls and 15 patients diagnosed with schizophrenia were presented with stimuli portraying positive (laugh), negative (cry) and neutral (control) emotional stimuli in visual, auditory modalities in isolation, and congruently or incongruently associated. Participants where requested to recognize and quantitatively rate the emotional value of the perceived stimuli, while electromyographic activity of Corrugator and Zygomaticus muscles was recorded. All participants correctly judged the perceived emotional stimuli and prioritized the visual over the auditory modality in identifying the emotion when they were incongruently associated (Audio-Visual Incongruent condition). The neutral emotional stimuli did not evoke any muscle responses and were judged by all participants as emotionally neutral. Control group responded with rapid and congruent mimicry to emotional stimuli, and in Incongruent condition muscle responses were driven by what participants saw rather than by what they heard. Patient group showed a similar pattern only with respect to negative stimuli, whereas showed a lack of or a non-specific Zygomaticus response when positive stimuli were presented. Finally, we found that only patients with reduced facial mimicry (Internalizers) judged both positive and negative emotions as significantly more neutral than controls. The relevance of these findings for studying emotional deficits in schizophrenia is discussed.

"To see or not to see: that is the question." The "Protection-Against-Schizophrenia" (PaSZ) model: evidence from congenital blindness and visuo-cognitive aberrations

The causes of schizophrenia are still unknown. For the last 100 years, though, both “absent” and “perfect” vision have been associated with a lower risk for schizophrenia. Hence, vision itself and aberrations in visual functioning may be fundamental to the development and etiological explanations of the disorder. In this paper, we present the “Protection-Against-Schizophrenia” (PaSZ) model, which grades the risk for developing schizophrenia as a function of an individual's visual capacity. We review two vision perspectives: (1) “Absent” vision or how congenital blindness contributes to PaSZ and (2) “perfect” vision or how aberrations in visual functioning are associated with psychosis. First, we illustrate that, although congenitally blind and sighted individuals acquire similar world representations, blind individuals compensate for behavioral shortcomings through neurofunctional and multisensory reorganization. These reorganizations may indicate etiological explanations for their PaSZ. Second, we demonstrate that visuo-cognitive impairments are fundamental for the development of schizophrenia. Deteriorated visual information acquisition and processing contribute to higher-order cognitive dysfunctions and subsequently to schizophrenic symptoms. Finally, we provide different specific therapeutic recommendations for individuals who suffer from visual impairments (who never developed “normal” vision) and individuals who suffer from visual deterioration (who previously had “normal” visual skills). Rather than categorizing individuals as “normal” and “mentally disordered,” the PaSZ model uses a continuous scale to represent psychiatrically relevant human behavior. This not only provides a scientific basis for more fine-grained diagnostic assessments, earlier detection, and more appropriate therapeutic assignments, but it also outlines a trajectory for unraveling the causes of abnormal psychotic human self- and world-perception.

Cocaine users manifest impaired prosodic and cross-modal emotion processing

BACKGROUND

A small number of previous studies have provided evidence that cocaine users (CU) exhibit impairments in complex social cognition tasks, while the more basic facial emotion recognition is widely unaffected. However, prosody and cross-modal emotion processing has not been systematically investigated in CU so far. Therefore, the aim of the present study was to assess complex multisensory emotion processing in CU in comparison to controls and to examine a potential association with drug use patterns.

METHOD

The abbreviated version of the comprehensive affect testing system (CATS-A) was used to measure emotion perception across the three channels of facial affect, prosody, and semantic content in 58 CU and 48 healthy control (HC) subjects who were matched for age, sex, verbal intelligence, and years of education.

RESULTS

CU had significantly lower scores than controls in the quotient scales of "emotion recognition" and "prosody recognition" and the subtests "conflicting prosody/meaning - attend to prosody" and "match emotional prosody to emotional face" either requiring to attend to prosody or to integrate cross-modal information. In contrast, no group difference emerged for the "affect recognition quotient." Cumulative cocaine doses and duration of cocaine use correlated negatively with emotion processing.

CONCLUSION

CU show impaired cross-modal integration of different emotion processing channels particularly with regard to prosody, whereas more basic aspects of emotion processing such as facial affect perception are comparable to the performance of HC.