Emotion Science in the Twenty-First Century. Time, Sex, and Behavior in Emotion Science: Over and Above

Do subtle cultural differences sculpt face pareidolia?

Face tuning to non-face images such as shadows or grilled toasts is termed face pareidolia. Face-pareidolia images represent a valuable tool for investigation of social cognition in mental disorders. Here we examined (i) whether, and, if so, how face pareidolia is affected by subtle cultural differences; and (ii) whether this impact is modulated by gender. With this purpose in mind, females and males from Northern Italy were administered a set of Face-n-Thing images, photographs of objects such as houses or waves to a varying degree resembling a face. Participants were presented with pareidolia images with canonical upright orientation and display inversion that heavily affects face pareidolia. In a two-alternative forced-choice paradigm, beholders had to indicate whether each image resembled a face. The outcome was compared with the findings obtained in the Southwest of Germany. With upright orientation, neither cultural background nor gender affected face pareidolia. As expected, display inversion generally mired face pareidolia. Yet, while display inversion led to a drastic reduction of face impression in German males as compared to females, in Italians, no gender differences were found. In a nutshell, subtle cultural differences do not sculpt face pareidolia, but instead affect face impression in a gender-specific way under unusual viewing conditions. Clarification of the origins of these effects requires tailored brain imaging work. Implications for transcultural psychiatry, in particular, for schizophrenia research, are highlighted and discussed.

Ties between reading faces, bodies, eyes, and autistic traits

While reading covered with masks faces during the COVID-19 pandemic, for efficient social interaction, we need to combine information from different sources such as the eyes (without faces hidden by masks) and bodies. This may be challenging for individuals with neuropsychiatric conditions, in particular, autism spectrum disorders. Here we examined whether reading of dynamic faces, bodies, and eyes are tied in a gender-specific way, and how these capabilities are related to autistic traits expression. Females and males accomplished a task with point-light faces along with a task with point-light body locomotion portraying different emotional expressions. They had to infer emotional content of displays. In addition, participants were administered the Reading the Mind in the Eyes Test, modified and Autism Spectrum Quotient questionnaire. The findings show that only in females, inferring emotions from dynamic bodies and faces are firmly linked, whereas in males, reading in the eyes is knotted with face reading. Strikingly, in neurotypical males only, accuracy of face, body, and eyes reading was negatively tied with autistic traits. The outcome points to gender-specific modes in social cognition: females rely upon merely dynamic cues while reading faces and bodies, whereas males most likely trust configural information. The findings are of value for examination of face and body language reading in neuropsychiatric conditions, in particular, autism, most of which are gender/sex-specific. This work suggests that if male individuals with autistic traits experience difficulties in reading covered with masks faces, these deficits may be unlikely compensated by reading (even dynamic) bodies and faces. By contrast, in females, reading covered faces as well as reading language of dynamic bodies and faces are not compulsorily connected to autistic traits preventing them from paying high costs for maladaptive social interaction.

Neural circuits underpinning face tuning in male depression

Reading bodies and faces is essential for efficient social interactions, though it may be thought-provoking for individuals with depression. Yet aberrations in the face sensitivity and underwriting neural circuits are not well understood, in particular, in male depression. Here, we use cutting-edge analyses of time course and dynamic topography of gamma oscillatory neuromagnetic cortical activity during administration of a task with Arcimboldo-like images. No difference in face tuning was found between individuals with depression and their neurotypical peers. Furthermore, this behavioral outcome nicely dovetails with magnetoencephalographic data: at early processing stages, the gamma oscillatory response to images resembling a face was rather similar in patients and controls. These bursts originated primarily from the right medioventral occipital cortex and lateral occipital cortex. At later processing stages, however, its topography altered remarkably in depression with profound engagement of the frontal circuits. Yet the primary difference in depressive individuals as compared with their neurotypical peers occurred over the left middle temporal cortices, a part of the social brain, engaged in feature integration and meaning retrieval. The outcome suggests compensatory recruitment of neural resources in male depression.

Reading language of the eyes

The need for assessment of social skills in clinical and neurotypical populations has led to the widespread, and still increasing use of the 'Reading the Mind in the Eyes Test' (RMET) developed more than two decades ago by Simon Baron-Cohen and colleagues for evaluation of social cognition in autism. By analyzing most recent clinical and brain imaging data, we illuminate a set of factors decisive for using the RMET. Converging evidence indicates: (i) In neurotypical individuals, RMET scores are tightly correlated with other social skills (empathy, emotional intelligence, and body language reading); (ii) The RMET assesses recognition of facial affect, but also heavily relies on receptive language skills, semantic knowledge, and memory; (iii) RMET performance is underwritten by the large-scale ensembles of neural networks well-outside the social brain; (iv) The RMET is limited in its capacity to differentiate between neuropsychiatric conditions as well as between stages and severity of a single disorder, though it reliably distinguishes individuals with altered social cognition or elevated pathological traits from neurotypical persons; (v) Merely gender (as a social construct) rather than neurobiological sex influences performance on the RMET; (vi) RMET scores do not substantially decline in healthy aging, and they are higher with higher education level, cognitive abilities, literacy, and mental well-being; (vii) Accuracy on the RMET, and engagement of the social brain, are greater when emotions are expressed and recognized by individuals with similar cultural/ethnic background. Further research is required to better inform usage of the RMET as a tool for swift and reliable examination of social cognition. In light of comparable visual input from the RMET images and faces covered by masks due to COVID-19 regulations, the analysis is of value for keeping efficient social interaction during the current pandemic, in particular, in professional settings related to social communication.

A CNN-Based Framework for Predicting Public Emotion and Multi-Level Behaviors Based on Network Public Opinion

Analysis of network public opinion can help to effectively predict the public emotion and the multi-level government behaviors. Due to the massive and multidimensional characteristics of network public opinion data, the in-depth value mining of public opinion is one of the research bottlenecks. Based on Term Frequency-Inverse Document Frequency (TF-IDF) and deep learning technologies, this paper proposes an advanced TF-IDF mechanism, namely TF-IDF-COR, to extract text feature representations of public opinions and develops a CNN-based prediction model to predict the tendency of publics' emotion and mental health. The proposed method can accurately judge the emotional tendency of network users. The main contribution of this paper is as follows: (1) based on the advantages of TF-IDF mechanism, we propose a TF-IDF-COR mechanism, which integrates the correlation coefficient of word embeddings to TF-IDF. (2) To make the extracted feature semantic information more comprehensive, CNN and TF-IDF-COR are combined to form an effective COR-CNN model for emotion and mental health prediction. Finally, experiments on Sina-Weibo and Twitter opinion data sets show that the improved TF-IDF-COR and the COR-CNN model have better classification performance than traditional classification models. In the experiment, we compare the proposed COR-CNN with support vector machine, k-nearest neighbors, and convolutional neural network in terms of accuracy and F1 score. Experiment results show that COR-CNN performs much better than the three baseline models.

Neural correlates of sex differences in communicative gestures and speech comprehension: A preliminary study

The goal of this study was to investigate whether the semantic processing of the audiovisual combination of communicative gestures with speech differs between men and women. We recorded event-related brain potentials in women and men during the presentation of communicative gestures that were either congruent or incongruent with the speech.Our results showed that incongruent gestures elicited an N400 effect over frontal sites compared to congruent ones in both groups. Moreover, the females showed an earlier N2 response to incongruent stimuli than congruent ones, while larger sustained negativity and late positivity in response to incongruent stimuli was observed only in males. These results suggest that women rapidly recognize and process audiovisual combinations of communicative gestures and speech (as early as 300 ms) whereas men analyze them at the later stages of the process.

Reading Covered Faces

Covering faces with masks, due to mandatory pandemic safety regulations, we can no longer rely on the habitual daily-life information. This may be thought-provoking for healthy people, but particularly challenging for individuals with neuropsychiatric and neurodevelopmental conditions. Au fait research on reading covered faces reveals that: 1) wearing masks hampers facial affect recognition, though it leaves reliable inferring basic emotional expressions; 2) by buffering facial affect, masks lead to narrowing of emotional spectrum and dampen veridical evaluation of counterparts; 3) masks may affect perceived face attractiveness; 4) covered (either by masks or other veils) faces have a certain signal function introducing perceptual biases and prejudices; 5) reading covered faces is gender- and age-specific, being more challenging for males and more variable even in healthy aging; 6) the hampering effects of masks on social cognition occur over the globe; and 7) reading covered faces is likely to be supported by the large-scale assemblies of the neural circuits far beyond the social brain. Challenges and limitations of ongoing research and parallels to the Reading the Mind in the Eyes Test are assessed. Clarification of how masks affect face reading in the real world, where we deal with dynamic faces and have entrée to additional valuable social signals such as body language, as well as the specificity of neural networks underlying reading covered faces calls for further tailored research.

Social cognition in individuals born preterm

Faces hold a substantial value for effective social interactions and sharing. Covering faces with masks, due to COVID-19 regulations, may lead to difficulties in using social signals, in particular, in individuals with neurodevelopmental conditions. Daily-life social participation of individuals who were born preterm is of immense importance for their quality of life. Here we examined face tuning in individuals (aged 12.79 ± 1.89 years) who were born preterm and exhibited signs of periventricular leukomalacia (PVL), a dominant form of brain injury in preterm birth survivors. For assessing the face sensitivity in this population, we implemented a recently developed experimental tool, a set of Face-n-Food images bordering on the style of Giuseppe Arcimboldo. The key benefit of these images is that single components do not trigger face processing. Although a coarse face schema is thought to be hardwired in the brain, former preterms exhibit substantial shortages in the face tuning not only compared with typically developing controls but also with individuals with autistic spectrum disorders. The lack of correlations between the face sensitivity and other cognitive abilities indicates that these deficits are domain-specific. This underscores impact of preterm birth sequelae for social functioning at large. Comparison of the findings with data in individuals with other neurodevelopmental and neuropsychiatric conditions provides novel insights into the origins of deficient face processing.

Face Tuning in Depression

The latest COVID-19 pandemic reveals that unexpected changes elevate depression bringing people apart, but also calling for social sharing. Yet the impact of depression on social cognition and functioning is not well understood. Assessment of social cognition is crucial not only for a better understanding of major depressive disorder (MDD), but also for screening, intervention, and remediation. Here by applying a novel experimental tool, a Face-n-Food task comprising a set of images bordering on the Giuseppe Arcimboldo style, we assessed the face tuning in patients with MDD and person-by-person matched controls. The key benefit of these images is that single components do not trigger face processing. Contrary to common beliefs, the outcome indicates that individuals with depression express intact face responsiveness. Yet, while in depression face sensitivity is tied with perceptual organization, in typical development, it is knotted with social cognition capabilities. Face tuning in depression, therefore, may rely upon altered behavioral strategies and underwriting brain mechanisms. To exclude a possible camouflaging effect of female social skills, we examined gender impact. Neither in depression nor in typical individuals had females excelled in face tuning. The outcome sheds light on the origins of the face sensitivity and alterations in social functioning in depression and mental well-being at large. Aberrant social functioning in depression is likely to be the result of deeply-rooted maladaptive strategies rather than of poor sensitivity to social signals. This has implications for mental well-being under the current pandemic conditions.

Face pareidolia in the brain: Impact of gender and orientation

Research on face sensitivity is of particular relevance during the rapidly evolving Covid-19 pandemic leading to social isolation, but also calling for intact interaction and sharing. Humans possess high sensitivity even to a coarse face scheme, seeing faces in non-face images where real faces do not exist. The advantage of non-face images is that single components do not trigger face processing. Here by implementing a novel set of Face-n-Thing images, we examined (i) how face tuning alters with changing display orientation, and (ii) whether it is affected by observers’ gender. Young females and males were presented with a set of Face-n-Thing images either with canonical upright orientation or inverted 180° in the image plane. Face impression was substantially impeded by display inversion. Furthermore, whereas with upright display orientation, no gender differences were found, with inversion, Face-n-Thing images elicited face impression in females significantly more often. The outcome sheds light on the origins of the face inversion effect in general. Moreover, the findings open a way for examination of face sensitivity and underwriting brain networks in neuropsychiatric conditions related to the current pandemic (such as depression and anxiety), most of which are gender/sex-specific.

Face pareidolia in schizophrenia

BACKGROUND

Faces convey valuable daily life social signals. As in most psychiatric conditions, non-verbal social cognition or its components including face processing may be aberrant in schizophrenia (SZ). Social participation of individuals with SZ is vital for their quality of life, and remediation of social abilities in this population is of high relevance both for society and clinical care.

METHOD

Tuning to faces in non-face images such as shadows, grilled toasts, or ink blots is called face pareidolia. Humans possess high sensitivity to facial signals: even fetuses and infants are well tuned to coarse face cues. Here we assessed face tuning in individuals with SZ and person-by-person matched controls by using a new experimental tool, a set of food-plate images bordering on the Giuseppe Arcimboldo style. The key benefit of these images is that single components do not trigger face processing.

RESULTS AND CONCLUSIONS

The outcome indicates that individuals with SZ exhibit aberrant face tuning in face-like non-face images (χ²(1) = 17.44, p = 0.0001) that can hamper adaptive interaction with peers and social participation hindering, in turn, clinical remediation. Face response rate in SZ patients was related to the scores on the event arrangement task tapping social cognition (Pearson product-moment correlation, r = 0.602, p = 0.01) and on picture completion task assessing visual perceptual organization (Spearman's rho = 0.614, p = 0.009). Therefore, poor performance on the face tuning task is unlikely to be accounted for by deviant general cognitive abilities, but rather by impairments in perceptual integration and social cognition. Comparison of these findings with data in autism and other neuropsychiatric conditions provides novel insights on the origins of face tuning in SZ and triggers brain imaging research.

Untangling the Ties Between Social Cognition and Body Motion: Gender Impact

We proved the viability of the general hypothesis that biological motion (BM) processing serves as a hallmark of social cognition. We assumed that BM processing and inferring emotions through BM (body language reading) are firmly linked and examined whether this tie is gender-specific. Healthy females and males completed two tasks with the same set of point-light BM displays portraying angry and neutral locomotion of female and male actors. For one task, perceivers had to indicate actor gender, while for the other, they had to infer the emotional content of locomotion. Thus, with identical visual input, we directed task demands either to BM processing or inferring of emotion. This design allows straight comparison between sensitivity to BM and recognition of emotions conveyed by the same BM. In addition, perceivers were administered a set of photographs from the Reading the Mind in the Eyes Test (RMET), with which they identified either emotional state or actor gender. Although there were no gender differences in performance on BM tasks, a tight link occurred between recognition accuracy of emotions and gender through BM in males. In females only, body language reading (both accuracy and response time) was associated with performance on the RMET. The outcome underscores gender-specific modes in visual social cognition and triggers investigation of body language reading in a wide range of neuropsychiatric disorders.

How is your mind-set? Proof of concept for the measurement of the level of emotional development

Background

In persons with intellectual and developmental disabilities, not only cognitive brain functions, but also socio-emotional processing networks may be impaired. This study aims to validate the Scale of Emotional Development—Short (SED-S) to provide an instrument for the assessment of socio-emotional brain functions.

Method

The SED-S was applied in 160 children aged 0–12 years. Criterion validity was investigated at item and scale level in terms of the agreement between the scale classification and the child’s chronological age. Additionally, interrater reliability and internal consistency were assessed.

Results

For the majority of items, the expected response pattern emerged, showing the highest response probabilities in the respective target age groups. Agreement between the classification of the different SED-S domains and chronological age was high (κ_{w =} 0.95; exact agreement = 80.6%). Interrater reliability at domain level ranged from κ_w = .98 to 1.00 and internal consistency was high (α = .99).

Conclusion

The study normed the SED-S in a sample of typically developing children and provides evidence for criterion validity on item, domain and scale level.

Social Cognition in Down Syndrome: Face Tuning in Face-Like Non-Face Images

Individuals with Down syndrome (DS) are widely believed to possess considerable socialization strengths. However, the findings on social cognition capabilities are controversial. In the present study, we investigated whether individuals with DS exhibit shortage in face tuning, one of the indispensable components of social cognition. For this purpose, we implemented a recently developed Face-n-Food paradigm with food-plate images composed of food ingredients such as fruits and vegetables. The key benefit of such face-like non-face images is that single elements do not facilitate face processing. In a spontaneous recognition task, 25 children with DS aged 9 to 18 years were presented with a set of Face-n-Food images bordering on the Giuseppe Arcimboldo style. The set of images was administered in a predetermined order from the least to most resembling a face. In DS individuals, thresholds for recognition of the Face-n-Food images as a face were drastically higher as compared not only with typically developing controls, but also with individuals with autistic spectrum disorders and Williams-Beuren syndrome. This outcome represents a significant step toward better conceptualization of the visual social world in DS and neurodevelopmental disorders in general.

Even subtle cultural differences affect face tuning

Culture shapes social cognition in many ways. Yet cultural impact on face tuning remains largely unclear. Here typically developing females and males from the French-speaking part of Switzerland were presented with a set of Arcimboldo-like Face-n-Food images composed of food ingredients and in different degree resembling a face. The outcome had been compared with previous findings obtained in young adults of the South-West Germany. In that study, males exhibit higher thresholds for face tuning on the Face-n-Food task than females. In Swiss participants, no gender differences exist in face tuning. Strikingly, males from the French-speaking part of Switzerland possess higher sensitivity to faces than their German peers, whereas no difference in face tuning occurs between females. The outcome indicates that even relatively subtle cultural differences as well as culture by gender interaction can modulate social cognition. Clarification of the nature of cultural impact on face tuning as well as social cognition at large is of substantial value for understanding a wide range of neuropsychiatric and neurodevelopmental conditions.