Sadness is unique: neural processing of emotions in speech prosody in musicians and non-musicians

Does music training improve emotion recognition and cognitive abilities? Longitudinal and correlational evidence from children

Music training is widely claimed to enhance nonmusical abilities, yet causal evidence remains inconclusive. Moreover, research tends to focus on cognitive over socioemotional outcomes. In two studies, we investigated whether music training improves emotion recognition in voices and faces among school-aged children. We also examined music-training effects on musical abilities, motor skills (fine and gross), broader socioemotional functioning, and cognitive abilities including nonverbal reasoning, executive functions, and auditory memory (short-term and working memory). Study 1 (N = 110) was a 2-year longitudinal intervention conducted in a naturalistic school setting, comparing music training to basketball training (active control) and no training (passive control). Music training improved fine-motor skills and auditory memory relative to controls, but it had no effect on emotion recognition or other cognitive and socioemotional abilities. Both music and basketball training improved gross-motor skills. Study 2 (N = 192) compared children without music training to peers attending a music school. Although music training correlated with better emotion recognition in speech prosody (tone of voice), this association disappeared after controlling for socioeconomic status, musical abilities, or short-term memory. In contrast, musical abilities correlated with emotion recognition in both prosody and faces, independently of training or other confounding variables. These findings suggest that music training enhances fine-motor skills and auditory memory, but it does not causally improve emotion recognition, other cognitive abilities, or socioemotional functioning. Observed advantages in emotion recognition likely stem from preexisting musical abilities and other confounding factors such as socioeconomic status.

Instrumental music training relates to intensity assessment but not emotional prosody recognition in Mandarin

Building on research demonstrating the benefits of music training for emotional prosody recognition in nontonal languages, this study delves into its unexplored influence on tonal languages. In tonal languages, the acoustic similarity between lexical tones and music, along with the dual role of pitch in conveying lexical and affective meanings, create a unique interplay. We evaluated 72 participants, half of whom had extensive instrumental music training, with the other half serving as demographically matched controls. All participants completed an online test consisting of 210 Chinese pseudosentences, each designed to express one of five emotions: happiness, sadness, fear, anger, or neutrality. Our robust statistical analyses, which included effect size estimates and Bayesian factors, revealed that music and nonmusic groups exhibit similar abilities in identifying the emotional prosody of various emotions. However, the music group attributed higher intensity ratings to emotional prosodies of happiness, fear, and anger compared to the nonmusic group. These findings suggest that while instrumental music training is not related to emotional prosody recognition, it does appear to be related to perceived emotional intensity. This dissociation between emotion recognition and intensity evaluation adds a new piece to the puzzle of the complex relationship between music training and emotion perception in tonal languages.

Spectrotemporal cues and attention jointly modulate fMRI network topology for sentence and melody perception

Speech and music are two fundamental modes of human communication. Lateralisation of key processes underlying their perception has been related both to the distinct sensitivity to low-level spectrotemporal acoustic features and to top-down attention. However, the interplay between bottom-up and top-down processes needs to be clarified. In the present study, we investigated the contribution of acoustics and attention to melodies or sentences to lateralisation in fMRI functional network topology. We used sung speech stimuli selectively filtered in temporal or spectral modulation domains with crossed and balanced verbal and melodic content. Perception of speech decreased with degradation of temporal information, whereas perception of melodies decreased with spectral degradation. Applying graph theoretical metrics on fMRI connectivity matrices, we found that local clustering, reflecting functional specialisation, linearly increased when spectral or temporal cues crucial for the task goal were incrementally degraded. These effects occurred in a bilateral fronto-temporo-parietal network for processing temporally degraded sentences and in right auditory regions for processing spectrally degraded melodies. In contrast, global topology remained stable across conditions. These findings suggest that lateralisation for speech and music partially depends on an interplay of acoustic cues and task goals under increased attentional demands.

Electrophysiological Correlates of Vocal Emotional Processing in Musicians and Non-Musicians

Musicians outperform non-musicians in vocal emotion recognition, but the underlying mechanisms are still debated. Behavioral measures highlight the importance of auditory sensitivity towards emotional voice cues. However, it remains unclear whether and how this group difference is reflected at the brain level. Here, we compared event-related potentials (ERPs) to acoustically manipulated voices between musicians (n = 39) and non-musicians (n = 39). We used parameter-specific voice morphing to create and present vocal stimuli that conveyed happiness, fear, pleasure, or sadness, either in all acoustic cues or selectively in either pitch contour (F0) or timbre. Although the fronto-central P200 (150-250 ms) and N400 (300-500 ms) components were modulated by pitch and timbre, differences between musicians and non-musicians appeared only for a centro-parietal late positive potential (500-1000 ms). Thus, this study does not support an early auditory specialization in musicians but suggests instead that musicality affects the manner in which listeners use acoustic voice cues during later, controlled aspects of emotion evaluation.

Singing ability is related to vocal emotion recognition: Evidence for shared sensorimotor processing across speech and music

The ability to recognize emotion in speech is a critical skill for social communication. Motivated by previous work that has shown that vocal emotion recognition accuracy varies by musical ability, the current study addressed this relationship using a behavioral measure of musical ability (i.e., singing) that relies on the same effector system used for vocal prosody production. In the current study, participants completed a musical production task that involved singing four-note novel melodies. To measure pitch perception, we used a simple pitch discrimination task in which participants indicated whether a target pitch was higher or lower than a comparison pitch. We also used self-report measures to address language and musical background. We report that singing ability, but not self-reported musical experience nor pitch discrimination ability, was a unique predictor of vocal emotion recognition accuracy. These results support a relationship between processes involved in vocal production and vocal perception, and suggest that sensorimotor processing of the vocal system is recruited for processing vocal prosody.

An RCT study showing few weeks of music lessons enhance audio-visual temporal processing

Music involves different senses and is emotional in nature, and musicians show enhanced detection of audio-visual temporal discrepancies and emotion recognition compared to non-musicians. However, whether musical training produces these enhanced abilities or if they are innate within musicians remains unclear. Thirty-one adult participants were randomly assigned to a music training, music listening, or control group who all completed a one-hour session per week for 11 weeks. The music training group received piano training, the music listening group listened to the same music, and the control group did their homework. Measures of audio-visual temporal discrepancy, facial expression recognition, autistic traits, depression, anxiety, stress and mood were completed and compared from the beginning to end of training. ANOVA results revealed that only the music training group showed a significant improvement in detection of audio-visual temporal discrepancies compared to the other groups for both stimuli (flash-beep and face-voice). However, music training did not improve emotion recognition from facial expressions compared to the control group, while it did reduce the levels of depression, stress and anxiety compared to baseline. This RCT study provides the first evidence of a causal effect of music training on improved audio-visual perception that goes beyond the music domain.

Enhanced salience of musical sounds in singers and instrumentalists

Music training has been linked to facilitated processing of emotional sounds. However, most studies have focused on speech, and less is known about musicians' brain responses to other emotional sounds and in relation to instrument-specific experience. The current study combined behavioral and EEG methods to address two novel questions related to the perception of auditory emotional cues: whether and how long-term music training relates to a distinct emotional processing of nonverbal vocalizations and music; and whether distinct training profiles (vocal vs. instrumental) modulate brain responses to emotional sounds from early to late processing stages. Fifty-eight participants completed an EEG implicit emotional processing task, in which musical and vocal sounds differing in valence were presented as nontarget stimuli. After this task, participants explicitly evaluated the same sounds regarding the emotion being expressed, their valence, and arousal. Compared with nonmusicians, musicians displayed enhanced salience detection (P2), attention orienting (P3), and elaborative processing (Late Positive Potential) of musical (vs. vocal) sounds in event-related potential (ERP) data. The explicit evaluation of musical sounds also was distinct in musicians: accuracy in the emotional recognition of musical sounds was similar across valence types in musicians, who also judged musical sounds to be more pleasant and more arousing than nonmusicians. Specific profiles of music training (singers vs. instrumentalists) did not relate to differences in the processing of vocal vs. musical sounds. Together, these findings reveal that music has a privileged status in the auditory system of long-term musically trained listeners, irrespective of their instrument-specific experience.

Symmetric in the striate but asymmetric in the extrastriate cortex when processing three-quarter faces: Neural underpinnings for aesthetic appreciations

Faces and their aesthetic appreciation are a core element of social interaction. Although studies have been made on facial processing when looking at faces with different perspectives, a direct comparison of faces in the left to the right perspective is missing. Portraits in classical Western art indicate a preference of the left compared to the right perspective, but the neural underpinnings of such an asymmetry still have to be clarified. Using functional magnetic resonance imaging, the current study focuses on the processing of three-quarter faces seen with different perspectives. Seventeen participants were asked to passively look at photographs of six male and six female faces with a neutral expression; the photographs were taken from the left, right, and frontal perspectives while keeping their focus on the eyes. The results showed that specific brain areas were involved in processing the three-quarter faces in either symmetric or asymmetric ways. Viewing left and right three-quarter faces resulted in two mirror-like activations in the striate cortex corresponding to the symmetric layout of the left and right perspectives. Viewing the left face resulted additionally in an enhanced activation also in the left extrastriate cortex. The right perspective of male faces elicited a lower activation compared to other perspectives in face-selective areas of the brain. Our findings suggest that the preference of the left three-quarter face emerges already in the early visual pathway presumably prior to facial identification, emotional processing, and aesthetic appreciation. Our observations may have general importance in disentangling different neural components and processing stages in the spatiotemporal characteristics of artistic expressions.

The three second time window in poems and language processing in general: Complementarity of discrete timing and temporal continuity

The existence of discrete time windows has triggered the search for permanence and continuity for artists (including poets) in multiple cultures throughout history. In this article, we argue that there exists a 3-s window in the temporal structure of poems as well as in the aesthetic appreciation of poetry by reviewing previous literature on the temporal aspects of poems. This 3-s window can also be considered to be a general temporal machinery underlying human behavior, including language production and perception in general. The reafference principle has provided us a unique frame for understanding cognitive processes. However, "time" was absent in the original two-stage reafference principle. Therefore, we propose a three-stage cycling model of language perception, taking into account time and time windows. We also inspect the possible neural implementations of the three stages: the generation, maintenance, and comparison of predictions (as well as the integration of predictions into the representational context). These three stages are embedded in a temporal window of ~3 s and are repeated in a cycling mode, resulting in the representational context being continuously updated. Thus, it is possible that "semantics" could be carried forward across different time windows, being a "glue" linking the discrete time windows and thus achieving the subjective feeling of temporal continuity. Candidates of such "semantic glue" could include semantic and syntactic structures as well as identity and emotion.

Music Education at School: Too Little and Too Late? Evidence From a Longitudinal Study on Music Training in Preadolescents

It is widely believed that intensive music training can boost cognitive and visuo-motor skills. However, this evidence is primarily based on retrospective studies; this makes it difficult to determine whether a cognitive advantage is caused by the intensive music training, or it is instead a factor influencing the choice of starting a music curriculum. To address these issues in a highly ecological setting, we tested longitudinally 128 students of a Middle School in Milan, at the beginning of the first class and, 1 year later, at the beginning of the second class. 72 students belonged to a Music curriculum (30 with previous music experience and 42 without) and 56 belonged to a Standard curriculum (44 with prior music experience and 12 without). Using a Principal Component Analysis, all the cognitive measures were grouped in four high-order factors, reflecting (a) General Cognitive Abilities, (b) Speed of Linguistic Elaboration, (c) Accuracy in Reading and Memory tests, and (d) Visuospatial and numerical skills. The longitudinal comparison of the four groups of students revealed that students from the Music curriculum had better performance in tests tackling General Cognitive Abilities, Visuospatial skills, and Accuracy in Reading and Memory tests. However, there were no significant curriculum-by-time interactions. Finally, the decision to have a musical experience before entering middle school was more likely to occur when the cultural background of the families was a high one. We conclude that a combination of family-related variables, early music experience, and pre-existent cognitive make-up is a likely explanation for the decision to enter a music curriculum at middle school.

Hemodynamic responses to emotional speech in two-month-old infants imaged using diffuse optical tomography

Emotional speech is one of the principal forms of social communication in humans. In this study, we investigated neural processing of emotional speech (happy, angry, sad and neutral) in the left hemisphere of 21 two-month-old infants using diffuse optical tomography. Reconstructed total hemoglobin (HbT) images were analysed using adaptive voxel-based clustering and region-of-interest (ROI) analysis. We found a distributed happy > neutral response within the temporo-parietal cortex, peaking in the anterior temporal cortex; a negative HbT response to emotional speech (the average of the emotional speech conditions < baseline) in the temporo-parietal cortex, neutral > angry in the anterior superior temporal sulcus (STS), happy > angry in the superior temporal gyrus and posterior superior temporal sulcus, angry < baseline in the insula, superior temporal sulcus and superior temporal gyrus and happy < baseline in the anterior insula. These results suggest that left STS is more sensitive to happy speech as compared to angry speech, indicating that it might play an important role in processing positive emotions in two-month-old infants. Furthermore, happy speech (relative to neutral) seems to elicit more activation in the temporo-parietal cortex, thereby suggesting enhanced sensitivity of temporo-parietal cortex to positive emotional stimuli at this stage of infant development.

East of West, West of East: a matter of global and local identity

Research is a very personal matter. On the basis of experiences in different countries with researchers from different cultures over many years, some observations will be described. The conceptual frame of this attempt is to look for anthropological universals and cultural specifics. Much can be learned from spatial representations in the arts. Whereas in the West since Renaissance time the central perspective has become dominant in visual art, in Eastern landscape paintings the "floating view" is typical. The claim that the central perspective corresponds to geometric laws and matches how we see the world is misleading for at least two reasons: It violates mechanisms of size constancy, and the visual world is spatially reduced in pictures to the perifoveal region only. Research on spatial attention has disclosed two different attentional systems being responsible either for near-fovea vision or for the far periphery. This fundamental principle as a global characteristic of visual processing is neglected in Western art. In Eastern art with a floating view geometric laws are violated, and different potential perspectives are integrated within a holistic pattern. The semantics of what shall be expressed becomes important irrespective of physical parameters. The latter may also create the unique phenomenon of becoming subjectively part of the picture confirming personal identity. Cultural specifics like in the arts (what one might expect) can surprisingly also be observed in theoretical considerations about visual processing. Whereas in the tradition of Western science visual percepts are built up with local elements like feature detectors, in an important Chinese theory global topological features are analyzed first. An important task of the brain is to create the identity of a percept on the basis of spatially and temporally distributed neural activities. It is, thus, an important theoretical question how to deal with the challenge to create and maintain the identity of a percept for some time. It is suggested that one should leave behind a monocausal reasoning for such explanations but adopt for analytical strategies the concept of complementarity as a generative principle.

Sadness and happiness are amplified in solitary listening to music

Previous studies have shown that music is a powerful means to convey affective states, but it remains unclear whether and how social context shape the intensity and quality of emotions perceived in music. Using a within-subject design, we studied this question in two experimental settings, i.e. when subjects were alone versus in company of others without direct social interaction or feedback. Non-vocal musical excerpts of the emotional qualities happiness or sadness were rated on arousal and valence dimensions. We found evidence for an amplification of perceived emotion in the solitary listening condition, i.e. happy music was rated as happier and more arousing when nobody else was around and, in an analogous manner, sad music was perceived as sadder. This difference might be explained by a shift of attention in the presence of others. The observed interaction of perceived emotion and social context did not differ for stimuli of different cultural origin.

Complementarity As Generative Principle: A Thought Pattern for Aesthetic Appreciations and Cognitive Appraisals in General

In experimental aesthetics the relationship between the arts and cognitive neuroscience has gained particular interest in recent years. But has cognitive neuroscience indeed something to offer when studying the arts? Here we present a theoretical frame within which the concept of complementarity as a generative or creative principle is proposed; neurocognitive processes are characterized by the duality of complementary activities like bottom-up and top-down control, or logistical functions like temporal control and content functions like perceptions in the neural machinery. On that basis a thought pattern is suggested for aesthetic appreciations and cognitive appraisals in general. This thought pattern is deeply rooted in the history of philosophy and art theory since antiquity; and complementarity also characterizes neural operations as basis for cognitive processes. We then discuss some challenges one is confronted with in experimental aesthetics; in our opinion, one serious problem is the lack of a taxonomy of functions in psychology and neuroscience which is generally accepted. This deficit makes it next to impossible to develop acceptable models which are similar to what has to be modeled. Another problem is the severe language bias in this field of research as knowledge gained in many languages over the ages remains inaccessible to most scientists. Thus, an inspection of research results or theoretical concepts is necessarily too narrow. In spite of these limitations we provide a selective summary of some results and viewpoints with a focus on visual art and its appreciation. It is described how questions of art and aesthetic appreciations using behavioral methods and in particular brain-imaging techniques are analyzed and evaluated focusing on such issues like the representation of artwork or affective experiences. Finally, we emphasize complementarity as a generative principle on a practical level when artists and scientists work directly together which can lead to new insights and broader perspectives on both sides.

Aesthetic Preferences for Eastern and Western Traditional Visual Art: Identity Matters

Western and Chinese artists have different traditions in representing the world in their paintings. While Western artists start since the Renaissance to represent the world with a central perspective and focus on salient objects in a scene, Chinese artists concentrate on context information in their paintings, mainly before the mid-19th century. We investigated whether the different typical representations influence the aesthetic preference for traditional Chinese and Western paintings in the different cultural groups. Traditional Chinese and Western paintings were presented randomly for an aesthetic evaluation to Chinese and Western participants. Both Chinese and Western paintings included two categories: landscapes and people in different scenes. Results showed a significant interaction between the source of the painting and the cultural group. For Chinese and Western paintings, a reversed pattern of aesthetic preference was observed: while Chinese participants gave higher aesthetic scores to traditional Chinese paintings than to Western paintings, Western participants tended to give higher aesthetic scores to traditional Western paintings than to Chinese paintings. We interpret this observation as indicator that personal identity is supported and enriched within cultural belongingness. Another important finding was that landscapes were more preferable than people in a scene across different cultural groups indicating a universal principle of preferences for landscapes. Thus, our results suggest that, on the one hand, the way that artists represent the world in their paintings influences the way that culturally embedded viewers perceive and appreciate paintings, but on the other hand, independent of the cultural background, anthropological universals are disclosed by the preference of landscapes.

Impaired socio-emotional processing in a developmental music disorder

Some individuals show a congenital deficit for music processing despite normal peripheral auditory processing, cognitive functioning, and music exposure. This condition, termed congenital amusia, is typically approached regarding its profile of musical and pitch difficulties. Here, we examine whether amusia also affects socio-emotional processing, probing auditory and visual domains. Thirteen adults with amusia and 11 controls completed two experiments. In Experiment 1, participants judged emotions in emotional speech prosody, nonverbal vocalizations (e.g., crying), and (silent) facial expressions. Target emotions were: amusement, anger, disgust, fear, pleasure, relief, and sadness. Compared to controls, amusics were impaired for all stimulus types, and the magnitude of their impairment was similar for auditory and visual emotions. In Experiment 2, participants listened to spontaneous and posed laughs, and either inferred the authenticity of the speaker's state, or judged how much laughs were contagious. Amusics showed decreased sensitivity to laughter authenticity, but normal contagion responses. Across the experiments, mixed-effects models revealed that the acoustic features of vocal signals predicted socio-emotional evaluations in both groups, but the profile of predictive acoustic features was different in amusia. These findings suggest that a developmental music disorder can affect socio-emotional cognition in subtle ways, an impairment not restricted to auditory information.