Consonance and dissonance perception. A critical review of the historical sources, multidisciplinary findings, and main hypotheses

Acoustic estimation of voice roughness

Roughness is a perceptual characteristic of sound that was first applied to musical consonance and dissonance, but it is increasingly recognized as a central aspect of voice quality in human and animal communication. It may be particularly important for asserting social dominance or attracting attention in urgent signals such as screams. To ensure that the results of roughness research are valid and consistent across studies, we need standard methodology for measuring it. I review the literature on roughness estimation, from classic psychoacoustics to more recent approaches, and present two collections of 602 human vocal samples whose roughness was rated by 162 listeners in perceptual experiments. Two algorithms for estimating roughness acoustically from modulation spectra are then presented and optimized to match the human ratings. One uses a bank of gammatone or Butterworth filters to obtain an auditory spectrogram, and a faster algorithm begins with a conventional spectrogram obtained with Short-Time Fourier transform; both explain ~ 50% of variance in average human ratings per stimulus. The range of modulation frequencies most relevant to roughness perception is [50, 200] Hz; this range can be selected with simple cutoff points or with a lognormal weighting function. Modulation and roughness spectrograms are proposed as visual aids for studying the dynamics of roughness in longer recordings. The described algorithms are implemented in the function modulationSpectrum() from the open-source R library soundgen. The audio recordings and their ratings are freely available from https://osf.io/gvcpx/ and can be used for benchmarking other algorithms.

Perceiving temporal structure within and between the senses: A multisensory/crossmodal perspective

The literature demonstrates that people perceive temporal structure in sequences of auditory, tactile, or visual stimuli. However, to date, much less attention has been devoted to studying the perception of temporal structure that results from the presentation of stimuli to the chemical senses and/or crossmodally. In this review, we examine the literature on the perception of temporal features in the unisensory, multisensory and crossmodal domains in an attempt to answer, among others, the following foundational questions: Is the ability to perceive the temporal structure of stimuli demonstrated beyond the spatial senses (i.e., in the chemical senses)? Is the intriguing idea of an amodal, or supramodal, temporal processor in the human brain empirically grounded? Is the perception of temporal structure in crossmodal patterns (even) possible? Does the ability to perceive temporal patterns convey any biological advantage to humans? Overall, the reviewed literature suggests that humans perceive rhythmic structures, such as beat and metre, across audition, vision and touch, exhibiting similar behavioural traits. In contrast, only a limited number of studies have demonstrated this ability in crossmodal contexts (e.g., audiotactile interactions). Similar evidence within the chemical senses remains scarce and unconvincing, posing challenges to the concept of an amodal temporal processor and raising questions about its potential biological advantages. These limitations highlight the need for further investigation. To address these gaps, we propose several directions for future research, which may provide valuable insights into the nature and mechanisms of temporal processing across sensory modalities.

Moderating effects of chord progressions on the emotional experience of major and minor chords

Numerous studies have shown the major chords express positive emotions, while minor chords convey negative emotions. However, several research suggest that the association between major/minor chords and emotional valence may vary due to certain musical contextual factors. This study investigates whether the emotional experience associated with major and minor chords is influenced by chord progressions. Employing subjective evaluations, participants were asked to evaluate the emotional valence, stability, and tension of individual chords (Experiment 1) and the final chords in chord sequences (Experiment 2). Results indicate that the pleasantness ratings of major chords are significantly higher than those of minor chords, while tension ratings are significantly lower. However, stability ratings between major and minor chords did not significantly differ. Notably, when major chords served as the final chords in stable termination progressions, their ratings in pleasantness and stability will be significantly higher compared to those of minor chords, and at the same time, they will also elicit lower tension ratings. While unstable terminations show no significant differences in ratings between major and minor chords across the dimensions of pleasantness, stability, and tension. Thus, this study reveals that chord progressions, as a musical context, influence the emotional valence of major and minor chords through variations in their stability and tension.

The evolution of human music in light of increased prosocial behavior: a new model

Together with language, music is perhaps our most distinctive behavioral trait. Following the lead of evolutionary linguistic research, different hypotheses have been proposed to explain why only humans perform music and how this ability might have evolved in the species. In this paper, we advance a new model of music evolution that builds on the theory of self-domestication, according to which the human phenotype is, at least in part, the outcome of a process similar to mammal domestication, triggered by a progressive reduction in reactive aggression levels in response to environmental changes. In the paper, we specifically argue that changes in aggression management through the course of human cultural evolution can account for the behaviors conducive to the emergence and evolution of music. We hypothesize 4 stages in the evolutionary development of music under the influence of environmental changes and evolution of social organization: starting from musilanguage, 1) proto-music gave rise to 2) personal and private forms of timbre-oriented music, then to 3) small-group ensembles of pitch-oriented music, at first of indefinite and then definite pitch, and finally to 4) collective (tonal) music. These stages parallel what has been hypothesized for languages and encompass the diversity of music types and genres described worldwide. Overall, music complexity emerges in a gradual fashion under the effects of enhanced abilities for cultural niche construction, resulting from the stable trend of reduction in reactive aggression towards the end of the Pleistocene, leading to the rise of hospitality codes, and succeeded by the increase in proactive aggression from the beginning of the Holocene onward. This paper addresses numerous controversies in the literature on the evolution of music by providing a clear structural definition of music, identifying its structural features that distinguish it from oral language, and summarizing the typology of operational functions of music and formats of its transmission. The proposed framework of structural approach to music arms a researcher with means to identify and comparatively analyze different schemes of tonal organization of music, placing them in the context of human social and cultural evolution. Especially valuable is the theory of so-called "personal song", described and analyzed here from ethological, social, cultural, cognitive, and musicological perspectives. Personal song seems to constitute a remnant of the proto-musical transition from animal communication to human music as we know it today. We interlink the emergence of personal song with the evolution of kinship, placing both of them on the timeline of cultural evolution - based on totality of ethnographic, archaeological, anthropological, genetic, and paleoclimatic data.

Pleasantness makes a good time: musical consonance shapes interpersonal synchronization in dyadic joint action

Introduction

Music making is a process by which humans across cultures come together to create patterns of sounds that are aesthetically pleasing. What remains unclear is how this aesthetic outcome affects the sensorimotor interaction between participants.

Method

Here we approach this question using an interpersonal sensorimotor synchronization paradigm to test whether the quality of a jointly created chord (consonant vs. dissonant) affects movement coordination. We recruited non-musician participants in dyads to perform a dyadic synchronization-continuation task (dSCT): on each trial, participants first synchronized their movements to a metronome (synchronization phase) and then continued tapping together at the same tempo without the metronome (continuation phase). Each tap yielded a note and participants heard both their own and that of their partner, thus creating a chord that was varied to be either consonant (Perf5 or Maj6) or dissonant (Min2 or Maj2). For each trial, participants also rated the pleasure they felt in creating the sounds together. Additionally, they completed questionnaires about social closeness to the other participant, musical reward sensitivity and musical training.

Results

Results showed that participants' taps were closer in time when they jointly created consonant (high pleasure) vs. dissonant (low pleasure) chords, and that pleasure experienced by the dyad in each trial predicted interpersonal synchronization. However, consonance did not affect individual synchronization with the metronome or individual tapping when the metronome was discontinued. The effect of consonance on synchronization was greater in dyads who reported feeling less close prior to the task.

Discussion

Together, these results highlight the role of consonance in shaping the temporal coordination of our actions with others. More broadly, this work shows that the aesthetic outcome of what we create together affects joint behaviors.

Loop and Enjoy: A Scoping Review of the Research on the Effects of Processing Fluency on Aesthetic Reactions to Auditory Stimuli

Processing fluency has been shown to affect how people aesthetically evaluate stimuli. While this effect is well documented for visual stimuli, the evidence accumulated for auditory stimuli has not yet been integrated. Our aim was to examine the relevant research on how processing fluency affects the aesthetic appreciation of auditory stimuli and to identify the extant knowledge gaps in this body of evidence. This scoping review of 19 studies reported across 13 articles found that, similarly to visual stimuli, fluency has a positive effect on liking of auditory stimuli. Additionally, we identified certain elements that impede the generalizability of the current research on the relationship between fluency and aesthetic reactions to auditory stimuli, such as a lack of consistency in the number of repeated exposures, the tendency to omit the affective component and the failure to account for personal variables such as musical abilities developed through musical training or the participants' personality or preferences. These results offer a starting point in developing novel and proper processing fluency manipulations of auditory stimuli and suggest several avenues for future research aiming to clarify the impact and importance of processing fluency and disfluency in this domain.

Should absolute pitch be considered as a unique kind of absolute sensory judgment in humans? A systematic and theoretical review of the literature

Absolute pitch is the name given to the rare ability to identify a musical note in an automatic and effortless manner without the need for a reference tone. Those individuals with absolute pitch can, for example, name the note they hear, identify all of the tones of a given chord, and/or name the pitches of everyday sounds, such as car horns or sirens. Hence, absolute pitch can be seen as providing a rare example of absolute sensory judgment in audition. Surprisingly, however, the intriguing question of whether such an ability presents unique features in the domain of sensory perception, or whether instead similar perceptual skills also exist in other sensory domains, has not been explicitly addressed previously. In this paper, this question is addressed by systematically reviewing research on absolute pitch using the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) method. Thereafter, we compare absolute pitch with two rare types of sensory experience, namely synaesthesia and eidetic memory, to understand if and how these phenomena exhibit similar features to absolute pitch. Furthermore, a common absolute perceptual ability that has been often compared to absolute pitch, namely colour perception, is also discussed. Arguments are provided supporting the notion that none of the examined abilities can be considered like absolute pitch. Therefore, we conclude by suggesting that absolute pitch does indeed appear to constitute a unique kind of absolute sensory judgment in humans, and we discuss some open issues and novel directions for future research in absolute pitch.

Consonance in the carillon

Previous psychological studies have shown that musical consonance is not only determined by the frequency ratios between tones, but also by the frequency spectra of those tones. However, these prior studies used artificial tones, specifically tones built from a small number of pure tones, which do not match the acoustic complexity of real musical instruments. The present experiment therefore investigates tones recorded from a real musical instrument, the Westerkerk Carillon, conducting a "dense rating" experiment where participants (N = 113) rated musical intervals drawn from the continuous range 0-15 semitones. Results show that the traditional consonances of the major third and the minor sixth become dissonances in the carillon and that small intervals (in particular 0.5-2.5 semitones) also become particularly dissonant. Computational modelling shows that these effects are primarily caused by interference between partials (e.g., beating), but that preference for harmonicity is also necessary to produce an accurate overall account of participants' preferences. The results support musicians' writings about the carillon and contribute to ongoing debates about the psychological mechanisms underpinning consonance perception, in particular disputing the recent claim that interference is largely irrelevant to consonance perception.

Dissonant music engages early visual processing

The neuroscientific examination of music processing in audio-visual contexts offers a valuable framework to assess how auditory information influences the emotional encoding of visual information. Using fMRI during naturalistic film viewing, we investigated the neural mechanisms underlying the effect of music on valence inferences during mental state attribution. Thirty-eight participants watched the same short-film accompanied by systematically controlled consonant or dissonant music. Subjects were instructed to think about the main character's intentions. The results revealed that increasing levels of dissonance led to more negatively valenced inferences, displaying the profound emotional impact of musical dissonance. Crucially, at the neuroscientific level and despite music being the sole manipulation, dissonance evoked the response of the primary visual cortex (V1). Functional/effective connectivity analysis showed a stronger coupling between the auditory ventral stream (AVS) and V1 in response to tonal dissonance and demonstrated the modulation of early visual processing via top-down feedback inputs from the AVS to V1. These V1 signal changes indicate the influence of high-level contextual representations associated with tonal dissonance on early visual cortices, serving to facilitate the emotional interpretation of visual information. Our results highlight the significance of employing systematically controlled music, which can isolate emotional valence from the arousal dimension, to elucidate the brain's sound-to-meaning interface and its distributive crossmodal effects on early visual encoding during naturalistic film viewing.

Development of tonality and consonance categorization ability and preferences in 4- to 6-year-old children

Consonance perception has been extensively studied in Western adults, but it is less clear how this perception develops in children during musical enculturation. We investigated how this development occurs in 4- to 6-year-old children by examining two complex musical skills (i.e., consonance and tonality preferences). Accordingly, we developed a child-focused approach to understand the underlying developmental processes of tonality and consonance preferences in 4- to 6-year-old children using a video interview format. As previous studies have confounded preference with perception, we examined each concept separately and measured perceptual abilities as categorization. For tonality, the ability to categorize tonal and atonal melodies developed by the age of 6 years. It is noteworthy that only children who could categorize successfully showed a preference for tonality at the age of 6. For consonance, we observed an early preference for consonance at 4 years of age, but this preference was only measurable with large differences between consonant and dissonant stimuli. We propose that tonality and consonance preferences develop during childhood with increasing categorization ability when the surrounding musical culture is marked by Western tonality and consonance.

Neuromagnetic representation of musical roundness in chord progressions

Introduction

Musical roundness perception relies on consonance/dissonance within a rule-based harmonic context, but also on individual characteristics of the listener. The present work tackles these aspects in a combined psychoacoustic and neurophysiological study, taking into account participant's musical aptitude.

Methods

Our paradigm employed cadence-like four-chord progressions, based on Western music theory. Chord progressions comprised naturalistic and artificial sounds; moreover, their single chords varied regarding consonance/dissonance and harmonic function. Thirty participants listened to the chord progressions while their cortical activity was measured with magnetoencephalography; afterwards, they rated the individual chord progressions with respect to their perceived roundness.

Results

Roundness ratings differed according to the degree of dissonance in the dominant chord at the progression's third position; this effect was pronounced in listeners with high musical aptitude. Interestingly, a corresponding pattern occurred in the neuromagnetic N1m response to the fourth chord (i.e., at the progression's resolution), again with somewhat stronger differentiation among musical listeners. The N1m magnitude seemed to increase during chord progressions that were considered particularly round, with the maximum difference after the final chord; here, however, the musical aptitude effect just missed significance.

Discussion

The roundness of chord progressions is reflected in participant's psychoacoustic ratings and in their transient cortical activity, with stronger differentiation among listeners with high musical aptitude. The concept of roundness might help to reframe consonance/dissonance to a more holistic, gestalt-like understanding that covers chord relations in Western music.

Sensory translation between audition and vision

Across the millennia, and across a range of disciplines, there has been a widespread desire to connect, or translate between, the senses in a manner that is meaningful, rather than arbitrary. Early examples were often inspired by the vivid, yet mostly idiosyncratic, crossmodal matches expressed by synaesthetes, often exploited for aesthetic purposes by writers, artists, and composers. A separate approach comes from those academic commentators who have attempted to translate between structurally similar dimensions of perceptual experience (such as pitch and colour). However, neither approach has succeeded in delivering consensually agreed crossmodal matches. As such, an alternative approach to sensory translation is needed. In this narrative historical review, focusing on the translation between audition and vision, we attempt to shed light on the topic by addressing the following three questions: (1) How is the topic of sensory translation related to synaesthesia, multisensory integration, and crossmodal associations? (2) Are there common processing mechanisms across the senses that can help to guarantee the success of sensory translation, or, rather, is mapping among the senses mediated by allegedly universal (e.g., amodal) stimulus dimensions? (3) Is the term 'translation' in the context of cross-sensory mappings used metaphorically or literally? Given the general mechanisms and concepts discussed throughout the review, the answers we come to regarding the nature of audio-visual translation are likely to apply to the translation between other perhaps less-frequently studied modality pairings as well.

Enjoying art: an evolutionary perspective on the esthetic experience from emotion elicitors

The ubiquity of human art prompted evolutionary psychologists to explore its origins as a potential adaptation to the environment. Here we focus on emotionally charged art and posit that affective affordances embedded into some artworks play a pivotal role in explaining why these artworks are enjoyed from an evolutionary perspective. Such features, recurring in various art forms, are interpreted as cues to the emotional state of others, enabling art consumers to engage in empathetic experiences and vicarious emotions. We explore the adaptive value of deriving pleasure from vicarious emotions, while also addressing the seemingly counterintuitive enjoyment of artworks that evoke negative emotions. We discuss the appreciation of vicarious emotions irrespective of their valence and maintain this appreciation to hold adaptive significance for three key reasons. Firstly, it aids art consumers in refining their interpretational schemes of internal states, potentially enhancing emotional regulation skills. Secondly, it contributes to a deeper understanding of the emotions of others, thereby fostering emotional intelligence and empathy. Lastly, the enjoyment of affectively charged artworks reinforces social cohesion by harmonizing the emotions of group members. This perspective provides a comprehensive framework for understanding the evolutionary underpinnings of the human capacity for art appreciation and emotional engagement.

Musical anhedonia, timbre, and the rewards of music listening

Pleasure in music has been linked to predictive coding of melodic and rhythmic patterns, subserved by connectivity between regions in the brain's auditory and reward networks. Specific musical anhedonics derive little pleasure from music and have altered auditory-reward connectivity, but no difficulties with music perception abilities and no generalized physical anhedonia. Recent research suggests that specific musical anhedonics experience pleasure in nonmusical sounds, suggesting that the implicated brain pathways may be specific to music reward. However, this work used sounds with clear real-world sources (e.g., babies laughing, crowds cheering), so positive hedonic responses could be based on the referents of these sounds rather than the sounds themselves. We presented specific musical anhedonics and matched controls with isolated short pleasing and displeasing synthesized sounds of varying timbres with no clear real-world referents. While the two groups found displeasing sounds equally displeasing, the musical anhedonics gave substantially lower pleasure ratings to the pleasing sounds, indicating that their sonic anhedonia is not limited to musical rhythms and melodies. Furthermore, across a large sample of participants, mean pleasure ratings for pleasing synthesized sounds predicted significant and similar variance in six dimensions of musical reward considered to be relatively independent, suggesting that pleasure in sonic timbres play a role in eliciting reward-related responses to music. We replicate the earlier findings of preserved pleasure ratings for semantically referential sounds in musical anhedonics and find that pleasure ratings of semantic referents, when presented without sounds, correlated with ratings for the sounds themselves. This association was stronger in musical anhedonics than in controls, suggesting the use of semantic knowledge as a compensatory mechanism for affective sound processing. Our results indicate that specific musical anhedonia is not entirely specific to melodic and rhythmic processing, and suggest that timbre merits further research as a source of pleasure in music.

The impact of crossmodal predictions on the neural processing of aesthetic stimuli

Neuroaesthetic research has focused on neural predictive processes involved in the encounter with art stimuli or the related evaluative judgements, and it has been mainly conducted unimodally. Here, with electroencephalography, magnetoencephalography and an affective priming protocol, we investigated whether and how the neural responses to non-representational aesthetic stimuli are top-down modulated by affective representational (i.e. semantically meaningful) predictions between audition and vision. Also, the neural chronometry of affect processing of these aesthetic stimuli was investigated. We hypothesized that the early affective components of crossmodal aesthetic responses are dependent on the affective and representational predictions formed in another sensory modality resulting in differentiated brain responses, and that audition and vision indicate different processing latencies for affect. The target stimuli were aesthetic visual patterns and musical chords, and they were preceded by a prime from the opposing sensory modality. We found that early auditory-cortex responses to chords were more affected by valence than the corresponding visual-cortex ones. Furthermore, the assessments of visual targets were more facilitated by affective congruency of crossmodal primes than the acoustic targets. These results indicate, first, that the brain uses early affective information for predictively guiding aesthetic responses; second, that an affective transfer of information takes place crossmodally, mainly from audition to vision, impacting the aesthetic assessment. This article is part of the theme issue 'Art, aesthetics and predictive processing: theoretical and empirical perspectives'.

Making sense of music: Insights from neurophysiology and connectivity analyses in naturalistic listening conditions

According to the latest frameworks, auditory perception and memory involve the constant prediction of future sound events by the brain, based on the continuous extraction of feature regularities from the environment. The neural hierarchical mechanisms for predictive processes in perception and memory for sounds are typically studied in relation to simple acoustic features in isolated sounds or sound patterns inserted in highly certain contexts. Such studies have identified reliable prediction formation and error signals, e.g., the N100 or the mismatch negativity (MMN) evoked responses. In real life, though, individuals often face situations in which uncertainty prevails and where making sense of sounds becomes a hard challenge. In music, not only deviations from predictions are masterly set up by composers to induce emotions but sometimes the sheer uncertainty of sound scenes is exploited for aesthetic purposes, especially in compositional styles such as Western atonal classical music. In very recent magnetoencephalography (MEG) and electroencephalography (EEG) studies, experimental and technical advances in stimulation paradigms and analysis approaches have permitted the identification of prediction-error responses from highly uncertain, atonal contexts and the extraction of prediction-related responses from real, continuous music. Moreover, functional connectivity analyses revealed the emergence of cortico-hippocampal interactions during the formation of auditory memories for more predictable vs. less predictable patterns. These findings contribute to understanding the general brain mechanisms that enable us to predict even highly uncertain sound environments and to possibly make sense of and appreciate even atonal music.

The influence of musical parameters and subjective musical ratings on perceptions of culture

Recent research suggests that music can affect evaluations of other groups and cultures. However, little is known about the objective and subjective musical parameters that influence these evaluations. We aimed to fill this gap through two studies. Study 1 collected responses from 52 American participants who listened to 30 folk-song melodies from different parts of the world. Linear mixed-effects models tested the influence of objective and subjective musical parameters of these melodies on evaluations of the cultures from which they originated. Musical parameters consistently predicted cultural evaluations. The most prominent musical parameter was musical velocity, a measure of number of pitch onsets, predicting more cultural warmth, competence and evolvedness and less cultural threat. Next, with a sample of 212 American participants, Study 2 used a within-subjects experiment to alter the tempo and dissonance for a subset of six melody excerpts from Study 1, testing for causal effects. Linear mixed-effects models revealed that both dissonance and slow tempo predicted more negative cultural evaluations. Together, both studies demonstrate how musical parameters can influence cultural perceptions. Avenues for future research are discussed.

Evidence for a universal association of auditory roughness with musical stability

We provide evidence that the roughness of chords-a psychoacoustic property resulting from unresolved frequency components-is associated with perceived musical stability (operationalized as finishedness) in participants with differing levels and types of exposure to Western or Western-like music. Three groups of participants were tested in a remote cloud forest region of Papua New Guinea (PNG), and two groups in Sydney, Australia (musicians and non-musicians). Unlike prominent prior studies of consonance/dissonance across cultures, we framed the concept of consonance as stability rather than as pleasantness. We find a negative relationship between roughness and musical stability in every group including the PNG community with minimal experience of musical harmony. The effect of roughness is stronger for the Sydney participants, particularly musicians. We find an effect of harmonicity-a psychoacoustic property resulting from chords having a spectral structure resembling a single pitched tone (such as produced by human vowel sounds)-only in the Sydney musician group, which indicates this feature's effect is mediated via a culture-dependent mechanism. In sum, these results underline the importance of both universal and cultural mechanisms in music cognition, and they suggest powerful implications for understanding the origin of pitch structures in Western tonal music as well as on possibilities for new musical forms that align with humans' perceptual and cognitive biases. They also highlight the importance of how consonance/dissonance is operationalized and explained to participants-particularly those with minimal prior exposure to musical harmony.