The multiple synaesthete E.S. — Neuroanatomical basis of interval-taste and tone-colour synaesthesia

Should absolute pitch be considered as a unique kind of absolute sensory judgment in humans? A systematic and theoretical review of the literature

Absolute pitch is the name given to the rare ability to identify a musical note in an automatic and effortless manner without the need for a reference tone. Those individuals with absolute pitch can, for example, name the note they hear, identify all of the tones of a given chord, and/or name the pitches of everyday sounds, such as car horns or sirens. Hence, absolute pitch can be seen as providing a rare example of absolute sensory judgment in audition. Surprisingly, however, the intriguing question of whether such an ability presents unique features in the domain of sensory perception, or whether instead similar perceptual skills also exist in other sensory domains, has not been explicitly addressed previously. In this paper, this question is addressed by systematically reviewing research on absolute pitch using the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) method. Thereafter, we compare absolute pitch with two rare types of sensory experience, namely synaesthesia and eidetic memory, to understand if and how these phenomena exhibit similar features to absolute pitch. Furthermore, a common absolute perceptual ability that has been often compared to absolute pitch, namely colour perception, is also discussed. Arguments are provided supporting the notion that none of the examined abilities can be considered like absolute pitch. Therefore, we conclude by suggesting that absolute pitch does indeed appear to constitute a unique kind of absolute sensory judgment in humans, and we discuss some open issues and novel directions for future research in absolute pitch.

Subtitled speech: Phenomenology of tickertape synesthesia

With effort, most literate persons can conjure more or less vague visual mental images of the written form of words they are hearing, an ability afforded by the links between sounds, meaning, and letters. However, as first reported by Francis Galton, persons with ticker-tape synesthesia (TTS) automatically perceive in their mind's eye accurate and vivid images of the written form of all utterances which they are hearing. We propose that TTS results from an atypical setup of the brain reading system, with an increased top-down influence of phonology on orthography. As a first descriptive step towards a deeper understanding of TTS, we identified 26 persons with TTS. Participants had to answer to a questionnaire aiming to describe the phenomenology of TTS along multiple dimensions, including visual and temporal features, triggering stimuli, voluntary control, interference with language processing, etc. We also assessed the synesthetic percepts elicited experimentally by auditory stimuli such as non-speech sounds, pseudowords, and words with various types of correspondence between sounds and letters. We discuss the potential cerebral substrates of those features, argue that TTS may provide a unique window in the mechanisms of written language processing and acquisition, and propose an agenda for future research.

Coloured hearing, colour music, colour organs, and the search for perceptually meaningful correspondences between colour and sound

There has long been interest in the nature of the relationship(s) between hue and pitch or, in other words, between colour and musical/pure tones, stretching back at least as far as Newton, Goethe, Helmholtz, and beyond. In this narrative historical review, we take a closer look at the motivations that have lain behind the various assertions that have been made in the literature concerning the analogies, and possible perceptual similarities, between colour and sound. During the last century, a number of experimental psychologists have also investigated the nature of the correspondence between these two primary dimensions of perceptual experience. The multitude of different crossmodal mappings that have been put forward over the centuries are summarized, and a distinction drawn between physical/structural and psychological correspondences. The latter being further sub-divided into perceptual and affective categories. Interest in physical correspondences has typically been motivated by the structural similarities (analogous mappings) between the organization of perceptible dimensions of auditory and visual experience. Emphasis has been placed both on the similarity in terms of the number of basic categories into which pitch and colour can be arranged and also on the fact that both can be conceptualized as circular dimensions. A distinction is drawn between those commentators who have argued for a dimensional alignment of pitch and hue (based on a structural mapping), and those who appear to have been motivated by the existence of specific correspondences between particular pairs of auditory and visual stimuli instead (often, as we will see, based on the idiosyncratic correspondences that have been reported by synaesthetes). Ultimately, though, the emotional-mediation account would currently appear to provide the most parsimonious account for whatever affinity the majority of people experience between musical sounds and colour.

Commercializing Sonic Seasoning in Multisensory Offline Experiential Events and Online Tasting Experiences

The term "sonic seasoning" refers to the deliberate pairing of sound/music with taste/flavour in order to enhance, or modify, the multisensory tasting experience. Although the recognition that people experience a multitude of crossmodal correspondences between stimuli in the auditory and chemical senses originally emerged from the psychophysics laboratory, the last decade has seen an explosion of interest in the use and application of sonic seasoning research findings, in a range of multisensory experiential events and online offerings. These marketing-led activations have included a variety of different approaches, from curating pre-composed music selections that have the appropriate sonic qualities (such as pitch or timbre), to the composition of bespoke music/soundscapes that match the specific taste/flavour of particular food or beverage products. Moreover, given that our experience of flavour often changes over time and frequently contains multiple distinct elements, there is also scope to more closely match the sonic seasoning to the temporal evolution of the various components (or notes) of the flavour experience. We review a number of case studies of the use of sonic seasoning, highlighting some of the challenges and opportunities associated with the various approaches, and consider the intriguing interplay between physical and digital (online) experiences. Taken together, the various examples reviewed here help to illustrate the growing commercial relevance of sonic seasoning research.

Perceiving Music Through the Lens of Synaesthesia and Absolute Pitch

Synaesthesia and absolute pitch (AP) are two rare conditions that occur more frequently within populations of artistic professionals. Current thinking surrounding synaesthesia and AP and their relationship to music perception form the focus of this article. Given that synaesthesia has rarely been discussed in the music literature, the article surveys and consolidates general neurobiological, psychological, and behavioural evidence to summarise what is currently known on this topic, in order to link this back to the conditions that most relate to music. In contrast, research on AP is now well established in the music literature, but the important gap of linking AP to other conditions such as synaesthesia has yet to be fully explored. This article investigates the potential relationship between synaesthesia and AP for musicians who possess both conditions by systematically comparing the definitions, classifications, prevalence, diagnoses, and impacts on music perception of synaesthesia and AP and provides insights into the varying states of the literature and knowledge of both conditions. In so doing, this article aims to facilitate a greater understanding of music and auditory forms of synaesthesia and their interaction with AP and encourage increased research effort on this important topic.

An internet-based survey of synesthesia in multiple sclerosis: Incidence, characteristics and implications

Objective Prior work raises the interesting possibility that both multiple sclerosis and synesthesia share a common etiology, that being immune system dysfunction, as well as neuroanatomical and neurochemical abnormalities, including those involving white matter and serotonergic pathways, respectively. Given these links between these two syndromes, we examined the possibility that prevalence of synesthesia would be elevated in a population of individuals with MS, relative to what is thought to be the prevalence in the neurotypical population. It was not known whether synesthesia might be a marker for subsequent development of MS, or if synesthesia might reflect neurological damage resulting from MS disease progression. Method Individuals with self- reported clinically definite MS were recruited online via the internet and social media using sites specifically relevant to the MS community. Data from 147 individuals who completed several questionnaires related to synesthesia were analyzed. Results Depending on criteria, between approximately 7 and 16% of individuals with MS reported synesthesia here. This is an estimated 1.57 to 3.55 times increased incidence of synesthesia here relative to previous findings in neurotypical samples. Limitations of the study include that this was an internet survey, and that synesthesia was not directly assessed in this sample. Conclusions Results suggest a link between the syndromes, primarily indicating that synesthesia may be a marker for subsequent MS development, and the implications and directions for future study are discussed.

Auditory aversion in absolute pitch possessors

Absolute pitch (AP) refers to the ability of identifying the pitch of a given tone without reliance on any reference pitch. The downside of possessing AP may be the experience of disturbance when exposed to out-of-tune tones. Here, we investigated this so-far unexplored phenomenon in AP, which we refer to as auditory aversion. Electroencephalography (EEG) was recorded in a sample of AP possessors and matched control musicians without AP while letting them perform a task underlying a so-called affective priming paradigm: Participants judged valenced pictures preceded by musical primes as quickly and accurately as possible. The primes were bimodal, presented as tones in combination with visual notations that either matched or mismatched the actually presented tone. Both samples performed better in judging unpleasant pictures over pleasant ones. In comparison with the control musicians, the AP possessors revealed a more profound discrepancy between the two valence conditions, and their EEG revealed later peaks at around 200 ms (P200) after prime onset. Their performance dropped when responding to pleasant pictures preceded by incongruent primes, especially when mistuned by one semitone. This interference was also reflected in an EEG deflection at around 400 ms (N400) after picture onset, preceding the behavior responses. These findings suggest that AP possessors process mistuned musical stimuli and pleasant pictures as affectively unrelated with each other, supporting an aversion towards out-of-tune tones in AP possessors. The longer prime-related P200 latencies exhibited by AP possessors suggest a delay in integrating musical stimuli, underlying an altered affinity towards pitch-label associations.

Assessing Lateral Interaction in the Synesthetic Visual Brain

In grapheme-color synesthesia, letters and numbers evoke abnormal colored perceptions. Although the underlying mechanisms are not known, it is largely thought that the synesthetic brain is characterized by atypical connectivity throughout various brain regions, including the visual areas. To study the putative impact of synesthesia on the visual brain, we assessed lateral interactions (i.e., local functional connectivity between neighboring neurons in the visual cortex) by recording steady-state visual evoked potentials (ssVEPs) over the occipital region in color-grapheme synesthetes (n = 6) and controls (n = 21) using the windmill/dartboard paradigm. Discrete Fourier Transform analysis was conducted to extract the fundamental frequency and the second harmonics of ssVEP responses from contrast-reversing stimuli presented at 4.27 Hz. Lateral interactions were assessed using two amplitude-based indices: Short-range and long-range lateral interactions. Results indicated that synesthetes had a statistically weaker signal coherence of the fundamental frequency component compared to the controls, but no group differences were observed on lateral interaction indices. However, a significant correlation was found between long-range lateral interactions and the type of synesthesia experience (projector versus associator). We conclude that the occipital activity related to lateral interactions in synesthetes does not substantially differ from that observed in controls. Further investigation is needed to understand the impact of synesthesia on visual processing, specifically in relation to subjective experiences of synesthete individuals.

Early tone categorization in absolute pitch musicians is subserved by the right-sided perisylvian brain

Absolute pitch (AP) is defined as the ability to identify and label tones without reference to keyality. In this context, the main question is whether early or late processing stages are responsible for this ability. We investigated the electrophysiological responses to tones in AP and relative pitch (RP) possessors while participants listened attentively to sine tones. Since event-related potentials are particularly suited for tracking tone encoding (N100 and P200), categorization (N200), and mnemonic functions (N400), we hypothesized that differences in early pitch processing stages would be reflected by increased N100 and P200-related areas in AP musicians. Otherwise, differences in later cognitive stages of tone processing should be mirrored by increased N200 and/or N400 areas in AP musicians. AP possessors exhibited larger N100 areas and a tendency towards enhanced P200 areas. Furthermore, the sources of these components were estimated and statistically compared between the two groups for a set of a priori defined regions of interest. AP musicians demonstrated increased N100-related current densities in the right superior temporal sulcus, middle temporal gyrus, and Heschl’s gyrus. Results are interpreted as indicating that early between-group differences in right-sided perisylvian brain regions might reflect auditory tone categorization rather than labelling mechanisms.

Music-colour synaesthesia: Concept, context and qualia

This review provides a commentary on coloured-hearing arising on hearing music: music-colour synaesthesia. Although traditionally explained by the hyperconnectivity theory (Ramachandran & Hubbard, 2001a) and the disinhibited feedback theory (Grossenbacher & Lovelace, 2001) as a purely perceptual phenomenon, the review of eight coloured-hearing neuroimaging studies shows that it may not be assumed that these explanations are directly translatable to music-colour synaesthesia. The concept of 'ideaesthesia' (Nikolić, 2009) and the role of conceptual and semantic inducers challenge the likelihood of a single mechanism underlying the cause of synaesthesia and argue for a move away from a purely sensory to sensory explanation. Finally, music-colour synaesthesia forms a challenge for established philosophical theories and the position of synaesthesia is considered within the larger context of musical qualia.

An extended research of crossmodal correspondence between color and sound in psychology and cognitive ergonomics

Based on the existing research on sound symbolism and crossmodal correspondence, this study proposed an extended research on cross-modal correspondence between various sound attributes and color properties in a group of non-synesthetes. In Experiment 1, we assessed the associations between each property of sounds and colors. Twenty sounds with five auditory properties (pitch, roughness, sharpness, tempo and discontinuity), each varied in four levels, were used as the sound stimuli. Forty-nine colors with different hues, saturation and brightness were used to match to those sounds. Result revealed that besides pitch and tempo, roughness and sharpness also played roles in sound-color correspondence. Reaction times of sound-hue were a little longer than the reaction times of sound-lightness. In Experiment 2, a speeded target discrimination task was used to assess whether the associations between sound attributes and color properties could invoke natural cross-modal correspondence and improve participants' cognitive efficiency in cognitive tasks. Several typical sound-color pairings were selected according to the results of Experiment 1. Participants were divided into two groups (congruent and incongruent). In each trial participants had to judge whether the presented color could appropriately be associated with the sound stimuli. Result revealed that participants responded more quickly and accurately in the congruent group than in the incongruent group. It was also found that there was no significant difference in reaction times and error rates between sound-hue and sound-lightness. The results of Experiment 1 and 2 indicate the existence of a robust crossmodal correspondence between multiple attributes of sound and color, which also has strong influence on cognitive tasks. The inconsistency of the reaction times between sound-hue and sound-lightness in Experiment 1 and 2 is probably owing to the difference in experimental protocol, which indicates that the complexity of experiment design may be an important factor in crossmodal correspondence phenomena.

Neuroanatomical basis of number synaesthesias: A voxel-based morphometry study

In synaesthesia, a specific sensory dimension leads to an involuntary sensation in another sensory dimension not commonly associated with it; for example, synaesthetes may experience a specific colour when listening or thinking of numbers or letters. Large-scale behavioural studies provide a rich description of different synaesthesia phenotypes, and a great amount of research has been oriented to uncovering whether a single or multiple brain mechanisms underlie these various synaesthesia phenotypes. Interestingly, most of the synaesthetic inducers are conceptual stimuli such as numbers, letters, and months. However, the impact of these concepts on the synaesthetic brain remains largely unexplored. Numbers appear as the most typical inducer in two common types of synaesthesia: grapheme-colour and sequence-space. Numbers are symbols that denote quantity information and their processing recruits a specific neural network. Therefore, numbers may play an important role in the brain mechanisms underlying some types of synaesthesia. We used voxel-based morphometry (VBM) to compare grey matter (GM) volume in synaesthetes and controls. Relative to controls, synaesthetes showed increase in GM in the right amygdala and in the left cerebellum. Within the synaestheste group, comparing synaesthetes who reported numbers as the inducer with synaesthetes who reported other stimuli as the inducer revealed increase in GM in the left angular gyrus, which is associated with the verbal aspect of number processing. These results reveal neuroanatomical differences between synaesthetes and controls, and show the impact of the type of inducer in the synaesthetic brain. We discuss these findings in line with current neurobiological models of synaesthesia.

Creating Colored Letters: Familial Markers of Grapheme–Color Synesthesia in Parietal Lobe Activation and Structure

Perception is inherently subjective, and individual differences in phenomenology are well illustrated by the phenomenon of synesthesia (highly specific, consistent, and automatic cross-modal experiences, in which the external stimulus corresponding to the additional sensation is absent). It is unknown why some people develop synesthesia and others do not. In the current study, we tested whether neural markers related to having synesthesia in the family were evident in brain function and structure. Relatives of synesthetes (who did not have any type of synesthesia themselves) and matched controls read specially prepared books with colored letters for several weeks and were scanned before and after reading using magnetic resonance imaging. Effects of acquired letter–color associations were evident in brain activation. Training-related activation (while viewing black letters) in the right angular gyrus of the parietal lobe was directly related to the strength of the learned letter–color associations (behavioral Stroop effect). Within this obtained angular gyrus ROI, the familial trait of synesthesia related to brain activation differences while participants viewed both black and colored letters. Finally, we compared brain structure using voxel-based morphometry and diffusion tensor imaging to test for group differences and training effects. One cluster in the left superior parietal lobe had significantly more coherent white matter in the relatives compared with controls. No evidence for experience-dependent plasticity was obtained. For the first time, we present evidence suggesting that the (nonsynesthete) relatives of grapheme–color synesthetes show atypical grapheme processing as well as increased brain connectivity.

When music is salty: The crossmodal associations between sound and taste

Here we investigate associations between complex auditory and complex taste stimuli. A novel piece of music was composed and recorded in four different styles of musical articulation to reflect the four basic tastes groups (sweet, sour, salty, bitter). In Experiment 1, participants performed above chance at pairing the music clips with corresponding taste words. Experiment 2 uses multidimensional scaling to interpret how participants categorize these musical stimuli, and to show that auditory categories can be organized in a similar manner as taste categories. Experiment 3 introduces four different flavors of custom-made chocolate ganache and shows that participants can match music clips with the corresponding taste stimuli with above-chance accuracy. Experiment 4 demonstrates the partial role of pleasantness in crossmodal mappings between sound and taste. The present findings confirm that individuals are able to make crossmodal associations between complex auditory and gustatory stimuli, and that valence may mediate multisensory integration in the general population.

The Merit of Synesthesia for Consciousness Research

Synesthesia is a phenomenon in which additional perceptual experiences are elicited by sensory stimuli or cognitive concepts. Synesthetes possess a unique type of phenomenal experiences not directly triggered by sensory stimulation. Therefore, for better understanding of consciousness it is relevant to identify the mental and physiological processes that subserve synesthetic experience. In the present work we suggest several reasons why synesthesia has merit for research on consciousness. We first review the research on the dynamic and rapidly growing field of the studies of synesthesia. We particularly draw attention to the role of semantics in synesthesia, which is important for establishing synesthetic associations in the brain. We then propose that the interplay between semantics and sensory input in synesthesia can be helpful for the study of the neural correlates of consciousness, especially when making use of ambiguous stimuli for inducing synesthesia. Finally, synesthesia-related alterations of brain networks and functional connectivity can be of merit for the study of consciousness.

Multisensory integration and cross-modal learning in synaesthesia: A unifying model

Recent research into synaesthesia has highlighted the role of learning, yet synaesthesia is clearly a genetic condition. Here we ask how can the idea that synaesthesia reflects innate, genetic differences be reconciled with models that suggest it is driven by learning. A number of lines of evidence suggest that synaesthesia relies on, or at least interacts with, processes of multisensory integration that are common across all people. These include multisensory activations that arise in early regions of the brain as well as feedback from longer-term cross-modal associations generated in memory. These cognitive processes may interact independently to influence the phenomenology of the synaesthetic experience, as well as the individual differences within particular types of synaesthesia. The theoretical framework presented here is consistent with both an innate difference as the fundamental driver of the condition of synaesthesia, and with experiential and semantic influences on the eventual phenotype that emerges. In particular, it proposes that the internally generated synaesthetic percepts are treated similarly to other sensory information as the brain is learning the multisensory attributes of objects and developing cross-modal associations that merge in the concept of the object.

Sweet and sour: music and taste associations

Purpose

– This paper aims to extend current understanding concerning the cross-modal correspondences between sounds and tastes by introducing new research tools and experimental data to study associations and their reflections between music and taste.

Design/methodology/approach

– The experiment design addresses the multidisciplinary approach by using cultural, chemical and statistical analysis methods.

Findings

– The paper provides further evidence that exposure to the “sweet” or “sour” musical pieces influences people’s food-related thinking processes and behaviors. It also demonstrates that sweet or sour elements in the music may reflect to actual sweetness (as measured by sugar content) and sourness (as measured by organic acid content) of foods developed in association with music carrying similar taste characteristics.

Research limitations/implications

– The findings should be replicated and expanded using larger consumer samples and wider repertoires of “taste music” and dependent variables. Also, the level of experimental control should be improved; e.g., the “sweet” and “sour” music were produced using different instruments, which may have an influence to the results.

Practical implications

– Ambient “taste music” that is congruent with the basic flavors of the dishes can be played in restaurants to highlight guests’ sensory experience.

Social implications

– By carefully considering the symbolic meanings of the music used in different social situations, it is possible to create multimodal experiences and even subconscious expectations in people’ minds.

Originality/value

– Cross-modal associations are made between the tastes and music. This can influence on perception of food and provide new ways to build multimodal gastronomic experiences.

A critical review of the neuroimaging literature on synesthesia

Synesthesia refers to additional sensations experienced by some people for specific stimulations, such as the systematic arbitrary association of colors to letters for the most studied type. Here, we review all the studies (based mostly on functional and structural magnetic resonance imaging) that have searched for the neural correlates of this subjective experience, as well as structural differences related to synesthesia. Most differences claimed for synesthetes are unsupported, due mainly to low statistical power, statistical errors, and methodological limitations. Our critical review therefore casts some doubts on whether any neural correlate of the synesthetic experience has been established yet. Rather than being a neurological condition (i.e., a structural or functional brain anomaly), synesthesia could be reconsidered as a special kind of childhood memory, whose signature in the brain may be out of reach with present brain imaging techniques.

Rates of white matter hyperintensities compatible with the radiological profile of multiple sclerosis within self-referred synesthete populations

Synesthesia is an inherited condition causing unusual secondary sensations (e.g, sounds might be experienced as both auditory and visual percepts). The condition has been linked with cognitive and perceptual benefits and is considered a benign alternative form of perception. Here, we investigate self-referred synesthete populations and their rates of radiologically determined white matter hyperintensities (WMH) of a type compatible with the McDonald imaging criteria for the diagnosis of multiple sclerosis (MS). MS is a chronic condition resulting in damage to myelination surrounding nerve fibers of the central nervous system (CNS). Magnetic resonance imaging (MRI) features highly suggestive of MS without overt clinical symptoms are termed radiologically isolated syndrome (RIS). We present data showing that the shared MRI profile of MS and RIS has been significantly overrepresented in synesthetes who have participated in neuroimaging research. We present validation of the clinical and MRI status of these synesthetes and an analysis showing the significant probability their unusual numbers may not have arisen by chance. We discuss how to interpret significant data based on small case numbers and consider the implications of our findings for synesthesia's clinical status.

Implicit interactions between number and space in digit-color synesthesia

In digit-color synesthesia, a variant of grapheme-color synesthesia, digits trigger an additional color percept. Recent work on number processing in synesthesia suggests that colors can implicitly elicit numerical representations in digit-color synesthetes implying that synesthesia is bidirectional. Furthermore, morphometric investigations revealed structural differences in the parietal cortex of grapheme-color synesthetes, i.e., in the brain region where interactions between number and space occur in non-synesthetic subjects. Based upon these previous findings, we here examined whether implicitly evoked numerical representations interact with spatial representations in synesthesia in such a way that even a non-numerical, visuo-spatial task (here: line bisection) is modulated, i.e., whether synesthetes exhibit a systematic bisection bias for colored lines. Thirteen digit-color synesthetes were asked to bisect two sets of lines which were colored in their individual synesthetic colors associated with a small or a large digit, respectively. For all colored line stimuli combined, digit-color synesthetes showed--like control subjects (n = 13, matched for age, gender, IQ and handedness)--a pseudo-neglect when bisecting colored lines. Measuring the color-induced change of the bisection bias (i.e., comparing the biases when bisecting lines colored according to a small number vs those lines corresponding to a large number) revealed that only digit-color synesthetes were significantly influenced by line color. The results provide further evidence for the bidirectional nature of synesthesia and support the concept of a mental number line. In addition, they extend previous reports on bidirectionality in synesthesia by showing that even non-numerical, visuo-spatial performance can be modulated by implicit bidirectional processes.

Cross-modal associations in synaesthesia: Vowel colours in the ear of the beholder

Human speech conveys many forms of information, but for some exceptional individuals (synaesthetes), listening to speech sounds can automatically induce visual percepts such as colours. In this experiment, grapheme–colour synaesthetes and controls were asked to assign colours, or shades of grey, to different vowel sounds. We then investigated whether the acoustic content of these vowel sounds influenced participants' colour and grey-shade choices. We found that both colour and grey-shade associations varied systematically with vowel changes. The colour effect was significant for both participant groups, but significantly stronger and more consistent for synaesthetes. Because not all vowel sounds that we used are “translatable” into graphemes, we conclude that acoustic–phonetic influences co-exist with established graphemic influences in the cross-modal correspondences of both synaesthetes and non-synaesthetes.

Achromatic synesthesias - a functional magnetic resonance imaging study

Grapheme-color synesthetes experience consistent, automatic and idiosyncratic colors associated with specific letters and numbers. Frequently, these specific associations exhibit achromatic synesthetic qualities (e.g. white, black or gray). In this study, we have investigated for the first time the neural basis of achromatic synesthesias, their relationship to chromatic synesthesias and the achromatic congruency effect in order to understand not only synesthetic color but also other components of the synesthetic experience. To achieve this aim, functional magnetic resonance imaging experiments were performed in a group of associator grapheme-color synesthetes and matched controls who were stimulated with real chromatic and achromatic stimuli (Mondrians), and with letters and numbers that elicited different types of grapheme-color synesthesias (i.e. chromatic and achromatic inducers which elicited chromatic but also achromatic synesthesias, as well as congruent and incongruent ones). The information derived from the analysis of Mondrians and chromatic/achromatic synesthesias suggests that real and synesthetic colors/achromaticity do not fully share neural mechanisms. The whole-brain analysis of BOLD signals in response to the complete set of synesthetic inducers revealed that the functional peculiarities of the synesthetic brain are distributed, and reflect different components of the synesthetic experience: a perceptual component, an (attentional) feature binding component, and an emotional component. Additionally, the inclusion of achromatic experiences has provided new evidence in favor of the emotional binding theory, a line of interpretation which constitutes a bridge between grapheme-color synesthesia and other developmental modalities of the phenomenon.

Why we are not all synesthetes (not even weakly so)

A little over a decade ago, Martino and Marks (Current Directions in Psychological Science 10:61-65, 2001) put forward the influential claim that cases of intuitive matchings between stimuli in different sensory modalities should be considered as a weak form of synesthesia. Over the intervening years, many other researchers have agreed-at the very least, implicitly-with this position (e.g., Bien, ten Oever, Goebel, & Sack NeuroImage 59:663-672, 2012; Eagleman Cortex 45:1266-1277, 2009; Esterman, Verstynen, Ivry, & Robertson Journal of Cognitive Neuroscience 18:1570-1576, 2006; Ludwig, Adachi, & Matzuzawa Proceedings of the National Academy of Sciences of the United States of America 108:20661-20665, 2011; Mulvenna & Walsh Trends in Cognitive Sciences 10:350-352, 2006; Sagiv & Ward 2006; Zellner, McGarry, Mattern-McClory, & Abreu Chemical Senses 33:211-222:2008). Here, though, we defend the separatist view, arguing that these cases are likely to form distinct kinds of phenomena despite their superficial similarities. We believe that crossmodal correspondences should be studied in their own right and not assimilated, either in terms of the name used or in terms of the explanation given, to synesthesia. To conflate these two phenomena is both inappropriate and potentially misleading. Below, we critically evaluate the evidence concerning the descriptive and constitutive features of crossmodal correspondences and synesthesia and highlight how they differ. Ultimately, we wish to provide a general definition of crossmodal correspondences as acquired, malleable, relative, and transitive pairings between sensory dimensions and to provide a framework in which to integrate the nonsystematic cataloguing of new cases of crossmodal correspondences, a tendency that has increased in recent years.

Synaesthesia in Chinese characters: the role of radical function and position

Grapheme-colour synaesthetes experience unusual colour percepts when they encounter letters and/or digits. Studies of English-speaking grapheme-colour synaesthetes have shown that synaesthetic colours are sometimes triggered by rule-based linguistic mechanisms (e.g., B might be blue). In contrast, little is known about synaesthesia in logographic languages such as Chinese. The current study shows the mechanisms by which synaesthetic speakers of Chinese colour their language. One hypothesis is that Chinese characters might be coloured by their constituent morphological units, known as radicals, and we tested this by eliciting synaesthetic colours for characters while manipulating features of the radicals within them. We found that both the function (semantic vs. phonetic) and position (left vs. right) of radicals influence the nature of the synaesthetic colour generated. Our data show that in Chinese, as in English, synaesthetic colours are influenced by systematic rules, rather than by random associations, and that these rules are based on existing psycholinguistic mechanisms of language processing.

Combined structural and functional imaging reveals cortical deactivations in grapheme-color synaesthesia

Synaesthesia is a heritable condition in which particular stimuli generate specific and consistent sensory percepts or associations in another modality or processing stream. Functional neuroimaging studies have identified potential correlates of these experiences, including, in some but not all cases, the hyperactivation of visuotemporal areas and of parietal areas thought to be involved in perceptual binding. Structural studies have identified a similarly variable spectrum of differences between synaesthetes and controls. However, it remains unclear the extent to which these neural correlates reflect the synaesthetic experience itself or additional phenotypes associated with the condition. Here, we acquired both structural and functional neuroimaging data comparing thirteen grapheme-color synaesthetes with eleven non-synaesthetes. Using voxel-based morphometry and diffusion tensor imaging, we identify a number of clusters of increased volume of gray matter, of white matter or of increased fractional anisotropy in synaesthetes vs. controls. To assess the possible involvement of these areas in the synaesthetic experience, we used nine areas of increased gray matter volume as regions of interest in an fMRI experiment that characterized the contrast in response to stimuli which induced synaesthesia (i.e., letters) vs. those which did not (non-meaningful symbols). Four of these areas showed sensitivity to this contrast in synaesthetes but not controls. Unexpectedly, in two of them, in left lateral occipital cortex and in postcentral gyrus, the letter stimuli produced a strong negative BOLD signal in synaesthetes. An additional whole-brain fMRI analysis identified 14 areas, three of which were driven mainly by a negative BOLD response to letters in synaesthetes. Our findings suggest that cortical deactivations may be involved in the conscious experience of internally generated synaesthetic percepts.

A taste for words and sounds: a case of lexical-gustatory and sound-gustatory synesthesia

Gustatory forms of synesthesia involve the automatic and consistent experience of tastes that are triggered by non-taste related inducers. We present a case of lexical-gustatory and sound-gustatory synesthesia within one individual, SC. Most words and a subset of non-linguistic sounds induce the experience of taste, smell and physical sensations for SC. SC's lexical-gustatory associations were significantly more consistent than those of a group of controls. We tested for effects of presentation modality (visual vs. auditory), taste-related congruency, and synesthetic inducer-concurrent direction using a priming task. SC's performance did not differ significantly from a trained control group. We used functional magnetic resonance imaging to investigate the neural correlates of SC's synesthetic experiences by comparing her brain activation to the literature on brain networks related to language, music, and sound processing, in addition to synesthesia. Words that induced a strong taste were contrasted to words that induced weak-to-no tastes (“tasty” vs. “tasteless” words). Brain activation was also measured during passive listening to music and environmental sounds. Brain activation patterns showed evidence that two regions are implicated in SC's synesthetic experience of taste and smell: the left anterior insula and left superior parietal lobe. Anterior insula activation may reflect the synesthetic taste experience. The superior parietal lobe is proposed to be involved in binding sensory information across sub-types of synesthetes. We conclude that SC's synesthesia is genuine and reflected in her brain activation. The type of inducer (visual-lexical, auditory-lexical, and non-lexical auditory stimuli) could be differentiated based on patterns of brain activity.

Pathways to seeing music: enhanced structural connectivity in colored-music synesthesia

Synesthesia, a condition in which a stimulus in one sensory modality consistently and automatically triggers concurrent percepts in another modality, provides a window into the neural correlates of cross-modal associations. While research on grapheme-color synesthesia has provided evidence for both hyperconnectivity-hyperbinding and disinhibited feedback as potential underlying mechanisms, less research has explored the neuroanatomical basis of other forms of synesthesia. In the current study we investigated the white matter correlates of colored-music synesthesia. As these synesthetes report seeing colors upon hearing musical sounds, we hypothesized that they might show unique patterns of connectivity between visual and auditory association areas. We used diffusion tensor imaging to trace the white matter tracts in temporal and occipital lobe regions in 10 synesthetes and 10 matched non-synesthete controls. Results showed that synesthetes possessed hemispheric patterns of fractional anisotropy, an index of white matter integrity, in the inferior fronto-occipital fasciculus (IFOF), a major white matter pathway that connects visual and auditory association areas to frontal regions. Specifically, white matter integrity within the right IFOF was significantly greater in synesthetes than controls. Furthermore, white matter integrity in synesthetes was correlated with scores on audiovisual tests of the Synesthesia Battery, especially in white matter underlying the right fusiform gyrus. Our findings provide the first evidence of a white matter substrate of colored-music synesthesia, and suggest that enhanced white matter connectivity is involved in enhanced cross-modal associations.

Absolute pitch exhibits phenotypic and genetic overlap with synesthesia

Absolute pitch (AP) and synesthesia are two uncommon cognitive traits that reflect increased neuronal connectivity and have been anecdotally reported to occur together in an individual. Here we systematically evaluate the occurrence of synesthesia in a population of 768 subjects with documented AP. Out of these 768 subjects, 151 (20.1%) reported synesthesia, most commonly with color. These self-reports of synesthesia were validated in a subset of 21 study subjects, using an established methodology. We further carried out combined linkage analysis of 53 multiplex families with AP and 36 multiplex families with synesthesia. We observed a peak NPL LOD = 4.68 on chromosome 6q, as well as evidence of linkage on chromosome 2, using a dominant model. These data establish the close phenotypic and genetic relationship between AP and synesthesia. The chromosome 6 linkage region contains 73 genes; several leading candidate genes involved in neurodevelopment were investigated by exon resequencing. However, further studies will be required to definitively establish the identity of the causative gene(s) in the region.

Pre-attentive modulation of brain responses to tones in coloured-hearing synesthetes

BACKGROUND

Coloured-hearing (CH) synesthesia is a perceptual phenomenon in which an acoustic stimulus (the inducer) initiates a concurrent colour perception (the concurrent). Individuals with CH synesthesia "see" colours when hearing tones, words, or music; this specific phenomenon suggesting a close relationship between auditory and visual representations. To date, it is still unknown whether the perception of colours is associated with a modulation of brain functions in the inducing brain area, namely in the auditory-related cortex and associated brain areas. In addition, there is an on-going debate as to whether attention to the inducer is necessarily required for eliciting a visual concurrent, or whether the latter can emerge in a pre-attentive fashion.

RESULTS

By using the EEG technique in the context of a pre-attentive mismatch negativity (MMN) paradigm, we show that the binding of tones and colours in CH synesthetes is associated with increased MMN amplitudes in response to deviant tones supposed to induce novel concurrent colour perceptions. Most notably, the increased MMN amplitudes we revealed in the CH synesthetes were associated with stronger intracerebral current densities originating from the auditory cortex, parietal cortex, and ventral visual areas.

CONCLUSIONS

The automatic binding of tones and colours in CH synesthetes is accompanied by an early pre-attentive process recruiting the auditory cortex, inferior and superior parietal lobules, as well as ventral occipital areas.

Relating anatomical and social connectivity: white matter microstructure predicts emotional empathy

Understanding cues to the internal states of others involves a widely distributed network of brain regions. Although white matter (WM) connections are likely crucial for communication between these regions, the role of anatomical connectivity in empathic processing remains unexplored. The present study tested for a relationship between anatomical connectivity and empathy by assessing the WM microstructural correlates of affective empathy, which promotes interpersonal understanding through emotional reactions, and cognitive empathy, which does so via perspective taking. Associations between fractional anisotropy (FA) and the emotional (empathic concern, EC) and cognitive (perspective taking, PT) dimensions of empathy as assessed by the Interpersonal Reactivity Index were examined. EC was positively associated with FA in tracts providing communicative pathways within the limbic system, between perception and action-related regions, and between perception and affect-related regions, independently of individual differences in age, gender, and other dimensions of interpersonal reactivity. These findings provide a neuroanatomical basis for the rapid, privileged processing of emotional sensory information and the automatic elicitation of responses to the affective displays of others.

Reduced audio-visual integration in synaesthetes indicated by the double-flash illusion

It has been suggested that synaesthesia is the result of a hyper-sensitive multimodal binding-mechanism. To address the question whether multi-modal integration is altered in synaesthetes in general, grapheme-colour and auditory-visual synaesthetes were studied using the double-flash illusion. This illusion is induced by a single light flash presented together with multiple beep sounds, which is then perceived as multiple flashes. By varying the separation of auditory and visual stimuli, the hypothesis of a widened temporal window of audio-visual integration in synaesthetes was tested. As hypothesised, the results show differences between synaesthetes and controls concerning multisensory integration, but surprisingly other than expected synaesthetes perceive a reduced number of illusions and have a smaller time-window of audio-visual integration compared to controls. This indicates that they do not have a hyper-sensitive binding mechanism. On the contrary, synaesthetes seem to integrate even less than controls between vision and audition.

Pseudo-synesthesia through reading books with colored letters

BACKGROUND

Synesthesia is a phenomenon where a stimulus produces consistent extraordinary subjective experiences. A relatively common type of synesthesia involves perception of color when viewing letters (e.g. the letter 'a' always appears as light blue). In this study, we examine whether traits typically regarded as markers of synesthesia can be acquired by simply reading in color.

METHODOLOGY/PRINCIPAL FINDINGS

Non-synesthetes were given specially prepared colored books to read. A modified Stroop task was administered before and after reading. A perceptual crowding task was administered after reading. Reading one book (>49,000 words) was sufficient to induce effects regarded as behavioral markers for synesthesia. The results of the Stroop tasks indicate that it is possible to learn letter-color associations through reading in color (F(1, 14) = 5.85, p = .030). Furthermore, Stroop effects correlated with subjective reports about experiencing letters in color (r(13) = 0.51, p = .05). The frequency of viewing letters is related to the level of association as seen by the difference in the Stroop effect size between upper- and lower-case letters (t(14) = 2.79, p = .014) and in a subgroup of participants whose Stroop effects increased as they continued to read in color. Readers did not show significant performance advantages on the crowding task compared to controls. Acknowledging the many differences between trainees and synesthetes, results suggest that it may be possible to acquire a subset of synesthetic behavioral traits in adulthood through training.

CONCLUSION/SIGNIFICANCE

To our knowledge, this is the first evidence of acquiring letter-color associations through reading in color. Reading in color appears to be a promising avenue in which we may explore the differences and similarities between synesthetes and non-synesthetes. Additionally, reading in color is a plausible method for a long-term 'synesthetic' training program.

A composition algorithm based on crossmodal taste-music correspondences

While there is broad consensus about the structural similarities between language and music, comparably less attention has been devoted to semantic correspondences between these two ubiquitous manifestations of human culture. We have investigated the relations between music and a narrow and bounded domain of semantics: the words and concepts referring to taste sensations. In a recent work, we found that taste words were consistently mapped to musical parameters. Bitter is associated with low-pitched and continuous music (legato), salty is characterized by silences between notes (staccato), sour is high pitched, dissonant and fast and sweet is consonant, slow and soft (Mesz et al., 2011). Here we extended these ideas, in a synergistic dialog between music and science, investigating whether music can be algorithmically generated from taste-words. We developed and implemented an algorithm that exploits a large corpus of classic and popular songs. New musical pieces were produced by choosing fragments from the corpus and modifying them to minimize their distance to the region in musical space that characterizes each taste. In order to test the capability of the produced music to elicit significant associations with the different tastes, musical pieces were produced and judged by a group of non-musicians. Results showed that participants could decode well above chance the taste-word of the composition. We also discuss how our findings can be expressed in a performance bridging music and cognitive science.

Disinhibited feedback as a cause of synesthesia: evidence from a functional connectivity study on auditory-visual synesthetes

In synesthesia, certain stimuli to one sensory modality lead to sensory perception in another unstimulated modality. In addition to other models, a two-stage model is discussed to explain this phenomenon, which combines two previously formulated hypotheses regarding synesthesia: direct cross-activation and hyperbinding. The direct cross-activation model postulates that direct connections between sensory-specific areas are responsible for co-activation and synesthetic perception. The hyperbinding hypothesis suggests that the inducing stimulus and the synesthetic sensation are coupled by a sensory nexus area, which may be located in the parietal cortex. This latter hypothesis is compatible with the disinhibited feedback model, which suggests unusual feedback from multimodal convergence areas as the cause of synesthesia. In this study, the relevance of these models was tested in a group (n=14) of auditory-visual synesthetes by performing a functional connectivity analysis on functional magnetic resonance imaging (fMRI) data. Different simple and complex sounds were used as stimuli, and functionally defined seed areas in the bilateral auditory cortex (AC) and the left inferior parietal cortex (IPC) were used for the connectivity calculations. We found no differences in the connectivity of the AC and the visual areas between synesthetes and controls. The main finding of the study was stronger connectivity of the left IPC with the left primary auditory and right primary visual cortex in the group of auditory-visual synesthetes. The results support the model of disinhibited feedback as a cause of synesthetic perception but do not suggest direct cross-activation.

A strong parietal hub in the small-world network of coloured-hearing synaesthetes during resting state EEG

We investigated whether functional brain networks are different in coloured-hearing synaesthetes compared with non-synaesthetes. Based on resting state electroencephalographic (EEG) activity, graph-theoretical analysis was applied to functional connectivity data obtained from different frequency bands (theta, alpha1, alpha2, and beta) of 12 coloured-hearing synaesthetes and 13 non-synaesthetes. The analysis of functional connectivity was based on estimated intra-cerebral sources of brain activation using standardized low-resolution electrical tomography. These intra-cerebral sources of brain activity were subjected to graph-theoretical analysis yielding measures representing small-world network characteristics (cluster coefficients and path length). In addition, brain regions with strong interconnections were identified (so-called hubs), and the interconnectedness of these hubs were quantified using degree as a measure of connectedness. Our analysis was guided by the two-stage model proposed by Hubbard and Ramachandran (2005). In this model, the parietal lobe is thought to play a pivotal role in binding together the synaesthetic perceptions (hyperbinding). In addition, we hypothesized that the auditory cortex and the fusiform gyrus would qualify as strong hubs in synaesthetes. Although synaesthetes and non-synaesthetes demonstrated a similar small-world network topology, the parietal lobe turned out to be a stronger hub in synaesthetes than in non-synaesthetes supporting the two-stage model. The auditory cortex was also identified as a strong hub in these coloured-hearing synaesthetes (for the alpha2 band). Thus, our a priori hypotheses receive strong support. Several additional hubs (for which no a priori hypothesis has been formulated) were found to be different in terms of the degree measure in synaesthetes, with synaesthetes demonstrating stronger degree measures indicating stronger interconnectedness. These hubs were found in brain areas known to be involved in controlling memory processes (alpha1: hippocampus and retrosplenial area), executive functions (alpha1 and alpha2: ventrolateral prefrontal cortex; theta: inferior frontal cortex), and the generation of perceptions (theta: extrastriate cortex; beta: subcentral area). Taken together this graph-theoretical analysis of the resting state EEG supports the two-stage model in demonstrating that the left-sided parietal lobe is a strong hub region, which is stronger functionally interconnected in synaesthetes than in non-synaesthetes. The right-sided auditory cortex is also a strong hub supporting the idea that coloured-hearing synaesthetes demonstrate a specific auditory cortex. A further important point is that these hub regions are even differently operating at rest supporting the idea that these hub characteristics are predetermining factors of coloured-hearing synaesthesia.

The neural basis of illusory gustatory sensations: two rare cases of lexical-gustatory synaesthesia

Lexical-gustatory synaesthesia is a rare phenomenon in which the individual experiences flavour sensations when they read, hear, or imagine words. In this study, we provide insight into the neural basis of this form of synaesthesia using functional neuroimaging. Words known to evoke pleasant, neutral, and unpleasant synaesthetic tastes and synaesthetically tasteless words were presented to two lexical-gustatory synaesthetes, during fMRI scanning. Ten non-synaesthetic participants were also scanned on the same list of words. The synaesthetic brain displayed a different pattern of activity to words when compared to the non-synaesthetes, with insula activation related to viewing words that elicited tastes that have an associated emotional valence (i.e., pleasant or unpleasant tastes). The subjective intensity of the synaesthesia was correlated with activity in the medial parietal lobes (precuneus/retrosplenial cortex), which are implicated in polymodal imagery and self-directed thought. This region has also previously been activated in studies of lexical-colour synaesthesia, suggesting its role may not be limited to the type of synaesthesia explored here.

Brain areas involved in synaesthesia: a review

Despite a recent upsurge of research, much remains unknown about the neurobiological mechanisms underlying synaesthesia. By integrating results obtained so far in Magnetic Resonance Imaging (MRI) studies, this contribution sheds light on the role of particular brain regions in synaesthetic experiences. First, in accordance with its sensory nature, it seems that the sensory brain areas corresponding to the type of synaesthetic experience are activated. Synaesthetic colour experiences can activate colour regions in occipito-temporal cortex, but this is not necessarily restricted to V4. Furthermore, sensory and motor brain regions have been obtained that extend beyond the particular type of synaesthesia studied. Second, differences in experimental setup, number and type of synaesthetes tested, and method to delineate regions of interest may help explain inconsistent results obtained in the BOLD-MRI (Blood Oxygen Level Dependent functional MRI) studies. Third, an overview of obtained results shows that a network of brain areas rather than a single brain region underlies synaesthesia. Six brain regions of overlapping results emerge, these regions are in sensory and motor regions as well as 'higher level' regions in parietal and frontal lobe. We propose that these regions are related to three different cognitive processes inherently part of synaesthesia; the sensory processes, the (attentional) 'binding' processes, and cognitive control processes. Finally, we discuss how these functional and structural brain properties might relate to the development of synaesthesia. In particular, we believe this relationship is better understood by separating the question what underlies the presence of synaesthesia ('trait') from what determines particular synaesthetic associations ('type').

Synaesthesia: cross activations, high interconnectivity, and a parietal hub

This review summarizes the most recent studies on synaesthesia, particularly studies on grapheme-colour synaesthesia, time-space synaesthesia, and coloured-hearing synaesthesia. Based on behavioural as well as neuroimaging studies, there is emerging evidence that synaesthesia is not only caused by the cross activation of two sensory areas but that it may require additional binding processes, which are assumed to take place in the parietal lobe. However, divergent results exist with respect to the lateralization of this effect, i.e. whether it is the left or right parietal lobe most responsible. Studies also indicate that attention modulates the synaesthetic experience. Furthermore, it has been shown that synaesthetes demonstrate a higher level of connectivity, thus supporting the view of a genetic pre-disposition of synaesthesia.

Survival of the synesthesia gene: why do people hear colors and taste words?

This Unsolved Mystery reviews the biological evidence for why synesthesia, a condition in which stimuli presented through one modality spontaneously evoke sensations in an unrelated modality, may have been conserved in the population.

Synesthesia is a perceptual experience in which stimuli presented through one modality will spontaneously evoke sensations in an unrelated modality. The condition occurs from increased communication between sensory regions and is involuntary, automatic, and stable over time. While synesthesia can occur in response to drugs, sensory deprivation, or brain damage, research has largely focused on heritable variants comprising roughly 4% of the general population. Genetic research on synesthesia suggests the phenomenon is heterogeneous and polygenetic, yet it remains unclear whether synesthesia ever provided a selective advantage or is merely a byproduct of some other useful selected trait. Progress in uncovering the genetic basis of synesthesia will help us understand why synesthesia has been conserved in the population.

Enhanced cortical excitability in grapheme-color synesthesia and its modulation

Synesthesia is an unusual condition characterized by the over-binding of two or more features and the concomitant automatic and conscious experience of atypical, ancillary images or perceptions [1–3]. Previous research suggests that synesthetes display enhanced modality-specific perceptual processing [4–7], but it remains unclear whether enhanced processing contributes to conscious awareness of color photisms. In three experiments, we investigated whether grapheme-color synesthesia is characterized by enhanced cortical excitability in primary visual cortex and the role played by this hyperexcitability in the expression of synesthesia. Using transcranial magnetic stimulation, we show that synesthetes display 3-fold lower phosphene thresholds than controls during stimulation of the primary visual cortex. We next used transcranial direct current stimulation to discriminate between two competing hypotheses of the role of hyperexcitability in the expression of synesthesia. We demonstrate that synesthesia can be selectively augmented with cathodal stimulation and attenuated with anodal stimulation of primary visual cortex. A control task revealed that the effect of the brain stimulation was specific to the experience of synesthesia. These results indicate that hyperexcitability acts as a source of noise in visual cortex that influences the availability of the neuronal signals underlying conscious awareness of synesthetic photisms.