Assessing sound symbolism: Investigating phonetic forms, visual shapes and letter fonts in an implicit bouba-kiki experimental paradigm

The size and shape of sound: The role of articulation and acoustics in iconicity and crossmodal correspondencesa)

Onomatopoeias like hiss and peep are iconic because their forms resemble their meanings. Iconicity can also involve forms and meanings in different modalities, such as when people match the nonce words bouba and kiki to round and angular objects, and mil and mal to small and large ones, also known as "sound symbolism." This paper focuses on what specific analogies motivate such correspondences in spoken language: do people associate shapes and size with how phonemes sound (auditory), or how they are produced (articulatory)? Based on a synthesis of empirical evidence probing the cognitive mechanisms underlying different types of sound symbolism, this paper argues that analogies based on acoustics alone are often sufficient, rendering extant articulatory explanations for many iconic phenomena superfluous. This paper further suggests that different types of crossmodal iconicity in spoken language can fruitfully be understood as an extension of onomatopoeia: when speakers iconically depict such perceptual characteristics as size and shape, they mimic the acoustics that are correlated with these characteristics in the natural world.

Synesthetic Correspondence: An Overview

Intramodal and cross-modal perceptual grouping based on the spatial proximity and temporal closeness between multiple sensory stimuli, as an operational principle has built a coherent and meaningful representation of the multisensory event/object. To implement and investigate the cross-modal perceptual grouping, researchers have employed excellent paradigms of spatial/temporal ventriloquism and cross-modal dynamic capture and have revealed the conditional constraints as well as the functional facilitations among various correspondence of sensory properties, with featured behavioral evidence, computational framework as well as brain oscillation patterns. Typically, synesthetic correspondence as a special type of cross-modal correspondence can shape the efficiency and effect-size of cross-modal interaction. For example, factors such as pitch/loudness in the auditory dimension with size/brightness in the visual dimension could modulate the strength of the cross-modal temporal capture. The empirical behavioral findings, as well as psychophysical and neurophysiological evidence to address the cross-modal perceptual grouping and synesthetic correspondence, were summarized in this review. Finally, the potential applications (such as artificial synesthesia device) and how synesthetic correspondence interface with semantics (sensory linguistics), as well as the promising research questions in this field have been discussed.

The bouba/kiki effect is robust across cultures and writing systems

The bouba/kiki effect-the association of the nonce word bouba with a round shape and kiki with a spiky shape-is a type of correspondence between speech sounds and visual properties with potentially deep implications for the evolution of spoken language. However, there is debate over the robustness of the effect across cultures and the influence of orthography. We report an online experiment that tested the bouba/kiki effect across speakers of 25 languages representing nine language families and 10 writing systems. Overall, we found strong evidence for the effect across languages, with bouba eliciting more congruent responses than kiki. Participants who spoke languages with Roman scripts were only marginally more likely to show the effect, and analysis of the orthographic shape of the words in different scripts showed that the effect was no stronger for scripts that use rounder forms for bouba and spikier forms for kiki. These results confirm that the bouba/kiki phenomenon is rooted in crossmodal correspondence between aspects of the voice and visual shape, largely independent of orthography. They provide the strongest demonstration to date that the bouba/kiki effect is robust across cultures and writing systems. This article is part of the theme issue 'Voice modulation: from origin and mechanism to social impact (Part II)'.

Vowel Length Expands Perceptual and Emotional Evaluations in Written Japanese Sound-Symbolic Words

In this study, we examined whether vowel length affected the perceptual and emotional evaluations of Japanese sound-symbolic words. The perceptual and emotional features of Japanese sound-symbolic words, which included short and long vowels, were evaluated by 209 native Japanese speakers. The results showed that subjective evaluations of familiarity, visual imageability, auditory imageability, tactile imageability, emotional valence, arousal, and length were significantly higher for sound-symbolic words with long vowels compared to those with short vowels. Additionally, a subjective evaluation of speed was significantly higher for written Japanese sound-symbolic words with short vowels than for those with long vowels. The current findings suggest that vowel length in written Japanese sound-symbolic words increases the perceptually and emotionally subjective evaluations of Japanese sound-symbolic words.

Stimulus Parameters Underlying Sound-Symbolic Mapping of Auditory Pseudowords to Visual Shapes

Sound symbolism refers to non-arbitrary mappings between the sounds of words and their meanings and is often studied by pairing auditory pseudowords such as "maluma" and "takete" with rounded and pointed visual shapes, respectively. However, it is unclear what auditory properties of pseudowords contribute to their perception as rounded or pointed. Here, we compared perceptual ratings of the roundedness/pointedness of large sets of pseudowords and shapes to their acoustic and visual properties using a novel application of representational similarity analysis (RSA). Representational dissimilarity matrices (RDMs) of the auditory and visual ratings of roundedness/pointedness were significantly correlated crossmodally. The auditory perceptual RDM correlated significantly with RDMs of spectral tilt, the temporal fast Fourier transform (FFT), and the speech envelope. Conventional correlational analyses showed that ratings of pseudowords transitioned from rounded to pointed as vocal roughness (as measured by the harmonics-to-noise ratio, pulse number, fraction of unvoiced frames, mean autocorrelation, shimmer, and jitter) increased. The visual perceptual RDM correlated significantly with RDMs of global indices of visual shape (the simple matching coefficient, image silhouette, image outlines, and Jaccard distance). Crossmodally, the RDMs of the auditory spectral parameters correlated weakly but significantly with those of the global indices of visual shape. Our work establishes the utility of RSA for analysis of large stimulus sets and offers novel insights into the stimulus parameters underlying sound symbolism, showing that sound-to-shape mapping is driven by acoustic properties of pseudowords and suggesting audiovisual cross-modal correspondence as a basis for language users' sensitivity to this type of sound symbolism.

Is un stylo sharper than une épée? Investigating the interaction of sound symbolism and grammatical gender in English and French speakers

While the arbitrariness of language has long been considered one of its defining features, there is growing evidence that non-arbitrariness also plays an important role. Here we investigated two sources of non-arbitrariness: systematicity (via grammatical gender) and iconicity (via shape sound symbolism). We manipulated these two elements orthogonally, allowing us to examine the effect of each. In Experiment 1, we found that French speakers associated nonwords containing feminine (masculine) endings with round (sharp) shapes. French speakers also associated nonwords containing round-sounding (sharp-sounding) phonemes with round (sharp) shapes. This was repeated using auditory presentation with both an English-speaking (Experiment 2a) and French-speaking (Experiment 2b) sample. As predicted, the English speakers showed no effects of grammatical gender, while the French speakers did. These results demonstrate that speakers of a language with grammatical gender associate different properties to words belonging to different genders. The results also show that sound symbolism can emerge in stimuli with existing associated information (i.e., endings indicative of grammatical gender, and the association that they evoke). Finally, while previous studies have looked at effects of arbitrary and non-arbitrary mappings contained in a single stimulus, this is the first study to demonstrate that different kinds of non-arbitrary mappings can have an effect when appearing in the same stimulus. Together these results add to our understanding of the importance of non-arbitrariness in language.