Interaction between grasping and articulation: How vowel and consonant pronunciation influences precision and power grip responses

The size and shape of sound: The role of articulation and acoustics in iconicity and crossmodal correspondencesa)

Onomatopoeias like hiss and peep are iconic because their forms resemble their meanings. Iconicity can also involve forms and meanings in different modalities, such as when people match the nonce words bouba and kiki to round and angular objects, and mil and mal to small and large ones, also known as "sound symbolism." This paper focuses on what specific analogies motivate such correspondences in spoken language: do people associate shapes and size with how phonemes sound (auditory), or how they are produced (articulatory)? Based on a synthesis of empirical evidence probing the cognitive mechanisms underlying different types of sound symbolism, this paper argues that analogies based on acoustics alone are often sufficient, rendering extant articulatory explanations for many iconic phenomena superfluous. This paper further suggests that different types of crossmodal iconicity in spoken language can fruitfully be understood as an extension of onomatopoeia: when speakers iconically depict such perceptual characteristics as size and shape, they mimic the acoustics that are correlated with these characteristics in the natural world.

Congruent vs. incongruent tasks in interdisciplinary stroke rehabilitation: a single-case report

PURPOSE

Stroke survivors may experience challenges in multiple domains (e.g., speech-language, dexterity, mobility) and pursue services from multiple professionals. Clinicians typically provide rehabilitation services in back-to-back sessions (multidisciplinary). Alternatively, two or more clinicians can co-treat a stroke survivor in one session (interdisciplinary). This pilot project examined task congruency in interdisciplinary stroke care.

METHOD

A stroke survivor chronically challenged by non-fluent aphasia and right hemiparesis completed spoken-naming and upper-limb tasks simultaneously. The concurrent tasks were presented in two conditions: congruent (i.e., naming a pictured item while tracing the first letter of the name) and incongruent (i.e., naming a pictured item while tracing a non-symbolic shape). The sequence of the two conditions was: baseline probes, congruent practice (eight weeks), no practice (eight weeks), incongruent practice (eight weeks), and no practice (eight weeks). The entire treatment program was implemented as independent home practice utilizing a computer.

RESULTS

The participant made significant improvements in naming and clinically meaningful gains in arm movements during the congruent condition, but not during the incongruent condition.

CONCLUSIONS

Data from this study suggest a potentially positive effect of simultaneous speech-language and upper-limb tasks. More research is warranted to further examine the role of task congruency in interdisciplinary stroke rehabilitation.

Sound symbolism in manual and vocal responses: phoneme-response interactions associated with grasping as well as vertical and size dimensions of keypresses

It has been shown that reading the vowel [i] and consonant [t] facilitates precision grip responses, while [ɑ] and [k] are associated with faster power grip responses. A similar effect has been observed when participants perform responses with small or large response keys. The present study investigated whether the vowels and consonants could produce different effects with the grip responses and keypresses when the speech units are read aloud (Experiment 1) or silently (Experiment 2). As a second objective, the study investigated whether the recently observed effect, in which the upper position of a visual stimulus is associated with faster vocalizations of the high vowel and the lower position is associated with the low vowel, can be observed in manual responses linking, for example, the [i] with responses of the upper key and [ɑ] with lower responses. Firstly, the study showed that when the consonants are overtly articulated, the interaction effect can be observed only with the grip responses, while the vowel production was shown to systematically influence small/large keypresses, as well as precision/power grip responses. Secondly, the vowel [i] and consonant [t] were associated with the upper responses, while [ɑ] and [k] were associated with the lower responses, particularly in the overt articulation task. The paper delves into the potential sound-symbolic implications of these phonetic elements, suggesting that their acoustic and articulatory characteristics might implicitly align them with specific response magnitudes, vertical positions, and grip types.

High-Pitched Sound is Open and Low-Pitched Sound is Closed: Representing the Spatial Meaning of Pitch Height

Research shows that high- and low-pitch sounds can be associated with various meanings. For example, high-pitch sounds are associated with small concepts, whereas low-pitch sounds are associated with large concepts. This study presents three experiments revealing that high-pitch sounds are also associated with open concepts and opening hand actions, while low-pitch sounds are associated with closed concepts and closing hand actions. In Experiment 1, this sound-meaning correspondence effect was shown using the two-alternative forced-choice task, while Experiments 2 and 3 used reaction time tasks to show this interaction. In Experiment 2, high-pitch vocalizations were found to facilitate opening hand gestures, and low-pitch vocalizations were found to facilitate closing hand gestures, when performed simultaneously. In Experiment 3, high-pitched vocalizations were produced particularly rapidly when the visual target stimulus presented an open object, and low-pitched vocalizations were produced particularly rapidly when the target presented a closed object. These findings are discussed concerning the meaning of intonational cues. They are suggested to be based on cross-modally representing conceptual spatial knowledge in sensory, motor, and affective systems. Additionally, this pitch-opening effect might share cognitive processes with other pitch-meaning effects.

Prominence effects in vocal iconicity: Implications for lexical access and language changea)

This paper explores how three cognitive and perceptual cues, vocal iconicity, resemblance-based mappings between form and meaning, and segment position and lexical stress, interact to affect word formation and language processing. The study combines an analysis of the word-internal positions that iconic segments occur in based on data from 245 language families with an experimental study in which participants representing more than 30 languages rated iconic and non-iconic pseudowords. The pseudowords were designed to systematically vary segment and stress placement across syllables. The results for study 1 indicate that segments used iconically appear approximately 0.26 segment positions closer toward the beginning of words compared to non-iconic segments. In study 2, it was found that iconic segments occurring in stressed syllables and non-iconic segments occurring in the second syllable were rated as significantly more fitting. These findings suggest that the interplay between vocal iconicity and prominence effects increases the predictive function of iconic segments by foregrounding sounds, which intrinsically carry semantic information. Consequently, these results contribute to the understanding of the widespread occurrence of vocal iconicity in human languages.

Sound-action symbolism in relation to precision manipulation and whole-hand grasp usage

It has been suggested that actions can provide a fruitful conceptual context for sound symbolism phenomena, and that tight interaction between manual and articulatory processes might cause that hand actions, in particular, are sound-symbolically associated with specific speech sounds. Experiment 1 investigated whether novel words, built from speech sounds that have been previously linked to precision or power grasp responses, are implicitly associated with perceived actions that present precision manipulation or whole-hand grasp tool-use or the corresponding utilisation pantomimes. In the two-alternative forced-choice task, the participants were more likely to match novel words to tool-use actions and corresponding pantomimes that were sound-symbolically congruent with the words. Experiment 2 showed that the same or even larger sound-action symbolism effect can be observed when the pantomimes present unfamiliar utilisation actions. Based on this we propose that the sound-action symbolism might originate from the same sensorimotor mechanisms that process the meaning of iconic gestural signs. The study presents a novel sound-action phenomenon and supports the view that hand-mouth interaction might manifest itself by associating specific speech sounds with grasp-related utilisations.

Beyond grasping: Syllables processing influences mere manual keypress

We aimed to better understand the link between vocalization and grasping. We especially test whether neurocognitive processes underlying this interaction are not grasping specific. To test this hypothesis, we used the procedure of a previous experiment, showing that silently reading the syllable KA and TI can facilitate power- and precision-grip responses, respectively. In our experiment, the participants have to silently read the syllable KA or TI but, according to the color of the syllables, have merely to press a large or small switch (we removed the grasping component of responses). Responses on the large switch were faster when the syllable KA was read compared with TI and conversely for the responses carried out on the small switch. This result supports that the influence of vocalization is not restricted to grasping responses, and, in addition, it supports an alternative, non-grasping-specific model of interactions between vocalization and grasping.

Neural basis of sound-symbolic pseudoword-shape correspondences

Non-arbitrary mapping between the sound of a word and its meaning, termed sound symbolism, is commonly studied through crossmodal correspondences between sounds and visual shapes, e.g., auditory pseudowords, like 'mohloh' and 'kehteh', are matched to rounded and pointed visual shapes, respectively. Here, we used functional magnetic resonance imaging (fMRI) during a crossmodal matching task to investigate the hypotheses that sound symbolism (1) involves language processing; (2) depends on multisensory integration; (3) reflects embodiment of speech in hand movements. These hypotheses lead to corresponding neuroanatomical predictions of crossmodal congruency effects in (1) the language network; (2) areas mediating multisensory processing, including visual and auditory cortex; (3) regions responsible for sensorimotor control of the hand and mouth. Right-handed participants (n = 22) encountered audiovisual stimuli comprising a simultaneously presented visual shape (rounded or pointed) and an auditory pseudoword ('mohloh' or 'kehteh') and indicated via a right-hand keypress whether the stimuli matched or not. Reaction times were faster for congruent than incongruent stimuli. Univariate analysis showed that activity was greater for the congruent compared to the incongruent condition in the left primary and association auditory cortex, and left anterior fusiform/parahippocampal gyri. Multivoxel pattern analysis revealed higher classification accuracy for the audiovisual stimuli when congruent than when incongruent, in the pars opercularis of the left inferior frontal (Broca's area), the left supramarginal, and the right mid-occipital gyri. These findings, considered in relation to the neuroanatomical predictions, support the first two hypotheses and suggest that sound symbolism involves both language processing and multisensory integration.

Neural Basis Of Sound-Symbolic Pseudoword-Shape Correspondences

Non-arbitrary mapping between the sound of a word and its meaning, termed sound symbolism, is commonly studied through crossmodal correspondences between sounds and visual shapes, e.g., auditory pseudowords, like 'mohloh' and 'kehteh', are matched to rounded and pointed visual shapes, respectively. Here, we used functional magnetic resonance imaging (fMRI) during a crossmodal matching task to investigate the hypotheses that sound symbolism (1) involves language processing; (2) depends on multisensory integration; (3) reflects embodiment of speech in hand movements. These hypotheses lead to corresponding neuroanatomical predictions of crossmodal congruency effects in (1) the language network; (2) areas mediating multisensory processing, including visual and auditory cortex; (3) regions responsible for sensorimotor control of the hand and mouth. Right-handed participants ( n = 22) encountered audiovisual stimuli comprising a simultaneously presented visual shape (rounded or pointed) and an auditory pseudoword ('mohloh' or 'kehteh') and indicated via a right-hand keypress whether the stimuli matched or not. Reaction times were faster for congruent than incongruent stimuli. Univariate analysis showed that activity was greater for the congruent compared to the incongruent condition in the left primary and association auditory cortex, and left anterior fusiform/parahippocampal gyri. Multivoxel pattern analysis revealed higher classification accuracy for the audiovisual stimuli when congruent than when incongruent, in the pars opercularis of the left inferior frontal (Broca's area), the left supramarginal, and the right mid-occipital gyri. These findings, considered in relation to the neuroanatomical predictions, support the first two hypotheses and suggest that sound symbolism involves both language processing and multisensory integration.

HIGHLIGHTS

fMRI investigation of sound-symbolic correspondences between auditory pseudowords and visual shapesFaster reaction times for congruent than incongruent audiovisual stimuliGreater activation in auditory and visual cortices for congruent stimuliHigher classification accuracy for congruent stimuli in language and visual areasSound symbolism involves language processing and multisensory integration.