Higher-level linguistic categories dominate lower-level acoustics in lexical tone processing

Tone Deafness in Music Does Not Preclude Distributional Learning of Nonnative Tonal Languages in Individuals With Congenital Amusia

PURPOSE

Previous studies have shown that individuals with congenital amusia exhibit deficient pitch processing across music and language domains. This study investigated whether adult Chinese-speaking listeners with amusia were still able to learn Thai lexical tones based on stimulus frequency of statistical distribution via distributional learning, despite their degraded lexical tone perception.

METHOD

Following a pretest-training-posttest design, 21 amusics and 23 typical, musically intact listeners were assigned into bimodal and unimodal distribution conditions. Listeners were asked to discriminate minimal pairs of Thai mid-level tone and falling tone superimposed on variable base syllables and uttered by different speakers. The perceptual accuracy for each test session and improvement from pretest to posttest were collected and analyzed between the two groups using generalized mixed-effects models.

RESULTS

When discriminating Thai lexical tones, amusics were less accurate than typical listeners. Nonetheless, similarly to control listeners, perceptual gains from pretest to posttest were observed in bimodally rather than unimodally trained amusics, as evidenced by both trained and nontrained test words.

CONCLUSIONS

Amusics are able to learn lexical tones in a second or foreign context of speech. This extends previous research by showing that amusics' distributional learning of linguistic pitch remains largely preserved despite their degraded pitch processing. It is thus likely that manifestations of amusia in speech could not result from their abnormal statistical learning mechanism. This study meanwhile provides a heuristic approach for future studies to apply this paradigm into amusics' treatment to mitigate their pitch-processing disorder.

Neural–Behavioral Relation in Phonetic Discrimination Modulated by Language Background

It is a well-demonstrated phenomenon that listeners can discriminate native phonetic contrasts better than nonnative ones. Recent neuroimaging studies have started to reveal the underlying neural mechanisms. By focusing on the mismatch negativity/response (MMN/R), a widely studied index of neural sensitivity to sound change, researchers have observed larger MMNs for native contrasts than for nonnative ones in EEG, but also a more focused and efficient neural activation pattern for native contrasts in MEG. However, direct relations between behavioral discrimination and MMN/R are rarely reported. In the current study, 15 native English speakers and 15 native Spanish speakers completed both a behavioral discrimination task and a separate MEG recording to measure MMR to a VOT-based speech contrast (i.e., pre-voiced vs. voiced stop consonant), which represents a phonetic contrast native to Spanish speakers but is nonnative to English speakers. At the group level, English speakers exhibited significantly lower behavioral sensitivity (d’) to the contrast but a more expansive MMR, replicating previous studies. Across individuals, a significant relation between behavioral sensitivity and the MMR was only observed in the Spanish group. Potential differences in the mechanisms underlying behavioral discrimination for the two groups are discussed.

Categorical Perception of Mandarin Pitch Directions by Cantonese-Speaking Musicians and Non-musicians

Purpose: This study is to investigate whether Cantonese-speaking musicians may show stronger CP than Cantonese-speaking non-musicians in perceiving pitch directions generated based on Mandarin tones. It also aims to examine whether musicians may be more effective in processing stimuli and more sensitive to subtle differences caused by vowel quality.

Methods: Cantonese-speaking musicians and non-musicians performed a categorical identification and a discrimination task on rising and falling continua of fundamental frequency generated based on Mandarin level, rising and falling tones on two vowels with nine duration values.

Results: Cantonese-speaking musicians exhibited a stronger categorical perception (CP) of pitch contours than non-musicians based on the identification and discrimination tasks. Compared to non-musicians, musicians were also more sensitive to the change of stimulus duration and to the intrinsic F₀ in pitch perception in pitch processing.

Conclusion: The CP was strengthened due to musical experience and musicians benefited more from increased stimulus duration and were more efficient in pitch processing. Musicians might be able to better use the extra time to form an auditory representation with more acoustic details. Even with more efficiency in pitch processing, musicians' ability to detect subtle pitch changes caused by intrinsic F₀ was not undermined, which is likely due to their superior ability to process temporal information. These results thus suggest musicians may have a great advantage in learning tones of a second language.

The Development of Categorical Perception of Segments and Suprasegments in Mandarin-Speaking Preschoolers

This study investigated the developmental trajectories of categorical perception (CP) of segments (i.e., stops) and suprasegments (i.e., lexical tones) in an attempt to examine the perceptual development of phonological categories and whether CP of suprasegments develops in parallel with that of segments. Forty-seven Mandarin-speaking monolingual preschoolers aged four to six years old, and fourteen adults completed both identification and discrimination tasks of the Tone 1-2 continuum and the /pa/-/p^ha/ continuum. Results revealed that children could perceive both lexical tones and aspiration of stops in a categorical manner by age four. The boundary position did not depend on age, with children having similar positions to adults regardless of speech continuum types. The boundary width, on the other hand, reached the adult-like level at age six for lexical tones, but not for stops. In addition, the within-category discrimination score did not differ significantly between children and adults for both continua. The between-category discrimination score improved with age and achieved the adult-like level at age five for lexical tones, but still not for stops even at age six. It suggests that the fine-grained perception of phonological categories is a protracted process, and the improvement and varying timeline of the development of segments and suprasegments are discussed in relation to statistical learning of the regularities of speech sounds in ambient language, ongoing maturation of perceptual systems, the memory mechanism underlying perceptual learning, and the intrinsic nature of speech elements.

Effects of Amateur Musical Experience on Categorical Perception of Lexical Tones by Native Chinese Adults: An ERP Study

Music impacting on speech processing is vividly evidenced in most reports involving professional musicians, while the question of whether the facilitative effects of music are limited to experts or may extend to amateurs remains to be resolved. Previous research has suggested that analogous to language experience, musicianship also modulates lexical tone perception but the influence of amateur musical experience in adulthood is poorly understood. Furthermore, little is known about how acoustic information and phonological information of lexical tones are processed by amateur musicians. This study aimed to provide neural evidence of cortical plasticity by examining categorical perception of lexical tones in Chinese adults with amateur musical experience relative to the non-musician counterparts. Fifteen adult Chinese amateur musicians and an equal number of non-musicians participated in an event-related potential (ERP) experiment. Their mismatch negativities (MMNs) to lexical tones from Mandarin Tone 2–Tone 4 continuum and non-speech tone analogs were measured. It was hypothesized that amateur musicians would exhibit different MMNs to their non-musician counterparts in processing two aspects of information in lexical tones. Results showed that the MMN mean amplitude evoked by within-category deviants was significantly larger for amateur musicians than non-musicians regardless of speech or non-speech condition. This implies the strengthened processing of acoustic information by adult amateur musicians without the need of focused attention, as the detection of subtle acoustic nuances of pitch was measurably improved. In addition, the MMN peak latency elicited by across-category deviants was significantly shorter than that by within-category deviants for both groups, indicative of the earlier processing of phonological information than acoustic information of lexical tones at the pre-attentive stage. The results mentioned above suggest that cortical plasticity can still be induced in adulthood, hence non-musicians should be defined more strictly than before. Besides, the current study enlarges the population demonstrating the beneficial effects of musical experience on perceptual and cognitive functions, namely, the effects of enhanced speech processing from music are not confined to a small group of experts but extend to a large population of amateurs.

Reduced Sensitivity to Between-Category Information but Preserved Categorical Perception of Lexical Tones in Tone Language Speakers With Congenital Amusia

Previous studies have shown that for congenital amusics, long-term tone language experience cannot compensate for lexical tone processing difficulties. However, it is still unknown whether such difficulties are merely caused by domain-transferred insensitivity in lower-level acoustic processing and/or by higher-level phonological processing of linguistic pitch as well. The current P300 study links and extends previous studies by uncovering the neurophysiological mechanisms underpinning lexical tone perception difficulties in Mandarin-speaking amusics. Both the behavioral index (d′) and P300 amplitude showed reduced within-category as well as between-category sensitivity among the Mandarin-speaking amusics regardless of the linguistic status of the signal. The results suggest that acoustic pitch processing difficulties in amusics are manifested profoundly and further persist into the higher-level phonological processing that involves the neural processing of different lexical tone categories. Our findings indicate that long-term tone language experience may not compensate for the reduced acoustic pitch processing in tone language speakers with amusia but rather may extend to the neural processing of the phonological information of lexical tones during the attentive stage. However, from both the behavioral and neural evidence, the peakedness scores of the d′ and P300 amplitude were comparable between amusics and controls. It seems that the basic categorical perception (CP) pattern of native lexical tones is preserved in Mandarin-speaking amusics, indicating that they may have normal or near normal long-term categorical memory.

How musical experience affects tone perception efficiency by musicians of tonal and non-tonal speakers?

Purpose

To investigate if, regardless of language background (tonal or non-tonal), musicians may show stronger CP than non-musicians; To examine if native speakers of English (English or non-tonal musicians henceforth) or Mandarin Chinese (Mandarin or tonal musicians henceforth) can better accommodate multiple functions of the same acoustic cue and if musicians’ sensitivity to pitch of lexical tones comes at the cost of slower processing.

Method

English and Mandarin Musicians and non-musicians performed a categorical identification and a discrimination task on rising and falling continua of fundamental frequency on two vowels with 9 duration values.

Results

Non-tonal musicians exhibited significantly stronger categorical perception of pitch contour than non-tonal non-musicians. However, tonal musicians did not consistently perceive the two types of pitch directions more categorically than tonal non-musicians. Both tonal and non-tonal musicians also benefited more from increasing stimulus duration in processing pitch changes than non-musicians and they generally require less time for pitch processing. Musicians were also more sensitive to intrinsic F0 in pitch perception and differences of pitch types.

Conclusion

The effect of musical training strengthens categorical perception more consistently in non-tonal speakers than tonal speakers. Overall, musicians benefit more from increased stimulus duration, due perhaps to their greater sensitivity to temporal information, thus allowing them to be better at forming a more robust auditory representation and matching sounds to internalized memory templates. Musicians also attended more to acoustic details such as intrinsic F0 and pitch types in pitch processing, and yet, overall, their categorization of pitch was not compromised by traces of these acoustic details from their auditory short-term working memory. These findings may lead to a better understanding of pitch perception deficits in special populations, particularly among individuals diagnosed with autism spectrum disorder (ASD).

Lower-level acoustics underlie higher-level phonological categories in lexical tone perception

The pitch-processing deficit associated with congenital amusia has been shown to be transferable to lexical tone processing. However, it remains unclear whether the tone perception difficulties of amusics are merely due to the domain-general deficit in acoustic processing or additionally caused by impaired higher-level phonological operations. Answers to this question can shed light on the influence of lower-level acoustic processing on higher-level phonological processing. Using a modified categorical perception paradigm, the present study indicates that the acoustic processing deficit systematically extends to higher-level phonological processing. These findings suggest that lower-level acoustics underlie higher-level phonological categories in lexical tone perception.

Cooperative cortical network for categorical processing of Chinese lexical tone

The variation of pitch in speech not only creates the intonation for affective communication but also signals different meaning of a word in tonal languages, like Chinese. Due to its subtle and brisk pitch contour distinction between tone categories, the underlying neural processing mechanism is largely unknown. Using direct recordings of the human brain, we found categorical neural responses to lexical tones over a distributed cooperative network that included not only the auditory areas in the temporal cortex but also motor areas in the frontal cortex. Strong causal links from the temporal cortex to the motor cortex were discovered, which provides new evidence of top-down influence and sensory–motor interaction during speech perception.

In tonal languages such as Chinese, lexical tone with varying pitch contours serves as a key feature to provide contrast in word meaning. Similar to phoneme processing, behavioral studies have suggested that Chinese tone is categorically perceived. However, its underlying neural mechanism remains poorly understood. By conducting cortical surface recordings in surgical patients, we revealed a cooperative cortical network along with its dynamics responsible for this categorical perception. Based on an oddball paradigm, we found amplified neural dissimilarity between cross-category tone pairs, rather than between within-category tone pairs, over cortical sites covering both the ventral and dorsal streams of speech processing. The bilateral superior temporal gyrus (STG) and the middle temporal gyrus (MTG) exhibited increased response latencies and enlarged neural dissimilarity, suggesting a ventral hierarchy that gradually differentiates the acoustic features of lexical tones. In addition, the bilateral motor cortices were also found to be involved in categorical processing, interacting with both the STG and the MTG and exhibiting a response latency in between. Moreover, the motor cortex received enhanced Granger causal influence from the semantic hub, the anterior temporal lobe, in the right hemisphere. These unique data suggest that there exists a distributed cooperative cortical network supporting the categorical processing of lexical tone in tonal language speakers, not only encompassing a bilateral temporal hierarchy that is shared by categorical processing of phonemes but also involving intensive speech–motor interactions over the right hemisphere, which might be the unique machinery responsible for the reliable discrimination of tone identities.

Musical training shapes neural responses to melodic and prosodic expectation

Current research on music processing and syntax or semantics in language suggests that music and language share partially overlapping neural resources. Pitch also constitutes a common denominator, forming melody in music and prosody in language. Further, pitch perception is modulated by musical training. The present study investigated how music and language interact on pitch dimension and whether musical training plays a role in this interaction. For this purpose, we used melodies ending on an expected or unexpected note (melodic expectancy being estimated by a computational model) paired with prosodic utterances which were either expected (statements with falling pitch) or relatively unexpected (questions with rising pitch). Participants' (22 musicians, 20 nonmusicians) ERPs and behavioural responses in a statement/question discrimination task were recorded. Participants were faster for simultaneous expectancy violations in the melodic and linguistic stimuli. Further, musicians performed better than nonmusicians, which may be related to their increased pitch tracking ability. At the neural level, prosodic violations elicited a front-central positive ERP around 150 ms after the onset of the last word/note, while musicians presented reduced P600 in response to strong incongruities (questions on low-probability notes). Critically, musicians' P800 amplitudes were proportional to their level of musical training, suggesting that expertise might shape the pitch processing of language. The beneficial aspect of expertise could be attributed to its strengthening effect of general executive functions. These findings offer novel contributions to our understanding of shared higher-order mechanisms between music and language processing on pitch dimension, and further demonstrate a potential modulation by musical expertise.

•

Melodic expectancy influences the processing of prosodic expectancy.

•

Musical expertise modulates pitch processing in music and language.

•

Musicians have a more refined response to pitch.

•

Musicians' neural responses are proportional to their level of musical expertise.

•

Possible association between the P200 neural component and behavioural facilitation.