Tuning Neural Phase Entrainment to Speech

Auditory and motor priming of metric structure improves understanding of degraded speech

Speech comprehension is enhanced when preceded (or accompanied) by a congruent rhythmic prime reflecting the metrical sentence structure. Although these phenomena have been described for auditory and motor primes separately, their respective and synergistic contribution has not been addressed. In this experiment, participants performed a speech comprehension task on degraded speech signals that were preceded by a rhythmic prime that could be auditory, motor or audiomotor. Both auditory and audiomotor rhythmic primes facilitated speech comprehension speed. While the presence of a purely motor prime (unpaced tapping) did not globally benefit speech comprehension, comprehension accuracy scaled with the regularity of motor tapping. In order to investigate inter-individual variability, participants also performed a Spontaneous Speech Synchronization test. The strength of the estimated perception-production coupling correlated positively with overall speech comprehension scores. These findings are discussed in the framework of the dynamic attending and active sensing theories.

Effects of Stimulus Rate and Periodicity on Auditory Cortical Entrainment to Continuous Sounds

The neural mechanisms underlying the exogenous coding and neural entrainment to repetitive auditory stimuli have seen a recent surge of interest. However, few studies have characterized how parametric changes in stimulus presentation alter entrained responses. We examined the degree to which the brain entrains to repeated speech (i.e., /ba/) and nonspeech (i.e., click) sounds using phase-locking value (PLV) analysis applied to multichannel human electroencephalogram (EEG) data. Passive cortico-acoustic tracking was investigated in N = 24 normal young adults utilizing EEG source analyses that isolated neural activity stemming from both auditory temporal cortices. We parametrically manipulated the rate and periodicity of repetitive, continuous speech and click stimuli to investigate how speed and jitter in ongoing sound streams affect oscillatory entrainment. Neuronal synchronization to speech was enhanced at 4.5 Hz (the putative universal rate of speech) and showed a differential pattern to that of clicks, particularly at higher rates. PLV to speech decreased with increasing jitter but remained superior to clicks. Surprisingly, PLV entrainment to clicks was invariant to periodicity manipulations. Our findings provide evidence that the brain's neural entrainment to complex sounds is enhanced and more sensitized when processing speech-like stimuli, even at the syllable level, relative to nonspeech sounds. The fact that this specialization is apparent even under passive listening suggests a priority of the auditory system for synchronizing to behaviorally relevant signals.

Neural dynamics underlying successful auditory short-term memory performance

Listeners often operate in complex acoustic environments, consisting of many concurrent sounds. Accurately encoding and maintaining such auditory objects in short-term memory is crucial for communication and scene analysis. Yet, the neural underpinnings of successful auditory short-term memory (ASTM) performance are currently not well understood. To elucidate this issue, we presented a novel, challenging auditory delayed match-to-sample task while recording MEG. Human participants listened to 'scenes' comprising three concurrent tone pip streams. The task was to indicate, after a delay, whether a probe stream was present in the just-heard scene. We present three key findings: First, behavioural performance revealed faster responses in correct versus incorrect trials as well as in 'probe present' versus 'probe absent' trials, consistent with ASTM search. Second, successful compared with unsuccessful ASTM performance was associated with a significant enhancement of event-related fields and oscillatory activity in the theta, alpha and beta frequency ranges. This extends previous findings of an overall increase of persistent activity during short-term memory performance. Third, using distributed source modelling, we found these effects to be confined mostly to sensory areas during encoding, presumably related to ASTM contents per se. Parietal and frontal sources then became relevant during the maintenance stage, indicating that effective STM operation also relies on ongoing inhibitory processes suppressing task-irrelevant information. In summary, our results deliver a detailed account of the neural patterns that differentiate successful from unsuccessful ASTM performance in the context of a complex, multi-object auditory scene.

Characterizing endogenous delta oscillations in human MEG

Rhythmic activity in the delta frequency range (0.5-3 Hz) is a prominent feature of brain dynamics. Here, we examined whether spontaneous delta oscillations, as found in invasive recordings in awake animals, can be observed in non-invasive recordings performed in humans with magnetoencephalography (MEG). In humans, delta activity is commonly reported when processing rhythmic sensory inputs, with direct relationships to behaviour. However, rhythmic brain dynamics observed during rhythmic sensory stimulation cannot be interpreted as an endogenous oscillation. To test for endogenous delta oscillations we analysed human MEG data during rest. For comparison, we additionally analysed two conditions in which participants engaged in spontaneous finger tapping and silent counting, arguing that internally rhythmic behaviours could incite an otherwise silent neural oscillator. A novel set of analysis steps allowed us to show narrow spectral peaks in the delta frequency range in rest, and during overt and covert rhythmic activity. Additional analyses in the time domain revealed that only the resting state condition warranted an interpretation of these peaks as endogenously periodic neural dynamics. In sum, this work shows that using advanced signal processing techniques, it is possible to observe endogenous delta oscillations in non-invasive recordings of human brain dynamics.

In Time with the Beat: Entrainment in Patients with Phonological Impairment, Apraxia of Speech, and Parkinson's Disease

In the present study, we investigated if individuals with neurogenic speech sound impairments of three types, Parkinson’s dysarthria, apraxia of speech, and aphasic phonological impairment, accommodate their speech to the natural speech rhythm of an auditory model, and if so, whether the effect is more significant after hearing metrically regular sentences as compared to those with an irregular pattern. This question builds on theories of rhythmic entrainment, assuming that sensorimotor predictions of upcoming events allow humans to synchronize their actions with an external rhythm. To investigate entrainment effects, we conducted a sentence completion task relating participants’ response latencies to the spoken rhythm of the prime heard immediately before. A further research question was if the perceived rhythm interacts with the rhythm of the participants’ own productions, i.e., the trochaic or iambic stress pattern of disyllabic target words. For a control group of healthy speakers, our study revealed evidence for entrainment when trochaic target words were preceded by regularly stressed prime sentences. Persons with Parkinson’s dysarthria showed a pattern similar to that of the healthy individuals. For the patient groups with apraxia of speech and with phonological impairment, considerably longer response latencies with differing patterns were observed. Trochaic target words were initiated with significantly shorter latencies, whereas the metrical regularity of prime sentences had no consistent impact on response latencies and did not interact with the stress pattern of the target words to be produced. The absence of an entrainment in these patients may be explained by the more severe difficulties in initiating speech at all. We discuss the results in terms of clinical implications for diagnostics and therapy in neurogenic speech disorders.

Does rhythmic priming improve grammatical processing in Hungarian-speaking children with and without developmental language disorder?

Research has described several features shared between musical rhythm and speech or language, and experimental studies consistently show associations between performance on tasks in the two domains as well as impaired rhythm processing in children with language disorders. Motivated by these results, in the current study our first aim was to explore whether a short exposure to a regular musical rhythm (i.e., rhythmic priming) can improve subsequent grammatical processing in preschool‐aged Hungarian‐speaking children with and without developmental language disorder (DLD). Second, we investigated whether rhythmic priming is specific to grammar processing by assessing priming in two additional domains: a linguistic but non‐grammatical task (picture naming) and a non‐linguistic task (nonverbal Stroop task). Third, to confirm that the rhythmic priming effect originates from the facilitating effect of the regular rhythm and not the negative effect of the control condition, we added a third condition, silence, for all the three tasks. Both groups of children showed better performance on the grammaticality judgment task in the regular compared to both the irregular and the silent conditions but no such effect appeared in the non‐grammatical and non‐linguistic tasks. These results suggest that (1) rhythmic priming can improve grammatical processing in Hungarian, a language with complex morphosyntax, both in children with and without DLD, (2) the effect is specific to grammar and (3) is a result of the facilitating effect of the regular rhythm. These results could motivate further research about integrating rhythmic priming into traditional speech‐language therapy. A video abstract of this article can be viewed at https://youtu.be/zKzGuIjZyvU.

What you hear first, is what you get: Initial metrical cue presentation modulates syllable detection in sentence processing

Auditory rhythms create powerful expectations for the listener. Rhythmic cues with the same temporal structure as subsequent sentences enhance processing compared with irregular or mismatched cues. In the present study, we focus on syllable detection following matched rhythmic cues. Cues were aligned with subsequent sentences at the syllable (low-level cue) or the accented syllable (high-level cue) level. A different group of participants performed the task without cues to provide a baseline. We hypothesized that unaccented syllable detection would be faster after low-level cues, and accented syllable detection would be faster after high-level cues. There was no difference in syllable detection depending on whether the sentence was preceded by a high-level or low-level cue. However, the results revealed a priming effect of the cue that participants heard first. Participants who heard a high-level cue first were faster to detect accented than unaccented syllables, and faster to detect accented syllables than participants who heard a low-level cue first. The low-level-first participants showed no difference between detection of accented and unaccented syllables. The baseline experiment confirmed that hearing a low-level cue first removed the benefit of the high-level grouping structure for accented syllables. These results suggest that the initially perceived rhythmic structure influenced subsequent cue perception and its influence on syllable detection. Results are discussed in terms of dynamic attending, temporal context effects, and implications for context effects in neural entrainment.

Cortical Tracking of Speech: Toward Collaboration between the Fields of Signal and Sentence Processing

In recent years, a growing number of studies have used cortical tracking methods to investigate auditory language processing. Although most studies that employ cortical tracking stem from the field of auditory signal processing, this approach should also be of interest to psycholinguistics-particularly the subfield of sentence processing-given its potential to provide insight into dynamic language comprehension processes. However, there has been limited collaboration between these fields, which we suggest is partly because of differences in theoretical background and methodological constraints, some mutually exclusive. In this paper, we first review the theories and methodological constraints that have historically been prioritized in each field and provide concrete examples of how some of these constraints may be reconciled. We then elaborate on how further collaboration between the two fields could be mutually beneficial. Specifically, we argue that the use of cortical tracking methods may help resolve long-standing debates in the field of sentence processing that commonly used behavioral and neural measures (e.g., ERPs) have failed to adjudicate. Similarly, signal processing researchers who use cortical tracking may be able to reduce noise in the neural data and broaden the impact of their results by controlling for linguistic features of their stimuli and by using simple comprehension tasks. Overall, we argue that a balance between the methodological constraints of the two fields will lead to an overall improved understanding of language processing as well as greater clarity on what mechanisms cortical tracking of speech reflects. Increased collaboration will help resolve debates in both fields and will lead to new and exciting avenues for research.

Speaking rhythmically can shape hearing

Evidence suggests that temporal predictions arising from the motor system can enhance auditory perception. However, in speech perception, we lack evidence of perception being modulated by production. Here we show a behavioural protocol that captures the existence of such auditory-motor interactions. Participants performed a syllable discrimination task immediately after producing periodic syllable sequences. Two speech rates were explored: a 'natural' (individually preferred) and a fixed 'non-natural' (2 Hz) rate. Using a decoding approach, we show that perceptual performance is modulated by the stimulus phase determined by a participant's own motor rhythm. Remarkably, for 'natural' and 'non-natural' rates, this finding is restricted to a subgroup of the population with quantifiable auditory-motor coupling. The observed pattern is compatible with a neural model assuming a bidirectional interaction of auditory and speech motor cortices. Crucially, the model matches the experimental results only if it incorporates individual differences in the strength of the auditory-motor connection.

Rhythmic Abilities of Children With Hearing Loss

OBJECTIVES

Children with hearing loss (HL), in spite of early cochlear implantation, often struggle considerably with language acquisition. Previous research has shown a benefit of rhythmic training on linguistic skills in children with HL, suggesting that improving rhythmic capacities could help attenuating language difficulties. However, little is known about general rhythmic skills of children with HL and how they relate to speech perception. The aim of this study is twofold: (1) to assess the abilities of children with HL in different rhythmic sensorimotor synchronization tasks compared to a normal-hearing control group and (2) to investigate a possible relation between sensorimotor synchronization abilities and speech perception abilities in children with HL.

DESIGN

A battery of sensorimotor synchronization tests with stimuli of varying acoustic and temporal complexity was used: a metronome, different musical excerpts, and complex rhythmic patterns. Synchronization abilities were assessed in 32 children (aged from 5 to 10 years) with a severe to profound HL mainly fitted with one or two cochlear implants (n = 28) or with hearing aids (n = 4). Working memory and sentence repetition abilities were also assessed. Performance was compared to an age-matched control group of 24 children with normal hearing. The comparison took into account variability in working memory capacities. For children with HL only, we computed linear regressions on speech, sensorimotor synchronization, and working memory abilities, including device-related variables such as onset of device use, type of device, and duration of use.

RESULTS

Compared to the normal-hearing group, children with HL performed poorly in all sensorimotor synchronization tasks, but the effect size was greater for complex as compared to simple stimuli. Group differences in working memory did not explain this result. Linear regression analysis revealed that working memory, synchronization to complex rhythms performances, age, and duration of device use predicted the number of correct syllables produced in a sentence repetition task.

CONCLUSION

Despite early cochlear implantation or hearing aid use, hearing impairment affects the quality of temporal processing of acoustic stimuli in congenitally deaf children. This deficit seems to be more severe with stimuli of increasing rhythmic complexity highlighting a difficulty in structuring sounds according to a temporal hierarchy.

The processing of rhythmic structures in music and prosody by children with developmental dyslexia and developmental language disorder

Rhythm perception seems to be crucial to language development. Many studies have shown that children with developmental dyslexia and developmental language disorder have difficulties in processing rhythmic structures. In this study, we investigated the relationships between prosody and musical processing in Italian children with typical and atypical development. The tasks aimed to reproduce linguistic prosodic structures through musical sequences, offering a direct comparison between the two domains without violating the specificities of each one. About 16 Typically Developing children, 16 children with a diagnosis of Developmental Dyslexia, and 16 with a diagnosis of developmental language disorder (age 10-13 years) participated in the experimental study. Three tasks were administered: an association task between a sentence and its humming version, a stress discrimination task (between couples of sounds reproducing the intonation of Italian trisyllabic words), and an association task between trisyllabic nonwords with different stress position and three-notes musical sequences with different musical stress. Children with developmental language disorder perform significantly lower than Typically Developing children on the humming test. By contrast, children with developmental dyslexia are significantly slower than TD in associating nonwords with musical sequences. Accuracy and speed in the experimental tests correlate with metaphonological, language, and word reading scores. Theoretical and clinical implications are discussed within a multidimensional model of neurodevelopmental disorders including prosodic and rhythmic skills at word and sentence level.

Music Training, Working Memory, and Neural Oscillations: A Review

This review focuses on reports that link music training to working memory and neural oscillations. Music training is increasingly associated with improvement in working memory, which is strongly related to both localized and distributed patterns of neural oscillations. Importantly, there is a small but growing number of reports of relationships between music training, working memory, and neural oscillations in adults. Taken together, these studies make important contributions to our understanding of the neural mechanisms that support effects of music training on behavioral measures of executive functions. In addition, they reveal gaps in our knowledge that hold promise for further investigation. The current review is divided into the main sections that follow: (1) discussion of behavioral measures of working memory, and effects of music training on working memory in adults; (2) relationships between music training and neural oscillations during temporal stages of working memory; (3) relationships between music training and working memory in children; (4) relationships between music training and working memory in older adults; and (5) effects of entrainment of neural oscillations on cognitive processing. We conclude that the study of neural oscillations is proving useful in elucidating the neural mechanisms of relationships between music training and the temporal stages of working memory. Moreover, a lifespan approach to these studies will likely reveal strategies to improve and maintain executive function during development and aging.

A stimulus-brain coupling analysis of regular and irregular rhythms in adults with dyslexia and controls

When listening to temporally regular rhythms, most people are able to extract the beat. Evidence suggests that the neural mechanism underlying this ability is the phase alignment of endogenous oscillations to the external stimulus, allowing for the prediction of upcoming events (i.e., dynamic attending). Relatedly, individuals with dyslexia may have deficits in the entrainment of neural oscillations to external stimuli, especially at low frequencies. The current experiment investigated rhythmic processing in adults with dyslexia and matched controls. Regular and irregular rhythms were presented to participants while electroencephalography was recorded. Regular rhythms contained the beat at 2 Hz; while acoustic energy was maximal at 4 Hz and 8 Hz. These stimuli allowed us to investigate whether the brain responds non-linearly to the beat-level of a rhythmic stimulus, and whether beat-based processing differs between dyslexic and control participants. Both groups showed enhanced stimulus-brain coherence for regular compared to irregular rhythms at the frequencies of interest, with an overrepresentation of the beat-level in the brain compared to the acoustic signal. In addition, we found evidence that controls extracted subtle temporal regularities from irregular stimuli, whereas dyslexics did not. Findings are discussed in relation to dynamic attending theory and rhythmic processing deficits in dyslexia.

Do Patients With Neurogenic Speech Sound Impairments Benefit From Auditory Priming With a Regular Metrical Pattern?

Purpose Earlier investigations based on word and sentence repetition tasks had revealed that the most prevalent metrical pattern in German (the trochee)-unlike the iambic pattern-facilitates articulation in patients with apraxia of speech (AOS; e.g., Aichert, Späth, & Ziegler, 2016), confirming that segmental and prosodic aspects of speech production interact. In this study, we investigated if articulation in apraxic speakers also benefits from auditory priming by speech with a regular rhythm. Furthermore, we asked if the advantage of regular speech rhythm, if present, is confined to impairments at the motor planning stage of speech production (i.e., AOS) or if it also applies to phonological encoding impairments. Method Twelve patients with AOS, 12 aphasic patients with postlexical phonological impairment (PI), and 36 neurologically healthy speakers were examined. A sequential synchronization paradigm based on a sentence completion task was conducted in conditions where we independently varied the metrical regularity of the prime sentence (regular vs. irregular prime sentence) and the metrical regularity of the target word (trochaic vs. iambic). Results Our data confirmed the facilitating effect of regular (trochaic) word stress on speech accuracy in patients with AOS (target effect). This effect could, for the first time, also be demonstrated in individuals with PI. Moreover, the study also revealed an influence of the metrical regularity of speech input in both patient groups (prime effect). Conclusions Patients with AOS and patients with PI exploited rhythmic cues in the speech of a model speaker for the initiation and the segmental realization of words. There seems to be a robust metrical influence on speech at both the phonological and the phonetic planning stages of speech production.

Prolonged exposure to highly rhythmic music affects brain dynamics and perception

Rhythmic stimulation is a powerful tool to improve temporal prediction and parsing of the auditory signal. However, for long duration of stimulation, the rhythmic and repetitive aspects of music have often been associated to a trance state. In this study we conceived an auditory monitoring task that allows tracking changes of psychophysical auditory thresholds. Participants performed the task while listening to rhythmically regular and an irregular (scrambled but spectrally identical) music that were presented with an intermittent (short) and continuous (long) type of stimulation. Results show that psychophysical auditory thresholds increase following a Continuous versus Intermittent stimulation and this is accompanied by a reduction of the amplitude of two event-related potentials to target stimuli. These effects are larger with regular music, thus do not simply derive from the duration of stimulation. Interestingly, they seem to be related to a frequency selective neural coupling as well as an increase of network connectivity in the alpha band between frontal and central regions. Our study shows that the idea that rhythmic presentation of sensory stimuli facilitates perception might be limited to short streams, while long, highly regular, repetitive and strongly engaging streams may have an opposite perceptual impact.

Pushing the Envelope: Developments in Neural Entrainment to Speech and the Biological Underpinnings of Prosody Perception

Prosodic cues in speech are indispensable for comprehending a speaker’s message, recognizing emphasis and emotion, parsing segmental units, and disambiguating syntactic structures. While it is commonly accepted that prosody provides a fundamental service to higher-level features of speech, the neural underpinnings of prosody processing are not clearly defined in the cognitive neuroscience literature. Many recent electrophysiological studies have examined speech comprehension by measuring neural entrainment to the speech amplitude envelope, using a variety of methods including phase-locking algorithms and stimulus reconstruction. Here we review recent evidence for neural tracking of the speech envelope and demonstrate the importance of prosodic contributions to the neural tracking of speech. Prosodic cues may offer a foundation for supporting neural synchronization to the speech envelope, which scaffolds linguistic processing. We argue that prosody has an inherent role in speech perception, and future research should fill the gap in our knowledge of how prosody contributes to speech envelope entrainment.

Cross-Modal Priming Effect of Rhythm on Visual Word Recognition and Its Relationships to Music Aptitude and Reading Achievement

Recent evidence suggests the existence of shared neural resources for rhythm processing in language and music. Such overlaps could be the basis of the facilitating effect of regular musical rhythm on spoken word processing previously reported for typical children and adults, as well as adults with Parkinson’s disease and children with developmental language disorders. The present study builds upon these previous findings by examining whether non-linguistic rhythmic priming also influences visual word processing, and the extent to which such cross-modal priming effect of rhythm is related to individual differences in musical aptitude and reading skills. An electroencephalogram (EEG) was recorded while participants listened to a rhythmic tone prime, followed by a visual target word with a stress pattern that either matched or mismatched the rhythmic structure of the auditory prime. Participants were also administered standardized assessments of musical aptitude and reading achievement. Event-related potentials (ERPs) elicited by target words with a mismatching stress pattern showed an increased fronto-central negativity. Additionally, the size of the negative effect correlated with individual differences in musical rhythm aptitude and reading comprehension skills. Results support the existence of shared neurocognitive resources for linguistic and musical rhythm processing, and have important implications for the use of rhythm-based activities for reading interventions.

Neural Entrainment Determines the Words We Hear

Low-frequency neural entrainment to rhythmic input has been hypothesized as a canonical mechanism that shapes sensory perception in time. Neural entrainment is deemed particularly relevant for speech analysis, as it would contribute to the extraction of discrete linguistic elements from continuous acoustic signals. However, its causal influence in speech perception has been difficult to establish. Here, we provide evidence that oscillations build temporal predictions about the duration of speech tokens that affect perception. Using magnetoencephalography (MEG), we studied neural dynamics during listening to sentences that changed in speech rate. We observed neural entrainment to preceding speech rhythms persisting for several cycles after the change in rate. The sustained entrainment was associated with changes in the perceived duration of the last word's vowel, resulting in the perception of words with different meanings. These findings support oscillatory models of speech processing, suggesting that neural oscillations actively shape speech perception.

New evidence of a rhythmic priming effect that enhances grammaticality judgments in children

Musical rhythm and the grammatical structure of language share a surprising number of characteristics that may be intrinsically related in child development. The current study aimed to understand the potential influence of musical rhythmic priming on subsequent spoken grammar task performance in children with typical development who were native speakers of English. Participants (ages 5-8 years) listened to rhythmically regular and irregular musical sequences (within-participants design) followed by blocks of grammatically correct and incorrect sentences upon which they were asked to perform a grammaticality judgment task. Rhythmically regular musical sequences improved performance in grammaticality judgment compared with rhythmically irregular musical sequences. No such effect of rhythmic priming was found in two nonlinguistic control tasks, suggesting a neural overlap between rhythm processing and mechanisms recruited during grammar processing. These findings build on previous research investigating the effect of rhythmic priming by extending the paradigm to a different language, testing a younger population, and employing nonlanguage control tasks. These findings of an immediate influence of rhythm on grammar states (temporarily augmented grammaticality judgment performance) also converge with previous findings of associations between rhythm and grammar traits (stable generalized grammar abilities) in children. Taken together, the results of this study provide additional evidence for shared neural processing for language and music and warrant future investigations of potentially beneficial effects of innovative musical material on language processing.

Auditory-Motor Rhythms and Speech Processing in French and German Listeners

Moving to a speech rhythm can enhance verbal processing in the listener by increasing temporal expectancies (Falk and Dalla Bella, 2016). Here we tested whether this hypothesis holds for prosodically diverse languages such as German (a lexical stress-language) and French (a non-stress language). Moreover, we examined the relation between motor performance and the benefits for verbal processing as a function of language. Sixty-four participants, 32 German and 32 French native speakers detected subtle word changes in accented positions in metrically structured sentences to which they previously tapped with their index finger. Before each sentence, they were cued by a metronome to tap either congruently (i.e., to accented syllables) or incongruently (i.e., to non-accented parts) to the following speech stimulus. Both French and German speakers detected words better when cued to tap congruently compared to incongruent tapping. Detection performance was predicted by participants' motor performance in the non-verbal cueing phase. Moreover, tapping rate while participants tapped to speech predicted detection differently for the two language groups, in particular in the incongruent tapping condition. We discuss our findings in light of the rhythmic differences of both languages and with respect to recent theories of expectancy-driven and multisensory speech processing.