Temporal modulations in speech and music

Enhanced Delta Band Neural Tracking of Degraded Fundamental Frequency Speech in Noisy Environments

Pitch variation of the fundamental frequency (F0) is critical to speech understanding, especially in noisy environments. Degrading the F0 contour reduces behaviorally measured speech intelligibility, posing greater challenges for tonal languages like Mandarin Chinese where the F0 pattern determines semantic meaning. However, neural tracking of Mandarin speech with degraded F0 information in noisy environments remains unclear. This study investigated neural envelope tracking of continuous Mandarin speech with three F0-flattening levels (original, flat-tone, and flat-all) under various signal-to-noise ratios (0, -9, and -12 dB). F0 contours were flattened at the word level for flat-tone and at the sentence level for flat-all Mandarin speech. Electroencephalography responses were indexed by the temporal response function in the delta (<4 Hz) and theta (4-8 Hz) frequency bands. Results show that delta-band envelope tracking is modulated by the degree of F0 flattening in a nonmonotonic manner. Notably, flat-tone Mandarin speech elicited the strongest envelope tracking compared with both original and flat-all speech, despite reduced F0 information. In contrast, the theta band, which primarily encodes speech signal-to-noise level, was not affected by F0 changes. In addition, listeners with better pitch-related music skills exhibited more efficient neural envelope speech tracking, despite being musically naive. These findings indicate that neural envelope tracking in the delta (but not theta) band is highly specific to F0 pitch variation and highlight the role of intrinsic musical skills for speech-in-noise benefits.

Extended High-Frequency Hearing Loss and Suprathreshold Auditory Processing: The Moderating Role of Auditory Working Memory

OBJECTIVES

Natural sounds, including speech, contain temporal fluctuations, and hearing loss influences the coding of these temporal features. However, how subclinical hearing loss may influence temporal variations remains unclear. In listeners with normal audiograms, hearing loss above 8 kHz is indicative of basal cochlear damage and may signal the onset of cochlear dysfunction. This study examined a conceptual framework to investigate the relationship between extended high-frequency hearing and suprathreshold auditory processing, particularly focusing on how cognitive factors, such as working memory, moderate these interactions.

DESIGN

Frequency modulation difference limens to slow (2 Hz) and fast (20 Hz) modulations, backward masking thresholds, and digit span measures were obtained in 44 normal-hearing listeners with varying degrees of extended high-frequency hearing thresholds.

RESULTS

Extended high-frequency hearing thresholds alone were not directly associated with frequency modulation difference limens or backward masking thresholds. However, working memory capacity-particularly as measured by the backward digit span-moderated the relationship between extended high-frequency thresholds and suprathreshold auditory performance. Among individuals with lower working memory capacity, elevated extended high-frequency thresholds were associated with reduced sensitivity to fast-rate frequency modulations and higher backward masking thresholds. It is important to note that this moderating effect was task-specific, as it was not observed for slow-rate modulations.

CONCLUSIONS

The impact of elevated extended high-frequency thresholds on suprathreshold auditory processing is influenced by working memory capacity. Individuals with reduced cognitive capacity are particularly vulnerable to the perceptual effects of subclinical cochlear damage. This suggests that cognitive resources act as a compensatory mechanism, helping to mitigate the effects of subclinical deficits, especially in tasks that are temporally challenging.

Rhythm training improves word-reading in children with dyslexia

Specific learning disorder with reading deficit (SLD-reading), or dyslexia, is one of the most common neurodevelopmental disorders. Intensive training with reading specialists is recommended, but delayed access to care is common. In this study, we tested an innovative user-friendly and accessible intervention, the medical device Mila-Learn, which is a video game based on cognitive and rhythmic training to improve phonological and reading ability. This randomized placebo-controlled trial (ClinicalTrials.gov NCT05154721) included children aged 7 to 11 y/o with SLD-reading. The children were in 2nd to 5th grade and had been in school for more than 3 years. The exclusion criteria were reading or writing remediation during the past 12 months, previous use of Mila-Learn, and severe chronic illness. The patients, who were all native French speakers, were recruited from throughout France and were randomly assigned to Mila-Learn or a matched-placebo game for an 8-week training. The primary variable was nonword decoding. The secondary variables included phonological skills, 2-min word-reading accuracy and speed, grapheme-to-phoneme conversion skills, and self-esteem. Between September 2021 and April 2023, 151 children were assigned to Mila-Learn (n = 75; male = 36; female = 39) or the placebo (n = 76; male = 42; female = 34). We registered 39 adverse events; only one was due to the protocol and was in the placebo group. We found no differences between the groups in nonword decoding in the intention-to-treat (N = 151; p = 0.39) or per-protocol analysis (N = 93; p = 0.21). However, the per-protocol analysis revealed the superiority of Mila-Learn over the placebo by 5.05 score points (95% CI 0.21; 10.3, p < 0.05) for 2-minute word-reading accuracy and by 5.44 score points (95% CI 0.57; 10.99, p < 0.05) for 2-min word-reading speed. We found no other significant effects. Mila-Learn is safe for children with SLD-reading who might benefit from this medical device.Study registration: ClinicalTrials.gov NCT05154721-13/12/2021.

"What" and "When" Predictions Jointly Modulate Speech Processing

Adaptive behavior rests on predictions based on statistical regularities in the environment. Such regularities pertain to stimulus contents ("what") and timing ("when"), and both interactively modulate sensory processing. In speech streams, predictions can be formed at multiple hierarchical levels of contents (e.g., syllables vs words) and timing (faster vs slower time scales). Whether and how these hierarchies map onto each other remains unknown. Under one hypothesis, neural hierarchies may link "what" and "when" predictions within sensory processing areas: with lower versus higher cortical regions mediating interactions for smaller versus larger units (syllables vs words). Alternatively, interactions between "what" and "when" regularities might rest on a generic, sensory-independent mechanism. To address these questions, we manipulated "what" and "when" regularities at two levels-single syllables and disyllabic pseudowords-while recording neural activity using magnetoencephalography (MEG) in healthy volunteers (N = 22). We studied how neural responses to syllable and/or pseudoword deviants are modulated by "when" regularity. "When" regularity modulated "what" mismatch responses with hierarchical specificity, such that responses to deviant pseudowords (vs syllables) were amplified by temporal regularity at slower (vs faster) time scales. However, both these interactive effects were source-localized to the same regions, including frontal and parietal cortices. Effective connectivity analysis showed that the integration of "what" and "when" regularity selectively modulated connectivity within regions, consistent with gain effects. This suggests that the brain integrates "what" and "when" predictions that are congruent with respect to their hierarchical level, but this integration is mediated by a shared and distributed cortical network.

Anatomically distinct cortical tracking of music and speech by slow (1-8Hz) and fast (70-120Hz) oscillatory activity

Music and speech encode hierarchically organized structural complexity at the service of human expressiveness and communication. Previous research has shown that populations of neurons in auditory regions track the envelope of acoustic signals within the range of slow and fast oscillatory activity. However, the extent to which cortical tracking is influenced by the interplay between stimulus type, frequency band, and brain anatomy remains an open question. In this study, we reanalyzed intracranial recordings from thirty subjects implanted with electrocorticography (ECoG) grids in the left cerebral hemisphere, drawn from an existing open-access ECoG database. Participants passively watched a movie where visual scenes were accompanied by either music or speech stimuli. Cross-correlation between brain activity and the envelope of music and speech signals, along with density-based clustering analyses and linear mixed-effects modeling, revealed both anatomically overlapping and functionally distinct mapping of the tracking effect as a function of stimulus type and frequency band. We observed widespread left-hemisphere tracking of music and speech signals in the Slow Frequency Band (SFB, band-passed filtered low-frequency signal between 1-8Hz), with near zero temporal lags. In contrast, cortical tracking in the High Frequency Band (HFB, envelope of the 70-120Hz band-passed filtered signal) was higher during speech perception, was more densely concentrated in classical language processing areas, and showed a frontal-to-temporal gradient in lag values that was not observed during perception of musical stimuli. Our results highlight a complex interaction between cortical region and frequency band that shapes temporal dynamics during processing of naturalistic music and speech signals.

Evaluating the role of age on speech-in-noise perception based primarily on temporal envelope information

Acoustic amplitude modulation (AM) patterns carry important information, particularly in speech. AM masking, influenced by frequency selectivity in the modulation domain, is considered a crucial factor for speech intelligibility in noisy environments. Based on recent evidence suggesting an age-related decline in AM frequency selectivity, this study investigated whether increased AM masking in older listeners is associated with reduced speech intelligibility. Speech reception thresholds (SRTs) were measured using tone-vocoded speech and maskers with no modulation, broadband AM, or narrowband AM at varying modulation frequencies. AM masked thresholds were assessed for a 4-Hz target modulation frequency. The study included young (N = 14, 19-25 years) and older (N = 14, 57-79 years) listeners with normal hearing. It was hypothesized that SRTs would be higher for the older group with modulated maskers and that the age-related increase in SRT would depend on the masker's modulation frequency content. The speech intelligibility results showed that maskers with broadband AM produced higher SRTs than unmodulated maskers. However, SRTs varied little with masker-modulation center frequency across the range tested (2-32 Hz). While older listeners exhibited lower AM frequency selectivity than young listeners, they did not consistently exhibit higher SRTs than their young counterparts across maskers. However, there was a trend for the effect of age to be greater for maskers with broadband AM than for unmodulated maskers. Overall, despite supportive trends, the results do not conclusively demonstrate that older listeners are more susceptible than young listeners to AM masking of speech.

Linear phase property of speech envelope tracking response in Heschl's gyrus and superior temporal gyrus

Understanding how the brain tracks speech during listening remains a challenge. The phase resetting hypothesis proposes that the envelope-tracking response is generated by resetting the phase of intrinsic nonlinear neural oscillations, whereas the evoked response hypothesis proposes that the envelope-tracking response is the linear superposition of transient responses evoked by a sequence of acoustic events in speech. Recent studies have demonstrated a linear phase property of the envelope-tracking response, supporting the evoked response hypothesis. However, the cortical regions aligning with the evoked response hypothesis remain unclear. To address this question, we directly recorded from the cortex using stereo-electroencephalography (SEEG) in nineteen epilepsy patients as they listened to natural speech, and we investigated whether the phase lag between the speech envelope and neural activity linearly changes across frequency. We found that the linear phase property of low-frequency (LF) (.5-40 Hz) envelope tracking was widely observed in Heschl's gyrus (HG) and superior temporal gyrus (STG), with additional sparser distribution in insula, postcentral gyrus, and precentral gyrus. Furthermore, the latency of LF envelope-tracking responses derived from phase-frequency curve exhibited an increase gradient along HG and in the posterior-to-anterior direction in STG. Our findings suggest that auditory cortex can track speech envelope in line with the evoked response hypothesis.

Corollary discharge signals during production are domain general: An intracerebral EEG case study with a professional musician

As measured by event-related potentials, self-produced sounds elicit an overall reduced response in the auditory cortex compared to identical externally presented stimuli. This study examines this modulatory effect with high-precision recordings in naturalistic settings and explores whether it is domain-general across speech or music. Using stereotactic EEG with a professional musician undergoing presurgical epilepsy evaluation, we recorded auditory cortical activity during music and speech production and perception tasks. Compared to externally presented sounds, self-produced sounds induce modulation of activity in the auditory cortex which vary across frequency and spatial location but is consistent across cognitive domains (speech/music) and different stimuli. Self-produced music and speech were associated with widespread low-frequency (4-8 Hz) suppression, mid-frequency (8-80 Hz) enhancement, and decreased encoding of acoustic features. These findings reveal the domain-general nature of motor-driven corollary discharge modulatory signals and their frequency-specific effects in auditory regions.

Musical neurodynamics

A great deal of research in the neuroscience of music suggests that neural oscillations synchronize with musical stimuli. Although neural synchronization is a well-studied mechanism underpinning expectation, it has even more far-reaching implications for music. In this Perspective, we survey the literature on the neuroscience of music, including pitch, harmony, melody, tonality, rhythm, metre, groove and affect. We describe how fundamental dynamical principles based on known neural mechanisms can explain basic aspects of music perception and performance, as summarized in neural resonance theory. Building on principles such as resonance, stability, attunement and strong anticipation, we propose that people anticipate musical events not through predictive neural models, but because brain-body dynamics physically embody musical structure. The interaction of certain kinds of sounds with ongoing pattern-forming dynamics results in patterns of perception, action and coordination that we collectively experience as music. Statistically universal structures may have arisen in music because they correspond to stable states of complex, pattern-forming dynamical systems. This analysis of empirical findings from the perspective of neurodynamic principles sheds new light on the neuroscience of music and what makes music powerful.

Predictive coding and dimension-selective attention enhance the lateralization of spoken language processing

Hemispheric lateralization in speech and language processing exemplifies functional brain specialization. Seminal work in patients with left hemisphere damage highlighted the left-hemispheric dominance in language functions. However, speech processing is not confined to the left hemisphere. Hence, some researchers associate lateralization with auditory processing asymmetries: slow temporal and fine spectral acoustic information is preferentially processed in right auditory regions, while faster temporal information is primarily handled by left auditory regions. Other scholars posit that lateralization relates more to linguistic processing, particularly for speech and speech-like stimuli. We argue that these seemingly distinct accounts are interdependent. Linguistic analysis of speech relies on top-down processes, such as predictive coding and dimension-selective auditory attention, which enhance lateralized processing by engaging left-lateralized sensorimotor networks. Our review highlights that lateralization is weaker for simple sounds, stronger for speech-like sounds, and strongest for meaningful speech. Evidence shows that predictive speech processing and selective attention enhance lateralization. We illustrate that these top-down processes rely on left-lateralized sensorimotor networks and provide insights into the role of these networks in speech processing.

CoVox: A dataset of contrasting vocalizations

The human voice is remarkably versatile and can vary greatly in sound depending on how it is used. An increasing number of studies have addressed the differences and similarities between the singing and the speaking voice. However, finding adequate stimuli material that is at the same time controlled and ecologically valid is challenging, and most datasets lack variability in terms of vocal styles performed by the same voice. Here, we describe a curated stimulus set of vocalizations where 22 female singers performed the same melody excerpts in three contrasting singing styles (as a lullaby, as a pop song, and as an opera aria) and spoke the text aloud in two speaking styles (as if speaking to an adult or to an infant). All productions were made with the songs' original lyrics, in Brazilian Portuguese, and with a/lu/sound. This ecologically valid dataset of 1320 vocalizations was validated through a forced-choice lab experiment (N = 25 for each stimulus) where lay listeners could recognize the intended vocalization style with high accuracy (proportion of correct recognition superior to 69% for all styles). We also provide acoustic characterization of the stimuli, depicting clear and contrasting acoustic profiles depending on the style of vocalization. All recordings are made freely available under a Creative Commons license and can be downloaded at https://osf.io/cgexn/ .

From Lab to Concert Hall: Effects of Live Performance on Neural Entrainment and Engagement

Neural entrainment to acoustic rhythms underlies intelligibility in speech as well as sensorimotor responses to music. This property of neural dynamics, where cortical oscillations align in phase and frequency with a periodic stimulus, is well-studied in the context of sensory encoding and perception. However, little is known about how affective components in naturalistic music influence neural entrainment. The present study investigates the effect of live versus recorded music on neural entrainment and tracking using phase-based and linear modeling approaches. 21 participants listened to 2 live and 2 recorded performances of fast and slow movements of solo violin while their EEG data were collected with a mobile system. Participants made behavioral ratings of engagement, spontaneity, pleasure, investment, focus, and distraction after each trial. Live performances were rated as more engaging, pleasurable, and spontaneous than recorded performances. Live trials showed significantly higher acoustic-EEG phase-locking than recorded trials in the frequency range associated with the note-rate of the fast excerpts. Furthermore, the effect of liveness on phase-locking was strongest in individuals who reported the greatest increases in pleasure and engagement for live over recorded trials. Finally, forward linear mapping revealed stronger neural tracking of spectral over amplitude-related acoustic features and a sensitivity to tempo in neural tracking. Altogether, results suggest that experiencing music live strengthens cerebro-acoustic relationships by enhancing rhythmically-driven neural entrainment alongside perceived pleasure and engagement.

Significance Statement

Neural oscillations entrain to rhythms in naturalistic acoustic stimuli, including speech and music. The rhythmic structure of music impacts the timescale of neural entrainment and facilitates the pleasurable urge to move with music, but less is known about how the live experience of music affects neural entrainment. Here, we measure phase-locking and neural tracking between listeners' EEG activity and naturalistic acoustics during live and recorded solo violin performances, demonstrating that neural-acoustic interactions are driven by musical rhythms and strengthened by the perception of liveness. Together, the study provides insight into neural mechanisms underlying the pleasure of live music, suggesting that the social and affective experience of liveness alters the strength of neural entrainment.

Moving rhythmically can facilitate naturalistic speech perception in a noisy environment

The motor system is known to process temporal information, and moving rhythmically while listening to a melody can improve auditory processing. In three interrelated behavioural experiments, we demonstrate that this effect translates to speech processing. Motor priming improves the efficiency of subsequent naturalistic speech-in-noise processing under specific conditions. (i) Moving rhythmically at the lexical rate (~1.8 Hz) significantly improves subsequent speech processing compared to moving at other rates, such as the phrasal or syllabic rates. (ii) The impact of such rhythmic motor priming is not influenced by whether it is self-generated or triggered by an auditory beat. (iii) Overt lexical vocalization, regardless of its semantic content, also enhances the efficiency of subsequent speech processing. These findings provide evidence for the functional role of the motor system in processing the temporal dynamics of naturalistic speech.

Adaptation to sentences and melodies when making judgments along a voice-nonvoice continuum

Adaptation to constant or repetitive sensory signals serves to improve detection of novel events in the environment and to encode incoming information more efficiently. Within the auditory modality, contrastive adaptation effects have been observed within a number of categories, including voice and musical instrument type. A recent study found contrastive perceptual shifts between voice and instrument categories following repetitive presentation of adaptors consisting of either vowels or instrument tones. The current study tested the generalizability of adaptation along a voice-instrument continuum, using more ecologically valid adaptors. Participants were presented with an adaptor followed by an ambiguous voice-instrument target, created by generating a 10-step morphed continuum between pairs of vowel and instrument sounds. Listeners' categorization of the target sounds was shifted contrastively by a spoken sentence or instrumental melody adaptor, regardless of whether the adaptor and the target shared the same speaker gender or similar pitch range (Experiment 1). However, no significant contrastive adaptation was observed when nonspeech vocalizations or nonpitched percussion sounds were used as the adaptors (Experiment 2). The results suggest that adaptation between voice and nonvoice categories does not rely on exact repetition of simple stimuli, nor does it solely reflect the result of a sound being categorized as being human or nonhuman sourced. The outcomes suggest future directions for determining the precise spectro-temporal properties of sounds that induce these voice-instrument contrastive adaptation effects.

Cortical and behavioral tracking of rhythm in music: Effects of pitch predictability, enjoyment, and expertise

The cortical tracking of stimulus features is a crucial neural requisite of how we process continuous music. We here tested whether cortical tracking of the beat, typically related to rhythm processing, is modulated by pitch predictability and other top-down factors. Participants listened to tonal (high pitch predictability) and atonal (low pitch predictability) music while undergoing electroencephalography. We analyzed their cortical tracking of the acoustic envelope. Cortical envelope tracking was stronger while listening to atonal music, potentially reflecting listeners' violated pitch expectations and increased attention allocation. Envelope tracking was also stronger with more expertise and enjoyment. Furthermore, we showed cortical tracking of pitch surprisal (using IDyOM), which suggests that listeners' expectations match those computed by the IDyOM model, with higher surprisal for atonal music. Behaviorally, we measured participants' ability to finger-tap to the beat of tonal and atonal sequences in two experiments. Finger-tapping performance was better in the tonal condition, indicating a positive effect of pitch predictability on behavioral rhythm processing. Cortical envelope tracking predicted tapping performance for tonal music, as did pitch-surprisal tracking for atonal music, indicating that high and low predictability might impose different processing regimes. Taken together, our results show various ways that top-down factors impact musical rhythm processing.

Temporal Structure of Music Improves the Cortical Encoding of Speech

Long- and short-term musical training has been proposed to improve the efficiency of cortical tracking of speech, which refers to the synchronization of brain oscillations and the acoustic temporal structure of external stimuli. Here, we study how musical sequences with different rhythm structures can guide the temporal dynamics of auditory oscillations synchronized with the speech envelope. For this purpose, we investigated the effects of prior exposure to rhythmically structured musical sequences on cortical tracking of speech in Basque-Spanish bilingual adults (Experiment 1; N = 33, 22 female, Mean age = 25 years). We presented participants with sentences in Basque and Spanish preceded by musical sequences that differed in their rhythmical structure. The rhythmical structure of the musical sequences was created to (1) reflect and match the syllabic structure of the sentences, (2) reflect a regular rhythm but not match the syllabic structure of the sentences, and (3) follow an irregular rhythm. Participants' brain responses were recorded using electroencephalography, and speech-brain coherence in the delta and theta bands was calculated. Results showed stronger speech-brain coherence in the delta band in the first condition, but only for Spanish stimuli. A follow-up experiment including a subset of the initial sample (Experiment 2; N = 20) was conducted to investigate whether language-specific stimuli properties influenced the Basque results. Similar to Experiment 1, we found stronger speech-brain coherence in the delta and theta bands when the sentences were preceded by musical sequences that matched their syllabic structure. These results suggest that not only the regularity in music is crucial for influencing cortical tracking of speech, but so is adjusting this regularity to optimally reflect the rhythmic characteristics of listeners' native language(s). Despite finding some language-specific differences across frequencies, we showed that rhythm, inherent in musical signals, guides the adaptation of brain oscillations, by adapting the temporal dynamics of the oscillatory activity to the rhythmic scaffolding of the musical signal.

Best Cochlear Locations for Delivering Interaural Timing Cues in Electric Hearing

Growing numbers of children and adults who are deaf are eligible to receive cochlear implants (CI), which provide access to everyday sound. CIs in both ears (bilateral CIs or BiCIs) are becoming standard of care in many countries. However, their effectiveness is limited because they do not adequately restore the acoustic cues essential for sound localization, particularly interaural time differences (ITDs) at low frequencies. The cochlea, the auditory sensory organ, typically transmits ITDs more effectively at the apical region, which is specifically "tuned" to low frequencies. We hypothesized that effective restoration of robust ITD perception through electrical stimulation with BiCIs depends on targeting cochlear locations that transmit information most effectively. Importantly, we show that these locations can occur anywhere along the cochlea, even on the opposite end of the frequency map from where ITD cues are most dominantly encoded in an acoustic hearing system.

Complexity in speech and music listening via neural manifold flows

Understanding the complex neural mechanisms underlying speech and music perception remains a multifaceted challenge. In this study, we investigated neural dynamics using human intracranial recordings. Employing a novel approach based on low-dimensional reduction techniques, the Manifold Density Flow (MDF), we quantified the complexity of brain dynamics during naturalistic speech and music listening and during resting state. Our results reveal higher complexity in patterns of interdependence between different brain regions during speech and music listening compared with rest, suggesting that the cognitive demands of speech and music listening drive the brain dynamics toward states not observed during rest. Moreover, speech listening has more complexity than music, highlighting the nuanced differences in cognitive demands between these two auditory domains. Additionally, we validated the efficacy of the MDF method through experimentation on a toy model and compared its effectiveness in capturing the complexity of brain dynamics induced by cognitive tasks with another established technique in the literature. Overall, our findings provide a new method to quantify the complexity of brain activity by studying its temporal evolution on a low-dimensional manifold, suggesting insights that are invisible to traditional methodologies in the contexts of speech and music perception.

The Less Meaningful the Understanding, the Faster the Feeling: Speech Comprehension Changes Perceptual Speech Tempo

The perception of speech tempo is influenced by both the acoustic properties of speech and the cognitive state of the listener. However, there is a lack of research on how speech comprehension affects the perception of speech tempo. This study aims to disentangle the impact of speech comprehension on the perception of speech tempo by manipulating linguistic structures and measuring perceptual speech tempo at explicit and implicit levels. Three experiments were conducted to explore these relationships. In Experiment 1, two explicit speech tasks revealed that listeners tend to overestimate the speech tempo of sentences with low comprehensibility, although this effect decreased with repeated exposure to the speech. Experiment 2, utilizing an implicit speech tempo task, replicated the main findings of Experiment 1. Furthermore, the results from the drift-diffusion model eliminated the possibility that participants' responses were based on the type of sentence. In Experiment 3, non-native Chinese speakers with varying levels of language proficiency completed the implicit speech rate task. The results showed that non-native Chinese speakers exhibited distinct behavioral patterns compared to native Chinese speakers, as they did not perceive differences in speech tempo between high and low comprehensibility conditions. These findings highlight the intricate relationship between the perception of speech tempo and the comprehensibility of processed speech.

Investigating the Neural Basis of the Loud-first Principle of the Iambic-Trochaic Law

The perception of rhythmic patterns is crucial for the recognition of words in spoken languages, yet it remains unclear how these patterns are represented in the brain. Here, we tested the hypothesis that rhythmic patterns are encoded by neural activity phase-locked to the temporal modulation of these patterns in the speech signal. To test this hypothesis, we analyzed EEGs evoked with long sequences of alternating syllables acoustically manipulated to be perceived as a series of different rhythmic groupings in English. We found that the magnitude of the EEG at the syllable and grouping rates of each sequence was significantly higher than the noise baseline, indicating that the neural parsing of syllables and rhythmic groupings operates at different timescales. Distributional differences between the scalp topographies associated with each timescale suggests a further mechanistic dissociation between the neural segmentation of syllables and groupings. In addition, we observed that the neural tracking of louder syllables, which in trochaic languages like English are associated with the beginning of rhythmic groupings, was more robust than the neural tracking of softer syllables. The results of further bootstrapping and brain-behavior analyses indicate that the perception of rhythmic patterns is modulated by the magnitude of grouping alternations in the neural signal. These findings suggest that the temporal coding of rhythmic patterns in stress-based languages like English is supported by temporal regularities that are linguistically relevant in the speech signal.

Measuring self-similarity in empirical signals to understand musical beat perception

Experiencing music often entails the perception of a periodic beat. Despite being a widespread phenomenon across cultures, the nature and neural underpinnings of beat perception remain largely unknown. In the last decade, there has been a growing interest in developing methods to probe these processes, particularly to measure the extent to which beat-related information is contained in behavioral and neural responses. Here, we propose a theoretical framework and practical implementation of an analytic approach to capture beat-related periodicity in empirical signals using frequency-tagging. We highlight its sensitivity in measuring the extent to which the periodicity of a perceived beat is represented in a range of continuous time-varying signals with minimal assumptions. We also discuss a limitation of this approach with respect to its specificity when restricted to measuring beat-related periodicity only from the magnitude spectrum of a signal and introduce a novel extension of the approach based on autocorrelation to overcome this issue. We test the new autocorrelation-based method using simulated signals and by re-analyzing previously published data and show how it can be used to process measurements of brain activity as captured with surface EEG in adults and infants in response to rhythmic inputs. Taken together, the theoretical framework and related methodological advances confirm and elaborate the frequency-tagging approach as a promising window into the processes underlying beat perception and, more generally, temporally coordinated behaviors.

Speech Recognition and Noise Adaptation in Realistic Noises

The recognition of isolated words in noise improves as words are delayed from the noise onset. This phenomenon, known as adaptation to noise, has been mostly investigated using synthetic noises. The aim here was to investigate whether adaptation occurs for realistic noises and to what extent it depends on the spectrum and level fluctuations of the noise. Forty-nine different realistic and synthetic noises were analyzed and classified according to how much they fluctuated in level over time and how much their spectra differed from the speech spectrum. Six representative noises were chosen that covered the observed range of level fluctuations and spectral differences but could still mask speech. For the six noises, speech reception thresholds (SRTs) were measured for natural and tone-vocoded words delayed 50 (early condition) and 800 ms (late condition) from the noise onset. Adaptation was calculated as the SRT improvement in the late relative to the early condition. Twenty-two adults with normal hearing participated in the experiments. For natural words, adaptation was small overall (mean = 0.5 dB) and similar across the six noises. For vocoded words, significant adaptation occurred for all six noises (mean = 1.3 dB) and was not statistically different across noises. For the tested noises, the amount of adaptation was independent of the spectrum and level fluctuations of the noise. The results suggest that adaptation in speech recognition can occur in realistic noisy environments.

Music perception and its correlation with auditory speech perception in pediatric Mandarin-speaking cochlear implant users

BACKGROUND

Cochlear implants (CI) help regain perception of sound for patients with sensorineural hearing loss. The ability to recognize music pitch may be crucial for recognizing and producing speech for Mandarin.

AIMS/OBJECTIVES

This study aims to search for possible influencing factors of music perception and correlations between music perception and auditory speech abilities among prelingually deaf pediatric Mandarin-speaking CI users.

MATERIAL AND METHODS

Music perception of 24 pediatric CI users and 12 normal hearing children was measured using the MuSIC test. Auditory speech perception of the 24 CI users was also measured and analyzed with their music perception results.

RESULTS

Pediatric CI users performed worse than normal hearing children in pitch, rhythm and melody discrimination tests (p < .05). Significant difference in pitch and melody discrimination tests between age at implantation <5 and >5 groups was found. There were significant correlations between perception of consonants, tones, and speech in a noisy environment and perception of music pitch and melody.

CONCLUSION AND SIGNIFICANCE

Prelingually deaf pediatric CI users who received implantation before the age of five perform better in music perception tests. Pediatric CI users with better music perception show better auditory speech perception of Mandarin.

Best Cochlear Locations for Delivering Interaural Timing Cues in Electric Hearing

Growing numbers of children and adults who are deaf are eligible to receive cochlear implants (CI), which provide access to everyday sound. CIs in both ears (bilateral CIs or BiCIs) are becoming standard of care in many countries. However, their effectiveness is limited because they do not adequately restore the acoustic cues essential for sound localization, particularly interaural time differences (ITDs) at low frequencies. The cochlea, the auditory sensory organ, typically transmits ITDs more effectively at the apical region, which is specifically "tuned" to low frequencies. We hypothesized that effective restoration of robust ITD perception through electrical stimulation with BiCIs depends on targeting cochlear locations that transmit information most effectively. Importantly, we show that these locations can occur anywhere along the cochlea, even on the opposite end of the frequency map from where ITD cues are most dominantly encoded in an acoustic hearing system.

Physiological Entrainment: A Key Mind-Body Mechanism for Cognitive, Motor and Affective Functioning, and Well-Being

BACKGROUND

The human sensorimotor system can naturally synchronize with environmental rhythms, such as light pulses or sound beats. Several studies showed that different styles and tempos of music, or other rhythmic stimuli, have an impact on physiological rhythms, including electrocortical brain activity, heart rate, and motor coordination. Such synchronization, also known as the "entrainment effect", has been identified as a crucial mechanism impacting cognitive, motor, and affective functioning.

OBJECTIVES

This review examines theoretical and empirical contributions to the literature on entrainment, with a particular focus on the physiological mechanisms underlying this phenomenon and its role in cognitive, motor, and affective functions. We also address the inconsistent terminology used in the literature and evaluate the range of measurement approaches used to assess entrainment phenomena. Finally, we propose a definition of "physiological entrainment" that emphasizes its role as a fundamental mechanism that encompasses rhythmic interactions between the body and its environment, to support information processing across bodily systems and to sustain adaptive motor responses.

METHODS

We reviewed the recent literature through the lens of the "embodied cognition" framework, offering a unified perspective on the phenomenon of physiological entrainment.

RESULTS

Evidence from the current literature suggests that physiological entrainment produces measurable effects, especially on neural oscillations, heart rate variability, and motor synchronization. Eventually, such physiological changes can impact cognitive processing, affective functioning, and motor coordination.

CONCLUSIONS

Physiological entrainment emerges as a fundamental mechanism underlying the mind-body connection. Entrainment-based interventions may be used to promote well-being by enhancing cognitive, motor, and affective functions, suggesting potential rehabilitative approaches to enhancing mental health.

The human auditory cortex concurrently tracks syllabic and phonemic timescales via acoustic spectral flux

Dynamical theories of speech processing propose that the auditory cortex parses acoustic information in parallel at the syllabic and phonemic timescales. We developed a paradigm to independently manipulate both linguistic timescales, and acquired intracranial recordings from 11 patients who are epileptic listening to French sentences. Our results indicate that (i) syllabic and phonemic timescales are both reflected in the acoustic spectral flux; (ii) during comprehension, the auditory cortex tracks the syllabic timescale in the theta range, while neural activity in the alpha-beta range phase locks to the phonemic timescale; (iii) these neural dynamics occur simultaneously and share a joint spatial location; (iv) the spectral flux embeds two timescales-in the theta and low-beta ranges-across 17 natural languages. These findings help us understand how the human brain extracts acoustic information from the continuous speech signal at multiple timescales simultaneously, a prerequisite for subsequent linguistic processing.

Unsupervised discovery of family specific vocal usage in the Mongolian gerbil

In nature, animal vocalizations can provide crucial information about identity, including kinship and hierarchy. However, lab-based vocal behavior is typically studied during brief interactions between animals with no prior social relationship, and under environmental conditions with limited ethological relevance. Here, we address this gap by establishing long-term acoustic recordings from Mongolian gerbil families, a core social group that uses an array of sonic and ultrasonic vocalizations. Three separate gerbil families were transferred to an enlarged environment and continuous 20-day audio recordings were obtained. Using a variational autoencoder (VAE) to quantify 583,237 vocalizations, we show that gerbils exhibit a more elaborate vocal repertoire than has been previously reported and that vocal repertoire usage differs significantly by family. By performing gaussian mixture model clustering on the VAE latent space, we show that families preferentially use characteristic sets of vocal clusters and that these usage preferences remain stable over weeks. Furthermore, gerbils displayed family-specific transitions between vocal clusters. Since gerbils live naturally as extended families in complex underground burrows that are adjacent to other families, these results suggest the presence of a vocal dialect which could be exploited by animals to represent kinship. These findings position the Mongolian gerbil as a compelling animal model to study the neural basis of vocal communication and demonstrates the potential for using unsupervised machine learning with uninterrupted acoustic recordings to gain insights into naturalistic animal behavior.

Neural Dynamics of the Processing of Speech Features: Evidence for a Progression of Features from Acoustic to Sentential Processing

When we listen to speech, our brain's neurophysiological responses "track" its acoustic features, but it is less well understood how these auditory responses are enhanced by linguistic content. Here, we recorded magnetoencephalography (MEG) responses while subjects listened to four types of continuous-speech-like passages: speech-envelope modulated noise, English-like non-words, scrambled words, and a narrative passage. Temporal response function (TRF) analysis provides strong neural evidence for the emergent features of speech processing in cortex, from acoustics to higher-level linguistics, as incremental steps in neural speech processing. Critically, we show a stepwise hierarchical progression of progressively higher order features over time, reflected in both bottom-up (early) and top-down (late) processing stages. Linguistically driven top-down mechanisms take the form of late N400-like responses, suggesting a central role of predictive coding mechanisms at multiple levels. As expected, the neural processing of lower-level acoustic feature responses is bilateral or right lateralized, with left lateralization emerging only for lexical-semantic features. Finally, our results identify potential neural markers, linguistic level late responses, derived from TRF components modulated by linguistic content, suggesting that these markers are indicative of speech comprehension rather than mere speech perception.

Can a human sing with an unseen artificial partner? Coordination dynamics when singing with an unseen human or artificial partner

This study investigated whether a singer's coordination patterns differ when singing with an unseen human partner versus an unseen artificial partner (VOCALOID 6 voice synthesis software). We used cross-correlation analysis to compare the correlation of the amplitude envelope time series between the partner's and the participant's singing voices. We also conducted a Granger causality test to determine whether the past amplitude envelope of the partner helps predict the future amplitude envelope of the participants, or if the reverse is true. We found more pronounced characteristics of anticipatory synchronization and increased similarity in the unfolding dynamics of the amplitude envelopes in the human-partner condition compared to the artificial-partner condition, despite the tempo fluctuations in the human-partner condition. The results suggested that subtle qualities of the human singing voice, possibly stemming from intrinsic dynamics of the human body, may contain information that enables human agents to align their singing behavior dynamics with a human partner.

Hearing and cognitive decline in aging differentially impact neural tracking of context-supported versus random speech across linguistic timescales

Cognitive decline and hearing loss are common in older adults and often co-occur while investigated separately, affecting the neural processing of speech. This study investigated the interaction between cognitive decline, hearing loss, and contextual cues in speech processing. Participants aged 60 years and older were assessed for cognitive decline using the Montreal Cognitive Assessment and for hearing ability using a four-frequency pure tone average. They listened to in-house-designed matrix-style sentences that either provided supportive context or were random, while we recorded their electroencephalography. Neurophysiological responses were analyzed through auditory evoked potentials and speech tracking at different linguistic timescales (i.e., phrase, word, syllable and phoneme rate) using phase-locking values. The results showed that cognitive decline was associated with decreased response accuracy in a speech recognition task. Cognitive decline significantly impacted the P2 component of auditory evoked potentials, while hearing loss influenced speech tracking at the word and phoneme rates, but not at the phrase or syllable rates. Contextual cues enhanced speech tracking at the syllable rate. These findings suggest that cognitive decline and hearing loss differentially affect the neural mechanisms underlying speech processing, with contextual cues playing a significant role in enhancing syllable rate tracking. This study emphasises the importance of considering both cognitive and auditory factors when studying speech processing in older people and highlights the need for further research to investigate the interplay between cognitive decline, hearing loss and contextual cues in speech processing.

Concurrent processing of the prosodic hierarchy is supported by cortical entrainment and phase-amplitude coupling

Models of phonology posit a hierarchy of prosodic units that is relatively independent from syntactic structure, requiring its own parsing. It remains unexplored how this prosodic hierarchy is represented in the brain. We investigated this foundational question by means of an electroencephalography (EEG) study. Thirty young adults listened to German sentences containing manipulations at different levels of the prosodic hierarchy. Evaluating speech-to-brain cortical entrainment and phase-amplitude coupling revealed that prosody's hierarchical structure is maintained at the neural level during spoken language comprehension. The faithfulness of this tracking varied as a function of the hierarchy's degree of intactness as well as systematic interindividual differences in audio-motor synchronization abilities. The results underscore the role of complex oscillatory mechanisms in configuring the continuous and hierarchical nature of the speech signal and situate prosody as a structure indispensable from theoretical perspectives on spoken language comprehension in the brain.

The Attractiveness of Average Speech Rhythms: Revisiting the Average Effect From a Crosslinguistic Perspective

The current study investigates the average effect: the tendency for humans to appreciate an averaged (face, bird, wristwatch, car, and so on) over an individual instance. The effect holds across cultures, despite varying conceptualizations of attractiveness. While much research has been conducted on the average effect in visual perception, much less is known about the extent to which this effect applies to language and speech. This study investigates the attractiveness of average speech rhythms in Dutch and Mandarin Chinese, two typologically different languages. This was tested in a series of perception experiments in either language in which native listeners chose the most attractive one from a pair of acoustically manipulated rhythms. For each language, two experiments were carried out to control for the potential influence of the acoustic manipulation on the average effect. The results confirm the average effect in both languages, and they do not exclude individual variation in the listeners' perception of attractiveness. The outcomes provide a new crosslinguistic perspective and give rise to alternative explanations to the average effect.

Simultaneous EEG recording of cortical tracking of speech and movement kinematics

RATIONALE

Cortical activity is coupled with streams of sensory stimulation. The coupling with the temporal envelope of heard speech is known as the cortical tracking of speech (CTS), and that with movement kinematics is known as the corticokinematic coupling (CKC). Simultaneous measurement of both couplings is desirable in clinical settings, but it is unknown whether the inherent dual-tasking condition has an impact on CTS or CKC.

AIM

We aim to determine whether and how CTS and CKC levels are affected when recorded simultaneously.

METHODS

Twenty-three healthy young adults underwent 64-channel EEG recordings while listening to stories and while performing repetitive finger-tapping movements in 3 conditions: separately (audio- or tapping-only) or simultaneously (audio-tapping). CTS and CKC values were estimated using coherence analysis between each EEG signal and speech temporal envelope (CTS) or finger acceleration (CKC). CTS was also estimated as the reconstruction accuracy of a decoding model.

RESULTS

Across recordings, CTS assessed with reconstruction accuracy was significant in 85 % of the subjects at phrasal frequency (0.5 Hz) and in 68 % at syllabic frequencies (4-8 Hz), and CKC was significant in over 85 % of the subjects at movement frequency and its first harmonic. Comparing CTS and CKC values evaluated in separate recordings to those in simultaneous recordings revealed no significant difference and moderate-to-high levels of correlation.

CONCLUSION

Despite the subtle behavioral effects, CTS and CKC are not evidently altered by the dual-task setting inherent to recording them simultaneously and can be evaluated simultaneously using EEG in clinical settings.

Cortical Tracking of Speech Is Reduced in Adults Who Stutter When Listening for Speaking

PURPOSE

The purpose of this study was to investigate cortical tracking of speech (CTS) in adults who stutter (AWS) compared to typically fluent adults (TFAs) to test the involvement of the speech-motor network in tracking rhythmic speech information.

METHOD

Participants' electroencephalogram was recorded while they simply listened to sentences (listening only) or completed them by naming a picture (listening for speaking), thus manipulating the upcoming involvement of speech production. We analyzed speech-brain coherence and brain connectivity during listening.

RESULTS

During the listening-for-speaking task, AWS exhibited reduced CTS in the 3- to 5-Hz range (theta), corresponding to the syllabic rhythm. The effect was localized in the left inferior parietal and right pre/supplementary motor regions. Connectivity analyses revealed that TFAs had stronger information transfer in the theta range in both tasks in fronto-temporo-parietal regions. When considering the whole sample of participants, increased connectivity from the right superior temporal cortex to the left sensorimotor cortex was correlated with faster naming times in the listening-for-speaking task.

CONCLUSIONS

Atypical speech-motor functioning in stuttering impacts speech perception, especially in situations requiring articulatory alertness. The involvement of frontal and (pre)motor regions in CTS in TFAs is highlighted. Further investigation is needed into speech perception in individuals with speech-motor deficits, especially when smooth transitioning between listening and speaking is required, such as in real-life conversational settings.

SUPPLEMENTAL MATERIAL

https://doi.org/10.23641/asha.27234885.

Entrainment of neural oscillations during language processing in Early-Stage schizophrenia

BACKGROUND

Impairments in language processing in schizophrenia (ScZ) are a central aspect of the disorder but the underlying pathophysiology mechanisms are unclear. In the current study, we tested the hypothesis that neural oscillations are impaired during speech tracking in early-stage ScZ and in participants at clinical high-risk for psychosis (CHR-P).

METHOD

Magnetoencephalography (MEG) was used in combination with source reconstructed time-series to examine delta and theta-band entrainment during continuous speech. Participants were presented with a 5-minute audio recording during which they either attened to the story or word level. MEG-data were obtained from n = 22 CHR-P participants, n = 23 early-stage ScZ-patients, and n = 44 healthy controls (HC). Data were analysed with a Mutual Information (MI) approach to compute statistical dependence between the MEG and auditory signal, thus estimating individual speech-tracking ability. MEG-activity was reconstructed in a language network (bilateral inferior frontal cortex [F3T; Broca's], superior temporal areas [STS3, STS4; Wernicke's areas], and primary auditory cortex [bilateral HES; Heschl's gyrus]). MEG-data were correlated with clinical symptoms.

RESULTS

Theta-band entrainment in left Heschl's gyrus, averaged across groups, was significantly lower in the STORY compared to WORD condition (p = 0.022), and averaged over conditions, significantly lower in CHR-Ps (p = 0.045), but intact in early ScZ patients (p = 0.303), compared to controls. Correlation analyses between MEG data and symptom indicated that lower theta-band tracking in CHR-Ps was linked to the severity of perceptual abnormalities (p = 0.018).

CONCLUSION

Our results show that CHR-P participants involve impairments in theta-band entrainment during speech tracking in left primary auditory cortex while higher-order speech processing areas were intact. Moreover, the severity of aberrant perceptual experiences in CHR-P participants correlated with deficits in theta-band entrainment. Together, these findings highlight the possibility that neural oscillations during language processing could reveal fundamental abnormalities in speech processing which may constitute candidate biomarkers for early detection and diagnosis of ScZ.

Interference of mid-level sound statistics underlie human speech recognition sensitivity in natural noise

Recognizing speech in noise, such as in a busy restaurant, is an essential cognitive skill where the task difficulty varies across environments and noise levels. Although there is growing evidence that the auditory system relies on statistical representations for perceiving 1-5 and coding4,6-9 natural sounds, it's less clear how statistical cues and neural representations contribute to segregating speech in natural auditory scenes. We demonstrate that human listeners rely on mid-level statistics to segregate and recognize speech in environmental noise. Using natural backgrounds and variants with perturbed spectro-temporal statistics, we show that speech recognition accuracy at a fixed noise level varies extensively across natural backgrounds (0% to 100%). Furthermore, for each background the unique interference created by summary statistics can mask or unmask speech, thus hindering or improving speech recognition. To identify the neural coding strategy and statistical cues that influence accuracy, we developed generalized perceptual regression, a framework that links summary statistics from a neural model to word recognition accuracy. Whereas a peripheral cochlear model accounts for only 60% of perceptual variance, summary statistics from a mid-level auditory midbrain model accurately predicts single trial sensory judgments, accounting for more than 90% of the perceptual variance. Furthermore, perceptual weights from the regression framework identify which statistics and tuned neural filters are influential and how they impact recognition. Thus, perception of speech in natural backgrounds relies on a mid-level auditory representation involving interference of multiple summary statistics that impact recognition beneficially or detrimentally across natural background sounds.

Rhythmic Temporal Cues Coordinate Cross-frequency Phase-amplitude Coupling during Memory Encoding

Accumulating evidence suggests that rhythmic temporal cues in the environment influence the encoding of information into long-term memory. Here, we test the hypothesis that these mnemonic effects of rhythm reflect the coupling of high-frequency (gamma) oscillations to entrained lower-frequency oscillations synchronized to the beat of the rhythm. In Study 1, we first test this hypothesis in the context of global effects of rhythm on memory, when memory is superior for visual stimuli presented in rhythmic compared with arrhythmic patterns at encoding [Jones, A., & Ward, E. V. Rhythmic temporal structure at encoding enhances recognition memory, Journal of Cognitive Neuroscience, 31, 1549-1562, 2019]. We found that rhythmic presentation of visual stimuli during encoding was associated with greater phase-amplitude coupling (PAC) between entrained low-frequency (delta) oscillations and higher-frequency (gamma) oscillations. In Study 2, we next investigated cross-frequency PAC in the context of local effects of rhythm on memory encoding, when memory is superior for visual stimuli presented in-synchrony compared with out-of-synchrony with a background auditory beat [Hickey, P., Merseal, H., Patel, A. D., & Race, E. Memory in time: Neural tracking of low-frequency rhythm dynamically modulates memory formation. Neuroimage, 213, 116693, 2020]. We found that the mnemonic effect of rhythm in this context was again associated with increased cross-frequency PAC between entrained low-frequency (delta) oscillations and higher-frequency (gamma) oscillations. Furthermore, the magnitude of gamma power modulations positively scaled with the subsequent memory benefit for in- versus out-of-synchrony stimuli. Together, these results suggest that the influence of rhythm on memory encoding may reflect the temporal coordination of higher-frequency gamma activity by entrained low-frequency oscillations.

Cortical tracking of language structures: Modality-dependent and independent responses

OBJECTIVES

The mental parsing of linguistic hierarchy is crucial for language comprehension, and while there is growing interest in the cortical tracking of auditory speech, the neurophysiological substrates for tracking written language are still unclear.

METHODS

We recorded electroencephalographic (EEG) responses from participants exposed to auditory and visual streams of either random syllables or tri-syllabic real words. Using a frequency-tagging approach, we analyzed the neural representations of physically presented (i.e., syllables) and mentally constructed (i.e., words) linguistic units and compared them between the two sensory modalities.

RESULTS

We found that tracking syllables is partially modality dependent, with anterior and posterior scalp regions more involved in the tracking of spoken and written syllables, respectively. The cortical tracking of spoken and written words instead was found to involve a shared anterior region to a similar degree, suggesting a modality-independent process for word tracking.

CONCLUSION

Our study suggests that basic linguistic features are represented in a sensory modality-specific manner, while more abstract ones are modality-unspecific during the online processing of continuous language input.

SIGNIFICANCE

The current methodology may be utilized in future research to examine the development of reading skills, especially the deficiencies in fluent reading among those with dyslexia.

Opposing neural processing modes alternate rhythmically during sustained auditory attention

During continuous tasks, humans show spontaneous fluctuations in performance, putatively caused by varying attentional resources allocated to process external information. If neural resources are used to process other, presumably "internal" information, sensory input can be missed and explain an apparent dichotomy of "internal" versus "external" attention. In the current study, we extract presumed neural signatures of these attentional modes in human electroencephalography (EEG): neural entrainment and α-oscillations (~10-Hz), linked to the processing and suppression of sensory information, respectively. We test whether they exhibit structured fluctuations over time, while listeners attend to an ecologically relevant stimulus, like speech, and complete a task that requires full and continuous attention. Results show an antagonistic relation between neural entrainment to speech and spontaneous α-oscillations in two distinct brain networks-one specialized in the processing of external information, the other reminiscent of the dorsal attention network. These opposing neural modes undergo slow, periodic fluctuations around ~0.07 Hz and are related to the detection of auditory targets. Our study might have tapped into a general attentional mechanism that is conserved across species and has important implications for situations in which sustained attention to sensory information is critical.

Unsupervised discovery of family specific vocal usage in the Mongolian gerbil

In nature, animal vocalizations can provide crucial information about identity, including kinship and hierarchy. However, lab-based vocal behavior is typically studied during brief interactions between animals with no prior social relationship, and under environmental conditions with limited ethological relevance. Here, we address this gap by establishing long-term acoustic recordings from Mongolian gerbil families, a core social group that uses an array of sonic and ultrasonic vocalizations. Three separate gerbil families were transferred to an enlarged environment and continuous 20-day audio recordings were obtained. Using a variational autoencoder (VAE) to quantify 583,237 vocalizations, we show that gerbils exhibit a more elaborate vocal repertoire than has been previously reported and that vocal repertoire usage differs significantly by family. By performing gaussian mixture model clustering on the VAE latent space, we show that families preferentially use characteristic sets of vocal clusters and that these usage preferences remain stable over weeks. Furthermore, gerbils displayed family-specific transitions between vocal clusters. Since gerbils live naturally as extended families in complex underground burrows that are adjacent to other families, these results suggest the presence of a vocal dialect which could be exploited by animals to represent kinship. These findings position the Mongolian gerbil as a compelling animal model to study the neural basis of vocal communication and demonstrates the potential for using unsupervised machine learning with uninterrupted acoustic recordings to gain insights into naturalistic animal behavior.

Cross-linguistic and acoustic-driven effects on multiscale neural synchrony to stress rhythms

We investigated how neural oscillations code the hierarchical nature of stress rhythms in speech and how stress processing varies with language experience. By measuring phase synchrony of multilevel EEG-acoustic tracking and intra-brain cross-frequency coupling, we show the encoding of stress involves different neural signatures (delta rhythms = stress foot rate; theta rhythms = syllable rate), is stronger for amplitude vs. duration stress cues, and induces nested delta-theta coherence mirroring the stress-syllable hierarchy in speech. Only native English, but not Mandarin, speakers exhibited enhanced neural entrainment at central stress (2 Hz) and syllable (4 Hz) rates intrinsic to natural English. English individuals with superior cortical-stress tracking capabilities also displayed stronger neural hierarchical coherence, highlighting a nuanced interplay between internal nesting of brain rhythms and external entrainment rooted in language-specific speech rhythms. Our cross-language findings reveal brain-speech synchronization is not purely a "bottom-up" but benefits from "top-down" processing from listeners' language-specific experience.

Dissociating prosodic from syntactic delta activity during natural speech comprehension

Decoding human speech requires the brain to segment the incoming acoustic signal into meaningful linguistic units, ranging from syllables and words to phrases. Integrating these linguistic constituents into a coherent percept sets the root of compositional meaning and hence understanding. One important cue for segmentation in natural speech is prosodic cues, such as pauses, but their interplay with higher-level linguistic processing is still unknown. Here, we dissociate the neural tracking of prosodic pauses from the segmentation of multi-word chunks using magnetoencephalography (MEG). We find that manipulating the regularity of pauses disrupts slow speech-brain tracking bilaterally in auditory areas (below 2 Hz) and in turn increases left-lateralized coherence of higher-frequency auditory activity at speech onsets (around 25-45 Hz). Critically, we also find that multi-word chunks-defined as short, coherent bundles of inter-word dependencies-are processed through the rhythmic fluctuations of low-frequency activity (below 2 Hz) bilaterally and independently of prosodic cues. Importantly, low-frequency alignment at chunk onsets increases the accuracy of an encoding model in bilateral auditory and frontal areas while controlling for the effect of acoustics. Our findings provide novel insights into the neural basis of speech perception, demonstrating that both acoustic features (prosodic cues) and abstract linguistic processing at the multi-word timescale are underpinned independently by low-frequency electrophysiological brain activity in the delta frequency range.

Amplitude envelope onset characteristics modulate phase locking for speech auditory-motor synchronization

The spontaneous speech-to-speech synchronization (SSS) test has been shown to be an effective behavioral method to estimate cortical speech auditory-motor coupling strength through phase-locking value (PLV) between auditory input and motor output. This study further investigated how amplitude envelope onset variations of the auditory speech signal may influence the speech auditory-motor synchronization. Sixty Mandarin-speaking adults listened to a stream of randomly presented syllables at an increasing speed while concurrently whispering in synchrony with the rhythm of the auditory stimuli whose onset consistency was manipulated, consisting of aspirated, unaspirated, and mixed conditions. The participants' PLVs for the three conditions in the SSS test were derived and compared. Results showed that syllable rise time affected the speech auditory-motor synchronization in a bifurcated fashion. Specifically, PLVs were significantly higher in the temporally more consistent conditions (aspirated or unaspirated) than those in the less consistent condition (mixed) for high synchronizers. In contrast, low synchronizers tended to be immune to the onset consistency. Overall, these results validated how syllable onset consistency in the rise time of amplitude envelope may modulate the strength of speech auditory-motor coupling. This study supports the application of the SSS test to examine individual differences in the integration of perception and production systems, which has implications for those with speech and language disorders that have difficulty with processing speech onset characteristics such as rise time.

Speech's syllabic rhythm and articulatory features produced under different auditory feedback conditions identify Parkinsonism

Diagnostic tests for Parkinsonism based on speech samples have shown promising results. Although abnormal auditory feedback integration during speech production and impaired rhythmic organization of speech are known in Parkinsonism, these aspects have not been incorporated into diagnostic tests. This study aimed to identify Parkinsonism using a novel speech behavioral test that involved rhythmically repeating syllables under different auditory feedback conditions. The study included 30 individuals with Parkinson's disease (PD) and 30 healthy subjects. Participants were asked to rhythmically repeat the PA-TA-KA syllable sequence, both whispering and speaking aloud under various listening conditions. The results showed that individuals with PD had difficulties in whispering and articulating under altered auditory feedback conditions, exhibited delayed speech onset, and demonstrated inconsistent rhythmic structure across trials compared to controls. These parameters were then fed into a supervised machine-learning algorithm to differentiate between the two groups. The algorithm achieved an accuracy of 85.4%, a sensitivity of 86.5%, and a specificity of 84.3%. This pilot study highlights the potential of the proposed behavioral paradigm as an objective and accessible (both in cost and time) test for identifying individuals with Parkinson's disease.

Predicting language outcome at birth

Even though most children acquire language effortlessly, not all do. Nowadays, language disorders are difficult to diagnose before 3-4 years of age, because diagnosis relies on behavioral criteria difficult to obtain early in life. Using electroencephalography, I investigated whether differences in newborns' neural activity when listening to sentences in their native language (French) and a rhythmically different unfamiliar language (English) relate to measures of later language development at 12 and 18 months. Here I show that activation differences in the theta band at birth predict language comprehension abilities at 12 and 18 months. These findings suggest that a neural measure of language discrimination at birth could be used in the early identification of infants at risk of developmental language disorders.

Improved tactile speech perception and noise robustness using audio-to-tactile sensory substitution with amplitude envelope expansion

Recent advances in haptic technology could allow haptic hearing aids, which convert audio to tactile stimulation, to become viable for supporting people with hearing loss. A tactile vocoder strategy for audio-to-tactile conversion, which exploits these advances, has recently shown significant promise. In this strategy, the amplitude envelope is extracted from several audio frequency bands and used to modulate the amplitude of a set of vibro-tactile tones. The vocoder strategy allows good consonant discrimination, but vowel discrimination is poor and the strategy is susceptible to background noise. In the current study, we assessed whether multi-band amplitude envelope expansion can effectively enhance critical vowel features, such as formants, and improve speech extraction from noise. In 32 participants with normal touch perception, tactile-only phoneme discrimination with and without envelope expansion was assessed both in quiet and in background noise. Envelope expansion improved performance in quiet by 10.3% for vowels and by 5.9% for consonants. In noise, envelope expansion improved overall phoneme discrimination by 9.6%, with no difference in benefit between consonants and vowels. The tactile vocoder with envelope expansion can be deployed in real-time on a compact device and could substantially improve clinical outcomes for a new generation of haptic hearing aids.

Auditory and motor priming of metric structure improves understanding of degraded speech

Speech comprehension is enhanced when preceded (or accompanied) by a congruent rhythmic prime reflecting the metrical sentence structure. Although these phenomena have been described for auditory and motor primes separately, their respective and synergistic contribution has not been addressed. In this experiment, participants performed a speech comprehension task on degraded speech signals that were preceded by a rhythmic prime that could be auditory, motor or audiomotor. Both auditory and audiomotor rhythmic primes facilitated speech comprehension speed. While the presence of a purely motor prime (unpaced tapping) did not globally benefit speech comprehension, comprehension accuracy scaled with the regularity of motor tapping. In order to investigate inter-individual variability, participants also performed a Spontaneous Speech Synchronization test. The strength of the estimated perception-production coupling correlated positively with overall speech comprehension scores. These findings are discussed in the framework of the dynamic attending and active sensing theories.

Spectro-temporal acoustical markers differentiate speech from song across cultures

Humans produce two forms of cognitively complex vocalizations: speech and song. It is debated whether these differ based primarily on culturally specific, learned features, or if acoustical features can reliably distinguish them. We study the spectro-temporal modulation patterns of vocalizations produced by 369 people living in 21 urban, rural, and small-scale societies across six continents. Specific ranges of spectral and temporal modulations, overlapping within categories and across societies, significantly differentiate speech from song. Machine-learning classification shows that this effect is cross-culturally robust, vocalizations being reliably classified solely from their spectro-temporal features across all 21 societies. Listeners unfamiliar with the cultures classify these vocalizations using similar spectro-temporal cues as the machine learning algorithm. Finally, spectro-temporal features are better able to discriminate song from speech than a broad range of other acoustical variables, suggesting that spectro-temporal modulation-a key feature of auditory neuronal tuning-accounts for a fundamental difference between these categories.

Globally, songs and instrumental melodies are slower and higher and use more stable pitches than speech: A Registered Report

Both music and language are found in all known human societies, yet no studies have compared similarities and differences between song, speech, and instrumental music on a global scale. In this Registered Report, we analyzed two global datasets: (i) 300 annotated audio recordings representing matched sets of traditional songs, recited lyrics, conversational speech, and instrumental melodies from our 75 coauthors speaking 55 languages; and (ii) 418 previously published adult-directed song and speech recordings from 209 individuals speaking 16 languages. Of our six preregistered predictions, five were strongly supported: Relative to speech, songs use (i) higher pitch, (ii) slower temporal rate, and (iii) more stable pitches, while both songs and speech used similar (iv) pitch interval size and (v) timbral brightness. Exploratory analyses suggest that features vary along a "musi-linguistic" continuum when including instrumental melodies and recited lyrics. Our study provides strong empirical evidence of cross-cultural regularities in music and speech.

The human auditory system uses amplitude modulation to distinguish music from speech

Music and speech are complex and distinct auditory signals that are both foundational to the human experience. The mechanisms underpinning each domain are widely investigated. However, what perceptual mechanism transforms a sound into music or speech and how basic acoustic information is required to distinguish between them remain open questions. Here, we hypothesized that a sound's amplitude modulation (AM), an essential temporal acoustic feature driving the auditory system across processing levels, is critical for distinguishing music and speech. Specifically, in contrast to paradigms using naturalistic acoustic signals (that can be challenging to interpret), we used a noise-probing approach to untangle the auditory mechanism: If AM rate and regularity are critical for perceptually distinguishing music and speech, judging artificially noise-synthesized ambiguous audio signals should align with their AM parameters. Across 4 experiments (N = 335), signals with a higher peak AM frequency tend to be judged as speech, lower as music. Interestingly, this principle is consistently used by all listeners for speech judgments, but only by musically sophisticated listeners for music. In addition, signals with more regular AM are judged as music over speech, and this feature is more critical for music judgment, regardless of musical sophistication. The data suggest that the auditory system can rely on a low-level acoustic property as basic as AM to distinguish music from speech, a simple principle that provokes both neurophysiological and evolutionary experiments and speculations.