Talkers produce more pronounced amplitude modulations when speaking in noise

Acoustic correlates of the syllabic rhythm of speech: Modulation spectrum or local features of the temporal envelope

The syllable is a perceptually salient unit in speech. Since both the syllable and its acoustic correlate, i.e., the speech envelope, have a preferred range of rhythmicity between 4 and 8 Hz, it is hypothesized that theta-band neural oscillations play a major role in extracting syllables based on the envelope. A literature survey, however, reveals inconsistent evidence about the relationship between speech envelope and syllables, and the current study revisits this question by analyzing large speech corpora. It is shown that the center frequency of speech envelope, characterized by the modulation spectrum, reliably correlates with the rate of syllables only when the analysis is pooled over minutes of speech recordings. In contrast, in the time domain, a component of the speech envelope is reliably phase-locked to syllable onsets. Based on a speaker-independent model, the timing of syllable onsets explains about 24% variance of the speech envelope. These results indicate that local features in the speech envelope, instead of the modulation spectrum, are a more reliable acoustic correlate of syllables.

Individual theta-band cortical entrainment to speech in quiet predicts word-in-noise comprehension

Speech elicits brain activity time-locked to its amplitude envelope. The resulting speech-brain synchrony (SBS) is thought to be crucial to speech parsing and comprehension. It has been shown that higher speech-brain coherence is associated with increased speech intelligibility. However, studies depending on the experimental manipulation of speech stimuli do not allow conclusion about the causality of the observed tracking. Here, we investigate whether individual differences in the intrinsic propensity to track the speech envelope when listening to speech-in-quiet is predictive of individual differences in speech-recognition-in-noise, in an independent task. We evaluated the cerebral tracking of speech in source-localized magnetoencephalography, at timescales corresponding to the phrases, words, syllables and phonemes. We found that individual differences in syllabic tracking in right superior temporal gyrus and in left middle temporal gyrus (MTG) were positively associated with recognition accuracy in an independent words-in-noise task. Furthermore, directed connectivity analysis showed that this relationship is partially mediated by top-down connectivity from premotor cortex-associated with speech processing and active sensing in the auditory domain-to left MTG. Thus, the extent of SBS-even during clear speech-reflects an active mechanism of the speech processing system that may confer resilience to noise.

Vocal Behavior of Teachers Reading with Raised Voice in a Noisy Environment

(1) Objective: Teaching is a particularly voice-demanding occupation. Voice training provided during teachers’ education is often insufficient and thus teachers are at risk of developing voice disorders. Vocal demands during teaching are not only characterized by speaking for long durations but also by speaking in noisy environments. This provokes the so-called Lombard effect, which intuitively leads to an increase in voice intensity, pitch and phonation time in laboratory studies. However, this effect has not been thoroughly investigated in realistic teaching scenarios. (2) Methods: This study thus examined how 13 experienced, but vocally untrained, teachers behaved when reading in a noisy compared to quiet background environment. The quiet and noisy conditions were provided by a live audience either listening quietly or making noise by talking to each other. By using a portable voice accumulator, the fundamental frequency, sound pressure level of the voice and the noise as well as the phonation time were recorded in both conditions. (3) Results: The results showed that the teachers mainly responded according to the Lombard effect. In addition, analysis of phonation time revealed that they failed to increase inhalation time and appeared to lose articulation through the shortening of voiceless consonants in the noisy condition. (4) Conclusions: The teachers demonstrated vocally demanding behavior when speaking in the noisy condition, which can lead to vocal fatigue and cause dysphonia. The findings underline the necessity for specific voice training in teachers’ education, and the content of such training is discussed in light of the results.

An oscillating computational model can track pseudo-rhythmic speech by using linguistic predictions

Neuronal oscillations putatively track speech in order to optimize sensory processing. However, it is unclear how isochronous brain oscillations can track pseudo-rhythmic speech input. Here we propose that oscillations can track pseudo-rhythmic speech when considering that speech time is dependent on content-based predictions flowing from internal language models. We show that temporal dynamics of speech are dependent on the predictability of words in a sentence. A computational model including oscillations, feedback, and inhibition is able to track pseudo-rhythmic speech input. As the model processes, it generates temporal phase codes, which are a candidate mechanism for carrying information forward in time. The model is optimally sensitive to the natural temporal speech dynamics and can explain empirical data on temporal speech illusions. Our results suggest that speech tracking does not have to rely only on the acoustics but could also exploit ongoing interactions between oscillations and constraints flowing from internal language models.

Using fuzzy string matching for automated assessment of listener transcripts in speech intelligibility studies

Many studies of speech perception assess the intelligibility of spoken sentence stimuli by means of transcription tasks ('type out what you hear'). The intelligibility of a given stimulus is then often expressed in terms of percentage of words correctly reported from the target sentence. Yet scoring the participants' raw responses for words correctly identified from the target sentence is a time-consuming task, and hence resource-intensive. Moreover, there is no consensus among speech scientists about what specific protocol to use for the human scoring, limiting the reliability of human scores. The present paper evaluates various forms of fuzzy string matching between participants' responses and target sentences, as automated metrics of listener transcript accuracy. We demonstrate that one particular metric, the token sort ratio, is a consistent, highly efficient, and accurate metric for automated assessment of listener transcripts, as evidenced by high correlations with human-generated scores (best correlation: r = 0.940) and a strong relationship to acoustic markers of speech intelligibility. Thus, fuzzy string matching provides a practical tool for assessment of listener transcript accuracy in large-scale speech intelligibility studies. See https://tokensortratio.netlify.app for an online implementation.

Effects of task and language nativeness on the Lombard effect and on its onset and offset timing

This study focuses on the differences in speech sound pressure levels (here, called speech loudness) of Lombard speech (i.e., speech produced in the presence of an energetic masker) associated with different tasks and language nativeness. Vocalizations were produced by native speakers of Japanese with normal hearing and limited English proficiency while performing four tasks: dialog, a competitive game (both communicative), soliloquy, and text passage reading (noncommunicative). Relative to the native language (L1), larger loudness increments were observed in the game and text reading when performed in the second language (L2). Communicative tasks yielded louder vocalizations and larger increments of speech loudness than did noncommunicative tasks regardless of the spoken language. The period in which speakers increased their loudness after the onset of the masker was about fourfold longer than the time in which they decreased their loudness after the offset of the masker. Results suggest that when relying on acoustic signals, speakers use similar vocalization strategies in L1 and L2, and these depend on the complexity of the task, the need for accurate pronunciation, and the presence of a listener. Results also suggest that speakers use different strategies depending on the onset or offset of an energetic masker.

Echolocating bats exhibit differential amplitude compensation for noise interference at a sub-call level

Flexible vocal production control enables sound communication in both favorable and unfavorable conditions. The Lombard effect, which describes a rise in call amplitude with increasing ambient noise, is a widely exploited strategy by vertebrates to cope with interfering noise. In humans, the Lombard effect influences the lexical stress through differential amplitude modulation at a sub-call syllable level, which so far has not been documented in animals. Here, we bridge this knowledge gap with two species of Hipposideros bats, which produce echolocation calls consisting of two functionally well-defined units: the constant-frequency (CF) and frequency-modulated (FM) components. We show that ambient noise induced a strong, but differential, Lombard effect in the CF and FM components of the echolocation calls. We further report that the differential amplitude compensation occurred only in the spectrally overlapping noise conditions, suggesting a functional role in releasing masking. Lastly, we show that both species of bats exhibited a robust Lombard effect in the spectrally non-overlapping noise conditions, which contrasts sharply with the existing evidence. Our data highlight echolocating bats as a potential mammalian model for understanding vocal production control.

Characterizing amplitude and frequency modulation cues in natural soundscapes: A pilot study on four habitats of a biosphere reserve

Natural soundscapes correspond to the acoustical patterns produced by biological and geophysical sound sources at different spatial and temporal scales for a given habitat. This pilot study aims to characterize the temporal-modulation information available to humans when perceiving variations in soundscapes within and across natural habitats. This is addressed by processing soundscapes from a previous study [Krause, Gage, and Joo. (2011). Landscape Ecol. 26, 1247] via models of human auditory processing extracting modulation at the output of cochlear filters. The soundscapes represent combinations of elevation, animal, and vegetation diversity in four habitats of the biosphere reserve in the Sequoia National Park (Sierra Nevada, USA). Bayesian statistical analysis and support vector machine classifiers indicate that: (i) amplitude-modulation (AM) and frequency-modulation (FM) spectra distinguish the soundscapes associated with each habitat; and (ii) for each habitat, diurnal and seasonal variations are associated with salient changes in AM and FM cues at rates between about 1 and 100 Hz in the low (<0.5 kHz) and high (>1-3 kHz) audio-frequency range. Support vector machine classifications further indicate that soundscape variations can be classified accurately based on these perceptually inspired representations.

Enhanced amplitude modulations contribute to the Lombard intelligibility benefit: Evidence from the Nijmegen Corpus of Lombard Speech

Speakers adjust their voice when talking in noise, which is known as Lombard speech. These acoustic adjustments facilitate speech comprehension in noise relative to plain speech (i.e., speech produced in quiet). However, exactly which characteristics of Lombard speech drive this intelligibility benefit in noise remains unclear. This study assessed the contribution of enhanced amplitude modulations to the Lombard speech intelligibility benefit by demonstrating that (1) native speakers of Dutch in the Nijmegen Corpus of Lombard Speech produce more pronounced amplitude modulations in noise vs in quiet; (2) more enhanced amplitude modulations correlate positively with intelligibility in a speech-in-noise perception experiment; (3) transplanting the amplitude modulations from Lombard speech onto plain speech leads to an intelligibility improvement, suggesting that enhanced amplitude modulations in Lombard speech contribute towards intelligibility in noise. Results are discussed in light of recent neurobiological models of speech perception with reference to neural oscillators phase-locking to the amplitude modulations in speech, guiding the processing of speech.

Effects of added absorption on the vocal exertions of talkers in a reverberant room

Occupational speech users such as schoolteachers develop voice disorders at higher rates than the general population. Previous research has suggested that room acoustics may influence these trends. The research reported in this paper utilized varying acoustical conditions in a reverberant room to assess the effects on vocal parameters of healthy talkers. Thirty-two participants were recorded while completing a battery of speech tasks under eight room conditions. Vocal parameters were derived from the recordings and the statistically significant effects of room acoustics were verified using mixed-model analysis of variance tests. Changes in reverberation time (T₂₀), early decay time (EDT), clarity index (C₅₀), speech transmission index (STI), and room gain (G_RG) all showed highly correlated effects on certain vocal parameters, including speaking level standard deviation, speaking rate, and the acoustic vocal quality index. As T₂₀, EDT, and G_RG increased, and as C₅₀ and STI decreased, vocal parameters showed tendencies toward dysphonic phonation. Empirically derived equations are proposed that describe the relationships between select room-acoustic parameters and vocal parameters. This study provides an increased understanding of the impact of room acoustics on voice production, which could assist acousticians in improving room designs to help mitigate unhealthy vocal exertion and, by extension, voice problems.

Hyper-articulation in Lombard speech: An active communicative strategy to enhance visible speech cues?

This study investigates the hypothesis that speakers make active use of the visual modality in production to improve their speech intelligibility in noisy conditions. Six native speakers of Canadian French produced speech in quiet conditions and in 85 dB of babble noise, in three situations: interacting face-to-face with the experimenter (AV), using the auditory modality only (AO), or reading aloud (NI, no interaction). The audio signal was recorded with the three-dimensional movements of their lips and tongue, using electromagnetic articulography. All the speakers reacted similarly to the presence vs absence of communicative interaction, showing significant speech modifications with noise exposure in both interactive and non-interactive conditions, not only for parameters directly related to voice intensity or for lip movements (very visible) but also for tongue movements (less visible); greater adaptation was observed in interactive conditions, though. However, speakers reacted differently to the availability or unavailability of visual information: only four speakers enhanced their visible articulatory movements more in the AV condition. These results support the idea that the Lombard effect is at least partly a listener-oriented adaptation. However, to clarify their speech in noisy conditions, only some speakers appear to make active use of the visual modality.