Development and Validation of Sentences Without Semantic Context to Complement the Basic English Lexicon Sentences

The Influence of Semantic Context on the Intelligibility Benefit From Speech Glimpses in Younger and Older Adults

PURPOSE

Speech is often masked by background sound that fluctuates over time. Fluctuations in masker intensity can reveal glimpses of speech that support speech intelligibility, but older adults have frequently been shown to benefit less from speech glimpses than younger adults when listening to sentences. Recent work, however, suggests that older adults may leverage speech glimpses as much, or more, when listening to naturalistic stories, potentially because of the availability of semantic context in stories. The current study directly investigated whether semantic context helps older adults benefit from speech glimpses released by a fluctuating (modulated) masker more than younger adults.

METHOD

In two experiments, we reduced and extended semantic information of sentence stimuli in modulated and unmodulated speech maskers for younger and older adults. Speech intelligibility was assessed.

RESULTS

We found that semantic context improves speech intelligibility in both younger and older adults. Both age groups also exhibit better speech intelligibility for a modulated than an unmodulated (stationary) masker, but the benefit from the speech glimpses was reduced in older compared to younger adults. Semantic context amplified the benefit gained from the speech glimpses, but there was no indication that the amplification by the semantic context led to a greater benefit in older adults. If anything, younger adults benefitted more.

CONCLUSIONS

The current results suggest that the deficit in the masking-release benefit in older adults generalizes to situations in which extended speech context is available. That previous research found a greater benefit in older than younger adults during story listening may suggest that other factors, such as thematic knowledge, motivation, or cognition, may amplify the benefit from speech glimpses under naturalistic listening conditions.

Adaptation to sentences and melodies when making judgments along a voice-nonvoice continuum

Adaptation to constant or repetitive sensory signals serves to improve detection of novel events in the environment and to encode incoming information more efficiently. Within the auditory modality, contrastive adaptation effects have been observed within a number of categories, including voice and musical instrument type. A recent study found contrastive perceptual shifts between voice and instrument categories following repetitive presentation of adaptors consisting of either vowels or instrument tones. The current study tested the generalizability of adaptation along a voice-instrument continuum, using more ecologically valid adaptors. Participants were presented with an adaptor followed by an ambiguous voice-instrument target, created by generating a 10-step morphed continuum between pairs of vowel and instrument sounds. Listeners' categorization of the target sounds was shifted contrastively by a spoken sentence or instrumental melody adaptor, regardless of whether the adaptor and the target shared the same speaker gender or similar pitch range (Experiment 1). However, no significant contrastive adaptation was observed when nonspeech vocalizations or nonpitched percussion sounds were used as the adaptors (Experiment 2). The results suggest that adaptation between voice and nonvoice categories does not rely on exact repetition of simple stimuli, nor does it solely reflect the result of a sound being categorized as being human or nonhuman sourced. The outcomes suggest future directions for determining the precise spectro-temporal properties of sounds that induce these voice-instrument contrastive adaptation effects.

Increasing Motivation Increases Intelligibility Benefits of Perceptual Training in Dysarthria

PURPOSE

Perceptual training offers a promising, listener-targeted option for improving intelligibility of dysarthric speech. Cognitive resources are required for learning, and theoretical models of listening effort and engagement account for a role of listener motivation in allocation of such resources. Here, we manipulate training instructions to enhance motivation to test the hypothesis that increased motivation increases the intelligibility benefits of perceptual training.

METHOD

Across two data collection sites, which differed with respect to many elements of study design including age of speaker with dysarthria, dysarthria type and severity, type of testing and training stimuli, and participant compensation, 84 neurotypical adults were randomly assigned to one of two training instruction conditions: enhanced instructions or standard instructions. Intelligibility, quantified as percent words correct, was measured before and after training.

RESULTS

Listeners who received the enhanced instructions achieved greater intelligibility improvements from training relative to listeners who received the standard instructions. This result was robust across data collection sites and the many differences in methodology.

CONCLUSIONS

This study provides evidence for the role of motivation in improved understanding of dysarthric speech-increasing motivation increases allocation of cognitive resources to the learning process, resulting in improved mapping of the degraded speech signal. This provides empirical support for theoretical models of listening effort and engagement. Clinically, the results show that a simple addition to the training instructions can elevate learning outcomes.

Repairing Misperceptions of Words Early in a Sentence is More Effortful Than Repairing Later Words, Especially for Listeners With Cochlear Implants

The process of repairing misperceptions has been identified as a contributor to effortful listening in people who use cochlear implants (CIs). The current study was designed to examine the relative cost of repairing misperceptions at earlier or later parts of a sentence that contained contextual information that could be used to infer words both predictively and retroactively. Misperceptions were enforced at specific times by replacing single words with noise. Changes in pupil dilation were analyzed to track differences in the timing and duration of effort, comparing listeners with typical hearing (TH) or with CIs. Increases in pupil dilation were time-locked to the moment of the missing word, with longer-lasting increases when the missing word was earlier in the sentence. Compared to listeners with TH, CI listeners showed elevated pupil dilation for longer periods of time after listening, suggesting a lingering effect of effort after sentence offset. When needing to mentally repair missing words, CI listeners also made more mistakes on words elsewhere in the sentence, even though these words were not masked. Changes in effort based on the position of the missing word were not evident in basic measures like peak pupil dilation and only emerged when the full-time course was analyzed, suggesting the timing analysis adds new information to our understanding of listening effort. These results demonstrate that some mistakes are more costly than others and incur different levels of mental effort to resolve the mistake, underscoring the information lost when characterizing speech perception with simple measures like percent-correct scores.

Testing the role of temporal coherence on speech intelligibility with noise and single-talker maskers

Temporal coherence, where sounds with aligned timing patterns are perceived as a single source, is considered an essential cue in auditory scene analysis. However, its effects have been studied primarily with simple repeating tones, rather than speech. This study investigated the role of temporal coherence in speech by introducing across-frequency asynchronies. The effect of asynchrony on the intelligibility of target sentences was tested in the presence of background speech-shaped noise or a single-talker interferer. Our hypothesis was that disrupting temporal coherence should not only reduce intelligibility but also impair listeners' ability to segregate the target speech from an interfering talker, leading to greater degradation for speech-in-speech than speech-in-noise tasks. Stimuli were filtered into eight frequency bands, which were then desynchronized with delays of 0-120 ms. As expected, intelligibility declined as asynchrony increased. However, the decline was similar for both noise and single-talker maskers. Primarily target, rather than masker, asynchrony affected performance for both natural (forward) and reversed-speech maskers, and for target sentences with low and high semantic context. The results suggest that temporal coherence may not be as critical a cue for speech segregation as it is for the non-speech stimuli traditionally used in studies of auditory scene analysis.

Perceptual Learning of Dysarthria in Adolescence

PURPOSE

As evidenced by perceptual learning studies involving adult listeners and speakers with dysarthria, adaptation to dysarthric speech is driven by signal predictability (speaker property) and a flexible speech perception system (listener property). Here, we extend adaptation investigations to adolescent populations and examine whether adult and adolescent listeners can learn to better understand an adolescent speaker with dysarthria.

METHOD

Classified by developmental stage, adult (n = 42) and adolescent (n = 40) listeners completed a three-phase perceptual learning protocol (pretest, familiarization, and posttest). During pretest and posttest, all listeners transcribed speech produced by a 13-year-old adolescent with spastic dysarthria associated with cerebral palsy. During familiarization, half of the adult and adolescent listeners engaged in structured familiarization (audio and lexical feedback) with the speech of the adolescent speaker with dysarthria; and the other half, with the speech of a neurotypical adolescent speaker (control).

RESULTS

Intelligibility scores increased from pretest to posttest for all listeners. However, listeners who received dysarthria familiarization achieved greater intelligibility improvements than those who received control familiarization. Furthermore, there was a significant effect of developmental stage, where the adults achieved greater intelligibility improvements relative to the adolescents.

CONCLUSIONS

This study provides the first tranche of evidence that adolescent dysarthric speech is learnable-a finding that holds even for adolescent listeners whose speech perception systems are not yet fully developed. Given the formative role that social interactions play during adolescence, these findings of improved intelligibility afford important clinical implications.

Predicting speech-in-speech recognition: Short-term audibility, talker sex, and listener factors

Speech-in-speech recognition can be challenging, and listeners vary considerably in their ability to accomplish this complex auditory-cognitive task. Variability in performance can be related to intrinsic listener factors as well as stimulus factors associated with energetic and informational masking. The current experiments characterized the effects of short-term audibility of the target, differences in target and masker talker sex, and intrinsic listener variables on sentence recognition in two-talker speech and speech-shaped noise. Participants were young adults with normal hearing. Each condition included the adaptive measurement of speech reception thresholds, followed by testing at a fixed signal-to-noise ratio (SNR). Short-term audibility for each keyword was quantified using a computational glimpsing model for target+masker mixtures. Scores on a psychophysical task of auditory stream segregation predicted speech recognition, with stronger effects for speech-in-speech than speech-in-noise. Both speech-in-speech and speech-in-noise recognition depended on the proportion of audible glimpses available in the target+masker mixture, even across stimuli presented at the same global SNR. Short-term audibility requirements varied systematically across stimuli, providing an estimate of the greater informational masking for speech-in-speech than speech-in-noise recognition and quantifying informational masking for matched and mismatched talker sex.

Masked-Speech Recognition for Linguistically Diverse Populations: A Focused Review and Suggestions for the Future

PURPOSE

Twenty years ago, von Hapsburg and Peña (2002) wrote a tutorial that reviewed the literature on speech audiometry and bilingualism and outlined valuable recommendations to increase the rigor of the evidence base. This review article returns to that seminal tutorial to reflect on how that advice was applied over the last 20 years and to provide updated recommendations for future inquiry.

METHOD

We conducted a focused review of the literature on masked-speech recognition for bilingual children and adults. First, we evaluated how studies published since 2002 described bilingual participants. Second, we reviewed the literature on native language masked-speech recognition. Third, we discussed theoretically motivated experimental work. Fourth, we outlined how recent research in bilingual speech recognition can be used to improve clinical practice.

RESULTS

Research conducted since 2002 commonly describes bilingual samples in terms of their language status, competency, and history. Bilingualism was not consistently associated with poor masked-speech recognition. For example, bilinguals who were exposed to English prior to age 7 years and who were dominant in English performed comparably to monolinguals for masked-sentence recognition tasks. To the best of our knowledge, there are no data to document the masked-speech recognition ability of these bilinguals in their other language compared to a second monolingual group, which is an important next step. Nonetheless, individual factors that commonly vary within bilingual populations were associated with masked-speech recognition and included language dominance, competency, and age of acquisition. We identified methodological issues in sampling strategies that could, in part, be responsible for inconsistent findings between studies. For instance, disparities in socioeconomic status (SES) between recruited bilingual and monolingual groups could cause confounding bias within the research design.

CONCLUSIONS

Dimensions of the bilingual linguistic profile should be considered in clinical practice to inform counseling and (re)habilitation strategies since susceptibility to masking is elevated in at least one language for most bilinguals. Future research should continue to report language status, competency, and history but should also report language stability and demand for use data. In addition, potential confounds (e.g., SES, educational attainment) when making group comparisons between monolinguals and bilinguals must be considered.

Preliminary Evaluation of Automated Speech Recognition Apps for the Hearing Impaired and Deaf

Objective

Automated speech recognition (ASR) systems have become increasingly sophisticated, accurate, and deployable on many digital devices, including on a smartphone. This pilot study aims to examine the speech recognition performance of ASR apps using audiological speech tests. In addition, we compare ASR speech recognition performance to normal hearing and hearing impaired listeners and evaluate if standard clinical audiological tests are a meaningful and quick measure of the performance of ASR apps.

Methods

Four apps have been tested on a smartphone, respectively AVA, Earfy, Live Transcribe, and Speechy. The Dutch audiological speech tests performed were speech audiometry in quiet (Dutch CNC-test), Digits-in-Noise (DIN)-test with steady-state speech-shaped noise, sentences in quiet and in averaged long-term speech-shaped spectrum noise (Plomp-test). For comparison, the app's ability to transcribe a spoken dialogue (Dutch and English) was tested.

Results

All apps scored at least 50% phonemes correct on the Dutch CNC-test for a conversational speech intensity level (65 dB SPL) and achieved 90–100% phoneme recognition at higher intensity levels. On the DIN-test, AVA and Live Transcribe had the lowest (best) signal-to-noise ratio +8 dB. The lowest signal-to-noise measured with the Plomp-test was +8 to 9 dB for Earfy (Android) and Live Transcribe (Android). Overall, the word error rate for the dialogue in English (19–34%) was lower (better) than for the Dutch dialogue (25–66%).

Conclusion

The performance of the apps was limited on audiological tests that provide little linguistic context or use low signal to noise levels. For Dutch audiological speech tests in quiet, ASR apps performed similarly to a person with a moderate hearing loss. In noise, the ASR apps performed more poorly than most profoundly deaf people using a hearing aid or cochlear implant. Adding new performance metrics including the semantic difference as a function of SNR and reverberation time could help to monitor and further improve ASR performance.

Visualization of Speech Perception Analysis via Phoneme Alignment: A Pilot Study

Objective: Speech tests assess the ability of people with hearing loss to comprehend speech with a hearing aid or cochlear implant. The tests are usually at the word or sentence level. However, few tests analyze errors at the phoneme level. So, there is a need for an automated program to visualize in real time the accuracy of phonemes in these tests. Method: The program reads in stimulus-response pairs and obtains their phonemic representations from an open-source digital pronouncing dictionary. The stimulus phonemes are aligned with the response phonemes via a modification of the Levenshtein Minimum Edit Distance algorithm. Alignment is achieved via dynamic programming with modified costs based on phonological features for insertion, deletions and substitutions. The accuracy for each phoneme is based on the F1-score. Accuracy is visualized with respect to place and manner (consonants) or height (vowels). Confusion matrices for the phonemes are used in an information transfer analysis of ten phonological features. A histogram of the information transfer for the features over a frequency-like range is presented as a phonemegram. Results: The program was applied to two datasets. One consisted of test data at the sentence and word levels. Stimulus-response sentence pairs from six volunteers with different degrees of hearing loss and modes of amplification were analyzed. Four volunteers listened to sentences from a mobile auditory training app while two listened to sentences from a clinical speech test. Stimulus-response word pairs from three lists were also analyzed. The other dataset consisted of published stimulus-response pairs from experiments of 31 participants with cochlear implants listening to 400 Basic English Lexicon sentences via different talkers at four different SNR levels. In all cases, visualization was obtained in real time. Analysis of 12,400 actual and random pairs showed that the program was robust to the nature of the pairs. Conclusion: It is possible to automate the alignment of phonemes extracted from stimulus-response pairs from speech tests in real time. The alignment then makes it possible to visualize the accuracy of responses via phonological features in two ways. Such visualization of phoneme alignment and accuracy could aid clinicians and scientists.

Role of semantic context and talker variability in speech perception of cochlear-implant users and normal-hearing listeners

This study assessed the impact of semantic context and talker variability on speech perception by cochlear-implant (CI) users and compared their overall performance and between-subjects variance with that of normal-hearing (NH) listeners under vocoded conditions. Thirty post-lingually deafened adult CI users were tested, along with 30 age-matched and 30 younger NH listeners, on sentences with and without semantic context, presented in quiet and noise, spoken by four different talkers. Additional measures included working memory, non-verbal intelligence, and spectral-ripple detection and discrimination. Semantic context and between-talker differences influenced speech perception to similar degrees for both CI users and NH listeners. Between-subjects variance for speech perception was greatest in the CI group but remained substantial in both NH groups, despite the uniformly degraded stimuli in these two groups. Spectral-ripple detection and discrimination thresholds in CI users were significantly correlated with speech perception, but a single set of vocoder parameters for NH listeners was not able to capture average CI performance in both speech and spectral-ripple tasks. The lack of difference in the use of semantic context between CI users and NH listeners suggests no overall differences in listening strategy between the groups, when the stimuli are similarly degraded.