Reconstructing wholes from parts: effects of modality, age, and hearing loss on word recognition

Effects of age on brainstem coding of speech glimpses in interrupted noise

Difficulty understanding speech in fluctuating backgrounds is common among older adults. Whereas younger adults are adept at interpreting speech based on brief moments when the signal-to-noise ratio is favorable, older adults use these glimpses of speech less effectively. Age-related declines in auditory brainstem function may degrade the fidelity of speech cues in fluctuating noise for older adults, such that brief glimpses of speech interrupted by noise segments are not faithfully represented in the neural code that reaches the cortex. This hypothesis was tested using electrophysiological recordings of the envelope following response (EFR) elicited by glimpses of speech-like stimuli varying in duration (42, 70, 210 ms) and interrupted by silence or intervening noise. Responses from adults aged 23-73 years indicated that both age and hearing sensitivity were associated with EFR temporal coherence and response magnitude. Age was better than hearing sensitivity for predicting temporal coherence, whereas hearing sensitivity was better than age for predicting response magnitude. Poorer-fidelity EFRs were observed with shorter glimpses and with the addition of intervening noise. However, losses of fidelity with glimpse duration and noise were not associated with participant age or hearing sensitivity. These results suggest that the EFR is sensitive to factors commonly associated with glimpsing but do not entirely account for age-related changes in speech recognition in fluctuating backgrounds.

The effect of modality onset asynchrony and processing time on the recognition of text-supplemented speech

This study examined the effect of modality onset asynchrony and response processing time for the recognition of text-supplemented speech. Speech and text were periodically interrupted by noise or black bars, respectively, to preserve 50% of the sentence and presented in unimodal and multimodal conditions. Sentence recognition and response errors were assessed for responses made simultaneous with the stimulus or after its presentation. Increased processing time allowed for the cognitive repair of initial response errors in working memory. Text-supplemented speech was best recognized with minimal temporal asynchrony. Overall, text supplementation facilitated the recognition of degraded speech when provided sufficient processing time.

Older adults' neural tracking of interrupted speech is a function of task difficulty

Age-related hearing loss is a highly prevalent condition, which manifests at both the auditory periphery and the brain. It leads to degraded auditory input, which needs to be repaired in order to achieve understanding of spoken language. It is still unclear how older adults with this condition draw on their neural resources to optimally process speech. By presenting interrupted speech to 26 healthy older adults with normal-for-age audiograms, this study investigated neural tracking of degraded auditory input. The electroencephalograms of the participants were recorded while they first listened to and then verbally repeated sentences interrupted by silence in varying interruption rates. Speech tracking was measured by inter-trial phase coherence in response to the stimuli. In interruption rates that corresponded to the theta frequency band, speech tracking was highly specific to the interruption rate and positively related to the understanding of interrupted speech. These results suggest that older adults' brain activity optimizes through the tracking of stimulus characteristics, and that this tracking aids in processing an incomplete auditory stimulus. Further investigation of speech tracking as a candidate training mechanism to alleviate age-related hearing loss is thus encouraged.

Older adult recognition error patterns when listening to interrupted speech and speech in steady-state noise

This study examined sentence recognition errors made by older adults in degraded listening conditions compared to a previous sample of younger adults. We examined speech recognition errors made by older normal-hearing adults who repeated sentences that were corrupted by steady-state noise (SSN) or periodically interrupted by noise to preserve 33%, 50%, or 66% of the sentence. Responses were transcribed and coded for the number and type of keyword errors. Errors increased with decreasing preservation of the sentence. Similar sentence recognition was observed between SSN and the greatest amount of interruption (33%). Errors were predominately at the word level rather than at the phoneme level and consisted of omission or substitution of keywords. Compared to younger listeners, older listeners made more total errors and omitted more whole words when speech was highly degraded. They also made more whole word substitutions when speech was more preserved. In addition, the semantic relatedness of the substitution errors to the sentence context varied according to the distortion condition, with greater context effects in SSN than interruption. Overall, older listeners made errors reflecting poorer speech representations. Error analyses provide a more detailed account of speech recognition by identifying changes in the type of errors made across listening conditions and listener groups.

Changes in the Auditory Association Cortex in Dementing Illnesses

OBJECTIVE

To evaluate the relationship between degree of cognitive impairment and gray-matter density changes in the auditory cortex.

STUDY DESIGN

Retrospective case-control.

PATIENTS

Six hundred sixty-three patients of a tertiary referral center cognitive disorders clinic.

INTERVENTION

Magnetic resonance imaging.

MAIN OUTCOME MEASURES

Ratios of gray matter density of the primary auditory cortex (A1) to whole brain and auditory association cortex (AAC) to whole brain in patients with Alzheimer's disease (AD) compared with mild cognitive impairment (MCI) and patients with a mini-mental state exam (MMSE) scores ≤25 versus >25.

RESULTS

After multivariate analysis, a statistically significant difference between AAC to brain ratios for patients with a MMSE ≤25 (n = 325) compared with >25 (n = 269) was found, with values -0.03 (95% CI -0.04 to -0.02, p < 0.0001) on the left and -0.04 (95% CI -0.06 to -0.03, p < 0.0001) on the right. The adjusted average difference of left and right AAC to brain ratios between AD patients (n = 218) compared with MCI patients (n = 121) was also statistically significant, at -0.03 (95% CI -0.05 to -0.01, p = 0.004) and -0.05 (95% CI -0.07 to -0.03, p < 0.0001), respectively. There was no statistically significant difference in the left or right A1 to brain ratios between the MMSE groups or between the AD and MCI groups.

CONCLUSIONS

The AAC for patients with MMSE ≤25 and for those with AD shows decreased gray matter density when compared with patients with better cognitive function. No difference was detected in A1, raising the possibility that patients may have intact neural hearing, but impaired ability to interpret sounds.

Contributions of Voice Expectations to Talker Selection in Younger and Older Adults With Normal Hearing

Focused attention on expected voice features, such as fundamental frequency (F0) and spectral envelope, may facilitate segregation and selection of a target talker in competing talker backgrounds. Age-related declines in attention may limit these abilities in older adults, resulting in poorer speech understanding in complex environments. To test this hypothesis, younger and older adults with normal hearing listened to sentences with a single competing talker. For most trials, listener attention was directed to the target by a cue phrase that matched the target talker's F0 and spectral envelope. For a small percentage of randomly occurring probe trials, the target's voice unexpectedly differed from the cue phrase in terms of F0 and spectral envelope. Overall, keyword recognition for the target talker was poorer for older adults than younger adults. Keyword recognition was poorer on probe trials than standard trials for both groups, and incorrect responses on probe trials contained keywords from the single-talker masker. No interaction was observed between age-group and the decline in keyword recognition on probe trials. Thus, reduced performance by older adults overall could not be attributed to declines in attention to an expected voice. Rather, other cognitive abilities, such as speed of processing and linguistic closure, were predictive of keyword recognition for younger and older adults. Moreover, the effects of age interacted with the sex of the target talker, such that older adults had greater difficulty understanding target keywords from female talkers than male talkers.

Perceptual Organization of Interrupted Speech and Text

PURPOSE

Visual recognition of interrupted text may predict speech intelligibility under adverse listening conditions. This study investigated the nature of the linguistic information and perceptual processes underlying this relationship.

METHOD

To directly compare the perceptual organization of interrupted speech and text, we examined the recognition of spoken and printed sentences interrupted at different rates in 14 adults with normal hearing. The interruption method approximated deletion and retention of rate-specific linguistic information (0.5-64 Hz) in speech by substituting either white space or silent intervals for text or speech in the original sentences.

RESULTS

A similar U-shaped pattern of cross-rate variation in performance was observed in both modalities, with minima at 2 Hz. However, at the highest and lowest interruption rates, recognition accuracy was greater for text than speech, whereas the reverse was observed at middle rates. An analysis of word duration and the frequency of word sampling across interruption rates suggested that the location of the function minima was influenced by perceptual reconstruction of whole words. Overall, the findings indicate a high degree of similarity in the perceptual organization of interrupted speech and text.

CONCLUSION

The observed rate-specific variation in the perception of speech and text may potentially affect the degree to which recognition accuracy in one modality is predictive of the other.

Age effects on perceptual organization of speech: Contributions of glimpsing, phonemic restoration, and speech segregation

In realistic listening environments, speech perception requires grouping together audible fragments of speech, filling in missing information, and segregating the glimpsed target from the background. The purpose of this study was to determine the extent to which age-related difficulties with these tasks can be explained by declines in glimpsing, phonemic restoration, and/or speech segregation. Younger and older adults with normal hearing listened to sentences interrupted with silence or envelope-modulated noise, presented either in quiet or with a competing talker. Older adults were poorer than younger adults at recognizing keywords based on short glimpses but benefited more when envelope-modulated noise filled silent intervals. Recognition declined with a competing talker but this effect did not interact with age. Results of cognitive tasks indicated that faster processing speed and better visual-linguistic closure were predictive of better speech understanding. Taken together, these results suggest that age-related declines in speech recognition may be partially explained by difficulty grouping short glimpses of speech into a coherent message.

Age effects on perceptual organization of speech: Contributions of glimpsing, phonemic restoration, and speech segregation

In realistic listening environments, speech perception requires grouping together audible fragments of speech, filling in missing information, and segregating the glimpsed target from the background. The purpose of this study was to determine the extent to which age-related difficulties with these tasks can be explained by declines in glimpsing, phonemic restoration, and/or speech segregation. Younger and older adults with normal hearing listened to sentences interrupted with silence or envelope-modulated noise, presented either in quiet or with a competing talker. Older adults were poorer than younger adults at recognizing keywords based on short glimpses but benefited more when envelope-modulated noise filled silent intervals. Recognition declined with a competing talker but this effect did not interact with age. Results of cognitive tasks indicated that faster processing speed and better visual-linguistic closure were predictive of better speech understanding. Taken together, these results suggest that age-related declines in speech recognition may be partially explained by difficulty grouping short glimpses of speech into a coherent message.

Reading Behind the Lines: The Factors Affecting the Text Reception Threshold in Hearing Aid Users

PURPOSE

The visual Text Reception Threshold (TRT) test (Zekveld et al., 2007) has been designed to assess modality-general factors relevant for speech perception in noise. In the last decade, the test has been adopted in audiology labs worldwide. The 1st aim of this study was to examine which factors best predict interindividual differences in the TRT. Second, we aimed to assess the relationships between the TRT and the speech reception thresholds (SRTs) estimated in various conditions.

METHOD

First, we reviewed studies reporting relationships between the TRT and the auditory and/or cognitive factors and formulated specific hypotheses regarding the TRT predictors. These hypotheses were tested using a prediction model applied to a rich data set of 180 hearing aid users. In separate association models, we tested the relationships between the TRT and the various SRTs and subjective hearing difficulties, while taking into account potential confounding variables.

RESULTS

The results of the prediction model indicate that the TRT is predicted by the ability to fill in missing words in incomplete sentences, by lexical access speed, and by working memory capacity. Furthermore, in line with previous studies, a moderate association between higher age, poorer pure-tone hearing acuity, and poorer TRTs was observed. Better TRTs were associated with better SRTs for the correct perception of 50% of Hagerman matrix sentences in a 4-talker babble, as well as with better subjective ratings of speech perception. Age and pure-tone hearing thresholds significantly confounded these associations. The associations of the TRT with SRTs estimated in other conditions and with subjective qualities of hearing were not statistically significant when adjusting for age and pure-tone average.

CONCLUSIONS

We conclude that the abilities tapped into by the TRT test include processes relevant for speeded lexical decision making when completing partly masked sentences and that these processes require working memory capacity. Furthermore, the TRT is associated with the SRT of hearing aid users as estimated in a challenging condition that includes informational masking and with experienced difficulties with speech perception in daily-life conditions. The current results underline the value of using the TRT test in studies involving speech perception and aid in the interpretation of findings acquired using the test.

Processing Mechanisms in Hearing-Impaired Listeners: Evidence from Reaction Times and Sentence Interpretation

OBJECTIVE

The authors aimed to determine whether hearing impairment affects sentence comprehension beyond phoneme or word recognition (i.e., on the sentence level), and to distinguish grammatically induced processing difficulties in structurally complex sentences from perceptual difficulties associated with listening to degraded speech. Effects of hearing impairment or speech in noise were expected to reflect hearer-specific speech recognition difficulties. Any additional processing time caused by the sustained perceptual challenges across the sentence may either be independent of or interact with top-down processing mechanisms associated with grammatical sentence structure.

DESIGN

Forty-nine participants listened to canonical subject-initial or noncanonical object-initial sentences that were presented either in quiet or in noise. Twenty-four participants had mild-to-moderate hearing impairment and received hearing-loss-specific amplification. Twenty-five participants were age-matched peers with normal hearing status. Reaction times were measured on-line at syntactically critical processing points as well as two control points to capture differences in processing mechanisms. An off-line comprehension task served as an additional indicator of sentence (mis)interpretation, and enforced syntactic processing.

RESULTS

The authors found general effects of hearing impairment and speech in noise that negatively affected perceptual processing, and an effect of word order, where complex grammar locally caused processing difficulties for the noncanonical sentence structure. Listeners with hearing impairment were hardly affected by noise at the beginning of the sentence, but were affected markedly toward the end of the sentence, indicating a sustained perceptual effect of speech recognition. Comprehension of sentences with noncanonical word order was negatively affected by degraded signals even after sentence presentation.

CONCLUSION

Hearing impairment adds perceptual processing load during sentence processing, but affects grammatical processing beyond the word level to the same degree as in normal hearing, with minor differences in processing mechanisms. The data contribute to our understanding of individual differences in speech perception and language understanding. The authors interpret their results within the ease of language understanding model.

Long-term musical experience and auditory and visual perceptual abilities under adverse conditions

Musicians have been shown to have enhanced speech perception in noise skills. It is unclear whether these improvements are limited to the auditory modality, as no research has examined musicians' visual perceptual abilities under degraded conditions. The current study examined associations between long-term musical experience and visual perception under noisy or degraded conditions. The performance of 11 musicians and 11 age-matched nonmusicians was compared on several auditory and visual perceptions in noise measures. Auditory perception tests included speech-in-noise tests and an environmental sound in noise test. Visual perception tasks included a fragmented sentences task, an object recognition task, and a lip-reading measure. Participants' vocabulary knowledge and nonverbal reasoning abilities were also assessed. Musicians outperformed nonmusicians on the speech perception in noise measures as well as the visual fragmented sentences task. Musicians also displayed better vocabulary knowledge in comparison to nonmusicians. Associations were found between perception of speech and visually degraded text. The findings show that long-term musical experience is associated with modality-general improvements in perceptual abilities. Possible systems supporting musicians' perceptual abilities are discussed.

Integration of partial information for spoken and written sentence recognition by older listeners

Older adults have difficulty understanding speech in challenging listening environments. Combining multisensory signals may facilitate speech recognition. This study measured recognition of interrupted spoken and written sentences by older adults for different preserved stimulus proportions. Unimodal performance was first examined when only interrupted text or speech stimuli were presented. Multimodal performance with concurrently presented text and speech stimuli was tested with delayed and simultaneous participant responses. Older listeners performed better in unimodal speech-only compared to text-only conditions across all proportions preserved. Performance was also better in delayed multimodal conditions. Comparison to a younger sample suggests age-related amodal processing declines.

Text as a Supplement to Speech in Young and Older Adults

OBJECTIVE

The purpose of this experiment was to quantify the contribution of visual text to auditory speech recognition in background noise. Specifically, the authors tested the hypothesis that partially accurate visual text from an automatic speech recognizer could be used successfully to supplement speech understanding in difficult listening conditions in older adults, with normal or impaired hearing. The working hypotheses were based on what is known regarding audiovisual speech perception in the elderly from speechreading literature. We hypothesized that (1) combining auditory and visual text information will result in improved recognition accuracy compared with auditory or visual text information alone, (2) benefit from supplementing speech with visual text (auditory and visual enhancement) in young adults will be greater than that in older adults, and (3) individual differences in performance on perceptual measures would be associated with cognitive abilities.

DESIGN

Fifteen young adults with normal hearing, 15 older adults with normal hearing, and 15 older adults with hearing loss participated in this study. All participants completed sentence recognition tasks in auditory-only, text-only, and combined auditory-text conditions. The auditory sentence stimuli were spectrally shaped to restore audibility for the older participants with impaired hearing. All participants also completed various cognitive measures, including measures of working memory, processing speed, verbal comprehension, perceptual and cognitive speed, processing efficiency, inhibition, and the ability to form wholes from parts. Group effects were examined for each of the perceptual and cognitive measures. Audiovisual benefit was calculated relative to performance on auditory- and visual-text only conditions. Finally, the relationship between perceptual measures and other independent measures were examined using principal-component factor analyses, followed by regression analyses.

RESULTS

Both young and older adults performed similarly on 9 out of 10 perceptual measures (auditory, visual, and combined measures). Combining degraded speech with partially correct text from an automatic speech recognizer improved the understanding of speech in both young and older adults, relative to both auditory- and text-only performance. In all subjects, cognition emerged as a key predictor for a general speech-text integration ability.

CONCLUSIONS

These results suggest that neither age nor hearing loss affected the ability of subjects to benefit from text when used to support speech, after ensuring audibility through spectral shaping. These results also suggest that the benefit obtained by supplementing auditory input with partially accurate text is modulated by cognitive ability, specifically lexical and verbal skills.

Lexical Influences on Spoken Spondaic Word Recognition in Hearing-Impaired Patients

Top-down contextual influences play a major part in speech understanding, especially in hearing-impaired patients with deteriorated auditory input. Those influences are most obvious in difficult listening situations, such as listening to sentences in noise but can also be observed at the word level under more favorable conditions, as in one of the most commonly used tasks in audiology, i.e., repeating isolated words in silence. This study aimed to explore the role of top-down contextual influences and their dependence on lexical factors and patient-specific factors using standard clinical linguistic material. Spondaic word perception was tested in 160 hearing-impaired patients aged 23-88 years with a four-frequency average pure-tone threshold ranging from 21 to 88 dB HL. Sixty spondaic words were randomly presented at a level adjusted to correspond to a speech perception score ranging between 40 and 70% of the performance intensity function obtained using monosyllabic words. Phoneme and whole-word recognition scores were used to calculate two context-influence indices (the j factor and the ratio of word scores to phonemic scores) and were correlated with linguistic factors, such as the phonological neighborhood density and several indices of word occurrence frequencies. Contextual influence was greater for spondaic words than in similar studies using monosyllabic words, with an overall j factor of 2.07 (SD = 0.5). For both indices, context use decreased with increasing hearing loss once the average hearing loss exceeded 55 dB HL. In right-handed patients, significantly greater context influence was observed for words presented in the right ears than for words presented in the left, especially in patients with many years of education. The correlations between raw word scores (and context influence indices) and word occurrence frequencies showed a significant age-dependent effect, with a stronger correlation between perception scores and word occurrence frequencies when the occurrence frequencies were based on the years corresponding to the patients' youth, showing a "historic" word frequency effect. This effect was still observed for patients with few years of formal education, but recent occurrence frequencies based on current word exposure had a stronger influence for those patients, especially for younger ones.

Integration of Partial Information Within and Across Modalities: Contributions to Spoken and Written Sentence Recognition

PURPOSE

This study evaluated the extent to which partial spoken or written information facilitates sentence recognition under degraded unimodal and multimodal conditions.

METHOD

Twenty young adults with typical hearing completed sentence recognition tasks in unimodal and multimodal conditions across 3 proportions of preservation. In the unimodal condition, performance was examined when only interrupted text or interrupted speech stimuli were available. In the multimodal condition, performance was examined when both interrupted text and interrupted speech stimuli were concurrently presented. Sentence recognition scores were obtained from simultaneous and delayed response conditions.

RESULTS

Significantly better performance was obtained for unimodal speech-only compared with text-only conditions across all proportions preserved. The multimodal condition revealed better performance when responses were delayed. During simultaneous responses, participants received equal benefit from speech information when the text was moderately and significantly degraded. The benefit from text in degraded auditory environments occurred only when speech was highly degraded.

CONCLUSIONS

The speech signal, compared with text, is robust against degradation likely due to its continuous, versus discrete, features. Allowing time for offline linguistic processing is beneficial for the recognition of partial sensory information in unimodal and multimodal conditions. Despite the perceptual differences between the 2 modalities, the results highlight the utility of multimodal speech + text signals.

Recognition performance on words interrupted (10 ips, 50% duty cycle) with two interruption patterns referenced to word onset: Young listeners with normal hearing for pure tones and older listeners with sensorineural hearing loss

OBJECTIVE

To determine in an interrupted word paradigm (Maryland CNCs; 10 ips, 50% duty cycle) if different locations of the interruption pattern produce different recognition performances.

DESIGN

Repeated measures using two interruption patterns that were complementary halves referenced to word onset; one started with word onset (0-ms), the other started 50 ms later. The hypothesis was that recognition performance would be better on the 0-ms condition than on the 50-ms condition, but there would be some words with the reverse relation.

STUDY SAMPLE

Twenty-four young adults with normal hearing for pure tones and 32 older adults (mean = 67 years) with sensorineural hearing loss participated.

RESULTS

With the young listeners mean recognition performance on the 0-ms condition (63.1%) was significantly better than the mean performance on the 50-ms condition (47.8%). About twice as many words had better performance on the 0-ms condition. With the older listeners, who were given only stimuli on which performances were > 58% by the young normals, performances on the two conditions were the same.

CONCLUSIONS

The hypothesis was supported with the young listeners. The equal performance by the older listeners on the two conditions was attributed to the manner in which the words were selected.

Age-related cochlear synaptopathy: an early-onset contributor to auditory functional decline

Aging listeners experience greater difficulty understanding speech in adverse listening conditions and exhibit degraded temporal resolution, even when audiometric thresholds are normal. When threshold evidence for peripheral involvement is lacking, central and cognitive factors are often cited as underlying performance declines. However, previous work has uncovered widespread loss of cochlear afferent synapses and progressive cochlear nerve degeneration in noise-exposed ears with recovered thresholds and no hair cell loss (Kujawa and Liberman 2009). Here, we characterize age-related cochlear synaptic and neural degeneration in CBA/CaJ mice never exposed to high-level noise. Cochlear hair cell and neuronal function was assessed via distortion product otoacoustic emissions and auditory brainstem responses, respectively. Immunostained cochlear whole mounts and plastic-embedded sections were studied by confocal and conventional light microscopy to quantify hair cells, cochlear neurons, and synaptic structures, i.e., presynaptic ribbons and postsynaptic glutamate receptors. Cochlear synaptic loss progresses from youth (4 weeks) to old age (144 weeks) and is seen throughout the cochlea long before age-related changes in thresholds or hair cell counts. Cochlear nerve loss parallels the synaptic loss, after a delay of several months. Key functional clues to the synaptopathy are available in the neural response; these can be accessed noninvasively, enhancing the possibilities for translation to human clinical characterization.

How linguistic closure and verbal working memory relate to speech recognition in noise--a review

The ability to recognize masked speech, commonly measured with a speech reception threshold (SRT) test, is associated with cognitive processing abilities. Two cognitive factors frequently assessed in speech recognition research are the capacity of working memory (WM), measured by means of a reading span (Rspan) or listening span (Lspan) test, and the ability to read masked text (linguistic closure), measured by the text reception threshold (TRT). The current article provides a review of recent hearing research that examined the relationship of TRT and WM span to SRTs in various maskers. Furthermore, modality differences in WM capacity assessed with the Rspan compared to the Lspan test were examined and related to speech recognition abilities in an experimental study with young adults with normal hearing (NH). Span scores were strongly associated with each other, but were higher in the auditory modality. The results of the reviewed studies suggest that TRT and WM span are related to each other, but differ in their relationships with SRT performance. In NH adults of middle age or older, both TRT and Rspan were associated with SRTs in speech maskers, whereas TRT better predicted speech recognition in fluctuating nonspeech maskers. The associations with SRTs in steady-state noise were inconclusive for both measures. WM span was positively related to benefit from contextual information in speech recognition, but better TRTs related to less interference from unrelated cues. Data for individuals with impaired hearing are limited, but larger WM span seems to give a general advantage in various listening situations.