Streaming of vowel sequences based on fundamental frequency in a cochlear-implant simulation

Lexical effects on talker discrimination in adult cochlear implant usersa)

The lexical and phonological content of an utterance impacts the processing of talker-specific details in normal-hearing (NH) listeners. Adult cochlear implant (CI) users demonstrate difficulties in talker discrimination, particularly for same-gender talker pairs, which may alter the reliance on lexical information in talker discrimination. The current study examined the effect of lexical content on talker discrimination in 24 adult CI users. In a remote AX talker discrimination task, word pairs-produced either by the same talker (ST) or different talkers with the same (DT-SG) or mixed genders (DT-MG)-were either lexically easy (high frequency, low neighborhood density) or lexically hard (low frequency, high neighborhood density). The task was completed in quiet and multi-talker babble (MTB). Results showed an effect of lexical difficulty on talker discrimination, for same-gender talker pairs in both quiet and MTB. CI users showed greater sensitivity in quiet as well as less response bias in both quiet and MTB for lexically easy words compared to lexically hard words. These results suggest that CI users make use of lexical content in same-gender talker discrimination, providing evidence for the contribution of linguistic information to the processing of degraded talker information by adult CI users.

Rapid Assessment of Non-Verbal Auditory Perception in Normal-Hearing Participants and Cochlear Implant Users

In the case of hearing loss, cochlear implants (CI) allow for the restoration of hearing. Despite the advantages of CIs for speech perception, CI users still complain about their poor perception of their auditory environment. Aiming to assess non-verbal auditory perception in CI users, we developed five listening tests. These tests measure pitch change detection, pitch direction identification, pitch short-term memory, auditory stream segregation, and emotional prosody recognition, along with perceived intensity ratings. In order to test the potential benefit of visual cues for pitch processing, the three pitch tests included half of the trials with visual indications to perform the task. We tested 10 normal-hearing (NH) participants with material being presented as original and vocoded sounds, and 10 post-lingually deaf CI users. With the vocoded sounds, the NH participants had reduced scores for the detection of small pitch differences, and reduced emotion recognition and streaming abilities compared to the original sounds. Similarly, the CI users had deficits for small differences in the pitch change detection task and emotion recognition, as well as a decreased streaming capacity. Overall, this assessment allows for the rapid detection of specific patterns of non-verbal auditory perception deficits. The current findings also open new perspectives about how to enhance pitch perception capacities using visual cues.

Effect of Vowel Auditory Training on the Speech-In-Noise Perception among Older Adults with Normal Hearing

Introduction:

Aging reduces the ability to understand speech in noise. Hearing rehabilitation is one of the ways to help older people communicate effectively. This study aimed to investigate the effect of vowel auditory training on the improvement of speech-in-noise (SIN) perception among elderly listeners.

Materials and Methods:

This study was conducted on 36 elderly listeners (17 males and 15 females) with the mean±SD of 67.6±6.33. They had the normal peripheral auditory ability but had difficulties in SIN perception. The samples were randomly divided into two groups of intervention and control. The intervention group underwent vowel auditory training; however, the control group received no training.

Results:

After vowel auditory training, the intervention group showed significant changes in the results of the SIN test at two signal-to-noise ratios of 0 and -10 and the Iranian version of the Speech, Spatial, and Qualities of Hearing Scale, compared to the control group (P<0.001). Regarding the Speech Auditory Brainstem Response test, the F₀ magnitude was higher in the intervention group (8.42±2.26), compared to the control group (6.68±1.87) (P<0.011).

Conclusion:

This study investigated the effect of vowel auditory training on the improvement of SIN perception which could be probably due to better F₀ encoding and receiving. This ability enhancement resulted in the easier perception of speech and its more proper separation from background noise which in turn enhanced the ability of the old people to follow the speech of a specific person and track the discussion.

The impact of speaking style on speech recognition in quiet and multi-talker babble in adult cochlear implant users

The current study examined sentence recognition across speaking styles (conversational, neutral, and clear) in quiet and multi-talker babble (MTB) for cochlear implant (CI) users and normal-hearing listeners under CI simulations. Listeners demonstrated poorer recognition accuracy in MTB than in quiet, but were relatively more accurate with clear speech overall. Within CI users, higher-performing participants were also more accurate in MTB when listening to clear speech. Lower performing users' accuracy was not impacted by speaking style. Clear speech may facilitate recognition in MTB for high-performing users, who may be better able to take advantage of clear speech cues.

The Sound of a Cochlear Implant Investigated in Patients With Single-Sided Deafness and a Cochlear Implant

HYPOTHESIS

A cochlear implant (CI) restores hearing in patients with profound sensorineural hearing loss by electrical stimulation of the auditory nerve. It is unknown how this electrical stimulation sounds.

BACKGROUND

Patients with single-sided deafness (SSD) and a CI form a unique population, since they can compare the sound of their CI with simulations of the CI sound played to their nonimplanted ear.

METHODS

We tested six stimuli (speech and music) in 10 SSD patients implanted with a CI (Cochlear Ltd). Patients listened to the original stimulus with their CI ear while their nonimplanted ear was masked. Subsequently, patients listened to two CI simulations, created with a vocoder, with their nonimplanted ear alone. They selected the CI simulation with greatest similarity to the sound as perceived by their CI ear and they graded similarity on a 1 to 10 scale. We tested three vocoders: two known from the literature, and one supplied by Cochlear Ltd. Two carriers (noise, sine) were tested for each vocoder.

RESULTS

Carrier noise and the vocoders from the literature were most often selected as best match to the sound as perceived by the CI ear. However, variability in selections was substantial both between patients and within patients between sound samples. The average grade for similarity was 6.8 for speech stimuli and 6.3 for music stimuli.

CONCLUSION

We obtained a fairly good impression of what a CI can sound like for SSD patients. This may help to better inform and educate patients and family members about the sound of a CI.

Discrimination of Voice Pitch and Vocal-Tract Length in Cochlear Implant Users

OBJECTIVES

When listening to two competing speakers, normal-hearing (NH) listeners can take advantage of voice differences between the speakers. Users of cochlear implants (CIs) have difficulty in perceiving speech on speech. Previous literature has indicated sensitivity to voice pitch (related to fundamental frequency, F0) to be poor among implant users, while sensitivity to vocal-tract length (VTL; related to the height of the speaker and formant frequencies), the other principal voice characteristic, has not been directly investigated in CIs. A few recent studies evaluated F0 and VTL perception indirectly, through voice gender categorization, which relies on perception of both voice cues. These studies revealed that, contrary to prior literature, CI users seem to rely exclusively on F0 while not utilizing VTL to perform this task. The objective of the present study was to directly and systematically assess raw sensitivity to F0 and VTL differences in CI users to define the extent of the deficit in voice perception.

DESIGN

The just-noticeable differences (JNDs) for F0 and VTL were measured in 11 CI listeners using triplets of consonant-vowel syllables in an adaptive three-alternative forced choice method.

RESULTS

The results showed that while NH listeners had average JNDs of 1.95 and 1.73 semitones (st) for F0 and VTL, respectively, CI listeners showed JNDs of 9.19 and 7.19 st. These JNDs correspond to differences of 70% in F0 and 52% in VTL. For comparison to the natural range of voices in the population, the F0 JND in CIs remains smaller than the typical male-female F0 difference. However, the average VTL JND in CIs is about twice as large as the typical male-female VTL difference.

CONCLUSIONS

These findings, thus, directly confirm that CI listeners do not seem to have sufficient access to VTL cues, likely as a result of limited spectral resolution, and, hence, that CI listeners' voice perception deficit goes beyond poor perception of F0. These results provide a potential common explanation not only for a number of deficits observed in CI listeners, such as voice identification and gender categorization, but also for competing speech perception.

The role of envelope periodicity in the perception of masked speech with simulated and real cochlear implants

In normal hearing, complex tones with pitch-related periodic envelope modulations are far less effective maskers of speech than aperiodic noise. Here, it is shown that this masker-periodicity benefit is diminished in noise-vocoder simulations of cochlear implants (CIs) and further reduced with real CIs. Nevertheless, both listener groups still benefitted significantly from masker periodicity, despite the lack of salient spectral pitch cues. The main reason for the smaller effect observed in CI users is thought to be an even stronger channel interaction than in the CI simulations, which smears out the random envelope modulations that are characteristic for aperiodic sounds. In contrast, neither interferers that were amplitude-modulated at a rate of 10 Hz nor maskers with envelopes specifically designed to reveal the target speech enabled a masking release in CI users. Hence, even at the high signal-to-noise ratios at which they were tested, CI users can still exploit pitch cues transmitted by the temporal envelope of a non-speech masker, whereas slow amplitude modulations of the masker envelope are no longer helpful.

Auditory stream segregation of iterated rippled noises by normal-hearing and hearing-impaired listeners

Individuals with hearing loss are thought to be less sensitive to the often subtle variations of acoustic information that support auditory stream segregation. Perceptual segregation can be influenced by differences in both the spectral and temporal characteristics of interleaved stimuli. The purpose of this study was to determine what stimulus characteristics support sequential stream segregation by normal-hearing and hearing-impaired listeners. Iterated rippled noises (IRNs) were used to assess the effects of tonality, spectral resolvability, and hearing loss on the perception of auditory streams in two pitch regions, corresponding to 250 and 1000 Hz. Overall, listeners with hearing loss were significantly less likely to segregate alternating IRNs into two auditory streams than were normally hearing listeners. Low pitched IRNs were generally less likely to segregate into two streams than were higher pitched IRNs. High-pass filtering was a strong contributor to reduced segregation for both groups. The tonality, or pitch strength, of the IRNs had a significant effect on streaming, but the effect was similar for both groups of subjects. These data demonstrate that stream segregation is influenced by many factors including pitch differences, pitch region, spectral resolution, and degree of stimulus tonality, in addition to the loss of auditory sensitivity.

Neural Entrainment to Speech Modulates Speech Intelligibility

Speech is crucial for communication in everyday life. Speech-brain entrainment, the alignment of neural activity to the slow temporal fluctuations (envelope) of acoustic speech input, is a ubiquitous element of current theories of speech processing. Associations between speech-brain entrainment and acoustic speech signal, listening task, and speech intelligibility have been observed repeatedly. However, a methodological bottleneck has prevented so far clarifying whether speech-brain entrainment contributes functionally to (i.e., causes) speech intelligibility or is merely an epiphenomenon of it. To address this long-standing issue, we experimentally manipulated speech-brain entrainment without concomitant acoustic and task-related variations, using a brain stimulation approach that enables modulating listeners' neural activity with transcranial currents carrying speech-envelope information. Results from two experiments involving a cocktail-party-like scenario and a listening situation devoid of aural speech-amplitude envelope input reveal consistent effects on listeners' speech-recognition performance, demonstrating a causal role of speech-brain entrainment in speech intelligibility. Our findings imply that speech-brain entrainment is critical for auditory speech comprehension and suggest that transcranial stimulation with speech-envelope-shaped currents can be utilized to modulate speech comprehension in impaired listening conditions.

Neural Correlates of Speech Segregation Based on Formant Frequencies of Adjacent Vowels

The neural substrates by which speech sounds are perceptually segregated into distinct streams are poorly understood. Here, we recorded high-density scalp event-related potentials (ERPs) while participants were presented with a cyclic pattern of three vowel sounds (/ee/-/ae/-/ee/). Each trial consisted of an adaptation sequence, which could have either a small, intermediate, or large difference in first formant (Δf₁) as well as a test sequence, in which Δf₁ was always intermediate. For the adaptation sequence, participants tended to hear two streams (“streaming”) when Δf₁ was intermediate or large compared to when it was small. For the test sequence, in which Δf₁ was always intermediate, the pattern was usually reversed, with participants hearing a single stream with increasing Δf₁ in the adaptation sequences. During the adaptation sequence, Δf₁-related brain activity was found between 100–250 ms after the /ae/ vowel over fronto-central and left temporal areas, consistent with generation in auditory cortex. For the test sequence, prior stimulus modulated ERP amplitude between 20–150 ms over left fronto-central scalp region. Our results demonstrate that the proximity of formants between adjacent vowels is an important factor in the perceptual organization of speech, and reveal a widely distributed neural network supporting perceptual grouping of speech sounds.

Dual-carrier processing to convey temporal fine structure cues: Implications for cochlear implants

Speech intelligibility in noise can be degraded by using vocoder processing to alter the temporal fine structure (TFS). Here it is argued that this degradation is not attributable to the loss of speech information potentially present in the TFS. Instead it is proposed that the degradation results from the loss of sound-source segregation information when two or more carriers (i.e., TFS) are substituted with only one as a consequence of vocoder processing. To demonstrate this segregation role, vocoder processing involving two carriers, one for the target and one for the background, was implemented. Because this approach does not preserve the speech TFS, it may be assumed that any improvement in intelligibility can only be a consequence of the preserved carrier duality and associated segregation cues. Three experiments were conducted using this "dual-carrier" approach. All experiments showed substantial sentence intelligibility in noise improvements compared to traditional single-carrier conditions. In several conditions, the improvement was so substantial that intelligibility approximated that for unprocessed speech in noise. A foreseeable and potentially promising implication for the dual-carrier approach involves implementation into cochlear implant speech processors, where it may provide the TFS cues necessary to segregate speech from noise.

Room and head coloration can induce obligatory stream segregation

Multiple sound reflections from room materials and a listener's head induce slight spectral modifications of sounds. This coloration depends on the listener and source positions, and on the room itself. This study investigated whether coloration could help segregate competing sources. Obligatory streaming was evaluated for diotic speech-shaped noises using a rhythmic discrimination task. Thresholds for detecting anisochrony were always significantly higher when stimuli differed in spectrum. The tested differences corresponded to three spatial configurations involving different levels of head and room coloration. These results suggest that, despite the generally deleterious effects of reverberation on speech intelligibility, coloration could favor source segregation.

Turning down the noise: the benefit of musical training on the aging auditory brain

Age-related decline in hearing abilities is a ubiquitous part of aging, and commonly impacts speech understanding, especially when there are competing sound sources. While such age effects are partially due to changes within the cochlea, difficulties typically exist beyond measurable hearing loss, suggesting that central brain processes, as opposed to simple peripheral mechanisms (e.g., hearing sensitivity), play a critical role in governing hearing abilities late into life. Current training regimens aimed to improve central auditory processing abilities have experienced limited success in promoting listening benefits. Interestingly, recent studies suggest that in young adults, musical training positively modifies neural mechanisms, providing robust, long-lasting improvements to hearing abilities as well as to non-auditory tasks that engage cognitive control. These results offer the encouraging possibility that musical training might be used to counteract age-related changes in auditory cognition commonly observed in older adults. Here, we reviewed studies that have examined the effects of age and musical experience on auditory cognition with an emphasis on auditory scene analysis. We infer that musical training may offer potential benefits to complex listening and might be utilized as a means to delay or even attenuate declines in auditory perception and cognition that often emerge later in life.

Age-related differences in the sequential organization of speech sounds

This study investigated the effects of age on listeners' tendency to group speech tokens into one or two auditory streams. Younger and older adults were presented with sequences of four vowel sounds, which were arranged according to the proximity of first-formant frequencies between adjacent vowels. In Experiment 1, participants were less accurate in identifying the order of the four vowels and more likely to report hearing two streams when the first-formant alternated between low and high frequency and the overall difference between adjacent vowels was large. This effect of first-formant continuity on temporal order judgments and probability of hearing two streams was higher in younger than in older adults. In Experiment 2, participants indicated whether there was rhythm irregularity in an otherwise isochronous sequence of four vowels. Young adults' thresholds were lower when successive first-formants ascended or descended monotonically (condition promoting integration) than when they alternated discontinuously (condition promoting streaming). This effect was not observed in older adults whose thresholds were comparable for both types of vowel sequences. These two experiments provide converging evidence for an age-related deficit in exploiting first-formant information between consecutive vowels, which appear to impede older adults' ability to sequentially group speech sounds over time.

Rate and onset cues can improve cochlear implant synthetic vowel recognition in noise

Understanding speech-in-noise is difficult for most cochlear implant (CI) users. Speech-in-noise segregation cues are well understood for acoustic hearing but not for electric hearing. This study investigated the effects of stimulation rate and onset delay on synthetic vowel-in-noise recognition in CI subjects. In experiment I, synthetic vowels were presented at 50, 145, or 795 pulse/s and noise at the same three rates, yielding nine combinations. Recognition improved significantly if the noise had a lower rate than the vowel, suggesting that listeners can use temporal gaps in the noise to detect a synthetic vowel. This hypothesis is supported by accurate prediction of synthetic vowel recognition using a temporal integration window model. Using lower rates a similar trend was observed in normal hearing subjects. Experiment II found that for CI subjects, a vowel onset delay improved performance if the noise had a lower or higher rate than the synthetic vowel. These results show that differing rates or onset times can improve synthetic vowel-in-noise recognition, indicating a need to develop speech processing strategies that encode or emphasize these cues.

Using Zebra-speech to study sequential and simultaneous speech segregation in a cochlear-implant simulation

Previous studies have suggested that cochlear implant users may have particular difficulties exploiting opportunities to glimpse clear segments of a target speech signal in the presence of a fluctuating masker. Although it has been proposed that this difficulty is associated with a deficit in linking the glimpsed segments across time, the details of this mechanism are yet to be explained. The present study introduces a method called Zebra-speech developed to investigate the relative contribution of simultaneous and sequential segregation mechanisms in concurrent speech perception, using a noise-band vocoder to simulate cochlear implants. One experiment showed that the saliency of the difference between the target and the masker is a key factor for Zebra-speech perception, as it is for sequential segregation. Furthermore, forward masking played little or no role, confirming that intelligibility was not limited by energetic masking but by across-time linkage abilities. In another experiment, a binaural cue was used to distinguish the target and the masker. It showed that the relative contribution of simultaneous and sequential segregation depended on the spectral resolution, with listeners relying more on sequential segregation when the spectral resolution was reduced. The potential of Zebra-speech as a segregation enhancement strategy for cochlear implants is discussed.

The effect of lip-reading on primary stream segregation

Lip-reading has been shown to improve the intelligibility of speech in multitalker situations, where auditory stream segregation naturally takes place. This study investigated whether the benefit of lip-reading is a result of a primary audiovisual interaction that enhances the obligatory streaming mechanism. Two behavioral experiments were conducted involving sequences of French vowels that alternated in fundamental frequency. In Experiment 1, subjects attempted to identify the order of items in a sequence. In Experiment 2, subjects attempted to detect a disruption to temporal isochrony across alternate items. Both tasks are disrupted by streaming, thus providing a measure of primary or obligatory streaming. Visual lip gestures articulating alternate vowels were synchronized with the auditory sequence. Overall, the results were consistent with the hypothesis that visual lip gestures enhance segregation by affecting primary auditory streaming. Moreover, increases in the naturalness of visual lip gestures and auditory vowels, and corresponding increases in audiovisual congruence may potentially lead to increases in the effect of visual lip gestures on streaming.

Use of intonation contours for speech recognition in noise by cochlear implant recipients

The corruption of intonation contours has detrimental effects on sentence-based speech recognition in normal-hearing listeners Binns and Culling [(2007). J. Acoust. Soc. Am. 122, 1765-1776]. This paper examines whether this finding also applies to cochlear implant (CI) recipients. The subjects' F0-discrimination and speech perception in the presence of noise were measured, using sentences with regular and inverted F0-contours. The results revealed that speech recognition for regular contours was significantly better than for inverted contours. This difference was related to the subjects' F0-discrimination providing further evidence that the perception of intonation patterns is important for the CI-mediated speech recognition in noise.

Objective and subjective psychophysical measures of auditory stream integration and segregation

The perceptual organization of sound sequences into auditory streams involves the integration of sounds into one stream and the segregation of sounds into separate streams. "Objective" psychophysical measures of auditory streaming can be obtained using behavioral tasks where performance is facilitated by segregation and hampered by integration, or vice versa. Traditionally, these two types of tasks have been tested in separate studies involving different listeners, procedures, and stimuli. Here, we tested subjects in two complementary temporal-gap discrimination tasks involving similar stimuli and procedures. One task was designed so that performance in it would be facilitated by perceptual integration; the other, so that performance would be facilitated by perceptual segregation. Thresholds were measured in both tasks under a wide range of conditions produced by varying three stimulus parameters known to influence stream formation: frequency separation, tone-presentation rate, and sequence length. In addition to these performance-based measures, subjective judgments of perceived segregation were collected in the same listeners under corresponding stimulus conditions. The patterns of results obtained in the two temporal-discrimination tasks, and the relationships between thresholds and perceived-segregation judgments, were mostly consistent with the hypothesis that stream segregation helped performance in one task and impaired performance in the other task. The tasks and stimuli described here may prove useful in future behavioral or neurophysiological experiments, which seek to manipulate and measure neural correlates of auditory streaming while minimizing differences between the physical stimuli.

Objective neural indices of speech-in-noise perception

Numerous factors contribute to understanding speech in noisy listening environments. There is a clinical need for objective biological assessment of auditory factors that contribute to the ability to hear speech in noise, factors that are free from the demands of attention and memory. Subcortical processing of complex sounds such as speech (auditory brainstem responses to speech and other complex stimuli [cABRs]) reflects the integrity of auditory function. Because cABRs physically resemble the evoking acoustic stimulus, they can provide objective indices of the neural transcription of specific acoustic elements (e.g., temporal, spectral) important for hearing speech. As with brainstem responses to clicks and tones, cABRs are clinically viable in individual subjects. Subcortical transcription of complex sounds is also clinically viable because of its known experience-dependence and role in auditory learning. Together with other clinical measures, cABRs can inform the underlying biological nature of listening and language disorders, inform treatment strategies, and provide an objective index of therapeutic outcomes. In this article, the authors review recent studies demonstrating the role of subcortical speech encoding in successful speech-in-noise perception.

Sequential stream segregation using temporal periodicity cues in cochlear implant recipients

Sequential stream segregation involves the ability of a listener to perceptually segregate two rapidly alternating sounds into different perceptual streams. By studying auditory streaming in cochlear implants (CIs), one can obtain a better understanding of the cues that CI recipients can use to segregate different sound sources, which may have relevance to such everyday activities as the understanding of speech in background noise. This study focuses on the ability of CI users to use temporal periodicity cues to perform auditory stream segregation. A rhythmic discrimination task involving sequences of alternating amplitude-modulated (AM) noises is used. The results suggest that most CI users can stream AM noise bursts at relatively low modulation frequencies (near 80 Hz AM), but that this ability diminishes at higher modulation frequencies. Additionally, the ability of CI users to perform streaming using temporal periodicity cues appears to be comparable to that of normal-hearing listeners. These results imply that CI subjects may in certain contexts (i.e., when the talker has a low fundamental frequency voice) be able to use temporal periodicity cues to segregate and thus understand the voices of competing talkers.

Concurrent-vowel and tone recognitions in acoustic and simulated electric hearing

Because of the poor spectral resolution in cochlear implants (CIs), fundamental frequency (F0) cues are not well preserved. Chinese-speaking CI users may have great difficulty understanding speech produced by competing talkers, due to conflicting tones. In this study, normal-hearing listeners' concurrent Chinese syllable recognition was measured with unprocessed speech and CI simulations. Concurrent syllables were constructed by summing two vowels from a male talker (with identical mean F0's) or one vowel from each of a male and a female talker (with a relatively large F0 separation). CI signal processing was simulated using four- and eight-channel noise-band vocoders; the degraded spectral resolution may limit listeners' ability to utilize talker and/or tone differences. The results showed that concurrent speech recognition was significantly poorer with the CI simulations than with unprocessed speech. There were significant interactions between the talker and speech-processing conditions, e.g., better tone and syllable recognitions with the male-female condition for unprocessed speech, and with the male-male condition for eight-channel speech. With the CI simulations, competing tones interfered with concurrent-tone and syllable recognitions, but not vowel recognition. Given limited pitch cues, subjects were unable to use F0 differences between talkers or tones for concurrent Chinese syllable recognition.