Enhancing temporal cues to voice pitch in continuous interleaved sampling cochlear implants

A Hundred Ways to Encode Sound Signals for Cochlear Implants

Cochlear implants are the most successful neural prostheses used to restore hearing in severe-to-profound hearing-impaired individuals. The field of cochlear implant coding investigates interdisciplinary approaches to translate acoustic signals into electrical pulses transmitted at the electrode-neuron interface, ranging from signal preprocessing algorithms, enhancement, and feature extraction methodologies to electric signal generation. In the last five decades, numerous coding strategies have been proposed clinically and experimentally. Initially developed to restore speech perception, increasing computational possibilities now allow coding of more complex signals, and new techniques to optimize the transmission of electrical signals are constantly gaining attention. This review provides insights into the history of multichannel coding and presents an extensive list of implemented strategies. The article briefly addresses each method and considers promising future directions of neural prostheses and possible signal processing, with the ultimate goal of providing a current big picture of the large field of cochlear implant coding.

Emotional prosodic expression of children with hearing aids or cochlear implants, rated by adults and peers

OBJECTIVE

The emotional prosodic expression potential of children with cochlear implants is poorer than that of normal hearing peers. Though little is known about children with hearing aids.

DESIGN

This study was set up to generate a better understanding of hearing aid users' prosodic identifiability compared to cochlear implant users and peers without hearing loss.

STUDY SAMPLE

Emotional utterances of 75 Dutch speaking children (7 - 12 yr; 26 CHA, 23 CCI, 26 CNH) were gathered. Utterances were evaluated blindly by normal hearing Dutch listeners: 22 children and 9 adults (17 - 24 yrs) for resemblance to three emotions (happiness, sadness, anger).

RESULTS

Emotions were more accurately recognised by adults than by children. Both children and adults correctly judged happiness significantly less often in CCI than in CNH. Also, adult listeners confused happiness with sadness more often in both CHA and CCI than in CNH.

CONCLUSIONS

Children and adults are able to accurately evaluate the emotions expressed through speech by children with varying degrees of hearing loss, ranging from mild to profound, nearly as well as they can with typically hearing children. The favourable outcomes emphasise the resilience of children with hearing loss in developing effective emotional communication skills.

Characterizing the relationship between modulation sensitivity and pitch resolution in cochlear implant users

Cochlear implants are medical devices that have restored hearing to approximately one million people around the world. Outcomes are impressive and most recipients attain excellent speech comprehension in quiet without relying on lip-reading cues, but pitch resolution is poor compared to normal hearing. Amplitude modulation of electrical stimulation is a primary cue for pitch perception in cochlear implant users. The experiments described in this article focus on the relationship between sensitivity to amplitude modulations and pitch resolution based on changes in the frequency of amplitude modulations. In the first experiment, modulation sensitivity and pitch resolution were measured in adults with no known hearing loss and in cochlear implant users with sounds presented to and processed by their clinical devices. Stimuli were amplitude-modulated sinusoids and amplitude-modulated narrow-band noises. Modulation detection and modulation frequency discrimination were measured for modulation frequencies centered on 110, 220, and 440 Hz. Pitch resolution based on changes in modulation frequency was measured for modulation depths of 25 %, 50 %, 100 %, and for a half-waved rectified modulator. Results revealed a strong linear relationship between modulation sensitivity and pitch resolution for cochlear implant users and peers with no known hearing loss. In the second experiment, cochlear implant users took part in analogous procedures of modulation sensitivity and pitch resolution but bypassing clinical sound processing using single-electrode stimulation. Results indicated that modulation sensitivity and pitch resolution was better conveyed by single-electrode stimulation than by clinical processors. Results at 440 Hz were worse, but also not well conveyed by clinical sound processing, so it remains unclear whether the 300 Hz perceptual limit described in the literature is a technological or biological limitation. These results highlight modulation depth and sensitivity as critical factors for pitch resolution in cochlear implant users and characterize the relationship that should inform the design of modulation enhancement algorithms for cochlear implants.

Comparison of Speech Perception Performance According to Prosody Change Between People With Normal Hearing and Cochlear Implant Users

BACKGROUND AND OBJECTIVES

Cochlear implants (CIs) are well known to improve audibility and speech recognition in individuals with hearing loss, but some individuals still struggle with many aspects in communication, such as prosody. This study explores how prosodic elements are perceived by those with normal hearing (NH) and CIs.

SUBJECTS AND METHODS

Thirteen individuals with NH and thirteen CI users participated in this study and completed speech perception, speech prosody perception, speech prosody production, pitch difference discrimination, and melodic contour perception testing.

RESULTS

NH listeners performed significantly better than CI users on speech perception, speech prosody perception (except for words with neutral meaning and a negative prosody change and when words were repeated twice), pitch difference discrimination, and melodic contour perception testing. No statistical significance was observed for speech prosody production for both groups.

CONCLUSIONS

Compared to NH listeners, CI users had limited ability to recognize prosodic elements. The study findings highlight the necessity of an assessment tool and signal processing algorithm for CIs, specifically targeting prosodic elements in clinical settings.

Communicating Emotion: Vocal Expression of Linguistic and Emotional Prosody in Children With Mild to Profound Hearing Loss Compared With That of Normal Hearing Peers

OBJECTIVES

Emotional prosody is known to play an important role in social communication. Research has shown that children with cochlear implants (CCIs) may face challenges in their ability to express prosody, as their expressions may have less distinct acoustic contrasts and therefore may be judged less accurately. The prosody of children with milder degrees of hearing loss, wearing hearing aids, has sparsely been investigated. More understanding of the prosodic expression by children with hearing loss, hearing aid users in particular, could create more awareness among healthcare professionals and parents on limitations in social communication, which awareness may lead to more targeted rehabilitation. This study aimed to compare the prosodic expression potential of children wearing hearing aids (CHA) with that of CCIs and children with normal hearing (CNH).

DESIGN

In this prospective experimental study, utterances of pediatric hearing aid users, cochlear implant users, and CNH containing emotional expressions (happy, sad, and angry) were recorded during a reading task. Of the utterances, three acoustic properties were calculated: fundamental frequency (F0), variance in fundamental frequency (SD of F0), and intensity. Acoustic properties of the utterances were compared within subjects and between groups.

RESULTS

A total of 75 children were included (CHA: 26, CCI: 23, and CNH: 26). Participants were between 7 and 13 years of age. The 15 CCI with congenital hearing loss had received the cochlear implant at median age of 8 months. The acoustic patterns of emotions uttered by CHA were similar to those of CCI and CNH. Only in CCI, we found no difference in F0 variation between happiness and anger, although an intensity difference was present. In addition, CCI and CHA produced poorer happy-sad contrasts than did CNH.

CONCLUSIONS

The findings of this study suggest that on a fundamental, acoustic level, both CHA and CCI have a prosodic expression potential that is almost on par with normal hearing peers. However, there were some minor limitations observed in the prosodic expression of these children, it is important to determine whether these differences are perceptible to listeners and could affect social communication. This study sets the groundwork for more research that will help us fully understand the implications of these findings and how they may affect the communication abilities of these children. With a clearer understanding of these factors, we can develop effective ways to help improve their communication skills.

A direct comparison of voice pitch processing in acoustic and electric hearing

In single-sided deafness patients fitted with a cochlear implant (CI) in the affected ear and preserved normal hearing in the other ear, acoustic and electric hearing can be directly compared without the need for an external control group. Although poor pitch perception is a crucial limitation when listening through CIs, it remains unclear how exactly the cortical processing of pitch information differs between acoustic and electric hearing. Hence, we separately presented both ears of 20 of these patients with vowel sequences in which the pitch contours were either repetitive or variable, while simultaneously recording functional near-infrared spectroscopy (fNIRS) and EEG data. Overall, the results showed smaller and delayed auditory cortex activity in electric hearing, particularly for the P2 event-related potential component, which appears to reflect the processing of voice pitch information. Both the fNIRS data and EEG source reconstructions furthermore showed that vowel sequences with variable pitch contours evoked additional activity in posterior right auditory cortex in electric but not acoustic hearing. This surprising discrepancy demonstrates, firstly, that the acoustic detail transmitted by CIs is sufficient to distinguish between speech sounds that only vary regarding their pitch information. Secondly, the absence of a condition difference when stimulating the normal-hearing ears suggests a saturation of cortical activity levels following unilateral deafness. Taken together, these results provide strong evidence in favour of using CIs in this patient group.

Modulation Depth Discrimination by Cochlear Implant Users

Cochlear implants (CIs) convey the amplitude envelope of speech by modulating high-rate pulse trains. However, not all of the envelope may be necessary to perceive amplitude modulations (AMs); the effective envelope depth may be limited by forward and backward masking from the envelope peaks. Three experiments used modulated pulse trains to measure which portions of the envelope can be effectively processed by CI users as a function of AM frequency. Experiment 1 used a three-interval forced-choice task to test the ability of CI users to discriminate less-modulated pulse trains from a fully modulated standard, without controlling for loudness. The stimuli in experiment 2 were identical, but a two-interval task was used in which participants were required to choose the less-modulated interval, ignoring loudness. Catch trials, in which judgements based on level or modulation depth would give opposing answers, were included. Experiment 3 employed novel stimuli whose modulation envelope could be modified below a variable point in the dynamic range, without changing the loudness of the stimulus. Overall, results showed that substantial portions of the envelope are not accurately encoded by CI users. In experiment 1, where loudness cues were available, participants on average were insensitive to changes in the bottom 30% of their dynamic range. In experiment 2, where loudness was controlled, participants appeared insensitive to changes in the bottom 50% of the dynamic range. In experiment 3, participants were insensitive to changes in the bottom 80% of the dynamic range. We discuss potential reasons for this insensitivity and implications for CI speech-processing strategies.

Mandarin tone recognition training with cochlear implant simulation: Amplitude envelope enhancement and cue weighting

With limited fundamental frequency (F0) cues, cochlear implant (CI) users recognize Mandarin tones using amplitude envelope. This study investigated whether tone recognition training with amplitude envelope enhancement may improve tone recognition and cue weighting with CIs. Three groups of CI-simulation listeners received training using vowels with amplitude envelope modified to resemble F0 contour (enhanced-amplitude-envelope training), training using natural vowels (natural-amplitude-envelope training), and exposure to natural vowels without training, respectively. Tone recognition with natural and enhanced amplitude envelope cues and cue weighting of amplitude envelope and F0 contour were measured in pre-, post-, and retention-tests. It was found that with similar pre-test performance, both training groups had better tone recognition than the no-training group after training. Only enhanced-amplitude-envelope training increased the benefits of amplitude envelope enhancement in the post- and retention-tests than in the pre-test. Neither training paradigm increased the cue weighting of amplitude envelope and F0 contour more than stimulus exposure. Listeners attending more to amplitude envelope in the pre-test tended to have better tone recognition with enhanced amplitude envelope cues before training and improve more in tone recognition after enhanced-amplitude-envelope training. The results suggest that auditory training and speech enhancement may bring maximum benefits to CI users when combined.

Further simulations of the effect of cochlear-implant pre-processing and head movement on interaural level differences

We simulated the effect of several automatic gain control (AGC) and AGC-like systems and head movement on the output levels, and resulting interaural level differences (ILDs) produced by bilateral cochlear-implant (CI) processors. The simulated AGC systems included unlinked AGCs with a range of parameter settings, linked AGCs, and two proprietary multi-channel systems used in contemporary CIs. The results show that over the range of values used clinically, the parameters that most strongly affect dynamic ILDs are the release time and compression ratio. Linking AGCs preserves ILDs at the expense of monaural level changes and, possibly, comfortable listening level. Multichannel AGCs can whiten output spectra, and/or distort the dynamic changes in ILD that occur during and after head movement. We propose that an unlinked compressor with a ratio of approximately 3:1 and a release time of 300-500 ms can preserve the shape of dynamic ILDs, without causing large spectral distortions or sacrificing listening comfort.

Place-Pitch Interval Perception With a Cochlear Implant

OBJECTIVES

Pitch is poorly perceived by cochlear implant (CI) users. However, as it is not well understood how pitch is encoded with electric stimulation, improving pitch representation with a CI is challenging. Changes in place of stimulation along the cochlea have been described as changes in pitch and can be accurately ranked by CI users. However, it remains unknown if place-pitch can be used to encode musical intervals, which are a necessary attribute of pitch. The objective of these experiments is to determine if place-pitch coding can be used to represent musical intervals with a CI.

DESIGN

In the first experiment, 10 CI users and 10 normal hearing (NH) controls were tested on their sensitivity to changes in the semitone spacing between each of the notes in the melody "Happy Birthday." The changes were implemented by uniformly expanding or compressing the frequency differences between each note in the melody. The participant's task was to scale how "out-of-tune" the melody was for various semitone spacing distortions. The notes were represented by pure-tones ≥440 Hz to minimize potential useful temporal information from the stimuli. A second experiment replicated the first experiment using single-sided deafened CI users allowing for a within-subject control. A third experiment verified that the CI users who participated in Experiment 1 were each able to determine pitch direction reliably.

RESULTS

Unlike NH listeners, CI listeners often ranked all distortions of interval spacing similarly in both the first and second experiment, and no effect of interval spacing was detected across CI users. Some participants found distorted interval spacings to be less out-of-tune than the nominally correct interval spacings. However, these patterns were inconsistent across listeners. Although performance was better for the NH listeners, the third experiment demonstrated that the CI listeners were able to reliably identify changes in pitch direction from place-pitch coding.

CONCLUSIONS

The data suggest that place-pitch intervals are not properly represented through a CI sound processor. Some limited support is found for place-pitch being useful for interval encoding as some participants demonstrated improved ratings for certain interval distortions. Presumably the interval representation for these participants could be improved by a change to the frequencies represented by each electrode. However, as these patterns vary across listeners, there is not a universal correction to frequency representation that will solve this issue. As results are similar for single-sided deafened CI users, the limitations in ratings are likely not limited by an eroded representation of the melody caused by an extended duration of deafness.

Neurophysiological Differences in Emotional Processing by Cochlear Implant Users, Extending Beyond the Realm of Speech

OBJECTIVE

Cochlear implants (CIs) restore a sense of hearing in deaf individuals. However, they do not transmit the acoustic signal with sufficient fidelity, leading to difficulties in recognizing emotions in voice and in music. The study aimed to explore the neurophysiological bases of these limitations.

DESIGN

Twenty-two adults (18 to 70 years old) with CIs and 22 age-matched controls with normal hearing participated. Event-related potentials (ERPs) were recorded in response to emotional bursts (happy, sad, or neutral) produced in each modality (voice or music) that were for the most part correctly identified behaviorally.

RESULTS

Compared to controls, the N1 and P2 components were attenuated and prolonged in CI users. To a smaller degree, N1 and P2 were also attenuated and prolonged in music compared to voice, in both populations. The N1-P2 complex was emotion-dependent (e.g., reduced and prolonged response to sadness), but this was also true in both populations. In contrast, the later portion of the response, between 600 and 850 ms, differentiated happy and sad from neutral stimuli in normal hearing but not in CI listeners.

CONCLUSIONS

The early portion of the ERP waveform reflected primarily the general reduction in sensory encoding by CI users (largely due to CI processing itself), whereas altered emotional processing (by CI users) could be found in the later portion of the ERP and extended beyond the realm of speech.

Temporal-pitch sensitivity in electric hearing with amplitude modulation and inserted pulses with short inter-pulse intervals

Listeners with cochlear implants (CIs) typically show poor sensitivity to the temporal-envelope pitch of high-rate pulse trains. Sensitivity to interaural time differences improves when adding pulses with short inter-pulse intervals (SIPIs) to high-rate pulse trains. In the current study, monaural temporal-pitch sensitivity with SIPI pulses was investigated for six CI listeners. Amplitude-modulated single-electrode stimuli, representing the coding of the fundamental frequency (F0) in the envelope of a high-rate carrier, were used. Two SIPI-insertion approaches, five modulation depths, two typical speech-F0s, and two carrier rates were tested. SIPI pulses were inserted either in every amplitude-modulation period (full-rate SIPI) to support the F0 cue or in every other amplitude-modulation period (half-rate SIPI) to circumvent a potential rate limitation at higher F0s. The results demonstrate that full-rate SIPI pulses improve temporal-pitch sensitivity across F0s and particularly at low modulation depths where envelope-pitch cues are weak. The half-rate SIPI pulses did not circumvent the limitation and further increased variability across listeners. Further, no effect of the carrier rate was found. Thus, the SIPI approach appears to be a promising approach to enhance CI listeners' access to temporal-envelope pitch cues at pulse rates used clinically.

Listeners' perception of lexical stress in the first words of infants with cochlear implants and normally hearing infants

Normally hearing (NH) infants are able to produce lexical stress in their first words, but congenitally hearing-impaired children with cochlear implants (CI) may find this more challenging, given the limited transmission of spectro-temporal information by the implant. Acoustic research has shown that the acoustic cues to stress in the first words of Dutch-acquiring CI infants are less pronounced (Pettinato, De Clerck, Verhoeven, & Gillis, 2017). The present study investigates how listeners perceive lexical stress in the first words of CI and NH infants. Two research questions are addressed: (1) How successful are CI and NH children in implementing the prosodic cues to prominence? (2) Is the degree of stress in CI and NH words perceived to be similar? The stimuli used in this study are disyllabic words (n = 1089) produced by 9 infants with CI and 9 NH infants acquiring Dutch. The words were presented to adult listeners in a listening experiment, in which they assessed the stress pattern on a continuous visual analogue scale (VAS) which expresses to what extent syllables are perceived as stressed. The results show that listeners perceive typical word stress production in the first words of infants with CI. The words of CI and NH infants were rated in agreement with the target stress pattern as often, and trochaic words were rated more frequently as such than iambic words. Listeners more frequently perceive unstressed syllables in the first words of infants with CI. However, for the words that are perceived to be clearly stressed, the degree of word stress is comparable in the two groups, and both infant groups are perceived to produce more contrast between stressed and unstressed syllables in trochees than in iambs. It is concluded that that acoustic differences between CI and NH infants' stress production are not necessarily perceptually salient.

Perceptual Discrimination of Speaking Style Under Cochlear Implant Simulation

OBJECTIVES

Real-life, adverse listening conditions involve a great deal of speech variability, including variability in speaking style. Depending on the speaking context, talkers may use a more casual, reduced speaking style or a more formal, careful speaking style. Attending to fine-grained acoustic-phonetic details characterizing different speaking styles facilitates the perception of the speaking style used by the talker. These acoustic-phonetic cues are poorly encoded in cochlear implants (CIs), potentially rendering the discrimination of speaking style difficult. As a first step to characterizing CI perception of real-life speech forms, the present study investigated the perception of different speaking styles in normal-hearing (NH) listeners with and without CI simulation.

DESIGN

The discrimination of three speaking styles (conversational reduced speech, speech from retold stories, and carefully read speech) was assessed using a speaking style discrimination task in two experiments. NH listeners classified sentence-length utterances, produced in one of the three styles, as either formal (careful) or informal (conversational). Utterances were presented with unmodified speaking rates in experiment 1 (31 NH, young adult Dutch speakers) and with modified speaking rates set to the average rate across all utterances in experiment 2 (28 NH, young adult Dutch speakers). In both experiments, acoustic noise-vocoder simulations of CIs were used to produce 12-channel (CI-12) and 4-channel (CI-4) vocoder simulation conditions, in addition to a no-simulation condition without CI simulation.

RESULTS

In both experiments 1 and 2, NH listeners were able to reliably discriminate the speaking styles without CI simulation. However, this ability was reduced under CI simulation. In experiment 1, participants showed poor discrimination of speaking styles under CI simulation. Listeners used speaking rate as a cue to make their judgements, even though it was not a reliable cue to speaking style in the study materials. In experiment 2, without differences in speaking rate among speaking styles, listeners showed better discrimination of speaking styles under CI simulation, using additional cues to complete the task.

CONCLUSIONS

The findings from the present study demonstrate that perceiving differences in three speaking styles under CI simulation is a difficult task because some important cues to speaking style are not fully available in these conditions. While some cues like speaking rate are available, this information alone may not always be a reliable indicator of a particular speaking style. Some other reliable speaking styles cues, such as degraded acoustic-phonetic information and variability in speaking rate within an utterance, may be available but less salient. However, as in experiment 2, listeners' perception of speaking styles may be modified if they are constrained or trained to use these additional cues, which were more reliable in the context of the present study. Taken together, these results suggest that dealing with speech variability in real-life listening conditions may be a challenge for CI users.

Effect of signal processing strategy and stimulation type on speech and auditory perception in adult cochlear implant users

OBJECTIVE

The objective of this study was to investigate the effects of signal processing strategy on speech understanding and auditory function for adult cochlear implant (CI) users with a focus on the effects of sequential versus paired stimulation.

DESIGN

Within-subjects, repeated measures design was utilised to compare performance between processing strategies and stimulation type on various measures of auditory function and subjective sound quality. Testing with subsequent strategies was completed after a total familiarisation time of two weeks.

STUDY SAMPLE

Ten post-lingually deafened adult CI users were recruited from a clinical population. Participants had a minimum of 13 months CI experience. Ages ranged from 25-78 years. All participants had long-term experience with the optima strategy; eight with sequential stimulation and two with paired stimulation.

RESULTS

We found no statistically significant effect of processing strategy. We observed an effect of stimulation type with sequential stimulation yielding significantly higher performance than paired stimulation for speech understanding in quiet and in noise, and subjective estimates of sound quality. No significant differences were noted across strategy or stimulation for music perception, spectral resolution or temporal resolution.

CONCLUSIONS

Many patients utilise paired stimulation - the default stimulation type in the clinical software; however, sequential stimulation yielded significantly higher outcomes on multiple measures.

Multimodal and Spectral Degradation Effects on Speech and Emotion Recognition in Adult Listeners

For cochlear implant (CI) users, degraded spectral input hampers the understanding of prosodic vocal emotion, especially in difficult listening conditions. Using a vocoder simulation of CI hearing, we examined the extent to which informative multimodal cues in a talker’s spoken expressions improve normal hearing (NH) adults’ speech and emotion perception under different levels of spectral degradation (two, three, four, and eight spectral bands). Participants repeated the words verbatim and identified emotions (among four alternative options: happy, sad, angry, and neutral) in meaningful sentences that are semantically congruent with the expression of the intended emotion. Sentences were presented in their natural speech form and in speech sampled through a noise-band vocoder in sound (auditory-only) and video (auditory–visual) recordings of a female talker. Visual information had a more pronounced benefit in enhancing speech recognition in the lower spectral band conditions. Spectral degradation, however, did not interfere with emotion recognition performance when dynamic visual cues in a talker’s expression are provided as participants scored at ceiling levels across all spectral band conditions. Our use of familiar sentences that contained congruent semantic and prosodic information have high ecological validity, which likely optimized listener performance under simulated CI hearing and may better predict CI users’ outcomes in everyday listening contexts.

Discrimination of Voice Pitch and Vocal-Tract Length in Cochlear Implant Users

OBJECTIVES

When listening to two competing speakers, normal-hearing (NH) listeners can take advantage of voice differences between the speakers. Users of cochlear implants (CIs) have difficulty in perceiving speech on speech. Previous literature has indicated sensitivity to voice pitch (related to fundamental frequency, F0) to be poor among implant users, while sensitivity to vocal-tract length (VTL; related to the height of the speaker and formant frequencies), the other principal voice characteristic, has not been directly investigated in CIs. A few recent studies evaluated F0 and VTL perception indirectly, through voice gender categorization, which relies on perception of both voice cues. These studies revealed that, contrary to prior literature, CI users seem to rely exclusively on F0 while not utilizing VTL to perform this task. The objective of the present study was to directly and systematically assess raw sensitivity to F0 and VTL differences in CI users to define the extent of the deficit in voice perception.

DESIGN

The just-noticeable differences (JNDs) for F0 and VTL were measured in 11 CI listeners using triplets of consonant-vowel syllables in an adaptive three-alternative forced choice method.

RESULTS

The results showed that while NH listeners had average JNDs of 1.95 and 1.73 semitones (st) for F0 and VTL, respectively, CI listeners showed JNDs of 9.19 and 7.19 st. These JNDs correspond to differences of 70% in F0 and 52% in VTL. For comparison to the natural range of voices in the population, the F0 JND in CIs remains smaller than the typical male-female F0 difference. However, the average VTL JND in CIs is about twice as large as the typical male-female VTL difference.

CONCLUSIONS

These findings, thus, directly confirm that CI listeners do not seem to have sufficient access to VTL cues, likely as a result of limited spectral resolution, and, hence, that CI listeners' voice perception deficit goes beyond poor perception of F0. These results provide a potential common explanation not only for a number of deficits observed in CI listeners, such as voice identification and gender categorization, but also for competing speech perception.

A tonal-language benefit for pitch in normally-hearing and cochlear-implanted children

In tonal languages, voice pitch inflections change the meaning of words, such that the brain processes pitch not merely as an acoustic characterization of sound but as semantic information. In normally-hearing (NH) adults, this linguistic pressure on pitch appears to sharpen its neural encoding and can lead to perceptual benefits, depending on the task relevance, potentially generalizing outside of the speech domain. In children, however, linguistic systems are still malleable, meaning that their encoding of voice pitch information might not receive as much neural specialization but might generalize more easily to ecologically irrelevant pitch contours. This would seem particularly true for early-deafened children wearing a cochlear implant (CI), who must exhibit great adaptability to unfamiliar sounds as their sense of pitch is severely degraded. Here, we provide the first demonstration of a tonal language benefit in dynamic pitch sensitivity among NH children (using both a sweep discrimination and labelling task) which extends partially to children with CI (i.e., in the labelling task only). Strong age effects suggest that sensitivity to pitch contours reaches adult-like levels early in tonal language speakers (possibly before 6 years of age) but continues to develop in non-tonal language speakers well into the teenage years. Overall, we conclude that language-dependent neuroplasticity can enhance behavioral sensitivity to dynamic pitch, even in extreme cases of auditory degradation, but it is most easily observable early in life.

The role of envelope periodicity in the perception of masked speech with simulated and real cochlear implants

In normal hearing, complex tones with pitch-related periodic envelope modulations are far less effective maskers of speech than aperiodic noise. Here, it is shown that this masker-periodicity benefit is diminished in noise-vocoder simulations of cochlear implants (CIs) and further reduced with real CIs. Nevertheless, both listener groups still benefitted significantly from masker periodicity, despite the lack of salient spectral pitch cues. The main reason for the smaller effect observed in CI users is thought to be an even stronger channel interaction than in the CI simulations, which smears out the random envelope modulations that are characteristic for aperiodic sounds. In contrast, neither interferers that were amplitude-modulated at a rate of 10 Hz nor maskers with envelopes specifically designed to reveal the target speech enabled a masking release in CI users. Hence, even at the high signal-to-noise ratios at which they were tested, CI users can still exploit pitch cues transmitted by the temporal envelope of a non-speech masker, whereas slow amplitude modulations of the masker envelope are no longer helpful.

A comparison of speech intonation production and perception abilities of Farsi speaking cochlear implanted and normal hearing children

INTRODUCTION

Cochlear implant prosthesis facilitates spoken language development and speech comprehension in children with severe-profound hearing loss. However, this prosthesis is limited in encoding information about fundamental frequency and pitch that are essentially for recognition of speech prosody. The purpose of the present study is to investigate the perception and production of intonation in cochlear implant children and comparison with normal hearing children.

METHOD

This study carried out on 25 cochlear implanted children and 50 children with normal hearing. First, using 10 action pictures statements and questions sentences were extracted. Fundamental frequency and pitch changes were identified using Praat software. Then, these sentences were judged by 7 adult listeners. In second stage 20 sentences were played for child and he/she determined whether it was in a question form or statement one.

RESULTS

Performance of cochlear implanted children in perception and production of intonation was significantly lower than children with normal hearing. The difference between fundamental frequency and pitch changes in cochlear implanted children and children with normal hearing was significant (P < 0/05). Cochlear implanted children performance in perception and production of intonation has significant correlation with child's age surgery and duration of prosthesis use (P < 0/05).

DISCUSSION

The findings of the current study show that cochlear prostheses have limited application in facilitating the perception and production of intonation in cochlear implanted children. It should be noted that the child's age at the surgery and duration of prosthesis's use is important in reduction of this limitation. According to these findings, speech and language pathologists should consider intervention of intonation in treatment program of cochlear implanted children.

Voice emotion perception and production in cochlear implant users

Voice emotion is a fundamental component of human social interaction and social development. Unfortunately, cochlear implant users are often forced to interface with highly degraded prosodic cues as a result of device constraints in extraction, processing, and transmission. As such, individuals with cochlear implants frequently demonstrate significant difficulty in recognizing voice emotions in comparison to their normal hearing counterparts. Cochlear implant-mediated perception and production of voice emotion is an important but relatively understudied area of research. However, a rich understanding of the voice emotion auditory processing offers opportunities to improve upon CI biomedical design and to develop training programs benefiting CI performance. In this review, we will address the issues, current literature, and future directions for improved voice emotion processing in cochlear implant users.

Encoding a Melody Using Only Temporal Information for Cochlear-Implant and Normal-Hearing Listeners

One way to provide pitch information to cochlear implant users is through amplitude-modulation rate. It is currently unknown whether amplitude-modulation rate can provide cochlear implant users with pitch information adequate for perceiving melodic information. In the present study, the notes of a song were encoded via amplitude-modulation rate of pulse trains on single electrodes at the apex or middle of long electrode arrays. The melody of the song was either physically correct or modified by compression or expansion. Nine cochlear implant users rated the extent to which the song was out of tune in the different conditions. Cochlear implant users on average did not show sensitivity to melody compression or expansion regardless of place of stimulation. These results were found despite the fact that three of the cochlear implant users showed the expected sensitivity to melody compression and expansion with the same task using acoustic pure tones in a contralateral acoustic ear. Normal-hearing listeners showed an inconsistent and weak effect of melody compression and expansion when the notes of the song were encoded with acoustic pulse rate. The results suggest that amplitude-modulation rate provides insufficient access to melodic information for cochlear-implant and normal-hearing listeners.

An integrated model of pitch perception incorporating place and temporal pitch codes with application to cochlear implant research

Although the neural mechanisms underlying pitch perception are not yet fully understood, there is general agreement that place and temporal representations of pitch are both used by the auditory system. This paper describes a neural network model of pitch perception that integrates both codes of pitch and explores the contributions of, and the interactions between, the two representations in simulated pitch ranking trials in normal and cochlear implant hearing. The model can replicate various psychophysical observations including the perception of the missing fundamental pitch and sensitivity to pitch interval sizes. As a case study, the model was used to investigate the efficiency of pitch perception cues in a novel sound processing scheme, Stimulation based on Auditory Modelling (SAM), that aims to improve pitch perception in cochlear implant hearing. Results showed that enhancement of the pitch perception cues would lead to better pitch ranking scores in the integrated model only if the place and temporal pitch cues were consistent.

A method to enhance the use of interaural time differences for cochlear implants in reverberant environments

The ability of normal-hearing (NH) listeners to exploit interaural time difference (ITD) cues conveyed in the modulated envelopes of high-frequency sounds is poor compared to ITD cues transmitted in the temporal fine structure at low frequencies. Sensitivity to envelope ITDs is further degraded when envelopes become less steep, when modulation depth is reduced, and when envelopes become less similar between the ears, common factors when listening in reverberant environments. The vulnerability of envelope ITDs is particularly problematic for cochlear implant (CI) users, as they rely on information conveyed by slowly varying amplitude envelopes. Here, an approach to improve access to envelope ITDs for CIs is described in which, rather than attempting to reduce reverberation, the perceptual saliency of cues relating to the source is increased by selectively sharpening peaks in the amplitude envelope judged to contain reliable ITDs. Performance of the algorithm with room reverberation was assessed through simulating listening with bilateral CIs in headphone experiments with NH listeners. Relative to simulated standard CI processing, stimuli processed with the algorithm generated lower ITD discrimination thresholds and increased extents of laterality. Depending on parameterization, intelligibility was unchanged or somewhat reduced. The algorithm has the potential to improve spatial listening with CIs.

Loudness Contour Can Influence Mandarin Tone Recognition: Vocoder Simulation and Cochlear Implants

Lexical tone recognition with current cochlear implants (CI) remains unsatisfactory due to significantly degraded pitch-related acoustic cues, which dominate the tone recognition by normal-hearing (NH) listeners. Several secondary cues (e.g., amplitude contour, duration, and spectral envelope) that influence tone recognition in NH listeners and CI users have been studied. This work proposes a loudness contour manipulation algorithm, namely Loudness-Tone (L-Tone), to investigate the effects of loudness contour on Mandarin tone recognition and the effectiveness of using loudness cue to enhance tone recognition for CI users. With L-Tone, the intensity of sound samples is multiplied by gain values determined by instantaneous fundamental frequencies (F0s) and pre-defined gain- F0 mapping functions. Perceptual experiments were conducted with a four-channel noise-band vocoder simulation in NH listeners and with CI users. The results suggested that 1) loudness contour is a useful secondary cue for Mandarin tone recognition, especially when pitch cues are significantly degraded; 2) L-Tone can be used to improve Mandarin tone recognition in both simulated and actual CI-hearing without significant negative effect on vowel and consonant recognition. L-Tone is a promising algorithm for incorporation into real-time CI processing and off-line CI rehabilitation training software.

Results using the OPAL strategy in Mandarin speaking cochlear implant recipients

OBJECTIVE

To evaluate the effectiveness of an experimental pitch-coding strategy for improving recognition of Mandarin lexical tone in cochlear implant (CI) recipients.

DESIGN

Adult CI recipients were tested on recognition of Mandarin tones in quiet and speech-shaped noise at a signal-to-noise ratio of +10 dB; Mandarin sentence speech-reception threshold (SRT) in speech-shaped noise; and pitch discrimination of synthetic complex-harmonic tones in quiet. Two versions of the experimental strategy were examined: (OPAL) linear (1:1) mapping of fundamental frequency (F0) to the coded modulation rate; and (OPAL+) transposed mapping of high F0s to a lower coded rate. Outcomes were compared to results using the clinical ACE™ strategy.

STUDY SAMPLE

Five Mandarin speaking users of Nucleus® cochlear implants.

RESULTS

A small but significant benefit in recognition of lexical tones was observed using OPAL compared to ACE in noise, but not in quiet, and not for OPAL+ compared to ACE or OPAL in quiet or noise. Sentence SRTs were significantly better using OPAL+ and comparable using OPAL to those using ACE. No differences in pitch discrimination thresholds were observed across strategies.

CONCLUSIONS

OPAL can provide benefits to Mandarin lexical tone recognition in moderately noisy conditions and preserve perception of Mandarin sentences in challenging noise conditions.

Cochlear Implant Rate Pitch and Melody Perception as a Function of Place and Number of Electrodes

Six Nucleus cochlear implant recipients participated in a study investigating the effect of place of stimulation on melody perception using rate-pitch cues. Each stimulus was a pulse train delivered on either a single electrode or multiple electrodes sequentially. Four spatial stimulation patterns were used: a single apical electrode, a single mid electrode, a pair of electrodes (apical and mid), and 11 electrodes (from apical to mid). Within one block of trials, all stimuli had the same spatial stimulation pattern, with pulse rate varying from 131 to 262 pps. An additional pulse rate range of 262 to 523 pps was tested with the single-electrode stimuli. Two experimental procedures were used: note ranking; and a modified melodies test with backwards and warp modification. In each trial of the modified melodies test, a familiar melody and a version with modified pitch were presented (in random order), and the subject’s task was to select the unmodified melody. There were no significant differences in performance for stimulation on 1, 2, or 11 electrodes, implying that recipients were unable to combine temporal information from different places in the cochlea to give a stronger pitch cue. No advantage of apical electrodes was found: at the lower pulse rates, there were no significant differences between electrodes; and at the higher pulse rates, scores on the apical electrode dropped more than those on the mid electrode.

Melodic pitch perception and lexical tone perception in Mandarin-speaking cochlear implant users

OBJECTIVES

To examine the relationship between lexical tone perception and melodic pitch perception in Mandarin-speaking cochlear implant (CI) users and to investigate the influence of previous acoustic hearing on CI users' speech and music perception.

DESIGN

Lexical tone perception and melodic contour identification (MCI) were measured in 21 prelingual and 11 postlingual young (aged 6-26 years) Mandarin-speaking CI users. Lexical tone recognition was measured for four tonal patterns: tone 1 (flat F0), tone 2 (rising F0), tone 3 (falling-rising F0), and tone 4 (falling F0). MCI was measured using nine five-note melodic patterns that contained changes in pitch contour, as well as different semitone spacing between notes.

RESULTS

Lexical tone recognition was generally good (overall mean = 81% correct), and there was no significant difference between subject groups. MCI performance was generally poor (mean = 23% correct). MCI performance was significantly better for postlingual (mean = 32% correct) than for prelingual CI participants (mean = 18% correct). After correcting for outliers, there was no significant correlation between lexical tone recognition and MCI performance for prelingual or postlingual CI participants. Age at deafness was significantly correlated with MCI performance only for postlingual participants. CI experience was significantly correlated with MCI performance for both prelingual and postlingual participants. Duration of deafness was significantly correlated with tone recognition only for prelingual participants.

CONCLUSIONS

Despite the prevalence of pitch cues in Mandarin, the present CI participants had great difficulty perceiving melodic pitch. The availability of amplitude and duration cues in lexical tones most likely compensated for the poor pitch perception observed with these CI listeners. Previous acoustic hearing experience seemed to benefit postlingual CI users' melodic pitch perception. Longer CI experience was associated with better MCI performance for both subject groups, suggesting that CI users' music perception may improve as they gain experience with their device.

Affective Properties of Mothers' Speech to Infants With Hearing Impairment and Cochlear Implants

PURPOSE

The affective properties of infant-directed speech influence the attention of infants with normal hearing to speech sounds. This study explored the affective quality of maternal speech to infants with hearing impairment (HI) during the 1st year after cochlear implantation as compared to speech to infants with normal hearing.

METHOD

Mothers of infants with HI and mothers of infants with normal hearing matched by age (NH-AM) or hearing experience (NH-EM) were recorded playing with their infants during 3 sessions over a 12-month period. Speech samples of 25 s were low-pass filtered, leaving intonation but not speech information intact. Sixty adults rated the stimuli along 5 scales: positive/negative affect and intention to express affection, to encourage attention, to comfort/soothe, and to direct behavior.

RESULTS

Low-pass filtered speech to HI and NH-EM groups was rated as more positive, affective, and comforting compared with the such speech to the NH-AM group. Speech to infants with HI and with NH-AM was rated as more directive than speech to the NH-EM group. Mothers decreased affective qualities in speech to all infants but increased directive qualities in speech to infants with NH-EM over time.

CONCLUSIONS

Mothers fine-tune communicative intent in speech to their infant's developmental stage. They adjust affective qualities to infants' hearing experience rather than to chronological age but adjust directive qualities of speech to the chronological age of their infants.

Modulation frequency discrimination with single and multiple channels in cochlear implant users

Temporal envelope cues convey important speech information for cochlear implant (CI) users. Many studies have explored CI users' single-channel temporal envelope processing. However, in clinical CI speech processors, temporal envelope information is processed by multiple channels. Previous studies have shown that amplitude modulation frequency discrimination (AMFD) thresholds are better when temporal envelopes are delivered to multiple rather than single channels. In clinical fitting, current levels on single channels must often be reduced to accommodate multi-channel loudness summation. As such, it is unclear whether the multi-channel advantage in AMFD observed in previous studies was due to coherent envelope information distributed across the cochlea or to greater loudness associated with multi-channel stimulation. In this study, single- and multi-channel AMFD thresholds were measured in CI users. Multi-channel component electrodes were either widely or narrowly spaced to vary the degree of overlap between neural populations. The reference amplitude modulation (AM) frequency was 100 Hz, and coherent modulation was applied to all channels. In Experiment 1, single- and multi-channel AMFD thresholds were measured at similar loudness. In this case, current levels on component channels were higher for single-than for multi-channel AM stimuli, and the modulation depth was approximately 100% of the perceptual dynamic range (i.e., between threshold and maximum acceptable loudness). Results showed no significant difference in AMFD thresholds between similarly loud single- and multi-channel modulated stimuli. In Experiment 2, single- and multi-channel AMFD thresholds were compared at substantially different loudness. In this case, current levels on component channels were the same for single- and multi-channel stimuli ("summation-adjusted" current levels) and the same range of modulation (in dB) was applied to the component channels for both single- and multi-channel testing. With the summation-adjusted current levels, loudness was lower with single than with multiple channels and the AM depth resulted in substantial stimulation below single-channel audibility, thereby reducing the perceptual range of AM. Results showed that AMFD thresholds were significantly better with multiple channels than with any of the single component channels. There was no significant effect of the distribution of electrodes on multi-channel AMFD thresholds. The results suggest that increased loudness due to multi-channel summation may contribute to the multi-channel advantage in AMFD, and that overall loudness may matter more than the distribution of envelope information in the cochlea.

Considering optogenetic stimulation for cochlear implants

Electrical cochlear implants are by far the most successful neuroprostheses and have been implanted in over 300,000 people worldwide. Cochlear implants enable open speech comprehension in most patients but are limited in providing music appreciation and speech understanding in noisy environments. This is generally considered to be due to low frequency resolution as a consequence of wide current spread from stimulation contacts. Accordingly, the number of independently usable stimulation channels is limited to less than a dozen. As light can be conveniently focused, optical stimulation might provide an alternative approach to cochlear implants with increased number of independent stimulation channels. Here, we focus on summarizing recent work on optogenetic stimulation as one way to develop optical cochlear implants. We conclude that proof of principle has been presented for optogenetic stimulation of the cochlea and central auditory neurons in rodents as well as for the technical realization of flexible μLED-based multichannel cochlear implants. Still, much remains to be done in order to advance the technique for auditory research and even more for eventual clinical translation. This article is part of a Special Issue entitled <Lasker Award>.

Envelope enhancement increases cortical sensitivity to interaural envelope delays with acoustic and electric hearing

Evidence from human psychophysical and animal electrophysiological studies suggests that sensitivity to interaural time delay (ITD) in the modulating envelope of a high-frequency carrier can be enhanced using half-wave rectified stimuli. Recent evidence has shown potential benefits of equivalent electrical stimuli to deaf individuals with bilateral cochlear implants (CIs). In the current study we assessed the effects of envelope shape on ITD sensitivity in the primary auditory cortex of normal-hearing ferrets, and profoundly-deaf animals with bilateral CIs. In normal-hearing animals, cortical sensitivity to ITDs (±1 ms in 0.1-ms steps) was assessed in response to dichotically-presented i) sinusoidal amplitude-modulated (SAM) and ii) half-wave rectified (HWR) tones (100-ms duration; 70 dB SPL) presented at the best-frequency of the unit over a range of modulation frequencies. In separate experiments, adult ferrets were deafened with neomycin administration and bilaterally-implanted with intra-cochlear electrode arrays. Electrically-evoked auditory brainstem responses (EABRs) were recorded in response to bipolar electrical stimulation of the apical pair of electrodes with singe biphasic current pulses (40 µs per phase) over a range of current levels to measure hearing thresholds. Subsequently, we recorded cortical sensitivity to ITDs (±800 µs in 80-µs steps) within the envelope of SAM and HWR biphasic-pulse trains (40 µs per phase; 6000 pulses per second, 100-ms duration) over a range of modulation frequencies. In normal-hearing animals, nearly a third of cortical neurons were sensitive to envelope-ITDs in response to SAM tones. In deaf animals with bilateral CI, the proportion of ITD-sensitive cortical neurons was approximately a fifth in response to SAM pulse trains. In normal-hearing and deaf animals with bilateral CI the proportion of ITD sensitive units and neural sensitivity to ITDs increased in response to HWR, compared with SAM stimuli. Consequently, novel stimulation strategies based on envelope enhancement may prove beneficial to individuals with bilateral cochlear implants.

Modulation enhancement in the electrical signal improves perception of interaural time differences with bimodal stimulation

Interaural timing cues are important for sound source localization and for binaural unmasking of speech that is spatially separated from interfering sounds. Users of a cochlear implant (CI) with residual hearing in the non-implanted ear (bimodal listeners) can only make very limited use of interaural timing cues with their clinical devices. Previous studies showed that bimodal listeners can be sensitive to interaural time differences (ITDs) for simple single- and three-channel stimuli. The modulation enhancement strategy (MEnS) was developed to improve the ITD perception of bimodal listeners. It enhances temporal modulations on all stimulated electrodes, synchronously with modulations in the acoustic signal presented to the non-implanted ear, based on measurement of the amplitude peaks occurring at the rate of the fundamental frequency in voiced phonemes. In the first experiment, ITD detection thresholds were measured using the method of constant stimuli for five bimodal listeners for an artificial vowel, processed with either the advanced combination encoder (ACE) strategy or with MEnS. With MEnS, detection thresholds were significantly lower, and for four subjects well within the physically relevant range. In the second experiment, the extent of lateralization was measured in three subjects with both strategies, and ITD sensitivity was determined using an adaptive procedure. All subjects could lateralize sounds based on ITD and sensitivity was significantly better with MEnS than with ACE. The current results indicate that ITD cues can be provided to bimodal listeners with modified sound processing.

The effect of enhancing temporal periodicity cues on Cantonese tone recognition by cochlear implantees

OBJECTIVES

This study investigates the efficacy of a cochlear implant (CI) processing method that enhances temporal periodicity cues of speech.

DESIGN

Subjects participated in word and tone identification tasks. Two processing conditions - the conventional advanced combination encoder (ACE) and tone-enhanced ACE were tested. Test materials were Cantonese disyllabic words recorded from one male and one female speaker. Speech-shaped noise was added to clean speech. The fundamental frequency information for periodicity enhancement was extracted from the clean speech. Electrical stimuli generated from the noisy speech with and without periodicity enhancement were presented via direct stimulation using a Laura 34 research processor. Subjects were asked to identify the presented word.

STUDY SAMPLE

Seven post-lingually deafened native Cantonese-speaking CI users.

RESULTS

Percent correct word, segmental structure, and tone identification scores were calculated. While word and segmental structure identification accuracy remained similar between the two processing conditions, tone identification in noise was better with tone-enhanced ACE than with conventional ACE. Significant improvement on tone perception was found only for the female voice.

CONCLUSIONS

Temporal periodicity cues are important to tone perception in noise. Pitch and tone perception by CI users could be improved when listeners received enhanced temporal periodicity cues.

Exploring the roles of spectral detail and intonation contour in speech intelligibility: an FMRI study

The melodic contour of speech forms an important perceptual aspect of tonal and nontonal languages and an important limiting factor on the intelligibility of speech heard through a cochlear implant. Previous work exploring the neural correlates of speech comprehension identified a left-dominant pathway in the temporal lobes supporting the extraction of an intelligible linguistic message, whereas the right anterior temporal lobe showed an overall preference for signals clearly conveying dynamic pitch information [Johnsrude, I. S., Penhune, V. B., & Zatorre, R. J. Functional specificity in the right human auditory cortex for perceiving pitch direction. Brain, 123, 155-163, 2000; Scott, S. K., Blank, C. C., Rosen, S., & Wise, R. J. Identification of a pathway for intelligible speech in the left temporal lobe. Brain, 123, 2400-2406, 2000]. The current study combined modulations of overall intelligibility (through vocoding and spectral inversion) with a manipulation of pitch contour (normal vs. falling) to investigate the processing of spoken sentences in functional MRI. Our overall findings replicate and extend those of Scott et al. [Scott, S. K., Blank, C. C., Rosen, S., & Wise, R. J. Identification of a pathway for intelligible speech in the left temporal lobe. Brain, 123, 2400-2406, 2000], where greater sentence intelligibility was predominately associated with increased activity in the left STS, and the greatest response to normal sentence melody was found in right superior temporal gyrus. These data suggest a spatial distinction between brain areas associated with intelligibility and those involved in the processing of dynamic pitch information in speech. By including a set of complexity-matched unintelligible conditions created by spectral inversion, this is additionally the first study reporting a fully factorial exploration of spectrotemporal complexity and spectral inversion as they relate to the neural processing of speech intelligibility. Perhaps surprisingly, there was little evidence for an interaction between the two factors-we discuss the implications for the processing of sound and speech in the dorsolateral temporal lobes.

Factors affecting the use of envelope interaural time differences in reverberation

At high frequencies, interaural time differences (ITDs) are conveyed by the sound envelope. Sensitivity to envelope ITDs depends crucially on the envelope shape. Reverberation degrades the envelope shape, reducing the modulation depth of the envelope and the slope of its flanks. Reverberation also reduces the envelope interaural coherence (i.e., the similarity of the envelopes at two ears). The current study investigates the extent to which these changes affect sensitivity to envelope ITDs. The first experiment measured ITD discrimination thresholds at low and high frequencies in a simulated room. The stimulus was either a low-frequency narrowband noise or the same noise transposed to a higher frequency. The results suggest that the effect of reverberation on ITD thresholds was multiplicative. Given that the threshold without reverberation was larger for the transposed than for the low-frequency stimulus, this meant that, in absolute terms, the thresholds for the transposed stimulus showed a much greater increase due to reverberation than those for the low-frequency stimulus. Three further experiments indicated that the effect of reverberation on the envelope ITD thresholds was due to the combined effect of the reduction in the envelope modulation depth and slopes, as well as the decrease in the envelope interaural coherence.

Listening to speech in a background of other talkers: effects of talker number and noise vocoding

Some of the most common interfering background sounds a listener experiences are the sounds of other talkers. In Experiment 1, recognition for natural Institute of Electrical and Electronics Engineers (IEEE) sentences was measured in normal-hearing adults at two fixed signal-to-noise ratios (SNRs) in 16 backgrounds with the same long-term spectrum: unprocessed speech babble (1, 2, 4, 8, and 16 talkers), noise-vocoded versions of the babbles (12 channels), noise modulated with the wide-band envelope of the speech babbles, and unmodulated noise. All talkers were adult males. For a given number of talkers, natural speech was always the most effective masker. The greatest changes in performance occurred as the number of talkers in the maskers increased from 1 to 2 or 4, with small changes thereafter. In Experiment 2, the same targets and maskers (1, 2, and 16 talkers) were used to measure speech reception thresholds (SRTs) adaptively. Periodicity in the target was also manipulated by noise-vocoding, which led to considerably higher SRTs. The greatest masking effect always occurred for the masker type most similar to the target, while the effects of the number of talkers were generally small. Implications are drawn with reference to glimpsing, informational vs energetic masking, overall SNR, and aspects of periodicity.

Enhancement of temporal cues to pitch in cochlear implants: effects on pitch ranking

The abilities to hear changes in pitch for sung vowels and understand speech using an experimental sound coding strategy (eTone) that enhanced coding of temporal fundamental frequency (F0) information were tested in six cochlear implant users, and compared with performance using their clinical (ACE) strategy. In addition, rate- and modulation rate-pitch difference limens (DLs) were measured using synthetic stimuli with F0s below 300 Hz to determine psychophysical abilities of each subject and to provide experience in attending to rate cues for the judgment of pitch. Sung-vowel pitch ranking tests for stimuli separated by three semitones presented across an F0 range of one octave (139-277 Hz) showed a significant benefit for the experimental strategy compared to ACE. Average d-prime (d') values for eTone (d' = 1.05) were approximately three time larger than for ACE (d' = 0.35). Similar scores for both strategies in the speech recognition tests showed that coding of segmental speech information by the experimental strategy was not degraded. Average F0 DLs were consistent with results from previous studies and for all subjects were less than or equal to approximately three semitones for F0s of 125 and 200 Hz.

Acoustic cue integration in speech intonation recognition with cochlear implants

The present article reports on the perceptual weighting of prosodic cues in question-statement identification by adult cochlear implant (CI) listeners. Acoustic analyses of normal-hearing (NH) listeners' production of sentences spoken as questions or statements confirmed that in English the last bisyllabic word in a sentence carries the dominant cues (F0, duration, and intensity patterns) for the contrast. Furthermore, these analyses showed that the F0 contour is the primary cue for the question-statement contrast, with intensity and duration changes conveying important but less reliable information. On the basis of these acoustic findings, the authors examined adult CI listeners' performance in two question-statement identification tasks. In Task 1, 13 CI listeners' question-statement identification accuracy was measured using naturally uttered sentences matched for their syntactic structures. In Task 2, the same listeners' perceptual cue weighting in question-statement identification was assessed using resynthesized single-word stimuli, within which fundamental frequency (F0), intensity, and duration properties were systematically manipulated. Both tasks were also conducted with four NH listeners with full-spectrum and noise-band-vocoded stimuli. Perceptual cue weighting was assessed by comparing the estimated coefficients in logistic models fitted to the data. Of the 13 CI listeners, 7 achieved high performance levels in Task 1. The results of Task 2 indicated that multiple sources of acoustic cues for question-statement identification were utilized to different extents depending on the listening conditions (e.g., full spectrum vs. spectrally degraded) or the listeners' hearing and amplification status (e.g., CI vs. NH).

Effect of cochlear implants on children's perception and production of speech prosody

Japanese 5- to 13-yr-olds who used cochlear implants (CIs) and a comparison group of normally hearing (NH) Japanese children were tested on their perception and production of speech prosody. For the perception task, they were required to judge whether semantically neutral utterances that were normalized for amplitude were spoken in a happy, sad, or angry manner. The performance of NH children was error-free. By contrast, child CI users performed well below ceiling but above chance levels on happy- and sad-sounding utterances but not on angry-sounding utterances. For the production task, children were required to imitate stereotyped Japanese utterances expressing disappointment and surprise as well as culturally typically representations of crow and cat sounds. NH 5- and 6-year-olds produced significantly poorer imitations than older hearing children, but age was unrelated to the imitation quality of child CI users. Overall, child CI user's imitations were significantly poorer than those of NH children, but they did not differ significantly from the imitations of the youngest NH group. Moreover, there was a robust correlation between the performance of child CI users on the perception and production tasks; this implies that difficulties with prosodic perception underlie their difficulties with prosodic imitation.