Processing F0 with cochlear implants: Modulation frequency discrimination and speech intonation recognition

The everyday speech environments of preschoolers with and without cochlear implants

Children who receive cochlear implants develop spoken language on a protracted timescale. The home environment facilitates speech-language development, yet it is relatively unknown how the environment differs between children with cochlear implants and typical hearing. We matched eighteen preschoolers with implants (31-65 months) to two groups of children with typical hearing: by chronological age and hearing age. Each child completed a long-form, naturalistic audio recording of their home environment (appx. 16 hours/child; >730 hours of observation) to measure adult speech input, child vocal productivity, and caregiver-child interaction. Results showed that children with cochlear implants and typical hearing were exposed to and engaged in similar amounts of spoken language with caregivers. However, the home environment did not reflect developmental stages as closely for children with implants, or predict their speech outcomes as strongly. Home-based speech-language interventions should focus on the unique input-outcome relationships for this group of children with hearing loss.

Meta-analysis on lexical tone recognition in cochlear implant users

OBJECTIVE

Lexical tone plays a vital role in speech communication in tonal languages. This study investigated lexical tone recognition in cochlear implant (CI) users, and identified potential factors that influence lexical tone recognition in the CI population.

DESIGN

We conducted a systematic search across eleven major databases, evaluated the risk of bias in the included studies, and conducted five meta-analyses.

STUDY SAMPLE

Forty studies that utilised a multi-item alternative forced-choice paradigm were included to evaluate the performance of lexical tone recognition in CI users.

RESULTS

CI users performed worse at recognising lexical tones than normal hearing (NH) controls. Furthermore, bimodal stimulation could benefit lexical tone recognition for CI users in both quiet and noisy conditions. Besides, the pooled results showed a negative correlation between tone recognition accuracy and age at implantation, as well as a positive correlation between tone recognition performance and the duration of CI experience.

CONCLUSIONS

This study indicates that CI users could not recognise lexical tones at the same level as the NH population. The bimodal intervention does have a more positive effect than unimodal implantation regardless of the listening environment. Moreover, earlier implantation and longer experience with the CI could facilitate lexical tone recognition.

Speech-in-noise, psychosocial, and heart rate variability outcomes of group singing or audiobook club interventions for older adults with unaddressed hearing loss: A SingWell Project multisite, randomized controlled trial, registered report protocol

BACKGROUND

Unaddressed age-related hearing loss is highly prevalent among older adults, typified by negative consequences for speech-in-noise perception and psychosocial wellbeing. There is promising evidence that group singing may enhance speech-in-noise perception and psychosocial wellbeing. However, there is a lack of robust evidence, primarily due to the literature being based on small sample sizes, single site studies, and a lack of randomized controlled trials. Hence, to address these concerns, this SingWell Project study utilizes an appropriately powered sample size, multisite, randomized controlled trial approach, with a robust preplanned statistical analysis.

OBJECTIVE

To explore if group singing may improve speech-in-noise perception and psychosocial wellbeing for older adults with unaddressed hearing loss.

METHODS

We designed an international, multisite, randomized controlled trial to explore the benefits of group singing for adults aged 60 years and older with unaddressed hearing loss (registered at clinicaltrials.gov, ID: NCT06580847). After undergoing an eligibility screening process and completing an information and consent form, we intend to recruit 210 participants that will be randomly assigned to either group singing or an audiobook club (control group) intervention for a training period of 12-weeks. The study has multiple timepoints for testing, that are broadly categorized as macro (i.e., pre- and post-measures across the 12-weeks), or micro timepoints (i.e., pre- and post-measures across a weekly training session). Macro measures include behavioural measures of speech and music perception, and psychosocial questionnaires. Micro measures include psychosocial questionnaires and heart-rate variability.

HYPOTHESES

We hypothesize that group singing may be effective at improving speech perception and psychosocial outcomes for adults aged 60 years and older with unaddressed hearing loss-more so than participants in the control group.

Lexical Stress Identification in Cochlear Implant-Simulated Speech by Non-Native Listeners

This study investigates whether a presumed difference in the perceptibility of cues to lexical stress in spectro-temporally degraded simulated cochlear implant (CI) speech affects how listeners weight these cues during a lexical stress identification task, specifically in their non-native language. Previous research suggests that in English, listeners predominantly rely on a reduction in vowel quality as a cue to lexical stress. In Dutch, changes in the fundamental frequency (F0) contour seem to have a greater functional weight than the vowel quality contrast. Generally, non-native listeners use the cue-weighting strategies from their native language in the non-native language. Moreover, few studies have suggested that these cues to lexical stress are differently perceptible in spectro-temporally degraded electric hearing, as CI users appear to make more effective use of changes in vowel quality than of changes in the F0 contour as cues to linguistic phenomena. In this study, native Dutch learners of English identified stressed syllables in CI-simulated and non-CI-simulated Dutch and English words that contained changes in the F0 contour and vowel quality as cues to lexical stress. The results indicate that neither the cue-weighting strategies in the native language nor in the non-native language are influenced by the perceptibility of cues in the spectro-temporally degraded speech signal. These results are in contrast to our expectations based on previous research and support the idea that cue weighting is a flexible and transferable process.

Speech-evoked cortical activities and speech recognition in adult cochlear implant listeners: a review of functional near-infrared spectroscopy studies

Cochlear implants (CIs) are the most successful neural prostheses, enabling individuals with severe to profound hearing loss to access sounds and understand speech. While CI has demonstrated success, speech perception outcomes vary largely among CI listeners, with significantly reduced performance in noise. This review paper summarizes prior findings on speech-evoked cortical activities in adult CI listeners using functional near-infrared spectroscopy (fNIRS) to understand (a) speech-evoked cortical processing in CI listeners compared to normal-hearing (NH) individuals, (b) the relationship between these activities and behavioral speech recognition scores, (c) the extent to which current fNIRS-measured speech-evoked cortical activities in CI listeners account for their differences in speech perception, and (d) challenges in using fNIRS for CI research. Compared to NH listeners, CI listeners had diminished speech-evoked activation in the middle temporal gyrus (MTG) and in the superior temporal gyrus (STG), except one study reporting an opposite pattern for STG. NH listeners exhibited higher inferior frontal gyrus (IFG) activity when listening to CI-simulated speech compared to natural speech. Among CI listeners, higher speech recognition scores correlated with lower speech-evoked activation in the STG, higher activation in the left IFG and left fusiform gyrus, with mixed findings in the MTG. fNIRS shows promise for enhancing our understanding of cortical processing of speech in CI listeners, though findings are mixed. Challenges include test-retest reliability, managing noise, replicating natural conditions, optimizing montage design, and standardizing methods to establish a strong predictive relationship between fNIRS-based cortical activities and speech perception in CI listeners.

Vocal control and speech production in cochlear implant listeners: A review within auditory-motor processing framework

A comprehensive literature review is conducted to summarize and discuss prior findings on how cochlear implants (CI) affect the users' abilities to produce and control vocal and articulatory movements within the auditory-motor integration framework of speech. Patterns of speech production pre- versus post-implantation, post-implantation adjustments, deviations from the typical ranges of speakers with normal hearing (NH), the effects of switching the CI on and off, as well as the impact of altered auditory feedback on vocal and articulatory speech control are discussed. Overall, findings indicate that CIs enhance the vocal and articulatory control aspects of speech production at both segmental and suprasegmental levels. While many CI users achieve speech quality comparable to NH individuals, some features still deviate in a group of CI users even years post-implantation. More specifically, contracted vowel space, increased vocal jitter and shimmer, longer phoneme and utterance durations, shorter voice onset time, decreased contrast in fricative production, limited prosodic patterns, and reduced intelligibility have been reported in subgroups of CI users compared to NH individuals. Significant individual variations among CI users have been observed in both the pace of speech production adjustments and long-term speech outcomes. Few controlled studies have explored how the implantation age and the duration of CI use influence speech features, leaving substantial gaps in our understanding about the effects of spectral resolution, auditory rehabilitation, and individual auditory-motor processing abilities on vocal and articulatory speech outcomes in CI users. Future studies under the auditory-motor integration framework are warranted to determine how suboptimal CI auditory feedback impacts auditory-motor processing and precise vocal and articulatory control in CI users.

The Role of Early Intact Auditory Experience on the Perception of Spoken Emotions, Comparing Prelingual to Postlingual Cochlear Implant Users

OBJECTIVES

Cochlear implants (CI) are remarkably effective, but have limitations regarding the transformation of the spectro-temporal fine structures of speech. This may impair processing of spoken emotions, which involves the identification and integration of semantic and prosodic cues. Our previous study found spoken-emotions-processing differences between CI users with postlingual deafness (postlingual CI) and normal hearing (NH) matched controls (age range, 19 to 65 years). Postlingual CI users over-relied on semantic information in incongruent trials (prosody and semantics present different emotions), but rated congruent trials (same emotion) similarly to controls. Postlingual CI's intact early auditory experience may explain this pattern of results. The present study examined whether CI users without intact early auditory experience (prelingual CI) would generally perform worse on spoken emotion processing than NH and postlingual CI users, and whether CI use would affect prosodic processing in both CI groups. First, we compared prelingual CI users with their NH controls. Second, we compared the results of the present study to our previous study ( Taitlebaum-Swead et al. 2022 ; postlingual CI).

DESIGN

Fifteen prelingual CI users and 15 NH controls (age range, 18 to 31 years) listened to spoken sentences composed of different combinations (congruent and incongruent) of three discrete emotions (anger, happiness, sadness) and neutrality (performance baseline), presented in prosodic and semantic channels (Test for Rating of Emotions in Speech paradigm). Listeners were asked to rate (six-point scale) the extent to which each of the predefined emotions was conveyed by the sentence as a whole (integration of prosody and semantics), or to focus only on one channel (rating the target emotion [RTE]) and ignore the other (selective attention). In addition, all participants performed standard tests of speech perception. Performance on the Test for Rating of Emotions in Speech was compared with the previous study (postlingual CI).

RESULTS

When asked to focus on one channel, semantics or prosody, both CI groups showed a decrease in prosodic RTE (compared with controls), but only the prelingual CI group showed a decrease in semantic RTE. When the task called for channel integration, both groups of CI users used semantic emotional information to a greater extent than their NH controls. Both groups of CI users rated sentences that did not present the target emotion higher than their NH controls, indicating some degree of confusion. However, only the prelingual CI group rated congruent sentences lower than their NH controls, suggesting reduced accumulation of information across channels. For prelingual CI users, individual differences in identification of monosyllabic words were significantly related to semantic identification and semantic-prosodic integration.

CONCLUSIONS

Taken together with our previous study, we found that the degradation of acoustic information by the CI impairs the processing of prosodic emotions, in both CI user groups. This distortion appears to lead CI users to over-rely on the semantic information when asked to integrate across channels. Early intact auditory exposure among CI users was found to be necessary for the effective identification of semantic emotions, as well as the accumulation of emotional information across the two channels. Results suggest that interventions for spoken-emotion processing should not ignore the onset of hearing loss.

Combined-electrical optogenetic stimulation but not channelrhodopsin kinetics improves the fidelity of high rate stimulation in the auditory pathway in mice

Novel stimulation methods are needed to overcome the limitations of contemporary cochlear implants. Optogenetics is a technique that confers light sensitivity to neurons via the genetic introduction of light-sensitive ion channels. By controlling neural activity with light, auditory neurons can be activated with higher spatial precision. Understanding the behaviour of opsins at high stimulation rates is an important step towards their translation. To elucidate this, we compared the temporal characteristics of auditory nerve and inferior colliculus responses to optogenetic, electrical, and combined optogenetic-electrical stimulation in virally transduced mice expressing one of two channelrhodopsins, ChR2-H134R or ChIEF, at stimulation rates up to 400 pulses per second (pps). At 100 pps, optogenetic responses in ChIEF mice demonstrated higher fidelity, less change in latency, and greater response stability compared to responses in ChR2-H134R mice, but not at higher rates. Combined stimulation improved the response characteristics in both cohorts at 400 pps, although there was no consistent facilitation of electrical responses. Despite these results, day-long stimulation (up to 13 h) led to severe and non-recoverable deterioration of the optogenetic responses. The results of this study have significant implications for the translation of optogenetic-only and combined stimulation techniques for hearing loss.

Prosodic and Visual Cues Facilitate Irony Comprehension by Mandarin-Speaking Children With Cochlear Implants

PURPOSE

This study investigated irony comprehension by Mandarin-speaking children with cochlear implants, focusing on how prosodic and visual cues contribute to their comprehension, and whether second-order Theory of Mind is required for using these cues.

METHOD

We tested 52 Mandarin-speaking children with cochlear implants (aged 3-7 years) and 52 age- and gender-matched children with normal hearing. All children completed a Theory of Mind test and a story comprehension test. Ironic stories were presented in three conditions, each providing different cues: (a) context-only, (b) context and prosody, and (c) context, prosody, and visual cues. Comparisons were conducted on the accuracy of story understanding across the three conditions to examine the role of prosodic and visual cues.

RESULTS

The results showed that, compared to the context-only condition, the additional prosodic and visual cues both improved the accuracy of irony comprehension for children with cochlear implants, similar to their normal-hearing peers. Furthermore, such improvements were observed for all children, regardless of whether they passed the second-order Theory of Mind test or not.

CONCLUSIONS

This study is the first to demonstrate the benefits of prosodic and visual cues in irony comprehension, without reliance on second-order Theory of Mind, for Mandarin-speaking children with cochlear implants. It implies potential insights for utilizing prosodic and visual cues in intervention strategies to promote irony comprehension.

Prelingually Deaf Children With Cochlear Implants Show Better Perception of Voice Cues and Speech in Competing Speech Than Postlingually Deaf Adults With Cochlear Implants

OBJECTIVES

Postlingually deaf adults with cochlear implants (CIs) have difficulties with perceiving differences in speakers' voice characteristics and benefit little from voice differences for the perception of speech in competing speech. However, not much is known yet about the perception and use of voice characteristics in prelingually deaf implanted children with CIs. Unlike CI adults, most CI children became deaf during the acquisition of language. Extensive neuroplastic changes during childhood could make CI children better at using the available acoustic cues than CI adults, or the lack of exposure to a normal acoustic speech signal could make it more difficult for them to learn which acoustic cues they should attend to. This study aimed to examine to what degree CI children can perceive voice cues and benefit from voice differences for perceiving speech in competing speech, comparing their abilities to those of normal-hearing (NH) children and CI adults.

DESIGN

CI children's voice cue discrimination (experiment 1), voice gender categorization (experiment 2), and benefit from target-masker voice differences for perceiving speech in competing speech (experiment 3) were examined in three experiments. The main focus was on the perception of mean fundamental frequency (F0) and vocal-tract length (VTL), the primary acoustic cues related to speakers' anatomy and perceived voice characteristics, such as voice gender.

RESULTS

CI children's F0 and VTL discrimination thresholds indicated lower sensitivity to differences compared with their NH-age-equivalent peers, but their mean discrimination thresholds of 5.92 semitones (st) for F0 and 4.10 st for VTL indicated higher sensitivity than postlingually deaf CI adults with mean thresholds of 9.19 st for F0 and 7.19 st for VTL. Furthermore, CI children's perceptual weighting of F0 and VTL cues for voice gender categorization closely resembled that of their NH-age-equivalent peers, in contrast with CI adults. Finally, CI children had more difficulties in perceiving speech in competing speech than their NH-age-equivalent peers, but they performed better than CI adults. Unlike CI adults, CI children showed a benefit from target-masker voice differences in F0 and VTL, similar to NH children.

CONCLUSION

Although CI children's F0 and VTL voice discrimination scores were overall lower than those of NH children, their weighting of F0 and VTL cues for voice gender categorization and their benefit from target-masker differences in F0 and VTL resembled that of NH children. Together, these results suggest that prelingually deaf implanted CI children can effectively utilize spectrotemporally degraded F0 and VTL cues for voice and speech perception, generally outperforming postlingually deaf CI adults in comparable tasks. These findings underscore the presence of F0 and VTL cues in the CI signal to a certain degree and suggest other factors contributing to the perception challenges faced by CI adults.

Effect of Age and Unaided Acoustic Hearing on Pediatric Cochlear Implant Users' Ability to Distinguish Yes/No Statements and Questions

PURPOSE

The purpose of this study was to evaluate the ability to discriminate yes/no questions from statements in three groups of children: bilateral cochlear implant (CI) users, nontraditional CI users with aidable hearing preoperatively in the ear to be implanted, and controls with normal hearing. Half of the nontraditional CI users had sufficient postoperative acoustic hearing in the implanted ear to use electric-acoustic stimulation, and half used a CI alone.

METHOD

Participants heard recorded sentences that were produced either as yes/no questions or as statements by three male and three female talkers. Three raters scored each participant response as either a question or a statement. Bilateral CI users (n = 40, 4-12 years old) and normal-hearing controls (n = 10, 4-12 years old) were tested binaurally in the free field. Nontraditional CI recipients (n = 22, 6-17 years old) were tested with direct audio input to the study ear.

RESULTS

For the bilateral CI users, performance was predicted by age but not by 125-Hz acoustic thresholds; just under half (n = 17) of the participants in this group had measurable 125-Hz thresholds in their better ear. For nontraditional CI recipients, better performance was predicted by lower 125-Hz acoustic thresholds in the test ear, and there was no association with participant age. Performance approached that of the normal-hearing controls for some participants in each group.

CONCLUSIONS

Results suggest that a 125-Hz acoustic hearing supports discrimination of yes/no questions and statements in pediatric CI users. Bilateral CI users with little or no acoustic hearing at 125 Hz develop the ability to perform this task, but that ability emerges later than for children with better acoustic hearing. These results underscore the importance of preserving acoustic hearing for pediatric CI users when possible.

Impact of cochlear implants use on voice production and quality

Cochlear implant users experience difficulties controlling their vocalizations compared to normal hearing peers. However, less is known about their voice quality. The primary aim of the present study was to determine if cochlear implant users' voice quality would be categorized as dysphonic by the Acoustic Voice Quality Index (AVQI) and smoothed cepstral peak prominence (CPPS). A secondary aim was to determine if vocal quality is further impacted when using bilateral implants compared to using only one implant. The final aim was to determine how residual hearing impacts voice quality. Twenty-seven cochlear implant users participated in the present study and were recorded while sustaining a vowel and while reading a standardized passage. These recordings were analyzed to calculate the AVQI and CPPS. The results indicate that CI users' voice quality was detrimentally affected by using their CI, raising to the level of a dysphonic voice. Specifically, when using their CI, mean AVQI scores were 4.0 and mean CPPS values were 11.4 dB, which indicates dysphonia. There were no significant differences in voice quality when comparing participants with bilateral implants to those with one implant. Finally, for participants with residual hearing, as hearing thresholds worsened, the likelihood of a dysphonic voice decreased.

Psychoacoustic and electroencephalographic responses to changes in amplitude modulation depth and frequency in relation to speech recognition in cochlear implantees

Temporal envelope modulations (TEMs) are one of the most important features that cochlear implant (CI) users rely on to understand speech. Electroencephalographic assessment of TEM encoding could help clinicians to predict speech recognition more objectively, even in patients unable to provide active feedback. The acoustic change complex (ACC) and the auditory steady-state response (ASSR) evoked by low-frequency amplitude-modulated pulse trains can be used to assess TEM encoding with electrical stimulation of individual CI electrodes. In this study, we focused on amplitude modulation detection (AMD) and amplitude modulation frequency discrimination (AMFD) with stimulation of a basal versus an apical electrode. In twelve adult CI users, we (a) assessed behavioral AMFD thresholds and (b) recorded cortical auditory evoked potentials (CAEPs), AMD-ACC, AMFD-ACC, and ASSR in a combined 3-stimulus paradigm. We found that the electrophysiological responses were significantly higher for apical than for basal stimulation. Peak amplitudes of AMFD-ACC were small and (therefore) did not correlate with speech-in-noise recognition. We found significant correlations between speech-in-noise recognition and (a) behavioral AMFD thresholds and (b) AMD-ACC peak amplitudes. AMD and AMFD hold potential to develop a clinically applicable tool for assessing TEM encoding to predict speech recognition in CI users.

Temporal pitch matching with bilateral cochlear implants

Interaural pitch matching is a common task used with bilateral cochlear implant (CI) users, although studies measuring this have largely focused on place-based pitch matches. Temporal-based pitch also plays an important role in CI users' perception, but interaural temporal-based pitch matching has not been well characterized for CI users. To investigate this, bilateral CI users were asked to match amplitude modulation frequencies of stimulation across ears. Comparisons were made to previous place-based pitch matching data that were collected using similar procedures. The results indicate that temporal-based pitch matching is particularly sensitive to the choice of reference ear.

How to vocode: Using channel vocoders for cochlear-implant research

The channel vocoder has become a useful tool to understand the impact of specific forms of auditory degradation-particularly the spectral and temporal degradation that reflect cochlear-implant processing. Vocoders have many parameters that allow researchers to answer questions about cochlear-implant processing in ways that overcome some logistical complications of controlling for factors in individual cochlear implant users. However, there is such a large variety in the implementation of vocoders that the term "vocoder" is not specific enough to describe the signal processing used in these experiments. Misunderstanding vocoder parameters can result in experimental confounds or unexpected stimulus distortions. This paper highlights the signal processing parameters that should be specified when describing vocoder construction. The paper also provides guidance on how to determine vocoder parameters within perception experiments, given the experimenter's goals and research questions, to avoid common signal processing mistakes. Throughout, we will assume that experimenters are interested in vocoders with the specific goal of better understanding cochlear implants.

Predictors of Emotional Prosody Identification by School-Age Children With Cochlear Implants and Their Peers With Normal Hearing

OBJECTIVES

Children with cochlear implants (CIs) vary widely in their ability to identify emotions in speech. The causes of this variability are unknown, but this knowledge will be crucial if we are to design improvements in technological or rehabilitative interventions that are effective for individual patients. The objective of this study was to investigate how well factors such as age at implantation, duration of device experience (hearing age), nonverbal cognition, vocabulary, and socioeconomic status predict prosody-based emotion identification in children with CIs, and how the key predictors in this population compare to children with normal hearing who are listening to either normal emotional speech or to degraded speech.

DESIGN

We measured vocal emotion identification in 47 school-age CI recipients aged 7 to 19 years in a single-interval, 5-alternative forced-choice task. None of the participants had usable residual hearing based on parent/caregiver report. Stimuli consisted of a set of semantically emotion-neutral sentences that were recorded by 4 talkers in child-directed and adult-directed prosody corresponding to five emotions: neutral, angry, happy, sad, and scared. Twenty-one children with normal hearing were also tested in the same tasks; they listened to both original speech and to versions that had been noise-vocoded to simulate CI information processing.

RESULTS

Group comparison confirmed the expected deficit in CI participants' emotion identification relative to participants with normal hearing. Within the CI group, increasing hearing age (correlated with developmental age) and nonverbal cognition outcomes predicted emotion recognition scores. Stimulus-related factors such as talker and emotional category also influenced performance and were involved in interactions with hearing age and cognition. Age at implantation was not predictive of emotion identification. Unlike the CI participants, neither cognitive status nor vocabulary predicted outcomes in participants with normal hearing, whether listening to original speech or CI-simulated speech. Age-related improvements in outcomes were similar in the two groups. Participants with normal hearing listening to original speech showed the greatest differences in their scores for different talkers and emotions. Participants with normal hearing listening to CI-simulated speech showed significant deficits compared with their performance with original speech materials, and their scores also showed the least effect of talker- and emotion-based variability. CI participants showed more variation in their scores with different talkers and emotions than participants with normal hearing listening to CI-simulated speech, but less so than participants with normal hearing listening to original speech.

CONCLUSIONS

Taken together, these results confirm previous findings that pediatric CI recipients have deficits in emotion identification based on prosodic cues, but they improve with age and experience at a rate that is similar to peers with normal hearing. Unlike participants with normal hearing, nonverbal cognition played a significant role in CI listeners' emotion identification. Specifically, nonverbal cognition predicted the extent to which individual CI users could benefit from some talkers being more expressive of emotions than others, and this effect was greater in CI users who had less experience with their device (or were younger) than CI users who had more experience with their device (or were older). Thus, in young prelingually deaf children with CIs performing an emotional prosody identification task, cognitive resources may be harnessed to a greater degree than in older prelingually deaf children with CIs or than children with normal hearing.

Communicating Emotion: Vocal Expression of Linguistic and Emotional Prosody in Children With Mild to Profound Hearing Loss Compared With That of Normal Hearing Peers

OBJECTIVES

Emotional prosody is known to play an important role in social communication. Research has shown that children with cochlear implants (CCIs) may face challenges in their ability to express prosody, as their expressions may have less distinct acoustic contrasts and therefore may be judged less accurately. The prosody of children with milder degrees of hearing loss, wearing hearing aids, has sparsely been investigated. More understanding of the prosodic expression by children with hearing loss, hearing aid users in particular, could create more awareness among healthcare professionals and parents on limitations in social communication, which awareness may lead to more targeted rehabilitation. This study aimed to compare the prosodic expression potential of children wearing hearing aids (CHA) with that of CCIs and children with normal hearing (CNH).

DESIGN

In this prospective experimental study, utterances of pediatric hearing aid users, cochlear implant users, and CNH containing emotional expressions (happy, sad, and angry) were recorded during a reading task. Of the utterances, three acoustic properties were calculated: fundamental frequency (F0), variance in fundamental frequency (SD of F0), and intensity. Acoustic properties of the utterances were compared within subjects and between groups.

RESULTS

A total of 75 children were included (CHA: 26, CCI: 23, and CNH: 26). Participants were between 7 and 13 years of age. The 15 CCI with congenital hearing loss had received the cochlear implant at median age of 8 months. The acoustic patterns of emotions uttered by CHA were similar to those of CCI and CNH. Only in CCI, we found no difference in F0 variation between happiness and anger, although an intensity difference was present. In addition, CCI and CHA produced poorer happy-sad contrasts than did CNH.

CONCLUSIONS

The findings of this study suggest that on a fundamental, acoustic level, both CHA and CCI have a prosodic expression potential that is almost on par with normal hearing peers. However, there were some minor limitations observed in the prosodic expression of these children, it is important to determine whether these differences are perceptible to listeners and could affect social communication. This study sets the groundwork for more research that will help us fully understand the implications of these findings and how they may affect the communication abilities of these children. With a clearer understanding of these factors, we can develop effective ways to help improve their communication skills.

Review of Binaural Processing With Asymmetrical Hearing Outcomes in Patients With Bilateral Cochlear Implants

Bilateral cochlear implants (BiCIs) result in several benefits, including improvements in speech understanding in noise and sound source localization. However, the benefit bilateral implants provide among recipients varies considerably across individuals. Here we consider one of the reasons for this variability: difference in hearing function between the two ears, that is, interaural asymmetry. Thus far, investigations of interaural asymmetry have been highly specialized within various research areas. The goal of this review is to integrate these studies in one place, motivating future research in the area of interaural asymmetry. We first consider bottom-up processing, where binaural cues are represented using excitation-inhibition of signals from the left ear and right ear, varying with the location of the sound in space, and represented by the lateral superior olive in the auditory brainstem. We then consider top-down processing via predictive coding, which assumes that perception stems from expectations based on context and prior sensory experience, represented by cascading series of cortical circuits. An internal, perceptual model is maintained and updated in light of incoming sensory input. Together, we hope that this amalgamation of physiological, behavioral, and modeling studies will help bridge gaps in the field of binaural hearing and promote a clearer understanding of the implications of interaural asymmetry for future research on optimal patient interventions.

Voice Discrimination in Quiet and in Background Noise by Simulated and Real Cochlear Implant Users

PURPOSE

Cochlear implant (CI) users demonstrate poor voice discrimination (VD) in quiet conditions based on the speaker's fundamental frequency (fo) and formant frequencies (i.e., vocal-tract length [VTL]). Our purpose was to examine the effect of background noise at levels that allow good speech recognition thresholds (SRTs) on VD via acoustic CI simulations and CI hearing.

METHOD

Forty-eight normal-hearing (NH) listeners who listened via noise-excited (n = 20) or sinewave (n = 28) vocoders and 10 prelingually deaf CI users (i.e., whose hearing loss began before language acquisition) participated in the study. First, the signal-to-noise ratio (SNR) that yields 70.7% correct SRT was assessed using an adaptive sentence-in-noise test. Next, the CI simulation listeners performed 12 adaptive VDs: six in quiet conditions, two with each cue (fo, VTL, fo + VTL), and six amid speech-shaped noise. The CI participants performed six VDs: one with each cue, in quiet and amid noise. SNR at VD testing was 5 dB higher than the individual's SRT in noise (SRTn +5 dB).

RESULTS

Results showed the following: (a) Better VD was achieved via the noise-excited than the sinewave vocoder, with the noise-excited vocoder better mimicking CI VD; (b) background noise had a limited negative effect on VD, only for the CI simulation listeners; and (c) there was a significant association between SNR at testing and VTL VD only for the CI simulation listeners.

CONCLUSIONS

For NH listeners who listen to CI simulations, noise that allows good SRT can nevertheless impede VD, probably because VD depends more on bottom-up sensory processing. Conversely, for prelingually deaf CI users, noise that allows good SRT hardly affects VD, suggesting that they rely strongly on bottom-up processing for both VD and speech recognition.

Prosodic Focus Interpretation in Spectrotemporally Degraded Speech by Non-Native Listeners

PURPOSE

This study assesses how spectrotemporal degradations that can occur in the sound transmission of a cochlear implant (CI) may influence the ability of non-native listeners to recognize the intended meaning of utterances based on the position of the prosodically focused word. Previous research suggests that perceptual accuracy and listening effort are negatively affected by CI processing (or CI simulations) or when the speech is presented in a non-native language, in a number of tasks and circumstances. How these two factors interact to affect prosodic focus interpretation, however, remains unclear.

METHOD

In an online experiment, normal-hearing (NH) adolescent and adult native Dutch learners of English and a small control group of NH native English adolescents listened to CI-simulated (eight-channel noise-band vocoded) and non-CI-simulated English sentences differing in prosodically marked focus. For assessing perceptual accuracy, listeners had to indicate which of four possible context questions the speaker answered. For assessing listening effort, a dual-task paradigm was used with a secondary free recall task.

RESULTS

The results indicated that prosodic focus interpretation was significantly less accurate in the CI-simulated condition compared with the non-CI-simulated condition but that listening effort was not increased. Moreover, there was no interaction between the influence of the degraded CI-simulated speech signal and listening groups in either their perceptual accuracy or listening effort.

CONCLUSION

Non-native listeners are not more strongly affected by spectrotemporal degradations than native listeners, and less proficient non-native listeners are not more strongly affected by these degradations than more proficient non-native listeners.

Cortical haemodynamic responses predict individual ability to recognise vocal emotions with uninformative pitch cues but do not distinguish different emotions

We investigated the cortical representation of emotional prosody in normal-hearing listeners using functional near-infrared spectroscopy (fNIRS) and behavioural assessments. Consistent with previous reports, listeners relied most heavily on F0 cues when recognizing emotion cues; performance was relatively poor-and highly variable between listeners-when only intensity and speech-rate cues were available. Using fNIRS to image cortical activity to speech utterances containing natural and reduced prosodic cues, we found right superior temporal gyrus (STG) to be most sensitive to emotional prosody, but no emotion-specific cortical activations, suggesting that while fNIRS might be suited to investigating cortical mechanisms supporting speech processing it is less suited to investigating cortical haemodynamic responses to individual vocal emotions. Manipulating emotional speech to render F0 cues less informative, we found the amplitude of the haemodynamic response in right STG to be significantly correlated with listeners' abilities to recognise vocal emotions with uninformative F0 cues. Specifically, listeners more able to assign emotions to speech with degraded F0 cues showed lower haemodynamic responses to these degraded signals. This suggests a potential objective measure of behavioural sensitivity to vocal emotions that might benefit neurodiverse populations less sensitive to emotional prosody or hearing-impaired listeners, many of whom rely on listening technologies such as hearing aids and cochlear implants-neither of which restore, and often further degrade, the F0 cues essential to parsing emotional prosody conveyed in speech.

The effect of stimulus type and testing method on talker discrimination of school-age children

Efficient talker discrimination (TD) improves speech understanding under multi-talker conditions. So far, TD of children has been assessed using various testing parameters, making it difficult to draw comparative conclusions. This study explored the effects of the stimulus type and variability on children's TD. Thirty-two children (7-10 years old) underwent eight TD assessments with fundamental frequency + formant changes using an adaptive procedure. Stimuli included consonant-vowel-consonant words or three-word sentences and were either fixed by run or by trial (changing throughout the run). Cognitive skills were also assessed. Thirty-one adults (18-35 years old) served as controls. The results showed (1) poorer TD for the fixed-by-trial than the fixed-by-run method, with both stimulus types for the adults but only with the words for the children; (2) poorer TD for the words than the sentences with the fixed-by-trial method only for the children; and (3) significant correlations between the children's age and TD. These results support a developmental trajectory in the use of perceptual anchoring for TD and in its reliance on comprehensive acoustic and linguistic information. The finding that the testing parameters may influence the top-down and bottom-up processing for TD should be considered when comparing data across studies or when planning new TD experiments.

Relationships Between the Auditory Nerve Sensitivity to Amplitude Modulation, Perceptual Amplitude Modulation Rate Discrimination Sensitivity, and Speech Perception Performance in Postlingually Deafened Adult Cochlear Implant Users

OBJECTIVE

This study assessed the relationships between the salience of amplitude modulation (AM) cues encoded at the auditory nerve (AN), perceptual sensitivity to changes in AM rate (i.e., AM rate discrimination threshold, AMRDT), and speech perception scores in postlingually deafened adult cochlear implant (CI) users.

DESIGN

Study participants were 18 postlingually deafened adults with Cochlear Nucleus devices, including five bilaterally implanted patients. For each of 23 implanted ears, neural encoding of AM cues at 20 Hz at the AN was evaluated at seven electrode locations across the electrode array using electrophysiological measures of the electrically evoked compound action potential (eCAP). The salience of AM neural encoding was quantified by the Modulated Response Amplitude Ratio (MRAR). Psychophysical measures of AMRDT for 20 Hz modulation were evaluated in 16 ears using a three-alternative, forced-choice procedure, targeting 79.4% correct on the psychometric function. AMRDT was measured at up to five electrode locations for each test ear, including the electrode pair that showed the largest difference in the MRAR. Consonant-Nucleus-Consonant (CNC) word scores presented in quiet and in speech-shaped noise at a signal to noise ratio (SNR) of +10 dB were measured in all 23 implanted ears. Simulation tests were used to assess the variations in correlation results when using the MRAR and AMRDT measured at only one electrode location in each participant to correlate with CNC word scores. Linear Mixed Models (LMMs) were used to evaluate the relationship between MRARs/AMRDTs measured at individual electrode locations and CNC word scores. Spearman Rank correlation tests were used to evaluate the strength of association between CNC word scores measured in quiet and in noise with (1) the variances in MRARs and AMRDTs, and (2) the averaged MRAR or AMRDT across multiple electrodes tested for each participant.

RESULTS

There was no association between the MRAR and AMRDT. Using the MRAR and AMRDT measured at only one, randomly selected electrode location to assess their associations with CNC word scores could lead to opposite conclusions. Both the results of LMMs and Spearman Rank correlation tests showed that CNC word scores measured in quiet or at 10 dB SNR were not significantly correlated with the MRAR or AMRDT. In addition, the results of Spearman Rank correlation tests showed that the variances in MRARs and AMRDTs were not significantly correlated with CNC word scores measured in quiet or in noise.

CONCLUSIONS

The difference in AN sensitivity to AM cues is not the primary factor accounting for the variation in AMRDTs measured at different stimulation sites within individual CI users. The AN sensitivity to AM per se may not be a crucial factor for CNC word perception in quiet or at 10 dB SNR in postlingually deafened adult CI users. Using electrophysiological or psychophysical results measured at only one electrode location to correlate with speech perception scores in CI users can lead to inaccurate, if not wrong, conclusions.

Adult Cochlear Implant Users Versus Typical Hearing Persons: An Automatic Analysis of Acoustic-Prosodic Parameters

PURPOSE

The aim of this study was to investigate the speech prosody of postlingually deaf cochlear implant (CI) users compared with control speakers without hearing or speech impairment.

METHOD

Speech recordings of 74 CI users (37 males and 37 females) and 72 age-balanced control speakers (36 males and 36 females) are considered. All participants are German native speakers and read Der Nordwind und die Sonne (The North Wind and the Sun), a standard text in pathological speech analysis and phonetic transcriptions. Automatic acoustic analysis is performed considering pitch, loudness, and duration features, including speech rate and rhythm.

RESULTS

In general, duration and rhythm features differ between CI users and control speakers. CI users read slower and have a lower voiced segment ratio compared with control speakers. A lower voiced ratio goes along with a prolongation of the voiced segments' duration in male and with a prolongation of pauses in female CI users. Rhythm features in CI users have higher variability in the duration of vowels and consonants than in control speakers. The use of bilateral CIs showed no advantages concerning speech prosody features in comparison to unilateral use of CI.

CONCLUSIONS

Even after cochlear implantation and rehabilitation, the speech of postlingually deaf adults deviates from the speech of control speakers, which might be due to changed auditory feedback. We suggest considering changes in temporal aspects of speech in future rehabilitation strategies.

SUPPLEMENTAL MATERIAL

https://doi.org/10.23641/asha.21579171.

Assessing the Relationship Between Pitch Perception and Neural Health in Cochlear Implant Users

Various neural health estimates have been shown to indicate the density of spiral ganglion neurons in animal and modeling studies of cochlear implants (CIs). However, when applied to human CI users, these neural health estimates based on psychophysical and electrophysiological measures are not consistently correlated with each other or with the speech recognition performance. This study investigated whether the neural health estimates have stronger correlations with the temporal and place pitch sensitivity than with the speech recognition performance. On five electrodes in 12 tested ears of eight adult CI users, polarity effect (PE), multipulse integration (MPI), and interphase gap (IPG) effect on the amplitude growth function (AGF) of electrically evoked compound action potential (ECAP) were measured to estimate neural health, while thresholds of amplitude modulation frequency ranking (AMFR) and virtual channel ranking (VCR) were measured to indicate temporal and place pitch sensitivity. AzBio sentence recognition in noise was measured using the clinical CI processor for each ear. The results showed significantly poorer AMFR and VCR thresholds on the basal electrodes than on the apical and middle electrodes. Across ears and electrodes, only the IPG offset effect on ECAP AGF had a nearly significant negative correlation with the VCR threshold after removing the outliers. No significant across-ear correlations were found between the mean neural health estimates, mean pitch-ranking thresholds, and AzBio sentence recognition score. This study suggests that the central axon demyelination reflected by the IPG offset effect may be important for the place pitch sensitivity of CI users and that the IPG offset effect may be used to predict the perceptual resolution of virtual channels for CI programming.

Asymmetric temporal envelope sensitivity: Within- and across-ear envelope comparisons in listeners with bilateral cochlear implants

For listeners with bilateral cochlear implants (BiCIs), patient-specific differences in the interface between cochlear implant (CI) electrodes and the auditory nerve can lead to degraded temporal envelope information, compromising the ability to distinguish between targets of interest and background noise. It is unclear how comparisons of degraded temporal envelope information across spectral channels (i.e., electrodes) affect the ability to detect differences in the temporal envelope, specifically amplitude modulation (AM) rate. In this study, two pulse trains were presented simultaneously via pairs of electrodes in different places of stimulation, within and/or across ears, with identical or differing AM rates. Results from 11 adults with BiCIs indicated that sensitivity to differences in AM rate was greatest when stimuli were paired between different places of stimulation in the same ear. Sensitivity from pairs of electrodes was predicted by the poorer electrode in the pair or the difference in fidelity between both electrodes in the pair. These findings suggest that electrodes yielding poorer temporal fidelity act as a bottleneck to comparisons of temporal information across frequency and ears, limiting access to the cues used to segregate sounds, which has important implications for device programming and optimizing patient outcomes with CIs.

The development of cortical processing of speech differs between children with cochlear implants and normal hearing and changes with parental singing

Objective

The aim of the present study was to investigate speech processing development in children with normal hearing (NH) and cochlear implants (CI) groups using a multifeature event-related potential (ERP) paradigm. Singing is associated to enhanced attention and speech perception. Therefore, its connection to ERPs was investigated in the CI group.

Methods

The paradigm included five change types in a pseudoword: two easy- (duration, gap) and three difficult-to-detect (vowel, pitch, intensity) with CIs. The positive mismatch responses (pMMR), mismatch negativity (MMN), P3a and late differentiating negativity (LDN) responses of preschoolers (below 6 years 9 months) and schoolchildren (above 6 years 9 months) with NH or CIs at two time points (T1, T2) were investigated with Linear Mixed Modeling (LMM). For the CI group, the association of singing at home and ERP development was modeled with LMM.

Results

Overall, responses elicited by the easy- and difficult to detect changes differed between the CI and NH groups. Compared to the NH group, the CI group had smaller MMNs to vowel duration changes and gaps, larger P3a responses to gaps, and larger pMMRs and smaller LDNs to vowel identity changes. Preschoolers had smaller P3a responses and larger LDNs to gaps, and larger pMMRs to vowel identity changes than schoolchildren. In addition, the pMMRs to gaps increased from T1 to T2 in preschoolers. More parental singing in the CI group was associated with increasing pMMR and less parental singing with decreasing P3a amplitudes from T1 to T2.

Conclusion

The multifeature paradigm is suitable for assessing cortical speech processing development in children. In children with CIs, cortical discrimination is often reflected in pMMR and P3a responses, and in MMN and LDN responses in children with NH. Moreover, the cortical speech discrimination of children with CIs develops late, and over time and age, their speech sound change processing changes as does the processing of children with NH. Importantly, multisensory activities such as parental singing can lead to improvement in the discrimination and attention shifting toward speech changes in children with CIs. These novel results should be taken into account in future research and rehabilitation.

A direct comparison of voice pitch processing in acoustic and electric hearing

In single-sided deafness patients fitted with a cochlear implant (CI) in the affected ear and preserved normal hearing in the other ear, acoustic and electric hearing can be directly compared without the need for an external control group. Although poor pitch perception is a crucial limitation when listening through CIs, it remains unclear how exactly the cortical processing of pitch information differs between acoustic and electric hearing. Hence, we separately presented both ears of 20 of these patients with vowel sequences in which the pitch contours were either repetitive or variable, while simultaneously recording functional near-infrared spectroscopy (fNIRS) and EEG data. Overall, the results showed smaller and delayed auditory cortex activity in electric hearing, particularly for the P2 event-related potential component, which appears to reflect the processing of voice pitch information. Both the fNIRS data and EEG source reconstructions furthermore showed that vowel sequences with variable pitch contours evoked additional activity in posterior right auditory cortex in electric but not acoustic hearing. This surprising discrepancy demonstrates, firstly, that the acoustic detail transmitted by CIs is sufficient to distinguish between speech sounds that only vary regarding their pitch information. Secondly, the absence of a condition difference when stimulating the normal-hearing ears suggests a saturation of cortical activity levels following unilateral deafness. Taken together, these results provide strong evidence in favour of using CIs in this patient group.

Is speech intelligibility what speech intelligibility tests test?

Natural, conversational speech signals contain sources of symbolic and iconic information, both of which are necessary for the full understanding of speech. But speech intelligibility tests, which are generally derived from written language, present only symbolic information sources, including lexical semantics and syntactic structures. Speech intelligibility tests exclude almost all sources of information about talkers, including their communicative intentions and their cognitive states and processes. There is no reason to suspect that either hearing impairment or noise selectively affect perception of only symbolic information. We must therefore conclude that diagnosis of good or poor speech intelligibility on the basis of standard speech tests is based on measurement of only a fraction of the task of speech perception. This paper presents a descriptive comparison of information sources present in three widely used speech intelligibility tests and spontaneous, conversational speech elicited using a referential communication task. The aim of this comparison is to draw attention to the differences in not just the signals, but the tasks of listeners perceiving these different speech signals and to highlight the implications of these differences for the interpretation and generalizability of speech intelligibility test results.

Age-Related Changes in Voice Emotion Recognition by Postlingually Deafened Listeners With Cochlear Implants

OBJECTIVES

Identification of emotional prosody in speech declines with age in normally hearing (NH) adults. Cochlear implant (CI) users have deficits in the perception of prosody, but the effects of age on vocal emotion recognition by adult postlingually deaf CI users are not known. The objective of the present study was to examine age-related changes in CI users' and NH listeners' emotion recognition.

DESIGN

Participants included 18 CI users (29.6 to 74.5 years) and 43 NH adults (25.8 to 74.8 years). Participants listened to emotion-neutral sentences spoken by a male and female talker in five emotions (happy, sad, scared, angry, neutral). NH adults heard them in four conditions: unprocessed (full spectrum) speech, 16-channel, 8-channel, and 4-channel noise-band vocoded speech. The adult CI users only listened to unprocessed (full spectrum) speech. Sensitivity (d') to emotions and Reaction Times were obtained using a single-interval, five-alternative, forced-choice paradigm.

RESULTS

For NH participants, results indicated age-related declines in Accuracy and d', and age-related increases in Reaction Time in all conditions. Results indicated an overall deficit, as well as age-related declines in overall d' for CI users, but Reaction Times were elevated compared with NH listeners and did not show age-related changes. Analysis of Accuracy scores (hit rates) were generally consistent with d' data.

CONCLUSIONS

Both CI users and NH listeners showed age-related deficits in emotion identification. The CI users' overall deficit in emotion perception, and their slower response times, suggest impaired social communication which may in turn impact overall well-being, particularly so for older CI users, as lower vocal emotion recognition scores have been associated with poorer subjective quality of life in CI patients.

The effect of simulated insertion depth differences on the vocal pitches of cochlear implant users

Cochlear implant (CI) users often produce different vocal pitches when using their left versus right CI. One possible explanation for this is that insertion depth differs across the two CIs. The goal of this study was to investigate the role of electrode insertion depth in the production of vocal pitch. Eleven individuals with bilateral CIs used maps simulating differences in insertion depth. Participants produced a sustained vowel and sang Happy Birthday. Approximately half the participants significantly shifted the pitch of their voice in response to different simulated insertion depths. The results suggest insertion depth differences can alter produced vocal pitch.

Difficulties Experienced by Older Listeners in Utilizing Voice Cues for Speaker Discrimination

Human listeners are assumed to apply different strategies to improve speech recognition in background noise. Young listeners with normal hearing (NH), e.g., have been shown to follow the voice of a particular speaker based on the fundamental (F0) and formant frequencies, which are both influenced by the gender, age, and size of the speaker. However, the auditory and cognitive processes that underlie the extraction and discrimination of these voice cues across speakers may be subject to age-related decline. The present study aimed to examine the utilization of F0 and formant cues for voice discrimination (VD) in older adults with hearing expected for their age. Difference limens (DLs) for VD were estimated in 15 healthy older adults (65–78 years old) and 35 young adults (18–35 years old) using only F0 cues, only formant frequency cues, and a combination of F0 + formant frequencies. A three-alternative forced-choice paradigm with an adaptive-tracking threshold-seeking procedure was used. Wechsler backward digit span test was used as a measure of auditory working memory. Trail Making Test (TMT) was used to provide cognitive information reflecting a combined effect of processing speed, mental flexibility, and executive control abilities. The results showed that (a) the mean VD thresholds of the older adults were poorer than those of the young adults for all voice cues, although larger variability was observed among the older listeners; (b) both age groups found the formant cues more beneficial for VD, compared to the F0 cues, and the combined (F0 + formant) cues resulted in better thresholds, compared to each cue separately; (c) significant associations were found for the older adults in the combined F0 + formant condition between VD and TMT scores, and between VD and hearing sensitivity, supporting the notion that a decline with age in both top-down and bottom-up mechanisms may hamper the ability of older adults to discriminate between voices. The present findings suggest that older listeners may have difficulty following the voice of a specific speaker and thus implementing doing so as a strategy for listening amid noise. This may contribute to understanding their reported difficulty listening in adverse conditions.

More Than Words: the Relative Roles of Prosody and Semantics in the Perception of Emotions in Spoken Language by Postlingual Cochlear Implant Users

OBJECTIVES

The processing of emotional speech calls for the perception and integration of semantic and prosodic cues. Although cochlear implants allow for significant auditory improvements, they are limited in the transmission of spectro-temporal fine-structure information that may not support the processing of voice pitch cues. The goal of the current study is to compare the performance of postlingual cochlear implant (CI) users and a matched control group on perception, selective attention, and integration of emotional semantics and prosody.

DESIGN

Fifteen CI users and 15 normal hearing (NH) peers (age range, 18-65 years) 1istened to spoken sentences composed of different combinations of four discrete emotions (anger, happiness, sadness, and neutrality) presented in prosodic and semantic channels-T-RES: Test for Rating Emotions in Speech. In three separate tasks, listeners were asked to attend to the sentence as a whole, thus integrating both speech channels (integration), or to focus on one channel only (rating of target emotion) and ignore the other (selective attention). Their task was to rate how much they agreed that the sentence conveyed each of the predefined emotions. In addition, all participants performed standard tests of speech perception.

RESULTS

When asked to focus on one channel, semantics or prosody, both groups rated emotions similarly with comparable levels of selective attention. When the task was called for channel integration, group differences were found. CI users appeared to use semantic emotional information more than did their NH peers. CI users assigned higher ratings than did their NH peers to sentences that did not present the target emotion, indicating some degree of confusion. In addition, for CI users, individual differences in speech comprehension over the phone and identification of intonation were significantly related to emotional semantic and prosodic ratings, respectively.

CONCLUSIONS

CI users and NH controls did not differ in perception of prosodic and semantic emotions and in auditory selective attention. However, when the task called for integration of prosody and semantics, CI users overused the semantic information (as compared with NH). We suggest that as CI users adopt diverse cue weighting strategies with device experience, their weighting of prosody and semantics differs from those used by NH. Finally, CI users may benefit from rehabilitation strategies that strengthen perception of prosodic information to better understand emotional speech.

Emotional Responses to Non-Speech Sounds for Hearing-aid and Bimodal Cochlear-Implant Listeners

The purpose of this project was to evaluate differences between groups and device configurations for emotional responses to non-speech sounds. Three groups of adults participated: 1) listeners with normal hearing with no history of device use, 2) hearing aid candidates with or without hearing aid experience, and 3) bimodal cochlear-implant listeners with at least 6 months of implant use. Participants (n = 18 in each group) rated valence and arousal of pleasant, neutral, and unpleasant non-speech sounds. Listeners with normal hearing rated sounds without hearing devices. Hearing aid candidates rated sounds while using one or two hearing aids. Bimodal cochlear-implant listeners rated sounds while using a hearing aid alone, a cochlear implant alone, or the hearing aid and cochlear implant simultaneously. Analysis revealed significant differences between groups in ratings of pleasant and unpleasant stimuli; ratings from hearing aid candidates and bimodal cochlear-implant listeners were less extreme (less pleasant and less unpleasant) than were ratings from listeners with normal hearing. Hearing aid candidates' ratings were similar with one and two hearing aids. Bimodal cochlear-implant listeners' ratings of valence were higher (more pleasant) in the configuration without a hearing aid (implant only) than in the two configurations with a hearing aid (alone or with an implant). These data support the need for further investigation into hearing device optimization to improve emotional responses to non-speech sounds for adults with hearing loss.

Pleasantness Ratings of Musical Dyads in Cochlear Implant Users

Cochlear implants have been used to restore hearing to more than half a million people around the world. The restored hearing allows most recipients to understand spoken speech without relying on visual cues. While speech comprehension in quiet is generally high for recipients, many complain about the sound of music. The present study examines consonance and dissonance perception in nine cochlear implant users and eight people with no known hearing loss. Participants completed web-based assessments to characterize low-level psychophysical sensitivities to modulation and pitch, as well as higher-level measures of musical pleasantness and speech comprehension in background noise. The underlying hypothesis is that sensitivity to modulation and pitch, in addition to higher levels of musical sophistication, relate to higher-level measures of music and speech perception. This hypothesis tested true with strong correlations observed between measures of modulation and pitch with measures of consonance ratings and speech recognition. Additionally, the cochlear implant users who were the most sensitive to modulations and pitch, and who had higher musical sophistication scores, had similar pleasantness ratings as those with no known hearing loss. The implication is that better coding and focused rehabilitation for modulation and pitch sensitivity will broadly improve perception of music and speech for cochlear implant users.

Adults with cochlear implants can use prosody to determine the clausal structure of spoken sentences

Speech prosody, including pitch contour, word stress, pauses, and vowel lengthening, can aid the detection of the clausal structure of a multi-clause sentence and this, in turn, can help listeners determine the meaning. However, for cochlear implant (CI) users, the reduced acoustic richness of the signal raises the question of whether CI users may have difficulty using sentence prosody to detect syntactic clause boundaries within sentences or whether this ability is rescued by the redundancy of the prosodic features that normally co-occur at clause boundaries. Twenty-two CI users, ranging in age from 19 to 77 years old, recalled three types of sentences: sentences in which the prosodic pattern was appropriate to the location of a clause boundary within the sentence (congruent prosody), sentences with reduced prosodic information, or sentences in which the location of the clause boundary and the prosodic marking of a clause boundary were placed in conflict. The results showed the presence of congruent prosody to be associated with superior sentence recall and a reduced processing effort as indexed by the pupil dilation. The individual differences in a standard test of word recognition (consonant-nucleus-consonant score) were related to the recall accuracy as well as the processing effort. The outcomes are discussed in terms of the redundancy of the prosodic features, which normally accompany a clause boundary and processing effort.

Estimating the reduced benefit of infant-directed speech in cochlear implant-related speech processing

Caregivers modify their speech when talking to infants, a specific type of speech known as infant-directed speech (IDS). This speaking style facilitates language learning compared to adult-directed speech (ADS) in infants with normal hearing (NH). While infants with NH and those with cochlear implants (CIs) prefer listening to IDS over ADS, it is yet unknown how CI processing may affect the acoustic distinctiveness between ADS and IDS, as well as the degree of intelligibility of these. This study analyzed speech of seven female adult talkers to model the effects of simulated CI processing on (1) acoustic distinctiveness between ADS and IDS, (2) estimates of intelligibility of caregivers' speech in ADS and IDS, and (3) individual differences in caregivers' ADS-to-IDS modification and estimated speech intelligibility. Results suggest that CI processing is substantially detrimental to the acoustic distinctiveness between ADS and IDS, as well as to the intelligibility benefit derived from ADS-to-IDS modifications. Moreover, the observed variability across individual talkers in acoustic implementation of ADS-to-IDS modification and the estimated speech intelligibility was significantly reduced due to CI processing. The findings are discussed in the context of the link between IDS and language learning in infants with CIs.

Accuracy and cue use in word segmentation for cochlear-implant listeners and normal-hearing listeners presented vocoded speech

Cochlear-implant (CI) listeners experience signal degradation, which leads to poorer speech perception than normal-hearing (NH) listeners. In the present study, difficulty with word segmentation, the process of perceptually parsing the speech stream into separate words, is considered as a possible contributor to this decrease in performance. CI listeners were compared to a group of NH listeners (presented with unprocessed speech and eight-channel noise-vocoded speech) in their ability to segment phrases with word segmentation ambiguities (e.g., "an iceman" vs "a nice man"). The results showed that CI listeners and NH listeners were worse at segmenting words when hearing processed speech than NH listeners were when presented with unprocessed speech. When viewed at a broad level, all of the groups used cues to word segmentation in similar ways. Detailed analyses, however, indicated that the two processed speech groups weighted top-down knowledge cues to word boundaries more and weighted acoustic cues to word boundaries less relative to NH listeners presented with unprocessed speech.

Sex-mismatch benefit for speech-in-speech recognition by pediatric and adult cochlear implant users

This project investigated whether pediatric (5-14 years) and adult (30-60 years) cochlear implant (CI) users benefit from a target/masker sex-mismatch for speech-in-speech recognition. Speech recognition thresholds were estimated in a two-male-talker or a two-female-talker masker. Target and masker speech were either sex-matched or sex-mismatched. For both age groups, performance for sex-matched talkers was worse for male than female speech. Sex-mismatch benefit was observed for the two-male-talker masker, indicating CI users can benefit from a target/masker sex mismatch. No benefit was observed for the two-female-talker masker, suggesting this effect may depend on the relative contributions of energetic and informational masking.

Perception of Child-Directed Versus Adult-Directed Emotional Speech in Pediatric Cochlear Implant Users

OBJECTIVES

Cochlear implants (CIs) are remarkable in allowing individuals with severe to profound hearing loss to perceive speech. Despite these gains in speech understanding, however, CI users often struggle to perceive elements such as vocal emotion and prosody, as CIs are unable to transmit the spectro-temporal detail needed to decode affective cues. This issue becomes particularly important for children with CIs, but little is known about their emotional development. In a previous study, pediatric CI users showed deficits in voice emotion recognition with child-directed stimuli featuring exaggerated prosody. However, the large intersubject variability and differential developmental trajectory known in this population incited us to question the extent to which exaggerated prosody would facilitate performance in this task. Thus, the authors revisited the question with both adult-directed and child-directed stimuli.

DESIGN

Vocal emotion recognition was measured using both child-directed (CDS) and adult-directed (ADS) speech conditions. Pediatric CI users, aged 7-19 years old, with no cognitive or visual impairments and who communicated through oral communication with English as the primary language participated in the experiment (n = 27). Stimuli comprised 12 sentences selected from the HINT database. The sentences were spoken by male and female talkers in a CDS or ADS manner, in each of the five target emotions (happy, sad, neutral, scared, and angry). The chosen sentences were semantically emotion-neutral. Percent correct emotion recognition scores were analyzed for each participant in each condition (CDS vs. ADS). Children also completed cognitive tests of nonverbal IQ and receptive vocabulary, while parents completed questionnaires of CI and hearing history. It was predicted that the reduced prosodic variations found in the ADS condition would result in lower vocal emotion recognition scores compared with the CDS condition. Moreover, it was hypothesized that cognitive factors, perceptual sensitivity to complex pitch changes, and elements of each child's hearing history may serve as predictors of performance on vocal emotion recognition.

RESULTS

Consistent with our hypothesis, pediatric CI users scored higher on CDS compared with ADS speech stimuli, suggesting that speaking with an exaggerated prosody-akin to "motherese"-may be a viable way to convey emotional content. Significant talker effects were also observed in that higher scores were found for the female talker for both conditions. Multiple regression analysis showed that nonverbal IQ was a significant predictor of CDS emotion recognition scores while Years using CI was a significant predictor of ADS scores. Confusion matrix analyses revealed a dependence of results on specific emotions; for the CDS condition's female talker, participants had high sensitivity (d' scores) to happy and low sensitivity to the neutral sentences while for the ADS condition, low sensitivity was found for the scared sentences.

CONCLUSIONS

In general, participants had higher vocal emotion recognition to the CDS condition which also had more variability in pitch and intensity and thus more exaggerated prosody, in comparison to the ADS condition. Results suggest that pediatric CI users struggle with vocal emotion perception in general, particularly to adult-directed speech. The authors believe these results have broad implications for understanding how CI users perceive emotions both from an auditory communication standpoint and a socio-developmental perspective.

Electro-Haptic Stimulation: A New Approach for Improving Cochlear-Implant Listening

Cochlear implants (CIs) have been remarkably successful at restoring speech perception for severely to profoundly deaf individuals. Despite their success, several limitations remain, particularly in CI users' ability to understand speech in noisy environments, locate sound sources, and enjoy music. A new multimodal approach has been proposed that uses haptic stimulation to provide sound information that is poorly transmitted by the implant. This augmenting of the electrical CI signal with haptic stimulation (electro-haptic stimulation; EHS) has been shown to improve speech-in-noise performance and sound localization in CI users. There is also evidence that it could enhance music perception. We review the evidence of EHS enhancement of CI listening and discuss key areas where further research is required. These include understanding the neural basis of EHS enhancement, understanding the effectiveness of EHS across different clinical populations, and the optimization of signal-processing strategies. We also discuss the significant potential for a new generation of haptic neuroprosthetic devices to aid those who cannot access hearing-assistive technology, either because of biomedical or healthcare-access issues. While significant further research and development is required, we conclude that EHS represents a promising new approach that could, in the near future, offer a non-invasive, inexpensive means of substantially improving clinical outcomes for hearing-impaired individuals.

Bimodal Benefit for Music Perception: Effect of Acoustic Bandwidth

Purpose The challenges associated with cochlear implant (CI)-mediated listening are well documented; however, they can be mitigated through the provision of aided acoustic hearing in the contralateral ear-a configuration termed bimodal hearing. This study extends previous literature to examine the effect of acoustic bandwidth in the non-CI ear for music perception. The primary aim was to determine the minimum and optimum acoustic bandwidth necessary to obtain bimodal benefit for music perception and speech perception. Method Participants included 12 adult bimodal listeners and 12 adult control listeners with normal hearing. Music perception was assessed via measures of timbre perception and subjective sound quality of real-world music samples. Speech perception was assessed via monosyllabic word recognition in quiet. Acoustic stimuli were presented to the non-CI ear in the following filter conditions: < 125, < 250, < 500, and < 750 Hz, and wideband (full bandwidth). Results Generally, performance for all stimuli improved with increasing acoustic bandwidth; however, the bandwidth that is both minimally and optimally beneficial may be dependent upon stimulus type. On average, music sound quality required wideband amplification, whereas speech recognition with a male talker in quiet required a narrower acoustic bandwidth (< 250 Hz) for significant benefit. Still, average speech recognition performance continued to improve with increasing bandwidth. Conclusion Further research is warranted to examine optimal acoustic bandwidth for additional stimulus types; however, these findings indicate that wideband amplification is most appropriate for speech and music perception in individuals with bimodal hearing.

The Sensitivity of the Electrically Stimulated Auditory Nerve to Amplitude Modulation Cues Declines With Advanced Age

OBJECTIVES

This study aimed to investigate effects of aging and duration of deafness on sensitivity of the auditory nerve (AN) to amplitude modulation (AM) cues delivered using trains of biphasic pulses in adult cochlear implant (CI) users.

DESIGN

There were 21 postlingually deaf adult CI users who participated in this study. All study participants used a Cochlear Nucleus device with a full electrode array insertion in the test ear. The stimulus was a 200-ms pulse train with a pulse rate of 2000 pulses per second. This carrier pulse train was sinusodially AM at four modulation rates (20, 40, 100, 200 Hz). The peak amplitude of the modulated pulse train was the maximum comfortable level (i.e., C level) measured for the carrier pulse train. The electrically evoked compound action potential (eCAP) to each of the 20 pulses selected over the last two AM cycles were measured. In addition, eCAPs to single pulses were measured with the probe levels corresponding to the levels of 20 selected pulses from each AM pulse train. There were seven electrodes across the array evaluated in 16 subjects (i.e., electrodes 3 or 4, 6, 9, 12, 15, 18, and 21). For the remaining five subjects, 4 to 5 electrodes were tested due to impedance issues or time constraints. The modulated response amplitude ratio (MRAR) was calculated as the ratio of the difference in the maximum and the minimum eCAP amplitude measured for the AM pulse train to that measured for the single pulse, and served as the dependent variable. Age at time of testing and duration of deafness measured/defined using three criteria served as the independent variables. Linear Mixed Models were used to assess the effects of age at testing and duration of deafness on the MRAR.

RESULTS

Age at testing had a strong, negative effect on the MRAR. For each subject, the duration of deafness varied substantially depending on how it was defined/measured, which demonstrates the difficulty of accurately measuring the duration of deafness in adult CI users. There was no clear or reliable trend showing a relationship between the MRAR measured at any AM rate and duration of deafness defined by any criteria. After controlling for the effect of age at testing, MRARs measured at 200 Hz and basal electrode locations (i.e., electrodes 3 and 6) were larger than those measured at any other AM rate and apical electrode locations (i.e., electrodes 18 and 21).

CONCLUSIONS

The AN sensitivity to AM cues implemented in the pulse-train stimulation significantly declines with advanced age. Accurately measuring duration of deafness in adult CI users is challenging, which, at least partially, might have accounted for the inconclusive findings in the relationship between the duration of deafness and the AN sensitivity to AM cues in this study.

Weighting of Prosodic and Lexical-Semantic Cues for Emotion Identification in Spectrally Degraded Speech and With Cochlear Implants

OBJECTIVES

Normally-hearing (NH) listeners rely more on prosodic cues than on lexical-semantic cues for emotion perception in speech. In everyday spoken communication, the ability to decipher conflicting information between prosodic and lexical-semantic cues to emotion can be important: for example, in identifying sarcasm or irony. Speech degradation in cochlear implants (CIs) can be sufficiently overcome to identify lexical-semantic cues, but the distortion of voice pitch cues makes it particularly challenging to hear prosody with CIs. The purpose of this study was to examine changes in relative reliance on prosodic and lexical-semantic cues in NH adults listening to spectrally degraded speech and adult CI users. We hypothesized that, compared with NH counterparts, CI users would show increased reliance on lexical-semantic cues and reduced reliance on prosodic cues for emotion perception. We predicted that NH listeners would show a similar pattern when listening to CI-simulated versions of emotional speech.

DESIGN

Sixteen NH adults and 8 postlingually deafened adult CI users participated in the study. Sentences were created to convey five lexical-semantic emotions (angry, happy, neutral, sad, and scared), with five sentences expressing each category of emotion. Each of these 25 sentences was then recorded with the 5 (angry, happy, neutral, sad, and scared) prosodic emotions by 2 adult female talkers. The resulting stimulus set included 125 recordings (25 Sentences × 5 Prosodic Emotions) per talker, of which 25 were congruent (consistent lexical-semantic and prosodic cues to emotion) and the remaining 100 were incongruent (conflicting lexical-semantic and prosodic cues to emotion). The recordings were processed to have 3 levels of spectral degradation: full-spectrum, CI-simulated (noise-vocoded) to have 8 channels and 16 channels of spectral information, respectively. Twenty-five recordings (one sentence per lexical-semantic emotion recorded in all five prosodies) were used for a practice run in the full-spectrum condition. The remaining 100 recordings were used as test stimuli. For each talker and condition of spectral degradation, listeners indicated the emotion associated with each recording in a single-interval, five-alternative forced-choice task. The responses were scored as proportion correct, where "correct" responses corresponded to the lexical-semantic emotion. CI users heard only the full-spectrum condition.

RESULTS

The results showed a significant interaction between hearing status (NH, CI) and congruency in identifying the lexical-semantic emotion associated with the stimuli. This interaction was as predicted, that is, CI users showed increased reliance on lexical-semantic cues in the incongruent conditions, while NH listeners showed increased reliance on the prosodic cues in the incongruent conditions. As predicted, NH listeners showed increased reliance on lexical-semantic cues to emotion when the stimuli were spectrally degraded.

CONCLUSIONS

The present study confirmed previous findings of prosodic dominance for emotion perception by NH listeners in the full-spectrum condition. Further, novel findings with CI patients and NH listeners in the CI-simulated conditions showed reduced reliance on prosodic cues and increased reliance on lexical-semantic cues to emotion. These results have implications for CI listeners' ability to perceive conflicts between prosodic and lexical-semantic cues, with repercussions for their identification of sarcasm and humor. Understanding instances of sarcasm or humor can impact a person's ability to develop relationships, follow conversation, understand vocal emotion and intended message of a speaker, following jokes, and everyday communication in general.

Effect of Channel Interaction on Vocal Cue Perception in Cochlear Implant Users

Speech intelligibility in multitalker settings is challenging for most cochlear implant (CI) users. One possibility for this limitation is the suboptimal representation of vocal cues in implant processing, such as the fundamental frequency (F0), and the vocal tract length (VTL). Previous studies suggested that while F0 perception depends on spectrotemporal cues, VTL perception relies largely on spectral cues. To investigate how spectral smearing in CIs affects vocal cue perception in speech-on-speech (SoS) settings, adjacent electrodes were simultaneously stimulated using current steering in 12 Advanced Bionics users to simulate channel interaction. In current steering, two adjacent electrodes are simultaneously stimulated forming a channel of parallel stimulation. Three such stimulation patterns were used: Sequential (one current steering channel), Paired (two channels), and Triplet stimulation (three channels). F0 and VTL just-noticeable differences (JNDs; Task 1), in addition to SoS intelligibility (Task 2) and comprehension (Task 3), were measured for each stimulation strategy. In Tasks 2 and 3, four maskers were used: the same female talker, a male voice obtained by manipulating both F0 and VTL (F0+VTL) of the original female speaker, a voice where only F0 was manipulated, and a voice where only VTL was manipulated. JNDs were measured relative to the original voice for the F0, VTL, and F0+VTL manipulations. When spectral smearing was increased from Sequential to Triplet, a significant deterioration in performance was observed for Tasks 1 and 2, with no differences between Sequential and Paired stimulation. Data from Task 3 were inconclusive. These results imply that CI users may tolerate certain amounts of channel interaction without significant reduction in performance on tasks relying on voice perception. This points to possibilities for using parallel stimulation in CIs for reducing power consumption.

Neural encoding of spectro-temporal cues at slow and near speech-rate in cochlear implant users

The ability to process rapid modulations in the spectro-temporal structure of sounds is critical for speech comprehension. For users of cochlear implants (CIs), spectral cues in speech are conveyed by differential stimulation of electrode contacts along the cochlea, and temporal cues in terms of the amplitude of stimulating electrical pulses, which track the amplitude-modulated (AM'ed) envelope of speech sounds. Whilst survival of inner-ear neurons and spread of electrical current are known factors that limit the representation of speech information in CI listeners, limitations in the neural representation of dynamic spectro-temporal cues common to speech are also likely to play a role. We assessed the ability of CI listeners to process spectro-temporal cues varying at rates typically present in human speech. Employing an auditory change complex (ACC) paradigm, and a slow (0.5Hz) alternating rate between stimulating electrodes, or different AM frequencies, to evoke a transient cortical ACC, we demonstrate that CI listeners-like normal-hearing listeners-are sensitive to transitions in the spectral- and temporal-domain. However, CI listeners showed impaired cortical responses when either spectral or temporal cues were alternated at faster, speech-like (6-7Hz), rates. Specifically, auditory change following responses-reliably obtained in normal-hearing listeners-were small or absent in CI users, indicating that cortical adaptation to alternating cues at speech-like rates is stronger under electrical stimulation. In CI listeners, temporal processing was also influenced by the polarity-behaviourally-and rate of presentation of electrical pulses-both neurally and behaviorally. Limitations in the ability to process dynamic spectro-temporal cues will likely impact speech comprehension in CI users.

Bimodal Benefits Revealed by Categorical Perception of Lexical Tones in Mandarin-Speaking Kindergarteners With a Cochlear Implant and a Contralateral Hearing Aid

Purpose Pitch reception poses challenges for individuals with cochlear implants (CIs), and adding a hearing aid (HA) in the nonimplanted ear is potentially beneficial. The current study used fine-scale synthetic speech stimuli to investigate the bimodal benefit for lexical tone categorization in Mandarin-speaking kindergarteners using a CI and an HA in opposite ears. Method The data were collected from 16 participants who were required to complete two classical tasks for speech categorical perception (CP) with CI + HA device condition and CI alone condition. Linear mixed-effects models were constructed to evaluate the identification and discrimination scores across different device conditions. Results The bimodal kindergarteners showed CP for the continuum varying from Mandarin Tone 1 and Tone 2. Moreover, the additional acoustic information from the contralateral HA contributes to improved lexical tone categorization, with a steeper slope, a higher discrimination score of between-category stimuli pair, and an improved peakedness score (i.e., an increased benefit magnitude for discriminations of between-category over within-category pairs) for the CI + HA condition than the CI alone condition. The bimodal kindergarteners with better residual hearing thresholds at 250 Hz level in the nonimplanted ear could perceive lexical tones more categorically. Conclusion The enhanced CP results with bimodal listening provide clear evidence for the clinical practice to fit a contralateral HA in the nonimplanted ear in kindergarteners with unilateral CIs with direct benefits from the low-frequency acoustic hearing.

Effects of noise on integration of acoustic and electric hearing within and across ears

In bimodal listening, cochlear implant (CI) users combine electric hearing (EH) in one ear and acoustic hearing (AH) in the other ear. In electric-acoustic stimulation (EAS), CI users combine EH and AH in the same ear. In quiet, integration of EH and AH has been shown to be better with EAS, but with greater sensitivity to tonotopic mismatch in EH. The goal of the present study was to evaluate how external noise might affect integration of AH and EH within or across ears. Recognition of monosyllabic words was measured for normal-hearing subjects listening to simulations of unimodal (AH or EH alone), EAS, and bimodal listening in quiet and in speech-shaped steady noise (10 dB, 0 dB signal-to-noise ratio). The input/output frequency range for AH was 0.1–0.6 kHz. EH was simulated using an 8-channel noise vocoder. The output frequency range was 1.2–8.0 kHz to simulate a shallow insertion depth. The input frequency range was either matched (1.2–8.0 kHz) or mismatched (0.6–8.0 kHz) to the output frequency range; the mismatched input range maximized the amount of speech information, while the matched input resulted in some speech information loss. In quiet, tonotopic mismatch differently affected EAS and bimodal performance. In noise, EAS and bimodal performance was similarly affected by tonotopic mismatch. The data suggest that tonotopic mismatch may differently affect integration of EH and AH in quiet and in noise.