Effect of cochlear implants on children's perception and production of speech prosody

Emotional prosodic expression of children with hearing aids or cochlear implants, rated by adults and peers

OBJECTIVE

The emotional prosodic expression potential of children with cochlear implants is poorer than that of normal hearing peers. Though little is known about children with hearing aids.

DESIGN

This study was set up to generate a better understanding of hearing aid users' prosodic identifiability compared to cochlear implant users and peers without hearing loss.

STUDY SAMPLE

Emotional utterances of 75 Dutch speaking children (7 - 12 yr; 26 CHA, 23 CCI, 26 CNH) were gathered. Utterances were evaluated blindly by normal hearing Dutch listeners: 22 children and 9 adults (17 - 24 yrs) for resemblance to three emotions (happiness, sadness, anger).

RESULTS

Emotions were more accurately recognised by adults than by children. Both children and adults correctly judged happiness significantly less often in CCI than in CNH. Also, adult listeners confused happiness with sadness more often in both CHA and CCI than in CNH.

CONCLUSIONS

Children and adults are able to accurately evaluate the emotions expressed through speech by children with varying degrees of hearing loss, ranging from mild to profound, nearly as well as they can with typically hearing children. The favourable outcomes emphasise the resilience of children with hearing loss in developing effective emotional communication skills.

Prosodic and Visual Cues Facilitate Irony Comprehension by Mandarin-Speaking Children With Cochlear Implants

PURPOSE

This study investigated irony comprehension by Mandarin-speaking children with cochlear implants, focusing on how prosodic and visual cues contribute to their comprehension, and whether second-order Theory of Mind is required for using these cues.

METHOD

We tested 52 Mandarin-speaking children with cochlear implants (aged 3-7 years) and 52 age- and gender-matched children with normal hearing. All children completed a Theory of Mind test and a story comprehension test. Ironic stories were presented in three conditions, each providing different cues: (a) context-only, (b) context and prosody, and (c) context, prosody, and visual cues. Comparisons were conducted on the accuracy of story understanding across the three conditions to examine the role of prosodic and visual cues.

RESULTS

The results showed that, compared to the context-only condition, the additional prosodic and visual cues both improved the accuracy of irony comprehension for children with cochlear implants, similar to their normal-hearing peers. Furthermore, such improvements were observed for all children, regardless of whether they passed the second-order Theory of Mind test or not.

CONCLUSIONS

This study is the first to demonstrate the benefits of prosodic and visual cues in irony comprehension, without reliance on second-order Theory of Mind, for Mandarin-speaking children with cochlear implants. It implies potential insights for utilizing prosodic and visual cues in intervention strategies to promote irony comprehension.

Prelingually Deaf Children With Cochlear Implants Show Better Perception of Voice Cues and Speech in Competing Speech Than Postlingually Deaf Adults With Cochlear Implants

OBJECTIVES

Postlingually deaf adults with cochlear implants (CIs) have difficulties with perceiving differences in speakers' voice characteristics and benefit little from voice differences for the perception of speech in competing speech. However, not much is known yet about the perception and use of voice characteristics in prelingually deaf implanted children with CIs. Unlike CI adults, most CI children became deaf during the acquisition of language. Extensive neuroplastic changes during childhood could make CI children better at using the available acoustic cues than CI adults, or the lack of exposure to a normal acoustic speech signal could make it more difficult for them to learn which acoustic cues they should attend to. This study aimed to examine to what degree CI children can perceive voice cues and benefit from voice differences for perceiving speech in competing speech, comparing their abilities to those of normal-hearing (NH) children and CI adults.

DESIGN

CI children's voice cue discrimination (experiment 1), voice gender categorization (experiment 2), and benefit from target-masker voice differences for perceiving speech in competing speech (experiment 3) were examined in three experiments. The main focus was on the perception of mean fundamental frequency (F0) and vocal-tract length (VTL), the primary acoustic cues related to speakers' anatomy and perceived voice characteristics, such as voice gender.

RESULTS

CI children's F0 and VTL discrimination thresholds indicated lower sensitivity to differences compared with their NH-age-equivalent peers, but their mean discrimination thresholds of 5.92 semitones (st) for F0 and 4.10 st for VTL indicated higher sensitivity than postlingually deaf CI adults with mean thresholds of 9.19 st for F0 and 7.19 st for VTL. Furthermore, CI children's perceptual weighting of F0 and VTL cues for voice gender categorization closely resembled that of their NH-age-equivalent peers, in contrast with CI adults. Finally, CI children had more difficulties in perceiving speech in competing speech than their NH-age-equivalent peers, but they performed better than CI adults. Unlike CI adults, CI children showed a benefit from target-masker voice differences in F0 and VTL, similar to NH children.

CONCLUSION

Although CI children's F0 and VTL voice discrimination scores were overall lower than those of NH children, their weighting of F0 and VTL cues for voice gender categorization and their benefit from target-masker differences in F0 and VTL resembled that of NH children. Together, these results suggest that prelingually deaf implanted CI children can effectively utilize spectrotemporally degraded F0 and VTL cues for voice and speech perception, generally outperforming postlingually deaf CI adults in comparable tasks. These findings underscore the presence of F0 and VTL cues in the CI signal to a certain degree and suggest other factors contributing to the perception challenges faced by CI adults.

A meta-analysis of expressive prosody in cochlear implant users

INTRODUCTION

Prosody is used to express indexical (identifying the talker), linguistic (e.g., question intonation, lexical stress), pragmatic (e.g., contrastive stress, sarcasm), and emotional/affective functions. It is manifested through changes in fundamental frequency (f0), intensity, and duration. F0 and intensity are degraded when perceived through a cochlear implant (CI). The purpose of this meta-analysis is to compare expressive prosody in speech produced by CI users versus normal hearing peers.

METHODS

A systematic search of the literature found 25 articles that met all inclusion criteria. These articles were assessed for quality, and data pertaining to the expression of f0, intensity, and duration, as well as classification accuracy and appropriateness ratings from normal hearing listeners, were extracted and meta-analyzed using random effects models.

RESULTS

The articles included in the meta-analysis were generally of acceptable or high quality. Meta-analyses revealed significant differences between individuals with CIs vs. normal hearing on all measures except mean f0, mean intensity, and rhythm. Effect sizes were generally medium to large. There was significant heterogeneity across studies, but little evidence of publication bias.

CONCLUSIONS

CI users speak with less variable f0, smaller f0 contours, more variable intensity, a slower speech rate, and reduced final lengthening at syntactic boundaries. These acoustic differences are reflected in significantly poorer ratings of speech produced by CI users compared to their normal hearing peers, as assessed by groups of normal hearing listeners. Because atypical expressive prosody is associated with negative outcomes, clinicians should consider targeting prosody when working with individuals who use CIs.

Effect of Age and Unaided Acoustic Hearing on Pediatric Cochlear Implant Users' Ability to Distinguish Yes/No Statements and Questions

PURPOSE

The purpose of this study was to evaluate the ability to discriminate yes/no questions from statements in three groups of children: bilateral cochlear implant (CI) users, nontraditional CI users with aidable hearing preoperatively in the ear to be implanted, and controls with normal hearing. Half of the nontraditional CI users had sufficient postoperative acoustic hearing in the implanted ear to use electric-acoustic stimulation, and half used a CI alone.

METHOD

Participants heard recorded sentences that were produced either as yes/no questions or as statements by three male and three female talkers. Three raters scored each participant response as either a question or a statement. Bilateral CI users (n = 40, 4-12 years old) and normal-hearing controls (n = 10, 4-12 years old) were tested binaurally in the free field. Nontraditional CI recipients (n = 22, 6-17 years old) were tested with direct audio input to the study ear.

RESULTS

For the bilateral CI users, performance was predicted by age but not by 125-Hz acoustic thresholds; just under half (n = 17) of the participants in this group had measurable 125-Hz thresholds in their better ear. For nontraditional CI recipients, better performance was predicted by lower 125-Hz acoustic thresholds in the test ear, and there was no association with participant age. Performance approached that of the normal-hearing controls for some participants in each group.

CONCLUSIONS

Results suggest that a 125-Hz acoustic hearing supports discrimination of yes/no questions and statements in pediatric CI users. Bilateral CI users with little or no acoustic hearing at 125 Hz develop the ability to perform this task, but that ability emerges later than for children with better acoustic hearing. These results underscore the importance of preserving acoustic hearing for pediatric CI users when possible.

Integrating Emotion Perception in Rehabilitation Programs for Cochlear Implant Users: A Call for a More Comprehensive Approach

PURPOSE

Postoperative rehabilitation programs for cochlear implant (CI) recipients primarily emphasize enhancing speech perception. However, effective communication in everyday social interactions necessitates consideration of diverse verbal social cues to facilitate language comprehension. Failure to discern emotional expressions may lead to maladjusted social behavior, underscoring the importance of integrating social cues perception into rehabilitation initiatives to enhance CI users' well-being. After conventional rehabilitation, CI users demonstrate varying levels of emotion perception abilities. This disparity notably impacts young CI users, whose emotion perception deficit can extend to social functioning, encompassing coping strategies and social competence, even when relying on nonauditory cues such as facial expressions. Knowing that emotion perception abilities generally decrease with age, acknowledging emotion perception impairments in aging CI users is crucial, especially since a direct correlation between quality-of-life scores and vocal emotion recognition abilities has been observed in adult CI users. After briefly reviewing the scope of CI rehabilitation programs and summarizing the mounting evidence on CI users' emotion perception deficits and their impact, we will present our recommendations for embedding emotional training as part of enriched and standardized evaluation/rehabilitation programs that can improve CI users' social integration and quality of life.

CONCLUSIONS

Evaluating all aspects, including emotion perception, in CI rehabilitation programs is crucial because it ensures a comprehensive approach that enhances speech comprehension and the emotional dimension of communication, potentially improving CI users' social interaction and overall well-being. The development of emotion perception training holds promises for CI users and individuals grappling with various forms of hearing loss and sensory deficits. Ultimately, adopting such a comprehensive approach has the potential to significantly elevate the overall quality of life for a broad spectrum of patients.

Comparison of Speech Perception Performance According to Prosody Change Between People With Normal Hearing and Cochlear Implant Users

BACKGROUND AND OBJECTIVES

Cochlear implants (CIs) are well known to improve audibility and speech recognition in individuals with hearing loss, but some individuals still struggle with many aspects in communication, such as prosody. This study explores how prosodic elements are perceived by those with normal hearing (NH) and CIs.

SUBJECTS AND METHODS

Thirteen individuals with NH and thirteen CI users participated in this study and completed speech perception, speech prosody perception, speech prosody production, pitch difference discrimination, and melodic contour perception testing.

RESULTS

NH listeners performed significantly better than CI users on speech perception, speech prosody perception (except for words with neutral meaning and a negative prosody change and when words were repeated twice), pitch difference discrimination, and melodic contour perception testing. No statistical significance was observed for speech prosody production for both groups.

CONCLUSIONS

Compared to NH listeners, CI users had limited ability to recognize prosodic elements. The study findings highlight the necessity of an assessment tool and signal processing algorithm for CIs, specifically targeting prosodic elements in clinical settings.

Executive functions in preschool and school-age cochlear implant users: do they differ from their hearing peers? A systematic review and meta-analysis

PURPOSE

Executive functions (EF) play a fundamental role in planning and executing goal-driven behaviours. The purpose of this systematic review and meta-analysis was to investigate EF skills mastered by preschool/school-age cochlear implanted children (CIC) without morpho-functional abnormalities and to compare their outcomes with typically hearing children (THC).

METHODS

Bibliographic search for observational studies of any language/date up to 16 December 2022 was performed with the following electronic databases: PubMed, Scopus, and Web of Science. After removal of duplicates, 2442 records were subjected to a three-stage screening process and 83 potentially eligible articles were identified. A total of 15 studies was included in the final analysis: 9 articles directly meeting the eligibility criteria plus 6 more studies thanks to the authors sharing their data set, specifically for participants who met present inclusion criteria.

RESULTS

Meta-analysis showed a statistically significant difference only for verbal short-term memory, whereas group differences for visuospatial short-term memory and verbal/visuospatial working memory were not significant. For fluency skills, meta-analysis revealed statistical significance for the semantic fluency task but not for the rapid naming test. Qualitative analysis reflected group similarities in flexibility but CIC's difficulties in auditory attention/planning skills. Controversial findings for inhibitory control skills were observed.

CONCLUSIONS

EF performance comparisons between CIC and THC show inter-skill and inter-test variances. Due to the paucity of existing studies, present findings should be interpreted with caution. Future research in this domain is strongly recommended.

Communicating Emotion: Vocal Expression of Linguistic and Emotional Prosody in Children With Mild to Profound Hearing Loss Compared With That of Normal Hearing Peers

OBJECTIVES

Emotional prosody is known to play an important role in social communication. Research has shown that children with cochlear implants (CCIs) may face challenges in their ability to express prosody, as their expressions may have less distinct acoustic contrasts and therefore may be judged less accurately. The prosody of children with milder degrees of hearing loss, wearing hearing aids, has sparsely been investigated. More understanding of the prosodic expression by children with hearing loss, hearing aid users in particular, could create more awareness among healthcare professionals and parents on limitations in social communication, which awareness may lead to more targeted rehabilitation. This study aimed to compare the prosodic expression potential of children wearing hearing aids (CHA) with that of CCIs and children with normal hearing (CNH).

DESIGN

In this prospective experimental study, utterances of pediatric hearing aid users, cochlear implant users, and CNH containing emotional expressions (happy, sad, and angry) were recorded during a reading task. Of the utterances, three acoustic properties were calculated: fundamental frequency (F0), variance in fundamental frequency (SD of F0), and intensity. Acoustic properties of the utterances were compared within subjects and between groups.

RESULTS

A total of 75 children were included (CHA: 26, CCI: 23, and CNH: 26). Participants were between 7 and 13 years of age. The 15 CCI with congenital hearing loss had received the cochlear implant at median age of 8 months. The acoustic patterns of emotions uttered by CHA were similar to those of CCI and CNH. Only in CCI, we found no difference in F0 variation between happiness and anger, although an intensity difference was present. In addition, CCI and CHA produced poorer happy-sad contrasts than did CNH.

CONCLUSIONS

The findings of this study suggest that on a fundamental, acoustic level, both CHA and CCI have a prosodic expression potential that is almost on par with normal hearing peers. However, there were some minor limitations observed in the prosodic expression of these children, it is important to determine whether these differences are perceptible to listeners and could affect social communication. This study sets the groundwork for more research that will help us fully understand the implications of these findings and how they may affect the communication abilities of these children. With a clearer understanding of these factors, we can develop effective ways to help improve their communication skills.

Evaluation of musical emotion perception and language development in children with cochlear implants

OBJECTIVES

While the primary purpose of cochlear implant (CI) fitting is to improve individuals' receptive and expressive skills, musical emotion perception (MEP) is generally ignored. This study assesses the MEP and language skills (LS) of children using CI.

METHODS

26 CI users and 26 matched healthy controls between the ages of 6 and 9 were included in the study. The Test of Language Development (TOLD) was applied to evaluate the LS of the participants, and the Montreal Emotion Identification Test (MEI) was applied to evaluate the MEP.

RESULTS

MEI test scores and all subtests of TOLD were statistically significantly lower in the CI group. Also, there was a statistically significant and moderate correlation between the listening subtest of TOLD and the MEI test.

CONCLUSIONS

MEP and language skills are poor in children with CI. Although language skills are primarily targeted in CI performance, improving MEP should also be included in rehabilitation programs. The relationship between music and the TOLD's listening subtest may provide evidence that listening skills can be improved by paying attention to the MEP, which is frequently ignored in rehabilitation programs.

Emotional prosody perception and production are linked in prelingually deaf children with cochlear implantsa)

Links between perception and production of emotional prosody by children with cochlear implants (CIs) have not been extensively explored. In this study, production and perception of emotional prosody were measured in 20 prelingually deaf school-age children with CIs. All were implanted by the age of 3, and most by 18 months. Emotion identification was well-predicted by prosody productions in terms of voice pitch modulation and duration. This finding supports the idea that in prelingually deaf children with CIs, production of emotional prosody is associated with access to auditory cues that support the perception of emotional prosody.

Vocal acoustic features may be objective biomarkers of negative symptoms in schizophrenia: A cross-sectional study

BACKGROUND

There are currently no objective biomarkers that allow the quantification of negative symptoms of schizophrenia. This study therefore explored the use of acoustic features in identifying the severity of negative symptoms in patients with schizophrenia.

METHODS

We recruited 79 inpatients who were diagnosed with schizophrenia according to the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (the schizophrenia group) at the Huilongguan Hospital in Beijing, China, and 79 healthy controls from the surrounding community (the control group). We assessed the clinical symptoms of the patients with schizophrenia using the Positive and Negative Syndrome Scale (PANSS) and the Brief Negative Symptom Scale (BNSS) and recorded the voice of each participant as they read emotionally positive, neutral, and negative texts. The Praat software was used to analyse and extract acoustic characteristics from the recordings, such as jitter, shimmer, and pitch. The acoustic differences between the two groups of participants and the relationship between acoustic characteristics and clinical symptoms in the patient group were analysed.

RESULTS

There were significant differences between the schizophrenia and control groups in pitch, voice breaks, jitter, shimmer, and the mean harmonics-to-noise ratio (p < 0.05). Jitter was negatively correlated with the blunted affect and alogia subscale scores of the BNSS, both in the positive and neutral emotion conditions, but the correlation disappeared in the negative emotion condition. However, shimmer exhibited a stable negative correlation with the blunted affect and alogia subscale scores of the BNSS in all three emotion conditions. A linear regression analysis showed that pitch, jitter, shimmer, and age were statistically significant predictors of BNSS subscale scores.

CONCLUSIONS

Acoustic emotional expression differs between patients with schizophrenia and healthy controls. Some acoustic characteristics are related to the severity of negative symptoms, regardless of semantic emotions, and may therefore be objective biomarkers of negative symptoms. A systematic method for assessing vocal acoustic characteristics could provide an accurate and feasible means of assessing negative symptoms in schizophrenia.

TWEET

Acoustic emotional expression differs between patients with schizophrenia and healthy controls. A systematic method for assessing vocal acoustics could provide an accurate and feasible means of assessing negative symptoms in schizophrenia.

Prosodic Development During the Early School-Age Years

PURPOSE

This study used a cross-sequential design to identify developmental changes in narrative speech rhythm and intonation. The aim was to provide a robust, clinically relevant characterization of normative changes in speech prosody across the early school-age years.

METHOD

Structured spontaneous narratives were elicited annually from 60 children over a 3-year period. Children were aged 5-7 years at study outset and then were aged 7-9 years at study offset. Articulation rate, prominence spacing, and intonational phrase length and duration were calculated for each narrative to index speech rhythm; measures of pitch variability and pitch range indexed intonation. Linear mixed-effects (LME) models tested for cohort-based and within-subject longitudinal change on the prosodic measures; linear regression was used to test for the simple effect of age-in-months within year on the measures.

RESULTS

The LME analyses indicated systematic longitudinal changes in speech rhythm across all measures except phrase duration; there were no longitudinal changes in pitch variability or pitch range across the school-age years. Linear regression results showed an increase in articulation rate with age; there were no systematic differences between age cohorts across years in the study.

CONCLUSIONS

The results indicate that speech rhythm continues to develop during the school-age years. The results also underscore the very strong relationship between the rate and rhythm characteristics of speech and so suggest an important influence of speech motor skills on rhythm production. Finally, the results on pitch variability and pitch range are interpreted to suggest that these are inadequate measures of typical intonation development during the school-age years.

Strategic perceptual weighting of acoustic cues for word stress in listeners with cochlear implants, acoustic hearing, or simulated bimodal hearing

Perception of word stress is an important aspect of recognizing speech, guiding the listener toward candidate words based on the perceived stress pattern. Cochlear implant (CI) signal processing is likely to disrupt some of the available cues for word stress, particularly vowel quality and pitch contour changes. In this study, we used a cue weighting paradigm to investigate differences in stress cue weighting patterns between participants listening with CIs and those with normal hearing (NH). We found that participants with CIs gave less weight to frequency-based pitch and vowel quality cues than NH listeners but compensated by upweighting vowel duration and intensity cues. Nonetheless, CI listeners' stress judgments were also significantly influenced by vowel quality and pitch, and they modulated their usage of these cues depending on the specific word pair in a manner similar to NH participants. In a series of separate online experiments with NH listeners, we simulated aspects of bimodal hearing by combining low-pass filtered speech with a vocoded signal. In these conditions, participants upweighted pitch and vowel quality cues relative to a fully vocoded control condition, suggesting that bimodal listening holds promise for restoring the stress cue weighting patterns exhibited by listeners with NH.

Parameter-Specific Morphing Reveals Contributions of Timbre to the Perception of Vocal Emotions in Cochlear Implant Users

Objectives:

Research on cochlear implants (CIs) has focused on speech comprehension, with little research on perception of vocal emotions. We compared emotion perception in CI users and normal-hearing (NH) individuals, using parameter-specific voice morphing.

Design:

Twenty-five CI users and 25 NH individuals (matched for age and gender) performed fearful-angry discriminations on bisyllabic pseudoword stimuli from morph continua across all acoustic parameters (Full), or across selected parameters (F0, Timbre, or Time information), with other parameters set to a noninformative intermediate level.

Results:

Unsurprisingly, CI users as a group showed lower performance in vocal emotion perception overall. Importantly, while NH individuals used timbre and fundamental frequency (F0) information to equivalent degrees, CI users were far more efficient in using timbre (compared to F0) information for this task. Thus, under the conditions of this task, CIs were inefficient in conveying emotion based on F0 alone. There was enormous variability between CI users, with low performers responding close to guessing level. Echoing previous research, we found that better vocal emotion perception was associated with better quality of life ratings.

Conclusions:

Some CI users can utilize timbre cues remarkably well when perceiving vocal emotions.

Voice emotion recognition by Mandarin-speaking pediatric cochlear implant users in Taiwan

OBJECTIVES

To explore the effects of obligatory lexical tone learning on speech emotion recognition and the cross-culture differences between United States and Taiwan for speech emotion understanding in children with cochlear implant.

METHODS

This cohort study enrolled 60 cochlear-implanted (cCI) Mandarin-speaking, school-aged children who underwent cochlear implantation before 5 years of age and 53 normal-hearing children (cNH) in Taiwan. The emotion recognition and the sensitivity of fundamental frequency (F0) changes for those school-aged cNH and cCI (6-17 years old) were examined in a tertiary referred center.

RESULTS

The mean emotion recognition score of the cNH group was significantly better than the cCI. Female speakers' vocal emotions are more easily to be recognized than male speakers' emotion. There was a significant effect of age at test on voice recognition performance. The average score of cCI with full-spectrum speech was close to the average score of cNH with eight-channel narrowband vocoder speech. The average performance of voice emotion recognition across speakers for cCI could be predicted by their sensitivity to changes in F0.

CONCLUSIONS

Better pitch discrimination ability comes with better voice emotion recognition for Mandarin-speaking cCI. Besides the F0 cues, cCI are likely to adapt their voice emotion recognition by relying more on secondary cues such as intensity and duration. Although cross-culture differences exist for the acoustic features of voice emotion, Mandarin-speaking cCI and their English-speaking cCI peer expressed a positive effect for age at test on emotion recognition, suggesting the learning effect and brain plasticity. Therefore, further device/processor development to improve presentation of pitch information and more rehabilitative efforts are needed to improve the transmission and perception of voice emotion in Mandarin.

LEVEL OF EVIDENCE

3.

Irony Comprehension in Children With Cochlear Implants: The Role of Language Competence, Theory of Mind, and Prosody Recognition

Purpose Aims of this research were (a) to investigate higher order linguistic and cognitive skills of Italian children with cochlear implants (CIs); (b) to correlate them with the comprehension of irony, which has never been systematically studied in this population; and (c) to identify the factors that facilitate the development of this competence. Method We tested 28 Italian children with CI (mean chronological age = 101 [SD = 25.60] months, age range: 60-144 months), and two control groups of normal-hearing (NH) peers matched for chronological age and for hearing age, on a series of tests assessing their cognitive abilities (nonverbal intelligence and theory of mind), linguistic skills (morphosyntax and prosody recognition), and irony comprehension. Results Despite having grammatical abilities in line with the group of NH children matched for hearing age, children with CI lag behind both groups of NH peers on the recognition of emotions through prosody and on the comprehension of ironic stories, even if these two abilities were not related. Conclusions This is the first study that targeted irony comprehension in children with CI, and we found that this competence, which is crucial for maintaining good social relationships with peers, is impaired in this population. In line with other studies, we found a correlation between this ability and advanced theory of mind skills, but at the same time, a deeper investigation is needed, to account for the high variability of performance in children with CI.

Perception of Child-Directed Versus Adult-Directed Emotional Speech in Pediatric Cochlear Implant Users

OBJECTIVES

Cochlear implants (CIs) are remarkable in allowing individuals with severe to profound hearing loss to perceive speech. Despite these gains in speech understanding, however, CI users often struggle to perceive elements such as vocal emotion and prosody, as CIs are unable to transmit the spectro-temporal detail needed to decode affective cues. This issue becomes particularly important for children with CIs, but little is known about their emotional development. In a previous study, pediatric CI users showed deficits in voice emotion recognition with child-directed stimuli featuring exaggerated prosody. However, the large intersubject variability and differential developmental trajectory known in this population incited us to question the extent to which exaggerated prosody would facilitate performance in this task. Thus, the authors revisited the question with both adult-directed and child-directed stimuli.

DESIGN

Vocal emotion recognition was measured using both child-directed (CDS) and adult-directed (ADS) speech conditions. Pediatric CI users, aged 7-19 years old, with no cognitive or visual impairments and who communicated through oral communication with English as the primary language participated in the experiment (n = 27). Stimuli comprised 12 sentences selected from the HINT database. The sentences were spoken by male and female talkers in a CDS or ADS manner, in each of the five target emotions (happy, sad, neutral, scared, and angry). The chosen sentences were semantically emotion-neutral. Percent correct emotion recognition scores were analyzed for each participant in each condition (CDS vs. ADS). Children also completed cognitive tests of nonverbal IQ and receptive vocabulary, while parents completed questionnaires of CI and hearing history. It was predicted that the reduced prosodic variations found in the ADS condition would result in lower vocal emotion recognition scores compared with the CDS condition. Moreover, it was hypothesized that cognitive factors, perceptual sensitivity to complex pitch changes, and elements of each child's hearing history may serve as predictors of performance on vocal emotion recognition.

RESULTS

Consistent with our hypothesis, pediatric CI users scored higher on CDS compared with ADS speech stimuli, suggesting that speaking with an exaggerated prosody-akin to "motherese"-may be a viable way to convey emotional content. Significant talker effects were also observed in that higher scores were found for the female talker for both conditions. Multiple regression analysis showed that nonverbal IQ was a significant predictor of CDS emotion recognition scores while Years using CI was a significant predictor of ADS scores. Confusion matrix analyses revealed a dependence of results on specific emotions; for the CDS condition's female talker, participants had high sensitivity (d' scores) to happy and low sensitivity to the neutral sentences while for the ADS condition, low sensitivity was found for the scared sentences.

CONCLUSIONS

In general, participants had higher vocal emotion recognition to the CDS condition which also had more variability in pitch and intensity and thus more exaggerated prosody, in comparison to the ADS condition. Results suggest that pediatric CI users struggle with vocal emotion perception in general, particularly to adult-directed speech. The authors believe these results have broad implications for understanding how CI users perceive emotions both from an auditory communication standpoint and a socio-developmental perspective.

Meta-Analysis on the Identification of Linguistic and Emotional Prosody in Cochlear Implant Users and Vocoder Simulations

OBJECTIVES

This study quantitatively assesses how cochlear implants (CIs) and vocoder simulations of CIs influence the identification of linguistic and emotional prosody in nontonal languages. By means of meta-analysis, it was explored how accurately CI users and normal-hearing (NH) listeners of vocoder simulations (henceforth: simulation listeners) identify prosody compared with NH listeners of unprocessed speech (henceforth: NH listeners), whether this effect of electric hearing differs between CI users and simulation listeners, and whether the effect of electric hearing is influenced by the type of prosody that listeners identify or by the availability of specific cues in the speech signal.

DESIGN

Records were found by searching the PubMed Central, Web of Science, Scopus, Science Direct, and PsycINFO databases (January 2018) using the search terms "cochlear implant prosody" and "vocoder prosody." Records (published in English) were included that reported results of experimental studies comparing CI users' and/or simulation listeners' identification of linguistic and/or emotional prosody in nontonal languages to that of NH listeners (all ages included). Studies that met the inclusion criteria were subjected to a multilevel random-effects meta-analysis.

RESULTS

Sixty-four studies reported in 28 records were included in the meta-analysis. The analysis indicated that CI users and simulation listeners were less accurate in correctly identifying linguistic and emotional prosody compared with NH listeners, that the identification of emotional prosody was more strongly compromised by the electric hearing speech signal than linguistic prosody was, and that the low quality of transmission of fundamental frequency (f0) through the electric hearing speech signal was the main cause of compromised prosody identification in CI users and simulation listeners. Moreover, results indicated that the accuracy with which CI users and simulation listeners identified linguistic and emotional prosody was comparable, suggesting that vocoder simulations with carefully selected parameters can provide a good estimate of how prosody may be identified by CI users.

CONCLUSIONS

The meta-analysis revealed a robust negative effect of electric hearing, where CIs and vocoder simulations had a similar negative influence on the identification of linguistic and emotional prosody, which seemed mainly due to inadequate transmission of f0 cues through the degraded electric hearing speech signal of CIs and vocoder simulations.

Higher Right Hemisphere Gamma Band Lateralization and Suggestion of a Sensitive Period for Vocal Auditory Emotional Stimuli Recognition in Unilateral Cochlear Implant Children: An EEG Study

In deaf children, huge emphasis was given to language; however, emotional cues decoding and production appear of pivotal importance for communication capabilities. Concerning neurophysiological correlates of emotional processing, the gamma band activity appears a useful tool adopted for emotion classification and related to the conscious elaboration of emotions. Starting from these considerations, the following items have been investigated: (i) whether emotional auditory stimuli processing differs between normal-hearing (NH) children and children using a cochlear implant (CI), given the non-physiological development of the auditory system in the latter group; (ii) whether the age at CI surgery influences emotion recognition capabilities; and (iii) in light of the right hemisphere hypothesis for emotional processing, whether the CI side influences the processing of emotional cues in unilateral CI (UCI) children. To answer these matters, 9 UCI (9.47 ± 2.33 years old) and 10 NH (10.95 ± 2.11 years old) children were asked to recognize nonverbal vocalizations belonging to three emotional states: positive (achievement, amusement, contentment, relief), negative (anger, disgust, fear, sadness), and neutral (neutral, surprise). Results showed better performances in NH than UCI children in emotional states recognition. The UCI group showed increased gamma activity lateralization index (LI) (relative higher right hemisphere activity) in comparison to the NH group in response to emotional auditory cues. Moreover, LI gamma values were negatively correlated with the percentage of correct responses in emotion recognition. Such observations could be explained by a deficit in UCI children in engaging the left hemisphere for more demanding emotional task, or alternatively by a higher conscious elaboration in UCI than NH children. Additionally, for the UCI group, there was no difference between the CI side and the contralateral side in gamma activity, but a higher gamma activity in the right in comparison to the left hemisphere was found. Therefore, the CI side did not appear to influence the physiologic hemispheric lateralization of emotional processing. Finally, a negative correlation was shown between the age at the CI surgery and the percentage of correct responses in emotion recognition and then suggesting the occurrence of a sensitive period for CI surgery for best emotion recognition skills development.

Bimodal Benefits Revealed by Categorical Perception of Lexical Tones in Mandarin-Speaking Kindergarteners With a Cochlear Implant and a Contralateral Hearing Aid

Purpose Pitch reception poses challenges for individuals with cochlear implants (CIs), and adding a hearing aid (HA) in the nonimplanted ear is potentially beneficial. The current study used fine-scale synthetic speech stimuli to investigate the bimodal benefit for lexical tone categorization in Mandarin-speaking kindergarteners using a CI and an HA in opposite ears. Method The data were collected from 16 participants who were required to complete two classical tasks for speech categorical perception (CP) with CI + HA device condition and CI alone condition. Linear mixed-effects models were constructed to evaluate the identification and discrimination scores across different device conditions. Results The bimodal kindergarteners showed CP for the continuum varying from Mandarin Tone 1 and Tone 2. Moreover, the additional acoustic information from the contralateral HA contributes to improved lexical tone categorization, with a steeper slope, a higher discrimination score of between-category stimuli pair, and an improved peakedness score (i.e., an increased benefit magnitude for discriminations of between-category over within-category pairs) for the CI + HA condition than the CI alone condition. The bimodal kindergarteners with better residual hearing thresholds at 250 Hz level in the nonimplanted ear could perceive lexical tones more categorically. Conclusion The enhanced CP results with bimodal listening provide clear evidence for the clinical practice to fit a contralateral HA in the nonimplanted ear in kindergarteners with unilateral CIs with direct benefits from the low-frequency acoustic hearing.

Music perception and training for pediatric cochlear implant users

INTRODUCTION

Cochlear implants (CIs) are biomedical devices that restore sound perception for people with severe-to-profound sensorineural hearing loss. Most postlingually deafened CI users are able to achieve excellent speech recognition in quiet environments. However, current CI sound processors remain limited in their ability to deliver fine spectrotemporal information, making it difficult for CI users to perceive complex sounds. Limited access to complex acoustic cues such as music, environmental sounds, lexical tones, and voice emotion may have significant ramifications on quality of life, social development, and community interactions.

AREAS COVERED

The purpose of this review article is to summarize the literature on CIs and music perception, with an emphasis on music training in pediatric CI recipients. The findings have implications on our understanding of noninvasive, accessible methods for improving auditory processing and may help advance our ability to improve sound quality and performance for implantees.

EXPERT OPINION

Music training, particularly in the pediatric population, may be able to continue to enhance auditory processing even after performance plateaus. The effects of these training programs appear generalizable to non-trained musical tasks, speech prosody and, emotion perception. Future studies should employ rigorous control groups involving a non-musical acoustic intervention, standardized auditory stimuli, and the provision of feedback.

Discourse Prosody in Children's Rhyme Speech Produced by Prelingually Deaf Mandarin-Speaking Children With Cochlear Implants

Purpose This study aimed to obtain a comprehensive understanding about how Mandarin-speaking children with cochlear implants (CIs) performed speech prosody in a connected discourse and to what extent their prosodic scenario differed from those normal-hearing (NH) peers. Method Fifteen prelingually deaf Mandarin-speaking children with unilateral multichannel CIs were chosen and 15 age-matched NH controls were recruited. Speech samples were spontaneously elicited by children's rhyme speech genre and subject to phonetic annotation. Acoustic analysis was conducted on all speech samples, mainly focusing on the measurements of duration and fundamental frequency (F0). Tempo measures included temporal fluency, syllable-lengthening, and rhythm metrics, whereas melodic measures included both local and global F0 variations under different prosodic domains. Results The CI children generally achieved compatible temporal performance with the NH children in spontaneous discourse, except that they were somewhat arbitrary when operationalizing lengthening strategy and pausing strategy at different prosodic boundaries. With regard to melodic performance, CI children may not sufficiently modulate local phonetic nuances of F0 variation, and meanwhile, they performed atypically in the global F0 declination pattern and overall F0 resetting pattern, failing to signal the specific structure of children's rhyme discourse. Early age at implantation and longer CI experience did not play a significant role in the temporal performance of the CI children but did facilitate their articulation of dynamic pitch variation in the spontaneous discourse to some extent. Conclusion CI children did exhibit atypical prosodic patterns in discourse context, especially the overall mapping between the prosodic manifestation and the discourse structure.

Prosodic control in imitative speech of preschool-age children with cochlear implants

This study aimed to (a) to investigate the acoustic characteristics of speech in preschool-age children with cochlear implants (CIs) compared to those in age-matched children with normal hearing (NH), (b) to identify which acoustic measures could differentiate children with CIs from children with NH, (c) to explore the relationships between child variables and the perceptually judged ability to control prosody in the CI group, and (d) to examine predictors of perceptually judged ability in children with CIs to control prosody. Study participants included 30 children with CIs (3-5.9 years old) and 30 age-matched children with NH. Children were asked to imitate 20 utterances, which were syntactically matched 10 statements and 10 questions. Fundamental frequency, intensity, and duration measures were obtained from the final word of each utterance. Ten adults rated the prosodic-contour appropriateness of the children with CIs using a 5-point scale. Children with CIs tended to produce less distinctive declarative and interrogative utterances compared to children with NH. The mean F0 significantly differentiated children with CIs from their NH peers. In children with CIs, perceptual ratings of prosodic-contour appropriateness were significantly correlated with age, duration of implant use, and percentage of consonants corrects. Duration of implant use was a significant factor predicting the ability to control prosody in children with CIs. These findings suggested that preschool-age children with CIs have difficulty in controlling pitch parameters, compared with the NH peers. The prosodic development in children with CIs was affected by the hearing experience via CIs.

Development of vocal emotion recognition in school-age children: The EmoHI test for hearing-impaired populations

Traditionally, emotion recognition research has primarily used pictures and videos, while audio test materials are not always readily available or are not of good quality, which may be particularly important for studies with hearing-impaired listeners. Here we present a vocal emotion recognition test with pseudospeech productions from multiple speakers expressing three core emotions (happy, angry, and sad): the EmoHI test. The high sound quality recordings make the test suitable for use with populations of children and adults with normal or impaired hearing. Here we present normative data for vocal emotion recognition development in normal-hearing (NH) school-age children using the EmoHI test. Furthermore, we investigated cross-language effects by testing NH Dutch and English children, and the suitability of the EmoHI test for hearing-impaired populations, specifically for prelingually deaf Dutch children with cochlear implants (CIs). Our results show that NH children's performance improved significantly with age from the youngest age group onwards (4-6 years: 48.9%, on average). However, NH children's performance did not reach adult-like values (adults: 94.1%) even for the oldest age group tested (10-12 years: 81.1%). Additionally, the effect of age on NH children's development did not differ across languages. All except one CI child performed at or above chance-level showing the suitability of the EmoHI test. In addition, seven out of 14 CI children performed within the NH age-appropriate range, and nine out of 14 CI children did so when performance was adjusted for hearing age, measured from their age at CI implantation. However, CI children showed great variability in their performance, ranging from ceiling (97.2%) to below chance-level performance (27.8%), which could not be explained by chronological age alone. The strong and consistent development in performance with age, the lack of significant differences across the tested languages for NH children, and the above-chance performance of most CI children affirm the usability and versatility of the EmoHI test.

Neurophysiological Differences in Emotional Processing by Cochlear Implant Users, Extending Beyond the Realm of Speech

OBJECTIVE

Cochlear implants (CIs) restore a sense of hearing in deaf individuals. However, they do not transmit the acoustic signal with sufficient fidelity, leading to difficulties in recognizing emotions in voice and in music. The study aimed to explore the neurophysiological bases of these limitations.

DESIGN

Twenty-two adults (18 to 70 years old) with CIs and 22 age-matched controls with normal hearing participated. Event-related potentials (ERPs) were recorded in response to emotional bursts (happy, sad, or neutral) produced in each modality (voice or music) that were for the most part correctly identified behaviorally.

RESULTS

Compared to controls, the N1 and P2 components were attenuated and prolonged in CI users. To a smaller degree, N1 and P2 were also attenuated and prolonged in music compared to voice, in both populations. The N1-P2 complex was emotion-dependent (e.g., reduced and prolonged response to sadness), but this was also true in both populations. In contrast, the later portion of the response, between 600 and 850 ms, differentiated happy and sad from neutral stimuli in normal hearing but not in CI listeners.

CONCLUSIONS

The early portion of the ERP waveform reflected primarily the general reduction in sensory encoding by CI users (largely due to CI processing itself), whereas altered emotional processing (by CI users) could be found in the later portion of the ERP and extended beyond the realm of speech.

Self-reported hearing quality of life measures in pediatric cochlear implant recipients with bilateral input

Objective: Self-reported hearing quality of life (QoL) for pediatric cochlear implant (CI) recipients was examined, asking whether 1) children with CIs have similar QoL as those with less severe hearing loss (HL); 2) children with different bilateral CI (BCI) device configurations report different QoL; and 3) do audiological, demographic and spoken language factors affect hearing QoL?Design: One hundred four children (ages 7-11 years) using bimodal devices or BCIs participated. The Hearing Environments and Reflection of Quality of Life (HEAR-QL) questionnaire, receptive language and speech perception tests were administered. HEAR-QL scores of CI recipients were compared to scores of age-mates with normal hearing and mild to profound HL.Results: HEAR-QL scores for CI participants were similar to those of children with less severe HL and did not differ with device configuration. Emotion identification and word recognition in noise correlated significantly with HEAR-QL scores.Discussion: CI recipients reported that HL hinders social participation. Better understanding of speech in noise and emotional content was associated with fewer hearing-related difficulties on the HEAR-QL.Conclusions: Noisy situations encountered in educational settings should be addressed for children with HL. The link between perception of emotion and hearing-related QoL for CI recipients should be further examined.

The CI MuMuFe - A New MMN Paradigm for Measuring Music Discrimination in Electric Hearing

Cochlear implants (CIs) allow good perception of speech while music listening is unsatisfactory, leading to reduced music enjoyment. Hence, a number of ongoing efforts aim to improve music perception with a CI. Regardless of the nature of these efforts, effect measurements must be valid and reliable. While auditory skills are typically examined by behavioral methods, recording of the mismatch negativity (MMN) response, using electroencephalography (EEG), has recently been applied successfully as a supplementary objective measure. Eleven adult CI users and 14 normally hearing (NH) controls took part in the present study. To measure their detailed discrimination of fundamental features of music we applied a new multifeature MMN-paradigm which presented four music deviants at four levels of magnitude, incorporating a novel "no-standard" approach to be tested with CI users for the first time. A supplementary test measured behavioral discrimination of the same deviants and levels. The MMN-paradigm elicited significant MMN responses to all levels of deviants in both groups. Furthermore, the CI-users' MMN amplitudes and latencies were not significantly different from those of NH controls. Both groups showed MMN strength that was in overall alignment with the deviation magnitude. In CI users, however, discrimination of pitch levels remained undifferentiated. On average, CI users' behavioral performance was significantly below that of the NH group, mainly due to poor pitch discrimination. Although no significant effects were found, CI users' behavioral results tended to be in accordance with deviation magnitude, most prominently manifested in discrimination of the rhythm deviant. In summary, the study indicates that CI users may be able to discriminate subtle changes in basic musical features both in terms of automatic neural responses and of attended behavioral detection. Despite high complexity, the new CI MuMuFe paradigm and the "no-standard" approach provided reliable results, suggesting that it may serve as a relevant tool in future CI research. For clinical use, future studies should investigate the possibility of applying the paradigm with the purpose of assessing discrimination skills not only at the group level but also at the individual level.

How Vocal Emotions Produced by Children With Cochlear Implants Are Perceived by Their Hearing Peers

Purpose Cochlear implants (CIs) transmit a degraded version of the acoustic input to the listener. This impacts the perception of harmonic pitch, resulting in deficits in the perception of voice features critical to speech prosody. Such deficits may relate to changes in how children with CIs (CCIs) learn to produce vocal emotions. The purpose of this study was to investigate happy and sad emotional speech productions by school-age CCIs, compared to productions by children with normal hearing (NH), postlingually deaf adults with CIs, and adults with NH. Method All individuals recorded the same emotion-neutral sentences in a happy manner and a sad manner. These recordings were then used as stimuli in an emotion recognition task performed by child and adult listeners with NH. Their performance was taken as a measure of how well the 4 groups of talkers communicated the 2 emotions. Results Results showed high variability in the identifiability of emotions produced by CCIs, relative to other groups. Some CCIs produced highly identifiable emotions, while others showed deficits. The postlingually deaf adults with CIs produced highly identifiable emotions and relatively small intersubject variability. Age at implantation was found to be a significant predictor of performance by CCIs. In addition, the NH listeners' age predicted how well they could identify the emotions produced by CCIs. Thus, older NH child listeners were better able to identify the CCIs' intended emotions than younger NH child listeners. In contrast to the deficits in their emotion productions, CCIs produced highly intelligible words in the sentences carrying the emotions. Conclusions These results confirm previous findings showing deficits in CCIs' productions of prosodic cues and indicate that early auditory experience plays an important role in vocal emotion productions by individuals with CIs.

Acoustics of Emotional Prosody Produced by Prelingually Deaf Children With Cochlear Implants

Purpose: Cochlear implants (CIs) provide reasonable levels of speech recognition quietly, but voice pitch perception is severely impaired in CI users. The central question addressed here relates to how access to acoustic input pre-implantation influences vocal emotion production by individuals with CIs. The objective of this study was to compare acoustic characteristics of vocal emotions produced by prelingually deaf school-aged children with cochlear implants (CCIs) who were implanted at the age of 2 and had no usable hearing before implantation with those produced by children with normal hearing (CNH), adults with normal hearing (ANH), and postlingually deaf adults with cochlear implants (ACI) who developed with good access to acoustic information prior to losing their hearing and receiving a CI. Method: A set of 20 sentences without lexically based emotional information was recorded by 13 CCI, 9 CNH, 9 ANH, and 10 ACI, each with a happy emotion and a sad emotion, without training or guidance. The sentences were analyzed for primary acoustic characteristics of the productions. Results: Significant effects of Emotion were observed in all acoustic features analyzed (mean voice pitch, standard deviation of voice pitch, intensity, duration, and spectral centroid). ACI and ANH did not differ in any of the analyses. Of the four groups, CCI produced the smallest acoustic contrasts between the emotions in voice pitch and emotions in its standard deviation. Effects of developmental age (highly correlated with the duration of device experience) and age at implantation (moderately correlated with duration of device experience) were observed, and interactions with the children's sex were also observed. Conclusion: Although prelingually deaf CCI and postlingually deaf ACI are listening to similar degraded speech and show similar deficits in vocal emotion perception, these groups are distinct in their productions of contrastive vocal emotions. The results underscore the importance of access to acoustic hearing in early childhood for the production of speech prosody and also suggest the need for a greater role of speech therapy in this area.

Recognition of emotional prosody by Mandarin-speaking adults with cochlear implants

To understand how cochlear implant processing affects emotional prosody recognition in tonal languages, how normal-hearing (NH) and cochlear-implanted (CI) adults identify four emotions ("angry," "happy," "sad," and "neutral") in short, semantically neutral, Mandarin sentences are compared. Depending on hearing status (CI, NH), adults heard natural speech and/or noise-vocoded speech conditions (4-, 8-, and 16-spectral channels). Results suggest that Mandarin-speaking adults with CIs recognize emotions with similar accuracy as NH listeners attending to spectrally degraded (4-channel) vocoded speech. The accuracy noted for Mandarin appears to be lower than that described in previous studies of English.

Emotional Understanding in Children with A Cochlear Implant

Emotional understanding plays an important role in the physical and mental health of children. To determine whether the development of emotional understanding is delayed in children with a cochlear implant (CI), 30 children with a CI and 30 matched children with typical hearing aged between 3 and 9 years old completed three tasks (facial expression, tone expression and scene of emotion matching), in which they identified four basic emotions (happiness, sadness, anger, and fear). The participants included 40 preschool children and 20 school-aged children. Compared with the children with typical hearing, the children with a CI were significantly delayed in the ability to understand emotion; the tone expression matching task was the most difficult task for all participants; and the school-aged children performed better than the preschool children, but this trend was only observed in the typical hearing group; and happiness was the easiest emotion to identify.

A tonal-language benefit for pitch in normally-hearing and cochlear-implanted children

In tonal languages, voice pitch inflections change the meaning of words, such that the brain processes pitch not merely as an acoustic characterization of sound but as semantic information. In normally-hearing (NH) adults, this linguistic pressure on pitch appears to sharpen its neural encoding and can lead to perceptual benefits, depending on the task relevance, potentially generalizing outside of the speech domain. In children, however, linguistic systems are still malleable, meaning that their encoding of voice pitch information might not receive as much neural specialization but might generalize more easily to ecologically irrelevant pitch contours. This would seem particularly true for early-deafened children wearing a cochlear implant (CI), who must exhibit great adaptability to unfamiliar sounds as their sense of pitch is severely degraded. Here, we provide the first demonstration of a tonal language benefit in dynamic pitch sensitivity among NH children (using both a sweep discrimination and labelling task) which extends partially to children with CI (i.e., in the labelling task only). Strong age effects suggest that sensitivity to pitch contours reaches adult-like levels early in tonal language speakers (possibly before 6 years of age) but continues to develop in non-tonal language speakers well into the teenage years. Overall, we conclude that language-dependent neuroplasticity can enhance behavioral sensitivity to dynamic pitch, even in extreme cases of auditory degradation, but it is most easily observable early in life.

Prosody perception and production by children with cochlear implants

The perception and production of emotional and linguistic (focus) prosody were compared in children with cochlear implants (CI) and normally hearing (NH) peers. Thirteen CI and thirteen hearing-age-matched school-aged NH children were tested, as baseline, on non-verbal emotion understanding, non-word repetition, and stimulus identification and naming. Main tests were verbal emotion discrimination, verbal focus position discrimination, acted emotion production, and focus production. Productions were evaluated by NH adult Dutch listeners. All scores between groups were comparable, except a lower score for the CI group for non-word repetition. Emotional prosody perception and production scores correlated weakly for CI children but were uncorrelated for NH children. In general, hearing age weakly predicted emotion production but not perception. Non-verbal emotional (but not linguistic) understanding predicted CI children's (but not controls') emotion perception and production. In conclusion, increasing time in sound might facilitate vocal emotional expression, possibly requiring independently maturing emotion perception skills.

Basic Measures of Prosody in Spontaneous Speech of Children With Early and Late Cochlear Implantation

PURPOSE

Relative to normally hearing (NH) peers, the speech of children with cochlear implants (CIs) has been found to have deviations such as a high fundamental frequency, elevated jitter and shimmer, and inadequate intonation. However, two important dimensions of prosody (temporal and spectral) have not been systematically investigated. Given that, in general, the resolution in CI hearing is best for the temporal dimension and worst for the spectral dimension, we expected this hierarchy to be reflected in the amount of CI speech's deviation from NH speech. Deviations, however, were expected to diminish with increasing device experience.

METHOD

Of 9 Dutch early- and late-implanted (division at 2 years of age) children and 12 hearing age-matched NH controls, spontaneous speech was recorded at 18, 24, and 30 months after implantation (CI) or birth (NH). Six spectral and temporal outcome measures were compared between groups, sessions, and genders.

RESULTS

On most measures, interactions of Group and/or Gender with Session were significant. For CI recipients as compared with controls, performance on temporal measures was not in general more deviant than spectral measures, although differences were found for individual measures. The late-implanted group had a tendency to be closer to the NH group than the early-implanted group. Groups converged over time.

CONCLUSIONS

Results did not support the phonetic dimension hierarchy hypothesis, suggesting that the appropriateness of the production of basic prosodic measures does not depend on auditory resolution. Rather, it seems to depend on the amount of control necessary for speech production.

Musical and vocal emotion perception for cochlear implants users

Cochlear implants can successfully restore hearing in profoundly deaf individuals and enable speech comprehension. However, the acoustic signal provided is severely degraded and, as a result, many important acoustic cues for perceiving emotion in voices and music are unavailable. The deficit of cochlear implant users in auditory emotion processing has been clearly established. Yet, the extent to which this deficit and the specific cues that remain available to cochlear implant users are unknown due to several confounding factors. Here we assessed the recognition of the most basic forms of auditory emotion and aimed to identify which acoustic cues are most relevant to recognize emotions through cochlear implants. To do so, we used stimuli that allowed vocal and musical auditory emotions to be comparatively assessed while controlling for confounding factors. These stimuli were used to evaluate emotion perception in cochlear implant users (Experiment 1) and to investigate emotion perception in natural versus cochlear implant hearing in the same participants with a validated cochlear implant simulation approach (Experiment 2). Our results showed that vocal and musical fear was not accurately recognized by cochlear implant users. Interestingly, both experiments found that timbral acoustic cues (energy and roughness) correlate with participant ratings for both vocal and musical emotion bursts in the cochlear implant simulation condition. This suggests that specific attention should be given to these cues in the design of cochlear implant processors and rehabilitation protocols (especially energy, and roughness). For instance, music-based interventions focused on timbre could improve emotion perception and regulation, and thus improve social functioning, in children with cochlear implants during development.

Benefits of Music Training for Perception of Emotional Speech Prosody in Deaf Children With Cochlear Implants

Objectives:

Children who use cochlear implants (CIs) have characteristic pitch processing deficits leading to impairments in music perception and in understanding emotional intention in spoken language. Music training for normal-hearing children has previously been shown to benefit perception of emotional prosody. The purpose of the present study was to assess whether deaf children who use CIs obtain similar benefits from music training. We hypothesized that music training would lead to gains in auditory processing and that these gains would transfer to emotional speech prosody perception.

Design:

Study participants were 18 child CI users (ages 6 to 15). Participants received either 6 months of music training (i.e., individualized piano lessons) or 6 months of visual art training (i.e., individualized painting lessons). Measures of music perception and emotional speech prosody perception were obtained pre-, mid-, and post-training. The Montreal Battery for Evaluation of Musical Abilities was used to measure five different aspects of music perception (scale, contour, interval, rhythm, and incidental memory). The emotional speech prosody task required participants to identify the emotional intention of a semantically neutral sentence under audio-only and audiovisual conditions.

Results:

Music training led to improved performance on tasks requiring the discrimination of melodic contour and rhythm, as well as incidental memory for melodies. These improvements were predominantly found from mid- to post-training. Critically, music training also improved emotional speech prosody perception. Music training was most advantageous in audio-only conditions. Art training did not lead to the same improvements.

Conclusions:

Music training can lead to improvements in perception of music and emotional speech prosody, and thus may be an effective supplementary technique for supporting auditory rehabilitation following cochlear implantation.

Neural Processing of Musical and Vocal Emotions Through Cochlear Implants Simulation

Cochlear implants (CIs) partially restore the sense of hearing in the deaf. However, the ability to recognize emotions in speech and music is reduced due to the implant's electrical signal limitations and the patient's altered neural pathways. Electrophysiological correlations of these limitations are not yet well established. Here we aimed to characterize the effect of CIs on auditory emotion processing and, for the first time, directly compare vocal and musical emotion processing through a CI-simulator. We recorded 16 normal hearing participants' electroencephalographic activity while listening to vocal and musical emotional bursts in their original form and in a degraded (CI-simulated) condition. We found prolonged P50 latency and reduced N100-P200 complex amplitude in the CI-simulated condition. This points to a limitation in encoding sound signals processed through CI simulation. When comparing the processing of vocal and musical bursts, we found a delay in latency with the musical bursts compared to the vocal bursts in both conditions (original and CI-simulated). This suggests that despite the cochlear implants' limitations, the auditory cortex can distinguish between vocal and musical stimuli. In addition, it adds to the literature supporting the complexity of musical emotion. Replicating this study with actual CI users might lead to characterizing emotional processing in CI users and could ultimately help develop optimal rehabilitation programs or device processing strategies to improve CI users' quality of life.

Vocal Emotion Identification by Children Using Cochlear Implants, Relations to Voice Quality, and Musical Interests

PURPOSE

Listening tests for emotion identification were conducted with 8-17-year-old children with hearing impairment (HI; N = 25) using cochlear implants, and their 12-year-old peers with normal hearing (N = 18). The study examined the impact of musical interests and acoustics of the stimuli on correct emotion identification.

METHOD

The children completed a questionnaire with their background information and noting musical interests. They then listened to vocal stimuli produced by actors (N = 5) and consisting of nonsense sentences and prolonged vowels ([a:], [i:], and [u:]; N = 32) expressing excitement, anger, contentment, and fear. The children's task was to identify the emotions they heard in the sample by choosing from the provided options. Acoustics of the samples were studied using Praat software, and statistics were examined using SPSS 24 software.

RESULTS

The children with HI identified the emotions with 57% accuracy and the normal hearing children with 75% accuracy. Female listeners were more accurate than male listeners in both groups. Those who were implanted before age of 3 years identified emotions more accurately than others (p < .05). No connection between the child's audiogram and correct identification was observed. Musical interests and voice quality parameters were found to be related to correct identification.

CONCLUSIONS

Implantation age, musical interests, and voice quality tended to have an impact on correct emotion identification. Thus, in developing the cochlear implants, it may be worth paying attention to the acoustic structures of vocal emotional expressions, especially the formant frequency of F3. Supporting the musical interests of children with HI may help their emotional development and improve their social lives.

Using prosody to infer discourse prominence in cochlear-implant users and normal-hearing listeners

Cochlear implants (CIs) provide speech perception to adults with severe-to-profound hearing loss, but the acoustic signal remains severely degraded. Limited access to pitch cues is thought to decrease sensitivity to prosody in CI users, but co-occurring changes in intensity and duration may provide redundant cues. The current study investigates how listeners use these cues to infer discourse prominence. CI users and normal-hearing (NH) listeners were presented with sentences varying in prosody (accented vs. unaccented words) while their eye-movements were measured to referents varying in discourse status (given vs. new categories). In Experiment 1, all listeners inferred prominence when prosody on nouns distinguished categories ("SANDWICH"→not sandals). In Experiment 2, CI users and NH listeners presented with natural speech inferred prominence when prosody on adjectives implied contrast across both categories and properties ("PINK horse"→not the orange horse). In contrast, NH listeners presented with simulated CI (vocoded) speech were sensitive to acoustic differences in prosody, but did not use these cues to infer discourse status. Together, this suggests that exploiting redundant cues for comprehension varies with the demands of language processing and prior experience with the degraded signal.

Children's and adults' perception of questions and statements from terminal fundamental frequency contours

The present study compared children's and adults' identification and discrimination of declarative questions and statements on the basis of terminal cues alone. Children (8-11 years, n = 41) and adults (n = 21) judged utterances as statements or questions from sentences with natural statement and question endings and with manipulated endings that featured intermediate fundamental frequency (F₀) values. The same adults and a different sample of children (n = 22) were also tested on their discrimination of the utterances. Children's judgments shifted more gradually across categories than those of adults, but their category boundaries were comparable. In the discrimination task, adults found cross-boundary comparisons more salient than within-boundary comparisons. Adults' performance on the identification and discrimination tasks is consistent with but not definitive regarding categorical perception of statements and questions. Children, by contrast, discriminated the cross-boundary comparisons no better than other comparisons. The findings indicate age-related sharpening in the perception of statements and questions based on terminal F₀ cues and the gradual emergence of distinct perceptual categories.

Voice emotion perception and production in cochlear implant users

Voice emotion is a fundamental component of human social interaction and social development. Unfortunately, cochlear implant users are often forced to interface with highly degraded prosodic cues as a result of device constraints in extraction, processing, and transmission. As such, individuals with cochlear implants frequently demonstrate significant difficulty in recognizing voice emotions in comparison to their normal hearing counterparts. Cochlear implant-mediated perception and production of voice emotion is an important but relatively understudied area of research. However, a rich understanding of the voice emotion auditory processing offers opportunities to improve upon CI biomedical design and to develop training programs benefiting CI performance. In this review, we will address the issues, current literature, and future directions for improved voice emotion processing in cochlear implant users.

Level of emotion comprehension in children with mid to long term cochlear implant use: How basic and more complex emotion recognition relates to language and age at implantation

OBJECTIVES

The current study was designed with three main aims: To document the level of emotional comprehension skills, from basic to more complex ones, reached by a wide sample of cochlear implant (CI) deaf children with at least 36 months of device use; To investigate subjective and audiological factors that can affect their emotional development; To identify, if present, a "critical age", in which early intervention might positively affect adequate emotional competence development.

DESIGN

This is an observational cohort study. Children with congenital severe/profound deafness were selected based on: aged by 4-11 years, minimum of 36 months of CI use, Italian as the primary language in the family; normal cognitive level and absence of associated disorders or socio-economic difficulties. Audiological characteristics and language development were assessed throughout standardized tests, to measure speech perception in quiet, lexical comprehension and production. The development of emotions' understanding was assessed using the Test of Emotion Comprehension (TEC) of Pons and Harris, a hierarchical developmental model, where emotion comprehension is organized in 3 Stages (external, mental and reflective). Statistical analysis was accomplished via the Spearman Rank Correlation Coefficient, to study the relationship between the personal and audiological characteristics; a multivariate linear regression analysis was carried out to find which variables were better associated with the standardized TEC values; a chi-squared test with Yate's continuity correction and Mann-Whitney U test were used to account for differences between continuous variables and proportions.

RESULTS

72 children (40 females, 32 males) with a mean age of 8.1 years were included. At TEC score, 57 children showed normal range performances (79.17% of recipients) and 15 fell below average (20.83% of recipients). The 16.63% of older subjects (range of age 8-12 years) didn't master the Stage 3 (reflective), which is normally acquired by 8 years of age and failed 2 or all the 3 items of this component. Subjects implanted within 18 months of age had better emotion comprehension skills. TEC results were also positively correlated with an early diagnosis, a longer implant use, better auditory skills and higher scores on lexical and morphosintactic tests. On the contrary, it was negatively correlated with the presence of siblings and the order of birth. The gender, the side and the severity of deafness, type of implant and strategy were not correlated.

CONCLUSIONS

Early implanted children have more chance to develop adequate emotion comprehension, especially when the complex aspects are included, due to the very strong link between listening and language skills and emotional development. Furthermore, longer CI auditory experience along with early intervention allows an adequate communication development which positively influences the acquisition of such competencies.

Deficits in the Sensitivity to Pitch Sweeps by School-Aged Children Wearing Cochlear Implants

Sensitivity to static changes in pitch has been shown to be poorer in school-aged children wearing cochlear implants (CIs) than children with normal hearing (NH), but it is unclear whether this is also the case for dynamic changes in pitch. Yet, dynamically changing pitch has considerable ecological relevance in terms of natural speech, particularly aspects such as intonation, emotion, or lexical tone information. Twenty one children with NH and 23 children wearing a CI participated in this study, along with 18 NH adults and 6 CI adults for comparison. Listeners with CIs used their clinically assigned settings with envelope-based coding strategies. Percent correct was measured in one- or three-interval two-alternative forced choice tasks, for the direction or discrimination of harmonic complexes based on a linearly rising or falling fundamental frequency. Sweep rates were adjusted per subject, in a logarithmic scale, so as to cover the full extent of the psychometric function. Data for up- and down-sweeps were fitted separately, using a maximum-likelihood technique. Fits were similar for up- and down-sweeps in the discrimination task, but diverged in the direction task because psychometric functions for down-sweeps were very shallow. Hits and false alarms were then converted into d′ and beta values, from which a threshold was extracted at a d′ of 0.77. Thresholds were very consistent between the two tasks and considerably higher (worse) for CI listeners than for their NH peers. Thresholds were also higher for children than adults. Factors such as age at implantation, age at profound hearing loss, and duration of CI experience did not play any major role in this sensitivity. Thresholds of dynamic pitch sensitivity (in either task) also correlated with thresholds for static pitch sensitivity and with performance in tasks related to speech prosody.

Affective Properties of Mothers' Speech to Infants With Hearing Impairment and Cochlear Implants

PURPOSE

The affective properties of infant-directed speech influence the attention of infants with normal hearing to speech sounds. This study explored the affective quality of maternal speech to infants with hearing impairment (HI) during the 1st year after cochlear implantation as compared to speech to infants with normal hearing.

METHOD

Mothers of infants with HI and mothers of infants with normal hearing matched by age (NH-AM) or hearing experience (NH-EM) were recorded playing with their infants during 3 sessions over a 12-month period. Speech samples of 25 s were low-pass filtered, leaving intonation but not speech information intact. Sixty adults rated the stimuli along 5 scales: positive/negative affect and intention to express affection, to encourage attention, to comfort/soothe, and to direct behavior.

RESULTS

Low-pass filtered speech to HI and NH-EM groups was rated as more positive, affective, and comforting compared with the such speech to the NH-AM group. Speech to infants with HI and with NH-AM was rated as more directive than speech to the NH-EM group. Mothers decreased affective qualities in speech to all infants but increased directive qualities in speech to infants with NH-EM over time.

CONCLUSIONS

Mothers fine-tune communicative intent in speech to their infant's developmental stage. They adjust affective qualities to infants' hearing experience rather than to chronological age but adjust directive qualities of speech to the chronological age of their infants.