Covariation of cochlear implant users' perception and production of vowel contrasts and their identification by listeners with normal hearing

Discrimination and sensorimotor adaptation of self-produced vowels in cochlear implant users

Humans rely on auditory feedback to monitor and adjust their speech for clarity. Cochlear implants (CIs) have helped over a million people restore access to auditory feedback, which significantly improves speech production. However, there is substantial variability in outcomes. This study investigates the extent to which CI users can use their auditory feedback to detect self-produced sensory errors and make adjustments to their speech, given the coarse spectral resolution provided by their implants. First, we used an auditory discrimination task to assess the sensitivity of CI users to small differences in formant frequencies of their self-produced vowels. Then, CI users produced words with altered auditory feedback in order to assess sensorimotor adaptation to auditory error. Almost half of the CI users tested can detect small, within-channel differences in their self-produced vowels, and they can utilize this auditory feedback towards speech adaptation. An acoustic hearing control group showed better sensitivity to the shifts in vowels, even in CI-simulated speech, and elicited more robust speech adaptation behavior than the CI users. Nevertheless, this study confirms that CI users can compensate for sensory errors in their speech and supports the idea that sensitivity to these errors may relate to variability in production.

The Impact of Bimodal Hearing on Speech Acoustics of Vowel Production in Adult Cochlear Implant Users

PURPOSE

This study aimed to investigate the acoustic changes in vowel production with different forms of auditory feedback via cochlear implant (CI), hearing aid (HA), and bimodal hearing (CI + HA).

METHOD

Ten post-lingually deaf adult bimodal CI users (aged 50-78 years) produced English vowels /i/, /ɛ/, /æ/, /ɑ/, /ʊ/, and /u/ in the context of /hVd/ during short-term use of no device (ND), HA, CI, and CI + HA. Segmental features (first formant frequency [F 1], second formant frequency [F 2], and vowel space area) and suprasegmental features (duration, intensity, and fundamental frequency [f o]) of vowel production were analyzed. Participants also categorized a vowel continuum synthesized from their own productions of /ɛ/ and /æ/ using HA, CI, and CI + HA.

RESULTS

F 1s of all vowels decreased; F 2s of front vowels but not back vowels increased; vowel space areas increased; and vowel durations, intensities, and f os decreased with statistical significance in the HA, CI, and CI + HA conditions relative to the ND condition. Only f os were lower, and vowel space areas were larger with CI and CI + HA than with HA. Average changes in f o, intensity, and F 1 from the ND condition to the HA, CI, and CI + HA conditions were positively correlated. Most participants did not show a typical psychometric function for vowel categorization, and thus, the relationship between vowel categorization and production was not tested.

CONCLUSIONS

The results suggest that acoustic, electric, and bimodal hearing have a measurable impact on vowel acoustics of post-lingually deaf adults when their hearing devices are turned on and off temporarily. Also, changes in f o and F 1 with the use of hearing devices may be largely driven by changes in intensity.

Intelligibility of speech produced by sighted and blind adults

Purpose

It is well known that speech uses both the auditory and visual modalities to convey information. In cases of congenital sensory deprivation, the feedback language learners have access to for mapping visible and invisible orofacial articulation is impoverished. Although the effects of blindness on the movements of the lips, jaw, and tongue have been documented in francophone adults, not much is known about their consequences for speech intelligibility. The objective of this study is to investigate the effects of congenital visual deprivation on vowel intelligibility in adult speakers of Canadian French.

Method

Twenty adult listeners performed two perceptual identification tasks in which vowels produced by congenitally blind adults and sighted adults were used as stimuli. The vowels were presented in the auditory, visual, and audiovisual modalities (experiment 1) and at different signal-to-noise ratios in the audiovisual modality (experiment 2). Correct identification scores were calculated. Sequential information analyses were also conducted to assess the amount of information transmitted to the listeners along the three vowel features of height, place of articulation, and rounding.

Results

The results showed that, although blind speakers did not differ from their sighted peers in the auditory modality, they had lower scores in the audiovisual and visual modalities. Some vowels produced by blind speakers are also less robust in noise than those produced by sighted speakers.

Conclusion

Together, the results suggest that adult blind speakers have learned to adapt to their sensory loss so that they can successfully achieve intelligible vowel targets in non-noisy conditions but that they produce less intelligible speech in noisy conditions. Thus, the trade-off between visible (lips) and invisible (tongue) articulatory cues observed between vowels produced by blind and sighted speakers is not equivalent in terms of perceptual efficiency.

Impact of visual and auditory deprivation on speech perception and production in adults

Speech perception relies on auditory and visual cues and there are strong links between speech perception and production. We aimed to evaluate the role of auditory and visual modalities on speech perception and production in adults with impaired hearing or sight versus those with normal hearing and sight. We examined speech perception and production of three isolated vowels (/i/, /y/, /u/), which were selected based on their different auditory and visual perceptual saliencies, in 12 deaf adults who used one or two cochlear implants (CIs), 14 congenitally blind adults, and 16 adults with normal sight and hearing. The results showed that the deaf adults who used a CI had worse vowel identification and discrimination perception and they also produced vowels that were less typical or precise than other participants. They had different tongue positions in speech production, which possibly partly explains the poorer quality of their spoken vowels. Blind individuals had larger lip openings and smaller lip protrusions for the rounded vowel and unrounded vowels, compared to the other participants, but they still produced vowels that were similar to those produced by the adults with normal sight and hearing. In summary, the deaf adults, even though they used CIs, had greater difficulty in producing accurate vowel targets than the blind adults, whereas the blind adults were still able to produce accurate vowel targets, even though they used different articulatory strategies.

Vowel intelligibility in children with cochlear implants: An acoustic and articulatory study

In child cochlear implant (CI) users, early implantation generally results in highly intelligible speech. However, for some children developing a high level of speech intelligibility may be problematic. Studies of speech production in CI users have principally been based on perceptual judgment and acoustic measures. Articulatory measures, such as those collected using ultrasound provide the opportunity to more precisely evaluate what makes child CI users more intelligible. This study investigates speech production and intelligibility in children with CI using acoustic and articulatory measures. Ten children with unilateral or bilateral CIs and 13 children with normal hearing (NH) participated in the study. Participants repeated five English vowels (/a/, /e/, /i/, /o/, /u/) with and without auditory feedback. Ultrasound was used to capture tongue positions and acoustic signals were recorded simultaneously. The results showed that, despite quite similar acoustic results, the two speaker groups made different use of the tongue to implement vowel contrasts. Indeed, the tongue position was lower in the feedback OFF condition than the feedback ON condition for all participants, but the magnitude of this difference was larger for CI users than for their NH peers. This difference led to diminished intelligibility scores for CI users. This study shows the limitation of acoustic measurements alone and demonstrates how the use of articulatory measurements can explain intelligibility patterns. Moreover, our results show that when cochlear implantation occurs early in life and auditory feedback is available, CI users' intelligibility is comparable to that of their NH peers.

Maintaining Distinctiveness at Increased Speaking Rates: A Comparison between Congenitally Blind and Sighted Speakers

OBJECTIVES

The effects of increased speaking rates on vowels have been well documented in sighted adults. It has been reported that in fast speech, vowels are less widely spaced acoustically than in their citation form. Vowel space compression has also been reported in congenitally blind speakers. The objective of the study was to investigate the interaction of vision and speaking rate in adult speakers.

PATIENTS AND METHODS

Contrast distances between vowels were examined in conversational and fast speech produced by 10 congenitally blind and 10 sighted French-Canadian adults. Acoustic analyses were carried out.

RESULTS

Compared with the sighted speakers, in the fast speaking condition, the blind speakers produced more vowels with contrast along the height, place of articulation, and rounding features located within the auditory target regions typical of French vowels.

CONCLUSION

Blind speakers relied more heavily than sighted speakers on auditory properties of vowels to maintain perceptual distinctiveness.

The Lombard effect observed in speech produced by cochlear implant users in noisy environments: A naturalistic study

The Lombard effect is an involuntary response speakers experience in the presence of noise during voice communication. This phenomenon is known to cause changes in speech production such as an increase in intensity, pitch structure, formant characteristics, etc., for enhanced audibility in noisy environments. Although well studied for normal hearing listeners, the Lombard effect has received little, if any, attention in the field of cochlear implants (CIs). The objective of this study is to analyze speech production of CI users who are postlingually deafened adults with respect to environmental context. A total of six adult CI users were recruited to produce spontaneous speech in various realistic environments. Acoustic-phonetic analysis was then carried out to characterize their speech production in these environments. The Lombard effect was observed in the speech production of all CI users who participated in this study in adverse listening environments. The results indicate that both suprasegmental (e.g., F0, glottal spectral tilt and vocal intensity) and segmental (e.g., F1 for /i/ and /u/) features were altered in such environments. The analysis from this study suggests that modification of speech production of CI users under the Lombard effect may contribute to some degree an intelligible communication in adverse noisy environments.

Speaking Clearly for the Blind: Acoustic and Articulatory Correlates of Speaking Conditions in Sighted and Congenitally Blind Speakers

Compared to conversational speech, clear speech is produced with longer vowel duration, greater intensity, increased contrasts between vowel categories, and decreased dispersion within vowel categories. Those acoustic correlates are produced by larger movements of the orofacial articulators, including visible (lips) and invisible (tongue) articulators. Thus, clear speech provides the listener with audible and visual cues that are used to increase the overall intelligibility of speech produced by the speaker. It is unclear how those cues are produced by visually impaired speakers who never had access to vision. In this paper, we investigate the acoustic and articulatory correlates of vowels in clear versus conversational speech, and in sighted and congenitally blind speakers. Participants were recorded using electroarticulography while producing multiple repetitions of the ten Quebec French oral vowels in carrier sentences in both speaking conditions. Articulatory variables (lip, jaw, and tongue positions) as well as acoustic variables (contrasts between vowels, within-vowel dispersion, pitch, duration, and intensity) were measured. Lip movements were larger when going from conversational to clear speech in sighted speakers only. On the other hand, tongue movements were affected to a larger extent in blind speakers compared to their sighted peers. These findings confirm that vision plays an important role in the maintenance of speech intelligibility.

Acoustic characteristics of vowels produced by Greek intelligible speakers with profound hearing impairment I: Examination of vowel space

PURPOSE

The study examines F1, F2 and F3 formant frequencies of vowels produced by six Greek intelligible speakers with profound hearing impairment and six speakers with normal hearing (three male and three female in each group).

METHOD

The formant frequencies are measured in words of the form /'pVCV/ where V = /i, , , , u/ and C = /p, t, k, s/. The study examines differences in formant frequencies between the two groups and as a function of gender. Three measures are calculated to examine the area of the vowel space and differences along the F1 and F2 axes between the groups and genders.

RESULT

The results show that the vowel space produced by the speakers with hearing impairment is considerably reduced. Greater reduction was evident for F2 compared to F1. Restricted formant frequency ranges and relatively large variation along F1 and/or F2 for selected vowels resulted in overlap among vowel categories. F3 frequencies were systematically lower and showed greater variation for the speakers with hearing impairment.

CONCLUSION

The paper discusses findings with reference to perceptual and production constraints affecting the speech of individuals with hearing impairment.

Exploring consequences of short- and long-term deafness on speech production: a lip-tube perturbation study

Studies have reported strong links between speech production and perception. We aimed to evaluate the role of long- and short-term auditory feedback alteration on speech production. Eleven adults with normal hearing (controls) and 17 cochlear implant (CI) users (7 pre-lingually deaf and 10 post-lingually deaf adults) were recruited. Short-term auditory feedback deprivation was induced by turning off the CI or by providing masking noise. Acoustic and articulatory measures were obtained during the production of /u/, with and without a tube inserted between the lips (perturbation), and with and without auditory feedback. F1 values were significantly different between the implant OFF and ON conditions for the pre-lingually deaf participants. In the absence of auditory feedback, the pre-lingually deaf participants moved the tongue more forward. Thus, a lack of normal auditory experience of speech may affect the internal representation of a vowel.

Vowel Production in Persian Deaf Children with Cochlear Implant: is the Age of Implantation an Important Factor?

Proper production of vowels has great significance in speech intelligibility. Evidence shows that cochlear implantation has a significant impact on language and reading abilities in some children immediately after the surgery. The aim of the present study is comparing the quality of 6 simple Persian vowels between two groups of cochlear-implanted children under and over 2 years old. This was a cross-sectional analytic study conducted on 70 children who were implanted under the age of 2, 70 children who were implanted over the age of 2 and 238 normal children as control group. For data analysis, the SFS win acoustic analysis was used. Result of this study showed that F2/i/, f1/e/, f2/e/, f2/∞/, f1/a/, F2/a/, f1/o/, F2/o/and F2/u/means had significant difference between three groups (P < 0.05). Children implanted under the age of 2 showed similar performance as normal children in vowel production. Early cochlear implantation (under the age of 2) affects the quality of simple Persian vowel production significantly as well as the increase of speech intelligibility.

Five decades of research in speech motor control: what have we learned, and where should we go from here?

PURPOSE

The author presents a view of research in speech motor control over the past 5 decades, as observed from within Ken Stevens's Speech Communication Group (SCG) in the Research Laboratory of Electronics at MIT.

METHOD

The author presents a limited overview of some important developments and discoveries. The perspective is based largely on the research interests of the Speech Motor Control Group (SMCG) within the SCG; thus, it is selective, focusing on normal motor control of the vocal tract in the production of sound segments and syllables. It also covers the particular theories and models that drove the research. Following a brief introduction, there are sections on methodological advances, scientific advances, and conclusions.

RESULTS

Scientific and methodological advances have been closely interrelated. Advances in instrumentation and computer hardware and software have made it possible to record and process increasingly large, multifaceted data sets; introduce new paradigms for feedback perturbation; image brain activity; and develop more sophisticated, computational physiological and neural models. Such approaches have led to increased understanding of the widespread variability in speech, motor-equivalent trading relations, sensory goals, and the nature of feedback and feedforward neural control mechanisms.

CONCLUSIONS

Some ideas about important future directions for speech research are presented.

The effects of behavioral speech therapy on speech sound production with adults who have cochlear implants

PURPOSE

In this study, the authors examined the treatment efficacy of a behavioral speech therapy protocol for adult cochlear implant recipients.

METHOD

The authors used a multiple-baseline, across-behaviors and -participants design to examine the effectiveness of a therapy program based on behavioral principles and methods to improve the production of target speech sounds in 3 adults with cochlear implants. The authors included probe items in a baseline protocol to assess generalization of target speech sounds to untrained exemplars. Pretest and posttest scores from the Arizona Articulation Proficiency Scale, Third Revision (Arizona-3; Fudala, 2000) and measurement of speech errors during spontaneous speech were compared, providing additional measures of target behavior generalization.

RESULTS

The results of this study provided preliminary evidence supporting the overall effectiveness and efficiency of a behavioral speech therapy program in increasing percent correct speech sound production in adult cochlear implant recipients. The generalization of newly trained speech skills to untrained words and to spontaneous speech was demonstrated.

CONCLUSION

These preliminary findings support the application of behavioral speech therapy techniques for training speech sound production in adults with cochlear implants. Implications for future research and the development of aural rehabilitation programs for adult cochlear implant recipients are discussed.

Movement goals and feedback and feedforward control mechanisms in speech production

Studies of speech motor control are described that support a theoretical framework in which fundamental control variables for phonemic movements are multi-dimensional regions in auditory and somatosensory spaces. Auditory feedback is used to acquire and maintain auditory goals and in the development and function of feedback and feedforward control mechanisms. Several lines of evidence support the idea that speakers with more acute sensory discrimination acquire more distinct goal regions and therefore produce speech sounds with greater contrast. Feedback modification findings indicate that fluently produced sound sequences are encoded as feedforward commands, and feedback control serves to correct mismatches between expected and produced sensory consequences.

An investigation of the relation between sibilant production and somatosensory and auditory acuity

The relation between auditory acuity, somatosensory acuity and the magnitude of produced sibilant contrast was investigated with data from 18 participants. To measure auditory acuity, stimuli from a synthetic sibilant continuum ([s]-[ʃ]) were used in a four-interval, two-alternative forced choice adaptive-staircase discrimination task. To measure somatosensory acuity, small plastic domes with grooves of different spacing were pressed against each participant's tongue tip and the participant was asked to identify one of four possible orientations of the grooves. Sibilant contrast magnitudes were estimated from productions of the words 'said,' 'shed,' 'sid,' and 'shid'. Multiple linear regression revealed a significant relation indicating that a combination of somatosensory and auditory acuity measures predicts produced acoustic contrast. When the participants were divided into high- and low-acuity groups based on their median somatosensory and auditory acuity measures, separate ANOVA analyses with sibilant contrast as the dependent variable yielded a significant main effect for each acuity group. These results provide evidence that sibilant productions have auditory as well as somatosensory goals and are consistent with prior results and the theoretical framework underlying the DIVA model of speech production.

An acoustic analysis of the vowel space in young and old cochlear-implant speakers

The main purpose of this study was to compare acoustically the vowel spaces of two groups of cochlear implantees (CI) with two age-matched normal hearing groups. Five young test persons (15-25 years) and five older test persons (55-70 years) with CI and two control groups of the same age with normal hearing were recorded. The speech material consisted of five German vowels V = /a, e, i, o, u/ in bilabial and alveolar contexts. The results showed no differences between the two groups on Euclidean distances for the first formant frequency. In contrast, Euclidean distances for F2 of the CI group were shorter than those of the control group, causing their overall vowel space to be compressed. The main differences between the groups are interpreted in terms of the extent to which the formants are associated with visual cues to the vowels. Further results were partially longer vowel durations for the CI speakers.

Production and perception of French vowels by congenitally blind adults and sighted adults

The goal of this study is to investigate the production and perception of French vowels by blind and sighted speakers. 12 blind adults and 12 sighted adults served as subjects. The auditory-perceptual abilities of each subject were evaluated by discrimination tests (AXB). At the production level, ten repetitions of the ten French oral vowels were recorded. Formant values and fundamental frequency values were extracted from the acoustic signal. Measures of contrasts between vowel categories were computed and compared for each feature (height, place of articulation, roundedness) and group (blind, sighted). The results reveal a significant effect of group (blind vs sighted) on production, with sighted speakers producing vowels that are spaced further apart in the vowel space than those of blind speakers. A group effect emerged for a subset of the perceptual contrasts examined, with blind speakers having higher peak discrimination scores than sighted speakers. Results suggest an important role of visual input in determining speech goals.

Effects of short- and long-term changes in auditory feedback on vowel and sibilant contrasts

PURPOSE

To assess the effects of short- and long-term changes in auditory feedback on vowel and sibilant contrasts and to evaluate hypotheses arising from a model of speech motor planning.

METHOD

The perception and production of vowel and sibilant contrasts were measured in 8 postlingually deafened adults prior to activation of their cochlear implant speech processors, 1 month postactivation, and 1 year postactivation. Measures were taken postactivation both with and without auditory feedback. Contrast measures were also made for a group of speakers with reportedly normal hearing speaking with masked and unmasked auditory feedback.

RESULTS

Vowel and sibilant contrasts, measured in the absence of auditory feedback after 1 month of prosthesis use, were diminished compared with their values measured before prosthesis. Contrasts measured in the absence of auditory feedback after 1 year's experience with the prosthesis were increased compared with their values after 1 month's experience. In both time samples, contrasts were enhanced when auditory feedback was restored.

CONCLUSION

The provision of prosthetic hearing to postlingually deafened adults impaired their phonemic contrasts at first, as their auditory feedback had novel characteristics. Once auditory feedback became recalibrated with prosthesis use, it could, in turn, revise feedforward commands that control the contrasts in its absence.

Interactions of speaking condition and auditory feedback on vowel production in postlingually deaf adults with cochlear implants

This study investigates the effects of speaking condition and auditory feedback on vowel production by postlingually deafened adults. Thirteen cochlear implant users produced repetitions of nine American English vowels prior to implantation, and at one month and one year after implantation. There were three speaking conditions (clear, normal, and fast), and two feedback conditions after implantation (implant processor turned on and off). Ten normal-hearing controls were also recorded once. Vowel contrasts in the formant space (expressed in mels) were larger in the clear than in the fast condition, both for controls and for implant users at all three time samples. Implant users also produced differences in duration between clear and fast conditions that were in the range of those obtained from the controls. In agreement with prior work, the implant users had contrast values lower than did the controls. The implant users' contrasts were larger with hearing on than off and improved from one month to one year postimplant. Because the controls and implant users responded similarly to a change in speaking condition, it is inferred that auditory feedback, although demonstrably important for maintaining normative values of vowel contrasts, is not needed to maintain the distinctiveness of those contrasts in different speaking conditions.

The distinctness of speakers' /s/-/S/ contrast is related to their auditory discrimination and use of an articulatory saturation effect

This study examines individual differences in producing the sibilant contrast in American English and the relation of those differences to 2 speaker characteristics: (a) use of a quantal biomechanical effect (called a "saturation effect") in producing the sibilants and (b) performance on a test of sibilant discrimination. Twenty participants produced the sibilants /s/ and /S/ in normal-, clear-, and fast-speaking conditions. The degree to which the participants used a saturation effect in producing /s/ and /S/ was assessed with a custom-made sensor that measured contact of the underside of the tongue tip with the lower alveolar ridge; such contact normally occurs during the production of /s/ but not /S/. The acuteness of the participants' discrimination of the sibilant contrast was measured using the ABX paradigm and synthesized sibilants. Differences among speakers in the degree of acoustic contrast between /s/ and /S/ that they produced proved related to differences among them in their use of contact contrastively and in their discriminative performance. The most distinct sibilant productions were obtained from participants who used contact in producing /s/ but not /S/ and who had high discrimination scores. The participants who did not use contact differentially when producing the 2 sibilants and who also discriminated the synthetic sibilants less well produced the least distinct sibilant contrasts. Intermediate degrees of sibilant contrast were found with participants who used contact differentially or discriminated well. These findings are compatible with a model of speech motor planning in which goals for phonemic speech movements are in somatosensory and auditory spaces.

The distinctness of speakers' productions of vowel contrasts is related to their discrimination of the contrasts

This study addresses the hypothesis that the more accurately a speaker discriminates a vowel contrast, the more distinctly the speaker produces that contrast. Measures of speech production and perception were collected from 19 young adult speakers of American English. In the production experiment, speakers repeated the words cod, cud, who'd, and hood in a carrier phrase at normal, clear, and fast rates. Articulatory movements and the associated acoustic signal were recorded, yielding measures of contrast distance between /a/ and /[see text for symbol]/ and between /u/ and /[see text for symbol]/. In the discrimination experiment, sets of seven natural-sounding stimuli ranging from cod to cud and who'd to hood were synthesized, based on productions by one male and one female speaker. The continua were then presented to each of the 19 speakers in labeling and discrimination tasks. Consistent with the hypothesis, speakers with discrimination scores above the median produced greater acoustic contrasts than speakers with discrimination scores at or below the median. Such a relation between speech production and perception is compatible with a model of speech production in which articulatory movements for vowels are planned primarily in auditory space.