Two-component hearing sensations produced by two-electrode stimulation in the cochlea of a deaf patient

A Hundred Ways to Encode Sound Signals for Cochlear Implants

Cochlear implants are the most successful neural prostheses used to restore hearing in severe-to-profound hearing-impaired individuals. The field of cochlear implant coding investigates interdisciplinary approaches to translate acoustic signals into electrical pulses transmitted at the electrode-neuron interface, ranging from signal preprocessing algorithms, enhancement, and feature extraction methodologies to electric signal generation. In the last five decades, numerous coding strategies have been proposed clinically and experimentally. Initially developed to restore speech perception, increasing computational possibilities now allow coding of more complex signals, and new techniques to optimize the transmission of electrical signals are constantly gaining attention. This review provides insights into the history of multichannel coding and presents an extensive list of implemented strategies. The article briefly addresses each method and considers promising future directions of neural prostheses and possible signal processing, with the ultimate goal of providing a current big picture of the large field of cochlear implant coding.

Evaluation of a Two-Formant Speech-Processing Strategy for a Multichannel Cochlear Prosthesis

Initial results with the two-formant speech-processing strategy (F0F1F2) confirm the advantage of a multichannel cochlear prosthesis capable of stimulating at different sites within the cochlea. The successful presentation of two spectra] components by varying the place of stimulation leads to the possibility of presenting further spectral information in this manner. Because virtually all multichannel implant patients demonstrate good “place” (electrode site) discrimination, these more refined coding strategies should lead to benefits for the majority of implantees. Already, with the F0F1F2 strategy, we have a system that appears to provide some effective auditory-alone communication ability for the average patient.

Speech-Processing Strategies and Their Implementation

Since 1979, the Australian speech-processing strategy has been based on the presentation of an estimate of F2 coded by electrode position and F0 coded by pulse rate. Although providing limited information, this strategy has produced good results with significant hearing-alone performance. This paper describes a number of strategies that provide further speech information in an attempt to increase hearing-alone performance to a level where the cochlear implant is able to operate in its own right rather than as an adjunct to lipreading. The strategies are all based on the addition of F1 to the existing strategy. Both electrode and temporal coding of F1 is described, and the performance and percepts produced are discussed. Amplitudes of the two formants must be carefully controlled to avoid masking. The implications of the strategies on the design of hardware are described.

Comparison of Two Cochlear Implant Speech-Processing Strategies

Speech processors extracting either the fundamental frequency (F0) alone, or the fundamental frequency combined with second formant information (F0-F2), have been evaluated on a totally deaf patient using a multiple-channel cochlear implant. A closed set test using 16 spondees and a modified rhyme test showed that for electrical stimulation alone the F0-F2 speech processor was significantly better than the F0 processor. The open set tests using phonetically balanced words and Central Institute for the Deaf everyday sentences showed that for electrical stimulation alone and electrical stimulation combined with lipreading, the results with the F0-F2 speech processor were all significantly better than with the F0 processor. Information transmission for consonant speech features was also better when using the F0-F2 processor.

Multidimensional scaling between acoustic and electric stimuli in cochlear implant users with contralateral hearing

This study investigated the perceptual relationship between acoustic and electric stimuli presented to CI users with functional contralateral hearing. Fourteen subjects with unilateral profound deafness implanted with a MED-EL CI scaled the perceptual differences between pure tones presented to the acoustic hearing ear and electric biphasic pulse trains presented to the implanted ear. The differences were analyzed with a multidimensional scaling (MDS) analysis. Additionally, speech performance in noise was tested using sentence material presented in different spatial configurations while patients listened with both their acoustic hearing and implanted ears. Results of alternating least squares scaling (ALSCAL) analysis consistently demonstrate that a change in place of stimulation is in the same perceptual dimension as a change in acoustic frequency. However, the relative perceptual differences between the acoustic and the electric stimuli varied greatly across subjects. A degree of perceptual separation between acoustic and electric stimulation (quantified by relative dimensional weightings from an INDSCAL analysis) was hypothesized that would indicate a change in perceptual quality, but also be predictive of performance with combined acoustic and electric hearing. Perceptual separation between acoustic and electric stimuli was observed for some subjects. However, no relationship between the degree of perceptual separation and performance was found.

The effect of visual cues on difficulty ratings for segregation of musical streams in listeners with impaired hearing

Background

Enjoyment of music is an important part of life that may be degraded for people with hearing impairments, especially those using cochlear implants. The ability to follow separate lines of melody is an important factor in music appreciation. This ability relies on effective auditory streaming, which is much reduced in people with hearing impairment, contributing to difficulties in music appreciation. The aim of this study was to assess whether visual cues could reduce the subjective difficulty of segregating a melody from interleaved background notes in normally hearing listeners, those using hearing aids, and those using cochlear implants.

Methodology/Principal Findings

Normally hearing listeners (N = 20), hearing aid users (N = 10), and cochlear implant users (N = 11) were asked to rate the difficulty of segregating a repeating four-note melody from random interleaved distracter notes. The pitch of the background notes was gradually increased or decreased throughout blocks, providing a range of difficulty from easy (with a large pitch separation between melody and distracter) to impossible (with the melody and distracter completely overlapping). Visual cues were provided on half the blocks, and difficulty ratings for blocks with and without visual cues were compared between groups. Visual cues reduced the subjective difficulty of extracting the melody from the distracter notes for normally hearing listeners and cochlear implant users, but not hearing aid users.

Conclusion/Significance

Simple visual cues may improve the ability of cochlear implant users to segregate lines of music, thus potentially increasing their enjoyment of music. More research is needed to determine what type of acoustic cues to encode visually in order to optimise the benefits they may provide.

Investigating the effects of stimulus duration and context on pitch perception by cochlear implant users

Cochlear implant sound processing strategies that use time-varying pulse rates to transmit fine structure information are one proposed method for improving the spectral representation of a sound with the eventual goal of improving speech recognition in noisy conditions, speech recognition in tonal languages, and music identification and appreciation. However, many of the perceptual phenomena associated with time-varying rates are not well understood. In this study, the effects of stimulus duration on both the place and rate-pitch percepts were investigated via psychophysical experiments. Four Nucleus CI24 cochlear implant users participated in these experiments, which included a short-duration pitch ranking task and three adaptive pulse rate discrimination tasks. When duration was fixed from trial-to-trial and rate was varied adaptively, results suggested that both the place-pitch and rate-pitch percepts may be independent of duration for durations above 10 and 20 ms, respectively. When duration was varied and pulse rates were fixed, performance was highly variable within and across subjects. Implications for multi-rate sound processing strategies are discussed.

The multiple-channel cochlear implant: the interface between sound and the central nervous system for hearing, speech, and language in deaf people-a personal perspective

The multiple-channel cochlear implant is the first sensori-neural prosthesis to effectively and safely bring electronic technology into a direct physiological relation with the central nervous system and human consciousness, and to give speech perception to severely-profoundly deaf people and spoken language to children. Research showed that the place and temporal coding of sound frequencies could be partly replicated by multiple-channel stimulation of the auditory nerve. This required safety studies on how to prevent the effects to the cochlea of trauma, electrical stimuli, biomaterials and middle ear infection. The mechanical properties of an array and mode of stimulation for the place coding of speech frequencies were determined. A fully implantable receiver-stimulator was developed, as well as the procedures for the clinical assessment of deaf people, and the surgical placement of the device. The perception of electrically coded sounds was determined, and a speech processing strategy discovered that enabled late-deafened adults to comprehend running speech. The brain processing systems for patterns of electrical stimuli reproducing speech were elucidated. The research was developed industrially, and improvements in speech processing made through presenting additional speech frequencies by place coding. Finally, the importance of the multiple-channel cochlear implant for early deafened children was established.

The development of the Nucleus Freedom Cochlear implant system

Cochlear Limited (Cochlear) released the fourth-generation cochlear implant system, Nucleus Freedom, in 2005. Freedom is based on 25 years of experience in cochlear implant research and development and incorporates advances in medicine, implantable materials, electronic technology, and sound coding. This article presents the development of Cochlear's implant systems, with an overview of the first 3 generations, and details of the Freedom system: the CI24RE receiver-stimulator, the Contour Advance electrode, the modular Freedom processor, the available speech coding strategies, the input processing options of Smart Sound to improve the signal before coding as electrical signals, and the programming software. Preliminary results from multicenter studies with the Freedom system are reported, demonstrating better levels of performance compared with the previous systems. The final section presents the most recent implant reliability data, with the early findings at 18 months showing improved reliability of the Freedom implant compared with the earlier Nucleus 3 System. Also reported are some of the findings of Cochlear's collaborative research programs to improve recipient outcomes. Included are studies showing the benefits from bilateral implants, electroacoustic stimulation using an ipsilateral and/or contralateral hearing aid, advanced speech coding, and streamlined speech processor programming.

Are Cochlear Implant Patients Suffering From Perceptual Dissonance?

Cochlear implants provide functional hearing to the majority of recipients and have gained widespread acceptance clinically, but the range of performance remains great and largely unexplained. Designs for implanted electrodes and electronics have converged, whereas novel speech processing strategies have proliferated. For each patient, the fitting audiologist must sort empirically through options that produce large but idiosyncratic differences in both objective performance and subjective preference. This review and analysis suggests that the place-pitch and rate-pitch theories on which cochlear implants have been designed are incomplete. The missing component may be related to the phase-locking of auditory nerve activity to both acoustic and electrical stimulation. This component is likely to be highly distorted by electrical stimulation but its importance as one of several different pitch encoding mechanisms may vary widely among patients. Systematic means to control these putative phase effects using modern, high-speed, and high-density cochlear implants may make it possible to identify more efficiently the best strategy for a given patient and to minimize the perceptual confusion that arises from conflicting cues.

Temporal pitch in electric hearing

Both place and temporal codes in the peripheral auditory system contain pitch information, however, their actual use by the brain is unclear. Here pitch data are reported from users of the cochlear implant, which provides the ability to change the temporal code independently from the place code. With fixed electrode stimulation, both frequency discrimination and pitch estimate data show that the cochlear implant users can only discern differences in pitch for frequencies up to about 300 Hz. An integration model can predict pitch estimation from frequency discrimination, reinforcing Fechner's hypothesis relating sensation magnitude to stimulus discriminability. The present results suggest that 300 Hz is the upper boundary of the temporal code and that the absolute place information should be included in the present pitch models. They further suggest that future cochlear implants need to increase the number of independent electrodes to restore normal pitch range and resolution.

Investigating perceptual features of electrode stimulation via a multidimensional scaling paradigm

To achieve the most effective speech processing for individuals with cochlear implants, it is important to understand the perceptual features associated with the stimulation parameters. In general, when electrodes are stimulated in order from apex to base, the pitch of the perceived sound changes in an orderly fashion from low to high. Some deviations from this assumed order have been documented. Also, pitch is the dominant perceptual attribute of a sound when the stimuli associated with different electrodes have been accurately loudness balanced. In this study, the results of a multidimensional scaling (MDS) paradigm were compared to the results of a pitch-ranking procedure for six subjects implanted with multichannel cochlear prostheses. Results indicate that there may be multiple percepts that change with electrode location. Not surprisingly, the dominant percept is strongly correlated with pitch. The results also indicate that the structure of the second percept is consistent across subjects, although not interpretable using the data measured in this study. Furthermore, results indicate that MDS data can be used to pinpoint indiscriminable electrodes more accurately than pitch data. The results of this study may have importance for the design of the next generation of speech processors.

Electrical stimulation of the auditory nerve: the coding of frequency, the perception of pitch and the development of cochlear implant speech processing strategies for profoundly deaf people

1. The development of speech processing strategies for multiple-channel cochlear implants has depended on encoding sound frequencies and intensities as temporal and spatial patterns of electrical stimulation of the auditory nerve fibres so that speech information of most importance of intelligibility could be transmitted. 2. Initial physiological studies showed that rate encoding of electrical stimulation above 200 pulses/s could not reproduce the normal response patterns in auditory neurons for acoustic stimulation in the speech frequency range above 200 Hz and suggested that place coding was appropriate for the higher frequencies. 3. Rate difference limens in the experimental animal were only similar to those for sound up to 200 Hz. 4. Rate difference limens in implant patients were similar to those obtained in the experimental animal. 5. Satisfactory rate discrimination could be made for durations of 50 and 100 ms, but not 25 ms. This made rate suitable for encoding longer duration suprasegmental speech information, but not segmental information, such as consonants. The rate of stimulation could also be perceived as pitch, discriminated at different electrode sites along the cochlea and discriminated for stimuli across electrodes. 6. Place pitch could be scaled according to the site of stimulation in the cochlea so that a frequency scale was preserved and it also had a different quality from rate pitch and was described as tonality. Place pitch could also be discriminated for the shorter durations (25 ms) required for identifying consonants. 7. The inaugural speech processing strategy encoded the second formant frequencies (concentrations of frequency energy in the mid frequency range of most importance for speech intelligibility) as place of stimulation, the voicing frequency as rate of stimulation and the intensity as current level. Our further speech processing strategies have extracted additional frequency information and coded this as place of stimulation. The most recent development, however, presents temporal frequency information as amplitude variations at a constant rate of stimulation. 8. As additional speech frequencies have been encoded as place of stimulation, the mean speech perception scores have continued to increase and are now better than the average scores that severely-profoundly deaf adults and children with some residual hearing obtain with a hearing aid.

Electrode discrimination by early-deafened cochlear implant patients

Electrode discrimination was measured in six cochlear-implant patients who became profoundly deaf prior to the full development of auditory and speech skills. The cochlear implant manufactured by Cochlear Limited was used. Comparisons were made between two electric stimulation paradigms and two data collection procedures. The paradigms consisted of electrode trajectories and electrodes with random variation in electric current levels. The data collection methods consisted of an adaptive procedure to obtain difference limens and a method of constant stimuli that gives psychometric functions of discrimination performance. Data were collected for reference electrodes at the apical, mid, and basal positions on the array. The psychometric functions showed that discrimination performance improved with increases in the spatial separation between reference and comparison electrodes. The difference limens were similar to the corresponding spatial separation estimated from the psychometric function in most cases. The discrimination performance of most patients was consistent across the two stimulation paradigms. Difference limens were 1-4 electrodes from the reference electrode for most patients. There were differences in performance across patients and the three positions on the array.

Channel interactions in patients using the Ineraid multichannel cochlear implant

Electrode interactions were investigated on two totally deaf patients fitted with the Ineraid multichannel cochlear implant. Currents were applied to the most apical electrode (the 'perturbation' electrode) and their effects on psychophysical thresholds on the other electrodes (the 'test' electrodes) of the intracochlear array were studied. Two experimental protocols were used. In experiment I, we used a detection protocol to study how the perception of signals presented on each test electrode was affected by subthreshold, simultaneous or non-simultaneous stimulation of the perturbation electrode. Strong electrode interactions were observed with simultaneous stimulation and monotonically decreased as a function of electrode separation. Electrode interactions were weak with non-simultaneous stimulation. In experiment II, we used a discrimination protocol to study how the perception of signals presented on the test electrode was affected by suprathreshold, non-simultaneous stimulation of the perturbation electrode. Subjects could discriminate stimulation of 'perturbation+test' versus 'perturbation alone' when the level of stimulation on the test electrode was near threshold. These results demonstrate that strong electrode interactions in the Ineraid multichannel cochlear implant system are generated by electrical field summation due to simultaneous stimulation of different electrodes, and that one can reduce electrode interactions by sequential activation of the electrodes. These observations might help to understand basic phenomena underlying recent significant improvements in speech recognition scores when switching from simultaneous to interleaved pulsatile stimulation in patients wearing the same cochlear implant system (Wilson et al., 1991).

Sound perception induced by extracranial magnetic stimulation in deaf patients

Two profoundly hard-of-hearing and deaf patients were examined by non-invasive extracranial magnetic stimulation (EMS) in an effort to determine whether EMS could evoke auditory sensations. The patients were fitted with standard earplugs and were stimulated at the auricle, the mastoid and the temporal lobe area. The threshold of auditory sensation (TAS) was determined at each stimulus position and found to be approximately 20-40% of the maximum EMS level (2.0 Tesla). The TAS was generally lowest in mastoid stimulation, but was variable, and dependent on the angle and position of the stimulating coil relative to the skull. Middle-ear muscle reflex (MEMR) tests performed by EMS of the auricle, mastoid and temporal lobe area contralateral to the probe ear were negative. It was concluded that EMS of the auditory system, particularly the mastoid area, can evoke auditory sensations in cochlea-deaf ears, and that this technique deserves further study as a non-invasive procedure for evaluating potential cochlear implant patients in conjunction with electrostimulation.

Speech perception using a two-formant 22-electrode cochlear prosthesis in quiet and in noise

A new speech-processing strategy has been developed for the Cochlear Pty. Ltd. 22-electrode cochlear prosthesis which codes an estimate of the first formant frequency in addition to the amplitude, voice pitch and second formant frequencies. Two groups of cochlear implant patients were tested 3 months after implant surgery, one group (n = 13) having used the old (F0F2) processing strategy and the other (n = 9) having used the new (F0F1F2) strategy. All patients underwent similar postoperative training programs. Results indicated significantly improved speech recognition for the F0F1F2 group, particularly on open set tests with audition alone. Additional testing with a smaller group of patients was carried out with competing noise (speech babble). Results for a closed set spondee test showed that patient performance was significantly degraded at a signal-to-noise ratio of 10 dB when using the F0F2 strategy, but was not significantly affected with the F0F1F2 strategy.

Cochlear Implant for Simultaneous Multichannel Stimulation

A new cochlear implant is described that can stimulate up to three bipolar electrode pairs simultaneously. It uses a scala tympani electrode array comprising 20 separate platinum ring electrodes. The bipolar mode of stimulation is used to minimize the spread of current in the cochlea. Nearly all of the electronics of the device are integrated into a single custom-designed large-scale integrated circuit. A prototype of the chip has been fabricated using an advanced complementary metal oxide semiconductor technology. Preliminary test results indicate that the device functions according to its specifications. The implant, which is still under development, will use a single transcutaneous inductive link to receive both controlling data and electric power from an external speech processor.

Temporal Coding of Speech Information for Cochlear Implant Patients

A modified speech-processing strategy incorporating the temporal coding of information strongly correlated with the first formant of speech was evaluated in a long-term clinical experiment with a single patient. The aim was to assess whether the patient could learn to extract information from the time domain in addition to the time domain cues for voice excitation frequency already received from the initial strategy. It was found that the patient gained no significant advantage from the modified strategy, but there was no disadvantage either, and the patient expressed a preference for the modified strategy for everyday use.

Perception of connected speech without lipreading, using a multi-channel hearing prosthesis

Four of 13 totally deaf patients implanted with the Nucleus multi-channel hearing prosthesis at the University of Melbourne have demonstrated the ability to understand connected speech without lipreading or other visual cues. These patients were able to repeat verbatim unknown material read by a tester at rates of up to 35 words per minute. They were also able to understand an average of 78% of key words in everyday sentences in ideal acoustic conditions and 51% of equivalent material over the telephone. These results show that with a good proportion of postlingually deaf patients, the multi-channel hearing prosthesis can not only act as an aid to lipreading, but also restore effective speech understanding without lipreading.

Surgery for an improved multiple-channel cochlear implant

An improved multiple-channel cochlear implant has been developed. The titanium container with enclosed electronics, the receiver coil, and the connector are embedded in medical-grade Silastic. The upper half of the implant has a diameter of 35 mm and a height of 4.5 mm, and the lower half a diameter of 23 mm and a height of 5 mm. The electrode array has also been designed to reduce the possibility of breakage due to repeated movements over many years. The surgery involves drilling a bed in the mastoid bone for the receiver-stimulator, and fixing the proximal electrode under the mastoid cortex. Gentle insertion of the electrode array through the round window and along the scala tympani is achieved with a specially designed microclaw .

Multichannel electrical stimulation of the auditory nerve in man. II. Channel interaction

A multichannel cochlear implant can be an effective prosthesis only if its channels are independent of each other. Presumably independence is achieved by stimulating different populations of surviving neurons. Two types of interaction might occur between channels: electrical current field summation peripheral to stimulation of the nerves and neural-perceptual interaction following stimulation. Two psychophysical techniques to assess channel independence are discussed. In one technique a masker is presented on one channel in order to adapt the nerves responding to that channel. The forward masked threshold of a signal is then measured on all other channels and elevation of threshold is assumed to indicate overlapping neural populations. In the second procedure channel interaction is evaluated by measuring the loudness summation of stimuli presented simultaneously to two channels. The magnitude, distribution, and phasic components of the loudness summation are measures of interaction between channels. Data from two subjects suggests that monopolar stimulation produces broader interaction patterns than bipolar stimulation as a function of electrode separation. Considerable differences in the extent of channel interaction were observed between the two subjects, possibly because of the difference in the absolute current levels needed for equivalent sensation levels.