A bio-inspired coding (BIC) strategy for cochlear implants

A Hundred Ways to Encode Sound Signals for Cochlear Implants

Cochlear implants are the most successful neural prostheses used to restore hearing in severe-to-profound hearing-impaired individuals. The field of cochlear implant coding investigates interdisciplinary approaches to translate acoustic signals into electrical pulses transmitted at the electrode-neuron interface, ranging from signal preprocessing algorithms, enhancement, and feature extraction methodologies to electric signal generation. In the last five decades, numerous coding strategies have been proposed clinically and experimentally. Initially developed to restore speech perception, increasing computational possibilities now allow coding of more complex signals, and new techniques to optimize the transmission of electrical signals are constantly gaining attention. This review provides insights into the history of multichannel coding and presents an extensive list of implemented strategies. The article briefly addresses each method and considers promising future directions of neural prostheses and possible signal processing, with the ultimate goal of providing a current big picture of the large field of cochlear implant coding.

Outcomes Using the Optimized Pitch and Language Strategy Versus the Advanced Combination Encoder Strategy in Mandarin-Speaking Cochlear Implant Recipients

OBJECTIVES

The experimental Optimized Pitch and Language (OPAL) strategy enhances coding of fundamental frequency (F0) information in the temporal envelope of electrical signals delivered to channels of a cochlear implant (CI). Previous studies with OPAL have explored performance on speech and lexical tone perception in Mandarin- and English-speaking CI recipients. However, it was not clear which cues to lexical tone (primary and/or secondary) were used by the Mandarin CI listeners. The primary aim of the present study was to investigate whether OPAL provides improved recognition of Mandarin lexical tones in both quiet and noisy environments compared with the Advanced Combination Encoder (ACE) strategy. A secondary aim was to investigate whether, and to what extent, removal of secondary (duration and intensity envelope) cues to lexical tone affected Mandarin tone perception.

DESIGN

Thirty-two CI recipients with an average age of 24 (range 7 to 57) years were enrolled in the study. All recipients had at least 1 year of experience using ACE. Each subject attended two testing sessions, the first to measure baseline performance, and the second to evaluate the effect of strategy after provision of some take-home experience using OPAL. A minimum take-home duration of approximately 4 weeks was prescribed in which subjects were requested to use OPAL as much as possible but were allowed to also use ACE when needed. The evaluation tests included recognition of Mandarin lexical tones in quiet and in noise (signal to noise ratio [SNR] +5 dB) using naturally produced tones and duration/intensity envelope normalized versions of the tones; Mandarin sentence in adaptive noise; Mandarin monosyllabic and disyllabic word in quiet; a subset of Speech, Spatial, and Qualities of hearing questionnaire (SSQ, speech hearing scale); and subjective preference for strategy in quiet and noise.

RESULTS

For both the natural and normalized lexical tone tests, mean scores for OPAL were significantly higher than ACE in quiet by 2.7 and 2.9%-points, respectively, and in noise by 7.4 and 7.2%-points, respectively. Monosyllabic word recognition in quiet using OPAL was significantly higher than ACE by approximately 7.5% points. Average SSQ ratings for OPAL were significantly higher than ACE by approximately 0.5 points on a 10-point scale. In quiet conditions, 14 subjects preferred OPAL, 7 expressed a preference for ACE, and 9 reported no preference. Compared with quiet, in noisy situations, there was a stronger preference for OPAL (19 recipients), a similar preference for ACE (7 recipients), while fewer expressed no preference. Average daily take-home use of ACE and OPAL was 4.9 and 7.1 hr, respectively.

CONCLUSIONS

For Mandarin-speaking CI recipients, OPAL provided significant improvements to lexical tone perception for natural and normalized tones in quiet and noise, monosyllabic word recognition in quiet, and subjective ratings of speech intelligibility. Subjects accessed both primary and secondary cues to lexical tone for perception in quiet and noise conditions. The benefits of lexical tone recognition were attributed to enhanced F0 rate cues encoded by OPAL, especially in a noisy environment. The OPAL strategy was well accepted by many of the Mandarin-speaking CI recipients.

InterlACE Sound Coding for Unilateral and Bilateral Cochlear Implants

OBJECTIVE

Cochlear implant signal processing strategies define the rules of how acoustic signals are converted into electrical stimulation patterns. Technological and anatomical limitations, however, impose constraints on the signal transmission and the accurate excitation of the auditory nerve. Acoustic signals are degraded throughout cochlear implant processing, and electrical signal interactions at the electrode-neuron interface constrain spectral and temporal precision. In this work, we propose a novel InterlACE signal processing strategy to counteract the occurring limitations.

METHODS

By replacing the maxima selection of the Advanced Combination Encoder strategy with a method that defines spatially and temporally alternating channels, InterlACE can compensate for discarded signal content of the conventional processing. The strategy can be extended bilaterally by introducing synchronized timing and channel selection. InterlACE was explored unilaterally and bilaterally by assessing speech intelligibility and spectral resolution. Five experienced bilaterally implanted cochlear implant recipients participated in the Oldenburg Sentence Recognition Test in background noise and the spectral ripple discrimination task.

RESULTS

The introduced alternating channel selection methodology shows promising outcomes for speech intelligibility but could not indicate better spectral ripple discrimination.

CONCLUSION

InterlACE processing positively affects speech intelligibility, increases available unilateral and bilateral signal content, and may potentially counteract signal interactions at the electrode-neuron interface.

SIGNIFICANCE

This work shows how cochlear implant channel selection can be modified and extended bilaterally. The clinical impact of the modifications needs to be explored with a larger sample size.

Neuromorphic hardware for somatosensory neuroprostheses

In individuals with sensory-motor impairments, missing limb functions can be restored using neuroprosthetic devices that directly interface with the nervous system. However, restoring the natural tactile experience through electrical neural stimulation requires complex encoding strategies. Indeed, they are presently limited in effectively conveying or restoring tactile sensations by bandwidth constraints. Neuromorphic technology, which mimics the natural behavior of neurons and synapses, holds promise for replicating the encoding of natural touch, potentially informing neurostimulation design. In this perspective, we propose that incorporating neuromorphic technologies into neuroprostheses could be an effective approach for developing more natural human-machine interfaces, potentially leading to advancements in device performance, acceptability, and embeddability. We also highlight ongoing challenges and the required actions to facilitate the future integration of these advanced technologies.

En route to sound coding strategies for optical cochlear implants

Hearing loss is the most common human sensory deficit. Severe-to-complete sensorineural hearing loss is often treated by electrical cochlear implants (eCIs) bypassing dysfunctional or lost hair cells by direct stimulation of the auditory nerve. The wide current spread from each intracochlear electrode array contact activates large sets of tonotopically organized neurons limiting spectral selectivity of sound coding. Despite many efforts, an increase in the number of independent eCI stimulation channels seems impossible to achieve. Light, which can be better confined in space than electric current may help optical cochlear implants (oCIs) to overcome eCI shortcomings. In this review, we present the current state of the optogenetic sound encoding. We highlight optical sound coding strategy development capitalizing on the optical stimulation that requires fine-grained, fast, and power-efficient real-time sound processing controlling dozens of microscale optical emitters as an emerging research area.

Facilitation properties in electrically evoked compound action potentials depending on spatial location and on threshold

Spiral ganglion neurons (SGNs) facilitation properties can be recorded utilizing electrically evoked compound action potential (ECAP). While intracochlear variation of the ECAP threshold in relation to its electrode channel is reported, no study investigated its impact on facilitation. In this study, we quantified intracochlear variation of the facilitation properties in cochlear implants (CI) using ECAPs. We hypothesized that the facilitation effect is dependent on the electrode channel and its ECAP threshold. Therefore, ECAPs were recorded in 23 CI subjects. For each subject, five default (channel-derived) and up to two additional (threshold-derived) stimulation sites were defined. Facilitation was quantified by the paradigm introduced by (Hey et al., 2017) with optimized parameter settings. For each channel the maximum facilitated amplitude was determined by a series of ECAP measurements. A linear mixed-effects model was used to investigate the impact of the electrode channel and ECAP threshold on the maximum facilitated amplitude. The maximum facilitated amplitude was found to be dependent on the ECAP threshold and independent on the electrode channel. We conclude that the facilitation paradigm is a useful and feasible tool to gain local information on the SGNs temporal processing patterns.

The effects of estimation accuracy, estimation approach, and number of selected channels using formant-priority channel selection for an "n-of-m" sound processing strategy for cochlear implants

Previously, selection of l channels was prioritized according to formant frequency locations in an l-of-n-of-m-based signal processing strategy to provide important voicing information independent of listening environments for cochlear implant (CI) users. In this study, ideal, or ground truth, formants were incorporated into the selection stage to determine the effect of accuracy on (1) subjective speech intelligibility, (2) objective channel selection patterns, and (3) objective stimulation patterns (current). An average +11% improvement (p < 0.05) was observed across six CI users in quiet, but not for noise or reverberation conditions. Analogous increases in channel selection and current for the upper range of F1 and a decrease across mid-frequencies with higher corresponding current, were both observed at the expense of noise-dominant channels. Objective channel selection patterns were analyzed a second time to determine the effects of estimation approach and number of selected channels (n). A significant effect of estimation approach was only observed in the noise and reverberation condition with minor differences in channel selection and significantly decreased stimulated current. Results suggest that estimation method, accuracy, and number of channels in the proposed strategy using ideal formants may improve intelligibility when corresponding stimulated current of formant channels are not masked by noise-dominant channels.

Restoration of the senses and human communication: Sustainable Development Goals 3 and 9

PURPOSE

This invited commentary addresses the importance of the senses in human communication, outlines advances achieved with cochlear implants, and new research directions to improve neural prostheses.

RESULT

In severely deaf people, cochlear implants restore speech understanding and enable children to achieve spoken language. Research in neural prostheses is advancing the restoration of hearing, vision, tactile senses, movement and the management of epilepsy. Bio-inspired stimulation strategies incorporating temporal and spatial characteristics of neural responses may deliver improved speech, vision and tactile perception using prostheses. To achieve stable long-term stimulation, chronic inflammation at the brain-electrode interface may be reduced using ROCK/Rho signalling pathway inhibitors and materials with brain-mimicking properties.

CONCLUSION

This commentary paper addresses two Sustainable Development Goals: industry, innovation and infrastructure (SDG 9) and good health and well-being (SDG 3).

Relative Weights of Temporal Envelope Cues in Different Frequency Regions for Mandarin Vowel, Consonant, and Lexical Tone Recognition

Objectives: Mandarin-speaking users of cochlear implants (CI) perform poorer than their English counterpart. This may be because present CI speech coding schemes are largely based on English. This study aims to evaluate the relative contributions of temporal envelope (E) cues to Mandarin phoneme (including vowel, and consonant) and lexical tone recognition to provide information for speech coding schemes specific to Mandarin.

Design: Eleven normal hearing subjects were studied using acoustic temporal E cues that were extracted from 30 continuous frequency bands between 80 and 7,562 Hz using the Hilbert transform and divided into five frequency regions. Percent-correct recognition scores were obtained with acoustic E cues presented in three, four, and five frequency regions and their relative weights calculated using the least-square approach.

Results: For stimuli with three, four, and five frequency regions, percent-correct scores for vowel recognition using E cues were 50.43–84.82%, 76.27–95.24%, and 96.58%, respectively; for consonant recognition 35.49–63.77%, 67.75–78.87%, and 87.87%; for lexical tone recognition 60.80–97.15%, 73.16–96.87%, and 96.73%. For frequency region 1 to frequency region 5, the mean weights in vowel recognition were 0.17, 0.31, 0.22, 0.18, and 0.12, respectively; in consonant recognition 0.10, 0.16, 0.18, 0.23, and 0.33; in lexical tone recognition 0.38, 0.18, 0.14, 0.16, and 0.14.

Conclusion: Regions that contributed most for vowel recognition was Region 2 (502–1,022 Hz) that contains first formant (F1) information; Region 5 (3,856–7,562 Hz) contributed most to consonant recognition; Region 1 (80–502 Hz) that contains fundamental frequency (F0) information contributed most to lexical tone recognition.

Bioengineering applications for hearing restoration: emerging biologically inspired and biointegrated designs

Cochlear implantation has become the standard of care for hearing loss not amenable to amplification by bypassing the structures of the cochlea and stimulating the spiral ganglion neurons directly. Since the first single channel electrodes were implanted, significant advancements have been made: multi-channel arrays are now standard, they are softer to avoid damage to the cochlea and pre-curved to better position the electrode array adjacent to the nerve, and surgical and stimulation techniques have helped to conform to the anatomy and physiology of the cochlea. However, even with these advances the experience does not approach that of normal hearing. In order to make significant advances in performance, the next generation of implants will require novel interface technology. Advances in regenerative techniques, optogenetics, piezoelectric materials, and bioengineered living scaffolds hold the promise for the next generation of implantable hearing devices, and hope for the restoration of natural hearing.

Combination and Comparison of Sound Coding Strategies Using Cochlear Implant Simulation With Mandarin Speech

Three cochlear implant (CI) sound coding strategies were combined in the same signal processing path and compared for speech intelligibility with vocoded Mandarin sentences. The three CI coding strategies, biologically-inspired hearing aid algorithm (BioAid), envelope enhancement (EE), and fundamental frequency modulation (F0mod), were combined with the advanced combination encoder (ACE) strategy. Hence, four singular coding strategies and four combinational coding strategies were derived. Mandarin sentences with speech-shape noise were processed using these coding strategies. Speech understanding of vocoded Mandarin sentences was evaluated using short-time objective intelligibility (STOI) and subjective sentence recognition tests with normal-hearing listeners. For signal-to-noise ratios at 5 dB or above, the EE strategy had slightly higher average scores in both STOI and listening tests compared to ACE. The addition of EE to BioAid slightly increased the mean scores for BioAid+EE, which was the combination strategy with the highest scores in both objective and subjective speech intelligibility. The benefits of BioAid, F0mod, and the four combinational coding strategies were not observed in CI simulation. The findings of this study may be useful for the future design of coding strategies and related studies with Mandarin.

Phenomenological model of auditory nerve population responses to cochlear implant stimulation

BACKGROUND

Models of auditory nerve fiber (ANF) responses to electrical stimulation are helpful to develop advanced coding for cochlear implants (CIs). A phenomenological model of ANF population responses to CI electrical stimulation with a lower computational complexity compared to a biophysical model would be beneficial to evaluate new CI coding strategies.

NEW METHOD

This study presents a phenomenological model which combines four temporal characteristics of ANFs (refractoriness, facilitation, accommodation and spike rate adaptation) in addition to a spatial spread of the electric field.

RESULTS

The model predicts the performances of CI subjects in the melodic contour identification (MCI) experiment. The simulations for the MCI experiment were consistent with CI recipients' experimental outcomes that were not predictable from the electrical stimulation patterns themselves.

COMPARISON WITH EXISTING METHODS

Previously, no phenomenological population model of ANFs has combined all four aforementioned temporal phenomena.

CONCLUSIONS

The proposed model would help the further investigations of ANFs responses to different electrical stimulation patterns and comparison of different sound coding strategies in CIs.

A sound coding strategy based on a temporal masking model for cochlear implants

Auditory masking occurs when one sound is perceptually altered by the presence of another sound. Auditory masking in the frequency domain is known as simultaneous masking and in the time domain is known as temporal masking or non-simultaneous masking. This works presents a sound coding strategy that incorporates a temporal masking model to select the most relevant channels for stimulation in a cochlear implant (CI). A previous version of the strategy, termed psychoacoustic advanced combination encoder (PACE), only used a simultaneous masking model for the same purpose, for this reason the new strategy has been termed temporal-PACE (TPACE). We hypothesized that a sound coding strategy that focuses on stimulating the auditory nerve with pulses that are as masked as possible can improve speech intelligibility for CI users. The temporal masking model used within TPACE attenuates the simultaneous masking thresholds estimated by PACE over time. The attenuation is designed to fall exponentially with a strength determined by a single parameter, the temporal masking half-life T½. This parameter gives the time interval at which the simultaneous masking threshold is halved. The study group consisted of 24 postlingually deaf subjects with a minimum of six months experience after CI activation. A crossover design was used to compare four variants of the new temporal masking strategy TPACE (T½ ranging between 0.4 and 1.1 ms) with respect to the clinical MP3000 strategy, a commercial implementation of the PACE strategy, in two prospective, within-subject, repeated-measure experiments. The outcome measure was speech intelligibility in noise at 15 to 5 dB SNR. In two consecutive experiments, the TPACE with T½ of 0.5 ms obtained a speech performance increase of 11% and 10% with respect to the MP3000 (T½ = 0 ms), respectively. The improved speech test scores correlated with the clinical performance of the subjects: CI users with above-average outcome in their routine speech tests showed higher benefit with TPACE. It seems that the consideration of short-acting temporal masking can improve speech intelligibility in CI users. The half-live with the highest average speech perception benefit (0.5 ms) corresponds to time scales that are typical for neuronal refractory behavior.