An evaluation framework for research platforms to advance cochlear implant/hearing aid technology: A case study with CCi-MOBILE

CompHEAR: A Customizable and Scalable Web-Enabled Auditory Performance Evaluation Platform for Cochlear Implant Sound Processing Research

Objective

Cochlear implants (CIs) are auditory prostheses for individuals with severe to profound hearing loss, offering substantial but incomplete restoration of hearing function by stimulating the auditory nerve using electrodes. However, progress in CI performance and innovation has been constrained by the inability to rapidly test multiple sound processing strategies. Current research interfaces provided by major CI manufacturers have limitations in supporting a wide range of auditory experiments due to portability, programming difficulties, and the lack of direct comparison between sound processing algorithms. To address these limitations, we present the CompHEAR research platform, designed specifically for the Cochlear Implant Hackathon, enabling researchers to conduct diverse auditory experiments on a large scale.

Study Design

Quasi-experimental.

Setting

Virtual.

Methods

CompHEAR is an open-source, user-friendly platform which offers flexibility and ease of customization, allowing researchers to set up a broad set of auditory experiments. CompHEAR employs a vocoder to simulate novel sound coding strategies for CIs. It facilitates even distribution of listening tasks among participants and delivers real-time metrics for evaluation. The software architecture underlies the platform's flexibility in experimental design and its wide range of applications in sound processing research.

Results

Performance testing of the CompHEAR platform ensured that it could support at least 10,000 concurrent users. The CompHEAR platform was successfully implemented during the COVID-19 pandemic and enabled global collaboration for the CI Hackathon (www.cihackathon.com).

Conclusion

The CompHEAR platform is a useful research tool that permits comparing diverse signal processing strategies across a variety of auditory tasks with crowdsourced judging. Its versatility, scalability, and ease of use can enable further research with the goal of promoting advancements in cochlear implant performance and improved patient outcomes.

A convolutional neural network-based framework for analysis and assessment of non-linguistic sound classification and enhancement for normal hearing and cochlear implant listeners

Naturalistic sounds encode salient acoustic content that provides situational context or subject/system properties essential for acoustic awareness, autonomy, safety, and improved quality of life for individuals with sensorineural hearing loss. Cochlear implants (CIs) are an assistive hearing device that restores auditory function in hearing impaired individuals. Most CI research advancements have focused on improving speech recognition in noisy, reverberant, or time-varying diverse environments. Relatively few studies have explored non-linguistic sound (NLS) perception among CIs, and those that have carried out such studies generally reported poor perception, suggesting a clear deficit in current CI sound processing systems. In this study, a convolutional neural network (CNN)-based NLS classification model is used as a framework to compare unprocessed and CI-simulated NLS classification and evaluate NLS perception targeted algorithms among CI listeners. Additionally, a NLS enhancement algorithm that focuses on improving identifiability and perception among CI listeners is proposed. The proposed NLS enhancement algorithm is evaluated based on identifiability performance using the CI-simulated NLS classification model. The proposed NLS classification framework was able to achieve near human-level performance with no significant effect of classification modality (model vs human subject) and achieved mean classification scores of 85.86% for NH (p = 0.3758) and 65.25% for CI (p = 0.1725). Among the four different feature-based methods of the proposed NLS enhancement algorithm, the "harmonicity"-based one achieved highest mean classification accuracy of 63.75%, when compared to baseline, and demonstrated significant improvement in performance (p = 0.0403). The resulting proposed comparative NLS classification framework contributes toward (i) advancement of NLS recognition studies, (ii) mitigation of CI user recruitment constraints and listener evaluation with NH listeners, (iii) development of a community shared testbed for comparative NLS studies, and (iv) advancement of NLS enhancement studies (identifiability and perceptual factors) among CI listeners.

Pitch perception with the temporal limits encoder for cochlear implants

The temporal-limits-encoder (TLE) strategy has been proposed to enhance the representation of temporal fine structure (TFS) in cochlear implants (CIs), which is vital for many aspects of sound perception but is typically discarded by most modern CI strategies. TLE works by computing an envelope modulator that is within the temporal pitch limits of CI electric hearing. This paper examines the TFS information encoded by TLE and evaluates the salience and usefulness of this information in CI users. Two experiments were conducted to compare pitch perception performance of TLE versus the widely-used Advanced Combinational Encoder (ACE) strategy. Experiment 1 investigated whether TLE processing improved pitch discrimination compared to ACE. Experiment 2 parametrically examined the effect of changing the lower frequency limit of the TLE modulator on pitch ranking. In both experiments, F0 difference limens were measured with synthetic harmonic complex tones using an adaptive procedure. Signal analysis of the outputs of TLE and ACE strategies showed that TLE introduces important temporal pitch cues that are not available with ACE. Results showed an improvement in pitch discrimination with TLE when the acoustic input had a lower F0 frequency. No significant effect of lower frequency limit was observed for pitch ranking, though a lower limit did tend to provide better outcomes. These results suggest that the envelope modulation introduced by TLE can improve pitch perception for CI listeners.

CCi-MOBILE: A Portable Real Time Speech Processing Platform for Cochlear Implant and Hearing Research

Experimental hardware-research interfaces form a crucial role during developmental stages of any medical, signal-monitoring system as it allows researchers to test and optimize output results before perfecting the design for the actual FDA approved medical device and large-scale production. These testing platforms, intake raw signals through which performance of novel algorithms can be analyzed and modified to generate the desired data points for an optimized output, allowing the advancement of the medical device. With cochlear implants (CIs) and hearing aids (HAs) becoming a more common solution for varying degrees of hearing impairment, having modern signal processing strategies tested for such speech sensitive systems is a necessity. But the rigid design requirements of commercial CI and HA processors make it difficult to explore novel algorithms for research investigations and conducting longitudinal studies. This study presents the design, development, clinical evaluation, and applications of CCi-MOBILE, a computationally powerful signal processing testing platform built for researchers in the hearing-impaired field. The custom-made, portable research platform allows researchers to design and perform complex speech processing algorithm assessment offline and in real-time. It can be operated through user-friendly, open-source software and is compatible with implants manufactured by Cochlear Corporation. The FPGA design and hardware processing pipeline for CI stimulation is discussed followed by results from an acute study with implant users' speech intelligibility in quiet and noisy conditions. The results show a consistent level of performance compared with CI users' clinical processor, thus confirming the viability of the platform in chronic CI based studies.

Effects of design and coupling parameters on the performance of electromagnetic transducers in round-window stimulation

Many studies have investigated factors contributing to large variations in the outcomes of round-window (RW) stimulation but most have focused on the floating mass transducer (FMT). To determine whether results for the FMT hold for a fixed-type transducer (FTT), this study constructs two coupled finite element models of the transducer and the human ear that incorporate the cochlear third windows and inner structures of these two electromagnetic transducers. We use these FE models of the human ear and transducers to investigate the influence of four design parameters and coupling conditions for the transducers, i.e., the support's Young's modulus, the coupling layer's cross sectional area and Young's modulus, and the transducer's cross sectional area. The results show that an increase in the support's Young's modulus reduces the output of the FMT but increases that of the FTT. Reducing the cross sectional area and Young's modulus of the coupling layer significantly increases the low-frequency response of the FMT but slightly reduces that of the FTT. Reducing the cross sectional area of the transducer increases the output of the FMT but reduces that of the FTT. This shows that inner structures of electromagnetic transducers should be considered in the optimal design parameters and coupling conditions for RW stimulation.

Deep Learning-Based Speech Enhancement With a Loss Trading Off the Speech Distortion and the Noise Residue for Cochlear Implants

The cochlea plays a key role in the transmission from acoustic vibration to neural stimulation upon which the brain perceives the sound. A cochlear implant (CI) is an auditory prosthesis to replace the damaged cochlear hair cells to achieve acoustic-to-neural conversion. However, the CI is a very coarse bionic imitation of the normal cochlea. The highly resolved time-frequency-intensity information transmitted by the normal cochlea, which is vital to high-quality auditory perception such as speech perception in challenging environments, cannot be guaranteed by CIs. Although CI recipients with state-of-the-art commercial CI devices achieve good speech perception in quiet backgrounds, they usually suffer from poor speech perception in noisy environments. Therefore, noise suppression or speech enhancement (SE) is one of the most important technologies for CI. In this study, we introduce recent progress in deep learning (DL), mostly neural networks (NN)-based SE front ends to CI, and discuss how the hearing properties of the CI recipients could be utilized to optimize the DL-based SE. In particular, different loss functions are introduced to supervise the NN training, and a set of objective and subjective experiments is presented. Results verify that the CI recipients are more sensitive to the residual noise than the SE-induced speech distortion, which has been common knowledge in CI research. Furthermore, speech reception threshold (SRT) in noise tests demonstrates that the intelligibility of the denoised speech can be significantly improved when the NN is trained with a loss function bias to more noise suppression than that with equal attention on noise residue and speech distortion.