Application of a pitch perception model to investigate the effect of stimulation field spread on the pitch ranking abilities of cochlear implant recipients

Estimating Pitch Information From Simulated Cochlear Implant Signals With Deep Neural Networks

Cochlear implant (CI) users, even with substantial speech comprehension, generally have poor sensitivity to pitch information (or fundamental frequency, F0). This insensitivity is often attributed to limited spectral and temporal resolution in the CI signals. However, the pitch sensitivity markedly varies among individuals, and some users exhibit fairly good sensitivity. This indicates that the CI signal contains sufficient information about F0, and users' sensitivity is predominantly limited by other physiological conditions such as neuroplasticity or neural health. We estimated the upper limit of F0 information that a CI signal can convey by decoding F0 from simulated CI signals (multi-channel pulsatile signals) with a deep neural network model (referred to as the CI model). We varied the number of electrode channels and the pulse rate, which should respectively affect spectral and temporal resolutions of stimulus representations. The F0-estimation performance generally improved with increasing number of channels and pulse rate. For the sounds presented under quiet conditions, the model performance was at best comparable to that of a control waveform model, which received raw-waveform inputs. Under conditions in which background noise was imposed, the performance of the CI model generally degraded by a greater degree than that of the waveform model. The pulse rate had a particularly large effect on predicted performance. These observations indicate that the CI signal contains some information for predicting F0, which is particularly sufficient for targets under quiet conditions. The temporal resolution (represented as pulse rate) plays a critical role in pitch representation under noisy conditions.

Unifying information theory and machine learning in a model of electrode discrimination in cochlear implants

Despite the development and success of cochlear implants over several decades, wide inter-subject variability in speech perception is reported. This suggests that cochlear implant user-dependent factors limit speech perception at the individual level. Clinical studies have demonstrated the importance of the number, placement, and insertion depths of electrodes on speech recognition abilities. However, these do not account for all inter-subject variability and to what extent these factors affect speech recognition abilities has not been studied. In this paper, an information theoretic method and machine learning technique are unified in a model to investigate the extent to which key factors limit cochlear implant electrode discrimination. The framework uses a neural network classifier to predict which electrode is stimulated for a given simulated activation pattern of the auditory nerve, and mutual information is then estimated between the actual stimulated electrode and predicted ones. We also investigate how and to what extent the choices of parameters affect the performance of the model. The advantages of this framework include i) electrode discrimination ability is quantified using information theory, ii) it provides a flexible framework that may be used to investigate the key factors that limit the performance of cochlear implant users, and iii) it provides insights for future modeling studies of other types of neural prostheses.

An integrated model of pitch perception incorporating place and temporal pitch codes with application to cochlear implant research

Although the neural mechanisms underlying pitch perception are not yet fully understood, there is general agreement that place and temporal representations of pitch are both used by the auditory system. This paper describes a neural network model of pitch perception that integrates both codes of pitch and explores the contributions of, and the interactions between, the two representations in simulated pitch ranking trials in normal and cochlear implant hearing. The model can replicate various psychophysical observations including the perception of the missing fundamental pitch and sensitivity to pitch interval sizes. As a case study, the model was used to investigate the efficiency of pitch perception cues in a novel sound processing scheme, Stimulation based on Auditory Modelling (SAM), that aims to improve pitch perception in cochlear implant hearing. Results showed that enhancement of the pitch perception cues would lead to better pitch ranking scores in the integrated model only if the place and temporal pitch cues were consistent.

Learning Pitch with STDP: A Computational Model of Place and Temporal Pitch Perception Using Spiking Neural Networks

Pitch perception is important for understanding speech prosody, music perception, recognizing tones in tonal languages, and perceiving speech in noisy environments. The two principal pitch perception theories consider the place of maximum neural excitation along the auditory nerve and the temporal pattern of the auditory neurons’ action potentials (spikes) as pitch cues. This paper describes a biophysical mechanism by which fine-structure temporal information can be extracted from the spikes generated at the auditory periphery. Deriving meaningful pitch-related information from spike times requires neural structures specialized in capturing synchronous or correlated activity from amongst neural events. The emergence of such pitch-processing neural mechanisms is described through a computational model of auditory processing. Simulation results show that a correlation-based, unsupervised, spike-based form of Hebbian learning can explain the development of neural structures required for recognizing the pitch of simple and complex tones, with or without the fundamental frequency. The temporal code is robust to variations in the spectral shape of the signal and thus can explain the phenomenon of pitch constancy.

Pitch is the perceptual correlate of sound frequency. Our auditory system has a sophisticated mechanism to process and perceive the neural information corresponding to pitch. This mechanism employs both the place and the temporal pattern of pitch-evoked neural events. Based on the known functions of the auditory system, we develop a computational model of pitch perception using a network of neurons with modifiable connections. We demonstrate that a well-known neural learning rule that is based on the timing of the neural events can identify and strengthen the neuronal connections that are most effective for the extraction of pitch. By providing an insight into how the auditory system interprets pitch information, the results of our study can be used to develop improved sound processing strategies for cochlear implants. In cochlear implant hearing, auditory percept is generated by stimulating the auditory neurons with controlled electrical impulses, enhancing which with the help of the model would lead to a better representation of pitch and would subsequently improve music perception and speech understanding in noisy environments in cochlear implant users.

The history and future of neural modeling for cochlear implants

This special issue of Network: Computation in Neural Systems on the topic of "Computational models of the electrically stimulated auditory system" incorporates review articles spanning a wide range of approaches to modeling cochlear implant stimulation of the auditory system. The purpose of this overview paper is to provide a historical context for the different modeling endeavors and to point toward how computational modeling could play a key role in the understanding, evaluation, and improvement of cochlear implants in the future.