Recognition of vocoded words and sentences in quiet and multi-talker babble with children and adults

A dual-modality study on the neural features of cochlear implant simulated tone and consonant perception

Accurately perceiving lexical tones and consonants is critical for understanding speech in tonal languages. Cochlear implant (CI) users exhibit reduced phonetic perception due to spectral loss in CI encoding, yet the underlying neural mechanisms remain unclear. This study combined electroencephalography and functional near-infrared spectroscopy (fNIRS) to investigate the neural processing mechanisms of CI-simulated channelized speech in 26 normal-hearing adults during the processing of tones (T1-T4) and consonants ("ba," "da," "ga," "za"). Results showed that the N1 amplitude in auditory evoked potentials was significantly lower for channelized speech than a natural human voice (NH), particularly for T2 and T4 tones, indicating a weaker perception of channelized speech. Functional connectivity analysis revealed that an NH exhibited significantly higher synchrony in the δ and θ frequency bands than channelized speech, which was more pronounced in the right temporal lobe. This finding was also observed with "za" consonants. fNIRS results showed stronger right temporal lobe activation for channelized speech, suggesting that the brain requires greater auditory effort to process channelized speech. Combining both modalities revealed neural compensatory mechanisms underlying channelized speech-manifesting as "low-efficiency perception with high cognitive load." This study provides potential biomarkers for CI rehabilitation assessment and a foundation for future research.

Right ear advantage in cochlear implant simulation: short- and long-term effects

OBJECTIVES

This study investigates the short-term effects of degraded auditory input on the right ear advantage (REA) and the REA following long-term exposure to vocoder-processed sounds, which simulate cochlear implant (CI) hearing. Vocoder processing allows normal hearing individuals to experience CI-like hearing conditions, enabling an exploration of how modifications to auditory input influence the REA.

DESIGN

A repeated-measures design was employed. Twenty-two normal-hearing participants completed dichotic word recognition tests under three auditory conditions: bilateral normal hearing, short-term vocoder-processed hearing, and long-term vocoder-processing hearing. REA was assessed after one month of training with vocoder-processed words to simulate long-term exposure.

STUDY SAMPLE

The study included 22 normal-hearing participants aged 19-28 years. All participants had normal hearing and no history of auditory or neurological disorders.

RESULTS

REA significantly decreased under the short-term vocoder condition compared to the normal hearing condition (p < 0.001). However, after long-term training, REA significantly improved (p < 0.001), and this improvement approached normal hearing levels (p = 0.28).

CONCLUSION

Our findings suggest that short-term exposure to vocoder-processed auditory input disrupts the REA, but extended training can restore it. These results provide insights into cortical plasticity and its role in auditory adaptation, with potential implications for developing rehabilitation strategies for CI users.

Employing deep learning model to evaluate speech information in acoustic simulations of Cochlear implants

Acoustic vocoders play a key role in simulating the speech information available to cochlear implant (CI) users. Traditionally, the intelligibility of vocoder CI simulations is assessed through speech recognition experiments with normally-hearing subjects, a process that can be time-consuming, costly, and subject to individual variability. As an alternative approach, we utilized an advanced deep learning speech recognition model to investigate the intelligibility of CI simulations. We evaluated model's performance on vocoder-processed words and sentences with varying vocoder parameters. The number of vocoder bands, frequency range, and envelope dynamic range were adjusted to simulate sound processing settings in CI devices. Additionally, we manipulated the low-cutoff frequency and intensity quantization of vocoder envelopes to simulate psychophysical temporal and intensity resolutions in CI patients. The results were evaluated within the context of the audio analysis performed in the model. Interestingly, the deep learning model, despite not being originally designed to mimic human speech processing, exhibited a human-like response to alterations in vocoder parameters, resembling existing human subject results. This approach offers significant time and cost savings compared to testing human subjects, and eliminates learning and fatigue effects during testing. Our findings demonstrate the potential of speech recognition models in facilitating auditory research.

Phonological discrimination and contrast detection in pupillometry

Introduction

The perception of phonemes is guided by both low-level acoustic cues and high-level linguistic context. However, differentiating between these two types of processing can be challenging. In this study, we explore the utility of pupillometry as a tool to investigate both low- and high-level processing of phonological stimuli, with a particular focus on its ability to capture novelty detection and cognitive processing during speech perception.

Methods

Pupillometric traces were recorded from a sample of 22 Danish-speaking adults, with self-reported normal hearing, while performing two phonological-contrast perception tasks: a nonword discrimination task, which included minimal-pair combinations specific to the Danish language, and a nonword detection task involving the detection of phonologically modified words within sentences. The study explored the perception of contrasts in both unprocessed speech and degraded speech input, processed with a vocoder.

Results

No difference in peak pupil dilation was observed when the contrast occurred between two isolated nonwords in the nonword discrimination task. For unprocessed speech, higher peak pupil dilations were measured when phonologically modified words were detected within a sentence compared to sentences without the nonwords. For vocoded speech, higher peak pupil dilation was observed for sentence stimuli, but not for the isolated nonwords, although performance decreased similarly for both tasks.

Conclusion

Our findings demonstrate the complexity of pupil dynamics in the presence of acoustic and phonological manipulation. Pupil responses seemed to reflect higher-level cognitive and lexical processing related to phonological perception rather than low-level perception of acoustic cues. However, the incorporation of multiple talkers in the stimuli, coupled with the relatively low task complexity, may have affected the pupil dilation.

Employing Deep Learning Model to Evaluate Speech Information in Acoustic Simulations of Auditory Implants

Acoustic simulations have played a prominent role in the development of speech processing and sound coding strategies for auditory neural implant devices. Traditionally evaluated using human subjects, acoustic simulations have been used to model the impact of implant signal processing as well as individual anatomy/physiology on speech perception. However, human subject testing is time-consuming, costly, and subject to individual variability. In this study, we propose a novel approach to perform simulations of auditory implants. Rather than using actual human participants, we utilized an advanced deep-learning speech recognition model to simulate the effects of some important signal processing as well as psychophysical/physiological factors on speech perception. Several simulation conditions were produced by varying number of spectral bands, input frequency range, envelope cut-off frequency, envelope dynamic range and envelope quantization. Our results demonstrate that the deep-learning model exhibits human-like robustness to simulation parameters in quiet and noise, closely resembling existing human subject results. This approach is not only significantly quicker and less expensive than traditional human studies, but it also eliminates individual human variables such as attention and learning. Our findings pave the way for efficient and accurate evaluation of auditory implant simulations, aiding the future development of auditory neural prosthesis technologies.

Employing Deep Learning Model to Evaluate Speech Information in Vocoder Simulations of Auditory Implants

Vocoder simulations have played a crucial role in the development of sound coding and speech processing techniques for auditory implant devices. Vocoders have been extensively used to model the effects of implant signal processing as well as individual anatomy and physiology on speech perception of implant users. Traditionally, such simulations have been conducted on human subjects, which can be time-consuming and costly. In addition, perception of vocoded speech varies significantly across individual subjects, and can be significantly affected by small amounts of familiarization or exposure to vocoded sounds. In this study, we propose a novel method that differs from traditional vocoder studies. Rather than using actual human participants, we use a speech recognition model to examine the influence of vocoder-simulated cochlear implant processing on speech perception. We used the OpenAI Whisper, a recently developed advanced open-source deep learning speech recognition model. The Whisper model's performance was evaluated on vocoded words and sentences in both quiet and noisy conditions with respect to several vocoder parameters such as number of spectral bands, input frequency range, envelope cut-off frequency, envelope dynamic range, and number of discriminable envelope steps. Our results indicate that the Whisper model exhibited human-like robustness to vocoder simulations, with performance closely mirroring that of human subjects in response to modifications in vocoder parameters. Furthermore, this proposed method has the advantage of being far less expensive and quicker than traditional human studies, while also being free from inter-individual variability in learning abilities, cognitive factors, and attentional states. Our study demonstrates the potential of employing advanced deep learning models of speech recognition in auditory prosthesis research.