Cortical Speech Processing in Postlingually Deaf Adult Cochlear Implant Users, as Revealed by Functional Near-Infrared Spectroscopy

An experiment was conducted to investigate the feasibility of using functional near-infrared spectroscopy (fNIRS) to image cortical activity in the language areas of cochlear implant (CI) users and to explore the association between the activity and their speech understanding ability. Using fNIRS, 15 experienced CI users and 14 normal-hearing participants were imaged while presented with either visual speech or auditory speech. Brain activation was measured from the prefrontal, temporal, and parietal lobe in both hemispheres, including the language-associated regions. In response to visual speech, the activation levels of CI users in an a priori region of interest (ROI)—the left superior temporal gyrus or sulcus—were negatively correlated with auditory speech understanding. This result suggests that increased cross-modal activity in the auditory cortex is predictive of poor auditory speech understanding. In another two ROIs, in which CI users showed significantly different mean activation levels in response to auditory speech compared with normal-hearing listeners, activation levels were significantly negatively correlated with CI users’ auditory speech understanding. These ROIs were located in the right anterior temporal lobe (including a portion of prefrontal lobe) and the left middle superior temporal lobe. In conclusion, fNIRS successfully revealed activation patterns in CI users associated with their auditory speech understanding.

Assessing hearing by measuring heartbeat: The effect of sound level

Functional near-infrared spectroscopy (fNIRS) is a non-invasive brain imaging technique that measures changes in oxygenated and de-oxygenated hemoglobin concentration and can provide a measure of brain activity. In addition to neural activity, fNIRS signals contain components that can be used to extract physiological information such as cardiac measures. Previous studies have shown changes in cardiac activity in response to different sounds. This study investigated whether cardiac responses collected using fNIRS differ for different loudness of sounds. fNIRS data were collected from 28 normal hearing participants. Cardiac response measures evoked by broadband, amplitude-modulated sounds were extracted for four sound intensities ranging from near-threshold to comfortably loud levels (15, 40, 65 and 90 dB Sound Pressure Level (SPL)). Following onset of the noise stimulus, heart rate initially decreased for sounds of 15 and 40 dB SPL, reaching a significantly lower rate at 15 dB SPL. For sounds at 65 and 90 dB SPL, increases in heart rate were seen. To quantify the timing of significant changes, inter-beat intervals were assessed. For sounds at 40 dB SPL, an immediate significant change in the first two inter-beat intervals following sound onset was found. At other levels, the most significant change appeared later (beats 3 to 5 following sound onset). In conclusion, changes in heart rate were associated with the level of sound with a clear difference in response to near-threshold sounds compared to comfortably loud sounds. These findings may be used alone or in conjunction with other measures such as fNIRS brain activity for evaluation of hearing ability.

Auditory Brainstem Representation of the Voice Pitch Contours in the Resolved and Unresolved Components of Mandarin Tones

Accurate perception of voice pitch plays a vital role in speech understanding, especially for tonal languages such as Mandarin. Lexical tones are primarily distinguished by the fundamental frequency (F0) contour of the acoustic waveform. It has been shown that the auditory system could extract the F0 from the resolved and unresolved harmonics, and the tone identification performance of resolved harmonics was better than unresolved harmonics. To evaluate the neural response to the resolved and unresolved components of Mandarin tones in quiet and in speech-shaped noise, we recorded the frequency-following response. In this study, four types of stimuli were used: speech with either only-resolved harmonics or only-unresolved harmonics, both in quiet and in speech-shaped noise. Frequency-following responses (FFRs) were recorded to alternating-polarity stimuli and were added or subtracted to enhance the neural response to the envelope (FFR_ENV) or fine structure (FFR_TFS), respectively. The neural representation of the F0 strength reflected by the FFR_ENV was evaluated by the peak autocorrelation value in the temporal domain and the peak phase-locking value (PLV) at F0 in the spectral domain. Both evaluation methods showed that the FFR_ENV F0 strength in quiet was significantly stronger than in noise for speech including unresolved harmonics, but not for speech including resolved harmonics. The neural representation of the temporal fine structure reflected by the FFR_TFS was assessed by the PLV at the harmonic near to F1 (4th of F0). The PLV at harmonic near to F1 (4th of F0) of FFR_TFS to resolved harmonics was significantly larger than to unresolved harmonics. Spearman's correlation showed that the FFR_ENV F0 strength to unresolved harmonics was correlated with tone identification performance in noise (0 dB SNR). These results showed that the FFR_ENV F0 strength to speech sounds with resolved harmonics was not affected by noise. In contrast, the response to speech sounds with unresolved harmonics, which were significantly smaller in noise compared to quiet. Our results suggest that coding resolved harmonics was more important than coding envelope for tone identification performance in noise.

Temporal Coding of Voice Pitch Contours in Mandarin Tones

Accurate perception of time-variant pitch is important for speech recognition, particularly for tonal languages with different lexical tones such as Mandarin, in which different tones convey different semantic information. Previous studies reported that the auditory nerve and cochlear nucleus can encode different pitches through phase-locked neural activities. However, little is known about how the inferior colliculus (IC) encodes the time-variant periodicity pitch of natural speech. In this study, the Mandarin syllable /ba/ pronounced with four lexical tones (flat, rising, falling then rising and falling) were used as stimuli. Local field potentials (LFPs) and single neuron activity were simultaneously recorded from 90 sites within contralateral IC of six urethane-anesthetized and decerebrate guinea pigs in response to the four stimuli. Analysis of the temporal information of LFPs showed that 93% of the LFPs exhibited robust encoding of periodicity pitch. Pitch strength of LFPs derived from the autocorrelogram was significantly (p < 0.001) stronger for rising tones than flat and falling tones. Pitch strength are also significantly increased (p < 0.05) with the characteristic frequency (CF). On the other hand, only 47% (42 or 90) of single neuron activities were significantly synchronized to the fundamental frequency of the stimulus suggesting that the temporal spiking pattern of single IC neuron could encode the time variant periodicity pitch of speech robustly. The difference between the number of LFPs and single neurons that encode the time-variant F0 voice pitch supports the notion of a transition at the level of IC from direct temporal coding in the spike trains of individual neurons to other form of neural representation.

Intensity Discrimination and Speech Recognition of Cochlear Implant Users

The relation between speech recognition and within-channel or across-channel (i.e., spectral tilt) intensity discrimination was measured in nine CI users (11 ears). Within-channel intensity difference limens (IDLs) were measured at four electrode locations across the electrode array. Spectral tilt difference limens were measured with (XIDL-J) and without (XIDL) level jitter. Only three subjects could perform the XIDL-J task with the amount of jitter required to limit use of within-channel cues. XIDLs (normalized to %DR) were correlated with speech recognition (r = 0.67, P = 0.019) and were highly correlated with IDLs. XIDLs were on average nearly 3 times larger than IDLs and did not vary consistently with the spatial separation of the two component electrodes. The overall pattern of results was consistent with a common underlying subject-dependent limitation in the two difference limen tasks, hypothesized to be perceptual variance (how the perception of a sound differs on different presentations), which may also underlie the correlation of XIDLs with speech recognition. Evidence that spectral tilt discrimination is more important for speech recognition than within-channel intensity discrimination was not unequivocally shown in this study. However, the results tended to support this proposition, with XIDLs more correlated with speech performance than IDLs, and the ratio XIDL/IDL also being correlated with speech recognition. If supported by further research, the importance of perceptual variance as a limiting factor in speech understanding for CI users has important implications for efforts to improve outcomes for those with poor speech recognition.

Brain Plasticity and Rehabilitation with a Cochlear Implant

The functional changes that occur in the brain due to deafness may affect the way the auditory system processes sound after cochlear implantation. Brain plasticity plays a crucial role in the success of cochlear implantation to facilitate or develop spoken language in profoundly deaf individuals. The functional plasticity that occurs in postlingually deaf adults during periods of deafness can both support and hinder speech understanding with a cochlear implant, depending on the nature and degree of functional changes. Evidence so far suggests that the strategies people use to communicate while deaf may influence whether the functional changes are adaptive or maladaptive. In the case of children with congenital deafness, evidence is very strong for a sensitive period in which auditory input must be restored if subsequent oral language is to be developed successfully. Successful oral language use and speech understanding in individuals implanted after 7 years of age depends strongly on the pre-implant use of hearing aids and auditory-verbal communication. Future research should focus on how to harness our growing knowledge of brain plasticity to optimize the outcomes of cochlear implantation in each individual.

The effect of presentation level and stimulation rate on speech perception and modulation detection for cochlear implant users

In order to improve speech understanding for cochlear implant users, it is important to maximize the transmission of temporal information. The combined effects of stimulation rate and presentation level on temporal information transfer and speech understanding remain unclear. The present study systematically varied presentation level (60, 50, and 40 dBA) and stimulation rate [500 and 2400 pulses per second per electrode (pps)] in order to observe how the effect of rate on speech understanding changes for different presentation levels. Speech recognition in quiet and noise, and acoustic amplitude modulation detection thresholds (AMDTs) were measured with acoustic stimuli presented to speech processors via direct audio input (DAI). With the 500 pps processor, results showed significantly better performance for consonant-vowel nucleus-consonant words in quiet, and a reduced effect of noise on sentence recognition. However, no rate or level effect was found for AMDTs, perhaps partly because of amplitude compression in the sound processor. AMDTs were found to be strongly correlated with the effect of noise on sentence perception at low levels. These results indicate that AMDTs, at least when measured with the CP910 Freedom speech processor via DAI, explain between-subject variance of speech understanding, but do not explain within-subject variance for different rates and levels.

Perceptual Spaces Induced by Cochlear Implant All-Polar Stimulation Mode

It has been argued that a main limitation of the cochlear implant is the spread of current induced by each electrode, which activates an inappropriately large range of sensory neurons. To reduce this spread, an alternative stimulation mode, the all-polar mode, was tested with five participants. It was designed to activate all the electrodes simultaneously with appropriate current levels and polarities to recruit narrower regions of auditory nerves at specific intracochlear electrode positions (denoted all-polar electrodes). In this study, the all-polar mode was compared with the current commercial stimulation mode: the monopolar mode. The participants were asked to judge the sound dissimilarity between pairs of two-electrode pulse-train stimuli that differed in the electrode positions and were presented in either monopolar or all-polar mode with pulses on the two electrodes presented either sequentially or simultaneously. The dissimilarity ratings were analyzed using a multidimensional scaling technique and three-dimensional stimulus perceptual spaces were produced. For all the conditions (mode and simultaneity), the first perceptual dimension was highly correlated with the position of the most apical activated electrode of the electrical stimulation and the second dimension with the position of the most basal electrode. In both sequential and simultaneous conditions, the monopolar and all-polar stimuli were significantly separated by a third dimension, which may indicate that all-polar stimuli have a perceptual quality that differs from monopolar stimuli. Overall, the results suggest that both modes might successfully represent spectral information in a sound processing strategy.

Clinical validation of a precision electromagnetic tremor measurement system in participants receiving deep brain stimulation for essential tremor

Tremor is characterized commonly through subjective clinical rating scales. Accelerometer-based techniques for objective tremor measurement have been developed in the past, yet these measures are usually presented as an unintuitive dimensionless index without measurement units. Here we have developed a tool (TREMBAL) to provide quantifiable and objective measures of tremor severity using electromagnetic motion tracking. We aimed to compare TREMBAL's objective measures with clinical tremor ratings and determine the test-retest reliability of our technique. Eight participants with ET receiving deep brain stimulation (DBS) therapy were consented. Tremor was simultaneously recorded using TREMBAL and video during DBS adjustment. After each adjustment, participants performed a hands-outstretched task (for postural tremor) and a finger-nose task (for kinetic tremor). Video recordings were de-identified, randomized, and shown to a panel of movement disorder specialists to obtain their ratings. Regression analysis and Pearson's correlations were used to determine agreement between datasets. Subsets of the trial were repeated to assess test-retest reliability. Tremor amplitude and velocity measures were in close agreement with mean clinical ratings (r  >  0.90) for both postural and kinetic tremors. Test-retest reliability for both translational and rotational components of tremor showed intra-class correlations  >0.80. TREMBAL assessments showed that tremor gradually improved with increasing DBS therapy-this was also supported by clinical observation. TREMBAL measurements are a sensitive, objective and reliable assessment of tremor severity. This tool may have application in clinical trials and in aiding automated optimization of deep brain stimulation.

Electrode Selection and Speech Understanding in Patients With Auditory Brainstem Implants

OBJECTIVES

The objective of this study was to evaluate whether speech understanding in auditory brainstem implant (ABI) users who have a tumor pathology could be improved by the selection of a subset of electrodes that were appropriately pitch ranked and distinguishable. It was hypothesized that disordered pitch or spectral percepts and channel interactions may contribute significantly to the poor outcomes in most ABI users.

DESIGN

A single-subject design was used with five participants. Pitch ranking information for all electrodes in the patients' clinic maps was obtained using a pitch ranking task and previous pitch ranking information from clinic sessions. A multidimensional scaling task was used to evaluate the stimulus space evoked by stimuli on the same set of electrodes. From this information, a subset of four to six electrodes was chosen and a new map was created, using just this subset, that the subjects took home for 1 month's experience. Closed-set consonant and vowel perception and sentences in quiet were tested at three sessions: with the clinic map before the test map was given, after 1 month with the test map, and after an additional 2 weeks with their clinic map.

RESULTS

The results of the pitch ranking and multidimensional scaling procedures confirmed that the ABI users did not have a well-ordered set of percepts related to electrode position, thus supporting the proposal that difficulty in processing of spectral information may contribute to poor speech understanding. However, none of the subjects benefited from a map that reduced the stimulation electrode set to a smaller number of electrodes that were well ordered in place pitch.

CONCLUSIONS

Although poor spectral processing may contribute to poor understanding in ABI users, it is not likely to be the sole contributor to poor outcomes.

The perception of complex pitch in cochlear implants: A comparison of monopolar and tripolar stimulation

Cochlear implant listeners typically perform poorly in tasks of complex pitch perception (e.g., musical pitch and voice pitch). One explanation is that wide current spread during implant activation creates channel interactions that may interfere with perception of temporal fundamental frequency information contained in the amplitude modulations within channels. Current focusing using a tripolar mode of stimulation has been proposed as a way of reducing channel interactions, minimising spread of excitation and potentially improving place and temporal pitch cues. The present study evaluated the effect of mode in a group of cochlear implant listeners on a pitch ranking task using male and female singing voices separated by either a half or a quarter octave. Results were variable across participants, but on average, pitch ranking was at chance level when the pitches were a quarter octave apart and improved when the difference was a half octave. No advantage was observed for tripolar over monopolar mode at either pitch interval, suggesting that previously published psychophysical advantages for focused modes may not translate into improvements in complex pitch ranking. Evaluation of the spectral centroid of the stimulation pattern, plus a lack of significant difference between male and female voices, suggested that participants may have had difficulty in accessing temporal pitch cues in either mode.

The Feasibility of Using Acoustic Markers of Speech for Optimizing Patient Outcomes during Randomized Amplitude Variation in Deep Brain Stimulation: A Proof of Principle Methods Study

Background

Deep brain stimulation (DBS) is an effective treatment for reducing symptoms of tremor. A common and typically subjectively determined adverse effect of DBS is dysarthria. Current assessment protocols are driven by the qualitative judgments of treating clinicians and lack the sensitivity and objectivity required to optimize patient outcomes where multiple stimulation parameters are trialed.

Objective

To examine the effect of DBS on speech in patients receiving stimulation to the posterior sub-thalamic area (PSA) via randomized manipulation of amplitude parameters.

Methods

Six patients diagnosed with tremor receiving treatment via DBS of the PSA were assessed in a double-blinded, within-subjects experimental protocol. Amplitude (i.e., voltage or current) was randomly adjusted across 10 settings, while speech samples (e.g., sustained vowel, counting to 10) were recorded to identify the patient-specific settings required for optimal therapeutic benefit (reduced tremor) with minimal adverse effects (altered speech). Speech production between stimulation parameters was quantified using acoustic analysis.

Results

Speech changed as a response to DBS but those changes were not uniform across patients nor were they generally in line with changes in amplitude with the exception of reduced vocal control and increased mean silence length in two patients. Speech outcomes did not correlate with changes in tremor.

Conclusion

Intra-individual changes in speech were detected as a response to modified amplitude; however, no clear pattern was observed across patients as a group. The use of objective acoustic measures allows for quantification of speech changes during DBS optimization protocols, even when those changes are subtle and potentially difficult to detect perceptually.

Effect of Pulse Rate and Polarity on the Sensitivity of Auditory Brainstem and Cochlear Implant Users to Electrical Stimulation

To further understand the response of the human brainstem to electrical stimulation, a series of experiments compared the effect of pulse rate and polarity on detection thresholds between auditory brainstem implant (ABI) and cochlear implant (CI) patients. Experiment 1 showed that for 400-ms pulse trains, ABI users’ thresholds dropped by about 2 dB as pulse rate was increased from 71 to 500 pps, but only by an average of 0.6 dB as rate was increased further to 3500 pps. This latter decrease was much smaller than the 7.7-dB observed for CI users. A similar result was obtained for pulse trains with a 40-ms duration. Furthermore, experiment 2 showed that the threshold difference between 500- and 3500-pps pulse trains remained much smaller for ABI than for CI users, even for durations as short as 2 ms, indicating the effect of a fast-acting mechanism. Experiment 3 showed that ABI users’ thresholds were lower for alternating-polarity than for fixed-polarity pulse trains, and that this difference was greater at 3500 pps than at 500 pps, consistent with the effect of pulse rate on ABI users’ thresholds being influenced by charge interactions between successive biphasic pulses. Experiment 4 compared thresholds and loudness between trains of asymmetric pulses of opposite polarity, in monopolar mode, and showed that in both cases less current was needed when the anodic, rather than the cathodic, current was concentrated into a short time interval. This finding is similar to that previously observed for CI users and is consistent with ABI users being more sensitive to anodic than cathodic current. We argue that our results constrain potential explanations for the differences in the perception of electrical stimulation by CI and ABI users, and have potential implications for future ABI stimulation strategies.

Loudness of time-varying stimuli with electric stimulation

McKay, Henshall, Farrell, and McDermott [J. Acoust. Soc. Am. 113, 2054-2063 (2003)] developed a practical method to estimate the loudness of periodic electrical signals presented through a cochlear implant. In the present work, this method was extended to time-varying sounds based on two models of time-varying loudness for normal listeners. To fit the model parameters, loudness balancing data was collected with six cochlear implant listeners. The pulse rate of a modulated pulse train was adjusted to equalize its loudness to a reference stimulus. The stimuli were single-electrode time-limited pulse bursts, repeated at a rate of 50 Hz, with on-times varying between 2 and 20 ms. The parameters of two different models of time-varying loudness were fitted to the results. For each model, parameters defining the time windows over which the electrical pulses contribute to instantaneous loudness were optimized. In each case, a good fit was obtained with the loudness balancing results. Therefore, the practical method was successfully extended to time-varying sounds by combining it with existing models of time-varying loudness for acoustic stimulation.

Interpulse interval discrimination within and across channels: comparison of monopolar and tripolar mode of stimulation

Perception of temporal patterns is crucial to speech understanding and music perception in normal hearing, and is fundamental in the design and implementation of processing strategies for cochlear implants. Two experiments described here investigated the effect of stimulation mode (monopolar versus tripolar) on interpulse interval discrimination using single-electrode stimulation (experiment 1) and dual-electrode stimulation (experiment 2). Experiment 1 required participants to discriminate stimuli containing different interpulse intervals and experiment 2 required listeners to discriminate between two dual-electrode stimuli that had the same temporal pattern on each electrode, but differed in inter-electrode timing. The hypotheses were that (i) stimulation mode would affect the ability to distinguish interpulse interval patterns on a single electrode and (ii) the electrode separation range in which subjects were sensitive to inter-electrode timing would be more restricted in tripolar than in monopolar stimulation. Results in nine cochlear implant users showed that mode did not have a significant mean effect on either the ability to discriminate interpulse intervals in single-electrode stimulation or the range of electrode separation in dual-electrode stimulation in which participants were sensitive to inter-electrode timing. In conclusion, tripolar stimulation did not show any advantage in delivering temporal information within or across channels in this group.

Estimating confidence intervals for information transfer analysis of confusion matrices

A non-parametric bootstrapping statistical method is introduced and investigated for estimating confidence intervals resulting from information transfer (IT) analysis of confusion matrices. Confidence intervals can be used to statistically compare ITs from two or more confusion matrices obtained in an experiment. Information transfer is a nonlinear analysis and does not satisfy many of the assumptions of a parametric method. The bootstrapping method accurately estimated IT confidence intervals as long as the confusion matrices contained a sufficiently large number of presentations per stimulus category, which is also a condition for reduced bias in IT analysis.

Auditory responses to electric and infrared neural stimulation of the rat cochlear nucleus

In an effort to improve the auditory brainstem implant, a prosthesis in which user outcomes are modest, we applied electric and infrared neural stimulation (INS) to the cochlear nucleus in a rat animal model. Electric stimulation evoked regions of neural activation in the inferior colliculus and short-latency, multipeaked auditory brainstem responses (ABRs). Pulsed INS, delivered to the surface of the cochlear nucleus via an optical fiber, evoked broad neural activation in the inferior colliculus. Strongest responses were recorded when the fiber was placed at lateral positions on the cochlear nucleus, close to the temporal bone. INS-evoked ABRs were multipeaked but longer in latency than those for electric stimulation; they resembled the responses to acoustic stimulation. After deafening, responses to electric stimulation persisted, whereas those to INS disappeared, consistent with a reported "optophonic" effect, a laser-induced acoustic artifact. Thus, for deaf individuals who use the auditory brainstem implant, INS alone did not appear promising as a new approach.

Place specificity measured in forward and interleaved masking in cochlear implants

Interleaved masking in cochlear implants is analogous to acoustic simultaneous masking and is relevant to speech processing strategies that interleave pulses on concurrently activated electrodes. In this study, spatial decay of masking as the distance between masker and probe increases was compared between forward and interleaved masking in the same group of cochlear implant users. Spatial masking patterns and the measures of place specificity were similar between forward and interleaved masking. Unlike acoustic hearing where broader tuning curves are obtained in simultaneous masking, the type of masking experiment did not influence the measure of place specificity in cochlear implants.

Place specificity of monopolar and tripolar stimuli in cochlear implants: the influence of residual masking

This experiment investigated whether place specificity of neural activity evoked by cochlear implant stimulation is improved in tripolar compared to monopolar mode using a forward masking protocol addressing some limitations of previous methods of measurement and analysis. The amount of residual masking (masking remaining at long masker-probe delays) was also measured, and its potential influence on the specificity measures was evaluated. The masker stimulus comprised equally loud interleaved mono- or tripolar stimulation on two electrodes equidistant from a central probe electrode in an apical and basal direction, reducing the influence of off-site listening. The effect of masker-probe distance on the threshold shift of the tripolar probe was analyzed to derive a measure of place specificity. On average, tripolar maskers were more place specific than monopolar maskers, although the mean effect was small. There was no significant effect of masker level on specificity or on the differences observed between modes. The mean influence of residual masking on normalized masking functions was similar for the two modes and, therefore, did not influence the comparison of specificity between the modes. However, variability in amount of residual masking was observed between subjects, and therefore should be considered in forward masking studies that compare place specificity across subjects.

Electrically evoked compound action potential artifact rejection by independent component analysis: technique validation

The electrically-evoked compound action potential (ECAP) is the synchronous whole auditory nerve activity in response to an electrical stimulus, and can be recorded in situ on cochlear implant (CI) electrodes. A novel procedure (ECAP-ICA) to isolate the ECAP from the stimulation artifact, based on independent component analysis (ICA), is described here. ECAPs with artifact (raw-ECAPs) were sequentially recorded for the same stimulus on 9 different intracochlear recording electrodes. The raw-ECAPs were fed to ICA, which separated them into independent sources. Restricting the ICA projection to 4 independent components did not induce under-fitting and was found to explain most of the raw-data variance. The sources were identified and only the source corresponding to the neural response was retained for artifact-free ECAP reconstruction. The validity of the ECAP-ICA procedure was supported as follows: N₁ and P₁ peaks occurred at usual latencies; and ECAP-ICA and artifact amplitude-growth functions (AGFs) had different slopes. Concatenation of raw-ECAPs from multiple stimulus currents, including some below the ECAP-ICA threshold, improved the source separation process. The main advantage of ECAP-ICA is that use of maskers or alternating polarity stimulation are not needed.

•

Novel technique to cancel artifact in electrically-evoked compound action potentials.

•

No maskers or alternate polarity are required.

•

ECAPs output from the procedure are compared to forward-masking technique.