An exploratory Study of EEG Alpha Oscillation and Pupil Dilation in Hearing-Aid Users During Effortful listening to Continuous Speech

An adaptive protocol to assess physiological responses as a function of task demand in speech-in-noise testing

BACKGROUND

Acoustic challenges impose demands on cognitive resources, known as listening effort (LE), which can substantially influence speech perception and communication. Standardized assessment protocols for monitoring LE are lacking, hindering the development of adaptive hearing assistive technology.

NEW METHOD

We employed an adaptive protocol, including a speech-in-noise test and personalized definition of task demand, to assess LE and its physiological correlates. Features extracted from electroencephalogram, galvanic skin response, electrocardiogram, respiration, pupil dilation, and blood volume pulse responses were analyzed as a function of task demand in 21 healthy participants with normal hearing.

RESULTS

Heightened sympathetic response was observed with higher task demand, evidenced by increased heart rate, blood pressure, and breath amplitude. Blood volume amplitude and breath amplitude exhibited higher sensitivity to changes in task demand.

COMPARISON WITH EXISTING METHODS

Notably, galvanic skin response showed higher amplitude during low task demand phases, indicating increased attention and engagement, aligning with findings from electroencephalogram signals and Lacey's attention theory.

CONCLUSIONS

The analysis of a range of physiological signals, spanning cardiovascular, central, and autonomic domains, demonstrated effectiveness in comprehensively examining LE. Future research should explore additional levels and manipulations of task demand, as well as the influence of individual motivation and hearing sensitivity, to further validate these outcomes and enhance the development of adaptive hearing assistive technology.

Investigation of Listening Effort in Tinnitus Patients by Providing Similar Peripheral Auditory Function With Control Group

INTRODUCTION

Previous studies have investigated listening effort in tinnitus patients compared to healthy individuals. These studies reported similar pure tone hearing thresholds between groups but did not investigate possible peripheral auditory dysfunction, which could affect the central auditory system and increase listening effort even when hearing thresholds are within the normal hearing range. This study aimed to investigate the presence of listening effort in tinnitus patients by controlling for peripheral auditory function (PAF).

METHODS

This study included 16 chronic tinnitus patients and 23 matched healthy controls, both with normal hearing thresholds. The subjects were assessed using 0.125-20 kHz pure-tone audiometry, a visual analogue scale (VAS), the Montreal Cognitive Assessment (MoCA), the Tinnitus Handicap Inventory (THI), the matrix test, auditory brainstem response (ABR), and electroencephalography (EEG). EEG alpha band activity was recorded from parietal electrodes (P3, P4, Pz).

RESULTS

The increase in alpha band power during the encoding phase of sentence presentation in tinnitus patients was less than that in the control group. We found higher VAS scores in tinnitus participants. We did not find significant differences in matrix test scores, ABR amplitude, or absolute latency values between groups. The EEG alpha power change and THI did not show a significant correlation.

CONCLUSION

To the best of our knowledge, this is the first study to investigate the listening effort of tinnitus patients and healthy controls using EEG alpha band power while controlling for hearing and PAF. Tinnitus patients may expend more listening-related effort despite having similar PAF to the control group.

Influence of Noise Reduction on Ocular Markers of Listening Effort in Hearing Aid Users in Darkness and Ambient Light

The combination of directional microphones and noise reduction (DIR + NR) in hearing aids offers substantial improvement in speech intelligibility and reduction in listening effort in spatial acoustic scenarios. Pupil dilation can be used to infer ocular markers of listening effort. However, pupillometry is also known to crucially depend on luminance. The present study investigates the effects of a state-of-the-art DIR + NR algorithm (implemented in commercial hearing aids) on pupil dilation of hearing aid users both in darkness and ambient light conditions. Speech intelligibility and peak pupil dilations (PPDs) of 29 experienced hearing aid users were measured during a spatial speech-in-noise-task at a signal-to-noise ratio (SNR) matching the individual's speech reception threshold. While speech intelligibility improvements due to DIR + NR were substantial (about 35 percentage points) and independent of luminance, PPDs were only significantly reduced due to DIR + NR in ambient light, but not in darkness. This finding suggests that the reduction in PPD due to DIR + NR (most likely through improvement in SNR) is dependent on luminance and should be interpreted with caution as a marker for listening effort. Relations of reduction in PPD due to DIR + NR in ambient light to subjectively reported long-term fatigue, age, and pure-tone average were not statistically significant, which indicates that all patients benefitted similarly in listening effort from DIR + NR, irrespective of these patient-specific factors. In conclusion, careful control of luminance needs to be taken in hearing aid studies inferring listening effort from pupillometry data.

Acoustically Transparent Hearing Aids Increase Physiological Markers of Listening Effort

While hearing aids are beneficial in compensating for hearing loss and suppressing ambient noise, they may also introduce an unwanted processing burden to the listener's sensory and cognitive system. To investigate such adverse side effects, hearing aids may be set to a 'transparent mode', aiming to replicate natural hearing through the open ear as best as possible. Such transparent hearing aids have previously been demonstrated to exhibit a small but significant disadvantage in speech intelligibility, with less conclusive effects on self-rated listening effort. Here we aimed to reproduce these findings and expand them with neurophysiological measures of invested listening effort, including parietal alpha power and pupil size. Invested listening effort was measured across five task difficulties, ranging from nearly impossible to easy, with normal-hearing participants in both aided and unaided conditions. Results well reproduced a hearing aid disadvantage for speech intelligibility and subjective listening effort ratings. As to be expected, pupil size and parietal alpha power followed an inverted u-shape, peaking at moderate task difficulties (around SRT50). However, the transparent hearing aid increased pupil size and parietal alpha power at medium task demand (between SRT20 and SRT80). These neurophysiological effects were larger than those observed in speech intelligibility and subjective listening effort, respectively. The results gain plausibility by yielding a substantial association of individual pupil size and individual parietal alpha power. In sum, our findings suggest that key neurophysiological measures of invested listening effort are sensitive to the individual additional burden on speech intelligibility that hearing aid processing can introduce.

Cortical encoding of phonetic onsets of both attended and ignored speech in hearing impaired individuals

Hearing impairment alters the sound input received by the human auditory system, reducing speech comprehension in noisy multi-talker auditory scenes. Despite such difficulties, neural signals were shown to encode the attended speech envelope more reliably than the envelope of ignored sounds, reflecting the intention of listeners with hearing impairment (HI). This result raises an important question: What speech-processing stage could reflect the difficulty in attentional selection, if not envelope tracking? Here, we use scalp electroencephalography (EEG) to test the hypothesis that the neural encoding of phonological information (i.e., phonetic boundaries and phonological categories) is affected by HI. In a cocktail-party scenario, such phonological difficulty might be reflected in an overrepresentation of phonological information for both attended and ignored speech sounds, with detrimental effects on the ability to effectively focus on the speaker of interest. To investigate this question, we carried out a re-analysis of an existing dataset where EEG signals were recorded as participants with HI, fitted with hearing aids, attended to one speaker (target) while ignoring a competing speaker (masker) and spatialised multi-talker background noise. Multivariate temporal response function (TRF) analyses indicated a stronger phonological information encoding for target than masker speech streams. Follow-up analyses aimed at disentangling the encoding of phonological categories and phonetic boundaries (phoneme onsets) revealed that neural signals encoded the phoneme onsets for both target and masker streams, in contrast with previously published findings with normal hearing (NH) participants and in line with our hypothesis that speech comprehension difficulties emerge due to a robust phonological encoding of both target and masker. Finally, the neural encoding of phoneme-onsets was stronger for the masker speech, pointing to a possible neural basis for the higher distractibility experienced by individuals with HI.

Distinct roles of SNR, speech Intelligibility, and attentional effort on neural speech tracking in noise

Robust neural encoding of speech in noise is influenced by several factors, including signal-to-noise ratio (SNR), speech intelligibility (SI), and attentional effort (AE). Yet, the interaction and distinct role of these factors remain unclear. In this study, fourteen native English speakers performed selective speech listening tasks at various SNR levels while EEG responses were recorded. Attentional performance was assessed using a repeated word detection task, and attentional effort was inferred from subjects' gaze velocity. Results indicate that both SNR and SI enhance neural tracking of target speech, with distinct effects influenced by the previously overlooked role of attentional effort. Specifically, at high levels of SI, increasing SNR leads to reduced attentional effort, which in turn decreases neural speech tracking. Our findings highlight the importance of differentiating the roles of SNR, SI, and AE in neural speech processing and advance our understanding of how noisy speech is processed in the auditory pathway.

Neural correlates of listening to nonnative-accented speech in multi-talker background noise

We examined the neural correlates underlying the semantic processing of native- and nonnative-accented sentences, presented in quiet or embedded in multi-talker noise. Implementing a semantic violation paradigm, 36 English monolingual young adults listened to American-accented (native) and Chinese-accented (nonnative) English sentences with or without semantic anomalies, presented in quiet or embedded in multi-talker noise, while EEG was recorded. After hearing each sentence, participants verbally repeated the sentence, which was coded and scored as an offline comprehension accuracy measure. In line with earlier behavioral studies, the negative impact of background noise on sentence repetition accuracy was higher for nonnative-accented than for native-accented sentences. At the neural level, the N400 effect for semantic anomaly was larger for native-accented than for nonnative-accented sentences, and was also larger for sentences presented in quiet than in noise, indicating impaired lexical-semantic access when listening to nonnative-accented speech or sentences embedded in noise. No semantic N400 effect was observed for nonnative-accented sentences presented in noise. Furthermore, the frequency of neural oscillations in the alpha frequency band (an index of online cognitive listening effort) was higher when listening to sentences in noise versus in quiet, but no difference was observed across the accent conditions. Semantic anomalies presented in background noise also elicited higher theta activity, whereas processing nonnative-accented anomalies was associated with decreased theta activity. Taken together, we found that listening to nonnative accents or background noise is associated with processing challenges during online semantic access, leading to decreased comprehension accuracy. However, the underlying cognitive mechanism (e.g., associated listening efforts) might manifest differently across accented speech processing and speech in noise processing.

Evaluating aperiodic and periodic neural activity as markers of listening effort in speech perception

Listening effort (LE) is critical to understanding speech perception in acoustically challenging environments. EEG alpha power has emerged as a potential neural correlate of LE. However, the magnitude and direction of the relationship between acoustic challenge and alpha power has been inconsistent in the literature. In the current study, a secondary data analysis of Silcox and Payne (2021), we examine the broadband 1/f-like exponent and offset of the EEG power spectrum as measures of aperiodic neural activity during effortful speech perception and the influence of this aperiodic activity on reliable estimation of periodic (i.e., alpha) neural activity. EEG was continuously recorded during sentence listening and the broadband (1-40 Hz) EEG power spectrum was computed for each participant for quiet and noise trials separately. Using the specparam algorithm, we decomposed the power spectrum into both aperiodic and periodic components and found that broadband aperiodic activity was sensitive to background noise during speech perception and additionally impacted the measurement of noise-induced changes on alpha oscillations. We discuss the implications of these results for the LE and neural speech processing literatures.

A Multimodal Approach to Measuring Listening Effort: A Systematic Review on the Effects of Auditory Task Demand on Physiological Measures and Their Relationship

OBJECTIVES

Listening effort involves the mental effort required to perceive an auditory stimulus, for example in noisy environments. Prolonged increased listening effort, for example due to impaired hearing ability, may increase risk of health complications. It is therefore important to identify valid and sensitive measures of listening effort. Physiological measures have been shown to be sensitive to auditory task demand manipulations and are considered to reflect changes in listening effort. Such measures include pupil dilation, alpha power, skin conductance level, and heart rate variability. The aim of the current systematic review was to provide an overview of studies to listening effort that used multiple physiological measures. The two main questions were: (1) what is the effect of changes in auditory task demand on simultaneously acquired physiological measures from various modalities? and (2) what is the relationship between the responses in these physiological measures?

DESIGN

Following Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, relevant articles were sought in PubMed, PsycInfo, and Web of Science and by examining the references of included articles. Search iterations with different combinations of psychophysiological measures were performed in conjunction with listening effort-related search terms. Quality was assessed using the Appraisal Tool for Cross-Sectional Studies.

RESULTS

A total of 297 articles were identified from three databases, of which 27 were included. One additional article was identified from reference lists. Of the total 28 included articles, 16 included an analysis regarding the relationship between the physiological measures. The overall quality of the included studies was reasonable.

CONCLUSIONS

The included studies showed that most of the physiological measures either show no effect to auditory task demand manipulations or a consistent effect in the expected direction. For example, pupil dilation increased, pre-ejection period decreased, and skin conductance level increased with increasing auditory task demand. Most of the relationships between the responses of these physiological measures were nonsignificant or weak. The physiological measures varied in their sensitivity to auditory task demand manipulations. One of the identified knowledge gaps was that the included studies mostly used tasks with high-performance levels, resulting in an underrepresentation of the physiological changes at lower performance levels. This makes it difficult to capture how the physiological responses behave across the full psychometric curve. Our results support the Framework for Understanding Effortful Listening and the need for a multimodal approach to listening effort. We furthermore discuss focus points for future studies.

Neurophysiological and Autonomic Dynamics of Threat Processing During Sustained Social Fear Generalization

Survival in dynamic environments requires that organisms learn to predict danger from situational cues. One key facet of threat prediction is generalization from a predictive cue to similar cues, ensuring that a cue-outcome contingency is applied beyond the original learning environment. Generalization has been observed in laboratory studies of aversive conditioning: behavioral and physiological processes generalize responses from a stimulus paired with threat (the CS+) to unpaired stimuli, with response magnitudes varying with CS+ similarity. In contrast, work focusing on sensory responses in visual cortex has found a sharpening pattern, in which responses to stimuli closely resembling the CS+ are maximally suppressed, potentially reflecting lateral inhibitory interactions with the CS+ representation. Originally demonstrated with simple visual cues, changes in visuocortical tuning have also been observed in threat generalization learning across facial identities. It is unclear to what extent these visuocortical changes represent transient or sustained effects and if generalization learning requires prior conditioning to the CS+. The present study addressed these questions using EEG and pupillometry in an aversive generalization paradigm involving hundreds of trials using a gradient of facial identities. Visuocortical ssVEP sharpening occurred after dozens of trials of generalization learning without prior differential conditioning, but diminished as learning continued. By contrast, generalization of alpha power suppression, pupil dilation, and self-reported valence and arousal was seen throughout the experiment. Findings are consistent with threat processing models emphasizing the role of changing visucocortical and attentional dynamics when forming, curating, and shaping fear memories as observers continue learning about stimulus-outcome contingencies.

Changes in pupil size track self-control failure

People are more likely to perform poorly on a self-control task following a previous task requiring self-control (ego-depletion), but the mechanism for this effect remains unclear. We used pupillometry to test the role of attentional effort in ego-depletion. We hypothesized that an elevated pupil diameter (PD)-a common physiological measure of effort-during an initial task requiring self-control should be negatively associated with performance on a subsequent control task. To test this hypothesis, participants were first assigned to either a high- or low-demand attention task (manipulation; a standard ego-depletion paradigm), after which all participants completed the same Stroop task. We then separately extracted both sustained (low-frequency) and phasic (high-frequency) changes in PD from both tasks to evaluate possible associations with lapses of cognitive control on the Stroop task. We first show that in the initial task, sustained PD was larger among participants who were assigned to the demanding attention condition. Furthermore, ego-depletion effects were serially mediated by PD: an elevated PD response emerged rapidly among the experimental group during the manipulation, persisted as an elevated baseline response during the Stroop task, and predicted worse accuracy on incongruent trials, revealing a potential indirect pathway to ego-depletion via sustained attention. Secondary analyses revealed another, independent and direct pathway via high levels of transient attentional control: participants who exhibited large phasic responses during the manipulation tended to perform worse on the subsequent Stroop task. We conclude by exploring the neuroscientific implications of these results within the context of current theories of self-control.

Attention Mobilization as a Modulator of Listening Effort: Evidence From Pupillometry

Listening to speech in noise can require substantial mental effort, even among younger normal-hearing adults. The task-evoked pupil response (TEPR) has been shown to track the increased effort exerted to recognize words or sentences in increasing noise. However, few studies have examined the trajectory of listening effort across longer, more natural, stretches of speech, or the extent to which expectations about upcoming listening difficulty modulate the TEPR. Seventeen younger normal-hearing adults listened to 60-s-long audiobook passages, repeated three times in a row, at two different signal-to-noise ratios (SNRs) while pupil size was recorded. There was a significant interaction between SNR, repetition, and baseline pupil size on sustained listening effort. At lower baseline pupil sizes, potentially reflecting lower attention mobilization, TEPRs were more sustained in the harder SNR condition, particularly when attention mobilization remained low by the third presentation. At intermediate baseline pupil sizes, differences between conditions were largely absent, suggesting these listeners had optimally mobilized their attention for both SNRs. Lastly, at higher baseline pupil sizes, potentially reflecting overmobilization of attention, the effect of SNR was initially reversed for the second and third presentations: participants initially appeared to disengage in the harder SNR condition, resulting in reduced TEPRs that recovered in the second half of the story. Together, these findings suggest that the unfolding of listening effort over time depends critically on the extent to which individuals have successfully mobilized their attention in anticipation of difficult listening conditions.

Combining Cardiovascular and Pupil Features Using k-Nearest Neighbor Classifiers to Assess Task Demand, Social Context, and Sentence Accuracy During Listening

In daily life, both acoustic factors and social context can affect listening effort investment. In laboratory settings, information about listening effort has been deduced from pupil and cardiovascular responses independently. The extent to which these measures can jointly predict listening-related factors is unknown. Here we combined pupil and cardiovascular features to predict acoustic and contextual aspects of speech perception. Data were collected from 29 adults (mean  =  64.6 years, SD  =  9.2) with hearing loss. Participants performed a speech perception task at two individualized signal-to-noise ratios (corresponding to 50% and 80% of sentences correct) and in two social contexts (the presence and absence of two observers). Seven features were extracted per trial: baseline pupil size, peak pupil dilation, mean pupil dilation, interbeat interval, blood volume pulse amplitude, pre-ejection period and pulse arrival time. These features were used to train k-nearest neighbor classifiers to predict task demand, social context and sentence accuracy. The k-fold cross validation on the group-level data revealed above-chance classification accuracies: task demand, 64.4%; social context, 78.3%; and sentence accuracy, 55.1%. However, classification accuracies diminished when the classifiers were trained and tested on data from different participants. Individually trained classifiers (one per participant) performed better than group-level classifiers: 71.7% (SD  =  10.2) for task demand, 88.0% (SD  =  7.5) for social context, and 60.0% (SD  =  13.1) for sentence accuracy. We demonstrated that classifiers trained on group-level physiological data to predict aspects of speech perception generalized poorly to novel participants. Individually calibrated classifiers hold more promise for future applications.

Validity and reliability of self-reported and neural measures of listening effort

Listening effort can be defined as a measure of cognitive resources used by listeners to perform a listening task. Various methods have been proposed to measure this effort, yet their reliability remains unestablished, a crucial step before their application in research or clinical settings. This study encompassed 32 participants undertaking speech-in-noise tasks across two sessions, approximately a week apart. They listened to sentences and word lists at varying signal-to-noise ratios (SNRs) (-9, -6, -3 and 0 dB), then retaining them for roughly 3 s. We evaluated the test-retest reliability of self-reported effort ratings, theta (4-7 Hz) and alpha (8-13 Hz) oscillatory power, suggested previously as neural markers of listening effort. Additionally, we examined the reliability of correct word percentages. Both relative and absolute reliability were assessed using intraclass correlation coefficients (ICC) and Bland-Altman analysis. We also computed the standard error of measurement (SEM) and smallest detectable change (SDC). Our findings indicated heightened frontal midline theta power for word lists compared to sentences during the retention phase under high SNRs (0 dB, -3 dB), likely indicating a greater memory load for word lists. We observed SNR's impact on alpha power in the right central region during the listening phase and frontal theta power during the retention phase in sentences. Overall, the reliability analysis demonstrated satisfactory between-session variability for correct words and effort ratings. However, neural measures (frontal midline theta power and right central alpha power) displayed substantial variability, even though group-level outcomes appeared consistent across sessions.

An Exploratory Study of the Effect of Tinnitus on Listening Effort Using EEG and Pupillometry

OBJECTIVE

Previous behavioral studies on listening effort in tinnitus patients did not consider extended high-frequency hearing thresholds and had conflicting results. This inconsistency may be related that listening effort is not evaluated by the central nervous system (CNS) and autonomic nervous system (ANS), which are directly related to tinnitus pathophysiology. This study matches hearing thresholds at all frequencies, including the extended high-frequency and reduces hearing loss to objectively evaluate listening effort over the CNS and ANS simultaneously in tinnitus patients.

STUDY DESIGN

Case-control study.

SETTING

University hospital.

METHODS

Sixteen chronic tinnitus patients and 23 matched healthy controls having normal pure-tone averages with symmetrical hearing thresholds were included. Subjects were evaluated with 0.125 to 20 kHz pure-tone audiometry, Montreal Cognitive Assessment Test (MoCA), Tinnitus Handicap Inventory (THI), Visual Analog Scale (VAS), electroencephalography (EEG), and pupillometry.

RESULTS

Pupil dilation and EEG alpha band in the "coding" phase of the sentence presented in tinnitus patients was less than in the control group (p < .05). VAS score was higher in the tinnitus group (p < .01). Also, there was no statistically significant relationship between EEG and pupillometry components and THI or MoCA (p > .05).

CONCLUSION

This study suggests that tinnitus patients may need to make an extra effort to listen. Also, pupillometry may not be sufficiently reliable to assess listening effort in ANS-related pathologies. Considering the possible listening difficulties in tinnitus patients, reducing the listening difficulties, especially in noisy environments, can be added to the goals of tinnitus therapy protocols.

Music in noise recognition: An EEG study of listening effort in cochlear implant users and normal hearing controls

Despite the plethora of studies investigating listening effort and the amount of research concerning music perception by cochlear implant (CI) users, the investigation of the influence of background noise on music processing has never been performed. Given the typical speech in noise recognition task for the listening effort assessment, the aim of the present study was to investigate the listening effort during an emotional categorization task on musical pieces with different levels of background noise. The listening effort was investigated, in addition to participants' ratings and performances, using EEG features known to be involved in such phenomenon, that is alpha activity in parietal areas and in the left inferior frontal gyrus (IFG), that includes the Broca's area. Results showed that CI users performed worse than normal hearing (NH) controls in the recognition of the emotional content of the stimuli. Furthermore, when considering the alpha activity corresponding to the listening to signal to noise ratio (SNR) 5 and SNR10 conditions subtracted of the activity while listening to the Quiet condition-ideally removing the emotional content of the music and isolating the difficulty level due to the SNRs- CI users reported higher levels of activity in the parietal alpha and in the homologous of the left IFG in the right hemisphere (F8 EEG channel), in comparison to NH. Finally, a novel suggestion of a particular sensitivity of F8 for SNR-related listening effort in music was provided.

Eye Movements Decrease during Effortful Speech Listening

Hearing impairment affects many older adults but is often diagnosed decades after speech comprehension in noisy situations has become effortful. Accurate assessment of listening effort may thus help diagnose hearing impairment earlier. However, pupillometry-the most used approach to assess listening effort-has limitations that hinder its use in practice. The current study explores a novel way to assess listening effort through eye movements. Building on cognitive and neurophysiological work, we examine the hypothesis that eye movements decrease when speech listening becomes challenging. In three experiments with human participants from both sexes, we demonstrate, consistent with this hypothesis, that fixation duration increases and spatial gaze dispersion decreases with increasing speech masking. Eye movements decreased during effortful speech listening for different visual scenes (free viewing, object tracking) and speech materials (simple sentences, naturalistic stories). In contrast, pupillometry was less sensitive to speech masking during story listening, suggesting pupillometric measures may not be as effective for the assessments of listening effort in naturalistic speech-listening paradigms. Our results reveal a critical link between eye movements and cognitive load, suggesting that neural activity in the brain regions that support the regulation of eye movements, such as frontal eye field and superior colliculus, are modulated when listening is effortful.SIGNIFICANCE STATEMENT Assessment of listening effort is critical for early diagnosis of age-related hearing loss. Pupillometry is most used but has several disadvantages. The current study explores a novel way to assess listening effort through eye movements. We examine the hypothesis that eye movements decrease when speech listening becomes effortful. We demonstrate, consistent with this hypothesis, that fixation duration increases and gaze dispersion decreases with increasing speech masking. Eye movements decreased during effortful speech listening for different visual scenes (free viewing, object tracking) and speech materials (sentences, naturalistic stories). Our results reveal a critical link between eye movements and cognitive load, suggesting that neural activity in brain regions that support the regulation of eye movements are modulated when listening is effortful.

Energetic and informational masking place dissociable demands on listening effort: Evidence from simultaneous electroencephalography and pupillometrya)

The task of processing speech masked by concurrent speech/noise can pose a substantial challenge to listeners. However, performance on such tasks may not directly reflect the amount of listening effort they elicit. Changes in pupil size and neural oscillatory power in the alpha range (8-12 Hz) are prominent neurophysiological signals known to reflect listening effort; however, measurements obtained through these two approaches are rarely correlated, suggesting that they may respond differently depending on the specific cognitive demands (and, by extension, the specific type of effort) elicited by specific tasks. This study aimed to compare changes in pupil size and alpha power elicited by different types of auditory maskers (highly confusable intelligible speech maskers, speech-envelope-modulated speech-shaped noise, and unmodulated speech-shaped noise maskers) in young, normal-hearing listeners. Within each condition, the target-to-masker ratio was set at the participant's individually estimated 75% correct point on the psychometric function. The speech masking condition elicited a significantly greater increase in pupil size than either of the noise masking conditions, whereas the unmodulated noise masking condition elicited a significantly greater increase in alpha oscillatory power than the speech masking condition, suggesting that the effort needed to solve these respective tasks may have different neural origins.

Effortful listening produces both enhancement and suppression of alpha in the EEG

Introduction

Adverse listening conditions can drive increased mental effort during listening. Neuromagnetic alpha oscillations (8-13 Hz) may index this listening effort, but inconsistencies regarding the direction of the relationship are abundant. We performed source analyses on high-density EEG data collected during a speech-on-speech listening task to address the possibility that opposing alpha power relationships among alpha producing brain sources drive this inconsistency.

Methods

Listeners (N=20) heard two simultaneously presented sentences of the form: Ready go to now. They either reported the color/number pair of a "Baron" call sign sentence (active: high effort), or ignored the stimuli (passive: low effort). Independent component analysis (ICA) was used to segregate temporally distinct sources in the EEG.

Results

Analysis of independent components (ICs) revealed simultaneous alpha enhancements (e.g., for somatomotor mu ICs) and suppressions (e.g., for left temporal ICs) for different brain sources. The active condition exhibited stronger enhancement for left somatomotor mu rhythm ICs, but stronger suppression for central occipital ICs.

Discussion

This study shows both alpha enhancement and suppression to be associated with increases in listening effort. Literature inconsistencies could partially relate to some source activities overwhelming others in scalp recordings.

Listening Effort in Tinnitus: A Pilot Study Employing a Light EEG Headset and Skin Conductance Assessment during the Listening to a Continuous Speech Stimulus under Different SNR Conditions

Background noise elicits listening effort. What else is tinnitus if not an endogenous background noise? From such reasoning, we hypothesized the occurrence of increased listening effort in tinnitus patients during listening tasks. Such a hypothesis was tested by investigating some indices of listening effort through electroencephalographic and skin conductance, particularly parietal and frontal alpha and electrodermal activity (EDA). Furthermore, tinnitus distress questionnaires (THI and TQ12-I) were employed. Parietal alpha values were positively correlated to TQ12-I scores, and both were negatively correlated to EDA; Pre-stimulus frontal alpha correlated with the THI score in our pilot study; finally, results showed a general trend of increased frontal alpha activity in the tinnitus group in comparison to the control group. Parietal alpha during the listening to stimuli, positively correlated to the TQ12-I, appears to reflect a higher listening effort in tinnitus patients and the perception of tinnitus symptoms. The negative correlation between both listening effort (parietal alpha) and tinnitus symptoms perception (TQ12-I scores) with EDA levels could be explained by a less responsive sympathetic nervous system to prepare the body to expend increased energy during the "fight or flight" response, due to pauperization of energy from tinnitus perception.

Speech recognition in noise task among children and young-adults: a pupillometry study

Introduction

Children experience unique challenges when listening to speech in noisy environments. The present study used pupillometry, an established method for quantifying listening and cognitive effort, to detect temporal changes in pupil dilation during a speech-recognition-in-noise task among school-aged children and young adults.

Methods

Thirty school-aged children and 31 young adults listened to sentences amidst four-talker babble noise in two signal-to-noise ratios (SNR) conditions: high accuracy condition (+10 dB and  + 6 dB, for children and adults, respectively) and low accuracy condition (+5 dB and + 2 dB, for children and adults, respectively). They were asked to repeat the sentences while pupil size was measured continuously during the task.

Results

During the auditory processing phase, both groups displayed pupil dilation; however, adults exhibited greater dilation than children, particularly in the low accuracy condition. In the second phase (retention), only children demonstrated increased pupil dilation, whereas adults consistently exhibited a decrease in pupil size. Additionally, the children's group showed increased pupil dilation during the response phase.

Discussion

Although adults and school-aged children produce similar behavioural scores, group differences in dilation patterns point that their underlying auditory processing differs. A second peak of pupil dilation among the children suggests that their cognitive effort during speech recognition in noise lasts longer than in adults, continuing past the first auditory processing peak dilation. These findings support effortful listening among children and highlight the need to identify and alleviate listening difficulties in school-aged children, to provide proper intervention strategies.

Neural α Oscillations and Pupil Size Differentially Index Cognitive Demand under Competing Audiovisual Task Conditions

Cognitive demand is thought to modulate two often used, but rarely combined, measures: pupil size and neural α (8-12 Hz) oscillatory power. However, it is unclear whether these two measures capture cognitive demand in a similar way under complex audiovisual-task conditions. Here we recorded pupil size and neural α power (using electroencephalography), while human participants of both sexes concurrently performed a visual multiple object-tracking task and an auditory gap detection task. Difficulties of the two tasks were manipulated independent of each other. Participants' performance decreased in accuracy and speed with increasing cognitive demand. Pupil size increased with increasing difficulty for both the auditory and the visual task. In contrast, α power showed diverging neural dynamics: parietal α power decreased with increasing difficulty in the visual task, but not with increasing difficulty in the auditory task. Furthermore, independent of task difficulty, within-participant trial-by-trial fluctuations in pupil size were negatively correlated with α power. Difficulty-induced changes in pupil size and α power, however, did not correlate, which is consistent with their different cognitive-demand sensitivities. Overall, the current study demonstrates that the dynamics of the neurophysiological indices of cognitive demand and associated effort are multifaceted and potentially modality-dependent under complex audiovisual-task conditions.SIGNIFICANCE STATEMENT Pupil size and oscillatory α power are associated with cognitive demand and effort, but their relative sensitivity under complex audiovisual-task conditions is unclear, as is the extent to which they share underlying mechanisms. Using an audiovisual dual-task paradigm, we show that pupil size increases with increasing cognitive demands for both audition and vision. In contrast, changes in oscillatory α power depend on the respective task demands: parietal α power decreases with visual demand but not with auditory task demand. Hence, pupil size and α power show different sensitivity to cognitive demands, perhaps suggesting partly different underlying neural mechanisms.

Streamlining experiment design in cognitive hearing science using OpenSesame

Auditory science increasingly builds on concepts and testing paradigms originated in behavioral psychology and cognitive neuroscience - an evolution of which the resulting discipline is now known as cognitive hearing science. Experimental cognitive hearing science paradigms call for hybrid cognitive and psychobehavioral tests such as those relating the attentional system, working memory, and executive functioning to low-level auditory acuity or speech intelligibility. Building complex multi-stimuli experiments can rapidly become time-consuming and error-prone. Platform-based experiment design can help streamline the implementation of cognitive hearing science experimental paradigms, promote the standardization of experiment design practices, and ensure reliability and control. Here, we introduce a set of features for the open-source python-based OpenSesame platform that allows the rapid implementation of custom behavioral and cognitive hearing science tests, including complex multichannel audio stimuli while interfacing with various synchronous inputs/outputs. Our integration includes advanced audio playback capabilities with multiple loudspeakers, an adaptive procedure, compatibility with standard I/Os and their synchronization through implementation of the Lab Streaming Layer protocol. We exemplify the capabilities of this extended OpenSesame platform with an implementation of the three-alternative forced choice amplitude modulation detection test and discuss reliability and performance. The new features are available free of charge from GitHub: https://github.com/elus-om/BRM_OMEXP .

Combining Multiple Psychophysiological Measures of Listening Effort: Challenges and Recommendations

About one-third of all recently published studies on listening effort have used at least one physiological measure, providing evidence of the popularity of such measures in listening effort research. However, the specific measures employed, as well as the rationales used to justify their inclusion, vary greatly between studies, leading to a literature that is fragmented and difficult to integrate. A unified approach that assesses multiple psychophysiological measures justified by a single rationale would be preferable because it would advance our understanding of listening effort. However, such an approach comes with a number of challenges, including the need to develop a clear definition of listening effort that links to specific physiological measures, customized equipment that enables the simultaneous assessment of multiple measures, awareness of problems caused by the different timescales on which the measures operate, and statistical approaches that minimize the risk of type-I error inflation. This article discusses in detail the various obstacles for combining multiple physiological measures in listening effort research and provides recommendations on how to overcome them.

Time Scales and Moments of Listening Effort Revealed in Pupillometry

This article offers a collection of observations that highlight the value of time course data in pupillometry and points out ways in which these observations create deeper understanding of listening effort. The main message is that listening effort should be considered on a moment-to-moment basis rather than as a singular amount. A review of various studies and the reanalysis of data reveal distinct signatures of effort before a stimulus, during a stimulus, in the moments after a stimulus, and changes over whole experimental testing sessions. Collectively these observations motivate questions that extend beyond the "amount" of effort, toward understanding how long the effort lasts, and how precisely someone can allocate effort at specific points in time or reduce effort at other times. Apparent disagreements between studies are reconsidered as informative lessons about stimulus selection and the nature of pupil dilation as a reflection of decision making rather than the difficulty of sensory encoding.

Exploring the Correlations Between Measures of Listening Effort in Adults and Children: A Systematic Review with Narrative Synthesis

Listening effort (LE) describes the cognitive resources needed to process an auditory message. Our understanding of this notion remains in its infancy, hindering our ability to appreciate how it impacts individuals with hearing impairment effectively. Despite the myriad of proposed measurement tools, a validated method remains elusive. This is complicated by the seeming lack of association between tools demonstrated via correlational analyses. This review aims to systematically review the literature relating to the correlational analyses between different measures of LE. Five databases were used- PubMed, Cochrane, EMBASE, PsychINFO, and CINAHL. The quality of the evidence was assessed using the GRADE criteria and risk of bias with ROBINS-I/GRADE tools. Each statistically significant analysis was classified using an approved system for medical correlations. The final analyses included 48 papers, equating to 274 correlational analyses, of which 99 reached statistical significance (36.1%). Within these results, the most prevalent classifications were poor or fair. Moreover, when moderate or very strong correlations were observed, they tended to be dependent on experimental conditions. The quality of evidence was graded as very low. These results show that measures of LE are poorly correlated and supports the multi-dimensional concept of LE. The lack of association may be explained by considering where each measure operates along the effort perception pathway. Moreover, the fragility of significant correlations to specific conditions further diminishes the hope of finding an all-encompassing tool. Therefore, it may be prudent to focus on capturing the consequences of LE rather than the notion itself.

The Involvement of Listening Effort in Explaining Bilingual Listening Under Adverse Listening Conditions

The current review examines listening effort to uncover how it is implicated in bilingual performance under adverse listening conditions. Various measures of listening effort, including physiological, behavioral, and subjective measures, have been employed to examine listening effort in bilingual children and adults. Adverse listening conditions, stemming from environmental factors, as well as factors related to the speaker or listener, have been examined. The existing literature, although relatively limited to date, points to increased listening effort among bilinguals in their nondominant second language (L2) compared to their dominant first language (L1) and relative to monolinguals. Interestingly, increased effort is often observed even when speech intelligibility remains unaffected. These findings emphasize the importance of considering listening effort alongside speech intelligibility. Building upon the insights gained from the current review, we propose that various factors may modulate the observed effects. These include the particular measure selected to examine listening effort, the characteristics of the adverse condition, as well as factors related to the particular linguistic background of the bilingual speaker. Critically, further research is needed to better understand the impact of these factors on listening effort. The review outlines avenues for future research that would promote a comprehensive understanding of listening effort in bilingual individuals.

Neural responses to naturalistic audiovisual speech are related to listening demand in cochlear implant users

There is a weak relationship between clinical and self-reported speech perception outcomes in cochlear implant (CI) listeners. Such poor correspondence may be due to differences in clinical and “real-world” listening environments and stimuli. Speech in the real world is often accompanied by visual cues, background environmental noise, and is generally in a conversational context, all factors that could affect listening demand. Thus, our objectives were to determine if brain responses to naturalistic speech could index speech perception and listening demand in CI users. Accordingly, we recorded high-density electroencephalogram (EEG) while CI users listened/watched a naturalistic stimulus (i.e., the television show, “The Office”). We used continuous EEG to quantify “speech neural tracking” (i.e., TRFs, temporal response functions) to the show’s soundtrack and 8–12 Hz (alpha) brain rhythms commonly related to listening effort. Background noise at three different signal-to-noise ratios (SNRs), +5, +10, and +15 dB were presented to vary the difficulty of following the television show, mimicking a natural noisy environment. The task also included an audio-only (no video) condition. After each condition, participants subjectively rated listening demand and the degree of words and conversations they felt they understood. Fifteen CI users reported progressively higher degrees of listening demand and less words and conversation with increasing background noise. Listening demand and conversation understanding in the audio-only condition was comparable to that of the highest noise condition (+5 dB). Increasing background noise affected speech neural tracking at a group level, in addition to eliciting strong individual differences. Mixed effect modeling showed that listening demand and conversation understanding were correlated to early cortical speech tracking, such that high demand and low conversation understanding occurred with lower amplitude TRFs. In the high noise condition, greater listening demand was negatively correlated to parietal alpha power, where higher demand was related to lower alpha power. No significant correlations were observed between TRF/alpha and clinical speech perception scores. These results are similar to previous findings showing little relationship between clinical speech perception and quality-of-life in CI users. However, physiological responses to complex natural speech may provide an objective measure of aspects of quality-of-life measures like self-perceived listening demand.

Speech to noise ratio improvement induces nonlinear parietal phase synchrony in hearing aid users

Objectives

Comprehension of speech in adverse listening conditions is challenging for hearing-impaired (HI) individuals. Noise reduction (NR) schemes in hearing aids (HAs) have demonstrated the capability to help HI to overcome these challenges. The objective of this study was to investigate the effect of NR processing (inactive, where the NR feature was switched off, vs. active, where the NR feature was switched on) on correlates of listening effort across two different background noise levels [+3 dB signal-to-noise ratio (SNR) and +8 dB SNR] by using a phase synchrony analysis of electroencephalogram (EEG) signals.

Design

The EEG was recorded while 22 HI participants fitted with HAs performed a continuous speech in noise (SiN) task in the presence of background noise and a competing talker. The phase synchrony within eight regions of interest (ROIs) and four conventional EEG bands was computed by using a multivariate phase synchrony measure.

Results

The results demonstrated that the activation of NR in HAs affects the EEG phase synchrony in the parietal ROI at low SNR differently than that at high SNR. The relationship between conditions of the listening task and phase synchrony in the parietal ROI was nonlinear.

Conclusion

We showed that the activation of NR schemes in HAs can non-linearly reduce correlates of listening effort as estimated by EEG-based phase synchrony. We contend that investigation of the phase synchrony within ROIs can reflect the effects of HAs in HI individuals in ecological listening conditions.

Cortical Correlates of Increased Postural Task Difficulty in Young Adults: A Combined Pupillometry and EEG Study

The pupillary response reflects mental effort (or cognitive workload) during cognitive and/or motor tasks including standing postural control. EEG has been shown to be a non-invasive measure to assess the cortical involvement of postural control. The purpose of this study was to understand the effect of increasing postural task difficulty on the pupillary response and EEG outcomes and their relationship in young adults. Fifteen adults completed multiple trials of standing: eyes open, eyes open while performing a dual-task (auditory two-back), eyes occluded, and eyes occluded with a dual-task. Participants stood on a force plate and wore an eye tracker and 256-channel EEG cap during the conditions. The power spectrum was analyzed for absolute theta (4−7 Hz), alpha (8−13 Hz), and beta (13−30 Hz) frequency bands. Increased postural task difficulty was associated with greater pupillary response (p < 0.001) and increased posterior region alpha power (p = 0.001) and fronto-central region theta/beta power ratio (p = 0.01). Greater pupillary response correlated with lower posterior EEG alpha power during eyes-occluded standing with (r = −0.67, p = 0.01) and without (r = −0.69, p = 0.01) dual-task. A greater pupillary response was associated with lower CoP displacement in the anterior−posterior direction during dual-task eyes-occluded standing (r = −0.60, p = 0.04). The pupillary response and EEG alpha power appear to capture similar cortical processes that are increasingly utilized during progressively more challenging postural task conditions. As the pupillary response also correlated with task performance, this measurement may serve as a valuable stand-alone or adjunct tool to understand the underlying neurophysiological mechanisms of postural control.

Listening Over Time: Single-Trial Tonic and Phasic Oscillatory Alpha-and Theta-Band Indicators of Listening-Related Fatigue

Objectives

Listening effort engages cognitive resources to support speech understanding in adverse listening conditions, and leads to fatigue over the longer term for people with hearing loss. Direct, neural measures of listening-related fatigue have not been developed. Here, event-related or phasic changes in alpha and theta oscillatory power during listening were used as measures of listening effort, and longer-term or tonic changes over the course of the listening task were assessed as measures of listening-related fatigue. In addition, influences of self-reported fatigue and degree of hearing loss on tonic changes in oscillatory power were examined.

Design

Participants were middle-aged adults (age 37–65 years; n = 12) with age-appropriate hearing. Sentences were presented in a background of multi-talker babble at a range of signal-to-noise ratios (SNRs) varying around the 80 percent threshold of individual listeners. Single-trial oscillatory power during both sentence and baseline intervals was analyzed with linear mixed-effect models that included as predictors trial number, SNR, subjective fatigue, and hearing loss.

Results

Alpha and theta power in both sentence presentation and baseline intervals increased as a function of trial, indicating listening-related fatigue. Further, tonic power increases across trials were affected by hearing loss and/or subjective fatigue, particularly in the alpha-band. Phasic changes in alpha and theta power generally tracked with SNR, with decreased alpha power and increased theta power at less favorable SNRs. However, for the alpha-band, the linear effect of SNR emerged only at later trials.

Conclusion

Tonic increases in oscillatory power in alpha- and theta-bands over the course of a listening task may be biomarkers for the development of listening-related fatigue. In addition, alpha-band power as an index of listening-related fatigue may be sensitive to individual differences attributable to level of hearing loss and the subjective experience of listening-related fatigue. Finally, phasic effects of SNR on alpha power emerged only after a period of listening, suggesting that this measure of listening effort could depend on the development of listening-related fatigue.

COVID-19 and Sudden Sensorineural Hearing Loss: A Systematic Review

A growing body of evidence suggests that patients with the 2019 Coronavirus disease (COVID-19) have a risk of developing sudden sensorineural hearing loss (SSNHL). The pathogenesis of COVID-19-related SSNHL remains unclear. This systematic review examined whether COVID-19 causes an increased incidence of SSNHL and the clinical characteristics of patients with COVID-19-related SSNHL according to the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) guidelines. SSNHL usually developed between a few days and 2 months after the diagnosis of COVID-19, and a proportion of patients developed it before the diagnosis of COVID-19. The literature is inconsistent regarding whether COVID-19 causes an increased incidence of SSNHL, and this matter remains unclear. This review included 23 patients with COVID-19-related SSNHL, all adult patients with an average age of 43.1 years. Of these patients, 60.9% had accompanying tinnitus symptoms. Glucocorticoids are the preferred medication to treat COVID-19-related SSNHL. Intratympanic administration may be considered to reduce the side effects of the drug. Hearing tests are suggested when hearing loss is suspected in COVID-19 individuals, and if SSNHL is detected, prompt and aggressive treatment is vital. Large-scale, multicenter research on the pathophysiology, treatment, and prognosis of COVID-19- related SSNHL should be conducted in the future.

EEG Alpha and Pupil Diameter Reflect Endogenous Auditory Attention Switching and Listening Effort

Everyday environments often contain distracting competing talkers and background noise, requiring listeners to focus their attention on one acoustic source and reject others. During this auditory attention task, listeners may naturally interrupt their sustained attention and switch attended sources. The effort required to perform this attention switch has not been well studied in the context of competing continuous speech. In this work, we developed two variants of endogenous attention switching and a sustained attention control. We characterized these three experimental conditions under the context of decoding auditory attention, while simultaneously evaluating listening effort and neural markers of spatial‐audio cues. A least‐squares, electroencephalography (EEG)‐based, attention decoding algorithm was implemented across all conditions. It achieved an accuracy of 69.4% and 64.0% when computed over nonoverlapping 10 and 5‐s correlation windows, respectively. Both decoders illustrated smooth transitions in the attended talker prediction through switches at approximately half of the analysis window size (e.g., the mean lag taken across the two switch conditions was 2.2 s when the 5‐s correlation window was used). Expended listening effort, as measured by simultaneous EEG and pupillometry, was also a strong indicator of whether the listeners sustained attention or performed an endogenous attention switch (peak pupil diameter measure [ p=0.034] and minimum parietal alpha power measure [ p=0.016]). We additionally found evidence of talker spatial cues in the form of centrotemporal alpha power lateralization ( p=0.0428). These results suggest that listener effort and spatial cues may be promising features to pursue in a decoding context, in addition to speech‐based features.

Hearing Aid Noise Reduction Lowers the Sustained Listening Effort During Continuous Speech in Noise-A Combined Pupillometry and EEG Study

OBJECTIVES

The investigation of auditory cognitive processes recently moved from strictly controlled, trial-based paradigms toward the presentation of continuous speech. This also allows the investigation of listening effort on larger time scales (i.e., sustained listening effort). Here, we investigated the modulation of sustained listening effort by a noise reduction algorithm as applied in hearing aids in a listening scenario with noisy continuous speech. The investigated directional noise reduction algorithm mainly suppresses noise from the background.

DESIGN

We recorded the pupil size and the EEG in 22 participants with hearing loss who listened to audio news clips in the presence of background multi-talker babble noise. We estimated how noise reduction (off, on) and signal-to-noise ratio (SNR; +3 dB, +8 dB) affect pupil size and the power in the parietal EEG alpha band (i.e., parietal alpha power) as well as the behavioral performance.

RESULTS

Our results show that noise reduction reduces pupil size, while there was no significant effect of the SNR. It is important to note that we found interactions of SNR and noise reduction, which suggested that noise reduction reduces pupil size predominantly under the lower SNR. Parietal alpha power showed a similar yet nonsignificant pattern, with increased power under easier conditions. In line with the participants' reports that one of the two presented talkers was more intelligible, we found a reduced pupil size, increased parietal alpha power, and better performance when people listened to the more intelligible talker.

CONCLUSIONS

We show that the modulation of sustained listening effort (e.g., by hearing aid noise reduction) as indicated by pupil size and parietal alpha power can be studied under more ecologically valid conditions. Mainly concluded from pupil size, we demonstrate that hearing aid noise reduction lowers sustained listening effort. Our study approximates to real-world listening scenarios and evaluates the benefit of the signal processing as can be found in a modern hearing aid.

EEG Phase Synchrony Reflects SNR Levels During Continuous Speech-in-Noise Tasks

Comprehension of speech in noise is a challenge for hearing-impaired (HI) individuals. Electroencephalography (EEG) provides a tool to investigate the effect of different levels of signal-to-noise ratio (SNR) of the speech. Most studies with EEG have focused on spectral power in well-defined frequency bands such as alpha band. In this study, we investigate how local functional connectivity, i.e. functional connectivity within a localized region of the brain, is affected by two levels of SNR. Twenty-two HI participants performed a continuous speech in noise task at two different SNRs (+3 dB and +8 dB). The local connectivity within eight regions of interest was computed by using a multivariate phase synchrony measure on EEG data. The results showed that phase synchrony increased in the parietal and frontal area as a response to increasing SNR. We contend that local connectivity measures can be used to discriminate between speech-evoked EEG responses at different SNRs.

Creating Clarity in Noisy Environments by Using Deep Learning in Hearing Aids

Hearing aids continue to acquire increasingly sophisticated sound-processing features beyond basic amplification. On the one hand, these have the potential to add user benefit and allow for personalization. On the other hand, if such features are to benefit according to their potential, they require clinicians to be acquainted with both the underlying technologies and the specific fitting handles made available by the individual hearing aid manufacturers. Ensuring benefit from hearing aids in typical daily listening environments requires that the hearing aids handle sounds that interfere with communication, generically referred to as “noise.” With this aim, considerable efforts from both academia and industry have led to increasingly advanced algorithms that handle noise, typically using the principles of directional processing and postfiltering. This article provides an overview of the techniques used for noise reduction in modern hearing aids. First, classical techniques are covered as they are used in modern hearing aids. The discussion then shifts to how deep learning, a subfield of artificial intelligence, provides a radically different way of solving the noise problem. Finally, the results of several experiments are used to showcase the benefits of recent algorithmic advances in terms of signal-to-noise ratio, speech intelligibility, selective attention, and listening effort.

Electrophysiological biomarkers of behavioral dimensions from cross-species paradigms

There has been a fundamental failure to translate preclinically supported research into clinically efficacious treatments for psychiatric disorders. One of the greatest impediments toward improving this species gap has been the difficulty of identifying translatable neurophysiological signals that are related to specific behavioral constructs. Here, we present evidence from three paradigms that were completed by humans and mice using analogous procedures, with each task eliciting candidate a priori defined electrophysiological signals underlying effortful motivation, reinforcement learning, and cognitive control. The effortful motivation was assessed using a progressive ratio breakpoint task, yielding a similar decrease in alpha-band activity over time in both species. Reinforcement learning was assessed via feedback in a probabilistic learning task with delta power significantly modulated by reward surprise in both species. Additionally, cognitive control was assessed in the five-choice continuous performance task, yielding response-locked theta power seen across species, and modulated by difficulty in humans. Together, these successes, and also the teachings from these failures, provide a roadmap towards the use of electrophysiology as a method for translating findings from the preclinical assays to the clinical settings.

Cortical alpha oscillations in cochlear implant users reflect subjective listening effort during speech-in-noise perception

Listening to speech in noise is effortful for individuals with hearing loss, even if they have received a hearing prosthesis such as a hearing aid or cochlear implant (CI). At present, little is known about the neural functions that support listening effort. One form of neural activity that has been suggested to reflect listening effort is the power of 8-12 Hz (alpha) oscillations measured by electroencephalography (EEG). Alpha power in two cortical regions has been associated with effortful listening-left inferior frontal gyrus (IFG), and parietal cortex-but these relationships have not been examined in the same listeners. Further, there are few studies available investigating neural correlates of effort in the individuals with cochlear implants. Here we tested 16 CI users in a novel effort-focused speech-in-noise listening paradigm, and confirm a relationship between alpha power and self-reported effort ratings in parietal regions, but not left IFG. The parietal relationship was not linear but quadratic, with alpha power comparatively lower when effort ratings were at the top and bottom of the effort scale, and higher when effort ratings were in the middle of the scale. Results are discussed in terms of cognitive systems that are engaged in difficult listening situations, and the implication for clinical translation.

EEG power spectral dynamics associated with listening in adverse conditions

Adverse listening conditions increase the demand on cognitive resources needed for speech comprehension. In an exploratory study, we aimed to identify independent power spectral features in the EEG useful for studying the cognitive processes involved in this effortful listening. Listeners performed the coordinate response measure task with a single-talker masker at a 0-dB signal-to-noise ratio. Sounds were left unfiltered or degraded with low-pass filtering. Independent component analysis (ICA) was used to identify independent components (ICs) in the EEG data, the power spectral dynamics of which were then analyzed. Frontal midline theta, left frontal, right frontal, left mu, right mu, left temporal, parietal, left occipital, central occipital, and right occipital clusters of ICs were identified. All IC clusters showed some significant listening-related changes in their power spectrum. This included sustained theta enhancements, gamma enhancements, alpha enhancements, alpha suppression, beta enhancements, and mu rhythm suppression. Several of these effects were absent or negligible using traditional channel analyses. Comparison of filtered to unfiltered speech revealed a stronger alpha suppression in the parietal and central occipital clusters of ICs for the filtered speech condition. This not only replicates recent findings showing greater alpha suppression as listening difficulty increases but also suggests that such alpha-band effects can stem from multiple cortical sources. We lay out the advantages of the ICA approach over the restrictive analyses that have been used as of late in the study of listening effort. We also make suggestions for moving into hypothesis-driven studies regarding the power spectral features that were revealed.