Cortical alpha oscillations in cochlear implant users reflect subjective listening effort during speech-in-noise perception

Impact of Respectfulness on Semantic Integration During Discourse Processing

Linguistic expressions of respectful terms are shaped by social status. Previous studies have shown respectful term usage affects online language processing. This study investigates its impact on semantic integration through three self-pace reading experiments, manipulating Respect Consistency (Respect vs. Disrespect) and Semantic Consistency (Semantic Consistent vs. Semantic Inconsistent). In Experiment 1, disrespect was manipulated by using the plain form of pronouns instead of the respectful form when addressing individuals of higher social status. The results showed longer reading times for semantically inconsistent sentences compared to consistent ones, reflecting the classic semantic integration effect. Nevertheless, this effect was only detected when respectful pronouns were employed. For Experiments 2 and 3, disrespect was operationalized by directly addressing individuals of higher social status by their personal names. A comparable interaction to that in Experiment 1 was identified solely in Experiment 3, which involved an appropriateness judgment task. In contrast, no such interaction was observed in Experiment 2, which involved a reading comprehension task. These results indicated that both disrespectful pronouns and addressing individuals by their personal names hinder semantic integration, but through different mechanisms. These findings provide important insights into the role of respectful term usage on semantic integration during discourse comprehension.

Speech outcomes in cochlear implant users depend on visual cross-modal cortical activity measured before or after implantation

Cochlear implants can partially restore hearing function in deaf individuals, but long-term speech listening outcomes vary widely across cochlear implant users. Visual cross-modal plasticity, where auditory cortical neurons upregulate visual inputs to assist visual processing, is one factor proposed to worsen cochlear implant users' speech outcomes because it may limit auditory processing capability. However, evidence for this view is conflicting, and the relationship of cross-modal activity to speech perception may depend on other variables such as the type of visual activity and when it is assessed. To clarify, we measured visual cross-modal activity during a silent lip reading task using EEG in a cross-sectional, observational study. The study tested visual brain activation in 14 individuals prior to receiving a cochlear implant, 15 individuals tested at least 1 year after receiving and using a cochlear implant and 13 typical hearing controls who did not use a cochlear implant or hearing aid. Cross-modal responses to the onset of a visual event were positively correlated to speech outcomes in cochlear implant users tested after surgery but were negatively correlated in those tested prior to cochlear implant surgery. In addition, cross-modal increases in neural oscillatory power in the alpha band (8-12 Hz) arising in the lip reading task were associated with worse speech outcomes in both cochlear implant user groups. Taken together, results redress claims that cross-modal plasticity is maladaptive for speech outcomes and instead suggest that this relationship depends on the time point of testing, stage of sensory processing and likely the relevance of the stimulus for speech. In addition, findings form the basis for new neural markers that are predictive of cochlear implant users' long-term speech ability.

Auditory working memory in noise in cochlear implant users: Insights from behavioural and neuronal measures

OBJECTIVE

We investigated auditory working-memory using behavioural measures and electroencephalography (EEG) in adult Cochlear Implant (CI) users with varying degrees of CI performance.

METHODS

24 adult CI listeners (age: M = 61.38, SD = 12.45) performed the Sternberg auditory-digit-in-working-memory task during which EEG, accuracy, and promptness were captured. Participants were presented with 2, 4, or 6 digits at Signal-to-Noise Ratios (SNR) of 0, +5 and +10dB. They had to identify a probe stimulus as present in the preceding sequence. ANOVA models were used to compare conditions.

RESULTS

ANOVA revealed that increasing memory load (ML) led to decreased task performance and CI performance interacted with ML and SNR. Centro-parietal alpha power increased during memory encoding but did not differ between conditions. Frontal alpha power was positively correlated with accuracy in conditions most affected by SNR (r = 0.57, r = 0.52) and theta power in conditions most affected by ML (r = 0.55, r = 0.57).

CONCLUSIONS

While parietal alpha power is modulated by the task, it is frontal alpha that relates quantitatively to sensory aspects of processing (noise) and frontal theta to memory load in this group of CI listeners.

SIGNIFICANCE

These results suggest that alpha and theta show distinct relationships to behaviour, providing additional insight into neurocognitive (auditory working-memory) processes in CI users.

Pupillometry and perceived listening effort for cochlear implant users-a comparison of three speech-in-noise tests

OBJECTIVE

Measuring listening effort using pupillometry is challenging in cochlear implant (CI) users. We assess three validated speech tests (Matrix, LIST, and DIN) to identify the optimal speech material for measuring peak-pupil-dilation (PPD) in CI users as a function of signal-to-noise ratio (SNR).

DESIGN

Speech tests were administered in quiet and two noisy conditions, namely at the speech recognition threshold (0 dB re SRT), i.e. the SNR where speech intelligibility (SI) was 50%, and at a more favourable SNR of +6 dB re SRT. PPDs and subjective ratings of effort were obtained.

STUDY SAMPLE

Eighteen unilaterally implanted CI users.

RESULTS

LIST sentences revealed significantly different PPDs between +6 and 0 dB re SRT and DIN triplets between quiet and +6 dB re SRT. PPDs obtained with the Matrix test were independent of SNR and yielded large PPDs and high subjective ratings even in quiet.

CONCLUSIONS

PPD is a sensitive measure for listening effort when processing LIST sentences near 0 dB re SRT and when processing DIN triplets at more favourable listening conditions around +6 dB re SRT. PPDs obtained with the Matrix test were insensitive to SNR, likely because it is demanding for CI users even in quiet.

Cochlear implantation in adults with acquired single-sided deafness improves cortical processing and comprehension of speech presented to the non-implanted ears: a longitudinal EEG study

Former studies have established that individuals with a cochlear implant (CI) for treating single-sided deafness experience improved speech processing after implantation. However, it is not clear how each ear contributes separately to improve speech perception over time at the behavioural and neural level. In this longitudinal EEG study with four different time points, we measured neural activity in response to various temporally and spectrally degraded spoken words presented monaurally to the CI and non-CI ears (5 left and 5 right ears) in 10 single-sided CI users and 10 age- and sex-matched individuals with normal hearing. Subjective comprehension ratings for each word were also recorded. Data from single-sided CI participants were collected pre-CI implantation, and at 3, 6 and 12 months after implantation. We conducted a time-resolved representational similarity analysis on the EEG data to quantify whether and how neural patterns became more similar to those of normal hearing individuals. At 6 months after implantation, the speech comprehension ratings for the degraded words improved in both ears. Notably, the improvement was more pronounced for the non-CI ears than the CI ears. Furthermore, the enhancement in the non-CI ears was paralleled by increased similarity to neural representational patterns of the normal hearing control group. The maximum of this effect coincided with peak decoding accuracy for spoken-word comprehension (600-1200 ms after stimulus onset). The present data demonstrate that cortical processing gradually normalizes within months after CI implantation for speech presented to the non-CI ear. CI enables the deaf ear to provide afferent input, which, according to our results, complements the input of the non-CI ear, gradually improving its function. These novel findings underscore the feasibility of tracking neural recovery after auditory input restoration using advanced multivariate analysis methods, such as representational similarity analysis.

Neuronal oscillations in cognition: Down syndrome as a model of mouse to human translation

Down syndrome (DS), a prevalent cognitive disorder resulting from trisomy of human chromosome 21 (Hsa21), poses a significant global health concern. Affecting approximately 1 in 800 live births worldwide, DS is the leading genetic cause of intellectual disability and a major predisposing factor for early-onset Alzheimer's dementia. The estimated global population of individuals with DS is 6 million, with increasing prevalence due to advances in DS health care. Global efforts are dedicated to unraveling the mechanisms behind the varied clinical outcomes in DS. Recent studies on DS mouse models reveal disrupted neuronal circuits, providing insights into DS pathologies. Yet, translating these findings to humans faces challenges due to limited systematic electrophysiological analyses directly comparing human and mouse. Additionally, disparities in experimental procedures between the two species pose hurdles to successful translation. This review provides a concise overview of neuronal oscillations in human and rodent cognition. Focusing on recent DS mouse model studies, we highlight disruptions in associated brain function. We discuss various electrophysiological paradigms and suggest avenues for exploring molecular dysfunctions contributing to DS-related cognitive impairments. Deciphering neuronal oscillation intricacies holds promise for targeted therapies to alleviate cognitive disabilities in DS individuals.

Evaluating aperiodic and periodic neural activity as markers of listening effort in speech perception

Listening effort (LE) is critical to understanding speech perception in acoustically challenging environments. EEG alpha power has emerged as a potential neural correlate of LE. However, the magnitude and direction of the relationship between acoustic challenge and alpha power has been inconsistent in the literature. In the current study, a secondary data analysis of Silcox and Payne (2021), we examine the broadband 1/f-like exponent and offset of the EEG power spectrum as measures of aperiodic neural activity during effortful speech perception and the influence of this aperiodic activity on reliable estimation of periodic (i.e., alpha) neural activity. EEG was continuously recorded during sentence listening and the broadband (1-40 Hz) EEG power spectrum was computed for each participant for quiet and noise trials separately. Using the specparam algorithm, we decomposed the power spectrum into both aperiodic and periodic components and found that broadband aperiodic activity was sensitive to background noise during speech perception and additionally impacted the measurement of noise-induced changes on alpha oscillations. We discuss the implications of these results for the LE and neural speech processing literatures.

Music training influences online temporal order processing during reading comprehension

Numerous studies have demonstrated the influence of musical expertise on spoken language processing; however, its effects on reading comprehension remain largely unexplored. This study aims to investigate the role of musical expertise in sentence comprehension, particularly concerning the processing of temporal order. Using two self-paced reading experiments, we examined individuals' responses to two-clause sentences connected by the temporal connectives "before" or "after". "After" sentences consistently presented events in their actual order of temporal occurrence, while "before" sentences described events in reverse temporal order. In both experiments, our analyses of reading times consistently uncovered a significant temporal order effect, with words immediately following the temporal connectives being processed slower in "before" sentences compared to "after" sentences. This suggests the presence of immediate online processing costs associated with "before" sentences. Notably, these processing costs were found to be attenuated in individuals with musical expertise compared to those without. However, analyses of comprehension accuracy showed no advantage of musicians over non-musicians. Specifically, in Experiment 1, the two groups showed no difference in comprehension accuracy, while in Experiment 2, musicians exhibited lower accuracy rates compared to non-musicians in both "before" and "after" sentences. These results suggest that musical expertise may attenuate online processing costs associated with complex linguistic constructs, but could not promote reading accuracy. We concluded that music training is associated with a restricted effect on written sentence comprehension.

Can Music Enhance Working Memory and Speech in Noise Perception in Cochlear Implant Users? Design Protocol for a Randomized Controlled Behavioral and Electrophysiological Study

BACKGROUND

A cochlear implant (CI) enables deaf people to understand speech but due to technical restrictions, users face great limitations in noisy conditions. Music training has been shown to augment shared auditory and cognitive neural networks for processing speech and music and to improve auditory-motor coupling, which benefits speech perception in noisy listening conditions. These are promising prerequisites for studying multi-modal neurologic music training (NMT) for speech-in-noise (SIN) perception in adult cochlear implant (CI) users. Furthermore, a better understanding of the neurophysiological correlates when performing working memory (WM) and SIN tasks after multi-modal music training with CI users may provide clinicians with a better understanding of optimal rehabilitation.

METHODS

Within 3 months, 81 post-lingual deafened adult CI recipients will undergo electrophysiological recordings and a four-week neurologic music therapy multi-modal training randomly assigned to one of three training focusses (pitch, rhythm, and timbre). Pre- and post-tests will analyze behavioral outcomes and apply a novel electrophysiological measurement approach that includes neural tracking to speech and alpha oscillation modulations to the sentence-final-word-identification-and-recall test (SWIR-EEG). Expected outcome: Short-term multi-modal music training will enhance WM and SIN performance in post-lingual deafened adult CI recipients and will be reflected in greater neural tracking and alpha oscillation modulations in prefrontal areas. Prospectively, outcomes could contribute to understanding the relationship between cognitive functioning and SIN besides the technical deficits of the CI. Targeted clinical application of music training for post-lingual deafened adult CI carriers to significantly improve SIN and positively impact the quality of life can be realized.

Effects of Background Noise and Linguistic Violations on Frontal Theta Oscillations During Effortful Listening

OBJECTIVES

Background noise and linguistic violations have been shown to increase the listening effort. The present study aims to examine the effects of the interaction between background noise and linguistic violations on subjective listening effort and frontal theta oscillations during effortful listening.

DESIGN

Thirty-two normal-hearing listeners participated in this study. The linguistic violation was operationalized as sentences versus random words (strings). Behavioral and electroencephalography data were collected while participants listened to sentences and strings in background noise at different signal to noise ratios (SNRs) (-9, -6, -3, 0 dB), maintained them in memory for about 3 sec in the presence of background noise, and then chose the correct sequence of words from a base matrix of words.

RESULTS

Results showed the interaction effects of SNR and speech type on effort ratings. Although strings were inherently more effortful than sentences, decreasing SNR from 0 to -9 dB (in 3 dB steps), increased effort rating more for sentences than strings in each step, suggesting the more pronounced effect of noise on sentence processing that strings in low SNRs. Results also showed a significant interaction between SNR and speech type on frontal theta event-related synchronization during the retention interval. This interaction indicated that strings exhibited higher frontal theta event-related synchronization than sentences at SNR of 0 dB, suggesting increased verbal working memory demand for strings under challenging listening conditions.

CONCLUSIONS

The study demonstrated that the interplay between linguistic violation and background noise shapes perceived effort and cognitive load during speech comprehension under challenging listening conditions. The differential impact of noise on processing sentences versus strings highlights the influential role of context and cognitive resource allocation in the processing of speech.

Alpha-Band Dynamics of Hearing Aid Wearers Performing the Repeat-Recall Test (RRT)

This study measured electroencephalographic activity in the alpha band, often associated with task difficulty, to physiologically validate self-reported effort ratings from older hearing-impaired listeners performing the Repeat-Recall Test (RRT)-an integrative multipart assessment of speech-in-noise performance, context use, and auditory working memory. Following a single-blind within-subjects design, 16 older listeners (mean age = 71 years, SD = 13, 9 female) with a moderate-to-severe degree of bilateral sensorineural hearing loss performed the RRT while wearing hearing aids at four fixed signal-to-noise ratios (SNRs) of -5, 0, 5, and 10 dB. Performance and subjective ratings of listening effort were assessed for complementary versions of the RRT materials with high/low availability of semantic context. Listeners were also tested with a version of the RRT that omitted the memory (i.e., recall) component. As expected, results showed alpha power to decrease significantly with increasing SNR from 0 through 10 dB. When tested with high context sentences, alpha was significantly higher in conditions where listeners had to recall the sentence materials compared to conditions where the recall requirement was omitted. When tested with low context sentences, alpha power was relatively high irrespective of the memory component. Within-subjects, alpha power was related to listening effort ratings collected across the different RRT conditions. Overall, these results suggest that the multipart demands of the RRT modulate both neural and behavioral measures of listening effort in directions consistent with the expected/designed difficulty of the RRT conditions.

Validity and reliability of self-reported and neural measures of listening effort

Listening effort can be defined as a measure of cognitive resources used by listeners to perform a listening task. Various methods have been proposed to measure this effort, yet their reliability remains unestablished, a crucial step before their application in research or clinical settings. This study encompassed 32 participants undertaking speech-in-noise tasks across two sessions, approximately a week apart. They listened to sentences and word lists at varying signal-to-noise ratios (SNRs) (-9, -6, -3 and 0 dB), then retaining them for roughly 3 s. We evaluated the test-retest reliability of self-reported effort ratings, theta (4-7 Hz) and alpha (8-13 Hz) oscillatory power, suggested previously as neural markers of listening effort. Additionally, we examined the reliability of correct word percentages. Both relative and absolute reliability were assessed using intraclass correlation coefficients (ICC) and Bland-Altman analysis. We also computed the standard error of measurement (SEM) and smallest detectable change (SDC). Our findings indicated heightened frontal midline theta power for word lists compared to sentences during the retention phase under high SNRs (0 dB, -3 dB), likely indicating a greater memory load for word lists. We observed SNR's impact on alpha power in the right central region during the listening phase and frontal theta power during the retention phase in sentences. Overall, the reliability analysis demonstrated satisfactory between-session variability for correct words and effort ratings. However, neural measures (frontal midline theta power and right central alpha power) displayed substantial variability, even though group-level outcomes appeared consistent across sessions.

Listening effort and fatigue among cochlear implant users: a scoping review

Introduction

In challenging listening situations, speech perception with a cochlear implant (CI) remains demanding and requires high levels of listening effort, which can lead to increased levels of listening-related fatigue. The body of literature on these topics increases as the number of CI users rises. This scoping review aims to provide an overview of the existing literature on listening effort, fatigue, and listening-related fatigue among CI users and the measurement techniques to evaluate them.

Methods

The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) Statements were used to conduct the scoping review. The search was performed on PubMed, Scopus, and Web of Science to identify all relevant studies.

Results

In total, 24 studies were included and suggests that CI users experience higher levels of listening effort when compared to normal hearing controls using scales, questionnaires and electroencephalogram measurements. However, executing dual-task paradigms did not reveal any difference in listening effort between both groups. Uncertainty exists regarding the difference in listening effort between unilateral, bilateral, and bimodal CI users with bilateral hearing loss due to ambiguous results. Only five studies were eligible for the research on fatigue and listening-related fatigue. Additionally, studies using objective measurement methods were lacking.

Discussion

This scoping review highlights the necessity for additional research on these topics. Moreover, there is a need for guidelines on how listening effort, fatigue, and listening-related fatigue should be measured to allow for study results that are comparable and support optimal rehabilitation strategies.

Examining cortical tracking of the speech envelope in post-stroke aphasia

Introduction

People with aphasia have been shown to benefit from rhythmic elements for language production during aphasia rehabilitation. However, it is unknown whether rhythmic processing is associated with such benefits. Cortical tracking of the speech envelope (CTenv) may provide a measure of encoding of speech rhythmic properties and serve as a predictor of candidacy for rhythm-based aphasia interventions.

Methods

Electroencephalography was used to capture electrophysiological responses while Spanish speakers with aphasia (n = 9) listened to a continuous speech narrative (audiobook). The Temporal Response Function was used to estimate CTenv in the delta (associated with word- and phrase-level properties), theta (syllable-level properties), and alpha bands (attention-related properties). CTenv estimates were used to predict aphasia severity, performance in rhythmic perception and production tasks, and treatment response in a sentence-level rhythm-based intervention.

Results

CTenv in delta and theta, but not alpha, predicted aphasia severity. Neither CTenv in delta, alpha, or theta bands predicted performance in rhythmic perception or production tasks. Some evidence supported that CTenv in theta could predict sentence-level learning in aphasia, but alpha and delta did not.

Conclusion

CTenv of the syllable-level properties was relatively preserved in individuals with less language impairment. In contrast, higher encoding of word- and phrase-level properties was relatively impaired and was predictive of more severe language impairments. CTenv and treatment response to sentence-level rhythm-based interventions need to be further investigated.

Music in noise recognition: An EEG study of listening effort in cochlear implant users and normal hearing controls

Despite the plethora of studies investigating listening effort and the amount of research concerning music perception by cochlear implant (CI) users, the investigation of the influence of background noise on music processing has never been performed. Given the typical speech in noise recognition task for the listening effort assessment, the aim of the present study was to investigate the listening effort during an emotional categorization task on musical pieces with different levels of background noise. The listening effort was investigated, in addition to participants' ratings and performances, using EEG features known to be involved in such phenomenon, that is alpha activity in parietal areas and in the left inferior frontal gyrus (IFG), that includes the Broca's area. Results showed that CI users performed worse than normal hearing (NH) controls in the recognition of the emotional content of the stimuli. Furthermore, when considering the alpha activity corresponding to the listening to signal to noise ratio (SNR) 5 and SNR10 conditions subtracted of the activity while listening to the Quiet condition-ideally removing the emotional content of the music and isolating the difficulty level due to the SNRs- CI users reported higher levels of activity in the parietal alpha and in the homologous of the left IFG in the right hemisphere (F8 EEG channel), in comparison to NH. Finally, a novel suggestion of a particular sensitivity of F8 for SNR-related listening effort in music was provided.

Effortful listening produces both enhancement and suppression of alpha in the EEG

Introduction

Adverse listening conditions can drive increased mental effort during listening. Neuromagnetic alpha oscillations (8-13 Hz) may index this listening effort, but inconsistencies regarding the direction of the relationship are abundant. We performed source analyses on high-density EEG data collected during a speech-on-speech listening task to address the possibility that opposing alpha power relationships among alpha producing brain sources drive this inconsistency.

Methods

Listeners (N=20) heard two simultaneously presented sentences of the form: Ready go to now. They either reported the color/number pair of a "Baron" call sign sentence (active: high effort), or ignored the stimuli (passive: low effort). Independent component analysis (ICA) was used to segregate temporally distinct sources in the EEG.

Results

Analysis of independent components (ICs) revealed simultaneous alpha enhancements (e.g., for somatomotor mu ICs) and suppressions (e.g., for left temporal ICs) for different brain sources. The active condition exhibited stronger enhancement for left somatomotor mu rhythm ICs, but stronger suppression for central occipital ICs.

Discussion

This study shows both alpha enhancement and suppression to be associated with increases in listening effort. Literature inconsistencies could partially relate to some source activities overwhelming others in scalp recordings.

Dynamic networks differentiate the language ability of children with cochlear implants

Background

Cochlear implantation (CI) in prelingually deafened children has been shown to be an effective intervention for developing language and reading skill. However, there is a substantial proportion of the children receiving CI who struggle with language and reading. The current study-one of the first to implement electrical source imaging in CI population was designed to identify the neural underpinnings in two groups of CI children with good and poor language and reading skill.

Methods

Data using high density electroencephalography (EEG) under a resting state condition was obtained from 75 children, 50 with CIs having good (HL) or poor language skills (LL) and 25 normal hearing (NH) children. We identified coherent sources using dynamic imaging of coherent sources (DICS) and their effective connectivity computing time-frequency causality estimation based on temporal partial directed coherence (TPDC) in the two CI groups compared to a cohort of age and gender matched NH children.

Findings

Sources with higher coherence amplitude were observed in three frequency bands (alpha, beta and gamma) for the CI groups when compared to normal hearing children. The two groups of CI children with good (HL) and poor (LL) language ability exhibited not only different cortical and subcortical source profiles but also distinct effective connectivity between them. Additionally, a support vector machine (SVM) algorithm using these sources and their connectivity patterns for each CI group across the three frequency bands was able to predict the language and reading scores with high accuracy.

Interpretation

Increased coherence in the CI groups suggest overall that the oscillatory activity in some brain areas become more strongly coupled compared to the NH group. Moreover, the different sources and their connectivity patterns and their association to language and reading skill in both groups, suggest a compensatory adaptation that either facilitated or impeded language and reading development. The neural differences in the two groups of CI children may reflect potential biomarkers for predicting outcome success in CI children.

Neural α Oscillations and Pupil Size Differentially Index Cognitive Demand under Competing Audiovisual Task Conditions

Cognitive demand is thought to modulate two often used, but rarely combined, measures: pupil size and neural α (8-12 Hz) oscillatory power. However, it is unclear whether these two measures capture cognitive demand in a similar way under complex audiovisual-task conditions. Here we recorded pupil size and neural α power (using electroencephalography), while human participants of both sexes concurrently performed a visual multiple object-tracking task and an auditory gap detection task. Difficulties of the two tasks were manipulated independent of each other. Participants' performance decreased in accuracy and speed with increasing cognitive demand. Pupil size increased with increasing difficulty for both the auditory and the visual task. In contrast, α power showed diverging neural dynamics: parietal α power decreased with increasing difficulty in the visual task, but not with increasing difficulty in the auditory task. Furthermore, independent of task difficulty, within-participant trial-by-trial fluctuations in pupil size were negatively correlated with α power. Difficulty-induced changes in pupil size and α power, however, did not correlate, which is consistent with their different cognitive-demand sensitivities. Overall, the current study demonstrates that the dynamics of the neurophysiological indices of cognitive demand and associated effort are multifaceted and potentially modality-dependent under complex audiovisual-task conditions.SIGNIFICANCE STATEMENT Pupil size and oscillatory α power are associated with cognitive demand and effort, but their relative sensitivity under complex audiovisual-task conditions is unclear, as is the extent to which they share underlying mechanisms. Using an audiovisual dual-task paradigm, we show that pupil size increases with increasing cognitive demands for both audition and vision. In contrast, changes in oscillatory α power depend on the respective task demands: parietal α power decreases with visual demand but not with auditory task demand. Hence, pupil size and α power show different sensitivity to cognitive demands, perhaps suggesting partly different underlying neural mechanisms.

Brightening the Study of Listening Effort with Functional Near-Infrared Spectroscopy: A Scoping Review

Listening effort is a long-standing area of interest in auditory cognitive neuroscience. Prior research has used multiple techniques to shed light on the neurophysiological mechanisms underlying listening during challenging conditions. Functional near-infrared spectroscopy (fNIRS) is growing in popularity as a tool for cognitive neuroscience research, and its recent advances offer many potential advantages over other neuroimaging modalities for research related to listening effort. This review introduces the basic science of fNIRS and its uses for auditory cognitive neuroscience. We also discuss its application in recently published studies on listening effort and consider future opportunities for studying effortful listening with fNIRS. After reading this article, the learner will know how fNIRS works and summarize its uses for listening effort research. The learner will also be able to apply this knowledge toward generation of future research in this area.

Combining Multiple Psychophysiological Measures of Listening Effort: Challenges and Recommendations

About one-third of all recently published studies on listening effort have used at least one physiological measure, providing evidence of the popularity of such measures in listening effort research. However, the specific measures employed, as well as the rationales used to justify their inclusion, vary greatly between studies, leading to a literature that is fragmented and difficult to integrate. A unified approach that assesses multiple psychophysiological measures justified by a single rationale would be preferable because it would advance our understanding of listening effort. However, such an approach comes with a number of challenges, including the need to develop a clear definition of listening effort that links to specific physiological measures, customized equipment that enables the simultaneous assessment of multiple measures, awareness of problems caused by the different timescales on which the measures operate, and statistical approaches that minimize the risk of type-I error inflation. This article discusses in detail the various obstacles for combining multiple physiological measures in listening effort research and provides recommendations on how to overcome them.

Performance Monitoring and Cognitive Inhibition during a Speech-in-Noise Task in Older Listeners

The goal of this study was to examine the effect of hearing loss on theta and alpha electroencephalography (EEG) frequency power measures of performance monitoring and cognitive inhibition, respectively, during a speech-in-noise task. It was hypothesized that hearing loss would be associated with an increase in the peak power of theta and alpha frequencies toward easier conditions compared to normal hearing adults. The shift would reflect how hearing loss modulates the recruitment of listening effort to easier listening conditions. Nine older adults with normal hearing (ONH) and 10 older adults with hearing loss (OHL) participated in this study. EEG data were collected from all participants while they completed the words-in-noise task. It hypothesized that hearing loss would also have an effect on theta and alpha power. The ONH group showed an inverted U -shape effect of signal-to-noise ratio (SNR), but there were limited effects of SNR on theta or alpha power in the OHL group. The results of the ONH group support the growing body of literature showing effects of listening conditions on alpha and theta power. The null results of listening condition in the OHL group add to a smaller body of literature, suggesting that listening effort research conditions should have near ceiling performance.

EEG Alpha Band Responses Reveal Amplification Benefits in Infants with Hearing Loss

Our objective was to examine the effects of hearing aid amplification on auditory detection and discrimination in infants who were hard of hearing (IHH) using a physiological measure of auditory perception. We recorded EEG from 41 sleeping IHH aged 1.04 to 5.62 months while presenting auditory stimuli in a mismatch response paradigm. Responses were recorded during two listening conditions for each participant: aided and unaided. Temporal envelopes of the mismatch response in the EEG alpha band (6-12 Hz) were extracted from the latent, time-frequency transformed data. Aided alpha band responses were greater than unaided responses for the deviant trials but were not different for the standard trials. Responses to the deviant trials were greater than responses to the standard trials for the aided conditions but were not different for the unaided conditions. These results suggest that the alpha band mismatch can be used to examine both detection and discrimination of speech and non-speech sounds in IHH. With further study, the alpha band mismatch could expand and refine our abilities to validate hearing aid fittings at younger ages than current clinical protocols allow.

Neural responses to naturalistic audiovisual speech are related to listening demand in cochlear implant users

There is a weak relationship between clinical and self-reported speech perception outcomes in cochlear implant (CI) listeners. Such poor correspondence may be due to differences in clinical and “real-world” listening environments and stimuli. Speech in the real world is often accompanied by visual cues, background environmental noise, and is generally in a conversational context, all factors that could affect listening demand. Thus, our objectives were to determine if brain responses to naturalistic speech could index speech perception and listening demand in CI users. Accordingly, we recorded high-density electroencephalogram (EEG) while CI users listened/watched a naturalistic stimulus (i.e., the television show, “The Office”). We used continuous EEG to quantify “speech neural tracking” (i.e., TRFs, temporal response functions) to the show’s soundtrack and 8–12 Hz (alpha) brain rhythms commonly related to listening effort. Background noise at three different signal-to-noise ratios (SNRs), +5, +10, and +15 dB were presented to vary the difficulty of following the television show, mimicking a natural noisy environment. The task also included an audio-only (no video) condition. After each condition, participants subjectively rated listening demand and the degree of words and conversations they felt they understood. Fifteen CI users reported progressively higher degrees of listening demand and less words and conversation with increasing background noise. Listening demand and conversation understanding in the audio-only condition was comparable to that of the highest noise condition (+5 dB). Increasing background noise affected speech neural tracking at a group level, in addition to eliciting strong individual differences. Mixed effect modeling showed that listening demand and conversation understanding were correlated to early cortical speech tracking, such that high demand and low conversation understanding occurred with lower amplitude TRFs. In the high noise condition, greater listening demand was negatively correlated to parietal alpha power, where higher demand was related to lower alpha power. No significant correlations were observed between TRF/alpha and clinical speech perception scores. These results are similar to previous findings showing little relationship between clinical speech perception and quality-of-life in CI users. However, physiological responses to complex natural speech may provide an objective measure of aspects of quality-of-life measures like self-perceived listening demand.

Speech to noise ratio improvement induces nonlinear parietal phase synchrony in hearing aid users

Objectives

Comprehension of speech in adverse listening conditions is challenging for hearing-impaired (HI) individuals. Noise reduction (NR) schemes in hearing aids (HAs) have demonstrated the capability to help HI to overcome these challenges. The objective of this study was to investigate the effect of NR processing (inactive, where the NR feature was switched off, vs. active, where the NR feature was switched on) on correlates of listening effort across two different background noise levels [+3 dB signal-to-noise ratio (SNR) and +8 dB SNR] by using a phase synchrony analysis of electroencephalogram (EEG) signals.

Design

The EEG was recorded while 22 HI participants fitted with HAs performed a continuous speech in noise (SiN) task in the presence of background noise and a competing talker. The phase synchrony within eight regions of interest (ROIs) and four conventional EEG bands was computed by using a multivariate phase synchrony measure.

Results

The results demonstrated that the activation of NR in HAs affects the EEG phase synchrony in the parietal ROI at low SNR differently than that at high SNR. The relationship between conditions of the listening task and phase synchrony in the parietal ROI was nonlinear.

Conclusion

We showed that the activation of NR schemes in HAs can non-linearly reduce correlates of listening effort as estimated by EEG-based phase synchrony. We contend that investigation of the phase synchrony within ROIs can reflect the effects of HAs in HI individuals in ecological listening conditions.

Listening Over Time: Single-Trial Tonic and Phasic Oscillatory Alpha-and Theta-Band Indicators of Listening-Related Fatigue

Objectives

Listening effort engages cognitive resources to support speech understanding in adverse listening conditions, and leads to fatigue over the longer term for people with hearing loss. Direct, neural measures of listening-related fatigue have not been developed. Here, event-related or phasic changes in alpha and theta oscillatory power during listening were used as measures of listening effort, and longer-term or tonic changes over the course of the listening task were assessed as measures of listening-related fatigue. In addition, influences of self-reported fatigue and degree of hearing loss on tonic changes in oscillatory power were examined.

Design

Participants were middle-aged adults (age 37–65 years; n = 12) with age-appropriate hearing. Sentences were presented in a background of multi-talker babble at a range of signal-to-noise ratios (SNRs) varying around the 80 percent threshold of individual listeners. Single-trial oscillatory power during both sentence and baseline intervals was analyzed with linear mixed-effect models that included as predictors trial number, SNR, subjective fatigue, and hearing loss.

Results

Alpha and theta power in both sentence presentation and baseline intervals increased as a function of trial, indicating listening-related fatigue. Further, tonic power increases across trials were affected by hearing loss and/or subjective fatigue, particularly in the alpha-band. Phasic changes in alpha and theta power generally tracked with SNR, with decreased alpha power and increased theta power at less favorable SNRs. However, for the alpha-band, the linear effect of SNR emerged only at later trials.

Conclusion

Tonic increases in oscillatory power in alpha- and theta-bands over the course of a listening task may be biomarkers for the development of listening-related fatigue. In addition, alpha-band power as an index of listening-related fatigue may be sensitive to individual differences attributable to level of hearing loss and the subjective experience of listening-related fatigue. Finally, phasic effects of SNR on alpha power emerged only after a period of listening, suggesting that this measure of listening effort could depend on the development of listening-related fatigue.

Working-Memory, Alpha-Theta Oscillations and Musical Training in Older Age: Research Perspectives for Speech-on-speech Perception

During the normal course of aging, perception of speech-on-speech or “cocktail party” speech and use of working memory (WM) abilities change. Musical training, which is a complex activity that integrates multiple sensory modalities and higher-order cognitive functions, reportedly benefits both WM performance and speech-on-speech perception in older adults. This mini-review explores the relationship between musical training, WM and speech-on-speech perception in older age (> 65 years) through the lens of the Ease of Language Understanding (ELU) model. Linking neural-oscillation literature associating speech-on-speech perception and WM with alpha-theta oscillatory activity, we propose that two stages of speech-on-speech processing in the ELU are underpinned by WM-related alpha-theta oscillatory activity, and that effects of musical training on speech-on-speech perception may be reflected in these frequency bands among older adults.

EEG Alpha and Pupil Diameter Reflect Endogenous Auditory Attention Switching and Listening Effort

Everyday environments often contain distracting competing talkers and background noise, requiring listeners to focus their attention on one acoustic source and reject others. During this auditory attention task, listeners may naturally interrupt their sustained attention and switch attended sources. The effort required to perform this attention switch has not been well studied in the context of competing continuous speech. In this work, we developed two variants of endogenous attention switching and a sustained attention control. We characterized these three experimental conditions under the context of decoding auditory attention, while simultaneously evaluating listening effort and neural markers of spatial‐audio cues. A least‐squares, electroencephalography (EEG)‐based, attention decoding algorithm was implemented across all conditions. It achieved an accuracy of 69.4% and 64.0% when computed over nonoverlapping 10 and 5‐s correlation windows, respectively. Both decoders illustrated smooth transitions in the attended talker prediction through switches at approximately half of the analysis window size (e.g., the mean lag taken across the two switch conditions was 2.2 s when the 5‐s correlation window was used). Expended listening effort, as measured by simultaneous EEG and pupillometry, was also a strong indicator of whether the listeners sustained attention or performed an endogenous attention switch (peak pupil diameter measure [ p=0.034] and minimum parietal alpha power measure [ p=0.016]). We additionally found evidence of talker spatial cues in the form of centrotemporal alpha power lateralization ( p=0.0428). These results suggest that listener effort and spatial cues may be promising features to pursue in a decoding context, in addition to speech‐based features.

Parkinson's disease affects the neural alpha oscillations associated with speech-in-noise processing

Parkinson's disease (PD) has increasingly been associated with auditory dysfunction, including alterations regarding the control of auditory information processing. Although these alterations may interfere with the processing of speech in degraded listening conditions, behavioural studies have generally found preserved speech-in-noise recognition in PD. However, behavioural speech audiometry does not capture the neurophysiological mechanisms supporting speech-in-noise processing. Therefore, the aim of this study was to investigate the neural oscillatory mechanisms associated with speech-in-noise processing in PD. Twelve persons with PD and 12 age- and gender-matched healthy controls (HCs) were included in this study. Persons with PD were studied in the medication off condition. All subjects underwent an audiometric screening and performed a sentence-in-noise recognition task under simultaneous electroencephalography (EEG) recording. Behavioural speech recognition scores and self-reported ratings of effort, performance, and motivation were collected. Time-frequency analysis of EEG data revealed no significant difference between persons with PD and HCs regarding delta-theta (2-8 Hz) inter-trial phase coherence to noise and sentence onset. In contrast, significantly increased alpha (8-12 Hz) power was found in persons with PD compared with HCs during the sentence-in-noise recognition task. Behaviourally, persons with PD demonstrated significantly decreased speech recognition scores, whereas no significant differences were found regarding effort, performance, and motivation ratings. These results suggest that persons with PD allocate more cognitive resources to support speech-in-noise processing. The interpretation of this finding is discussed in the context of a top-down mediated compensation mechanism for inefficient filtering and degradation of auditory input in PD.