Effects of aging and noise on real-time spoken word recognition: evidence from eye movements

Age-related differences in processing of emotions in speech disappear with babble noise in the background

Older adults process emotional speech differently than young adults, relying less on prosody (tone) relative to semantics (words). This study aimed to elucidate the mechanisms underlying these age-related differences via an emotional speech-in-noise test. A sample of 51 young and 47 older adults rated spoken sentences with emotional content on both prosody and semantics, presented on the background of wideband speech-spectrum noise (sensory interference) or on the background of multi-talker babble (sensory/cognitive interference). The presence of wideband noise eliminated age-related differences in semantics but not in prosody when processing emotional speech. Conversely, the presence of babble resulted in the elimination of age-related differences across all measures. The results suggest that both sensory and cognitive-linguistic factors contribute to age-related changes in emotional speech processing. Because real world conditions typically involve noisy background, our results highlight the importance of testing under such conditions.

Efficiency of spoken word recognition slows across the adult lifespan

Spoken word recognition is a critical hub during language processing, linking hearing and perception to meaning and syntax. Words must be recognized quickly and efficiently as speech unfolds to be successfully integrated into conversation. This makes word recognition a computationally challenging process even for young, normal hearing adults. Older adults often experience declines in hearing and cognition, which could be linked by age-related declines in the cognitive processes specific to word recognition. However, it is unclear whether changes in word recognition across the lifespan can be accounted for by hearing or domain-general cognition. Participants (N = 107) responded to spoken words in a Visual World Paradigm task while their eyes were tracked to assess the real-time dynamics of word recognition. We examined several indices of word recognition from early adolescence through older adulthood (ages 11-78). The timing and proportion of eye fixations to target and competitor images reveals that spoken word recognition became more efficient through age 25 and began to slow in middle age, accompanied by declines in the ability to resolve competition (e.g., suppressing sandwich to recognize sandal). There was a unique effect of age even after accounting for differences in inhibitory control, processing speed, and hearing thresholds. This suggests a limited age range where listeners are peak performers.

Recognizing non-native spoken words in background noise increases interference from the native language

Listeners frequently recognize spoken words in the presence of background noise. Previous research has shown that noise reduces phoneme intelligibility and hampers spoken-word recognition - especially for non-native listeners. In the present study, we investigated how noise influences lexical competition in both the non-native and the native language, reflecting the degree to which both languages are co-activated. We recorded the eye movements of native Dutch participants as they listened to English sentences containing a target word while looking at displays containing four objects. On target-present trials, the visual referent depicting the target word was present, along with three unrelated distractors. On target-absent trials, the target object (e.g., wizard) was absent. Instead, the display contained an English competitor, overlapping with the English target in phonological onset (e.g., window), a Dutch competitor, overlapping with the English target in phonological onset (e.g., wimpel, pennant), and two unrelated distractors. Half of the sentences was masked by speech-shaped noise; the other half was presented in quiet. Compared to speech in quiet, noise delayed fixations to the target objects on target-present trials. For target-absent trials, we observed that the likelihood for fixation biases towards the English and Dutch onset competitors (over the unrelated distractors) was larger in noise than in quiet. Our data thus show that the presence of background noise increases lexical competition in the task-relevant non-native (English) and in the task-irrelevant native (Dutch) language. The latter reflects stronger interference of one's native language during non-native spoken-word recognition under adverse conditions.

Purposeful listening in challenging conditions: A study of prediction during consecutive interpreting in noise

Prediction is often used during language comprehension. However, studies of prediction have tended to focus on L1 listeners in quiet conditions. Thus, it is unclear how listeners predict outside the laboratory and in specific communicative settings. Here, we report two eye-tracking studies which used a visual-world paradigm to investigate whether prediction during a consecutive interpreting task differs from prediction during a listening task in L2 listeners, and whether L2 listeners are able to predict in the noisy conditions that might be associated with this communicative setting. In a first study, thirty-six Dutch-English bilinguals either just listened to, or else listened to and then consecutively interpreted, predictable sentences presented on speech-shaped sound. In a second study, another thirty-six Dutch-English bilinguals carried out the same tasks in clear speech. Our results suggest that L2 listeners predict the meaning of upcoming words in noisy conditions. However, we did not find that predictive eye movements depended on task, nor that L2 listeners predicted upcoming word form. We also did not find a difference in predictive patterns when we compared our two studies. Thus, L2 listeners predict in noisy circumstances, supporting theories which posit that prediction regularly takes place in comprehension, but we did not find evidence that a subsequent production task or noise affects semantic prediction.

Delayed Lexical Access and Cascading Effects on Spreading Semantic Activation During Spoken Word Recognition in Children With Hearing Aids and Cochlear Implants: Evidence From Eye-Tracking

OBJECTIVE

The objective of this study was to characterize the dynamics of real-time lexical access, including lexical competition among phonologically similar words, and spreading semantic activation in school-age children with hearing aids (HAs) and children with cochlear implants (CIs). We hypothesized that developing spoken language via degraded auditory input would lead children with HAs or CIs to adapt their approach to spoken word recognition, especially by slowing down lexical access.

DESIGN

Participants were children ages 9- to 12-years old with normal hearing (NH), HAs, or CIs. Participants completed a Visual World Paradigm task in which they heard a spoken word and selected the matching picture from four options. Competitor items were either phonologically similar, semantically similar, or unrelated to the target word. As the target word unfolded, children's fixations to the target word, cohort competitor, rhyme competitor, semantically related item, and unrelated item were recorded as indices of ongoing lexical access and spreading semantic activation.

RESULTS

Children with HAs and children with CIs showed slower fixations to the target, reduced fixations to the cohort competitor, and increased fixations to the rhyme competitor, relative to children with NH. This wait-and-see profile was more pronounced in the children with CIs than the children with HAs. Children with HAs and children with CIs also showed delayed fixations to the semantically related item, although this delay was attributable to their delay in activating words in general, not to a distinct semantic source.

CONCLUSIONS

Children with HAs and children with CIs showed qualitatively similar patterns of real-time spoken word recognition. Findings suggest that developing spoken language via degraded auditory input causes long-term cognitive adaptations to how listeners recognize spoken words, regardless of the type of hearing device used. Delayed lexical access directly led to delays in spreading semantic activation in children with HAs and CIs. This delay in semantic processing may impact these children's ability to understand connected speech in everyday life.

Listen up! ADHD slows spoken-word processing in adverse listening conditions: Evidence from eye movements

BACKGROUND

Cognitive skills such as sustained attention, inhibition and working memory are essential for speech processing, yet are often impaired in people with ADHD. Offline measures have indicated difficulties in speech recognition on multi-talker babble (MTB) background for young adults with ADHD (yaADHD). However, to-date no study has directly tested online speech processing in adverse conditions for yaADHD.

AIMS

Gauging the effects of ADHD on segregating the spoken target-word from its sound-sharing competitor, in MTB and working-memory (WM) load.

METHODS AND PROCEDURES

Twenty-four yaADHD and 22 matched controls that differ in sustained attention (SA) but not in WM were asked to follow spoken instructions presented on MTB to touch a named object, while retaining one (low-load) or four (high-load) digit/s for later recall. Their eye fixations were tracked.

OUTCOMES AND RESULTS

In the high-load condition, speech processing was less accurate and slowed by 140ms for yaADHD. In the low-load condition, the processing advantage shifted from early perceptual to later cognitive stages. Fixation transitions (hesitations) were inflated for yaADHD.

CONCLUSIONS AND IMPLICATIONS

ADHD slows speech processing in adverse listening conditions and increases hesitation, as speech unfolds in time. These effects, detected only by online eyetracking, relate to attentional difficulties. We suggest online speech processing as a novel purview on ADHD. WHAT THIS PAPER ADDS?: We suggest speech processing in adverse listening conditions as a novel vantage point on ADHD. Successful speech recognition in noise is essential for performance across daily settings: academic, employment and social interactions. It involves several executive functions, such as inhibition and sustained attention. Impaired performance in these functions is characteristic of ADHD. However, to date there is only scant research on speech processing in ADHD. The current study is the first to investigate online speech processing as the word unfolds in time using eyetracking for young adults with ADHD (yaADHD). This method uncovered slower speech processing in multi-talker babble noise for yaADHD compared to matched controls. The performance of yaADHD indicated increased hesitation between the spoken word and sound-sharing alternatives (e.g., CANdle-CANdy). These delays and hesitations, on the single word level, could accumulate in continuous speech to significantly impair communication in ADHD, with severe implications on their quality of life and academic success. Interestingly, whereas yaADHD and controls were matched on WM standardized tests, WM load appears to affect speech processing for yaADHD more than for controls. This suggests that ADHD may lead to inefficient deployment of WM resources that may not be detected when WM is tested alone. Note that these intricate differences could not be detected using traditional offline accuracy measures, further supporting the use of eyetracking in speech tasks. Finally, communication is vital for active living and wellbeing. We suggest paying attention to speech processing in ADHD in treatment and when considering accessibility and inclusion.

I'm not sure that curve means what you think it means: Toward a [more] realistic understanding of the role of eye-movement generation in the Visual World Paradigm

The Visual World Paradigm (VWP) is a powerful experimental paradigm for language research. Listeners respond to speech in a "visual world" containing potential referents of the speech. Fixations to these referents provides insight into the preliminary states of language processing as decisions unfold. The VWP has become the dominant paradigm in psycholinguistics and extended to every level of language, development, and disorders. Part of its impact is the impressive data visualizations which reveal the millisecond-by-millisecond time course of processing, and advances have been made in developing new analyses that precisely characterize this time course. All theoretical and statistical approaches make the tacit assumption that the time course of fixations is closely related to the underlying activation in the system. However, given the serial nature of fixations and their long refractory period, it is unclear how closely the observed dynamics of the fixation curves are actually coupled to the underlying dynamics of activation. I investigated this assumption with a series of simulations. Each simulation starts with a set of true underlying activation functions and generates simulated fixations using a simple stochastic sampling procedure that respects the sequential nature of fixations. I then analyzed the results to determine the conditions under which the observed fixations curves match the underlying functions, the reliability of the observed data, and the implications for Type I error and power. These simulations demonstrate that even under the simplest fixation-based models, observed fixation curves are systematically biased relative to the underlying activation functions, and they are substantially noisier, with important implications for reliability and power. I then present a potential generative model that may ultimately overcome many of these issues.

On the definition of noise

Urbanization has exposed people to extreme sound levels. Although researchers have investigated the ability of people to listen, analyze, and distinguish sound, the concept of noise has not been clearly articulated from a human perspective. The lack of knowledge on how people perceive noise limits our capacity to control it in a targeted manner. This study aimed to interpret the definition of noise from the public perspective based on a grounded theory approach. Seventy-eight participants were interviewed about noise, and four categories of perceived understanding of noise were identified: challenges, definitions of noise, opportunities, and action. As one of the challenges, urbanization is associated with increased noise levels around the human environment. In terms of definition, perceiving sound as noise is considered to be a result of the complex and dynamic process that includes sound, the environment, and humans. Sound and humans interact with the environment. In terms of opportunities, noise may have positive roles on certain occasions, dispelling the misconception that noise is exclusively negative. In addition, we found that noise perception has gradually shifted from noise control to noise utilization. In terms of action, noise can be controlled at the sound sources, susceptible target groups, susceptible behaviors and states, locations, and times where noise is perceived with high frequency. In this study, we investigated several aspects of noise, ranging from noise control, soundscape definition, and 'soundscape indices' (SSID) integration and application. Our findings provide an additional basis for developing better definitions, control, and utilization strategies of noise in the future, thereby improving the quality of the sound environment.

Decoding the temporal dynamics of spoken word and nonword processing from EEG

The efficiency of spoken word recognition is essential for real-time communication. There is consensus that this efficiency relies on an implicit process of activating multiple word candidates that compete for recognition as the acoustic signal unfolds in real-time. However, few methods capture the neural basis of this dynamic competition on a msec-by-msec basis. This is crucial for understanding the neuroscience of language, and for understanding hearing, language and cognitive disorders in people for whom current behavioral methods are not suitable. We applied machine-learning techniques to standard EEG signals to decode which word was heard on each trial and analyzed the patterns of confusion over time. Results mirrored psycholinguistic findings: Early on, the decoder was equally likely to report the target (e.g., baggage) or a similar sounding competitor (badger), but by around 500 msec, competitors were suppressed. Follow up analyses show that this is robust across EEG systems (gel and saline), with fewer channels, and with fewer trials. Results are robust within individuals and show high reliability. This suggests a powerful and simple paradigm that can assess the neural dynamics of speech decoding, with potential applications for understanding lexical development in a variety of clinical disorders.

Speech Processing as a Far-Transfer Gauge of Serious Games for Cognitive Training in Aging: Randomized Controlled Trial of Web-Based Effectivate Training

BACKGROUND

The number of serious games for cognitive training in aging (SGCTAs) is proliferating in the market and attempting to combat one of the most feared aspects of aging-cognitive decline. However, the efficacy of many SGCTAs is still questionable. Even the measures used to validate SGCTAs are up for debate, with most studies using cognitive measures that gauge improvement in trained tasks, also known as near transfer. This study takes a different approach, testing the efficacy of the SGCTA-Effectivate-in generating tangible far-transfer improvements in a nontrained task-the Eye tracking of Word Identification in Noise Under Memory Increased Load (E-WINDMIL)-which tests speech processing in adverse conditions.

OBJECTIVE

This study aimed to validate the use of a real-time measure of speech processing as a gauge of the far-transfer efficacy of an SGCTA designed to train executive functions.

METHODS

In a randomized controlled trial that included 40 participants, we tested 20 (50%) older adults before and after self-administering the SGCTA Effectivate training and compared their performance with that of the control group of 20 (50%) older adults. The E-WINDMIL eye-tracking task was administered to all participants by blinded experimenters in 2 sessions separated by 2 to 8 weeks.

RESULTS

Specifically, we tested the change between sessions in the efficiency of segregating the spoken target word from its sound-sharing alternative, as the word unfolds in time. We found that training with the SGCTA Effectivate improved both early and late speech processing in adverse conditions, with higher discrimination scores in the training group than in the control group (early processing: F_1,38=7.371; P=.01; η_p²=0.162 and late processing: F_1,38=9.003; P=.005; η_p²=0.192).

CONCLUSIONS

This study found the E-WINDMIL measure of speech processing to be a valid gauge for the far-transfer effects of executive function training. As the SGCTA Effectivate does not train any auditory task or language processing, our results provide preliminary support for the ability of Effectivate to create a generalized cognitive improvement. Given the crucial role of speech processing in healthy and successful aging, we encourage researchers and developers to use speech processing measures, the E-WINDMIL in particular, to gauge the efficacy of SGCTAs. We advocate for increased industry-wide adoption of far-transfer metrics to gauge SGCTAs.

Pragmatic inferences in aging and human-robot communication

Despite the increase in research on older adults' communicative behavior, little work has explored patterns of age-related change in pragmatic inferencing and how these patterns are adapted depending on the situation-specific context. In two eye-tracking experiments, participants followed instructions like "Click on the greenhouse", which were either played over speakers or spoken live by a co-present robot partner. Implicit inferential processes were measured by exploring the extent to which listeners temporarily (mis)understood the unfolding noun to be a modified phrase referring to a competitor object in the display (green hat). This competitor was accompanied by either another member of the same category or an unrelated item (tan hat vs. dice). Experiment 1 (no robot) showed clear evidence of contrastive inferencing in both younger and older adults (more looks to the green hat when the tan hat was also present). Experiment 2 explored the ability to suppress these contrastive inferences when the robot talker was known to lack any color perception, making descriptions like "green hat" implausible. Younger but not older listeners were able to suppress contrastive inferences in this context, suggesting older adults could not keep the relevant limitations in mind and/or were more likely to spontaneously ascribe human attributes to the robot. Together, the findings enhance our understanding of pragmatic inferencing in aging.

Age-Related Changes in the Perception of Emotions in Speech: Assessing Thresholds of Prosody and Semantics Recognition in Noise for Young and Older Adults

Older adults process emotions in speech differently than do young adults. However, it is unclear whether these age-related changes impact all speech channels to the same extent, and whether they originate from a sensory or a cognitive source. The current study adopted a psychophysical approach to directly compare young and older adults’ sensory thresholds for emotion recognition in two channels of spoken-emotions: prosody (tone) and semantics (words). A total of 29 young adults and 26 older adults listened to 50 spoken sentences presenting different combinations of emotions across prosody and semantics. They were asked to recognize the prosodic or semantic emotion, in separate tasks. Sentences were presented on the background of speech-spectrum noise ranging from SNR of −15 dB (difficult) to +5 dB (easy). Individual recognition thresholds were calculated (by fitting psychometric functions) separately for prosodic and semantic recognition. Results indicated that: (1). recognition thresholds were better for young over older adults, suggesting an age-related general decrease across channels; (2). recognition thresholds were better for prosody over semantics, suggesting a prosodic advantage; (3). importantly, the prosodic advantage in thresholds did not differ between age groups (thus a sensory source for age-related differences in spoken-emotions processing was not supported); and (4). larger failures of selective attention were found for older adults than for young adults, indicating that older adults experienced larger difficulties in inhibiting irrelevant information. Taken together, results do not support a sole sensory source, but rather an interplay of cognitive and sensory sources for age-related differences in spoken-emotions processing.

Children’s and adults’ use of fictional discourse and semantic knowledge for prediction in language processing

Using real-time eye-movement measures, we asked how a fantastical discourse context competes with stored representations of real-world events to influence the moment-by-moment interpretation of a story by 7-year-old children and adults. Seven-year-olds were less effective at bypassing stored real-world knowledge during real-time interpretation than adults. Our results suggest that children privilege stored semantic knowledge over situation-specific information presented in a fictional story context. We suggest that 7-year-olds’ canonical semantic and conceptual relations are sufficiently strongly rooted in statistical patterns in language that have consolidated over time that they overwhelm new and unexpected information even when the latter is fantastical and highly salient.

One Size Does Not Fit All: Examining the Effects of Working Memory Capacity on Spoken Word Recognition in Older Adults Using Eye Tracking

Difficulties understanding speech form one of the most prevalent complaints among older adults. Successful speech perception depends on top-down linguistic and cognitive processes that interact with the bottom-up sensory processing of the incoming acoustic information. The relative roles of these processes in age-related difficulties in speech perception, especially when listening conditions are not ideal, are still unclear. In the current study, we asked whether older adults with a larger working memory capacity process speech more efficiently than peers with lower capacity when speech is presented in noise, with another task performed in tandem. Using the Eye-tracking of Word Identification in Noise Under Memory Increased Load (E-WINDMIL) an adapted version of the "visual world" paradigm, 36 older listeners were asked to follow spoken instructions presented in background noise, while retaining digits for later recall under low (single-digit) or high (four-digits) memory load. In critical trials, instructions (e.g., "point at the candle") directed listeners' gaze to pictures of objects whose names shared onset or offset sounds with the name of a competitor that was displayed on the screen at the same time (e.g., candy or sandal). We compared listeners with different memory capacities on the time course for spoken word recognition under the two memory loads by testing eye-fixations on a named object, relative to fixations on an object whose name shared phonology with the named object. Results indicated two trends. (1) For older adults with lower working memory capacity, increased memory load did not affect online speech processing, however, it impaired offline word recognition accuracy. (2) The reverse pattern was observed for older adults with higher working memory capacity: increased task difficulty significantly decreases online speech processing efficiency but had no effect on offline word recognition accuracy. Results suggest that in older adults, adaptation to adverse listening conditions is at least partially supported by cognitive reserve. Therefore, additional cognitive capacity may lead to greater resilience of older listeners to adverse listening conditions. The differential effects documented by eye movements and accuracy highlight the importance of using both online and offline measures of speech processing to explore age-related changes in speech perception.

Linguistic Redundancy and its Effects on Younger and Older Adults' Real-Time Comprehension and Memory

Redundant modifiers can facilitate referential interpretation by narrowing attention to intended referents. This is intriguing because, on traditional accounts, redundancy should impair comprehension. Little is known, however, about the effects of redundancy on older adults' comprehension. Older adults may show different patterns due to age-related decline (e.g., processing speed and memory) or their greater proclivity for linguistic redundancy, as suggested in language production studies. The present study explores the effects of linguistic redundancy on younger and older listeners' incremental referential processing, judgments of informativity, and downstream memory performance. In an eye tracking task, gaze was monitored as listeners followed instructions from a social robot referring to a unique object within a multi-object display. Critical trials were varied in terms of modifier type ("…closed/purple/[NONE] umbrella") and whether displays contained another object matching target properties (closed purple notebook), making modifiers less effective at narrowing attention. Relative to unmodified descriptions, redundant color modifiers facilitated comprehension, particularly when they narrowed attention to a single referent. Descriptions with redundant state modifiers always impaired real-time comprehension. In contrast, memory measures showed faster recognition of objects previously described with redundant state modifiers. Although color and state descriptions had different effects on referential processing and memory, informativity judgments showed participants perceived them as informationally redundant to the same extent relative to unmodified descriptions. Importantly, the patterns did not differ by listener age. Together, the results show that the effects of linguistic redundancy are stable across adulthood but vary as a function of modifier type, visual context, and the measured phenomenon.

Effects of word familiarity and receptive vocabulary size on speech-in-noise recognition among young adults with normal hearing

Having a large receptive vocabulary benefits speech-in-noise recognition for young children, though this is not always the case for older children or adults. These observations could indicate that effects of receptive vocabulary size on speech-in-noise recognition differ depending on familiarity of the target words, with effects observed only for more recently acquired and less frequent words. Two experiments were conducted to evaluate effects of vocabulary size on open-set speech-in-noise recognition for adults with normal hearing. Targets were words acquired at 4, 9, 12 and 15 years of age, and they were presented at signal-to-noise ratios (SNRs) of -5 and -7 dB. Percent correct scores tended to fall with increasing age of acquisition (AoA), with the caveat that performance at -7 dB SNR was better for words acquired at 9 years of age than earlier- or later-acquired words. Similar results were obtained whether the AoA of the target words was blocked or mixed across trials. Differences in word duration appear to account for nonmonotonic effects of AoA. For all conditions, a positive correlation was observed between recognition and vocabulary size irrespective of target word AoA, indicating that effects of vocabulary size are not limited to recently acquired words. This dataset does not support differential assessment of AoA, lexical frequency, and other stimulus features known to affect lexical access.

Synchrony in Old Age: Playing the Mirror Game Improves Cognitive Performance

OBJECTIVES

Studies have shown that synchronized motion between people positively affects a range of emotional and social functions. The mirror-game is a synchrony-based paradigm, common to theater, performance arts, and therapy, which includes dyadic synchronized motion, playfulness, and spontaneity. The goal of the current study is to examine the effects of the mirror-game on subjective and cognitive indices in late life.

METHODS

Thirty-four older adults (aged 71-98) participated in a within-group study design. Participants conducted two sessions of 9-minute movement activities: the mirror-game and the control condition - a physical exercise class. Several measures were taken before and after experimental sessions to assess socio-emotional and attentional functions.

RESULTS

The mirror-game enhanced performance on the attention sub-scale and led to faster detections of spoken words in noise. Further, it enhanced perceived partner responsiveness and led to an increase in positive reported experience.

CONCLUSIONS

Our preliminary findings suggest that the mirror-game, rather than the exercise class, may have an immediate impact on mood and some attentional functions.

CLINICAL IMPLICATIONS

The mirror-game is a novel intervention, with potential benefits of social-emotional and cognitive functioning, which can be easily implemented into the daily routine care of older adults. Future studies should explore the effect of the mirror-game on additional cognitive and socio-emotional aspects.

The Presence of Background Noise Extends the Competitor Space in Native and Non-Native Spoken-Word Recognition: Insights from Computational Modeling

Oral communication often takes place in noisy environments, which challenge spoken‐word recognition. Previous research has suggested that the presence of background noise extends the number of candidate words competing with the target word for recognition and that this extension affects the time course and accuracy of spoken‐word recognition. In this study, we further investigated the temporal dynamics of competition processes in the presence of background noise, and how these vary in listeners with different language proficiency (i.e., native and non‐native) using computational modeling. We developed ListenIN (Listen‐In‐Noise), a neural‐network model based on an autoencoder architecture, which learns to map phonological forms onto meanings in two languages and simulates native and non‐native spoken‐word comprehension. We also examined the model's activation states during online spoken‐word recognition. These analyses demonstrated that the presence of background noise increases the number of competitor words, which are engaged in phonological competition and that this happens in similar ways intra and interlinguistically and in native and non‐native listening. Taken together, our results support accounts positing a “many‐additional‐competitors scenario” for the effects of noise on spoken‐word recognition.

More Than Words: the Relative Roles of Prosody and Semantics in the Perception of Emotions in Spoken Language by Postlingual Cochlear Implant Users

OBJECTIVES

The processing of emotional speech calls for the perception and integration of semantic and prosodic cues. Although cochlear implants allow for significant auditory improvements, they are limited in the transmission of spectro-temporal fine-structure information that may not support the processing of voice pitch cues. The goal of the current study is to compare the performance of postlingual cochlear implant (CI) users and a matched control group on perception, selective attention, and integration of emotional semantics and prosody.

DESIGN

Fifteen CI users and 15 normal hearing (NH) peers (age range, 18-65 years) 1istened to spoken sentences composed of different combinations of four discrete emotions (anger, happiness, sadness, and neutrality) presented in prosodic and semantic channels-T-RES: Test for Rating Emotions in Speech. In three separate tasks, listeners were asked to attend to the sentence as a whole, thus integrating both speech channels (integration), or to focus on one channel only (rating of target emotion) and ignore the other (selective attention). Their task was to rate how much they agreed that the sentence conveyed each of the predefined emotions. In addition, all participants performed standard tests of speech perception.

RESULTS

When asked to focus on one channel, semantics or prosody, both groups rated emotions similarly with comparable levels of selective attention. When the task was called for channel integration, group differences were found. CI users appeared to use semantic emotional information more than did their NH peers. CI users assigned higher ratings than did their NH peers to sentences that did not present the target emotion, indicating some degree of confusion. In addition, for CI users, individual differences in speech comprehension over the phone and identification of intonation were significantly related to emotional semantic and prosodic ratings, respectively.

CONCLUSIONS

CI users and NH controls did not differ in perception of prosodic and semantic emotions and in auditory selective attention. However, when the task called for integration of prosody and semantics, CI users overused the semantic information (as compared with NH). We suggest that as CI users adopt diverse cue weighting strategies with device experience, their weighting of prosody and semantics differs from those used by NH. Finally, CI users may benefit from rehabilitation strategies that strengthen perception of prosodic information to better understand emotional speech.

Lexical Access Changes Based on Listener Needs: Real-Time Word Recognition in Continuous Speech in Cochlear Implant Users

OBJECTIVES

A key challenge in word recognition is the temporary ambiguity created by the fact that speech unfolds over time. In normal hearing (NH) listeners, this temporary ambiguity is resolved through incremental processing and competition among lexical candidates. Post-lingually deafened cochlear implant (CI) users show similar incremental processing and competition but with slight delays. However, even brief delays could lead to drastic changes when compounded across multiple words in a phrase. This study asks whether words presented in non-informative continuous speech (a carrier phrase) are processed differently than in isolation and whether NH listeners and CI users exhibit different effects of a carrier phrase.

DESIGN

In a Visual World Paradigm experiment, listeners heard words either in isolation or in non-informative carrier phrases (e.g., "click on the…" ). Listeners selected the picture corresponding to the target word from among four items including the target word (e.g., mustard ), a cohort competitor (e.g., mustache ), a rhyme competitor (e.g., custard ), and an unrelated item (e.g., penguin ). Eye movements were tracked as an index of the relative activation of each lexical candidate as competition unfolds over the course of word recognition. Participants included 21 post-lingually deafened cochlear implant users and 21 NH controls. A replication experiment presented in the Supplemental Digital Content, http://links.lww.com/EANDH/A999 included an additional 22 post-lingually deafened CI users and 18 NH controls.

RESULTS

Both CI users and the NH controls were accurate at recognizing the words both in continuous speech and in isolation. The time course of lexical activation (indexed by the fixations) differed substantially between groups. CI users were delayed in fixating the target relative to NH controls. Additionally, CI users showed less competition from cohorts than NH controls (even as previous studies have often report increased competition). However, CI users took longer to suppress the cohort and suppressed it less fully than the NH controls. For both CI users and NH controls, embedding words in carrier phrases led to more immediacy in lexical access as observed by increases in cohort competition relative to when words were presented in isolation. However, CI users were not differentially affected by the carriers.

CONCLUSIONS

Unlike prior work, CI users appeared to exhibit "wait-and-see" profile, in which lexical access is delayed minimizing early competition. However, CI users simultaneously sustained competitor activation late in the trial, possibly to preserve flexibility. This hybrid profile has not been observed previously. When target words are heard in continuous speech, both CI users and NH controls more heavily weight early information. However, CI users (but not NH listeners) also commit less fully to the target, potentially keeping options open if they need to recover from a misperception. This mix of patterns reflects a lexical system that is extremely flexible and adapts to fit the needs of a listener.

Effect of Noise Reduction on Cortical Speech-in-Noise Processing and Its Variance due to Individual Noise Tolerance

OBJECTIVES

Despite the widespread use of noise reduction (NR) in modern digital hearing aids, our neurophysiological understanding of how NR affects speech-in-noise perception and why its effect is variable is limited. The current study aimed to (1) characterize the effect of NR on the neural processing of target speech and (2) seek neural determinants of individual differences in the NR effect on speech-in-noise performance, hypothesizing that an individual's own capability to inhibit background noise would inversely predict NR benefits in speech-in-noise perception.

DESIGN

Thirty-six adult listeners with normal hearing participated in the study. Behavioral and electroencephalographic responses were simultaneously obtained during a speech-in-noise task in which natural monosyllabic words were presented at three different signal-to-noise ratios, each with NR off and on. A within-subject analysis assessed the effect of NR on cortical evoked responses to target speech in the temporal-frontal speech and language brain regions, including supramarginal gyrus and inferior frontal gyrus in the left hemisphere. In addition, an across-subject analysis related an individual's tolerance to noise, measured as the amplitude ratio of auditory-cortical responses to target speech and background noise, to their speech-in-noise performance.

RESULTS

At the group level, in the poorest signal-to-noise ratio condition, NR significantly increased early supramarginal gyrus activity and decreased late inferior frontal gyrus activity, indicating a switch to more immediate lexical access and less effortful cognitive processing, although no improvement in behavioral performance was found. The across-subject analysis revealed that the cortical index of individual noise tolerance significantly correlated with NR-driven changes in speech-in-noise performance.

CONCLUSIONS

NR can facilitate speech-in-noise processing despite no improvement in behavioral performance. Findings from the current study also indicate that people with lower noise tolerance are more likely to get more benefits from NR. Overall, results suggest that future research should take a mechanistic approach to NR outcomes and individual noise tolerance.

Listening Effort Measured Using a Dual-task Paradigm in Adults With Different Amounts of Noise Exposure

OBJECTIVES

Excessive noise levels can result in hearing damage and/or hearing-related symptoms, thereby leading to impaired communication and, eventually a decrease in the quality of life. Moreover, in daily practice, subjects often indicate that listening in noisy situations is a challenging and often exhausting experience, even in the absence of a clinically significant hearing loss. Hence, a person's perceived difficulty of the listening situation will also be important to consider. It has been suggested that beyond the peripheral factors, there are some central cognitive correlates of speech understanding that are essential for communication abilities. The aim of the present study was to evaluate the effect of the amount of noise exposure on hearing as measured by pure-tone audiometry and otoacoustic emissions (OAEs) on the one hand and listening effort measured using a dual-task paradigm on the other hand.

DESIGN

The study included 152 adults between 18 and 40 years of age. First, participants completed a self-administered questionnaire regarding the amount of noise exposure. Second, hearing status was evaluated using pure-tone audiometry and transient evoked OAEs (TEOAEs) as well as distortion product OAEs (DPOAEs). Finally, listening effort was evaluated using a dual-task paradigm, which consisted of a primary speech-recognition task in different listening conditions and a secondary visual memory task that were performed both separately and simultaneously. Based on the quartiles of their subjective estimation of noise exposure, the participants were categorized into a group with low noise exposure (lower quartile), moderate noise exposure (two middle quartiles), and high noise exposure (upper quartile).

RESULTS

There was no significant difference in hearing thresholds between the participants with low, moderate, and high noise exposure for each frequency of the pure-tone audiometry. In contrast, more absent TEOAEs and DPOAEs were found in the high noise exposed group. Regarding the primary speech-recognition task of the dual-task paradigm, no significant difference in speech recognition was found between the different groups of noise exposure. For the amount of listening effort, it was found that across all listening conditions subjects with high noise exposure expend significantly more listening effort compared with subjects with low and moderate noise exposure.

CONCLUSIONS

This study is a first exploration of the effects of different amounts of noise exposure on listening effort showing that, listening effort is increased in subjects with high noise exposure compared with subjects with low and medium noise exposure. The most plausible hypothesis pertains to an effect of noise exposure on the peripheral and central auditory system, or a combination of effects on both the auditory system and the high-level cognitive functions necessary for speech understanding in the subjects with high levels of noise exposure. As such, a test for listening effort would be useful as part of a comprehensive test battery within the assessment of subjects exposed to noise.

The Recognition of Whispered Speech in Real-Time

OBJECTIVES

Whispered speech offers a unique set of challenges to speech perception and word recognition. The goals of the present study were twofold: First, to determine how listeners recognize whispered speech. Second, to inform major theories of spoken word recognition by considering how recognition changes when major cues to phoneme identity are reduced or largely absent compared with normal voiced speech.

DESIGN

Using eye tracking in the Visual World Paradigm, we examined how listeners recognize whispered speech. After hearing a target word (normal or whispered), participants selected the corresponding image from a display of four-a target (e.g., money), a word that shares sounds with the target at the beginning (cohort competitor, e.g., mother), a word that shares sounds with the target at the end (rhyme competitor, e.g., honey), and a phonologically unrelated word (e.g., whistle). Eye movements to each object were monitored to measure (1) how fast listeners process whispered speech, and (2) how strongly they consider lexical competitors (cohorts and rhymes) as the speech signal unfolds.

RESULTS

Listeners were slower to recognize whispered words. Compared with normal speech, listeners displayed slower reaction times to click the target image, were slower to fixate the target, and fixated the target less overall. Further, we found clear evidence that the dynamics of lexical competition are altered during whispered speech recognition. Relative to normal speech, words that overlapped with the target at the beginning (cohorts) displayed slower, reduced, and delayed activation, whereas words that overlapped with the target at the end (rhymes) exhibited faster, more robust, and longer lasting activation.

CONCLUSION

When listeners are confronted with whispered speech, they engage in a "wait-and-see" approach. Listeners delay lexical access, and by the time they begin to consider what word they are hearing, the beginning of the word has largely come and gone, and activation for cohorts is reduced. However, delays in lexical access actually increase consideration of rhyme competitors; the delay pushes lexical activation to a point later in processing, and the recognition system puts more weight on the word-final overlap between the target and the rhyme.

Cognitive and Physiological Measures of Listening Effort During Degraded Speech Perception: Relating Dual-Task and Pupillometry Paradigms

Purpose Listening effort is quickly becoming an important metric for assessing speech perception in less-than-ideal situations. However, the relationship between the construct of listening effort and the measures used to assess it remains unclear. We compared two measures of listening effort: a cognitive dual task and a physiological pupillometry task. We sought to investigate the relationship between these measures of effort and whether engaging effort impacts speech accuracy. Method In Experiment 1, 30 participants completed a dual task and a pupillometry task that were carefully matched in stimuli and design. The dual task consisted of a spoken word recognition task and a visual match-to-sample task. In the pupillometry task, pupil size was monitored while participants completed a spoken word recognition task. Both tasks presented words at three levels of listening difficulty (unmodified, eight-channel vocoding, and four-channel vocoding) and provided response feedback on every trial. We refined the pupillometry task in Experiment 2 (n = 31); crucially, participants no longer received response feedback. Finally, we ran a new group of subjects on both tasks in Experiment 3 (n = 30). Results In Experiment 1, accuracy in the visual task decreased with increased signal degradation in the dual task, but pupil size was sensitive to accuracy and not vocoding condition. After removing feedback in Experiment 2, changes in pupil size were predicted by listening condition, suggesting the task was now sensitive to engaged effort. Both tasks were sensitive to listening difficulty in Experiment 3, but there was no relationship between the tasks and neither task predicted speech accuracy. Conclusions Consistent with previous work, we found little evidence for a relationship between different measures of listening effort. We also found no evidence that effort predicts speech accuracy, suggesting that engaging more effort does not lead to improved speech recognition. Cognitive and physiological measures of listening effort are likely sensitive to different aspects of the construct of listening effort. Supplemental Material https://doi.org/10.23641/asha.16455900.

Keep it simple: Identification of basic versus complex emotions in spoken language in individuals with autism spectrum disorder without intellectual disability: A meta-analysis study

Daily functioning involves identifying emotions in spoken language, a fundamental aspect of social interactions. To date, there is inconsistent evidence in the literature on whether individuals with autism spectrum disorder without intellectual disability (ASD-without-ID) experience difficulties in identification of spoken emotions. We conducted a meta-analysis (literature search following the PRISMA guidelines), with 26 data sets (taken from 23 peer-reviewed journal articles) comparing individuals with ASD-without-ID (N = 614) and typically-developed (TD) controls (N = 640), from nine countries and in seven languages (published until February 2020). In our analyses there was no sufficient evidence to suggest that individuals with HF-ASD differ from matched controls in the identification of simple prosodic emotions (e.g., sadness, happiness). However, individuals with ASD-without-ID were found to perform significantly worse than controls in identification of complex prosodic emotions (e.g., envy and boredom). The level of the semantic content of the stimuli presented (e.g., sentences vs. strings of digits) was not found to have an impact on the results. In conclusion, the difference in findings between simple and complex emotions calls for a new-look on emotion processing in ASD-without-ID. Intervention programs may rely on the intact abilities of individuals with ASD-without-ID to process simple emotions and target improved performance with complex emotions. LAY SUMMARY: Individuals with autism spectrum disorder without intellectual disability (ASD-without-ID) do not differ from matched controls in the identification of simple prosodic emotions (e.g., sadness, happiness). However, they were found to perform significantly worse than controls in the identification of complex prosodic emotions (e.g., envy, boredom). This was found in a meta-analysis of 26 data sets with 1254 participants from nine countries and in seven languages. Intervention programs may rely on the intact abilities of individuals with ASD-without-ID to process simple emotions.

Speech aging: Production and perception

In this overview we describe literature on how speech production and speech perception change in healthy or normal aging across the adult lifespan. In the production section we review acoustic characteristics that have been investigated as potentially distinguishing younger and older adults. In the speech perception section studies concerning speaker age estimation and those investigating older listeners' perception are addressed. Our discussion focuses on major themes and other fruitful areas for future research. This article is categorized under: Linguistics > Language in Mind and Brain Linguistics > Linguistic Theory Psychology > Development and Aging.

Pre- and post-target cortical processes predict speech-in-noise performance

Understanding speech in noise (SiN) is a complex task that recruits multiple cortical subsystems. There is a variance in individuals' ability to understand SiN that cannot be explained by simple hearing profiles, which suggests that central factors may underlie the variance in SiN ability. Here, we elucidated a few cortical functions involved during a SiN task and their contributions to individual variance using both within- and across-subject approaches. Through our within-subject analysis of source-localized electroencephalography, we investigated how acoustic signal-to-noise ratio (SNR) alters cortical evoked responses to a target word across the speech recognition areas, finding stronger responses in left supramarginal gyrus (SMG, BA40 the dorsal lexicon area) with quieter noise. Through an individual differences approach, we found that listeners show different neural sensitivity to the background noise and target speech, reflected in the amplitude ratio of earlier auditory-cortical responses to speech and noise, named as an internal SNR. Listeners with better internal SNR showed better SiN performance. Further, we found that the post-speech time SMG activity explains a further amount of variance in SiN performance that is not accounted for by internal SNR. This result demonstrates that at least two cortical processes contribute to SiN performance independently: pre-target time processing to attenuate neural representation of background noise and post-target time processing to extract information from speech sounds.

Age-Related Differences in the Online Processing of Spoken Semantic Context and the Effect of Semantic Competition: Evidence From Eye Gaze

Purpose The study examined age-related differences in the use of semantic context and in the effect of semantic competition in spoken sentence processing. We used offline (response latency) and online (eye gaze) measures, using the "visual world" eye-tracking paradigm. Method Thirty younger and 30 older adults heard sentences related to one of four images presented on a computer monitor. They were asked to touch the image corresponding to the final word of the sentence (target word). Three conditions were used: a nonpredictive sentence, a predictive sentence suggesting one of the four images on the screen (semantic context), and a predictive sentence suggesting two possible images (semantic competition). Results Online eye gaze data showed no age-related differences with nonpredictive sentences, but revealed slowed processing for older adults when context was presented. With the addition of semantic competition to context, older adults were slower to look at the target word after it had been heard. In contrast, offline latency analysis did not show age-related differences in the effects of context and competition. As expected, older adults were generally slower to touch the image than younger adults. Conclusions Traditional offline measures were not able to reveal the complex effect of aging on spoken semantic context processing. Online eye gaze measures suggest that older adults were slower than younger adults to predict an indicated object based on semantic context. Semantic competition affected online processing for older adults more than for younger adults, with no accompanying age-related differences in latency. This supports an early age-related inhibition deficit, interfering with processing, and not necessarily with response execution.

School-Age Children Adapt the Dynamics of Lexical Competition in Suboptimal Listening Conditions

Although the ability to understand speech in adverse listening conditions is paramount for effective communication across the life span, little is understood about how this critical processing skill develops. This study asks how the dynamics of spoken word recognition (i.e., lexical access and competition) change during soft speech in 8- to 11-year-olds (n = 26). Lexical competition and access for speech at lower intensity levels was measured using eye-tracking and the visual world paradigm. Overall the results suggest that soft speech influences the magnitude and timing of lexical access and competition. These results suggest that lexical competition is a cognitive process that can be adapted in the school-age years to help cope with increased uncertainty due to alterations in the speech signal.

Focusing on Emotional and Social Intelligence Stimulation of People With Dementia by Playing a Serious Game—Proof of Concept Study

Context: Dementia is one of the top five chronic diseases, which has an overwhelming impact on patients' life quality, family, and caregivers. Currently, research relating to people with dementia (PwD) focuses on the deterioration of cognitive abilities. A more innovative approach, and the one taken by this paper, is focusing on methods to maintain and improve functionality, communication and quality of life of PwD by building on remaining capacities in the yet unexplored domain of emotional and social intelligence (ESI). The use of serious games for PwD (SG4D) aimed at building social and emotional capacity is a budding field of research.

Objectives: Proof of concept that the, low cost, easy to deploy SG4D, called “My Brain Works” (MBW), co-designed with PwD, enhances ESI, based on the Bar-On ESI model.

Methods: 27 PwD, clients at MELABEV dementia day center, participated in a mixed methods 12 weeks pilot, proof of concept study using a tablet SG4D co-designed with PwD. Quantitative performance data was collected automatically by the tablet during game sessions. In this paper we focus on the analysis of the qualitative and quantitative data related to ESI, observed by 10 different researchers, during each game session.

Results: Quantitative data revealed: both the PwD with high and low MoCA scores had similar average ESI scores. Qualitative analysis revealed that the PwD demonstrated 9 sub-components of the Bar-On ESI Model.

Conclusion: While there is no drug to stop cognitive decline associated with dementia, interventions related to ESI, on the other hand, may improve functioning and quality of life. Despite declines in cognitive abilities, our study shows that a tablet based SG4D can stimulate their ESI and evoke responses in self-awareness, empathy, social and communication capacities. Using SG4D to exercise and maintain social skills is an area that may be promising in the future and may help counter the negative effects of social isolation and loneliness. Such games, while not focusing on cognitive improvement, may also impact on cognitive functioning and help bridge the gap between caregiver and PwD. More research is needed with larger sample sizes.

Task-irrelevant Auditory Event-related Potentials as Mental Workload Indicators: A Between-task Comparison Study. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2020;2020:3216-3219. [PMID: 33018689 DOI: 10.1109/embc44109.2020.9175957]

Real-time monitoring of mental workload (MWL) is a crucial step to build closed-loop adaptive aiding systems for human-machine systems. MWL estimators based on spontaneous electroencephalography (EEG) and event-related potentials (ERPs) have shown great potentials to achieve this goal. However, the previous studies show that the between-task robustness of these EEG/ERP-based MWL estimators is still an unsolved intractable question. This study attempts to examine the task-irrelevant auditory event-related potentials (tir-aERPs) as MWL indicators. A working memory task (verbal n-back) and a visuo-motor task (multi-attribute task battery, MATB), both with two difficulty levels (easy and hard), were used in the experiment, along with task-irrelevant auditory probes that did not need any response from the participants. EEG was recorded from ten participants when they were performing the tasks. The tir-aERPs elicited by the auditory probes were extracted and analyzed. The results show that the amplitudes of N1, early P3a (eP3a) and the late reorienting negativity (RON) significantly decreased with the increasing MWL in both n-back and MATB. Task type has no obvious influence on the amplitudes and topological layout of the MWL-sensitive tir-aERPs features. These results suggest that the tir-aERPs are potentially more constant MWL indicators across very different task types. Therefore, the tir-aERPs should be taken into consideration in future task-independent MWL monitoring studies.

The Two Sides of Linguistic Context: Eye-Tracking as a Measure of Semantic Competition in Spoken Word Recognition Among Younger and Older Adults

Studies of spoken word recognition have reliably shown that both younger and older adults' recognition of acoustically degraded words is facilitated by the presence of a linguistic context. Against this benefit, older adults' word recognition can be differentially hampered by interference from other words that could also fit the context. These prior studies have primarily used off-line response measures such as the signal-to-noise ratio needed for a target word to be correctly identified. Less clear is the locus of these effects; whether facilitation and interference have their influence primarily during response selection, or whether their effects begin to operate even before a sentence-final target word has been uttered. This question was addressed by tracking 20 younger and 20 older adults' eye fixations on a visually presented target word that corresponded to the final word of a contextually constraining or neutral sentence, accompanied by a second word on the computer screen that in some cases could also fit the sentence context. Growth curve analysis of the time-course of eye-gaze on a target word showed facilitation and inhibition effects begin to appear even as a spoken sentence is unfolding in time. Consistent with an age-related inhibition deficit, older adults' word recognition was slowed by the presence of a semantic competitor to a degree not observed for younger adults, with this effect operating early in the recognition process.

Cognitive processes underlying spoken word recognition during soft speech

In two eye-tracking experiments using the Visual World Paradigm, we examined how listeners recognize words when faced with speech at lower intensities (40, 50, and 65 dBA). After hearing the target word, participants (n = 32) clicked the corresponding picture from a display of four images - a target (e.g., money), a cohort competitor (e.g., mother), a rhyme competitor (e.g., honey) and an unrelated item (e.g., whistle) - while their eye-movements were tracked. For slightly soft speech (50 dBA), listeners demonstrated an increase in cohort activation, whereas for rhyme competitors, activation started later and was sustained longer in processing. For very soft speech (40 dBA), listeners waited until later in processing to activate potential words, as illustrated by a decrease in activation for cohorts, and an increase in activation for rhymes. Further, the extent to which words were considered depended on word length (mono- vs. bi-syllabic words), and speech-extrinsic factors such as the surrounding listening environment. These results advance current theories of spoken word recognition by considering a range of speech levels more typical of everyday listening environments. From an applied perspective, these results motivate models of how individuals who are hard of hearing approach the task of recognizing spoken words.

Autonomic Nervous System Correlates of Speech Categorization Revealed Through Pupillometry

Human perception requires the many-to-one mapping between continuous sensory elements and discrete categorical representations. This grouping operation underlies the phenomenon of categorical perception (CP)-the experience of perceiving discrete categories rather than gradual variations in signal input. Speech perception requires CP because acoustic cues do not share constant relations with perceptual-phonetic representations. Beyond facilitating perception of unmasked speech, we reasoned CP might also aid the extraction of target speech percepts from interfering sound sources (i.e., noise) by generating additional perceptual constancy and reducing listening effort. Specifically, we investigated how noise interference impacts cognitive load and perceptual identification of unambiguous (i.e., categorical) vs. ambiguous stimuli. Listeners classified a speech vowel continuum (/u/-/a/) at various signal-to-noise ratios (SNRs [unmasked, 0 and -5 dB]). Continuous recordings of pupil dilation measured processing effort, with larger, later dilations reflecting increased listening demand. Critical comparisons were between time-locked changes in eye data in response to unambiguous (i.e., continuum endpoints) tokens vs. ambiguous tokens (i.e., continuum midpoint). Unmasked speech elicited faster responses and sharper psychometric functions, which steadily declined in noise. Noise increased pupil dilation across stimulus conditions, but not straightforwardly. Noise-masked speech modulated peak pupil size (i.e., [0 and -5 dB] > unmasked). In contrast, peak dilation latency varied with both token and SNR. Interestingly, categorical tokens elicited earlier pupil dilation relative to ambiguous tokens. Our pupillary data suggest CP reconstructs auditory percepts under challenging listening conditions through interactions between stimulus salience and listeners' internalized effort and/or arousal.

Aging and eye tracking: in the quest for objective biomarkers

Recent applications of eye tracking for diagnosis, prognosis and follow-up of therapy in age-related neurological or psychological deficits have been reviewed. The review is focused on active aging, neurodegeneration and cognitive impairments. The potential impacts and current limitations of using characterizing features of eye movements and pupillary responses (oculometrics) as objective biomarkers in the context of aging are discussed. A closer look into the findings, especially with respect to cognitive impairments, suggests that eye tracking is an invaluable technique to study hidden aspects of aging that have not been revealed using any other noninvasive tool. Future research should involve a wider variety of oculometrics, in addition to saccadic metrics and pupillary responses, including nonlinear and combinatorial features as well as blink- and fixation-related metrics to develop biomarkers to trace age-related irregularities associated with cognitive and neural deficits.

Differences in Working Memory Capacity Affect Online Spoken Word Recognition: Evidence From Eye Movements

Individual differences in working memory capacity have been gaining recognition as playing an important role in speech comprehension, especially in noisy environments. Using the visual world eye-tracking paradigm, a recent study by Hadar and coworkers found that online spoken word recognition was slowed when listeners were required to retain in memory a list of four spoken digits (high load) compared with only one (low load). In the current study, we recognized that the influence of a digit preload might be greater for individuals who have a more limited memory span. We compared participants with higher and lower memory spans on the time course for spoken word recognition by testing eye-fixations on a named object, relative to fixations on an object whose name shared phonology with the named object. Results show that when a low load was imposed, differences in memory span had no effect on the time course of preferential fixations. However, with a high load, listeners with lower span were delayed by ∼550 ms in discriminating target from sound-sharing competitors, relative to higher span listeners. This follows an assumption that the interference effect of a memory preload is not a fixed value, but rather, its effect is greater for individuals with a smaller memory span. Interestingly, span differences affected the timeline for spoken word recognition in noise, but not offline accuracy. This highlights the significance of using eye-tracking as a measure for online speech processing. Results further emphasize the importance of considering differences in cognitive capacity, even when testing normal hearing young adults.

Inclusion of Older Adults in the Research and Design of Digital Technology

Digital technology holds a promise to improve older adults' well-being and promote ageing in place. However, there seems to be a discrepancy between digital technologies that are developed and what older adults actually want and need. Ageing is stereotypically framed as a problem needed to be fixed, and older adults are considered to be frail and incompetent. Not surprisingly, many of the technologies developed for the use of older adults focus on care. The exclusion of older adults from the research and design of digital technology is often based on such negative stereotypes. In this opinion article, we argue that the inclusion rather than exclusion of older adults in the design process and research of digital technology is essential if technology is to fulfill the promise of improving well-being. We emphasize why this is important while also providing guidelines, evidence from the literature, and examples on how to do so. We unequivocally state that designers and researchers should make every effort to ensure the involvement of older adults in the design process and research of digital technology. Based on this paper, we suggest that ageism in the design process of digital technology might play a role as a possible barrier of adopting technology.

Psychobiological Responses Reveal Audiovisual Noise Differentially Challenges Speech Recognition

OBJECTIVES

In noisy environments, listeners benefit from both hearing and seeing a talker, demonstrating audiovisual (AV) cues enhance speech-in-noise (SIN) recognition. Here, we examined the relative contribution of auditory and visual cues to SIN perception and the strategies used by listeners to decipher speech in noise interference(s).

DESIGN

Normal-hearing listeners (n = 22) performed an open-set speech recognition task while viewing audiovisual TIMIT sentences presented under different combinations of signal degradation including visual (AVn), audio (AnV), or multimodal (AnVn) noise. Acoustic and visual noises were matched in physical signal-to-noise ratio. Eyetracking monitored participants' gaze to different parts of a talker's face during SIN perception.

RESULTS

As expected, behavioral performance for clean sentence recognition was better for A-only and AV compared to V-only speech. Similarly, with noise in the auditory channel (AnV and AnVn speech), performance was aided by the addition of visual cues of the talker regardless of whether the visual channel contained noise, confirming a multimodal benefit to SIN recognition. The addition of visual noise (AVn) obscuring the talker's face had little effect on speech recognition by itself. Listeners' eye gaze fixations were biased toward the eyes (decreased at the mouth) whenever the auditory channel was compromised. Fixating on the eyes was negatively associated with SIN recognition performance. Eye gazes on the mouth versus eyes of the face also depended on the gender of the talker.

CONCLUSIONS

Collectively, results suggest listeners (1) depend heavily on the auditory over visual channel when seeing and hearing speech and (2) alter their visual strategy from viewing the mouth to viewing the eyes of a talker with signal degradations, which negatively affects speech perception.

Card playing enhances speech perception among aging adults: comparison with aging musicians

Speech perception and auditory processing have been shown to be enhanced among aging musicians as compared to non-musicians. In the present study, the aim was to test whether these functions are also enhanced among those who are engaged in a non-musical mentally challenging leisure activity (card playing). Three groups of 23 aging adults, aged 60-80 years, were recruited for the study: Musicians, Card players, and Controls. Participants were matched for age, gender, Wechsler Adult Intelligence Scale-III Matrix Reasoning, and Digit Span scores. Their performance was measured using auditory spectral and spatial temporal order judgment tests, and four tasks of speech perception in conditions of: no background noise, background noise of speech frequencies, background noise of white noise, and 60% compressed speech. Musicians were better in auditory and speech perception than the other two groups. Card players were similar to Controls in auditory perception tasks, but were better in the speech perception tasks. Non-musician aging adults may be able to improve their speech perception ability by engaging in leisure activity requiring cognitive effort.

Auditory temporal processing, reading, and phonological awareness among aging adults

Auditory temporal processing (ATP) has been related in the literature to both speech perception as well as reading and phonological awareness. In aging adults, it is known to be related to difficulties in speech perception. In the present study, we aimed to test whether an age-related deficit in ATP would also be accompanied by poor reading and phonological awareness. Thirty-eight aging adults were compared to 55 readers with dyslexia and 42 young normal readers on temporal order judgment (TOJ), speech perception, reading, and phonological awareness tests. Aging adults had longer TOJ thresholds than young normal readers, but shorter than readers with dyslexia; however, they had lower speech perception accuracy than both groups. Phonological awareness of the aging adults was better than readers with dyslexia, but poorer than young normal readers, although their reading accuracy was similar to that of the young controls. This is the first report on poor phonological awareness among aging adults. Suprisingly, it was not accompanied by difficulties in reading ability, and might instead be related to aging adults' difficulties in speech perception. This newly discovered relationship between ATP and phonological awareness among aging adults appears to extend the existing understanding of this relationship, and suggests it should be explored in other groups with ATP deficits.

Eyes and ears: Using eye tracking and pupillometry to understand challenges to speech recognition

Although human speech recognition is often experienced as relatively effortless, a number of common challenges can render the task more difficult. Such challenges may originate in talkers (e.g., unfamiliar accents, varying speech styles), the environment (e.g. noise), or in listeners themselves (e.g., hearing loss, aging, different native language backgrounds). Each of these challenges can reduce the intelligibility of spoken language, but even when intelligibility remains high, they can place greater processing demands on listeners. Noisy conditions, for example, can lead to poorer recall for speech, even when it has been correctly understood. Speech intelligibility measures, memory tasks, and subjective reports of listener difficulty all provide critical information about the effects of such challenges on speech recognition. Eye tracking and pupillometry complement these methods by providing objective physiological measures of online cognitive processing during listening. Eye tracking records the moment-to-moment direction of listeners' visual attention, which is closely time-locked to unfolding speech signals, and pupillometry measures the moment-to-moment size of listeners' pupils, which dilate in response to increased cognitive load. In this paper, we review the uses of these two methods for studying challenges to speech recognition.

Gap encoding by parvalbumin-expressing interneurons in auditory cortex

Synaptic inhibition shapes the temporal processing of sounds in auditory cortex, but the contribution of specific inhibitory cell types to temporal processing remains unclear. We recorded from parvalbumin-expressing (PV+) interneurons in auditory cortex to determine how they encode gaps in noise, a model of temporal processing more generally. We found that PV+ cells had stronger and more prevalent on-responses, off-responses, and postresponse suppression compared with presumed pyramidal cells. We summarize this pattern of differences as "deeper modulation" of gap responses in PV+ cells. Response latencies were also markedly faster for PV+ cells. We found a similar pattern of deeper modulation and faster latencies for responses to white noise bursts, suggesting that these are general properties of on- and off-responses in PV+ cells rather than specific features of gap encoding. These findings are consistent with a role for PV+ cells in providing dynamic gain control by pooling local activity. NEW & NOTEWORTHY We found that parvalbumin-expressing (PV+) interneurons in auditory cortex showed more deeply modulated responses to both gaps in noise and bursts of noise, suggesting that they are optimized for the rapid detection of stimulus transients.

Waiting for lexical access: Cochlear implants or severely degraded input lead listeners to process speech less incrementally

Spoken language unfolds over time. Consequently, there are brief periods of ambiguity, when incomplete input can match many possible words. Typical listeners solve this problem by immediately activating multiple candidates which compete for recognition. In two experiments using the visual world paradigm, we examined real-time lexical competition in prelingually deaf cochlear implant (CI) users, and normal hearing (NH) adults listening to severely degraded speech. In Experiment 1, adolescent CI users and NH controls matched spoken words to arrays of pictures including pictures of the target word and phonological competitors. Eye-movements to each referent were monitored asa measure of how strongly that candidate was considered over time. Relative to NH controls, CI users showed a large delay in fixating any object, less competition from onset competitors (e.g., sandwich after hearing sandal), and increased competition from rhyme competitors (e.g., candle after hearing sandal). Experiment 2 observed the same pattern with NH listeners hearing highly degraded speech. These studies suggests that in contrast to all prior studies of word recognition in typical listeners, listeners recognizing words in severely degraded conditions can exhibit a substantively different pattern of dynamics, waiting to begin lexical access until substantial information has accumulated.

Designing Serious Computer Games for People With Moderate and Advanced Dementia: Interdisciplinary Theory-Driven Pilot Study

Background

The field of serious games for people with dementia (PwD) is mostly driven by game-design principals typically applied to games created by and for younger individuals. Little has been done developing serious games to help PwD maintain cognition and to support functionality.

Objectives

We aimed to create a theory-based serious game for PwD, with input from a multi-disciplinary team familiar with aging, dementia, and gaming theory, as well as direct input from end users (the iterative process). Targeting enhanced self-efficacy in daily activities, the goal was to generate a game that is acceptable, accessible and engaging for PwD.

Methods

The theory-driven game development was based on the following learning theories: learning in context, errorless learning, building on capacities, and acknowledging biological changes—all with the aim to boost self-efficacy. The iterative participatory process was used for game screen development with input of 34 PwD and 14 healthy community dwelling older adults, aged over 65 years. Development of game screens was informed by the bio-psychological aging related disabilities (ie, motor, visual, and perception) as well as remaining neuropsychological capacities (ie, implicit memory) of PwD. At the conclusion of the iterative development process, a prototype game with 39 screens was used for a pilot study with 24 PwD and 14 healthy community dwelling older adults. The game was played twice weekly for 10 weeks.

Results

Quantitative analysis showed that the average speed of successful screen completion was significantly longer for PwD compared with healthy older adults. Both PwD and controls showed an equivalent linear increase in the speed for task completion with practice by the third session (P<.02). Most important, the rate of improved processing speed with practice was not statistically different between PwD and controls. This may imply that some form of learning occurred for PwD at a nonsignificantly different rate than for controls. Qualitative results indicate that PwD found the game engaging and fun. Healthy older adults found the game too easy. Increase in self-reported self-efficacy was documented with PwD only.

Conclusions

Our study demonstrated that PwD’s speed improved with practice at the same rate as healthy older adults. This implies that when tasks are designed to match PwD’s abilities, learning ensues. In addition, this pilot study of a serious game, designed for PwD, was accessible, acceptable, and enjoyable for end users. Games designed based on learning theories and input of end users and a multi-disciplinary team familiar with dementia and aging may have the potential of maintaining capacity and improving functionality of PwD. A larger longer study is needed to confirm our findings and evaluate the use of these games in assessing cognitive status and functionality.

Oral-diadochokinetic rates for Hebrew-speaking healthy ageing population: non-word versus real-word repetition

BACKGROUND

Oral-diadochokinesis (oral-DDK) tasks are extensively used in the evaluation of motor speech abilities. Currently, validated normative data for older adults (aged 65 years and older) are missing in Hebrew. The effect of task stimuli (non-word versus real-word repetition) is also non-clear in the population of older adult Hebrew speakers.

AIMS

(1) To establish a norm for oral-DDK rate for older adult (aged 65 years and older) Hebrew speakers, and to investigate the possible effect of age and gender on performance rate; and (2) to examine the effects of stimuli (non-word versus real word) on oral-DDK rates.

METHODS & PROCEDURES

In experiment 1, 88 healthy older Hebrew speakers (60-95 years, 48 females and 40 males) were audio-recorded while performing an oral-DDK task (repetition of /pataka/), and repetition rates (syllables/s) were coded. In experiment 2, the effect of real-word repetition was evaluated. Sixty-eight older Hebrew speakers (aged 66-95 years, 43 females and 25 males) were asked to repeat 'pataka' (non-word) and 'bodeket' (Hebrew real word).

OUTCOMES & RESULTS

Experiment 1: Oral-DDK performance for older adult Hebrew speakers was 5.07 syllables/s (SD = 1.16 syllables/s), across age groups and gender. Comparison of this data with Hebrew norms for younger adults (and equivalent data in English) shows the following gradient of oral-DDK rates: ages 15-45 > 65-74 > 75-86 years. Gender was not a significant factor in our data. Experiment 2: Repetition of real words was faster than that of non-words, by 13.5%.

CONCLUSIONS & IMPLICATIONS

The paper provides normative values for oral-DDK rates for older Hebrew speakers. The data show the large impact of ageing on oro-motor functions. The analysis further indicates that speech and language pathologists should consider separate norms for clients of 65-74 years and those of 75-86 years. Hebrew rates were found to be different from English norms for the oldest group, shedding light on the impact of language on these norms. Finally, the data support using a dual-protocol (real- and non-word repetition) with older adults to improve differential diagnosis of normal and pathological ageing in this task.

Effort Not Speed Characterizes Comprehension of Spoken Sentences by Older Adults with Mild Hearing Impairment

In spite of the rapidity of everyday speech, older adults tend to keep up relatively well in day-to-day listening. In laboratory settings older adults do not respond as quickly as younger adults in off-line tests of sentence comprehension, but the question is whether comprehension itself is actually slower. Two unique features of the human eye were used to address this question. First, we tracked eye-movements as 20 young adults and 20 healthy older adults listened to sentences that referred to one of four objects pictured on a computer screen. Although the older adults took longer to indicate the referenced object with a cursor-pointing response, their gaze moved to the correct object as rapidly as that of the younger adults. Second, we concurrently measured dilation of the pupil of the eye as a physiological index of effort. This measure revealed that although poorer hearing acuity did not slow processing, success came at the cost of greater processing effort.