An Automated Approach to Examining Pausing in the Speech of People With Dementia

Speech patterns during memory recall relates to early tau burden across adulthood

INTRODUCTION

Early cognitive decline may manifest in subtle differences in speech.

METHODS

We examined 238 cognitively unimpaired adults from the Framingham Heart Study (32-75 years) who completed amyloid and tau PET imaging. Speech patterns during delayed recall of a story memory task were quantified via five speech markers, and their associations with global amyloid status and regional tau signal were examined.

RESULTS

Total utterance time, number of between-utterance pauses, speech rate, and percentage of unique words significantly correlated with delayed recall score although the shared variance was low (2%-15%). Delayed recall score was not significantly different between β-amyoid-positive (Aβ+) and -negative (Aβ-) groups and was not associated with regional tau signal. However, longer and more between-utterance pauses, and slower speech rate were associated with increased tau signal across medial temporal and early neocortical regions.

DISCUSSION

Subtle speech changes during memory recall may reflect cognitive impairment associated with early Alzheimer's disease pathology.

HIGHLIGHTS

Speech during delayed memory recall relates to tau PET signal across adulthood. Delayed memory recall score was not associated with tau PET signal. Speech shows greater sensitivity to detecting subtle cognitive changes associated with early tau accumulation. Our cohort spans adulthood, while most PET imaging studies focus on older adults.

Linguistic changes in neurodegenerative diseases relate to clinical symptoms

Background

The detection and characterization of speech changes may help in the identification and monitoring of neurodegenerative diseases. However, there is limited research validating the relationship between speech changes and clinical symptoms across a wide range of neurodegenerative diseases.

Method

We analyzed speech recordings from 109 patients who were diagnosed with various neurodegenerative diseases, including Alzheimer's disease, Frontotemporal Dementia, and Vascular Cognitive Impairment, in a cognitive neurology memory clinic. Speech recordings of an open-ended picture description task were processed using the Winterlight speech analysis platform which generates >500 speech features, including the acoustics of speech and linguistic properties of spoken language. We investigated the relationship between the speech features and clinical assessments including the Mini Mental State Examination (MMSE), Mattis Dementia Rating Scale (DRS), Western Aphasia Battery (WAB), and Boston Naming Task (BNT) in a heterogeneous patient population.

Result

Linguistic features including lexical and syntactic features were significantly correlated with clinical assessments in patients, across diagnoses. Lower MMSE and DRS scores were associated with the use of shorter words and fewer prepositional phrases. Increased impairment on WAB and BNT was correlated with the use of fewer nouns but more pronouns. Patients also differed from healthy adults as their speech duration was significantly shorter with more pauses.

Conclusion

Linguistic changes such as the use of simpler vocabularies and syntax were detectable in patients with different neurodegenerative diseases and correlated with cognitive decline. Speech has the potential to be a sensitive measure for detecting cognitive impairments across various neurodegenerative diseases.

Speech changes in old age: Methodological considerations for speech-based discrimination of healthy ageing and Alzheimer's disease

BACKGROUND

Recent evidence suggests that speech substantially changes in ageing. As a complex neurophysiological process, it can accurately reflect changes in the motor and cognitive systems underpinning human speech. Since healthy ageing is not always easily discriminable from early stages of dementia based on cognitive and behavioural hallmarks, speech is explored as a preclinical biomarker of pathological itineraries in old age. A greater and more specific impairment of neuromuscular activation, as well as  a specific cognitive and linguistic impairment in dementia, unchain discriminating changes in speech. Yet, there is no consensus on such discriminatory speech parameters, neither on how they should be elicited and assessed.

AIMS

To provide a state-of-the-art on speech parameters that allow for early discrimination between healthy and pathological ageing; the aetiology of these parameters; the effect of the type of experimental stimuli on speech elicitation and the predictive power of different speech parameters; and the most promising methods for speech analysis and their clinical implications.

METHODS & PROCEDURES

A scoping review methodology is used in accordance with the PRISMA model. Following a systematic search of PubMed, PsycINFO and CINAHL, 24 studies are included and analysed in the review.

MAIN CONTRIBUTION

The results of this review yield three key questions for the clinical assessment of speech in ageing. First, acoustic and temporal parameters are more sensitive to changes in pathological ageing and, of these two, temporal variables are more affected by cognitive impairment. Second, different types of stimuli can trigger speech parameters with different degree of accuracy for the discrimination of clinical groups. Tasks with higher cognitive load are more precise in eliciting higher levels of accuracy. Finally, automatic speech analysis for the discrimination of healthy and pathological ageing should be improved for both research and clinical practice.

CONCLUSIONS & IMPLICATIONS

Speech analysis is a promising non-invasive tool for the preclinical screening of healthy and pathological ageing. The main current challenges of speech analysis in ageing are the automatization of its clinical assessment and the consideration of the speaker's cognitive background during evaluation.

WHAT THIS PAPER ADDS

What is already known on the subject Societal aging  goes hand in hand with the rising incidence of ageing-related neurodegenerations, mainly Alzheimer's disease (AD). This is particularly noteworthy in countries with longer life expectancies. Healthy ageing and early stages of AD share a set of cognitive and behavioural characteristics. Since there is no cure for dementias, developing methods for accurate discrimination of healthy ageing and early AD is currently a priority. Speech has been described as one of the most significantly impaired features in AD. Neuropathological alterations in motor and cognitive systems would underlie specific speech impairment in dementia. Since speech can be evaluated quickly, non-invasively and inexpensively, its value for the clinical assessment of ageing itineraries may be particularly high. What this paper adds to existing knowledge Theoretical and experimental advances in the assessment of speech as a marker of AD have developed rapidly over the last decade. Yet, they are not always known to clinicians. Furthermore, there is a need to provide an updated state-of-the-art on which speech features are discriminatory to AD, how they can be assessed, what kind of results they can yield, and how such results should be interpreted. This article provides an updated overview of speech profiling, methods of speech measurement and analysis, and the clinical power of speech assessment for early discrimination of AD as the most common cause of dementia. What are the potential or actual clinical implications of this work? This article provides an overview of the predictive potential of different speech parameters in relation to AD cognitive impairment. In addition, it discusses the effect that the cognitive state, the type of elicitation task and the type of assessment method may have on the results of the speech-based analysis in ageing.

DementiaBank: Theoretical Rationale, Protocol, and Illustrative Analyses

PURPOSE

Dementia from Alzheimer's disease (AD) is characterized primarily by a significant decline in memory abilities; however, language abilities are also commonly affected and may precede the decline of other cognitive abilities. To study the progression of language, there is a need for open-access databases that can be used to build algorithms to produce translational models sensitive enough to detect early declines in language abilities. DementiaBank is an open-access repository of transcribed video/audio data from communicative interactions from people with dementia, mild cognitive impairment (MCI), and controls. The aims of this tutorial are to (a) describe the newly established standardized DementiaBank discourse protocol, (b) describe the Delaware corpus data, and (c) provide examples of automated linguistic analyses that can be conducted with the Delaware corpus data and describe additional DementiaBank resources.

METHOD

The DementiaBank discourse protocol elicits four types of discourse: picture description, story narrative, procedural, and personal narrative. The Delaware corpus currently includes data from 20 neurotypical adults and 33 adults with MCI from possible AD who completed the DementiaBank discourse protocol and a cognitive-linguistic battery. Language samples were video- and audio-recorded, transcribed, coded, and uploaded to DementiaBank. The protocol materials and transcription programs can be accessed for free via the DementiaBank website.

RESULTS

Illustrative analyses show the potential of the Delaware corpus data to help understand discourse metrics at the individual and group levels. In addition, they highlight analyses that could be used across TalkBank's other clinical banks (e.g., AphasiaBank). Information is also included on manual and automatic speech recognition transcription methods.

CONCLUSIONS

DementiaBank is a shared online database that can facilitate research efforts to address the gaps in knowledge about language changes associated with MCI and dementia from AD. Identifying early language markers could lead to improved assessment and treatment approaches for adults at risk for dementia.

The Role of Silence in Verbal Fluency Tasks - A New Approach for the Detection of Mild Cognitive Impairment

OBJECTIVE

Most recordings of verbal fluency tasks include substantial amounts of task-irrelevant content that could provide clinically valuable information for the detection of mild cognitive impairment (MCI). We developed a method for the analysis of verbal fluency, focusing not on the task-relevant words but on the silent segments, the hesitations, and the irrelevant utterances found in the voice recordings.

METHODS

Phonemic ('k', 't', 'a') and semantic (animals, food items, actions) verbal fluency data were collected from healthy control (HC; n = 25; M_age = 67.32) and MCI (n = 25; M_age = 71.72) participants. After manual annotation of the voice samples, 10 temporal parameters were computed based on the silent and the task-irrelevant segments. Traditional fluency measures, based on word count (correct words, errors, repetitions) were also employed in order to compare the outcome of the two methods.

RESULTS

Two silence-based parameters (the number of silent pauses and the average length of silent pauses) and the average word transition time differed significantly between the two groups in the case of all three semantic fluency tasks. Subsequent receiver operating characteristic (ROC) analysis showed that these three temporal parameters had classification abilities similar to the traditional measure of counting correct words.

CONCLUSION

In our approach for verbal fluency analysis, silence-related parameters displayed classification ability similar to the most widely used traditional fluency measure. Based on these results, an automated tool using voiced-unvoiced segmentation may be developed enabling swift and cost-effective verbal fluency-based MCI screening.

Evaluating Web-Based Automatic Transcription for Alzheimer Speech Data: Transcript Comparison and Machine Learning Analysis

Background

Speech data for medical research can be collected noninvasively and in large volumes. Speech analysis has shown promise in diagnosing neurodegenerative disease. To effectively leverage speech data, transcription is important, as there is valuable information contained in lexical content. Manual transcription, while highly accurate, limits the potential scalability and cost savings associated with language-based screening.

Objective

To better understand the use of automatic transcription for classification of neurodegenerative disease, namely, Alzheimer disease (AD), mild cognitive impairment (MCI), or subjective memory complaints (SMC) versus healthy controls, we compared automatically generated transcripts against transcripts that went through manual correction.

Methods

We recruited individuals from a memory clinic (“patients”) with a diagnosis of mild-to-moderate AD, (n=44, 30%), MCI (n=20, 13%), SMC (n=8, 5%), as well as healthy controls (n=77, 52%) living in the community. Participants were asked to describe a standardized picture, read a paragraph, and recall a pleasant life experience. We compared transcripts generated using Google speech-to-text software to manually verified transcripts by examining transcription confidence scores, transcription error rates, and machine learning classification accuracy. For the classification tasks, logistic regression, Gaussian naive Bayes, and random forests were used.

Results

The transcription software showed higher confidence scores (P<.001) and lower error rates (P>.05) for speech from healthy controls compared with patients. Classification models using human-verified transcripts significantly (P<.001) outperformed automatically generated transcript models for both spontaneous speech tasks. This comparison showed no difference in the reading task. Manually adding pauses to transcripts had no impact on classification performance. However, manually correcting both spontaneous speech tasks led to significantly higher performances in the machine learning models.

Conclusions

We found that automatically transcribed speech data could be used to distinguish patients with a diagnosis of AD, MCI, or SMC from controls. We recommend a human verification step to improve the performance of automatic transcripts, especially for spontaneous tasks. Moreover, human verification can focus on correcting errors and adding punctuation to transcripts. However, manual addition of pauses is not needed, which can simplify the human verification step to more efficiently process large volumes of speech data.

Estimating Sentence-like Structure in Synthetic Languages Using Information Topology

Estimating sentence-like units and sentence boundaries in human language is an important task in the context of natural language understanding. While this topic has been considered using a range of techniques, including rule-based approaches and supervised and unsupervised algorithms, a common aspect of these methods is that they inherently rely on a priori knowledge of human language in one form or another. Recently we have been exploring synthetic languages based on the concept of modeling behaviors using emergent languages. These synthetic languages are characterized by a small alphabet and limited vocabulary and grammatical structure. A particular challenge for synthetic languages is that there is generally no a priori language model available, which limits the use of many natural language processing methods. In this paper, we are interested in exploring how it may be possible to discover natural ‘chunks’ in synthetic language sequences in terms of sentence-like units. The problem is how to do this with no linguistic or semantic language model. Our approach is to consider the problem from the perspective of information theory. We extend the basis of information geometry and propose a new concept, which we term information topology, to model the incremental flow of information in natural sequences. We introduce an information topology view of the incremental information and incremental tangent angle of the Wasserstein-1 distance of the probabilistic symbolic language input. It is not suggested as a fully viable alternative for sentence boundary detection per se but provides a new conceptual method for estimating the structure and natural limits of information flow in language sequences but without any semantic knowledge. We consider relevant existing performance metrics such as the F-measure and indicate limitations, leading to the introduction of a new information-theoretic global performance based on modeled distributions. Although the methodology is not proposed for human language sentence detection, we provide some examples using human language corpora where potentially useful results are shown. The proposed model shows potential advantages for overcoming difficulties due to the disambiguation of complex language and potential improvements for human language methods.

Towards Interpretability of Speech Pause in Dementia Detection Using Adversarial Learning

Speech pause is an effective biomarker in dementia detection. Recent deep learning models have exploited speech pauses to achieve highly accurate dementia detection, but have not exploited the interpretability of speech pauses, i.e., what and how positions and lengths of speech pauses affect the result of dementia detection. In this paper, we will study the positions and lengths of dementia-sensitive pauses using adversarial learning approaches. Specifically, we first utilize an adversarial attack approach by adding the perturbation to the speech pauses of the testing samples, aiming to reduce the confidence levels of the detection model. Then, we apply an adversarial training approach to evaluate the impact of the perturbation in training samples on the detection model. We examine the interpretability from the perspectives of model accuracy, pause context, and pause length. We found that some pauses are more sensitive to dementia than other pauses from the model's perspective, e.g., speech pauses near to the verb "is". Increasing lengths of sensitive pauses or adding sensitive pauses leads the model inference to Alzheimer's Disease (AD), while decreasing the lengths of sensitive pauses or deleting sensitive pauses leads to non-AD.

Temporal speech parameters detect mild cognitive impairment in different languages: validation and comparison of the Speech-GAP Test® in English and Hungarian

BACKGROUND

The development of automatic speech recognition (ASR) technology allows the analysis of temporal (time-based) speech parameters characteristic of mild cognitive impairment (MCI). However, no information has been available on whether the analysis of spontaneous speech can be used with the same efficiency in different language environments.

OBJECTIVE

The main goal of this international pilot study is to address the question whether the Speech-Gap Test® (S-GAP Test®), previously tested in the Hungarian language, is appropriate for and applicable to the recognition of MCI in other languages such as English.

METHOD

After an initial screening of 88 individuals, English-speaking (n = 33) and Hungarian-speaking (n = 33) participants were classified as having MCI or as healthy controls (HC) based on Petersen's criteria. Speech of each participant was recorded via a spontaneous speech task. 15 temporal parameters were determined and calculated by means of ASR.

RESULTS

Seven temporal parameters in the English-speaking sample and 5 in the Hungarian-speaking sample showed significant differences between the MCI and the HC group. Receiver operating characteristics (ROC) analysis clearly distinguished the English-speaking MCI cases from the HC group based on speech tempo and articulation tempo with 100% sensitivity, and on three more temporal parameters with high sensitivity (85.7%). In the Hungarian-speaking sample, the ROC analysis showed similar sensitivity rates (92.3%).

CONCLUSION

The results of this study in different native-speaking populations suggest that changes in acoustic parameters detected by the S-GAP Test® might be present across different languages.

A Comparison of Speech Features between Mild Cognitive Impairment and Healthy Aging Groups

Background and Purpose

Language dysfunction is a symptom common to patients with Alzheimer's disease (AD). Speech feature analysis may be a patient-friendly screening test for early-stage AD. We aimed to investigate the speech features of amnestic mild cognitive impairment (aMCI) compared to normal controls (NCs).

Methods

Spoken responses to test questions were recorded with a microphone placed 15 cm in front of each participant. Speech samples delivered in response to four spoken test prompts (free speech test, Mini-Mental State Examination [MMSE], picture description test, and sentence repetition test) were obtained from 98 patients with aMCI and 139 NCs. Each recording was transcribed, with speech features noted. The frequency of the ten speech features assessed was evaluated to compare speech abilities between the test groups.

Results

Among the ten speech features, the frequency of pauses (p=0.001) and mumbles (p=0.001) were significantly higher in patients with aMCI than in NCs. Moreover, MMSE score was found to negatively correlate with the frequency of pauses (r=−0.441, p<0.001) and mumbles (r=−0.341, p<0.001).

Conclusions

Frequent pauses and mumbles reflect cognitive decline in aMCI patients in episodic and semantic memory tests. Speech feature analysis may prove to be a speech-based biomarker for screening early-stage cognitive impairment.

How Content Authored by People with Dementia Affects Attitudes towards Dementia

Negative attitudes shape experiences with stigmatized conditions such as dementia, from affecting social relationships to influencing willingness to adopt technology. Consequently, attitudinal change has been identified as one lever to improve life for people with stigmatized conditions. Though recognized as a scaleable approach, social media has not been studied in terms of how it should best be designed or deployed to target attitudes and understanding of dementia. Through a mixed methods design with 123 undergraduate college students, we study the effect of being exposed to dementia-related media, including content produced by people with dementia. We selected undergraduate college students as the target of our intervention, as they represent the next generation that will work and interact with individuals with dementia. Our analysis describes changes over the period of two weeks in attitudes and understanding of the condition. The shifts in understanding of dementia that we found in our qualitative analysis were not captured by the instrument we selected to assess understanding of dementia. While small improvements in positive and overall attitudes were seen across all interventions and the control, we observe a different pattern with negative attitudes, where transcriptions of content produced by people with dementia significantly reduced negative attitudes. The discussion presents implications for supporting people with dementia as content producers, doing so in ways that best affect attitudes and understanding by drawing on research on cues and interactive media, and supporting students in changing their perspectives towards people with dementia.