Learning diagnostic models using speech and language measures

Evaluation of Speech-Based Digital Biomarkers: Review and Recommendations

Speech represents a promising novel biomarker by providing a window into brain health, as shown by its disruption in various neurological and psychiatric diseases. As with many novel digital biomarkers, however, rigorous evaluation is currently lacking and is required for these measures to be used effectively and safely. This paper outlines and provides examples from the literature of evaluation steps for speech-based digital biomarkers, based on the recent V3 framework (Goldsack et al., 2020). The V3 framework describes 3 components of evaluation for digital biomarkers: verification, analytical validation, and clinical validation. Verification includes assessing the quality of speech recordings and comparing the effects of hardware and recording conditions on the integrity of the recordings. Analytical validation includes checking the accuracy and reliability of data processing and computed measures, including understanding test-retest reliability, demographic variability, and comparing measures to reference standards. Clinical validity involves verifying the correspondence of a measure to clinical outcomes which can include diagnosis, disease progression, or response to treatment. For each of these sections, we provide recommendations for the types of evaluation necessary for speech-based biomarkers and review published examples. The examples in this paper focus on speech-based biomarkers, but they can be used as a template for digital biomarker development more generally.

Quantifying the Uncertainty of Parameters Measured in Spontaneous Speech of Speakers With Dementia

Purpose Corpus analyses of spontaneous language fragments of varying length provide useful insights in the language change caused by brain damage, such as caused by some forms of dementia. Sample size is an important experimental parameter to consider when designing spontaneous language analyses studies. Sample length influences the confidence levels of analyses. Machine learning approaches often favor to use as much language as available, whereas language evaluation in a clinical setting is often based on truncated samples to minimize annotation labor and to limit any discomfort for participants. This article investigates, using Bayesian estimation of machine learned models, what the ideal text length should be to minimize model uncertainty. Method We use the Stanford parser to extract linguistic variables and train a statistic model to distinguish samples by speakers with no brain damage from samples by speakers with probable Alzheimer's disease. We compare the results to previously published models that used CLAN for linguistic analysis. Results The uncertainty around six individual variables and its relation to sample length are reported. The same model with linguistic variables that is used in all three experiments can predict group membership better than a model without them. One variable (concept density) is more informative when measured using the Stanford tools than when measured using CLAN. Conclusion For our corpus of German speech, the optimal sample length is found to be around 700 words long. Longer samples do not provide more information.

A Conversational Robot for Older Adults with Alzheimer’s Disease

Amid the rising cost of Alzheimer’s disease (AD), assistive health technologies can reduce care-giving burden by aiding in assessment, monitoring, and therapy. This article presents a pilot study testing the feasibility and effect of a conversational robot in a cognitive assessment task with older adults with AD. We examine the robot interactions through dialogue and miscommunication analysis, linguistic feature analysis, and the use of a qualitative analysis, in which we report key themes that were prevalent throughout the study. While conversations were typically better with human conversation partners (being longer, with greater engagement and less misunderstanding), we found that the robot was generally well liked by participants and that it was able to capture their interest in dialogue. Miscommunication due to issues of understanding and intelligibility did not seem to deter participants from their experience. Furthermore, in automatically extracting linguistic features, we examine how non-acoustic aspects of language change across participants with varying degrees of cognitive impairment, highlighting the robot’s potential as a monitoring tool. This pilot study is an exploration of how conversational robots can be used to support individuals with AD.

Using narratives in differential diagnosis of neurodegenerative syndromes

PURPOSE

Language decline has been associated with healthy aging and with various neurodegenerative conditions, making it challenging to differentiate among these conditions. This study examined the utility of linguistic measures derived from a short narrative language sample for 1) identifying language characteristics and cut-off scores to differentiate between healthy aging, Primary Progressive Aphasia (PPA), Mild Cognitive Impairment (MCI), and Alzheimer's dementia (AD); and 2) differentiating among PPA variants in which language is the primary impairment.

METHOD

Participants were 25 neurologically healthy English speakers, 20 individuals with MCI, 20 with AD, and 26 with PPA (non-fluent/agrammatic N = 10, logopenic N = 9, semantic N = 7). Narrative language samples of the Cookie Theft Picture of persons with healthy aging, MCI and AD were retrospectively obtained from the DementiaBank database (https://talkbank.org/DementiaBank/) and PPA samples were obtained from an ongoing research study. The language samples were analyzed for fluency, word retrieval success, grammatical accuracy, and errors using automated and manual analysis methods. The sensitivity and specificity of various language measures was computed.

RESULTS

Participants with PPA scored lower than neurologically healthy and MCI groups on fluency (words per minute and disfluencies), word retrieval (Correct Information Units and number of errors), and sentence grammaticality. PPA and AD groups did not differ on language measures. Agrammatic PPA participants scored lower than logopenic and semantic PPA groups on several measures, while logopenic and semantic PPA did not differ on any measures.

CONCLUSION

Measures derived from brief language samples and analyzed using mostly automated methods are clinically useful in differentiating PPA from healthy aging and MCI, and agrammatic PPA from other variants. The sensitivity and specificity of these measures is modest and can be improved when coupled with clinical presentation.

Automatic Assessment of Speech Impairment in Cantonese-speaking People with Aphasia

Aphasia is a common type of acquired language impairment resulting from dysfunction in specific brain regions. Analysis of narrative spontaneous speech, e.g., story-telling, is an essential component of standardized clinical assessment on people with aphasia (PWA). Subjective assessment by trained speech-language pathologists (SLP) have many limitations in efficiency, effectiveness and practicality. This paper describes a fully automated system for speech assessment of Cantonese-speaking PWA. A deep neural network (DNN) based automatic speech recognition (ASR) system is developed for aphasic speech by multi-task training with both in-domain and out-of-domain speech data. Story-level embedding and Siamese network are applied to derive robust text features, which can be used to quantify the difference between aphasic speech and unimpaired one. The proposed text features are combined with conventional acoustic features to cover different aspects of speech and language impairment in PWA. Experimental results show a high correlation between predicted scores and subject assessment scores. The best correlation value achieved with ASR-generated transcription is .827, as compared with .844 achieved with manual transcription. The Siamese network significantly outperforms story-level embedding in generating text features for automatic assessment.

Classification of Huntington Disease using Acoustic and Lexical Features

Speech is a critical biomarker for Huntington Disease (HD), with changes in speech increasing in severity as the disease progresses. Speech analyses are currently conducted using either transcriptions created manually by trained professionals or using global rating scales. Manual transcription is both expensive and time-consuming and global rating scales may lack sufficient sensitivity and fidelity [1]. Ultimately, what is needed is an unobtrusive measure that can cheaply and continuously track disease progression. We present first steps towards the development of such a system, demonstrating the ability to automatically differentiate between healthy controls and individuals with HD using speech cues. The results provide evidence that objective analyses can be used to support clinical diagnoses, moving towards the tracking of symptomatology outside of laboratory and clinical environments.

Connected Speech in Neurodegenerative Language Disorders: A Review

Language assessment has a crucial role in the clinical diagnosis of several neurodegenerative diseases. The analysis of extended speech production is a precious source of information encompassing the phonetic, phonological, lexico-semantic, morpho-syntactic, and pragmatic levels of language organization. The knowledge about the distinctive linguistic variables identifying language deficits associated to different neurodegenerative diseases has progressively improved in the last years. However, the heterogeneity of such variables and of the way they are measured and classified limits any generalization and makes the comparison among studies difficult. Here we present an exhaustive review of the studies focusing on the linguistic variables derived from the analysis of connected speech samples, with the aim of characterizing the language disorders of the most prevalent neurodegenerative diseases, including primary progressive aphasia, Alzheimer's disease, movement disorders, and amyotrophic lateral sclerosis. A total of 61 studies have been included, considering only those reporting group analysis and comparisons with a group of healthy persons. This review first analyzes the differences in the tasks used to elicit connected speech, namely picture description, story narration, and interview, considering the possible different contributions to the assessment of different linguistic domains. This is followed by an analysis of the terminologies and of the methods of measurements of the variables, indicating the need for harmonization and standardization. The final section reviews the linguistic domains affected by each different neurodegenerative disease, indicating the variables most consistently impaired at each level and suggesting the key variables helping in the differential diagnosis among diseases. While a large amount of valuable information is already available, the review highlights the need of further work, including the development of automated methods, to take advantage of the richness of connected speech analysis for both research and clinical purposes.

Leveraging psycholinguistic resources and emotional sequence models for suicide note emotion annotation

We describe the submission entered by SRI International and UC Davis for the I2B2 NLP Challenge Track 2. Our system is based on a machine learning approach and employs a combination of lexical, syntactic, and psycholinguistic features. In addition, we model the sequence and locations of occurrence of emotions found in the notes. We discuss the effect of these features on the emotion annotation task, as well as the nature of the notes themselves. We also explore the use of bootstrapping to help account for what appeared to be annotator fatigue in the data. We conclude a discussion of future avenues for improving the approach for this task, and also discuss how annotations at the word span level may be more appropriate for this task than annotations at the sentence level.

Review of extracting information from the Social Web for health personalization

In recent years the Web has come into its own as a social platform where health consumers are actively creating and consuming Web content. Moreover, as the Web matures, consumers are gaining access to personalized applications adapted to their health needs and interests. The creation of personalized Web applications relies on extracted information about the users and the content to personalize. The Social Web itself provides many sources of information that can be used to extract information for personalization apart from traditional Web forms and questionnaires. This paper provides a review of different approaches for extracting information from the Social Web for health personalization. We reviewed research literature across different fields addressing the disclosure of health information in the Social Web, techniques to extract that information, and examples of personalized health applications. In addition, the paper includes a discussion of technical and socioethical challenges related to the extraction of information for health personalization.