1
|
Popp Z, Low S, Igwe A, Rahman MS, Kim M, Khan R, Oh E, Kumar A, De Anda‐Duran I, Ding H, Hwang PH, Sunderaraman P, Shih LC, Lin H, Kolachalama VB, Au R. Shifting From Active to Passive Monitoring of Alzheimer Disease: The State of the Research. J Am Heart Assoc 2024; 13:e031247. [PMID: 38226518 PMCID: PMC10926806 DOI: 10.1161/jaha.123.031247] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/17/2024]
Abstract
Most research using digital technologies builds on existing methods for staff-administered evaluation, requiring a large investment of time, effort, and resources. Widespread use of personal mobile devices provides opportunities for continuous health monitoring without active participant engagement. Home-based sensors show promise in evaluating behavioral features in near real time. Digital technologies across these methodologies can detect precise measures of cognition, mood, sleep, gait, speech, motor activity, behavior patterns, and additional features relevant to health. As a neurodegenerative condition with insidious onset, Alzheimer disease and other dementias (AD/D) represent a key target for advances in monitoring disease symptoms. Studies to date evaluating the predictive power of digital measures use inconsistent approaches to characterize these measures. Comparison between different digital collection methods supports the use of passive collection methods in settings in which active participant engagement approaches are not feasible. Additional studies that analyze how digital measures across multiple data streams can together improve prediction of cognitive impairment and early-stage AD are needed. Given the long timeline of progression from normal to diagnosis, digital monitoring will more easily make extended longitudinal follow-up possible. Through the American Heart Association-funded Strategically Focused Research Network, the Boston University investigative team deployed a platform involving a wide range of technologies to address these gaps in research practice. Much more research is needed to thoroughly evaluate limitations of passive monitoring. Multidisciplinary collaborations are needed to establish legal and ethical frameworks for ensuring passive monitoring can be conducted at scale while protecting privacy and security, especially in vulnerable populations.
Collapse
Affiliation(s)
- Zachary Popp
- Department of Anatomy & NeurobiologyBoston University Chobanian & Avedisian School of MedicineBostonMAUSA
- Boston University Alzheimer’s Disease Research Center, Boston University Chobanian & Avedisian School of MedicineBostonMAUSA
| | - Spencer Low
- Department of Anatomy & NeurobiologyBoston University Chobanian & Avedisian School of MedicineBostonMAUSA
- Boston University Alzheimer’s Disease Research Center, Boston University Chobanian & Avedisian School of MedicineBostonMAUSA
- Department of EpidemiologyBoston University School of Public HealthBostonMAUSA
| | - Akwaugo Igwe
- Department of Anatomy & NeurobiologyBoston University Chobanian & Avedisian School of MedicineBostonMAUSA
- Boston University Alzheimer’s Disease Research Center, Boston University Chobanian & Avedisian School of MedicineBostonMAUSA
| | - Md Salman Rahman
- Department of Anatomy & NeurobiologyBoston University Chobanian & Avedisian School of MedicineBostonMAUSA
- Boston University Alzheimer’s Disease Research Center, Boston University Chobanian & Avedisian School of MedicineBostonMAUSA
| | - Minzae Kim
- Department of Anatomy & NeurobiologyBoston University Chobanian & Avedisian School of MedicineBostonMAUSA
- Boston UniversityBostonMAUSA
| | - Raiyan Khan
- Department of Anatomy & NeurobiologyBoston University Chobanian & Avedisian School of MedicineBostonMAUSA
- Boston UniversityBostonMAUSA
| | - Emily Oh
- Department of Anatomy & NeurobiologyBoston University Chobanian & Avedisian School of MedicineBostonMAUSA
- Boston UniversityBostonMAUSA
| | - Ankita Kumar
- Department of Anatomy & NeurobiologyBoston University Chobanian & Avedisian School of MedicineBostonMAUSA
- Boston UniversityBostonMAUSA
| | - Ileana De Anda‐Duran
- Department of EpidemiologyTulane University School of Public Health & Tropical MedicineNew OrleansLAUSA
| | - Huitong Ding
- Department of Anatomy & NeurobiologyBoston University Chobanian & Avedisian School of MedicineBostonMAUSA
- Framingham Heart StudyBoston University Chobanian & Avedisian School of Medicine School of MedicineBostonMAUSA
| | - Phillip H. Hwang
- Department of EpidemiologyBoston University School of Public HealthBostonMAUSA
| | - Preeti Sunderaraman
- Boston University Alzheimer’s Disease Research Center, Boston University Chobanian & Avedisian School of MedicineBostonMAUSA
- Framingham Heart StudyBoston University Chobanian & Avedisian School of Medicine School of MedicineBostonMAUSA
- Department of NeurologyBoston University Chobanian & Avedisian School of Medicine School of MedicineBostonMAUSA
| | - Ludy C. Shih
- Boston University Alzheimer’s Disease Research Center, Boston University Chobanian & Avedisian School of MedicineBostonMAUSA
- Framingham Heart StudyBoston University Chobanian & Avedisian School of Medicine School of MedicineBostonMAUSA
- Department of NeurologyBoston University Chobanian & Avedisian School of Medicine School of MedicineBostonMAUSA
| | - Honghuang Lin
- Department of MedicineUniversity of Massachusetts Chan Medical SchoolWorcesterMA
| | - Vijaya B. Kolachalama
- Boston University Alzheimer’s Disease Research Center, Boston University Chobanian & Avedisian School of MedicineBostonMAUSA
- Department of MedicineBoston University Chobanian & Avedisian School of Medicine School of MedicineBostonMAUSA
| | - Rhoda Au
- Department of Anatomy & NeurobiologyBoston University Chobanian & Avedisian School of MedicineBostonMAUSA
- Boston University Alzheimer’s Disease Research Center, Boston University Chobanian & Avedisian School of MedicineBostonMAUSA
- Department of EpidemiologyBoston University School of Public HealthBostonMAUSA
- Framingham Heart StudyBoston University Chobanian & Avedisian School of Medicine School of MedicineBostonMAUSA
- Department of NeurologyBoston University Chobanian & Avedisian School of Medicine School of MedicineBostonMAUSA
- Department of MedicineBoston University Chobanian & Avedisian School of Medicine School of MedicineBostonMAUSA
| |
Collapse
|
2
|
Yamada Y, Shinkawa K, Nemoto M, Nemoto K, Arai T. A mobile application using automatic speech analysis for classifying Alzheimer's disease and mild cognitive impairment. COMPUT SPEECH LANG 2023. [DOI: 10.1016/j.csl.2023.101514] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/16/2023]
|
3
|
Mavragani A, Kimura D, Kosugi A, Shinkawa K, Takase T, Kobayashi M, Yamada Y, Nemoto M, Watanabe R, Ota M, Higashi S, Nemoto K, Arai T, Nishimura M. Screening of Mild Cognitive Impairment Through Conversations With Humanoid Robots: Exploratory Pilot Study. JMIR Form Res 2023; 7:e42792. [PMID: 36637896 PMCID: PMC9883738 DOI: 10.2196/42792] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/01/2022] [Revised: 11/23/2022] [Accepted: 12/01/2022] [Indexed: 12/04/2022] Open
Abstract
BACKGROUND The rising number of patients with dementia has become a serious social problem worldwide. To help detect dementia at an early stage, many studies have been conducted to detect signs of cognitive decline by prosodic and acoustic features. However, many of these methods are not suitable for everyday use as they focus on cognitive function or conversational speech during the examinations. In contrast, conversational humanoid robots are expected to be used in the care of older people to help reduce the work of care and monitoring through interaction. OBJECTIVE This study focuses on early detection of mild cognitive impairment (MCI) through conversations between patients and humanoid robots without a specific examination, such as neuropsychological examination. METHODS This was an exploratory study involving patients with MCI and cognitively normal (CN) older people. We collected the conversation data during neuropsychological examination (Mini-Mental State Examination [MMSE]) and everyday conversation between a humanoid robot and 94 participants (n=47, 50%, patients with MCI and n=47, 50%, CN older people). We extracted 17 types of prosodic and acoustic features, such as the duration of response time and jitter, from these conversations. We conducted a statistical significance test for each feature to clarify the speech features that are useful when classifying people into CN people and patients with MCI. Furthermore, we conducted an automatic classification experiment using a support vector machine (SVM) to verify whether it is possible to automatically classify these 2 groups by the features identified in the statistical significance test. RESULTS We obtained significant differences in 5 (29%) of 17 types of features obtained from the MMSE conversational speech. The duration of response time, the duration of silent periods, and the proportion of silent periods showed a significant difference (P<.001) and met the reference value r=0.1 (small) of the effect size. Additionally, filler periods (P<.01) and the proportion of fillers (P=.02) showed a significant difference; however, these did not meet the reference value of the effect size. In contrast, we obtained significant differences in 16 (94%) of 17 types of features obtained from the everyday conversations with the humanoid robot. The duration of response time, the duration of speech periods, jitter (local, relative average perturbation [rap], 5-point period perturbation quotient [ppq5], difference of difference of periods [ddp]), shimmer (local, amplitude perturbation quotient [apq]3, apq5, apq11, average absolute differences between the amplitudes of consecutive periods [dda]), and F0cov (coefficient of variation of the fundamental frequency) showed a significant difference (P<.001). In addition, the duration of response time, the duration of silent periods, the filler period, and the proportion of fillers showed significant differences (P<.05). However, only jitter (local) met the reference value r=0.1 (small) of the effect size. In the automatic classification experiment for the classification of participants into CN and MCI groups, the results showed 66.0% accuracy in the MMSE conversational speech and 68.1% accuracy in everyday conversations with the humanoid robot. CONCLUSIONS This study shows the possibility of early and simple screening for patients with MCI using prosodic and acoustic features from everyday conversations with a humanoid robot with the same level of accuracy as the MMSE.
Collapse
Affiliation(s)
| | | | | | | | - Toshiro Takase
- Healthcare and Life Science, IBM Consulting, IBM Japan, Ltd, Tokyo, Japan
| | | | | | - Miyuki Nemoto
- Department of Psychiatry, Division of Clinical Medicine, Faculty of Medicine, University of Tsukuba, Ibaraki, Japan
| | - Ryohei Watanabe
- Department of Psychiatry, Division of Clinical Medicine, Faculty of Medicine, University of Tsukuba, Ibaraki, Japan
| | - Miho Ota
- Department of Psychiatry, Division of Clinical Medicine, Faculty of Medicine, University of Tsukuba, Ibaraki, Japan
| | - Shinji Higashi
- Department of Psychiatry, Ibaraki Medical Center, Tokyo Medical University, Ibaraki, Japan
| | - Kiyotaka Nemoto
- Department of Psychiatry, Division of Clinical Medicine, Faculty of Medicine, University of Tsukuba, Ibaraki, Japan
| | - Tetsuaki Arai
- Department of Psychiatry, Division of Clinical Medicine, Faculty of Medicine, University of Tsukuba, Ibaraki, Japan
| | - Masafumi Nishimura
- Department of Informatics, Graduate School of Intergraded Science and Technology, Shizuoka University, Hamamatsu, Japan
| |
Collapse
|
4
|
Polin C, Gellé T, Auditeau E, Adou C, Clément JP, Calvet B. Repetitive Behaviors in Alzheimer's Disease: A Systematic Review and Meta-Analysis. J Alzheimers Dis 2023; 96:483-497. [PMID: 37781801 DOI: 10.3233/jad-230380] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/03/2023]
Abstract
BACKGROUND Repetitive behaviors (RBs) are a well-known symptom of Alzheimer's disease (AD); however, they have been little studied and have not been the subject of any specific literature review. OBJECTIVE To conduct a systematic review of all studies to document RBs in AD. METHODS An extensive literature search combining five databases and a meta-analysis were conducted to investigate the frequency, nature, and cognitive correlates of RBs in AD. RESULTS Ten studies were included in the review. Seven studies out of ten investigated the frequency of RBs in patients with AD, which ranged from 52.3% to 87%. A meta-analysis showed an overall frequency of 66.3% (95% CI: 55.5; 77.1) of patients exhibiting RBs in AD, but important heterogeneity was observed between studies. Three studies investigated the predominant nature of RBs in AD. Verbal RBs, complex behavioral stereotypies, and simple motor stereotypies have been identified to different degrees depending on the level of dementia. Most verbal RBs are underpinned by episodic memory impairment, while simple motor stereotypies and complex behavioral stereotypies are mostly underpinned by executive dysfunction. CONCLUSIONS The current review seems to suggest that there are two types of mechanisms underpinning RBs involved in AD. The first is observed especially in the mild stages of the disease and is mediated by episodic memory impairment. The second occurs later and is mediated by executive impairment. Additional studies should be conducted to improve the knowledge about RBs in AD and thus improve their management.Systematic review registration number: PROSPERO 2022: CRD42022310027.
Collapse
Affiliation(s)
- Clément Polin
- Centre Mémoire de Ressources et de Recherche du Limousin, Pôle Universitaire de Psychiatrie de l'Adulte, de l'Agé et d'Addictologie, Centre Hospitalier Esquirol, Limoges, France
- Inserm U1094, IRD U270, University of Limoges, CHU Limoges, EpiMaCT - Epidemiology of Chronic Diseases in Tropical Zone, Institute of Epidemiology and Tropical Neurology, OmegaHealth, Limoges, France
| | - Thibaut Gellé
- Inserm U1094, IRD U270, University of Limoges, CHU Limoges, EpiMaCT - Epidemiology of Chronic Diseases in Tropical Zone, Institute of Epidemiology and Tropical Neurology, OmegaHealth, Limoges, France
| | - Emilie Auditeau
- Inserm U1094, IRD U270, University of Limoges, CHU Limoges, EpiMaCT - Epidemiology of Chronic Diseases in Tropical Zone, Institute of Epidemiology and Tropical Neurology, OmegaHealth, Limoges, France
| | - Caroline Adou
- Inserm U1094, IRD U270, University of Limoges, CHU Limoges, EpiMaCT - Epidemiology of Chronic Diseases in Tropical Zone, Institute of Epidemiology and Tropical Neurology, OmegaHealth, Limoges, France
| | - Jean-Pierre Clément
- Centre Mémoire de Ressources et de Recherche du Limousin, Pôle Universitaire de Psychiatrie de l'Adulte, de l'Agé et d'Addictologie, Centre Hospitalier Esquirol, Limoges, France
- Inserm U1094, IRD U270, University of Limoges, CHU Limoges, EpiMaCT - Epidemiology of Chronic Diseases in Tropical Zone, Institute of Epidemiology and Tropical Neurology, OmegaHealth, Limoges, France
| | - Benjamin Calvet
- Centre Mémoire de Ressources et de Recherche du Limousin, Pôle Universitaire de Psychiatrie de l'Adulte, de l'Agé et d'Addictologie, Centre Hospitalier Esquirol, Limoges, France
- Inserm U1094, IRD U270, University of Limoges, CHU Limoges, EpiMaCT - Epidemiology of Chronic Diseases in Tropical Zone, Institute of Epidemiology and Tropical Neurology, OmegaHealth, Limoges, France
| |
Collapse
|
5
|
Whelan R, Barbey FM, Cominetti MR, Gillan CM, Rosická AM. Developments in scalable strategies for detecting early markers of cognitive decline. Transl Psychiatry 2022; 12:473. [PMID: 36351888 PMCID: PMC9645320 DOI: 10.1038/s41398-022-02237-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/02/2022] [Revised: 10/24/2022] [Accepted: 10/26/2022] [Indexed: 11/10/2022] Open
Abstract
Effective strategies for early detection of cognitive decline, if deployed on a large scale, would have individual and societal benefits. However, current detection methods are invasive or time-consuming and therefore not suitable for longitudinal monitoring of asymptomatic individuals. For example, biological markers of neuropathology associated with cognitive decline are typically collected via cerebral spinal fluid, cognitive functioning is evaluated from face-to-face assessments by experts and brain measures are obtained using expensive, non-portable equipment. Here, we describe scalable, repeatable, relatively non-invasive and comparatively inexpensive strategies for detecting the earliest markers of cognitive decline. These approaches are characterized by simple data collection protocols conducted in locations outside the laboratory: measurements are collected passively, by the participants themselves or by non-experts. The analysis of these data is, in contrast, often performed in a centralized location using sophisticated techniques. Recent developments allow neuropathology associated with potential cognitive decline to be accurately detected from peripheral blood samples. Advances in smartphone technology facilitate unobtrusive passive measurements of speech, fine motor movement and gait, that can be used to predict cognitive decline. Specific cognitive processes can be assayed using 'gamified' versions of standard laboratory cognitive tasks, which keep users engaged across multiple test sessions. High quality brain data can be regularly obtained, collected at-home by users themselves, using portable electroencephalography. Although these methods have great potential for addressing an important health challenge, there are barriers to be overcome. Technical obstacles include the need for standardization and interoperability across hardware and software. Societal challenges involve ensuring equity in access to new technologies, the cost of implementation and of any follow-up care, plus ethical issues.
Collapse
Affiliation(s)
- Robert Whelan
- School of Psychology, Trinity College Dublin, Dublin, Ireland. .,Global Brain Health Institute, Trinity College Dublin, Dublin, Ireland.
| | - Florentine M. Barbey
- grid.8217.c0000 0004 1936 9705School of Psychology, Trinity College Dublin, Dublin, Ireland ,Cumulus Neuroscience Ltd, Dublin, Ireland
| | - Marcia R. Cominetti
- grid.8217.c0000 0004 1936 9705Global Brain Health Institute, Trinity College Dublin, Dublin, Ireland ,grid.411247.50000 0001 2163 588XDepartment of Gerontology, Universidade Federal de São Carlos, São Carlos, Brazil
| | - Claire M. Gillan
- grid.8217.c0000 0004 1936 9705School of Psychology, Trinity College Dublin, Dublin, Ireland ,grid.8217.c0000 0004 1936 9705Global Brain Health Institute, Trinity College Dublin, Dublin, Ireland
| | - Anna M. Rosická
- grid.8217.c0000 0004 1936 9705School of Psychology, Trinity College Dublin, Dublin, Ireland
| |
Collapse
|
6
|
Yamada Y, Shinkawa K, Kobayashi M, Badal VD, Glorioso D, Lee EE, Daly R, Nebeker C, Twamley EW, Depp C, Nemoto M, Nemoto K, Kim HC, Arai T, Jeste DV. Automated Analysis of Drawing Process to Estimate Global Cognition in Older Adults: Preliminary International Validation on the US and Japan Data Sets. JMIR Form Res 2022; 6:e37014. [PMID: 35511253 PMCID: PMC9121219 DOI: 10.2196/37014] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/03/2022] [Revised: 03/25/2022] [Accepted: 04/05/2022] [Indexed: 01/22/2023] Open
Abstract
BACKGROUND With the aging of populations worldwide, early detection of cognitive impairments has become a research and clinical priority, particularly to enable preventive intervention for dementia. Automated analysis of the drawing process has been studied as a promising means for lightweight, self-administered cognitive assessment. However, this approach has not been sufficiently tested for its applicability across populations. OBJECTIVE The aim of this study was to evaluate the applicability of automated analysis of the drawing process for estimating global cognition in community-dwelling older adults across populations in different nations. METHODS We collected drawing data with a digital tablet, along with Montreal Cognitive Assessment (MoCA) scores for assessment of global cognition, from 92 community-dwelling older adults in the United States and Japan. We automatically extracted 6 drawing features that characterize the drawing process in terms of the drawing speed, pauses between drawings, pen pressure, and pen inclinations. We then investigated the association between the drawing features and MoCA scores through correlation and machine learning-based regression analyses. RESULTS We found that, with low MoCA scores, there tended to be higher variability in the drawing speed, a higher pause:drawing duration ratio, and lower variability in the pen's horizontal inclination in both the US and Japan data sets. A machine learning model that used drawing features to estimate MoCA scores demonstrated its capability to generalize from the US dataset to the Japan dataset (R2=0.35; permutation test, P<.001). CONCLUSIONS This study presents initial empirical evidence of the capability of automated analysis of the drawing process as an estimator of global cognition that is applicable across populations. Our results suggest that such automated analysis may enable the development of a practical tool for international use in self-administered, automated cognitive assessment.
Collapse
Affiliation(s)
| | | | | | - Varsha D Badal
- Department of Psychiatry, University of California San Diego, La Jolla, CA, United States.,Sam and Rose Stein Institute for Research on Aging, University of California San Diego, La Jolla, CA, United States
| | - Danielle Glorioso
- Department of Psychiatry, University of California San Diego, La Jolla, CA, United States.,Sam and Rose Stein Institute for Research on Aging, University of California San Diego, La Jolla, CA, United States
| | - Ellen E Lee
- Department of Psychiatry, University of California San Diego, La Jolla, CA, United States.,Sam and Rose Stein Institute for Research on Aging, University of California San Diego, La Jolla, CA, United States.,VA San Diego Healthcare System, San Diego, CA, United States
| | - Rebecca Daly
- Department of Psychiatry, University of California San Diego, La Jolla, CA, United States.,Sam and Rose Stein Institute for Research on Aging, University of California San Diego, La Jolla, CA, United States
| | - Camille Nebeker
- Department of Family Medicine and Public Health, University of California San Diego, La Jolla, CA, United States
| | - Elizabeth W Twamley
- Department of Psychiatry, University of California San Diego, La Jolla, CA, United States.,Sam and Rose Stein Institute for Research on Aging, University of California San Diego, La Jolla, CA, United States.,VA San Diego Healthcare System, San Diego, CA, United States
| | - Colin Depp
- Department of Psychiatry, University of California San Diego, La Jolla, CA, United States.,Sam and Rose Stein Institute for Research on Aging, University of California San Diego, La Jolla, CA, United States
| | - Miyuki Nemoto
- Department of Psychiatry, Division of Clinical Medicine, Faculty of Medicine, University of Tsukuba, Ibaraki, Japan
| | - Kiyotaka Nemoto
- Department of Psychiatry, Division of Clinical Medicine, Faculty of Medicine, University of Tsukuba, Ibaraki, Japan
| | - Ho-Cheol Kim
- AI and Cognitive Software, IBM Almaden Research Center, San Jose, CA, United States
| | - Tetsuaki Arai
- Department of Psychiatry, Division of Clinical Medicine, Faculty of Medicine, University of Tsukuba, Ibaraki, Japan
| | - Dilip V Jeste
- Department of Psychiatry, University of California San Diego, La Jolla, CA, United States.,Sam and Rose Stein Institute for Research on Aging, University of California San Diego, La Jolla, CA, United States.,Department of Neurosciences, University of California San Diego, La Jolla, CA, United States
| |
Collapse
|
7
|
Yamada Y, Shinkawa K, Nemoto M, Arai T. Automatic Assessment of Loneliness in Older Adults Using Speech Analysis on Responses to Daily Life Questions. Front Psychiatry 2021; 12:712251. [PMID: 34966297 PMCID: PMC8710612 DOI: 10.3389/fpsyt.2021.712251] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/20/2021] [Accepted: 11/19/2021] [Indexed: 11/13/2022] Open
Abstract
Loneliness is a perceived state of social and emotional isolation that has been associated with a wide range of adverse health effects in older adults. Automatically assessing loneliness by passively monitoring daily behaviors could potentially contribute to early detection and intervention for mitigating loneliness. Speech data has been successfully used for inferring changes in emotional states and mental health conditions, but its association with loneliness in older adults remains unexplored. In this study, we developed a tablet-based application and collected speech responses of 57 older adults to daily life questions regarding, for example, one's feelings and future travel plans. From audio data of these speech responses, we automatically extracted speech features characterizing acoustic, prosodic, and linguistic aspects, and investigated their associations with self-rated scores of the UCLA Loneliness Scale. Consequently, we found that with increasing loneliness scores, speech responses tended to have less inflections, longer pauses, reduced second formant frequencies, reduced variances of the speech spectrum, more filler words, and fewer positive words. The cross-validation results showed that regression and binary-classification models using speech features could estimate loneliness scores with an R 2 of 0.57 and detect individuals with high loneliness scores with 95.6% accuracy, respectively. Our study provides the first empirical results suggesting the possibility of using speech data that can be collected in everyday life for the automatic assessments of loneliness in older adults, which could help develop monitoring technologies for early detection and intervention for mitigating loneliness.
Collapse
Affiliation(s)
| | | | - Miyuki Nemoto
- Dementia Medical Center, University of Tsukuba Hospital, Tsukuba, Japan
| | - Tetsuaki Arai
- Division of Clinical Medicine, Department of Psychiatry, Faculty of Medicine, University of Tsukuba, Tsukuba, Japan
| |
Collapse
|
8
|
Yamada Y, Shinkawa K, Kobayashi M, Nishimura M, Nemoto M, Tsukada E, Ota M, Nemoto K, Arai T. Tablet-Based Automatic Assessment for Early Detection of Alzheimer's Disease Using Speech Responses to Daily Life Questions. Front Digit Health 2021; 3:653904. [PMID: 34713127 PMCID: PMC8521899 DOI: 10.3389/fdgth.2021.653904] [Citation(s) in RCA: 13] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/15/2021] [Accepted: 02/22/2021] [Indexed: 01/09/2023] Open
Abstract
Health-monitoring technologies for automatically detecting the early signs of Alzheimer's disease (AD) have become increasingly important. Speech responses to neuropsychological tasks have been used for quantifying changes resulting from AD and differentiating AD and mild cognitive impairment (MCI) from cognitively normal (CN). However, whether and how other types of speech tasks with less burden on older adults could be used for detecting early signs of AD remains unexplored. In this study, we developed a tablet-based application and compared speech responses to daily life questions with those to neuropsychological tasks in terms of differentiating MCI from CN. We found that in daily life questions, around 80% of speech features showing significant differences between CN and MCI overlapped those showing significant differences in both our study and other studies using neuropsychological tasks, but the number of significantly different features as well as their effect sizes from life questions decreased compared with those from neuropsychological tasks. On the other hand, the results of classification models for detecting MCI by using the speech features showed that daily life questions could achieve high accuracy, i.e., 86.4%, comparable to neuropsychological tasks by using eight questions against all five neuropsychological tasks. Our results indicate that, while daily life questions may elicit weaker but statistically discernable differences in speech responses resulting from MCI than neuropsychological tasks, combining them could be useful for detecting MCI with comparable performance to using neuropsychological tasks, which could help develop health-monitoring technologies for early detection of AD in a less burdensome manner.
Collapse
Affiliation(s)
| | | | | | - Masafumi Nishimura
- Department of Informatics, Graduate School of Integrated Science and Technology, Shizuoka University, Shizuoka, Japan
| | - Miyuki Nemoto
- Department of Psychiatry, University of Tsukuba Hospital, Ibaraki, Japan
| | - Eriko Tsukada
- Department of Psychiatry, University of Tsukuba Hospital, Ibaraki, Japan
| | - Miho Ota
- Department of Psychiatry, Faculty of Medicine, University of Tsukuba, Ibaraki, Japan
| | - Kiyotaka Nemoto
- Department of Psychiatry, Faculty of Medicine, University of Tsukuba, Ibaraki, Japan
| | - Tetsuaki Arai
- Department of Psychiatry, Faculty of Medicine, University of Tsukuba, Ibaraki, Japan
| |
Collapse
|
9
|
Yamada Y, Shinkawa K, Kobayashi M, Takagi H, Nemoto M, Nemoto K, Arai T. Using Speech Data From Interactions With a Voice Assistant to Predict the Risk of Future Accidents for Older Drivers: Prospective Cohort Study. J Med Internet Res 2021; 23:e27667. [PMID: 33830066 PMCID: PMC8063093 DOI: 10.2196/27667] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/02/2021] [Revised: 03/08/2021] [Accepted: 03/15/2021] [Indexed: 01/27/2023] Open
Abstract
Background With the rapid growth of the older adult population worldwide, car accidents involving this population group have become an increasingly serious problem. Cognitive impairment, which is assessed using neuropsychological tests, has been reported as a risk factor for being involved in car accidents; however, it remains unclear whether this risk can be predicted using daily behavior data. Objective The objective of this study was to investigate whether speech data that can be collected in everyday life can be used to predict the risk of an older driver being involved in a car accident. Methods At baseline, we collected (1) speech data during interactions with a voice assistant and (2) cognitive assessment data—neuropsychological tests (Mini-Mental State Examination, revised Wechsler immediate and delayed logical memory, Frontal Assessment Battery, trail making test-parts A and B, and Clock Drawing Test), Geriatric Depression Scale, magnetic resonance imaging, and demographics (age, sex, education)—from older adults. Approximately one-and-a-half years later, we followed up to collect information about their driving experiences (with respect to car accidents) using a questionnaire. We investigated the association between speech data and future accident risk using statistical analysis and machine learning models. Results We found that older drivers (n=60) with accident or near-accident experiences had statistically discernible differences in speech features that suggest cognitive impairment such as reduced speech rate (P=.048) and increased response time (P=.040). Moreover, the model that used speech features could predict future accident or near-accident experiences with 81.7% accuracy, which was 6.7% higher than that using cognitive assessment data, and could achieve up to 88.3% accuracy when the model used both types of data. Conclusions Our study provides the first empirical results that suggest analysis of speech data recorded during interactions with voice assistants could help predict future accident risk for older drivers by capturing subtle impairments in cognitive function.
Collapse
Affiliation(s)
| | | | | | | | - Miyuki Nemoto
- Department of Psychiatry, University of Tsukuba Hospital, Ibaraki, Japan
| | - Kiyotaka Nemoto
- Department of Psychiatry, Faculty of Medicine, University of Tsukuba, Ibaraki, Japan
| | - Tetsuaki Arai
- Department of Psychiatry, Faculty of Medicine, University of Tsukuba, Ibaraki, Japan
| |
Collapse
|
10
|
Lu ZH, Wang JX, Li X. Revealing Opinions for COVID-19 Questions Using a Context Retriever, Opinion Aggregator, and Question-Answering Model: Model Development Study. J Med Internet Res 2021; 23:e22860. [PMID: 33739287 PMCID: PMC7984426 DOI: 10.2196/22860] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/25/2020] [Revised: 11/16/2020] [Accepted: 01/31/2021] [Indexed: 01/24/2023] Open
Abstract
Background COVID-19 has challenged global public health because it is highly contagious and can be lethal. Numerous ongoing and recently published studies about the disease have emerged. However, the research regarding COVID-19 is largely ongoing and inconclusive. Objective A potential way to accelerate COVID-19 research is to use existing information gleaned from research into other viruses that belong to the coronavirus family. Our objective is to develop a natural language processing method for answering factoid questions related to COVID-19 using published articles as knowledge sources. Methods Given a question, first, a BM25-based context retriever model is implemented to select the most relevant passages from previously published articles. Second, for each selected context passage, an answer is obtained using a pretrained bidirectional encoder representations from transformers (BERT) question-answering model. Third, an opinion aggregator, which is a combination of a biterm topic model and k-means clustering, is applied to the task of aggregating all answers into several opinions. Results We applied the proposed pipeline to extract answers, opinions, and the most frequent words related to six questions from the COVID-19 Open Research Dataset Challenge. By showing the longitudinal distributions of the opinions, we uncovered the trends of opinions and popular words in the articles published in the five time periods assessed: before 1990, 1990-1999, 2000-2009, 2010-2018, and since 2019. The changes in opinions and popular words agree with several distinct characteristics and challenges of COVID-19, including a higher risk for senior people and people with pre-existing medical conditions; high contagion and rapid transmission; and a more urgent need for screening and testing. The opinions and popular words also provide additional insights for the COVID-19–related questions. Conclusions Compared with other methods of literature retrieval and answer generation, opinion aggregation using our method leads to more interpretable, robust, and comprehensive question-specific literature reviews. The results demonstrate the usefulness of the proposed method in answering COVID-19–related questions with main opinions and capturing the trends of research about COVID-19 and other relevant strains of coronavirus in recent years.
Collapse
Affiliation(s)
- Zhao-Hua Lu
- Department of Biostatistics, St. Jude Children's Research Hospital, Memphis, TN, United States
| | - Jade Xiaoqing Wang
- Department of Biostatistics, St. Jude Children's Research Hospital, Memphis, TN, United States
| | - Xintong Li
- Department of Linguistics, The Ohio State University, Columbus, OH, United States
| |
Collapse
|
11
|
Yamada Y, Shinkawa K, Kobayashi M, Caggiano V, Nemoto M, Nemoto K, Arai T. Combining Multimodal Behavioral Data of Gait, Speech, and Drawing for Classification of Alzheimer's Disease and Mild Cognitive Impairment. J Alzheimers Dis 2021; 84:315-327. [PMID: 34542076 PMCID: PMC8609704 DOI: 10.3233/jad-210684] [Citation(s) in RCA: 15] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 08/16/2021] [Indexed: 11/30/2022]
Abstract
BACKGROUND Gait, speech, and drawing behaviors have been shown to be sensitive to the diagnosis of Alzheimer's disease (AD) and mild cognitive impairment (MCI). However, previous studies focused on only analyzing individual behavioral modalities, although these studies suggested that each of these modalities may capture different profiles of cognitive impairments associated with AD. OBJECTIVE We aimed to investigate if combining behavioral data of gait, speech, and drawing can improve classification performance compared with the use of individual modality and if each of these behavioral data can be associated with different cognitive and clinical measures for the diagnosis of AD and MCI. METHODS Behavioral data of gait, speech, and drawing were acquired from 118 AD, MCI, and cognitively normal (CN) participants. RESULTS Combining all three behavioral modalities achieved 93.0% accuracy for classifying AD, MCI, and CN, and only 81.9% when using the best individual behavioral modality. Each of these behavioral modalities was statistically significantly associated with different cognitive and clinical measures for diagnosing AD and MCI. CONCLUSION Our findings indicate that these behaviors provide different and complementary information about cognitive impairments such that classification of AD and MCI is superior to using either in isolation.
Collapse
Affiliation(s)
| | | | | | - Vittorio Caggiano
- Healthcare and Life Sciences, IBM Research, Yorktown Heights, NY, USA
| | - Miyuki Nemoto
- Department of Psychiatry, University of Tsukuba Hospital, Tsukuba, Ibaraki, Japan
| | - Kiyotaka Nemoto
- Department of Psychiatry, Faculty of Medicine, University of Tsukuba, Tsukuba, Ibaraki, Japan
| | - Tetsuaki Arai
- Department of Psychiatry, Faculty of Medicine, University of Tsukuba, Tsukuba, Ibaraki, Japan
| |
Collapse
|
12
|
Ditthapron A, O Agu E, C Lammert A. Privacy-Preserving Deep Speaker Separation for Smartphone-Based Passive Speech Assessment. IEEE OPEN JOURNAL OF ENGINEERING IN MEDICINE AND BIOLOGY 2021; 2:304-313. [PMID: 35402977 PMCID: PMC8940203 DOI: 10.1109/ojemb.2021.3063994] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/11/2021] [Revised: 02/24/2021] [Accepted: 03/01/2021] [Indexed: 12/03/2022] Open
Abstract
Goal: Smartphones can be used to passively assess and monitor patients’ speech impairments caused by ailments such as Parkinson’s disease, Traumatic Brain Injury (TBI), Post-Traumatic Stress Disorder (PTSD) and neurodegenerative diseases such as Alzheimer’s disease and dementia. However, passive audio recordings in natural settings often capture the speech of non-target speakers (cross-talk). Consequently, speaker separation, which identifies the target speakers’ speech in audio recordings with two or more speakers’ voices, is a crucial pre-processing step in such scenarios. Prior speech separation methods analyzed raw audio. However, in order to preserve speaker privacy, passively recorded smartphone audio and machine learning-based speech assessment are often performed on derived speech features such as Mel-Frequency Cepstral Coefficients (MFCCs). In this paper, we propose a novel Deep MFCC bAsed SpeaKer Separation (Deep-MASKS). Methods: Deep-MASKS uses an autoencoder to reconstruct MFCC components of an individual’s speech from an i-vector, x-vector or d-vector representation of their speech learned during the enrollment period. Deep-MASKS utilizes a Deep Neural Network (DNN) for MFCC signal reconstructions, which yields a more accurate, higher-order function compared to prior work that utilized a mask. Unlike prior work that operates on utterances, Deep-MASKS operates on continuous audio recordings. Results: Deep-MASKS outperforms baselines, reducing the Mean Squared Error (MSE) of MFCC reconstruction by up to 44% and the number of additional bits required to represent clean speech entropy by 36%.
Collapse
Affiliation(s)
- Apiwat Ditthapron
- Computer Science DepartmentWorcester Polytechnic Institute Worcester MA 01609 USA
| | - Emmanuel O Agu
- Computer Science DepartmentWorcester Polytechnic Institute Worcester MA 01609 USA
| | - Adam C Lammert
- Biomedical Engineering DepartmentWorcester Polytechnic Institute Worcester MA 01609 USA
| |
Collapse
|
13
|
Badal VD, Nebeker C, Shinkawa K, Yamada Y, Rentscher KE, Kim HC, Lee EE. Do Words Matter? Detecting Social Isolation and Loneliness in Older Adults Using Natural Language Processing. Front Psychiatry 2021; 12:728732. [PMID: 34867518 PMCID: PMC8635064 DOI: 10.3389/fpsyt.2021.728732] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/22/2021] [Accepted: 10/08/2021] [Indexed: 01/13/2023] Open
Abstract
Introduction: Social isolation and loneliness (SI/L) are growing problems with serious health implications for older adults, especially in light of the COVID-19 pandemic. We examined transcripts from semi-structured interviews with 97 older adults (mean age 83 years) to identify linguistic features of SI/L. Methods: Natural Language Processing (NLP) methods were used to identify relevant interview segments (responses to specific questions), extract the type and number of social contacts and linguistic features such as sentiment, parts-of-speech, and syntactic complexity. We examined: (1) associations of NLP-derived assessments of social relationships and linguistic features with validated self-report assessments of social support and loneliness; and (2) important linguistic features for detecting individuals with higher level of SI/L by using machine learning (ML) models. Results: NLP-derived assessments of social relationships were associated with self-reported assessments of social support and loneliness, though these associations were stronger in women than in men. Usage of first-person plural pronouns was negatively associated with loneliness in women and positively associated with emotional support in men. ML analysis using leave-one-out methodology showed good performance (F1 = 0.73, AUC = 0.75, specificity = 0.76, and sensitivity = 0.69) of the binary classification models in detecting individuals with higher level of SI/L. Comparable performance were also observed when classifying social and emotional support measures. Using ML models, we identified several linguistic features (including use of first-person plural pronouns, sentiment, sentence complexity, and sentence similarity) that most strongly predicted scores on scales for loneliness and social support. Discussion: Linguistic data can provide unique insights into SI/L among older adults beyond scale-based assessments, though there are consistent gender differences. Future research studies that incorporate diverse linguistic features as well as other behavioral data-streams may be better able to capture the complexity of social functioning in older adults and identification of target subpopulations for future interventions. Given the novelty, use of NLP should include prospective consideration of bias, fairness, accountability, and related ethical and social implications.
Collapse
Affiliation(s)
- Varsha D Badal
- Department of Psychiatry, University of California San Diego, La Jolla, CA, United States.,Sam and Rose Stein Institute for Research on Aging, University of California San Diego, La Jolla, CA, United States
| | - Camille Nebeker
- Herbert Wertheim School of Public Health and Longevity Science, University of California San Diego, La Jolla, CA, United States
| | | | | | - Kelly E Rentscher
- Cousins Center for Psychoneuroimmunology, Semel Institute for Neuroscience and Human Behavior, University of California, Los Angeles, Los Angeles, CA, United States
| | - Ho-Cheol Kim
- AI and Cognitive Software, IBM Research-Almaden, San Jose, CA, United States
| | - Ellen E Lee
- Department of Psychiatry, University of California San Diego, La Jolla, CA, United States.,Sam and Rose Stein Institute for Research on Aging, University of California San Diego, La Jolla, CA, United States.,VA San Diego Healthcare System, La Jolla, CA, United States
| |
Collapse
|
14
|
Cheng J, Duan Y, Zhang F, Shi J, Li H, Wang F, Li H. The Role of lncRNA TUG1 in the Parkinson Disease and Its Effect on Microglial Inflammatory Response. Neuromolecular Med 2020; 23:327-334. [PMID: 33085068 DOI: 10.1007/s12017-020-08626-y] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/08/2020] [Accepted: 10/12/2020] [Indexed: 10/23/2022]
Abstract
Parkinson's disease (PD) is a common neurodegenerative disease in the middle-aged and elderly populations. The purpose of this study was to investigate the clinical value of lncRNA TUG1 in PD and its effect on the microglial inflammatory response. A total of 181 subjects were recruited for the study, including 97 patients with PD (male/female 50/47) and 84 healthy individuals (male/female 41/43). There was no significant difference for gender and age distribution between the groups. The expression of serum TUG1 was determined by qRT-PCR. The receiver operating curve (ROC) was applied for diagnostic value analysis. CCK-8 was used to detect the effect of TUG1 on the proliferation of BV2 cells. The motor coordination ability of mice was tested by the rotarod and pole tests. ELISA was used to detect serum pro-inflammatory factors. TUG1 was highly expressed in the serum of PD patients. Serum TUG1 can distinguish PD patients to form healthy controls with the AUC of 0.902. Serum TUG1 was positively correlated with the levels of UPDRS, IL-6, IL-1β, and TNF-α in PD patients. Cell experiment results showed that the downregulation of TUG1 significantly inhibited cell proliferation and the release of TNF-α, IL-6, and IL-1β. Besides, animal experiments suggested that the downregulation of TUG1 significantly improved the motor coordination ability of the PD mice and inhibited the expression of inflammatory factors. lncRNA TUG1 is a latent biomarker of PD patients. TUG1 downregulation may inhibit the inflammatory response in the progression of PD. These findings provide a possible target for the early diagnosis and therapeutic intervention of PD.
Collapse
Affiliation(s)
- Jiang Cheng
- Department of Neurology, General Hospital of Ningxia Medical University, Ningxia Key Laboratory of Cerebrocranial Diseases, Incubation Base of National Key Laboratory, NO.804 Shengli Street, Xingqing District, Yinchuan, 750004, Ningxia, China
| | - Yangyang Duan
- School of Clinical Medicine, Ningxia Medical University, Yinchuan, 750004, Ningxia, China
| | - Fengting Zhang
- School of Clinical Medicine, Ningxia Medical University, Yinchuan, 750004, Ningxia, China
| | - Jin Shi
- School of Clinical Medicine, Ningxia Medical University, Yinchuan, 750004, Ningxia, China
| | - Hui Li
- Department of Computer Science, Jiangsu Ocean University, Lianyungang, 222000, Jiangsu, China
| | - Feng Wang
- Department of Neurosurgery, General Hospital of Ningxia Medical University, Ningxia Key Laboratory of Cerebrocranial Diseases, Incubation Base of National Key Laboratory, NO.804 Shengli Street, Xingqing District, Yinchuan, 750004, Ningxia, China.
| | - Haining Li
- Department of Neurology, General Hospital of Ningxia Medical University, Ningxia Key Laboratory of Cerebrocranial Diseases, Incubation Base of National Key Laboratory, NO.804 Shengli Street, Xingqing District, Yinchuan, 750004, Ningxia, China.
| |
Collapse
|