Diagnostic accuracy of keystroke dynamics as digital biomarkers for fine motor decline in neuropsychiatric disorders: a systematic review and meta-analysis

Digital Neuropsychology beyond Computerized Cognitive Assessment: Applications of Novel Digital Technologies

Compared with other health disciplines, there is a stagnation in technological innovation in the field of clinical neuropsychology. Traditional paper-and-pencil tests have a number of shortcomings, such as low-frequency data collection and limitations in ecological validity. While computerized cognitive assessment may help overcome some of these issues, current computerized paradigms do not address the majority of these limitations. In this paper, we review recent literature on the applications of novel digital health approaches, including ecological momentary assessment, smartphone-based assessment and sensors, wearable devices, passive driving sensors, smart homes, voice biomarkers, and electronic health record mining, in neurological populations. We describe how each digital tool may be applied to neurologic care and overcome limitations of traditional neuropsychological assessment. Ethical considerations, limitations of current research, as well as our proposed future of neuropsychological practice are also discussed.

Evaluating the Acceptability and Feasibility of Collecting Passive Smartphone Data to Estimate Psychological Functioning in U.S. Service Members and Veterans: A Pilot Study

INTRODUCTION

This study investigated the acceptability and feasibility of digital phenotyping in a military sample with a history of traumatic brain injury and co-occurring psychological and cognitive symptoms. The first aim was to evaluate the acceptability of digital phenotyping by (1a) quantifying the proportion of participants willing to download the app and rates of dropout and app discontinuation and (1b) reviewing the stated reasons for both refusing and discontinuing use of the app. The second aim was to investigate technical feasibility by (2a) characterizing the amount and frequency of transferred data and (2b) documenting technical challenges. Exploratory aim 3 sought to leverage data on phone and keyboard interactions to predict if a participant (a) is depressed and (b) has depression that improves over the course of the study.

MATERIALS AND METHODS

A passive digital phenotyping app (Mindstrong Discovery) functioned in the background of the participants' smartphones and passively collected phone usage and typing kinematics data.

RESULTS

Fifteen out of 16 participants (93.8%) consented to install the app on their personal smartphone devices. Four participants (26.7%) discontinued the use of the app partway through the study, primarily because of keyboard usability and technical issues. Fourteen out of 15 participants (93.3%) had at least one data transfer, and the median number of days with data was 40 out of a possible 57 days. The exploratory machine learning models predicting depression status and improvement in depression performed better than chance.

CONCLUSIONS

The findings of this pilot study suggest that digital phenotyping is acceptable and feasible in a military sample and provides support for future larger investigations of this technology.

A synergetic turn in cognitive neuroscience of brain diseases

Despite significant improvements in our understanding of brain diseases, many barriers remain. Cognitive neuroscience faces four major challenges: complex structure-function associations; disease phenotype heterogeneity; the lack of transdiagnostic models; and oversimplified cognitive approaches restricted to the laboratory. Here, we propose a synergetics framework that can help to perform the necessary dimensionality reduction of complex interactions between the brain, body, and environment. The key solutions include low-dimensional spatiotemporal hierarchies for brain-structure associations, whole-brain modeling to handle phenotype diversity, model integration of shared transdiagnostic pathophysiological pathways, and naturalistic frameworks balancing experimental control and ecological validity. Creating whole-brain models with reduced manifolds combined with ecological measures can improve our understanding of brain disease and help identify novel interventions. Synergetics provides an integrated framework for future progress in clinical and cognitive neuroscience, pushing the boundaries of brain health and disease toward more mature, naturalistic approaches.

Generalizing Parkinson's disease detection using keystroke dynamics: a self-supervised approach

OBJECTIVE

Passive monitoring of touchscreen interactions generates keystroke dynamic signals that can be used to detect and track neurological conditions such as Parkinson's disease (PD) and psychomotor impairment with minimal burden on the user. However, this typically requires datasets with clinically confirmed labels collected in standardized environments, which is challenging, especially for a large subject pool. This study validates the efficacy of a self-supervised learning method in reducing the reliance on labels and evaluates its generalizability.

MATERIALS AND METHODS

We propose a new type of self-supervised loss combining Barlow Twins loss, which attempts to create similar feature representations with reduced feature redundancy for samples coming from the same subject, and a Dissimilarity loss, which promotes uncorrelated features for samples generated by different subjects. An encoder is first pre-trained using this loss on unlabeled data from an uncontrolled setting, then fine-tuned with clinically validated data. Our experiments test the model generalizability with controls and subjects with PD on 2 independent datasets.

RESULTS

Our approach showed better generalization compared to previous methods, including a feature engineering strategy, a deep learning model pre-trained on Parkinsonian signs, and a traditional supervised model.

DISCUSSION

The absence of standardized data acquisition protocols and the limited availability of annotated datasets compromise the generalizability of supervised models. In these contexts, self-supervised models offer the advantage of learning more robust patterns from the data, bypassing the need for ground truth labels.

CONCLUSION

This approach has the potential to accelerate the clinical validation of touchscreen typing software for neurodegenerative diseases.

Brief webcam test of hand movements predicts episodic memory, executive function, and working memory in a community sample of cognitively asymptomatic older adults

INTRODUCTION

Low-cost simple tests for preclinical Alzheimer's disease are a research priority. We evaluated whether remote unsupervised webcam recordings of finger-tapping were associated with cognitive performance in older adults.

METHODS

A total of 404 cognitively-asymptomatic participants (64.6 [6.77] years; 70.8% female) completed 10-second finger-tapping tests (Tasmanian [TAS] Test) and cognitive tests (Cambridge Neuropsychological Test Automated Battery [CANTAB]) online at home. Regression models including hand movement features were compared with null models (comprising age, sex, and education level); change in Akaike Information Criterion greater than 2 (ΔAIC > 2) denoted statistical difference.

RESULTS

Hand movement features improved prediction of episodic memory, executive function, and working memory scores (ΔAIC > 2). Dominant hand features outperformed nondominant hand features for episodic memory (ΔAIC = 2.5), executive function (ΔAIC = 4.8), and working memory (ΔAIC = 2.2).

DISCUSSION

This brief webcam test improved prediction of cognitive performance compared to age, sex, and education. Finger-tapping holds potential as a remote language-agnostic screening tool to stratify community cohorts at risk for cognitive decline.

Estimating presymptomatic episodic memory impairment using simple hand movement tests: A cross-sectional study of a large sample of older adults

INTRODUCTION

Finding low-cost methods to detect early-stage Alzheimer's disease (AD) is a research priority for neuroprotective drug development. Presymptomatic Alzheimer's is associated with gait impairment but hand motor tests, which are more accessible, have hardly been investigated. This study evaluated how home-based Tasmanian (TAS) Test keyboard tapping tests predict episodic memory performance.

METHODS

1169 community participants (65.8 ± 7.4 years old; 73% female) without cognitive symptoms completed online single-key and alternate-key tapping tests and episodic memory, working memory, and executive function cognitive tests.

RESULTS

All single-key (R2 adj  = 8.8%, ΔAIC = 5.2) and alternate-key (R2 adj  = 9.1%, ΔAIC = 8.8) motor features predicted episodic memory performance relative to demographic and mood confounders only (R2 adj  = 8.1%). No tapping features improved estimation of working memory.

DISCUSSION

Brief self-administered online hand movement tests predict asymptomatic episodic memory impairment. This provides a potential low-cost home-based method for stratification of enriched cohorts.

HIGHLIGHTS

We devised two brief online keyboard tapping tests to assess hand motor function. 1169 cognitively asymptomatic adults completed motor- and cognitive tests online. Impaired hand motor function predicted reduced episodic memory performance. This brief self-administered test may aid stratification of community cohorts.

Toward Personalized Medicine Approaches for Parkinson Disease Using Digital Technologies

Parkinson disease (PD) is a complex neurodegenerative disorder that afflicts over 10 million people worldwide, resulting in debilitating motor and cognitive impairment. In the United States alone (with approximately 1 million cases), the economic burden for treating and caring for persons with PD exceeds US $50 billion and myriad therapeutic approaches are under development, including both symptomatic- and disease-modifying agents. The challenges presented in addressing PD are compounded by observations that numerous, statistically distinct patient phenotypes present with a wide variety of motor and nonmotor symptomatic profiles, varying responses to current standard-of-care symptom-alleviating medications (L-DOPA and dopaminergic agonists), and different disease trajectories. The existence of these differing phenotypes highlights the opportunities in personalized approaches to symptom management and disease control. The prodromal period of PD can span across several decades, allowing the potential to leverage the unique array of composite symptoms presented to trigger early interventions. This may be especially beneficial as disease progression in PD (alongside Alzheimer disease and Huntington disease) may be influenced by biological processes such as oxidative stress, offering the potential for individual lifestyle factors to be tailored to delay disease onset. In this viewpoint, we offer potential scenarios where emerging diagnostic and monitoring strategies might be tailored to the individual patient under the tenets of P4 medicine (predict, prevent, personalize, and participate). These approaches may be especially relevant as the causative factors and biochemical pathways responsible for the observed neurodegeneration in patients with PD remain areas of fluid debate. The numerous observational patient cohorts established globally offer an excellent opportunity to test and refine approaches to detect, characterize, control, modify the course, and ultimately stop progression of this debilitating disease. Such approaches may also help development of parallel interventive strategies in other diseases such as Alzheimer disease and Huntington disease, which share common traits and etiologies with PD. In this overview, we highlight near-term opportunities to apply P4 medicine principles for patients with PD and introduce the concept of composite orthogonal patient monitoring.

Depressive Disorder Remote Detection through Touchscreen Typing Behaviour

Depressive Disorder (DD) is a leading cause of disability worldwide. Passive tools for screening the symptoms of DD are essential in monitoring and limiting the spread of the disease. From an alternative perspective, individuals' kinetic expression and activities, including smartphone interaction, reflect their mental status. Such widely available data in everyday life form a promising source of information on keystroke dynamics and their characteristics. This work explores how keystroke dynamics derived from touchscreen typing patterns have revealed the diagnosis of mental disorders, particularly depressive disorders. Different deep learning approaches were established to detect patients' depressive tendencies denoted by the self-reported Patient Health Questionnaire-9 (PHQ-9) score based on keystroke digital biomarkers. In particular, Convolutional Neural Networks (CNN), Long-Short-Term-Memory (LSTM), and CNN-LSTM models were examined and compared. The keystroke sequences are captured unobtrusively during routine interaction with touchscreen smartphones in a non-clinical setting. This study used 23,264 typing sessions provided by 10 DD patients and 14 healthy controls (HC). The proposed approach was investigated under two keystroke feature combinations and validated utilizing a leave-one-subject-out (LOSO) cross-validation scheme. The best-performing LSTM-with-hold-time (LSTM-HT) model achieved an Area Under Curve (AUC) of 0.86 with the correlated probabilities for subjects' status [95% confidence interval (CI):0.66-1.00, sensitivity/specificity (SE/SP) of 0.8/0.93].Clinical relevance- The findings of this research have the potential to contribute to improving digital tools for objectively screening mental disorders in the wild. Moreover, they would potentially provide the users and their attending psychiatrists with information regarding the evolution of their mental health.

Parkinsonian Tremor Detection with Compact Convolutional Transformer from Bispectrum Representation of tri-Axial Accelerometer Signals

After the breakthroughs of Transformer networks in Natural Language Processing (NLP) tasks, they have led to exciting progress in visual tasks as well. Nonetheless, there has been a parallel growth in the number of parameters and the amount of training data, which led to the conclusion that Transformers are not suited for small datasets. This paper is the first to convey the feasibility of Compact Convolutional Transformers (CCT) for the prediction of Parkinsonian postural tremor based on the Bispectrum (BS) representation of IMU accelerometer time series. The dataset includes tri-axial accelerometer signals collected unobtrusively in-the-wild while subjects are on a phone call, and labelled by neurologists and signal processing experts. The BS is a noise-immune, higher-order representation that reflects a signal's deviation from Gaussianity and measures quadratic phase coupling. We performed comparative classification experiments using the CCT, pre-trained CNNs such as VGG-16 and ResNet-50, and the conventional Vision Transformer (ViT). Our model achieves competitive prediction accuracy and F1 score of 96% with only 1.016 M trainable parameters, compared to the ViT with 21.659 M trainable parameters, in a five-fold cross-validation scheme. Our model also outperforms pre-trained CNNs such as VGG-16 and ResNet-50. Furthermore, we show that the performance gains are maintained when training on a larger dataset of BS images. Our effort here is motivated by the hypothesis that data-efficient transformers outperform transfer learning using pre-trained CNNs, paving the way for promising deep learning architecture for small-scale, novel and noisy medical imaging datasets.Clinical relevance- Novel deep learning model for unobtrusive prediction of Parkinsonian Postural Tremor from Bispectrum image representation of tri-axial accelerometer signals collected in-the-wild.

Alzheimer's early detection in post-acute COVID-19 syndrome: a systematic review and expert consensus on preclinical assessments

Introduction

The risk of developing Alzheimer's disease (AD) in older adults increasingly is being discussed in the literature on Post-Acute COVID-19 Syndrome (PACS). Remote digital Assessments for Preclinical AD (RAPAs) are becoming more important in screening for early AD, and should always be available for PACS patients, especially for patients at risk of AD. This systematic review examines the potential for using RAPA to identify impairments in PACS patients, scrutinizes the supporting evidence, and describes the recommendations of experts regarding their use.

Methods

We conducted a thorough search using the PubMed and Embase databases. Systematic reviews (with or without meta-analysis), narrative reviews, and observational studies that assessed patients with PACS on specific RAPAs were included. The RAPAs that were identified looked for impairments in olfactory, eye-tracking, graphical, speech and language, central auditory, or spatial navigation abilities. The recommendations' final grades were determined by evaluating the strength of the evidence and by having a consensus discussion about the results of the Delphi rounds among an international Delphi consensus panel called IMPACT, sponsored by the French National Research Agency. The consensus panel included 11 international experts from France, Switzerland, and Canada.

Results

Based on the available evidence, olfaction is the most long-lasting impairment found in PACS patients. However, while olfaction is the most prevalent impairment, expert consensus statements recommend that AD olfactory screening should not be used on patients with a history of PACS at this point in time. Experts recommend that olfactory screenings can only be recommended once those under study have reported full recovery. This is particularly important for the deployment of the olfactory identification subdimension. The expert assessment that more long-term studies are needed after a period of full recovery, suggests that this consensus statement requires an update in a few years.

Conclusion

Based on available evidence, olfaction could be long-lasting in PACS patients. However, according to expert consensus statements, AD olfactory screening is not recommended for patients with a history of PACS until complete recovery has been confirmed in the literature, particularly for the identification sub-dimension. This consensus statement may require an update in a few years.

A Review on Smartphone Keystroke Dynamics as a Digital Biomarker for Understanding Neurocognitive Functioning

Can digital technologies provide a passive unobtrusive means to observe and study cognition outside of the laboratory? Previously, cognitive assessments and monitoring were conducted in a laboratory or clinical setting, allowing for a cross-sectional glimpse of cognitive states. In the last decade, researchers have been utilizing technological advances and devices to explore ways of assessing cognition in the real world. We propose that the virtual keyboard of smartphones, an increasingly ubiquitous digital device, can provide the ideal conduit for passive data collection to study cognition. Passive data collection occurs without the active engagement of a participant and allows for near-continuous, objective data collection. Most importantly, this data collection can occur in the real world, capturing authentic datapoints. This method of data collection and its analyses provide a more comprehensive and potentially more suitable insight into cognitive states, as intra-individual cognitive fluctuations over time have shown to be an early manifestation of cognitive decline. We review different ways passive data, centered around keystroke dynamics, collected from smartphones, have been used to assess and evaluate cognition. We also discuss gaps in the literature where future directions of utilizing passive data can continue to provide inferences into cognition and elaborate on the importance of digital data privacy and consent.

Associations Between Smartphone Keystroke Metadata and Mental Health Symptoms in Adolescents: Findings From the Future Proofing Study

BACKGROUND

Mental disorders are prevalent during adolescence. Among the digital phenotypes currently being developed to monitor mental health symptoms, typing behavior is one promising candidate. However, few studies have directly assessed associations between typing behavior and mental health symptom severity, and whether these relationships differs between genders.

OBJECTIVE

In a cross-sectional analysis of a large cohort, we tested whether various features of typing behavior derived from keystroke metadata were associated with mental health symptoms and whether these relationships differed between genders.

METHODS

A total of 934 adolescents from the Future Proofing study undertook 2 typing tasks on their smartphones through the Future Proofing app. Common keystroke timing and frequency features were extracted across tasks. Mental health symptoms were assessed using the Patient Health Questionnaire-Adolescent version, the Children's Anxiety Scale-Short Form, the Distress Questionnaire 5, and the Insomnia Severity Index. Bivariate correlations were used to test whether keystroke features were associated with mental health symptoms. The false discovery rates of P values were adjusted to q values. Machine learning models were trained and tested using independent samples (ie, 80% train 20% test) to identify whether keystroke features could be combined to predict mental health symptoms.

RESULTS

Keystroke timing features showed a weak negative association with mental health symptoms across participants. When split by gender, females showed weak negative relationships between keystroke timing features and mental health symptoms, and weak positive relationships between keystroke frequency features and mental health symptoms. The opposite relationships were found for males (except for dwell). Machine learning models using keystroke features alone did not predict mental health symptoms.

CONCLUSIONS

Increased mental health symptoms are weakly associated with faster typing, with important gender differences. Keystroke metadata should be collected longitudinally and combined with other digital phenotypes to enhance their clinical relevance.

TRIAL REGISTRATION

Australian and New Zealand Clinical Trial Registry, ACTRN12619000855123; https://www.anzctr.org.au/Trial/Registration/TrialReview.aspx?id=377664&isReview=true.

The convergence of traditional and digital biomarkers through AI-assisted biosensing: A new era in translational diagnostics?

Advances in consumer electronics, alongside the fields of microfluidics and nanotechnology have brought to the fore low-cost wearable/portable smart devices. Although numerous smart devices that track digital biomarkers have been successfully translated from bench-to-bedside, only a few follow the same fate when it comes to track traditional biomarkers. Current practices still involve laboratory-based tests, followed by blood collection, conducted in a clinical setting as they require trained personnel and specialized equipment. In fact, real-time, passive/active and robust sensing of physiological and behavioural data from patients that can feed artificial intelligence (AI)-based models can significantly improve decision-making, diagnosis and treatment at the point-of-procedure, by circumventing conventional methods of sampling, and in person investigation by expert pathologists, who are scarce in developing countries. This review brings together conventional and digital biomarker sensing through portable and autonomous miniaturized devices. We first summarise the technological advances in each field vs the current clinical practices and we conclude by merging the two worlds of traditional and digital biomarkers through AI/ML technologies to improve patient diagnosis and treatment. The fundamental role, limitations and prospects of AI in realizing this potential and enhancing the existing technologies to facilitate the development and clinical translation of "point-of-care" (POC) diagnostics is finally showcased.

A scoping review of neurodegenerative manifestations in explainable digital phenotyping

Neurologists nowadays no longer view neurodegenerative diseases, like Parkinson's and Alzheimer's disease, as single entities, but rather as a spectrum of multifaceted symptoms with heterogeneous progression courses and treatment responses. The definition of the naturalistic behavioral repertoire of early neurodegenerative manifestations is still elusive, impeding early diagnosis and intervention. Central to this view is the role of artificial intelligence (AI) in reinforcing the depth of phenotypic information, thereby supporting the paradigm shift to precision medicine and personalized healthcare. This suggestion advocates the definition of disease subtypes in a new biomarker-supported nosology framework, yet without empirical consensus on standardization, reliability and interpretability. Although the well-defined neurodegenerative processes, linked to a triad of motor and non-motor preclinical symptoms, are detected by clinical intuition, we undertake an unbiased data-driven approach to identify different patterns of neuropathology distribution based on the naturalistic behavior data inherent to populations in-the-wild. We appraise the role of remote technologies in the definition of digital phenotyping specific to brain-, body- and social-level neurodegenerative subtle symptoms, emphasizing inter- and intra-patient variability powered by deep learning. As such, the present review endeavors to exploit digital technologies and AI to create disease-specific phenotypic explanations, facilitating the understanding of neurodegenerative diseases as "bio-psycho-social" conditions. Not only does this translational effort within explainable digital phenotyping foster the understanding of disease-induced traits, but it also enhances diagnostic and, eventually, treatment personalization.

Common multi-day rhythms in smartphone behavior

The idea that abnormal human activities follow multi-day rhythms is found in ancient beliefs on the moon to modern clinical observations in epilepsy and mood disorders. To explore multi-day rhythms in healthy human behavior our analysis includes over 300 million smartphone touchscreen interactions logging up to 2 years of day-to-day activities (N401 subjects). At the level of each individual, we find a complex expression of multi-day rhythms where the rhythms occur scattered across diverse smartphone behaviors. With non-negative matrix factorization, we extract the scattered rhythms to reveal periods ranging from 7 to 52 days - cutting across age and gender. The rhythms are likely free-running - instead of being ubiquitously driven by the moon - as they did not show broad population-level synchronization even though the sampled population lived in northern Europe. We propose that multi-day rhythms are a common trait, but their consequences are uniquely experienced in day-to-day behavior.

Exploring the Complex Phenotypes of Impaired Finger Dexterity in Mild-to-moderate Stage Parkinson's Disease: A Time-Series Analysis

BACKGROUND

Impaired dexterity is an early motor symptom in Parkinson's disease (PD) that significantly impacts the daily activity of patients; however, what constitutes complex dexterous movements remains controversial.

OBJECTIVE

To explore the characteristics of finger dexterity in mild-to-moderate stage PD.

METHODS

We quantitatively assessed finger dexterity in 48 mild-to-moderate stage PD patients and 49 age-matched controls using a simple alternating two-finger typing test for 15 seconds. Time-series analyses of various kinematic parameters with machine learning were compared between sides and groups.

RESULTS

Both the more and less affected hands of patients with PD had significantly lower typing frequency and slower typing velocity than the non-dominant and the dominant hands of controls (p = 0.019, p = 0.016, p < 0.001, p < 0.001). The slope of the typing velocity decreased with time, indicating a sequence effect in the PD group. A typing duration of 6 seconds was determined sufficient to discriminate PD patients from controls. Typing error, repetition, and repetition rate were significantly higher in the more affected hands of patients with PD than in the non-dominant hand of controls (p < 0.001, p = 0.03, p < 0.001). The error rate was constant, whereas the repetition rate was steep during the initiation of typing. A predictive model of the more affected hand demonstrated an accuracy of 70% in differentiating PD patients from controls.

CONCLUSION

Our study demonstrated complex components of impaired finger dexterity in mild-to-moderate stage PD, namely bradykinesia with sequence effects, error, and repetition at the initiation of movement, suggesting that multiple neural networks may be involved in dexterity deficits in PD.

Associations between smartphone keystroke dynamics and cognition in MS

Objective

Examine the associations between smartphone keystroke dynamics and cognitive functioning among persons with multiple sclerosis (MS).

Methods

Sixteen persons with MS with no self-reported upper extremity or typing difficulties and 10 healthy controls (HCs) completed six weeks of remote monitoring of their keystroke dynamics (i.e., how they typed on their smartphone keyboards). They also completed a comprehensive neuropsychological assessment and symptom ratings about fatigue, depression, and anxiety at baseline.

Results

A total of 1,335,787 keystrokes were collected, which were part of 30,968 typing sessions. The MS group typed slower (P < .001) and more variably (P = .032) than the HC group. Faster typing speed was associated with better performance on measures of processing speed (P = .016), attention (P = .022), and executive functioning (cognitive flexibility: P = .029; behavioral inhibition: P = .002; verbal fluency: P = .039), as well as less severe impact from fatigue (P < .001) and less severe anxiety symptoms (P = .007). Those with better cognitive functioning and less severe symptoms showed a stronger correlation between the use of backspace and autocorrection events (P < .001).

Conclusion

Typing speed may be sensitive to cognitive functions subserved by the frontal-subcortical brain circuits. Individuals with better cognitive functioning and less severe symptoms may be better at monitoring their typing errors. Keystroke dynamics have the potential to be used as an unobtrusive remote monitoring method for real-life cognitive functioning among persons with MS, which may improve the detection of relapses, evaluate treatment efficacy, and track disability progression.

A longitudinal observational study of home-based conversations for detecting early dementia: protocol for the CUBOId TV task

INTRODUCTION

Limitations in effective dementia therapies mean that early diagnosis and monitoring are critical for disease management, but current clinical tools are impractical and/or unreliable, and disregard short-term symptom variability. Behavioural biomarkers of cognitive decline, such as speech, sleep and activity patterns, can manifest prodromal pathological changes. They can be continuously measured at home with smart sensing technologies, and permit leveraging of interpersonal interactions for optimising diagnostic and prognostic performance. Here we describe the ContinUous behavioural Biomarkers Of cognitive Impairment (CUBOId) study, which explores the feasibility of multimodal data fusion for in-home monitoring of mild cognitive impairment (MCI) and early Alzheimer's disease (AD). The report focuses on a subset of CUBOId participants who perform a novel speech task, the 'TV task', designed to track changes in ecologically valid conversations with disease progression.

METHODS AND ANALYSIS

CUBOId is a longitudinal observational study. Participants have diagnoses of MCI or AD, and controls are their live-in partners with no such diagnosis. Multimodal activity data were passively acquired from wearables and in-home fixed sensors over timespans of 8-25 months. At two time points participants completed the TV task over 5 days by recording audio of their conversations as they watched a favourite TV programme, with further testing to be completed after removal of the sensor installations. Behavioural testing is supported by neuropsychological assessment for deriving ground truths on cognitive status. Deep learning will be used to generate fused multimodal activity-speech embeddings for optimisation of diagnostic and predictive performance from speech alone.

ETHICS AND DISSEMINATION

CUBOId was approved by an NHS Research Ethics Committee (Wales REC; ref: 18/WA/0158) and is sponsored by University of Bristol. It is supported by the National Institute for Health Research Clinical Research Network West of England. Results will be reported at conferences and in peer-reviewed scientific journals.

Detection of Mental Fatigue in the General Population: Feasibility Study of Keystroke Dynamics as a Real-world Biomarker

Background

Mental fatigue is a common and potentially debilitating state that can affect individuals’ health and quality of life. In some cases, its manifestation can precede or mask early signs of other serious mental or physiological conditions. Detecting and assessing mental fatigue can be challenging nowadays as it relies on self-evaluation and rating questionnaires, which are highly influenced by subjective bias. Introducing more objective, quantitative, and sensitive methods to characterize mental fatigue could be critical to improve its management and the understanding of its connection to other clinical conditions.

Objective

This paper aimed to study the feasibility of using keystroke biometrics for mental fatigue detection during natural typing. As typing involves multiple motor and cognitive processes that are affected by mental fatigue, our hypothesis was that the information captured in keystroke dynamics can offer an interesting mean to characterize users’ mental fatigue in a real-world setting.

Methods

We apply domain transformation techniques to adapt and transform TypeNet, a state-of-the-art deep neural network, originally intended for user authentication, to generate a network optimized for the fatigue detection task. All experiments were conducted using 3 keystroke databases that comprise different contexts and data collection protocols.

Results

Our preliminary results showed area under the curve performances ranging between 72.2% and 80% for fatigue versus rested sample classification, which is aligned with previously published models on daily alertness and circadian cycles. This demonstrates the potential of our proposed system to characterize mental fatigue fluctuations via natural typing patterns. Finally, we studied the performance of an active detection approach that leverages the continuous nature of keystroke biometric patterns for the assessment of users’ fatigue in real time.

Conclusions

Our results suggest that the psychomotor patterns that characterize mental fatigue manifest during natural typing, which can be quantified via automated analysis of users’ daily interaction with their device. These findings represent a step towards the development of a more objective, accessible, and transparent solution to monitor mental fatigue in a real-world environment.

Modified SqueezeNet Architecture for Parkinson's Disease Detection Based on Keypress Data

Parkinson's disease (PD) is the most common form of Parkinsonism, which is a group of neurological disorders with PD-like motor impairments. The disease affects over 6 million people worldwide and is characterized by motor and non-motor symptoms. The affected person has trouble in controlling movements, which may affect simple daily-life tasks, such as typing on a computer. We propose the application of a modified SqueezeNet convolutional neural network (CNN) for detecting PD based on the subject's key-typing patterns. First, the data are pre-processed using data standardization and the Synthetic Minority Oversampling Technique (SMOTE), and then a Continuous Wavelet Transformation is applied to generate spectrograms used for training and testing a modified SqueezeNet model. The modified SqueezeNet model achieved an accuracy of 90%, representing a noticeable improvement in comparison to other approaches.

Scent-delivery devices as a digital healthcare tool for olfactory training: A pilot focus group study in Parkinson's disease patients

Parkinson's disease (PD) patients display a combination of motor and non-motor symptoms. The most common non-motor symptom is scent (olfactory) impairment, occurring at least four years prior to motor symptom onset. Recent and growing interest in digital healthcare technology used in PD has resulted in more technologies developed for motor rather than non-motor symptoms. Human-computer interaction (HCI), which uses computer technology to explore human activity and work, could be combined with digital healthcare technologies to better understand and support olfaction via scent training - leading to the development of a scent-delivery device (SDD). In this pilot study, three PD patients were invited to an online focus group to explore the association between PD and olfaction, understand HCI and sensory technologies and were demonstrated a new multichannel SDD with an associated mobile app. Participants had a preconceived link, a result of personal experience, between olfactory impairment and PD. Participants felt that healthcare professionals did not take olfactory dysfunction concerns seriously prior to PD diagnosis. Two were not comfortable with sharing scent loss experiences with others. Participants expected the multichannel SDD to be small, portable and easy-to-use, with customisable cartridges to deliver chosen scents and the mobile app to create a sense of community. None of the participants regularly performed scent training but would consider doing so if some scent function could be regained. Standardised digital SDDs for regular healthcare check-ups may facilitate improvement in olfactory senses in PD patients and potential earlier PD diagnosis, allowing earlier therapeutic and symptomatic PD management.