Smartphone-derived keystroke dynamics are sensitive to relevant changes in multiple sclerosis

Digital Neuropsychology beyond Computerized Cognitive Assessment: Applications of Novel Digital Technologies

Compared with other health disciplines, there is a stagnation in technological innovation in the field of clinical neuropsychology. Traditional paper-and-pencil tests have a number of shortcomings, such as low-frequency data collection and limitations in ecological validity. While computerized cognitive assessment may help overcome some of these issues, current computerized paradigms do not address the majority of these limitations. In this paper, we review recent literature on the applications of novel digital health approaches, including ecological momentary assessment, smartphone-based assessment and sensors, wearable devices, passive driving sensors, smart homes, voice biomarkers, and electronic health record mining, in neurological populations. We describe how each digital tool may be applied to neurologic care and overcome limitations of traditional neuropsychological assessment. Ethical considerations, limitations of current research, as well as our proposed future of neuropsychological practice are also discussed.

[Digital medicine in neurological research-Between hype and evidence]

BACKGROUND

The rapid advancement of digital medicine and health technologies in neurology offers both significant potential and challenges. This article outlines fundamental aspects of digital medicine related to neurological research and highlights application examples of digital technologies in neurological research.

AIM

To provide a comprehensive overview of current digital developments in neurology and their impact on neurological research.

MATERIAL AND METHODS

In this narrative review articles from various sources and references related to digital medicine and health technologies in neurology were compiled and analyzed.

RESULTS AND DISCUSSION

The data presented indicate that digital health technologies and digital therapeutics have the potential to decisively shape neurological care and research; however, it is emphasized that a critical evaluation and evidence-based approach to these technologies are essential to determine their actual value in neurology.

The effect of socially evaluated multitasking stress on typing rhythms

Individuals have unique typing rhythms characterized by specific keystroke dynamics. Changes in state and cardiovascular responding are well documented manifestations of the fight-flight response to stress. However, as stress also leads to changes in muscle tone and motor control, typing rhythms may also be impacted. We aim to determine which individuals are experiencing stress through their typing rhythms and identify universal keystroke markers of stress. Participants (N = 116) typed 80 repetitions of a 6-word, 30-character phrase before and after 15 min of critically evaluated multitasking stress. Cardiovascular, hemodynamic, and state variables were compared across baseline, stress, and recovery periods and measures of typing rhythm were derived for each period and classified using machine-learning algorithms. Critically evaluated multitasking led to significant changes in all stress measures, demonstrating highly robust stress reactivity. Machine learning algorithms accurately classified stressed typing for each individual based on their typing rhythms; however, no universal keystroke markers of stress were identified. Using typing rhythms. We were able to determine whether an individual was stressed or not, but the markers used for classification differed between individuals. These individual changes may provide opportunities for identifying stressful periods through keystroke monitoring, as well as the potential for early identification of disorders which may impact fine motor control. Typing rhythms could therefore be used to monitor health and well-being in individuals who use keyboards in various situations. This is the first rigorous assessment of stress and typing rhythms and has led to the development of a feasible and highly reproducible research protocol.

Wearable Sensor Technologies to Assess Motor Functions in People With Multiple Sclerosis: Systematic Scoping Review and Perspective

BACKGROUND

Wearable sensor technologies have the potential to improve monitoring in people with multiple sclerosis (MS) and inform timely disease management decisions. Evidence of the utility of wearable sensor technologies in people with MS is accumulating but is generally limited to specific subgroups of patients, clinical or laboratory settings, and functional domains.

OBJECTIVE

This review aims to provide a comprehensive overview of all studies that have used wearable sensors to assess, monitor, and quantify motor function in people with MS during daily activities or in a controlled laboratory setting and to shed light on the technological advances over the past decades.

METHODS

We systematically reviewed studies on wearable sensors to assess the motor performance of people with MS. We scanned PubMed, Scopus, Embase, and Web of Science databases until December 31, 2022, considering search terms "multiple sclerosis" and those associated with wearable technologies and included all studies assessing motor functions. The types of results from relevant studies were systematically mapped into 9 predefined categories (association with clinical scores or other measures; test-retest reliability; group differences, 3 types; responsiveness to change or intervention; and acceptability to study participants), and the reporting quality was determined through 9 questions. We followed the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) reporting guidelines.

RESULTS

Of the 1251 identified publications, 308 were included: 176 (57.1%) in a real-world context, 107 (34.7%) in a laboratory context, and 25 (8.1%) in a mixed context. Most publications studied physical activity (196/308, 63.6%), followed by gait (81/308, 26.3%), dexterity or tremor (38/308, 12.3%), and balance (34/308, 11%). In the laboratory setting, outcome measures included (in addition to clinical severity scores) 2- and 6-minute walking tests, timed 25-foot walking test, timed up and go, stair climbing, balance tests, and finger-to-nose test, among others. The most popular anatomical landmarks for wearable placement were the waist, wrist, and lower back. Triaxial accelerometers were most commonly used (229/308, 74.4%). A surge in the number of sensors embedded in smartphones and smartwatches has been observed. Overall, the reporting quality was good.

CONCLUSIONS

Continuous monitoring with wearable sensors could optimize the management of people with MS, but some hurdles still exist to full clinical adoption of digital monitoring. Despite a possible publication bias and vast heterogeneity in the outcomes reported, our review provides an overview of the current literature on wearable sensor technologies used for people with MS and highlights shortcomings, such as the lack of harmonization, transparency in reporting methods and results, and limited data availability for the research community. These limitations need to be addressed for the growing implementation of wearable sensor technologies in clinical routine and clinical trials, which is of utmost importance for further progress in clinical research and daily management of people with MS.

TRIAL REGISTRATION

PROSPERO CRD42021243249; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=243249.

The convergence of traditional and digital biomarkers through AI-assisted biosensing: A new era in translational diagnostics?

Advances in consumer electronics, alongside the fields of microfluidics and nanotechnology have brought to the fore low-cost wearable/portable smart devices. Although numerous smart devices that track digital biomarkers have been successfully translated from bench-to-bedside, only a few follow the same fate when it comes to track traditional biomarkers. Current practices still involve laboratory-based tests, followed by blood collection, conducted in a clinical setting as they require trained personnel and specialized equipment. In fact, real-time, passive/active and robust sensing of physiological and behavioural data from patients that can feed artificial intelligence (AI)-based models can significantly improve decision-making, diagnosis and treatment at the point-of-procedure, by circumventing conventional methods of sampling, and in person investigation by expert pathologists, who are scarce in developing countries. This review brings together conventional and digital biomarker sensing through portable and autonomous miniaturized devices. We first summarise the technological advances in each field vs the current clinical practices and we conclude by merging the two worlds of traditional and digital biomarkers through AI/ML technologies to improve patient diagnosis and treatment. The fundamental role, limitations and prospects of AI in realizing this potential and enhancing the existing technologies to facilitate the development and clinical translation of "point-of-care" (POC) diagnostics is finally showcased.

Towards a future where Alzheimer's disease pathology is stopped before the onset of dementia

Alzheimer's disease (AD) is a major healthcare challenge with no curative treatment at present. To address this challenge, we need a paradigm shift, where we focus on pre-dementia stages of AD. In this Perspective, we outline a strategy to move towards a future with personalized medicine for AD by preparing for and investing in effective and patient-orchestrated diagnosis, prediction and prevention of the dementia stage. While focusing on AD, this Perspective also discusses studies that do not specify the cause of dementia. Future personalized prevention strategies encompass multiple components, including tailored combinations of disease-modifying interventions and lifestyle. By empowering the public and patients to be more actively engaged in the management of their health and disease and by developing improved strategies for diagnosis, prediction and prevention, we can pave the way for a future with personalized medicine, in which AD pathology is stopped to prevent or delay the onset of dementia.

Dense Sampling Approaches for Psychiatry Research: Combining Scanners and Smartphones

Together, data from brain scanners and smartphones have sufficient coverage of biology, psychology, and environment to articulate between-person differences in the interplay within and across biological, psychological, and environmental systems thought to underlie psychopathology. An important next step is to develop frameworks that combine these two modalities in ways that leverage their coverage across layers of human experience to have maximum impact on our understanding and treatment of psychopathology. We review literature published in the last 3 years highlighting how scanners and smartphones have been combined to date, outline and discuss the strengths and weaknesses of existing approaches, and sketch a network science framework heretofore underrepresented in work combining scanners and smartphones that can push forward our understanding of health and disease.

Disease severity classification using passively collected smartphone-based keystroke dynamics within multiple sclerosis

Multiple Sclerosis (MS) is a progressive demyelinating disease of the central nervous system characterised by a wide range of motor and non-motor symptoms. The level of disability of people with MS (pwMS) is based on a wide range of clinical measures, though their frequency of evaluation and inaccuracies coming from objective and self-reported evaluations limits these assessments. Alternatively, remote health monitoring through devices can offer a cost-efficient solution to gather more reliable, objective measures continuously. Measuring smartphone keyboard interactions is a promising tool since typing and, thus, keystroke dynamics are likely influenced by symptoms that pwMS can experience. Therefore, this paper aims to investigate whether keyboard interactions gathered on a person's smartphone can provide insight into the clinical status of pwMS leveraging machine learning techniques. In total, 24 Healthy Controls (HC) and 102 pwMS were followed for one year. Next to continuous data generated via smartphone interactions, clinical outcome measures were collected and used as targets to train four independent multivariate binary classification pipelines in discerning pwMS versus HC and estimating the level of disease severity, manual dexterity and cognitive capabilities. The final models yielded an AUC-ROC in the hold-out set above 0.7, with the highest performance obtained in estimating the level of fine motor skills (AUC-ROC=0.753). These findings show that keyboard interactions combined with machine learning techniques can be used as an unobtrusive monitoring tool to estimate various levels of clinical disability in pwMS from daily activities and with a high frequency of sampling without increasing patient burden.

Integrating Rehabilomics into the Multi-Omics Approach in the Management of Multiple Sclerosis: The Way for Precision Medicine?

Over recent years, significant improvements have been made in the understanding of (epi)genetics and neuropathophysiological mechanisms driving the different forms of multiple sclerosis (MS). For example, the role and importance of the bidirectional communications between the brain and the gut-also referred to as the gut-brain axis-in the pathogenesis of MS is receiving increasing interest in recent years and is probably one of the most promising areas of research for the management of people with MS. However, despite these important advances, it must be noted that these data are not-yet-used in rehabilitation. Neurorehabilitation is a cornerstone of MS patient management, and there are many techniques available to clinicians and patients, including technology-supported rehabilitation. In this paper, we will discuss how new findings on the gut microbiome could help us to better understand how rehabilitation can improve motor and cognitive functions. We will also see how the data gathered during the rehabilitation can help to get a better diagnosis of the patients. Finally, we will discuss how these new techniques can better guide rehabilitation to lead to precision rehabilitation and ultimately increase the quality of patient care.

Associations between smartphone keystroke dynamics and cognition in MS

Objective

Examine the associations between smartphone keystroke dynamics and cognitive functioning among persons with multiple sclerosis (MS).

Methods

Sixteen persons with MS with no self-reported upper extremity or typing difficulties and 10 healthy controls (HCs) completed six weeks of remote monitoring of their keystroke dynamics (i.e., how they typed on their smartphone keyboards). They also completed a comprehensive neuropsychological assessment and symptom ratings about fatigue, depression, and anxiety at baseline.

Results

A total of 1,335,787 keystrokes were collected, which were part of 30,968 typing sessions. The MS group typed slower (P < .001) and more variably (P = .032) than the HC group. Faster typing speed was associated with better performance on measures of processing speed (P = .016), attention (P = .022), and executive functioning (cognitive flexibility: P = .029; behavioral inhibition: P = .002; verbal fluency: P = .039), as well as less severe impact from fatigue (P < .001) and less severe anxiety symptoms (P = .007). Those with better cognitive functioning and less severe symptoms showed a stronger correlation between the use of backspace and autocorrection events (P < .001).

Conclusion

Typing speed may be sensitive to cognitive functions subserved by the frontal-subcortical brain circuits. Individuals with better cognitive functioning and less severe symptoms may be better at monitoring their typing errors. Keystroke dynamics have the potential to be used as an unobtrusive remote monitoring method for real-life cognitive functioning among persons with MS, which may improve the detection of relapses, evaluate treatment efficacy, and track disability progression.

The Use of Smartphone Keystroke Dynamics to Passively Monitor Upper Limb and Cognitive Function in Multiple Sclerosis: Longitudinal Analysis

Background

Typing on smartphones, which has become a near daily activity, requires both upper limb and cognitive function. Analysis of keyboard interactions during regular typing, that is, keystroke dynamics, could therefore potentially be utilized for passive and continuous monitoring of function in patients with multiple sclerosis.

Objective

To determine whether passively acquired smartphone keystroke dynamics correspond to multiple sclerosis outcomes, we investigated the association between keystroke dynamics and clinical outcomes (upper limb and cognitive function). This association was investigated longitudinally in order to study within-patient changes independently of between-patient differences.

Methods

During a 1-year follow-up, arm function and information processing speed were assessed every 3 months in 102 patients with multiple sclerosis with the Nine-Hole Peg Test and Symbol Digit Modalities Test, respectively. Keystroke-dynamics data were continuously obtained from regular typing on the participants’ own smartphones. Press-and-release latency of the alphanumeric keys constituted the fine motor score cluster, while latency of the punctuation and backspace keys constituted the cognition score cluster. The association over time between keystroke clusters and the corresponding clinical outcomes was assessed with linear mixed models with subjects as random intercepts. By centering around the mean and calculating deviation scores within subjects, between-subject and within-subject effects were distinguished.

Results

Mean (SD) scores for the fine motor score cluster and cognition score cluster were 0.43 (0.16) and 0.94 (0.41) seconds, respectively. The fine motor score cluster was significantly associated with the Nine-Hole Peg Test: between-subject β was 15.9 (95% CI 12.2-19.6) and within-subject β was 6.9 (95% CI 2.0-11.9). The cognition score cluster was significantly associated with the Symbol Digit Modalities Test between subjects (between-subject β –11.2, 95% CI –17.3 to –5.2) but not within subjects (within-subject β –0.4, 95% CI –5.6 to 4.9).

Conclusions

Smartphone keystroke dynamics were longitudinally associated with multiple sclerosis outcomes. Worse arm function corresponded with longer latency in typing both across and within patients. Worse processing speed corresponded with higher latency in using punctuation and backspace keys across subjects. Hence, keystroke dynamics are a potential digital biomarker for remote monitoring and predicting clinical outcomes in patients with multiple sclerosis.

Trial Registration

Netherlands Trial Register NTR7268; https://trialsearch.who.int/Trial2.aspx?TrialID=NTR7268

Cognitive Writing Process Characteristics in Alzheimer's Disease

In this article, we explore if the observation of writing behavior can assist in the screening and follow-up of mild cognitive impairment (MCI) and mild dementia due to Alzheimer's disease (AD). To this end, we examined the extent to which overall writing process measures and pausing behavior during writing differed between 15 cognitively impaired patients and 15 age- and gender-matched healthy controls. Participants completed two typed picture description tasks that were registered with Inputlog, a keystroke logging program that captures keyboard activity during text production. The following variables were analyzed with mixed-effects models: time on task; number of characters, pauses and Pause-bursts per minute; proportion of pause time; duration of Pause-bursts; and pause time between words. For pause time between words, also the effect of pauses preceding specific word categories was analyzed. Results showed a main effect of group on all variables. In addition, for pause time between words a main effect of part-of-speech was found as well. Results indicate that writing process analysis can possibly serve as a supplementary tool for the screening and follow-up of AD.