Predicting Mood Disturbance Severity with Mobile Phone Keystroke Metadata: A BiAffect Digital Phenotyping Study

Detecting adolescent depression through passive monitoring of linguistic markers in smartphone communication

BACKGROUND

Cross sectional studies have identified linguistic correlates of major depressive disorder (MDD) in smartphone communication. However, it is unclear whether monitoring these linguistic characteristics can detect when an individual is experiencing MDD, which would facilitate timely intervention.

METHODS

Approximately 1.2 million messages typed into smartphone social communication apps (e.g. texting, social media) were passively collected from 90 adolescents with a range of depression severity over a 12-month period. Sentiment (i.e. positive vs. negative valence of text), proportions of first-person singular pronouns (e.g. 'I'), and proportions of absolutist words (e.g. 'all') were computed for each message and converted to weekly aggregates temporally aligned with weekly MDD statuses obtained from retrospective interviews. Idiographic, multilevel logistic regression models tested whether within-person deviations in these linguistic features were associated with the probability of concurrently meeting threshold for MDD.

RESULTS

Using more first-person singular pronouns in smartphone communication relative to one's own average was associated with higher odds of meeting threshold for MDD in the concurrent week (OR = 1.29; p = .007). Sentiment (OR = 1.07; p = .54) and use of absolutist words (OR = 0.99; p = .90) were not related to weekly MDD.

CONCLUSIONS

Passively monitoring use of first-person singular pronouns in adolescents' smartphone communication may help detect MDD, providing novel opportunities for early intervention.

Smartphone keyboard dynamics predict affect in suicidal ideation

While digital phenotyping provides opportunities for unobtrusive, real-time mental health assessments, the integration of its modalities is not trivial due to high dimensionalities and discrepancies in sampling frequencies. We provide an integrated pipeline that solves these issues by transforming all modalities to the same time unit, applying temporal independent component analysis (ICA) to high-dimensional modalities, and fusing the modalities with linear mixed-effects models. We applied our approach to integrate high-quality, daily self-report data with BiAffect keyboard dynamics derived from a clinical suicidality sample of mental health outpatients. Applying the ICA to the self-report data (104 participants, 5712 days of data) revealed components related to well-being, anhedonia, and irritability and social dysfunction. Mixed-effects models (55 participants, 1794 days) showed that less phone movement while typing was associated with more anhedonia (β = -0.12, p = 0.00030). We consider this method to be widely applicable to dense, longitudinal digital phenotyping data.

Investigation of daily patterns for smartphone keystroke dynamics based on loneliness and social isolation

This study examined the relationship between loneliness levels and daily patterns of mobile keystroke dynamics in healthy individuals. Sixty-six young healthy Koreans participated in the experiment. Over five weeks, the participants used a custom Android keyboard. We divided the participants into four groups based on their level of loneliness (no loneliness, moderate loneliness, severe loneliness, and very severe loneliness). The very severe loneliness group demonstrated significantly higher typing counts during sleep time than the other three groups (one-way ANOVA, F = 3.75, p < 0.05). In addition, the average cosine similarity value of weekday and weekend typing patterns in the very severe loneliness group was higher than that in the no loneliness group (Welch's t-test, t = 2.27, p < 0.05). This meant that the no loneliness group's weekday and weekend typing patterns varied, whereas the very severe loneliness group's weekday and weekend typing patterns did not. Our results indicated that individuals with very high levels of loneliness tended to use mobile keyboards during late-night hours and did not significantly change their smartphone usage behavior between weekdays and weekends. These findings suggest that mobile keystroke dynamics have the potential to be used for the early detection of loneliness and the development of targeted interventions.

Differential temporal utility of passively sensed smartphone features for depression and anxiety symptom prediction: a longitudinal cohort study

While studies show links between smartphone data and affective symptoms, we lack clarity on the temporal scale, specificity (e.g., to depression vs. anxiety), and person-specific (vs. group-level) nature of these associations. We conducted a large-scale (n = 1013) smartphone-based passive sensing study to identify within- and between-person digital markers of depression and anxiety symptoms over time. Participants (74.6% female; M age = 40.9) downloaded the LifeSense app, which facilitated continuous passive data collection (e.g., GPS, app and device use, communication) across 16 weeks. Hierarchical linear regression models tested the within- and between-person associations of 2-week windows of passively sensed data with depression (PHQ-8) or generalized anxiety (GAD-7). We used a shifting window to understand the time scale at which sensed features relate to mental health symptoms, predicting symptoms 2 weeks in the future (distal prediction), 1 week in the future (medial prediction), and 0 weeks in the future (proximal prediction). Spending more time at home relative to one's average was an early signal of PHQ-8 severity (distal β = 0.219, p = 0.012) and continued to relate to PHQ-8 at medial (β = 0.198, p = 0.022) and proximal (β = 0.183, p = 0.045) windows. In contrast, circadian movement was proximally related to (β = -0.131, p = 0.035) but did not predict (distal β = 0.034, p = 0.577; medial β = -0.089, p = 0.138) PHQ-8. Distinct communication features (i.e., call/text or app-based messaging) related to PHQ-8 and GAD-7. Findings have implications for identifying novel treatment targets, personalizing digital mental health interventions, and enhancing traditional patient-provider interactions. Certain features (e.g., circadian movement) may represent correlates but not true prospective indicators of affective symptoms. Conversely, other features like home duration may be such early signals of intra-individual symptom change, indicating the potential utility of prophylactic intervention (e.g., behavioral activation) in response to person-specific increases in these signals.

Digital and precision clinical trials: innovations for testing mental health medications, devices, and psychosocial treatments

Mental health treatment advances - including neuropsychiatric medications and devices, psychotherapies, and cognitive treatments - lag behind other fields of clinical medicine such as cardiovascular care. One reason for this gap is the traditional techniques used in mental health clinical trials, which slow the pace of progress, produce inequities in care, and undermine precision medicine goals. Newer techniques and methodologies, which we term digital and precision trials, offer solutions. These techniques consist of (1) decentralized (i.e., fully-remote) trials which improve the speed and quality of clinical trials and increase equity of access to research, (2) precision measurement which improves success rate and is essential for precision medicine, and (3) digital interventions, which offer increased reach of, and equity of access to, evidence-based treatments. These techniques and their rationales are described in detail, along with challenges and solutions for their utilization. We conclude with a vignette of a depression clinical trial using these techniques.

Smartphone keyboard dynamics predict affect in suicidal ideation

While digital phenotyping provides opportunities for unobtrusive, real-time mental health assessments, the integration of its modalities is not trivial due to high dimensionalities and discrepancies in sampling frequencies. We provide an integrated pipeline that solves these issues by transforming all modalities to the same time unit, applying temporal independent component analysis (ICA) to high-dimensional modalities, and fusing the modalities with linear mixed-effects models. We applied our approach to integrate high-quality, daily self-report data with BiAffect keyboard dynamics derived from a clinical suicidality sample of mental health outpatients. Applying the ICA to the self-report data (104 participants, 5712 days of data) revealed components related to well-being, anhedonia, and irritability and social dysfunction. Mixed-effects models (55 participants, 1794 days) showed that less phone movement while typing was associated with more anhedonia (β = -0.12, p = 0.00030). We consider this method to be widely applicable to dense, longitudinal digital phenotyping data.

Digital Phenotyping: Data-Driven Psychiatry to Redefine Mental Health

The term "digital phenotype" refers to the digital footprint left by patient-environment interactions. It has potential for both research and clinical applications but challenges our conception of health care by opposing 2 distinct approaches to medicine: one centered on illness with the aim of classifying and curing disease, and the other centered on patients, their personal distress, and their lived experiences. In the context of mental health and psychiatry, the potential benefits of digital phenotyping include creating new avenues for treatment and enabling patients to take control of their own well-being. However, this comes at the cost of sacrificing the fundamental human element of psychotherapy, which is crucial to addressing patients' distress. In this viewpoint paper, we discuss the advances rendered possible by digital phenotyping and highlight the risk that this technology may pose by partially excluding health care professionals from the diagnosis and therapeutic process, thereby foregoing an essential dimension of care. We conclude by setting out concrete recommendations on how to improve current digital phenotyping technology so that it can be harnessed to redefine mental health by empowering patients without alienating them.

The effect of socially evaluated multitasking stress on typing rhythms

Individuals have unique typing rhythms characterized by specific keystroke dynamics. Changes in state and cardiovascular responding are well documented manifestations of the fight-flight response to stress. However, as stress also leads to changes in muscle tone and motor control, typing rhythms may also be impacted. We aim to determine which individuals are experiencing stress through their typing rhythms and identify universal keystroke markers of stress. Participants (N = 116) typed 80 repetitions of a 6-word, 30-character phrase before and after 15 min of critically evaluated multitasking stress. Cardiovascular, hemodynamic, and state variables were compared across baseline, stress, and recovery periods and measures of typing rhythm were derived for each period and classified using machine-learning algorithms. Critically evaluated multitasking led to significant changes in all stress measures, demonstrating highly robust stress reactivity. Machine learning algorithms accurately classified stressed typing for each individual based on their typing rhythms; however, no universal keystroke markers of stress were identified. Using typing rhythms. We were able to determine whether an individual was stressed or not, but the markers used for classification differed between individuals. These individual changes may provide opportunities for identifying stressful periods through keystroke monitoring, as well as the potential for early identification of disorders which may impact fine motor control. Typing rhythms could therefore be used to monitor health and well-being in individuals who use keyboards in various situations. This is the first rigorous assessment of stress and typing rhythms and has led to the development of a feasible and highly reproducible research protocol.

Better Biomarkers, Faster Drugs, Stronger Models: Progress Towards Precision Psychiatry

The 21st century has brought novel therapies and new therapeutic targets for major depressive disorder (MDD). Until recently all antidepressant medications targeted monoamines-serotonin, norepinephrine, and dopamine- and their regulatory systems. But growing evidence has suggested that individuals who fail to respond to a monoaminergic treatment are likely to fail to respond to other monoaminergic options. The emergence in recent years of treatment targets beyond the monoaminergic systems (e.g. κ-opioid antagonists, ketamine and other NMDA modulators, neurosteroids) has cultivated hopes for not only greater efficacy in treating depression, but also improved precision in targeting specific phenotypes and symptoms. Concurrently, an expanding repertoire of diagnostic and assessment tools-such as smartphone-based experience sampling and brain imaging-is moving the field toward more reliable and symptom-specific measurement with greater descriptive and prescriptive power. Taken together, these diagnostic tools and treatment options herald a new era of "precision psychiatry"-the selection and implementation of an optimal treatment for an individual patient's particular needs. Anhedonia offers an example of the new precision psychiatry. Anhedonia has moved from merely one among several criteria for depression to a transdiagnostic psychopathology which can be understood neurobiologically, assessed quantitatively, and centered as a primary target in research and development of novel pharmacotherapies. We describe functional testing of reward circuits in the development of kappa-opioid antagonists for anhedonia. This offers a lens for understanding how and under what circumstances other novel treatments, such as psychedelics, might find a place in the future landscape of precision psychiatric care.

A Review on Smartphone Keystroke Dynamics as a Digital Biomarker for Understanding Neurocognitive Functioning

Can digital technologies provide a passive unobtrusive means to observe and study cognition outside of the laboratory? Previously, cognitive assessments and monitoring were conducted in a laboratory or clinical setting, allowing for a cross-sectional glimpse of cognitive states. In the last decade, researchers have been utilizing technological advances and devices to explore ways of assessing cognition in the real world. We propose that the virtual keyboard of smartphones, an increasingly ubiquitous digital device, can provide the ideal conduit for passive data collection to study cognition. Passive data collection occurs without the active engagement of a participant and allows for near-continuous, objective data collection. Most importantly, this data collection can occur in the real world, capturing authentic datapoints. This method of data collection and its analyses provide a more comprehensive and potentially more suitable insight into cognitive states, as intra-individual cognitive fluctuations over time have shown to be an early manifestation of cognitive decline. We review different ways passive data, centered around keystroke dynamics, collected from smartphones, have been used to assess and evaluate cognition. We also discuss gaps in the literature where future directions of utilizing passive data can continue to provide inferences into cognition and elaborate on the importance of digital data privacy and consent.

The Minimum Feature Set for Designing Mobile Apps to Support Bipolar Disorder-Affected Patients: Proposal of Essential Functions and Requirements

Research conducted on mobile apps providing mental health services has concluded that patients with mental disorders tend to use such apps to maintain mental health balance technology may help manage and monitor issues like bipolar disorder (BP). This study was conducted in four steps to identify the features of designing a mobile application for BP-affected patients including (1) a literature search, (2) analyzing existing mobile apps to examine their efficiency, (3) interviewing patients affected with BP to discover their needs, and 4) exploring the points of view of experts using a dynamic narrative survey. Literature search and mobile app analysis resulted in 45 features, which were later reduced to 30 after the experts were surveyed about the project. The features included the following: mood monitoring, sleep schedule, energy level evaluation, irritability, speech level, communication, sexual activity, self-confidence level, suicidal thoughts, guilt, concentration level, aggressiveness, anxiety, appetite, smoking or drug abuse, blood pressure, the patient's weight and the side effects of medication, reminders, mood data scales, diagrams or charts of the collected data, referring the collected data to a psychologist, educational information, sending feedbacks to patients using the application, and standard tests for mood assessment. The first phase of analysis should consider an expert and patient view survey, mood and medication tracking, as well as communication with other people in the same situation are the most features to be considered. The present study has identified the necessity of apps intended to manage and monitor bipolar patients to maximize efficiency and minimize relapse and side effects.

Predictors of adherence to electronic self-monitoring in patients with bipolar disorder: a contactless study using Growth Mixture Models

BACKGROUND

Several studies have reported on the feasibility of electronic (e-)monitoring using computers or smartphones in patients with mental disorders, including bipolar disorder (BD). While studies on e-monitoring have examined the role of demographic factors, such as age, gender, or socioeconomic status and use of health apps, to our knowledge, no study has examined clinical characteristics that might impact adherence with e-monitoring in patients with BD. We analyzed adherence to e-monitoring in patients with BD who participated in an ongoing e-monitoring study and evaluated whether demographic and clinical factors would predict adherence.

METHODS

Eighty-seven participants with BD in different phases of the illness were included. Patterns of adherence for wearable use, daily and weekly self-rating scales over 15 months were analyzed to identify adherence trajectories using growth mixture models (GMM). Multinomial logistic regression models were fitted to compute the effects of predictors on GMM classes.

RESULTS

Overall adherence rates were 79.5% for the wearable; 78.5% for weekly self-ratings; and 74.6% for daily self-ratings. GMM identified three latent class subgroups: participants with (i) perfect; (ii) good; and (iii) poor adherence. On average, 34.4% of participants showed "perfect" adherence; 37.1% showed "good" adherence; and 28.2% showed poor adherence to all three measures. Women, participants with a history of suicide attempt, and those with a history of inpatient admission were more likely to belong to the group with perfect adherence.

CONCLUSIONS

Participants with higher illness burden (e.g., history of admission to hospital, history of suicide attempts) have higher adherence rates to e-monitoring. They might see e-monitoring as a tool for better documenting symptom change and better managing their illness, thus motivating their engagement.

Associations Between Smartphone Keystroke Metadata and Mental Health Symptoms in Adolescents: Findings From the Future Proofing Study

BACKGROUND

Mental disorders are prevalent during adolescence. Among the digital phenotypes currently being developed to monitor mental health symptoms, typing behavior is one promising candidate. However, few studies have directly assessed associations between typing behavior and mental health symptom severity, and whether these relationships differs between genders.

OBJECTIVE

In a cross-sectional analysis of a large cohort, we tested whether various features of typing behavior derived from keystroke metadata were associated with mental health symptoms and whether these relationships differed between genders.

METHODS

A total of 934 adolescents from the Future Proofing study undertook 2 typing tasks on their smartphones through the Future Proofing app. Common keystroke timing and frequency features were extracted across tasks. Mental health symptoms were assessed using the Patient Health Questionnaire-Adolescent version, the Children's Anxiety Scale-Short Form, the Distress Questionnaire 5, and the Insomnia Severity Index. Bivariate correlations were used to test whether keystroke features were associated with mental health symptoms. The false discovery rates of P values were adjusted to q values. Machine learning models were trained and tested using independent samples (ie, 80% train 20% test) to identify whether keystroke features could be combined to predict mental health symptoms.

RESULTS

Keystroke timing features showed a weak negative association with mental health symptoms across participants. When split by gender, females showed weak negative relationships between keystroke timing features and mental health symptoms, and weak positive relationships between keystroke frequency features and mental health symptoms. The opposite relationships were found for males (except for dwell). Machine learning models using keystroke features alone did not predict mental health symptoms.

CONCLUSIONS

Increased mental health symptoms are weakly associated with faster typing, with important gender differences. Keystroke metadata should be collected longitudinally and combined with other digital phenotypes to enhance their clinical relevance.

TRIAL REGISTRATION

Australian and New Zealand Clinical Trial Registry, ACTRN12619000855123; https://www.anzctr.org.au/Trial/Registration/TrialReview.aspx?id=377664&isReview=true.

Smartphone-derived Virtual Keyboard Dynamics Coupled with Accelerometer Data as a Window into Understanding Brain Health: Smartphone Keyboard and Accelerometer as Window into Brain Health

We examine the feasibility of using accelerometer data exclusively collected during typing on a custom smartphone keyboard to study whether typing dynamics are associated with daily variations in mood and cognition. As part of an ongoing digital mental health study involving mood disorders, we collected data from a well-characterized clinical sample (N = 85) and classified accelerometer data per typing session into orientation (upright vs. not) and motion (active vs. not). The mood disorder group showed lower cognitive performance despite mild symptoms (depression/mania). There were also diurnal pattern differences with respect to cognitive performance: individuals with higher cognitive performance typed faster and were less sensitive to time of day. They also exhibited more well-defined diurnal patterns in smartphone keyboard usage: they engaged with the keyboard more during the day and tapered their usage more at night compared to those with lower cognitive performance, suggesting a healthier usage of their phone.

The foundation and architecture of precision medicine in neurology and psychiatry

Neurological and psychiatric diseases have high degrees of genetic and pathophysiological heterogeneity, irrespective of clinical manifestations. Traditional medical paradigms have focused on late-stage syndromic aspects of these diseases, with little consideration of the underlying biology. Advances in disease modeling and methodological design have paved the way for the development of precision medicine (PM), an established concept in oncology with growing attention from other medical specialties. We propose a PM architecture for central nervous system diseases built on four converging pillars: multimodal biomarkers, systems medicine, digital health technologies, and data science. We discuss Alzheimer's disease (AD), an area of significant unmet medical need, as a case-in-point for the proposed framework. AD can be seen as one of the most advanced PM-oriented disease models and as a compelling catalyzer towards PM-oriented neuroscience drug development and advanced healthcare practice.

Association Between the Severity of Depressive Symptoms and Human-Smartphone Interactions: Longitudinal Study

BACKGROUND

Various behavioral sensing research studies have found that depressive symptoms are associated with human-smartphone interaction behaviors, including lack of diversity in unique physical locations, entropy of time spent in each location, sleep disruption, session duration, and typing speed. These behavioral measures are often tested against the total score of depressive symptoms, and the recommended practice to disaggregate within- and between-person effects in longitudinal data is often neglected.

OBJECTIVE

We aimed to understand depression as a multidimensional process and explore the association between specific dimensions and behavioral measures computed from passively sensed human-smartphone interactions. We also aimed to highlight the nonergodicity in psychological processes and the importance of disaggregating within- and between-person effects in the analysis.

METHODS

Data used in this study were collected by Mindstrong Health, a telehealth provider that focuses on individuals with serious mental illness. Depressive symptoms were measured by the Diagnostic and Statistical Manual of Mental Disorders Fifth Edition (DSM-5) Self-Rated Level 1 Cross-Cutting Symptom Measure-Adult Survey every 60 days for a year. Participants' interactions with their smartphones were passively recorded, and 5 behavioral measures were developed and were expected to be associated with depressive symptoms according to either theoretical proposition or previous empirical evidence. Multilevel modeling was used to explore the longitudinal relations between the severity of depressive symptoms and these behavioral measures. Furthermore, within- and between-person effects were disaggregated to accommodate the nonergodicity commonly found in psychological processes.

RESULTS

This study included 982 records of DSM Level 1 depressive symptom measurements and corresponding human-smartphone interaction data from 142 participants (age range 29-77 years; mean age 55.1 years, SD 10.8 years; 96 female participants). Loss of interest in pleasurable activities was associated with app count (γ₁₀=-0.14; P=.01; within-person effect). Depressed mood was associated with typing time interval (γ₀₅=0.88; P=.047; within-person effect) and session duration (γ₀₅=-0.37; P=.03; between-person effect).

CONCLUSIONS

This study contributes new evidence for associations between human-smartphone interaction behaviors and the severity of depressive symptoms from a dimensional perspective, and it highlights the importance of considering the nonergodicity of psychological processes and analyzing the within- and between-person effects separately.

A Novel Approach to Clustering Accelerometer Data for Application in Passive Predictions of Changes in Depression Severity

The treatment of mood disorders, which can become a lifelong process, varies widely in efficacy between individuals. Most options to monitor mood rely on subjective self-reports and clinical visits, which can be burdensome and may not portray an accurate representation of what the individual is experiencing. A passive method to monitor mood could be a useful tool for those with these disorders. Some previously proposed models utilized sensors from smartphones and wearables, such as the accelerometer. This study examined a novel approach of processing accelerometer data collected from smartphones only while participants of the open-science branch of the BiAffect study were typing. The data were modeled by von Mises-Fisher distributions and weighted networks to identify clusters relating to different typing positions unique for each participant. Longitudinal features were derived from the clustered data and used in machine learning models to predict clinically relevant changes in depression from clinical and typing measures. Model accuracy was approximately 95%, with 97% area under the ROC curve (AUC). The accelerometer features outperformed the vast majority of clinical and typing features, which suggested that this new approach to analyzing accelerometer data could contribute towards unobtrusive detection of changes in depression severity without the need for clinical input.

Disease severity classification using passively collected smartphone-based keystroke dynamics within multiple sclerosis

Multiple Sclerosis (MS) is a progressive demyelinating disease of the central nervous system characterised by a wide range of motor and non-motor symptoms. The level of disability of people with MS (pwMS) is based on a wide range of clinical measures, though their frequency of evaluation and inaccuracies coming from objective and self-reported evaluations limits these assessments. Alternatively, remote health monitoring through devices can offer a cost-efficient solution to gather more reliable, objective measures continuously. Measuring smartphone keyboard interactions is a promising tool since typing and, thus, keystroke dynamics are likely influenced by symptoms that pwMS can experience. Therefore, this paper aims to investigate whether keyboard interactions gathered on a person's smartphone can provide insight into the clinical status of pwMS leveraging machine learning techniques. In total, 24 Healthy Controls (HC) and 102 pwMS were followed for one year. Next to continuous data generated via smartphone interactions, clinical outcome measures were collected and used as targets to train four independent multivariate binary classification pipelines in discerning pwMS versus HC and estimating the level of disease severity, manual dexterity and cognitive capabilities. The final models yielded an AUC-ROC in the hold-out set above 0.7, with the highest performance obtained in estimating the level of fine motor skills (AUC-ROC=0.753). These findings show that keyboard interactions combined with machine learning techniques can be used as an unobtrusive monitoring tool to estimate various levels of clinical disability in pwMS from daily activities and with a high frequency of sampling without increasing patient burden.

Real-time associations among MS symptoms and cognitive dysfunction using ecological momentary assessment

Introduction

Multiple sclerosis (MS) is characterized by a wide range of disabling symptoms, including cognitive dysfunction, fatigue, depression, anxiety, pain, and sleep difficulties. The current study aimed to examine real-time associations between non-cognitive and cognitive symptoms (latter measured both objectively and subjectively in real-time) using smartphone-administered ecological momentary assessment (EMA).

Methods

Forty-five persons with MS completed EMA four times per day for 3 weeks. For each EMA, participants completed mobile versions of the Trail-Making Test part B (mTMT-B) and a finger tapping task, as well as surveys about symptom severity. Multilevel models were conducted to account for within-person and within-day clustering.

Results

A total of 3,174 EMA sessions were collected; compliance rate was 84%. There was significant intra-day variability in mTMT-B performance (p < 0.001) and levels of self-reported fatigue (p < 0.001). When participants reported depressive symptoms that were worse than their usual levels, they also performed worse on the mTMT-B (p < 0.001), independent of upper extremity motor functioning. Other self-reported non-cognitive symptoms were not associated with real-time performance on the mTMT-B [p > 0.009 (Bonferroni-corrected)]. In contrast, when self-reported fatigue (p < 0.001), depression (p < 0.001), anxiety (p < 0.001), and pain (p < 0.001) were worse than the individual's typical levels, they also reported more severe cognitive dysfunction at the same time. Further, there was a statistical trend that self-reported cognitive dysfunction (not mTMT-B performance) predicted one's self-reported sense of accomplishment in real-time.

Discussion

The current study was the first to identify divergent factors that influence subjectively and objectively measured cognitive functioning in real time among persons with MS. Notably, it is when symptom severity was worse than the individual's usual levels (and not absolute levels) that led to cognitive fluctuations, which supports the use of EMA in MS symptom monitoring.

Cross-Platform Detection of Psychiatric Hospitalization via Social Media Data: Comparison Study

BACKGROUND

Previous research has shown the feasibility of using machine learning models trained on social media data from a single platform (eg, Facebook or Twitter) to distinguish individuals either with a diagnosis of mental illness or experiencing an adverse outcome from healthy controls. However, the performance of such models on data from novel social media platforms unseen in the training data (eg, Instagram and TikTok) has not been investigated in previous literature.

OBJECTIVE

Our study examined the feasibility of building machine learning classifiers that can effectively predict an upcoming psychiatric hospitalization given social media data from platforms unseen in the classifiers' training data despite the preliminary evidence on identity fragmentation on the investigated social media platforms.

METHODS

Windowed timeline data of patients with a diagnosis of schizophrenia spectrum disorder before a known hospitalization event and healthy controls were gathered from 3 platforms: Facebook (254/268, 94.8% of participants), Twitter (51/268, 19% of participants), and Instagram (134/268, 50% of participants). We then used a 3 × 3 combinatorial binary classification design to train machine learning classifiers and evaluate their performance on testing data from all available platforms. We further compared results from models in intraplatform experiments (ie, training and testing data belonging to the same platform) to those from models in interplatform experiments (ie, training and testing data belonging to different platforms). Finally, we used Shapley Additive Explanation values to extract the top predictive features to explain and compare the underlying constructs that predict hospitalization on each platform.

RESULTS

We found that models in intraplatform experiments on average achieved an F₁-score of 0.72 (SD 0.07) in predicting a psychiatric hospitalization because of schizophrenia spectrum disorder, which is 68% higher than the average of models in interplatform experiments at an F₁-score of 0.428 (SD 0.11). When investigating the key drivers for divergence in construct validities between models, an analysis of top features for the intraplatform models showed both low predictive feature overlap between the platforms and low pairwise rank correlation (<0.1) between the platforms' top feature rankings. Furthermore, low average cosine similarity of data between platforms within participants in comparison with the same measurement on data within platforms between participants points to evidence of identity fragmentation of participants between platforms.

CONCLUSIONS

We demonstrated that models built on one platform's data to predict critical mental health treatment outcomes such as hospitalization do not generalize to another platform. In our case, this is because different social media platforms consistently reflect different segments of participants' identities. With the changing ecosystem of social media use among different demographic groups and as web-based identities continue to become fragmented across platforms, further research on holistic approaches to harnessing these diverse data sources is required.

Smartphone accelerometer data as a proxy for clinical data in modeling of bipolar disorder symptom trajectory

Being able to track and predict fluctuations in symptoms of mental health disorders such as bipolar disorder outside the clinic walls is critical for expanding access to care for the global population. To that end, we analyze a dataset of 291 individuals from a smartphone app targeted at bipolar disorder, which contains rich details about their smartphone interactions (including typing dynamics and accelerometer motion) collected everyday over several months, along with more traditional clinical features. The aim is to evaluate whether smartphone accelerometer data could serve as a proxy for traditional clinical data, either by itself or in combination with typing dynamics. Results show that accelerometer data improves the predictive performance of machine learning models by nearly 5% over those previously reported in the literature based only on clinical data and typing dynamics. This suggests it is possible to elicit essentially the same "information" about bipolar symptomology using different data sources, in a variety of settings.

Associations between smartphone keystroke dynamics and cognition in MS

Objective

Examine the associations between smartphone keystroke dynamics and cognitive functioning among persons with multiple sclerosis (MS).

Methods

Sixteen persons with MS with no self-reported upper extremity or typing difficulties and 10 healthy controls (HCs) completed six weeks of remote monitoring of their keystroke dynamics (i.e., how they typed on their smartphone keyboards). They also completed a comprehensive neuropsychological assessment and symptom ratings about fatigue, depression, and anxiety at baseline.

Results

A total of 1,335,787 keystrokes were collected, which were part of 30,968 typing sessions. The MS group typed slower (P < .001) and more variably (P = .032) than the HC group. Faster typing speed was associated with better performance on measures of processing speed (P = .016), attention (P = .022), and executive functioning (cognitive flexibility: P = .029; behavioral inhibition: P = .002; verbal fluency: P = .039), as well as less severe impact from fatigue (P < .001) and less severe anxiety symptoms (P = .007). Those with better cognitive functioning and less severe symptoms showed a stronger correlation between the use of backspace and autocorrection events (P < .001).

Conclusion

Typing speed may be sensitive to cognitive functions subserved by the frontal-subcortical brain circuits. Individuals with better cognitive functioning and less severe symptoms may be better at monitoring their typing errors. Keystroke dynamics have the potential to be used as an unobtrusive remote monitoring method for real-life cognitive functioning among persons with MS, which may improve the detection of relapses, evaluate treatment efficacy, and track disability progression.

Passive Sensing in the Prediction of Suicidal Thoughts and Behaviors: Protocol for a Systematic Review

BACKGROUND

Suicide is a severe public health problem, resulting in a high number of attempts and deaths each year. Early detection of suicidal thoughts and behaviors (STBs) is key to preventing attempts. We discuss passive sensing of digital and behavioral markers to enhance the detection and prediction of STBs.

OBJECTIVE

The paper presents the protocol for a systematic review that aims to summarize existing research on passive sensing of STBs and evaluate whether the STB prediction can be improved using passive sensing compared to prior prediction models.

METHODS

A systematic search will be conducted in the scientific databases MEDLINE, PubMed, Embase, PsycINFO, and Web of Science. Eligible studies need to investigate any passive sensor data from smartphones or wearables to predict STBs. The predictive value of passive sensing will be the primary outcome. The practical implications and feasibility of the studies will be considered as secondary outcomes. Study quality will be assessed using the Prediction Model Risk of Bias Assessment Tool (PROBAST). If studies are sufficiently homogenous, we will conduct a meta-analysis of the predictive value of passive sensing on STBs.

RESULTS

The review process started in July 2022 with data extraction in September 2022. Results are expected in December 2022.

CONCLUSIONS

Despite intensive research efforts, the ability to predict STBs is little better than chance. This systematic review will contribute to our understanding of the potential of passive sensing to improve STB prediction. Future research will be stimulated since gaps in the current literature will be identified and promising next steps toward clinical implementation will be outlined.

TRIAL REGISTRATION

OSF Registries osf-registrations-hzxua-v1; https://osf.io/hzxua.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/42146.

Modern views of machine learning for precision psychiatry

In light of the National Institute of Mental Health (NIMH)'s Research Domain Criteria (RDoC), the advent of functional neuroimaging, novel technologies and methods provide new opportunities to develop precise and personalized prognosis and diagnosis of mental disorders. Machine learning (ML) and artificial intelligence (AI) technologies are playing an increasingly critical role in the new era of precision psychiatry. Combining ML/AI with neuromodulation technologies can potentially provide explainable solutions in clinical practice and effective therapeutic treatment. Advanced wearable and mobile technologies also call for the new role of ML/AI for digital phenotyping in mobile mental health. In this review, we provide a comprehensive review of ML methodologies and applications by combining neuroimaging, neuromodulation, and advanced mobile technologies in psychiatry practice. We further review the role of ML in molecular phenotyping and cross-species biomarker identification in precision psychiatry. We also discuss explainable AI (XAI) and neuromodulation in a closed human-in-the-loop manner and highlight the ML potential in multi-media information extraction and multi-modal data fusion. Finally, we discuss conceptual and practical challenges in precision psychiatry and highlight ML opportunities in future research.

The Use of Smartphone Keystroke Dynamics to Passively Monitor Upper Limb and Cognitive Function in Multiple Sclerosis: Longitudinal Analysis

Background

Typing on smartphones, which has become a near daily activity, requires both upper limb and cognitive function. Analysis of keyboard interactions during regular typing, that is, keystroke dynamics, could therefore potentially be utilized for passive and continuous monitoring of function in patients with multiple sclerosis.

Objective

To determine whether passively acquired smartphone keystroke dynamics correspond to multiple sclerosis outcomes, we investigated the association between keystroke dynamics and clinical outcomes (upper limb and cognitive function). This association was investigated longitudinally in order to study within-patient changes independently of between-patient differences.

Methods

During a 1-year follow-up, arm function and information processing speed were assessed every 3 months in 102 patients with multiple sclerosis with the Nine-Hole Peg Test and Symbol Digit Modalities Test, respectively. Keystroke-dynamics data were continuously obtained from regular typing on the participants’ own smartphones. Press-and-release latency of the alphanumeric keys constituted the fine motor score cluster, while latency of the punctuation and backspace keys constituted the cognition score cluster. The association over time between keystroke clusters and the corresponding clinical outcomes was assessed with linear mixed models with subjects as random intercepts. By centering around the mean and calculating deviation scores within subjects, between-subject and within-subject effects were distinguished.

Results

Mean (SD) scores for the fine motor score cluster and cognition score cluster were 0.43 (0.16) and 0.94 (0.41) seconds, respectively. The fine motor score cluster was significantly associated with the Nine-Hole Peg Test: between-subject β was 15.9 (95% CI 12.2-19.6) and within-subject β was 6.9 (95% CI 2.0-11.9). The cognition score cluster was significantly associated with the Symbol Digit Modalities Test between subjects (between-subject β –11.2, 95% CI –17.3 to –5.2) but not within subjects (within-subject β –0.4, 95% CI –5.6 to 4.9).

Conclusions

Smartphone keystroke dynamics were longitudinally associated with multiple sclerosis outcomes. Worse arm function corresponded with longer latency in typing both across and within patients. Worse processing speed corresponded with higher latency in using punctuation and backspace keys across subjects. Hence, keystroke dynamics are a potential digital biomarker for remote monitoring and predicting clinical outcomes in patients with multiple sclerosis.

Trial Registration

Netherlands Trial Register NTR7268; https://trialsearch.who.int/Trial2.aspx?TrialID=NTR7268

Digitale Phänotypisierung in der Psychologie – ein Quantensprung in der psychologischen Forschung?

Zusammenfassung. Digitale Phänotypisierung stellt einen neuen, leistungsstarken Ansatz zur Realisierung psychodiagnostischer Aufgaben in vielen Bereichen der Psychologie und Medizin dar. Die Grundidee besteht aus der Nutzung digitaler Spuren aus dem Alltag, um deren Vorhersagekraft für verschiedenste Anwendungsmöglichkeiten zu überprüfen und zu nutzen. Voraussetzungen für eine erfolgreiche Umsetzung sind elaborierte Smart Sensing Ansätze sowie Big Data-basierte Extraktions- (Data Mining) und Machine Learning-basierte Analyseverfahren. Erste empirische Studien verdeutlichen das hohe Potential, aber auch die forschungsmethodischen sowie ethischen und rechtlichen Herausforderungen, um über korrelative Zufallsbefunde hinaus belastbare Befunde zu gewinnen. Hierbei müssen rechtliche und ethische Richtlinien sicherstellen, dass die Erkenntnisse in einer für Einzelne und die Gesellschaft als Ganzes wünschenswerten Weise genutzt werden. Für die Psychologie als Lehr- und Forschungsdomäne bieten sich durch Digitale Phänotypisierung vielfältige Möglichkeiten, die zum einen eine gelebte Zusammenarbeit verschiedener Fachbereiche und zum anderen auch curriculare Erweiterungen erfordern. Die vorliegende narrative Übersicht bietet eine theoretische, nicht-technische Einführung in das Forschungsfeld der Digitalen Phänotypisierung, mit ersten empirischen Befunden sowie einer Diskussion der Möglichkeiten und Grenzen sowie notwendigen Handlungsfeldern.

Capturing Cognitive Aging in Vivo: Application of a Neuropsychological Framework for Emerging Digital Tools

As the global burden of dementia continues to plague our healthcare systems, efficient, objective, and sensitive tools to detect neurodegenerative disease and capture meaningful changes in everyday cognition are increasingly needed. Emerging digital tools present a promising option to address many drawbacks of current approaches, with contexts of use that include early detection, risk stratification, prognosis, and outcome measurement. However, conceptual models to guide hypotheses and interpretation of results from digital tools are lacking and are needed to sort and organize the large amount of continuous data from a variety of sensors. In this viewpoint, we propose a neuropsychological framework for use alongside a key emerging approach—digital phenotyping. The Variability in Everyday Behavior (VIBE) model is rooted in established trends from the neuropsychology, neurology, rehabilitation psychology, cognitive neuroscience, and computer science literature and links patterns of intraindividual variability, cognitive abilities, and everyday functioning across clinical stages from healthy to dementia. Based on the VIBE model, we present testable hypotheses to guide the design and interpretation of digital phenotyping studies that capture everyday cognition in vivo. We conclude with methodological considerations and future directions regarding the application of the digital phenotyping approach to improve the efficiency, accessibility, accuracy, and ecological validity of cognitive assessment in older adults.

Digital tools for the assessment of pharmacological treatment for depressive disorder: State of the art

Depression is an invalidating disorder, marked by phenotypic heterogeneity. Clinical assessments for treatment adjustments and data-collection for pharmacological research often rely on subjective representations of functioning. Better phenotyping through digital applications may add unseen information and facilitate disentangling the clinical characteristics and impact of depression and its pharmacological treatment in everyday life. Researchers, physicians, and patients benefit from well-understood digital phenotyping approaches to assess the treatment efficacy and side-effects. This review discusses the current possibilities and pitfalls of wearables and technology for the assessment of the pharmacological treatment of depression. Their applications in the whole spectrum of treatment for depression, including diagnosis, treatment of an episode, and monitoring of relapse risk and prevention are discussed. Multiple aspects are to be considered, including concerns that come with collecting sensitive data and health recordings. Also, privacy and trust are addressed. Available applications range from questionnaire-like apps to objective assessment of behavioural patterns and promises in handling suicidality. Nonetheless, interpretation and integration of this high-resolution information with other phenotyping levels, remains challenging. This review provides a state-of-the-art description of wearables and technology in digital phenotyping for monitoring pharmacological treatment in depression, focusing on the challenges and opportunities of its application in clinical trials and research.

The Role of Remote Monitoring in Evaluating Fatigue in Multiple Sclerosis: A Review

Fatigue is one of the most common multiple sclerosis (MS) symptoms. Despite this, monitoring and measuring fatigue (subjective lack of energy)- and fatigability (objectively measurable and quantifiable performance decline)- in people with MS have remained challenging. Traditionally, administration of self-report questionnaires during in-person visits has been used to measure fatigue. However, remote measurement and monitoring of fatigue and fatigability have become feasible in the past decade. Traditional questionnaires can be administered through the web in any setting. The ubiquitous availability of smartphones allows for momentary and frequent measurement of MS fatigue in the ecological home-setting. This approach reduces the recall bias inherent in many traditional questionnaires and demonstrates the fluctuation of fatigue that cannot be captured by standard measures. Wearable devices can assess patients' fatigability and activity levels, often influenced by the severity of subjective fatigue. Remote monitoring of fatigue, fatigability, and activity in real-world situations can facilitate quantifying symptom-severity in clinical and research settings. Combining remote measures of fatigue as well as objective fatigability in a single construct, composite score, may provide a more comprehensive outcome. The more granular data obtained through remote monitoring techniques may also help with the development of interventions aimed at improving fatigue and lowering the burden of this disabling symptom.

Exploring the digital footprint of depression: a PRISMA systematic literature review of the empirical evidence

BACKGROUND

This PRISMA systematic literature review examined the use of digital data collection methods (including ecological momentary assessment [EMA], experience sampling method [ESM], digital biomarkers, passive sensing, mobile sensing, ambulatory assessment, and time-series analysis), emphasizing on digital phenotyping (DP) to study depression. DP is defined as the use of digital data to profile health information objectively.

AIMS

Four distinct yet interrelated goals underpin this study: (a) to identify empirical research examining the use of DP to study depression; (b) to describe the different methods and technology employed; (c) to integrate the evidence regarding the efficacy of digital data in the examination, diagnosis, and monitoring of depression and (d) to clarify DP definitions and digital mental health records terminology.

RESULTS

Overall, 118 studies were assessed as eligible. Considering the terms employed, "EMA", "ESM", and "DP" were the most predominant. A variety of DP data sources were reported, including voice, language, keyboard typing kinematics, mobile phone calls and texts, geocoded activity, actigraphy sensor-related recordings (i.e., steps, sleep, circadian rhythm), and self-reported apps' information. Reviewed studies employed subjectively and objectively recorded digital data in combination with interviews and psychometric scales.

CONCLUSIONS

Findings suggest links between a person's digital records and depression. Future research recommendations include (a) deriving consensus regarding the DP definition and (b) expanding the literature to consider a person's broader contextual and developmental circumstances in relation to their digital data/records.

Digital phenotyping in psychiatry: A scoping review

BACKGROUND

Digital phenotyping has been defined as the moment-by-moment assessment of an illness state through digital means, promising objective, quantifiable data on psychiatric patients' conditions, and could potentially improve diagnosis and management of mental illness. As it is a rapidly growing field, it is to be expected that new literature is being published frequently.

OBJECTIVE

We conducted this scoping review to assess the current state of literature on digital phenotyping and offer some discussion on the current trends and future direction of this area of research.

METHODS

We searched four databases, PubMed, Ovid MEDLINE, PsycINFO and Web of Science, from inception to August 25th, 2021. We included studies written in English that 1) investigated or applied their findings to diagnose psychiatric disorders and 2) utilized passive sensing for management or diagnosis. Protocols were excluded. A narrative synthesis approach was used, due to the heterogeneity and variability in outcomes and outcome types reported.

RESULTS

Of 10506 unique records identified, we included a total of 107 articles. The number of published studies has increased over tenfold from 2 in 2014 to 28 in 2020, illustrating the field's rapid growth. However, a significant proportion of these (49% of all studies and 87% of primary studies) were proof of concept, pilot or correlational studies examining digital phenotyping's potential. Most (62%) of the primary studies published evaluated individuals with depression (21%), BD (18%) and SZ (23%) (Appendix 1).

CONCLUSION

There is promise shown in certain domains of data and their clinical relevance, which have yet to be fully elucidated. A consensus has yet to be reached on the best methods of data collection and processing, and more multidisciplinary collaboration between physicians and other fields is needed to unlock the full potential of digital phenotyping and allow for statistically powerful clinical trials to prove clinical utility.

Use of Artificial Intelligence in Clinical Neurology

Artificial intelligence is already innovating in the provision of neurologic care. This review explores key artificial intelligence concepts; their application to neurologic diagnosis, prognosis, and treatment; and challenges that await their broader adoption. The development of new diagnostic biomarkers, individualization of prognostic information, and improved access to treatment are among the plethora of possibilities. These advances, however, reflect only the tip of the iceberg for the ways in which artificial intelligence may transform neurologic care in the future.

Diagnostic accuracy of keystroke dynamics as digital biomarkers for fine motor decline in neuropsychiatric disorders: a systematic review and meta-analysis

The unmet timely diagnosis requirements, that take place years after substantial neural loss and neuroperturbations in neuropsychiatric disorders, affirm the dire need for biomarkers with proven efficacy. In Parkinson's disease (PD), Mild Cognitive impairment (MCI), Alzheimers disease (AD) and psychiatric disorders, it is difficult to detect early symptoms given their mild nature. We hypothesize that employing fine motor patterns, derived from natural interactions with keyboards, also knwon as keystroke dynamics, could translate classic finger dexterity tests from clinics to populations in-the-wild for timely diagnosis, yet, further evidence is required to prove this efficiency. We have searched PubMED, Medline, IEEEXplore, EBSCO and Web of Science for eligible diagnostic accuracy studies employing keystroke dynamics as an index test for the detection of neuropsychiatric disorders as the main target condition. We evaluated the diagnostic performance of keystroke dynamics across 41 studies published between 2014 and March 2022, comprising 3791 PD patients, 254 MCI patients, and 374 psychiatric disease patients. Of these, 25 studies were included in univariate random-effect meta-analysis models for diagnostic performance assessment. Pooled sensitivity and specificity are 0.86 (95% Confidence Interval (CI) 0.82-0.90, I2 = 79.49%) and 0.83 (CI 0.79-0.87, I2 = 83.45%) for PD, 0.83 (95% CI 0.65-1.00, I2 = 79.10%) and 0.87 (95% CI 0.80-0.93, I2 = 0%) for psychomotor impairment, and 0.85 (95% CI 0.74-0.96, I2 = 50.39%) and 0.82 (95% CI 0.70-0.94, I2 = 87.73%) for MCI and early AD, respectively. Our subgroup analyses conveyed the diagnosis efficiency of keystroke dynamics for naturalistic self-reported data, and the promising performance of multimodal analysis of naturalistic behavioral data and deep learning methods in detecting disease-induced phenotypes. The meta-regression models showed the increase in diagnostic accuracy and fine motor impairment severity index with age and disease duration for PD and MCI. The risk of bias, based on the QUADAS-2 tool, is deemed low to moderate and overall, we rated the quality of evidence to be moderate. We conveyed the feasibility of keystroke dynamics as digital biomarkers for fine motor decline in naturalistic environments. Future work to evaluate their performance for longitudinal disease monitoring and therapeutic implications is yet to be performed. We eventually propose a partnership strategy based on a "co-creation" approach that stems from mechanistic explanations of patients' characteristics derived from data obtained in-clinics and under ecologically valid settings. The protocol of this systematic review and meta-analysis is registered in PROSPERO; identifier CRD42021278707. The presented work is supported by the KU-KAIST joint research center.

Predicting perinatal health outcomes using smartphone-based digital phenotyping and machine learning in a prospective Swedish cohort (Mom2B): study protocol

INTRODUCTION

Perinatal complications, such as perinatal depression and preterm birth, are major causes of morbidity and mortality for the mother and the child. Prediction of high risk can allow for early delivery of existing interventions for prevention. This ongoing study aims to use digital phenotyping data from the Mom2B smartphone application to develop models to predict women at high risk for mental and somatic complications.

METHODS AND ANALYSIS

All Swedish-speaking women over 18 years, who are either pregnant or within 3 months postpartum are eligible to participate by downloading the Mom2B smartphone app. We aim to recruit at least 5000 participants with completed outcome measures. Throughout the pregnancy and within the first year postpartum, both active and passive data are collected via the app in an effort to establish a participant's digital phenotype. Active data collection consists of surveys related to participant background information, mental and physical health, lifestyle, and social circumstances, as well as voice recordings. Participants' general smartphone activity, geographical movement patterns, social media activity and cognitive patterns can be estimated through passive data collection from smartphone sensors and activity logs. The outcomes will be measured using surveys, such as the Edinburgh Postnatal Depression Scale, and through linkage to national registers, from where information on registered clinical diagnoses and received care, including prescribed medication, can be obtained. Advanced machine learning and deep learning techniques will be applied to these multimodal data in order to develop accurate algorithms for the prediction of perinatal depression and preterm birth. In this way, earlier intervention may be possible.

ETHICS AND DISSEMINATION

Ethical approval has been obtained from the Swedish Ethical Review Authority (dnr: 2019/01170, with amendments), and the project fully fulfils the General Data Protection Regulation (GDPR) requirements. All participants provide consent to participate and can withdraw their participation at any time. Results from this project will be disseminated in international peer-reviewed journals and presented in relevant conferences.

Wearables in Schizophrenia: Update on Current and Future Clinical Applications

Schizophrenia affects 1% of the world population and is associated with a reduction in life expectancy of 20 years. The increasing prevalence of both consumer technology and clinical-grade wearable technology offers new metrics to guide clinical decision-making remotely and in real time. Herein, recent literature is reviewed to determine the potential utility of wearables in schizophrenia, including their utility in diagnosis, first-episode psychosis, and relapse prevention and their acceptability to patients. Several studies have further confirmed the validity of various devices in their ability to track sleep—an especially useful metric in schizophrenia, as sleep disturbances may be predictive of disease onset or the acute worsening of psychotic symptoms. Through machine learning, wearable-obtained heart rate and motor activity were used to differentiate between controls and patients with schizophrenia. Wearables can capture the autonomic dysregulation that has been detected when patients are actively experiencing paranoia, hallucinations, or delusions. Multiple platforms are currently being researched, such as Health Outcomes Through Positive Engagement and Self-Empowerment, Mobile Therapeutic Attention for Treatment-Resistant Schizophrenia, and Sleepsight, that may ultimately link patient data to clinicians. The future is bright for wearables in schizophrenia, as the recent literature exemplifies their potential to offer real-time insights to guide diagnosis and management.

The relationship between text message sentiment and self-reported depression

BACKGROUND

Personal sensing has shown promise for detecting behavioral correlates of depression, but there is little work examining personal sensing of cognitive and affective states. Digital language, particularly through personal text messages, is one source that can measure these markers.

METHODS

We correlated privacy-preserving sentiment analysis of text messages with self-reported depression symptom severity. We enrolled 219 U.S. adults in a 16 week longitudinal observational study. Participants installed a personal sensing app on their phones, which administered self-report PHQ-8 assessments of their depression severity, collected phone sensor data, and computed anonymized language sentiment scores from their text messages. We also trained machine learning models for predicting end-of-study self-reported depression status using on blocks of phone sensor and text features.

RESULTS

In correlation analyses, we find that degrees of depression, emotional, and personal pronoun language categories correlate most strongly with self-reported depression, validating prior literature. Our classification models which predict binary depression status achieve a leave-one-out AUC of 0.72 when only considering text features and 0.76 when combining text with other networked smartphone sensors.

LIMITATIONS

Participants were recruited from a panel that over-represented women, caucasians, and individuals with self-reported depression at baseline. As language use differs across demographic factors, generalizability beyond this population may be limited. The study period also coincided with the initial COVID-19 outbreak in the United States, which may have affected smartphone sensor data quality.

CONCLUSIONS

Effective depression prediction through text message sentiment, especially when combined with other personal sensors, could enable comprehensive mental health monitoring and intervention.

Applying machine learning to smartphone based cognitive and sleep assessments in schizophrenia

Background

Methods

Results

Discussion

Measurement practices exacerbate the generalizability crisis: Novel digital measures can help

Psychology's tendency to focus on confirmatory analyses before ensuring constructs are clearly defined and accurately measured is exacerbating the generalizability crisis. Our growing use of digital behaviors as predictors has revealed the fragility of subjective measures and the latent constructs they scaffold. However, new technologies can provide opportunities to improve conceptualizations, theories, and measurement practices.

Digital phenotype of mood disorders: A conceptual and critical review

BACKGROUND

Mood disorders are commonly diagnosed and staged using clinical features that rely merely on subjective data. The concept of digital phenotyping is based on the idea that collecting real-time markers of human behavior allows us to determine the digital signature of a pathology. This strategy assumes that behaviors are quantifiable from data extracted and analyzed through digital sensors, wearable devices, or smartphones. That concept could bring a shift in the diagnosis of mood disorders, introducing for the first time additional examinations on psychiatric routine care.

OBJECTIVE

The main objective of this review was to propose a conceptual and critical review of the literature regarding the theoretical and technical principles of the digital phenotypes applied to mood disorders.

METHODS

We conducted a review of the literature by updating a previous article and querying the PubMed database between February 2017 and November 2021 on titles with relevant keywords regarding digital phenotyping, mood disorders and artificial intelligence.

RESULTS

Out of 884 articles included for evaluation, 45 articles were taken into account and classified by data source (multimodal, actigraphy, ECG, smartphone use, voice analysis, or body temperature). For depressive episodes, the main finding is a decrease in terms of functional and biological parameters [decrease in activities and walking, decrease in the number of calls and SMS messages, decrease in temperature and heart rate variability (HRV)], while the manic phase produces the reverse phenomenon (increase in activities, number of calls and HRV).

CONCLUSION

The various studies presented support the potential interest in digital phenotyping to computerize the clinical characteristics of mood disorders.

The crossroads of digital phenotyping

The term 'Digital Phenotyping' has started to appear with increasing regularity in medical research, especially within psychiatry. This aims to bring together digital traces (e.g., from smartphones), medical data (e.g., electronic health records), and lived experiences (e.g., daily activity, location, social contact), to better monitor, intervene, and diagnose various psychiatric conditions. However, is this notion any different from digital traces or the quantified self? While digital phenotyping has the potential to transform and revolutionize medicine as we know it; there are a number of challenges that must be addressed if research is to blossom. At present, these issues include; (1) methodological issues, for example, the lack of clear theoretical links between digital markers (e.g., battery life, interactions with smartphones) and condition relapses, (2) the current tools being employed, where they typically have a number of security or privacy issues, and are invasive by nature, (3) analytical methods and approaches, where I question whether research should start in larger-scale epidemiological scale or in smaller (and potentially highly vulnerable) patient populations as is the current norm, (4) the current lack of security and privacy regulation adherence of apps used, and finally, (5) how do such technologies become integrated into various healthcare systems? This aims to provide deep insight into how the Digital Phenotyping could provide huge promise if we critically reflect now and gather clinical insights with a number of other disciplines such as epidemiology, computer- and the social sciences to move forward.

Breaking the Data Value-Privacy Paradox in Mobile Mental Health Systems Through User-Centered Privacy Protection: A Web-Based Survey Study

BACKGROUND

Mobile mental health systems (MMHS) have been increasingly developed and deployed in support of monitoring, management, and intervention with regard to patients with mental disorders. However, many of these systems rely on patient data collected by smartphones or other wearable devices to infer patients' mental status, which raises privacy concerns. Such a value-privacy paradox poses significant challenges to patients' adoption and use of MMHS; yet, there has been limited understanding of it.

OBJECTIVE

To address the significant literature gap, this research aims to investigate both the antecedents of patients' privacy concerns and the effects of privacy concerns on their continuous usage intention with regard to MMHS.

METHODS

Using a web-based survey, this research collected data from 170 participants with MMHS experience recruited from online mental health communities and a university community. The data analyses used both repeated analysis of variance and partial least squares regression.

RESULTS

The results showed that data type (P=.003), data stage (P<.001), privacy victimization experience (P=.01), and privacy awareness (P=.08) have positive effects on privacy concerns. Specifically, users report higher privacy concerns for social interaction data (P=.007) and self-reported data (P=.001) than for biometrics data; privacy concerns are higher for data transmission (P=.01) and data sharing (P<.001) than for data collection. Our results also reveal that privacy concerns have an effect on attitude toward privacy protection (P=.001), which in turn affects continuous usage intention with regard to MMHS.

CONCLUSIONS

This study contributes to the literature by deepening our understanding of the data value-privacy paradox in MMHS research. The findings offer practical guidelines for breaking the paradox through the design of user-centered and privacy-preserving MMHS.

A digital self-report survey of mood for bipolar disorder

OBJECTIVES

Bipolar disorder (BP) is commonly researched in digital settings. As a result, standardized digital tools are needed to measure mood. We sought to validate a new survey that is brief, validated in digital form, and able to separately measure manic and depressive severity.

METHODS

We introduce a 6-item digital survey, called digiBP, for measuring mood in BP. It has three depressive items (depressed mood, fidgeting, fatigue), two manic items (increased energy, rapid speech), and one mixed item (irritability); and recovers two scores (m and d) to measure manic and depressive severity. In a secondary analysis of individuals with BP who monitored their symptoms over 6 weeks (n = 43), we perform a series of analyses to validate the digiBP survey internally, externally, and as a longitudinal measure.

RESULTS

We first verify a conceptual model for the survey in which items load onto two factors ("manic" and "depressive"). We then show weekly averages of m and d scores from digiBP can explain significant variation in weekly scores from the Young Mania Rating Scale (R²  = 0.47) and SIGH-D (R²  = 0.58). Lastly, we examine the utility of the survey as a longitudinal measure by predicting an individual's future m and d scores from their past m and d scores.

CONCLUSIONS

While further validation is warranted in larger, diverse populations, these validation analyses should encourage researchers to consider digiBP for their next digital study of BP.

Apps and gaps in bipolar disorder: A systematic review on electronic monitoring for episode prediction

BACKGROUND

Long-term clinical monitoring in bipolar disorder (BD) is an important therapeutic tool. The availability of smartphones and wearables has sparked the development of automated applications to remotely monitor patients. This systematic review focus on the current state of electronic (e-) monitoring for episode prediction in BD.

METHODS

We systematically reviewed the literature on e-monitoring for episode prediction in adult BD patients. The systematic review was done according to the guidelines for reporting of systematic reviews and meta-analyses (PRISMA) and was registered in PROSPERO on April 29, 2020 (CRD42020155795). We conducted a search of Web of Science, MEDLINE, EMBASE, and PsycINFO (all 2000-2020) databases. We identified and extracted data from 17 published reports on 15 relevant studies.

RESULTS

Studies were heterogeneous and most had substantial methodological and technical limitations. Models varied widely in their performance. Published metrics were too heterogeneous to lend themselves to a meta-analysis. Four studies reported sensitivity (range: 0.21 - 0.95); and two reported specificity for prediction of mood episodes (range: 0.36 - 0.99). Two studies reported accuracy (range: 0.64 - 0.88) and four reported area under the curve (AUC; range: 0.52-0.95). Overall, models were better in predicting manic or hypomanic episodes, but their performance depended on feature type.

LIMITATIONS

Our conclusions are tempered by the lack of appropriate information impeding our ability to synthesize the available evidence.

CONCLUSIONS

Given the clinical variability in BD, predicting mood episodes remains a challenging task. Emerging e-monitoring technology for episode prediction in BD requires more development before it can be adopted clinically.

Portable technologies for digital phenotyping of bipolar disorder: A systematic review

BACKGROUND

Bias-prone psychiatric interviews remain the mainstay of bipolar disorder (BD) assessment. The development of digital phenotyping promises to improve BD management. We present a systematic review of the evidence about the use of portable digital devices for the identification of BD, BD types and BD mood states and for symptom assessment.

METHODS

We searched Web of Knowledge^SM, Scopus ®, IEEE Xplore, and ACM Digital Library databases (until 5/1/2021) for articles evaluating the use of portable/wearable digital devices, such as smartphone apps, wearable sensors, audio and/or visual recordings, and multimodal tools. The protocol is registered in PROSPERO (CRD42020200086).

RESULTS

We included 62 studies (2325 BD; 724 healthy controls, HC): 27 using smartphone apps, either for recording self-assessments (n = 10) or for passively gathering metadata (n = 7) or both (n = 10); 15 using wearable sensors for physiological parameters; 17 analysing audio and/or video recordings; 3 using multiple technologies. Two thirds of the included studies applied artificial intelligence (AI)-based approaches. They achieved fair to excellent classification performances.

LIMITATIONS

The included studies had small sample sizes and marked heterogeneity. Evidence of overfitting emerged, limiting generalizability. The absence of clear guidelines about reporting classification performances, with no shared standard metrics, makes results hardly interpretable and comparable.

CONCLUSIONS

New technologies offer a noteworthy opportunity to BD digital phenotyping with objectivity and high granularity. AI-based models could deliver important support in clinical decision-making. Further research and cooperation between different stakeholders are needed for addressing methodological, ethical and socio-economic considerations.

Reproducible Analysis Pipeline for Data Streams: Open-Source Software to Process Data Collected With Mobile Devices

Smartphone and wearable devices are widely used in behavioral and clinical research to collect longitudinal data that, along with ground truth data, are used to create models of human behavior. Mobile sensing researchers often program data processing and analysis code from scratch even though many research teams collect data from similar mobile sensors, platforms, and devices. This leads to significant inefficiency in not being able to replicate and build on others' work, inconsistency in quality of code and results, and lack of transparency when code is not shared alongside publications. We provide an overview of Reproducible Analysis Pipeline for Data Streams (RAPIDS), a reproducible pipeline to standardize the preprocessing, feature extraction, analysis, visualization, and reporting of data streams coming from mobile sensors. RAPIDS is formed by a group of R and Python scripts that are executed on top of reproducible virtual environments, orchestrated by a workflow management system, and organized following a consistent file structure for data science projects. We share open source, documented, extensible and tested code to preprocess, extract, and visualize behavioral features from data collected with any Android or iOS smartphone sensing app as well as Fitbit and Empatica wearable devices. RAPIDS allows researchers to process mobile sensor data in a rigorous and reproducible way. This saves time and effort during the data analysis phase of a project and facilitates sharing analysis workflows alongside publications.

Naturalistic smartphone keyboard typing reflects processing speed and executive function

OBJECTIVE

The increase in smartphone usage has enabled the possibility of more accessible ways to conduct neuropsychological evaluations. The objective of this study was to determine the feasibility of using smartphone typing dynamics with mood scores to supplement cognitive assessment through trail making tests.

METHODS

Using a custom-built keyboard, naturalistic keypress dynamics were unobtrusively recorded in individuals with bipolar disorder (n = 11) and nonbipolar controls (n = 8) on an Android smartphone. Keypresses were matched to digital trail making tests part B (dTMT-B) administered daily in two periods and weekly mood assessments. Following comparison of dTMT-Bs to the pencil-and-paper equivalent, longitudinal mixed-effects models were used to analyze daily dTMT-B performance as a function of typing and mood.

RESULTS

Comparison of the first dTMT-B to paper TMT-B showed adequate reliability (intraclass correlations = 0.74). In our model, we observed that participants who typed slower took longer to complete dTMT-B (b = 0.189, p < .001). This trend was also seen in individual fluctuations in typing speed and dTMT-B performance (b = 0.032, p = .004). Moreover, participants who were more depressed completed the dTMT-B slower than less depressed participants (b = 0.189, p < .001). A practice effect was observed for the dTMT-Bs.

CONCLUSION

Typing speed in combination with depression scores has the potential to infer aspects of cognition (visual attention, processing speed, and task switching) in people's natural environment to complement formal in-person neuropsychological assessments that commonly include the trail making test.

Digital Phenotyping in Child and Adolescent Psychiatry: A Perspective

Digital phenotyping (DP) provides opportunities to study child and adolescent psychiatry from a novel perspective. DP combines objective data obtained from digital sensors with participant-generated "active data," in order to understand better an individual's behavior and environmental interactions. Although this new method has led to advances in adult psychiatry, its use in child psychiatry has been more limited. This review aims to demonstrate potential benefits of DP methodology and passive data collection by reviewing studies specifically in child and adolescent psychiatry. Twenty-six studies were identified that collected passive data from four different categories: accelerometer/actigraph data, physiological data, GPS data, and step count. Study topics ranged from the associations between manic symptomology and cardiac parameters to the role of daily emotions, sleep, and social interactions in treatment for pediatric anxiety. Reviewed studies highlighted the diverse ways in which objective data can augment naturalistic self-report methods in child and adolescent psychiatry to allow for more objective, ecologically valid, and temporally resolved conclusions. Though limitations exist-including a lack of participant adherence and device failure and misuse-DP technology may represent a new and effective method for understanding pediatric cognition, behavior, disease etiology, and treatment efficacy.

Evaluating the utility of digital phenotyping to predict health outcomes in schizophrenia: protocol for the HOPE-S observational study

INTRODUCTION

The course of schizophrenia illness is characterised by recurrent relapses which are associated with adverse clinical outcomes such as treatment-resistance, functional and cognitive decline. Early identification is essential and relapse prevention remains a primary treatment goal for long-term management of schizophrenia. With the ubiquity of devices such as smartphones, objective digital biomarkers can be harnessed and may offer alternative means for symptom monitoring and relapse prediction. The acceptability of digital sensors (smartphone and wrist-wearable device) and the association between the captured digital data with clinical and health outcomes in individuals with schizophrenia will be examined.

METHODS AND ANALYSIS

In this study, we aim to recruit 100 individuals with schizophrenia spectrum disorders who are recently discharged from the Institute of Mental Health (IMH), Singapore. Participants are followed up for 6 months, where digital, clinical, cognitive and functioning data are collected while health utilisation data are obtained at the 6 month and 1 year timepoint from study enrolment. Associations between digital, clinical and health outcomes data will be examined. A data-driven machine learning approach will be used to develop prediction algorithms to detect clinically significant outcomes. Study findings will inform the design, data collection procedures and protocol of future interventional randomised controlled trial, testing the effectiveness of digital phenotyping in clinical management of individuals with schizophrenia spectrum disorders.

ETHICS AND DISSEMINATION

Ethics approval has been granted by the National Healthcare Group (NHG) Domain Specific Review Board (DSRB Reference no.: 2019/00720). The results will be published in peer-reviewed journals and presented at conferences.

TRIAL REGISTRATION NUMBER

NCT04230590.