Smartphone Sensor Data for Identifying and Monitoring Symptoms of Mood Disorders: A Longitudinal Observational Study

Smartphone digital phenotyping in mental health disorders: A review of raw sensors utilized, machine learning processing pipelines, and derived behavioral features

With increased access to digital technology, there has been a surge in the use of and interest in digital phenotyping as a tool to calculate various features from raw smart device data. However, the increased usage of digital phenotyping has created confusion. The vast number of sensors that can be utilized to collect passive data, and diverse methods utilized to convert that sensor data into features has introduced conflicting results and conclusions into the literature. Consequently, there is an identified need for standardizing how digital phenotyping data is measured and collected. This review evaluates the different sensors and methods utilized in digital phenotyping research across 112 papers, with the goal of finding the most common platforms, sensors, and methods for each behavioral measure. This should help guide future digital phenotyping research, and resolve some existing confusion in the field. Information on each study's data sensor variables were tracked and consolidated into a double-coded Codebook. Variables assessed included but were not limited to data sensors, features extracted from data sensors, statistical methods used, phone type, patient access to phones, and characteristics of patient population. This review found that most studies used Android devices (n = 67) or both Android and iPhone (n = 38) with an average duration of 14.3 weeks. The GPS sensor was also found to be the most frequently used sensor. This review underscores the need for standardization in methodological reporting, sensor utilization, and feature extraction across mental health studies.

Multimodal Digital Phenotyping Study in Patients With Major Depressive Episodes and Healthy Controls (Mobile Monitoring of Mood): Observational Longitudinal Study

BACKGROUND

Mood disorders are among the most common mental health conditions worldwide. Wearables and consumer-grade personal digital devices create digital traces that can be collected, processed, and analyzed, offering a unique opportunity to quantify and monitor individuals with mental disorders in their natural living environments.

OBJECTIVE

This study comprised (1) 3 subcohorts of patients with a major depressive episode, either with major depressive disorder, bipolar disorder, or concurrent borderline personality disorder, and (2) a healthy control group. We investigated whether differences in behavioral patterns could be observed at the group level, that is, patients versus healthy controls. We studied the volume and temporal patterns of smartphone screen and app use, communication, sleep, mobility, and physical activity. We investigated whether patients or controls exhibited more homogenous temporal patterns of activity when compared with other individuals in the same group. We examined which variables were associated with the severity of depression.

METHODS

In total, 188 participants were recruited to complete a 2-phase study. In the first 2 weeks, data from bed sensors, actigraphy, smartphones, and 5 sets of daily questions were collected. In the second phase, which lasted up to 1 year, only passive smartphone data and biweekly 9-item Patient Health Questionnaire data were collected. Survival analysis, statistical tests, and linear mixed models were performed.

RESULTS

Survival analysis showed no statistically significant difference in adherence. Most participants did not stay in the study for 1 year. Weekday location variance showed lower values for patients (control: mean -10.04, SD 2.73; patient: mean -11.91, SD 2.50; Mann-Whitney U [MWU] test P=.004). Normalized entropy of location was lower among patients (control: mean 2.10, SD 1.38; patient: mean 1.57, SD 1.10; MWU test P=.05). The temporal communication patterns of controls were more diverse compared to those of patients (MWU test P<.001). In contrast, patients exhibited more varied temporal patterns of smartphone use compared to the controls. We found that the duration of incoming calls (β=-0.08, 95% CI -0.12 to -0.04; P<.001) and the SD of activity magnitude (β=-2.05, 95% CI -4.18 to -0.20; P=.02) over the 14 days before the 9-item Patient Health Questionnaire records were negatively associated with depression severity. Conversely, the duration of outgoing calls showed a positive association with depression severity (β=0.05, 95% CI 0.00-0.09; P=.02).

CONCLUSIONS

Our work shows the important features for future analyses of behavioral markers of mood disorders. However, among outpatients with mild to moderate depressive disorders, the group-level differences from healthy controls in any single modality remain relatively modest. Therefore, future studies need to combine data from multiple modalities to detect more subtle differences and identify individualized signatures. The high dropout rates for longer study periods remain a challenge and limit the generalizability.

Wearables in Chronomedicine and Interpretation of Circadian Health

Wearable devices have gained increasing attention for use in multifunctional applications related to health monitoring, particularly in research of the circadian rhythms of cognitive functions and metabolic processes. In this comprehensive review, we encompass how wearables can be used to study circadian rhythms in health and disease. We highlight the importance of these rhythms as markers of health and well-being and as potential predictors for health outcomes. We focus on the use of wearable technologies in sleep research, circadian medicine, and chronomedicine beyond the circadian domain and emphasize actigraphy as a validated tool for monitoring sleep, activity, and light exposure. We discuss various mathematical methods currently used to analyze actigraphic data, such as parametric and non-parametric approaches, linear, non-linear, and neural network-based methods applied to quantify circadian and non-circadian variability. We also introduce novel actigraphy-derived markers, which can be used as personalized proxies of health status, assisting in discriminating between health and disease, offering insights into neurobehavioral and metabolic status. We discuss how lifestyle factors such as physical activity and light exposure can modulate brain functions and metabolic health. We emphasize the importance of establishing reference standards for actigraphic measures to further refine data interpretation and improve clinical and research outcomes. The review calls for further research to refine existing tools and methods, deepen our understanding of circadian health, and develop personalized healthcare strategies.

Digital biomarkers for non-motor symptoms in Parkinson's disease: the state of the art

Digital biomarkers that remotely monitor symptoms have the potential to revolutionize outcome assessments in future disease-modifying trials in Parkinson's disease (PD), by allowing objective and recurrent measurement of symptoms and signs collected in the participant's own living environment. This biomarker field is developing rapidly for assessing the motor features of PD, but the non-motor domain lags behind. Here, we systematically review and assess digital biomarkers under development for measuring non-motor symptoms of PD. We also consider relevant developments outside the PD field. We focus on technological readiness level and evaluate whether the identified digital non-motor biomarkers have potential for measuring disease progression, covering the spectrum from prodromal to advanced disease stages. Furthermore, we provide perspectives for future deployment of these biomarkers in trials. We found that various wearables show high promise for measuring autonomic function, constipation and sleep characteristics, including REM sleep behavior disorder. Biomarkers for neuropsychiatric symptoms are less well-developed, but show increasing accuracy in non-PD populations. Most biomarkers have not been validated for specific use in PD, and their sensitivity to capture disease progression remains untested for prodromal PD where the need for digital progression biomarkers is greatest. External validation in real-world environments and large longitudinal cohorts remains necessary for integrating non-motor biomarkers into research, and ultimately also into daily clinical practice.

From smartphone data to clinically relevant predictions: A systematic review of digital phenotyping methods in depression

BACKGROUND

Smartphone-based digital phenotyping enables potentially clinically relevant information to be collected as individuals go about their day. This could improve monitoring and interventions for people with Major Depressive Disorder (MDD). The aim of this systematic review was to investigate current digital phenotyping features and methods used in MDD.

METHODS

We searched PubMed, PsycINFO, Embase, Scopus and Web of Science (10/11/2023) for articles including: (1) MDD population, (2) smartphone-based features, (3) validated ratings. Risk of bias was assessed using several sources. Studies were compared within analysis goals (correlating features with depression, predicting symptom severity, diagnosis, mood state/episode, other). Twenty-four studies (9801 participants) were included.

RESULTS

Studies achieved moderate performance. Common themes included challenges from complex and missing data (leading to a risk of bias), and a lack of external validation.

DISCUSSION

Studies made progress towards relating digital phenotypes to clinical variables, often focusing on time-averaged features. Methods investigating temporal dynamics more directly may be beneficial for patient monitoring. European Research Council consolidator grant: 101001118, Prospero: CRD42022346264, Open Science Framework: https://osf.io/s7ay4.

The Development of Game-Based Digital Mental Health Interventions: Bridging the Paradigms of Health Care and Entertainment

Game elements are increasingly used to improve user engagement in digital mental health interventions, and specific game mechanics may yield therapeutic effects per se and thereby contribute to digital mental health intervention efficacy. However, only a few commercial game-based interventions are available. We suggest that the key challenge in their development reflects the tension between the 2 underlying paradigms, health care and entertainment, which have disparate goals and processes in digital development. We describe 3 approaches currently used to negotiate the 2 paradigms: the gamification of health care software, designing serious games, and purpose shifting existing entertainment games. We advanced an integrative framework to focus attention on 4 key themes in intervention development: target audience, engagement, mechanisms of action, and health-related effectiveness. On each theme, we show how the 2 paradigms contrast and can complement each other. Finally, we consider the 4 interdependent themes through the new product development phases from concept to production. Our viewpoint provides an integrative synthesis that facilitates the research, design, and development of game-based digital mental health interventions.

A Systematic Review of Location Data for Depression Prediction

Depression contributes to a wide range of maladjustment problems. With the development of technology, objective measurement for behavior and functional indicators of depression has become possible through the passive sensing technology of digital devices. Focusing on location data, we systematically reviewed the relationship between depression and location data. We searched Scopus, PubMed, and Web of Science databases by combining terms related to passive sensing and location data with depression. Thirty-one studies were included in this review. Location data demonstrated promising predictive power for depression. Studies examining the relationship between individual location data variables and depression, homestay, entropy, and the normalized entropy variable of entropy dimension showed the most consistent and significant correlations. Furthermore, variables of distance, irregularity, and location showed significant associations in some studies. However, semantic location showed inconsistent results. This suggests that the process of geographical movement is more related to mood changes than to semantic location. Future research must converge across studies on location-data measurement methods.

Associations Between Smartphone Keystroke Metadata and Mental Health Symptoms in Adolescents: Findings From the Future Proofing Study

BACKGROUND

Mental disorders are prevalent during adolescence. Among the digital phenotypes currently being developed to monitor mental health symptoms, typing behavior is one promising candidate. However, few studies have directly assessed associations between typing behavior and mental health symptom severity, and whether these relationships differs between genders.

OBJECTIVE

In a cross-sectional analysis of a large cohort, we tested whether various features of typing behavior derived from keystroke metadata were associated with mental health symptoms and whether these relationships differed between genders.

METHODS

A total of 934 adolescents from the Future Proofing study undertook 2 typing tasks on their smartphones through the Future Proofing app. Common keystroke timing and frequency features were extracted across tasks. Mental health symptoms were assessed using the Patient Health Questionnaire-Adolescent version, the Children's Anxiety Scale-Short Form, the Distress Questionnaire 5, and the Insomnia Severity Index. Bivariate correlations were used to test whether keystroke features were associated with mental health symptoms. The false discovery rates of P values were adjusted to q values. Machine learning models were trained and tested using independent samples (ie, 80% train 20% test) to identify whether keystroke features could be combined to predict mental health symptoms.

RESULTS

Keystroke timing features showed a weak negative association with mental health symptoms across participants. When split by gender, females showed weak negative relationships between keystroke timing features and mental health symptoms, and weak positive relationships between keystroke frequency features and mental health symptoms. The opposite relationships were found for males (except for dwell). Machine learning models using keystroke features alone did not predict mental health symptoms.

CONCLUSIONS

Increased mental health symptoms are weakly associated with faster typing, with important gender differences. Keystroke metadata should be collected longitudinally and combined with other digital phenotypes to enhance their clinical relevance.

TRIAL REGISTRATION

Australian and New Zealand Clinical Trial Registry, ACTRN12619000855123; https://www.anzctr.org.au/Trial/Registration/TrialReview.aspx?id=377664&isReview=true.

Developing "MinDag" - an app to capture symptom variation and illness mechanisms in bipolar disorder

Introduction

The illness course of bipolar disorder (BD) is highly heterogeneous with substantial variation between individuals with the same BD subtype and within individuals over time. This heterogeneity is not well-delineated and hampers the development of more targeted treatment. Furthermore, although lifestyle-related behaviors are believed to play a role in the illness course, such mechanisms are poorly understood. To address some of these knowledge gaps, we aimed to develop an app for collection of multi-dimensional longitudinal data on BD-relevant symptoms and lifestyle-related behaviors.

Methods

An app named MinDag was developed at the Norwegian Center for Mental Disorders Research in Oslo, Norway. The app was designed to tap into selected areas: mood, sleep, functioning/activities (social, occupational, physical exercise, leisure), substance use, emotional reactivity, and psychotic experiences. Ethical, security and usability issues were highly prioritized throughout the development and for the final app solution. We conducted beta- and pilot testing to eliminate technical problems and enhance usability and acceptability.

Results

The final version of MinDag comprises six modules; three which are presented for the user once daily (the Sleep module in the morning and the Mood and Functoning/Activities modules in the evening) and three which are presented once weekly (Substance Use, Emotional Reactivity, and Psychotic Experiences modules). In general, MinDag was well received in both in the beta-testing and the pilot study, and the participants provided valuable feedback that was taken into account in the final development. MinDag is now in use as part of the research protocol at the NORMENT center and in a specialized treatment unit for BD at Oslo University Hospital in Norway.

Discussion

We believe that MinDag will generate unique longitudinal data well suited for capturing the heterogeneity of BD and clarifying important unresolved issues such as how life-style related behavior may influence BD symptoms. Also, the experiences and knowledge derived from the development of MinDag may contribute to improving the security, acceptability, and benefit of digital tools in mental health.