Utilizing daily mood diaries and wearable sensor data to predict depression and suicidal ideation among medical interns

Smartphone-based text obtained via passive sensing as it relates to direct suicide risk assessment

BACKGROUND

Recent research highlights the dynamics of suicide risk, resulting in a shift toward real-time methodologies, such as ecological momentary assessment (EMA), to improve suicide risk identification. However, EMA's reliance on active self-reporting introduces challenges, including participant burden and reduced response rates during crises. This study explores the potential of Screenomics-a passive digital phenotyping method that captures intensive, real-time smartphone screenshots-to detect suicide risk through text-based analysis.

METHOD

Seventy-nine participants with past-month suicidal ideation or behavior completed daily EMA prompts and provided smartphone data over 28 days, resulting in approximately 7.5 million screenshots. Text from screenshots was analyzed using a validated dictionary encompassing suicide-related and general risk language.

RESULTS

Results indicated significant associations between passive and active suicidal ideation and suicide planning with specific language patterns. Detection of words related to suicidal thoughts and general risk-related words strongly correlated with self-reported suicide risk, with distinct between- and within-person effects highlighting the dynamic nature of suicide risk factors.

CONCLUSIONS

This study demonstrates the feasibility of leveraging smartphone text data for real-time suicide risk detection, offering a scalable, low-burden alternative to traditional methods. Findings suggest that dynamic, individualized monitoring via passive data collection could enhance suicide prevention efforts by enabling timely, tailored interventions. Future research should refine language models and explore diverse populations to extend the generalizability of this innovative approach.

Machine Learning Models to Identify Individuals With Imminent Suicide Risk Using a Wearable Device: A Pilot Study

OBJECTIVE

We aimed to determine whether individuals at immediate risk of suicide could be identified using data from a commercially available wearable device.

METHODS

Thirty-nine participants experiencing acute depressive episodes and 20 age- and sex-matched healthy controls wore a commercially available wearable device (Galaxy Watch Active2) for two months. We collected data on activities, sleep, and physiological metrics like heart rate and heart rate variability using the wearable device. Participants rated their mood spontaneously twice daily on a Likert scale displayed on the device. Mood ratings by clinicians were performed at weeks 0, 2, 4, and 8. The suicide risk was assessed using the Hamilton Depression Rating Scale's suicide item score (HAMD-3). We developed two predictive models using machine learning: a single-level model that processed all data simultaneously to identify those at immediate suicide risk (HAMD-3 scores ≥1) and a multilevel model. We compared the predictions of imminent suicide risk from both models.

RESULTS

Both the single-step and multi-step models effectively predicted imminent suicide risk. The multi-step model outperformed the single-step model in predicting imminent suicide risk with area under the curve scores of 0.89 compared to 0.88. In the multi-step model, the HAMD total score and heart rate variability were most significant, whereas in the single-step model, the HAMD total score and diagnosis were key predictors.

CONCLUSION

Wearable devices are a promising tool for identifying individuals at immediate risk of suicide. Future research with more refined temporal resolution is recommended.

Attitudes and barriers to mobile mental health interventions among first-year college students: a mixed-methods study

OBJECTIVE

This mixed-methods study examined attitudes, barriers, and preferences for mobile mental health interventions among first-year college students.

PARTICIPANTS

351 students (64% women; 51% non-Hispanic White; 66% Heterosexual) from two campuses completed self-report assessments and 10 completed individual semi-structured interviews.

METHODS

Paired t-tests compared attitudes for various mHealth applications and logistic regressions examined sociodemographic and clinical characteristics of mental health app users. Themes, topics, and quotes from interviews were derived through rapid qualitative analysis.

RESULTS

Mental health applications were less used and perceived to be less helpful than other mHealth applications. Past mental health app use was best predicted by past use of formal mental health care.

CONCLUSIONS

Mobile health interventions have significant potential to diversify mental health services for students. Despite limited engagement with these resources, openness to digital interventions among students is quite high. Improving intervention features and increasing problem-recognition to facilitate help-seeking may result in greater uptake.

Suicide risk among residents and PhD students: A systematic review of the literature

Residents and PhD students (any discipline) are susceptible to various mental health issues, including suicidal thoughts and behaviors. This systematic review aimed to (1) estimate the prevalence of suicide-related outcomes among residents/PhD students and (2) assess the associated variables. PubMed, PsycINFO, and Scopus databases were searched for articles documenting quantitative information about suicide-related outcomes among residents and PhD students from inception until April 30, 2023. Sixty studies were included. Estimates of the current prevalence of the following suicide-related outcomes were: death wishes (DW), 9.1%; suicidal ideation (SI), 8.6%; suicidal planning (SP), 3.2%; non-suicidal self-injury (NSSI), 1.9%; suicide attempt(s) (SA), .8%. Additionally, estimates of the lifetime prevalence were: lifetime SI (L-SI), 25.9%; lifetime SP (L-SP), 10.0%; lifetime SA (L-SA), 3.1%. Depression, burnout, hopelessness, loneliness, low quality of the relationship with the supervisor and experiencing workplace mistreatment frequently co-occurred with the assessed outcomes. Many outcomes (DW, SI, SP, L-SI, L-SP, L-SA) had a higher prevalence compared to the general population, while some (SI, NSSI, SA) were lower compared to undergraduates. Interventions for individuals at risk in this population are vital together with the modification of the work environment and the promotion of a supportive academic and professional culture to reduce suicide risk.

A systematic review on passive sensing for the prediction of suicidal thoughts and behaviors

Passive sensing data from smartphones and wearables may help improve the prediction of suicidal thoughts and behaviors (STB). In this systematic review, we explored the feasibility and predictive validity of passive sensing for STB. On June 24, 2024, we systematically searched Medline, Embase, Web of Science, PubMed, and PsycINFO. Studies were eligible if they investigated the association between STB and passive sensing, or the feasibility of passive sensing in this context. From 2107 unique records, we identified eleven prediction studies, ten feasibility studies, and seven protocols. Studies indicated generally lower model performance for passive compared to active data, with three out of four studies finding no incremental value. PROBAST ratings revealed major shortcomings in methodology and reporting. Studies suggested that passive sensing is feasible in high-risk populations. In conclusion, there is limited evidence on the predictive value of passive sensing for STB. We highlight important quality characteristics for future research.

Using Large Language Models to Detect Depression From User-Generated Diary Text Data as a Novel Approach in Digital Mental Health Screening: Instrument Validation Study

BACKGROUND

Depressive disorders have substantial global implications, leading to various social consequences, including decreased occupational productivity and a high disability burden. Early detection and intervention for clinically significant depression have gained attention; however, the existing depression screening tools, such as the Center for Epidemiologic Studies Depression Scale, have limitations in objectivity and accuracy. Therefore, researchers are identifying objective indicators of depression, including image analysis, blood biomarkers, and ecological momentary assessments (EMAs). Among EMAs, user-generated text data, particularly from diary writing, have emerged as a clinically significant and analyzable source for detecting or diagnosing depression, leveraging advancements in large language models such as ChatGPT.

OBJECTIVE

We aimed to detect depression based on user-generated diary text through an emotional diary writing app using a large language model (LLM). We aimed to validate the value of the semistructured diary text data as an EMA data source.

METHODS

Participants were assessed for depression using the Patient Health Questionnaire and suicide risk was evaluated using the Beck Scale for Suicide Ideation before starting and after completing the 2-week diary writing period. The text data from the daily diaries were also used in the analysis. The performance of leading LLMs, such as ChatGPT with GPT-3.5 and GPT-4, was assessed with and without GPT-3.5 fine-tuning on the training data set. The model performance comparison involved the use of chain-of-thought and zero-shot prompting to analyze the text structure and content.

RESULTS

We used 428 diaries from 91 participants; GPT-3.5 fine-tuning demonstrated superior performance in depression detection, achieving an accuracy of 0.902 and a specificity of 0.955. However, the balanced accuracy was the highest (0.844) for GPT-3.5 without fine-tuning and prompt techniques; it displayed a recall of 0.929.

CONCLUSIONS

Both GPT-3.5 and GPT-4.0 demonstrated relatively reasonable performance in recognizing the risk of depression based on diaries. Our findings highlight the potential clinical usefulness of user-generated text data for detecting depression. In addition to measurable indicators, such as step count and physical activity, future research should increasingly emphasize qualitative digital expression.

Sleep, mental health and physical health in new shift workers transitioning to shift work: Systematic review and meta-analysis

This systematic review and meta-analysis (PROSPERO registration CRD42022309827) aimed to describe how shift work impacts new workers' sleep, mental health, and physical health during the transition to shift work and to consolidate information regarding predictors of shift work tolerance (SWT) during this transition period. Inclusion criteria included: new shift workers; sleep, mental health, or physical health outcomes; prospective study design with the first timepoint assessing workers within three months of starting shift work; and written in English. Searches from six databases returned 12,172 articles as of August 2023. The final sample included 48 papers. Publication quality and risk of bias was assessed using the critical appraisal skills program. Forty-five studies investigated longitudinal changes in sleep, mental health, or physical health outcomes and 29 studies investigated predictors of SWT (i.e., better sleep, mental and physical health). Sleep and mental health outcomes worsened following the onset of shift work, while physical health did not significantly change. Pre-shift work mental health, sleep, and work characteristics predicted SWT later in workers' careers. Shift work adversely impacts new workers' sleep and mental health early in their career, and interventions before beginning shift work are needed to promote better SWT.

Effects of frequent assessments on the severity of suicidal thoughts: an ecological momentary assessment study

Objective

In recent years, there has been a significant increase in research using ecological momentary assessment (EMA) to explore suicidal thoughts and behaviors (STBs). Meanwhile, concerns have been raised regarding the potential impacts of frequent and intense STBs assessments on the study participants.

Methods

From November 2021 to June 2023, a total of 83 adolescent and young adult outpatients (Mage = 21.0, SDage = 6.3, 71.1% female), who were diagnosed with mood disorders, were recruited from three psychiatric clinics in China. Smartphone-based EMA was used to measure suicidal thoughts three times per day at randomly selected times. We examined the change of suicidal thoughts in each measurement and within 1 day to evaluate potential adverse effects using Bayesian multilevel models.

Results

The 3,105 effective surveys were nested in 83 participants (median follow-up days: 14 days). The results of two-level models indicated that suicidal thoughts decreased during the monitoring period. However, this effect varied among different individuals in the two-level model.

Conclusion

Our findings did not support the notion that repeated assessment of suicidal thoughts is iatrogenic, but future research should continue to investigate the impact of frequent assessment on suicidal thoughts, taking into account individual differences and utilizing larger sample sizes.

Smartphone keyboard dynamics predict affect in suicidal ideation

While digital phenotyping provides opportunities for unobtrusive, real-time mental health assessments, the integration of its modalities is not trivial due to high dimensionalities and discrepancies in sampling frequencies. We provide an integrated pipeline that solves these issues by transforming all modalities to the same time unit, applying temporal independent component analysis (ICA) to high-dimensional modalities, and fusing the modalities with linear mixed-effects models. We applied our approach to integrate high-quality, daily self-report data with BiAffect keyboard dynamics derived from a clinical suicidality sample of mental health outpatients. Applying the ICA to the self-report data (104 participants, 5712 days of data) revealed components related to well-being, anhedonia, and irritability and social dysfunction. Mixed-effects models (55 participants, 1794 days) showed that less phone movement while typing was associated with more anhedonia (β = -0.12, p = 0.00030). We consider this method to be widely applicable to dense, longitudinal digital phenotyping data.

The effect of cognitive behavioral therapy text messages on mood: A micro-randomized trial

The StayWell at Home intervention, a 60-day text-messaging program based on Cognitive Behavioral Therapy (CBT) principles, was developed to help adults cope with the adverse effects of the global pandemic. Participants in StayWell at Home were found to show reduced depressive and anxiety symptoms after participation. However, it remains unclear whether the intervention improved mood and which intervention components were most effective at improving user mood during the pandemic. Thus, utilizing a micro-randomized trial (MRT) design, we examined two intervention components to inform the mechanisms of action that improve mood: 1) text messages delivering CBT-informed coping strategies (i.e., behavioral activation, other coping skills, or social support); 2) time at which messages were sent. Data from two independent trials of StayWell are included in this paper. The first trial included 303 adults aged 18 or older, and the second included 266 adults aged 18 or older. Participants were recruited via online platforms (e.g., Facebook ads) and partnerships with community-based agencies aiming to reach diverse populations, including low-income individuals and people of color. The results of this paper indicate that participating in the program improved and sustained self-reported mood ratings among participants. We did not find significant differences between the type of message delivered and mood ratings. On the other hand, the results from Phase 1 indicated that delivering any type of message in the 3 pm-6 pm time window improved mood significantly over sending a message in the 9 am-12 pm time window. The StayWell at Home program increases in mood ratings appeared more pronounced during the first two to three weeks of the intervention and were maintained for the remainder of the study period. The current paper provides evidence that low-burden text-message interventions may effectively address behavioral health concerns among diverse communities.

Machine Learning for Multimodal Mental Health Detection: A Systematic Review of Passive Sensing Approaches

As mental health (MH) disorders become increasingly prevalent, their multifaceted symptoms and comorbidities with other conditions introduce complexity to diagnosis, posing a risk of underdiagnosis. While machine learning (ML) has been explored to mitigate these challenges, we hypothesized that multiple data modalities support more comprehensive detection and that non-intrusive collection approaches better capture natural behaviors. To understand the current trends, we systematically reviewed 184 studies to assess feature extraction, feature fusion, and ML methodologies applied to detect MH disorders from passively sensed multimodal data, including audio and video recordings, social media, smartphones, and wearable devices. Our findings revealed varying correlations of modality-specific features in individualized contexts, potentially influenced by demographics and personalities. We also observed the growing adoption of neural network architectures for model-level fusion and as ML algorithms, which have demonstrated promising efficacy in handling high-dimensional features while modeling within and cross-modality relationships. This work provides future researchers with a clear taxonomy of methodological approaches to multimodal detection of MH disorders to inspire future methodological advancements. The comprehensive analysis also guides and supports future researchers in making informed decisions to select an optimal data source that aligns with specific use cases based on the MH disorder of interest.

Ecological Momentary Assessment and Machine Learning for Predicting Suicidal Ideation Among Sexual and Gender Minority Individuals

Importance

Suicidality poses a serious global health concern, particularly in the sexual and gender minority population. While various studies have focused on investigating chronic stressors, the precise prediction effect of daily experiences on suicide ideation remains uncertain.

Objective

To test the extent to which mood fluctuations and contextual stressful events experienced by sexual and gender minority individuals may predict later short- and long-term suicide ideation.

Design, Setting, and Participants

This diagnostic study collected twice-daily data on mood states and stressful events from sexual and gender minority individuals over 25 days throughout 3 waves of the Chinese Lunar New Year (before, during, and after), and follow-up surveys assessing suicidal ideation were conducted 1, 3, and 8 months later. Online recruitment advertisements were used to recruit young adults throughout China. Eligible participants were self-identified as sexual and gender minority individuals aged 18 to 29 years. Those who were diagnosed with psychotic disorders (eg, schizophrenia spectrum or schizotypal disorder) or prevented from objective factors (ie, not having a phone or having an irregular sleep rhythm) were excluded. Data were collected from January to October 2022.

Main Outcomes and Measures

To predict short-term (1 month) and longer-term (3 and 8 months) suicidal ideation, the study tested several approaches by using machine learning including chronic stress baseline data (baseline approach), dynamic patterns of mood states and stressful events (ecological momentary assessment [EMA] approach), and a combination of baseline data and dynamic patterns (EMA plus baseline approach).

Results

A total of 103 sexual and gender minority individuals participated in the study (mean [SD] age, 24.2 [2.5] years; 72 [70%] female). Of these, 19 (18.4%; 95% CI, 10.9%-25.9%), 25 (24.8%; 95% CI, 16.4%-33.2%), 30 (29.4%; 95% CI, 20.6%-38.2%), and 32 (31.1%; 95% CI, 22.2%-40.0%) reported suicidal ideation at baseline, 1, 3, and 8 months follow-up, respectively. The EMA approach showed better performance than the baseline and baseline plus EMA approaches at 1-month follow-up (area under the receiver operating characteristic curve [AUC], 0.80; 95% CI, 0.78-0.81) and slightly better performance on the prediction of suicidal ideation at 3 and 8 months' follow-up. In addition, the best approach predicting suicidal ideation was obtained during Lunar New Year period at 1-month follow-up, which had a mean AUC of 0.77 (95% CI, 0.74-0.79) and better performance at 3 and 8 months' follow-up (AUC, 0.74; 95% CI, 0.72-0.76 and AUC, 0.72; 95% CI, 0.69-0.74, respectively).

Conclusions and Relevance

The findings in this study emphasize the importance of contextual risk factors experienced by sexual and gender minority individuals at different stages. The use of machine learning may facilitate the identification of individuals who are at risk and aid in the development of personalized process-based early prevention programs to mitigate future suicide risk.

Using machine learning with intensive longitudinal data to predict depression and suicidal ideation among medical interns over time

BACKGROUND

Use of intensive longitudinal methods (e.g. ecological momentary assessment, passive sensing) and machine learning (ML) models to predict risk for depression and suicide has increased in recent years. However, these studies often vary considerably in length, ML methods used, and sources of data. The present study examined predictive accuracy for depression and suicidal ideation (SI) as a function of time, comparing different combinations of ML methods and data sources.

METHODS

Participants were 2459 first-year training physicians (55.1% female; 52.5% White) who were provided with Fitbit wearable devices and assessed daily for mood. Linear [elastic net regression (ENR)] and non-linear (random forest) ML algorithms were used to predict depression and SI at the first-quarter follow-up assessment, using two sets of variables (daily mood features only, daily mood features + passive-sensing features). To assess accuracy over time, models were estimated iteratively for each of the first 92 days of internship, using data available up to that point in time.

RESULTS

ENRs using only the daily mood features generally had the best accuracy for predicting mental health outcomes, and predictive accuracy within 1 standard error of the full 92 day models was attained by weeks 7-8. Depression at 92 days could be predicted accurately (area under the curve >0.70) after only 14 days of data collection.

CONCLUSIONS

Simpler ML methods may outperform more complex methods until passive-sensing features become better specified. For intensive longitudinal studies, there may be limited predictive value in collecting data for more than 2 months.

Ecological Momentary Assessments and Passive Sensing in the Prediction of Short-Term Suicidal Ideation in Young Adults

Importance

Advancements in technology, including mobile-based ecological momentary assessments (EMAs) and passive sensing, have immense potential to identify short-term suicide risk. However, the extent to which EMA and passive data, particularly in combination, have utility in detecting short-term risk in everyday life remains poorly understood.

Objective

To examine whether and what combinations of self-reported EMA and sensor-based assessments identify next-day suicidal ideation.

Design, Setting, and Participants

In this intensive longitudinal prognostic study, participants completed EMAs 4 times daily and wore a sensor wristband (Fitbit Charge 3) for 8 weeks. Multilevel machine learning methods, including penalized generalized estimating equations and classification and regression trees (CARTs) with repeated 5-fold cross-validation, were used to optimize prediction of next-day suicidal ideation based on time-varying features from EMAs (affective, cognitive, behavioral risk factors) and sensor data (sleep, activity, heart rate). Young adult patients who visited an emergency department with recent suicidal ideation and/or suicide attempt were recruited. Identified via electronic health record screening, eligible individuals were contacted remotely to complete enrollment procedures. Participants (aged 18 to 25 years) completed 14 708 EMA observations (64.4% adherence) and wore a sensor wristband approximately half the time (55.6% adherence). Data were collected between June 2020 and July 2021. Statistical analysis was performed from January to March 2023.

Main Outcomes and Measures

The outcome was presence of next-day suicidal ideation.

Results

Among 102 enrolled participants, 83 (81.4%) were female; 6 (5.9%) were Asian, 5 (4.9%) were Black or African American, 9 (8.8%) were more than 1 race, and 76 (74.5%) were White; mean (SD) age was 20.9 (2.1) years. The best-performing model incorporated features from EMAs and showed good predictive accuracy (mean [SE] cross-validated area under the receiver operating characteristic curve [AUC], 0.84 [0.02]), whereas the model that incorporated features from sensor data alone showed poor prediction (mean [SE] cross-validated AUC, 0.56 [0.02]). Sensor-based features did not improve prediction when combined with EMAs. Suicidal ideation-related features were the strongest predictors of next-day ideation. When suicidal ideation features were excluded, an alternative EMA model had acceptable predictive accuracy (mean [SE] cross-validated AUC, 0.76 [0.02]). Both EMA models included features at different timescales reflecting within-day, end-of-day, and time-varying cumulative effects.

Conclusions and Relevance

In this prognostic study, self-reported risk factors showed utility in identifying near-term suicidal thoughts. Best-performing models required self-reported information, derived from EMAs, whereas sensor-based data had negligible predictive accuracy. These results may have implications for developing decision algorithms identifying near-term suicidal thoughts to guide risk monitoring and intervention delivery in everyday life.