Examining Passively Collected Smartphone-Based Data in the Days Prior to Psychiatric Hospitalization for a Suicidal Crisis: Comparative Case Analysis

BACKGROUND

Digital phenotyping has seen a broad increase in application across clinical research; however, little research has implemented passive assessment approaches for suicide risk detection. There is a significant potential for a novel form of digital phenotyping, termed screenomics, which captures smartphone activity via screenshots.

OBJECTIVE

This paper focuses on a comprehensive case review of 2 participants who reported past 1-month active suicidal ideation, detailing their passive (ie, obtained via screenomics screenshot capture) and active (ie, obtained via ecological momentary assessment [EMA]) risk profiles that culminated in suicidal crises and subsequent psychiatric hospitalizations. Through this analysis, we shed light on the timescale of risk processes as they unfold before hospitalization, as well as introduce the novel application of screenomics within the field of suicide research.

METHODS

To underscore the potential benefits of screenomics in comprehending suicide risk, the analysis concentrates on a specific type of data gleaned from screenshots-text-captured prior to hospitalization, alongside self-reported EMA responses. Following a comprehensive baseline assessment, participants completed an intensive time sampling period. During this period, screenshots were collected every 5 seconds while one's phone was in use for 35 days, and EMA data were collected 6 times a day for 28 days. In our analysis, we focus on the following: suicide-related content (obtained via screenshots and EMA), risk factors theoretically and empirically relevant to suicide risk (obtained via screenshots and EMA), and social content (obtained via screenshots).

RESULTS

Our analysis revealed several key findings. First, there was a notable decrease in EMA compliance during suicidal crises, with both participants completing fewer EMAs in the days prior to hospitalization. This contrasted with an overall increase in phone usage leading up to hospitalization, which was particularly marked by heightened social use. Screenomics also captured prominent precipitating factors in each instance of suicidal crisis that were not well detected via self-report, specifically physical pain and loneliness.

CONCLUSIONS

Our preliminary findings underscore the potential of passively collected data in understanding and predicting suicidal crises. The vast number of screenshots from each participant offers a granular look into their daily digital interactions, shedding light on novel risks not captured via self-report alone. When combined with EMA assessments, screenomics provides a more comprehensive view of an individual's psychological processes in the time leading up to a suicidal crisis.

The Idiosyncrasies of Everyday Digital Lives: Using the Human Screenome Project to Study User Behavior on Smartphones

Most methods used to make theory-relevant observations of technology use rely on self-report or application logging data where individuals' digital experiences are purposively summarized into aggregates meant to describe how the average individual engages with broadly defined segments of content. This aggregation and averaging masks heterogeneity in how and when individuals actually engage with their technology. In this study, we use screenshots (N > 6 million) collected every five seconds that were sequenced and processed using text and image extraction tools into content-, context-, and temporally-informative "screenomes" from 132 smartphone users over several weeks to examine individuals' digital experiences. Analyses of screenomes highlight extreme between-person and within-person heterogeneity in how individuals switch among and titrate their engagement with different content. Our simple quantifications of textual and graphical content and flow throughout the day illustrate the value screenomes have for the study of individuals' smartphone use and the cognitive and psychological processes that drive use. We demonstrate how temporal, textual, graphical, and topical features of people's smartphone screens can lay the foundation for expanding the Human Screenome Project with full-scale mining that will inform researchers' knowledge of digital life.

Screenomics: A New Approach for Observing and Studying Individuals' Digital Lives

AIMS

This study describes when and how adolescents engage with their fast-moving and dynamic digital environment as they go about their daily lives. We illustrate a new approach - screenomics - for capturing, visualizing, and analyzing screenomes, the record of individuals' day-to-day digital experiences.

SAMPLE

Over 500,000 smartphone screenshots provided by four Latino/Hispanic youth, age 14-15 years, from low-income, racial/ethnic minority neighborhoods.

METHOD

Screenomes collected from smartphones for one to three months, as sequences of smartphone screenshots obtained every five seconds that the device is activated, are analyzed using computational machinery for processing images and text, machine learning algorithms, human-labeling, and qualitative inquiry.

FINDINGS

Adolescents' digital lives differ substantially across persons, days, hours, and minutes. Screenomes highlight the extent of switching among multiple applications, and how each adolescent is exposed to different content at different times for different durations - with apps, food-related content, and sentiment as illustrative examples.

IMPLICATIONS

We propose that the screenome provides the fine granularity of data needed to study individuals' digital lives, for testing existing theories about media use, and for generation of new theory about the interplay between digital media and development.