Improving prediction of real-time loneliness and companionship type using geosocial features of personal smartphone data

Investigation of daily patterns for smartphone keystroke dynamics based on loneliness and social isolation

This study examined the relationship between loneliness levels and daily patterns of mobile keystroke dynamics in healthy individuals. Sixty-six young healthy Koreans participated in the experiment. Over five weeks, the participants used a custom Android keyboard. We divided the participants into four groups based on their level of loneliness (no loneliness, moderate loneliness, severe loneliness, and very severe loneliness). The very severe loneliness group demonstrated significantly higher typing counts during sleep time than the other three groups (one-way ANOVA, F = 3.75, p < 0.05). In addition, the average cosine similarity value of weekday and weekend typing patterns in the very severe loneliness group was higher than that in the no loneliness group (Welch's t-test, t = 2.27, p < 0.05). This meant that the no loneliness group's weekday and weekend typing patterns varied, whereas the very severe loneliness group's weekday and weekend typing patterns did not. Our results indicated that individuals with very high levels of loneliness tended to use mobile keyboards during late-night hours and did not significantly change their smartphone usage behavior between weekdays and weekends. These findings suggest that mobile keystroke dynamics have the potential to be used for the early detection of loneliness and the development of targeted interventions.

Personal Location as Health-Related Data: Public Knowledge, Public Concern, and Personal Action

OBJECTIVES

Personal health information (PHI), including health status and behaviors, are often associated with personal locations. Smart devices and other technologies routinely collect personal location. Therefore, technologies collecting personal location do not just create generic questions of privacy, but specific concerns related to PHI.

METHODS

To assess public opinion on the relationship between health, personal location, and privacy, a national survey of US residents was administered online in March 2020. Respondents answered questions about their use of smart devices and knowledge of location tracking. They also identified which of the locations they could visit were most private and how to balance possibilities that locations may be private but can also be useful to share.

RESULTS

Of respondents that used smart devices (n = 688), a majority (71.1%) indicated they knew they had applications tracking their location, with respondents who were younger (P < .001) and male (P = .002) and with more education (P = .045) more likely to indicate "yes." When all respondents (N = 828) identified the locations on a hypothetical map they felt were most private, health-related locations (substance use treatment center, hospital, urgent care) were the most selected.

CONCLUSIONS

The historical notion of PHI is no longer adequate and the public need greater education on how data from smart devices may be used to predict health status and behaviors. The COVID-19 pandemic brought increased attention to personal location as a tool for public health. Given healthcare's dependence upon trust, the field needs to lead the conversation and be viewed as protecting privacy while usefully leveraging location data.

Maternal Social Loneliness Detection Using Passive Sensing Through Continuous Monitoring in Everyday Settings: Longitudinal Study

BACKGROUND

Maternal loneliness is associated with adverse physical and mental health outcomes for both the mother and her child. Detecting maternal loneliness noninvasively through wearable devices and passive sensing provides opportunities to prevent or reduce the impact of loneliness on the health and well-being of the mother and her child.

OBJECTIVE

The aim of this study is to use objective health data collected passively by a wearable device to predict maternal (social) loneliness during pregnancy and the postpartum period and identify the important objective physiological parameters in loneliness detection.

METHODS

We conducted a longitudinal study using smartwatches to continuously collect physiological data from 31 women during pregnancy and the postpartum period. The participants completed the University of California, Los Angeles (UCLA) loneliness questionnaire in gestational week 36 and again at 12 weeks post partum. Responses to this questionnaire and background information of the participants were collected through our customized cross-platform mobile app. We leveraged participants' smartwatch data from the 7 days before and the day of their completion of the UCLA questionnaire for loneliness prediction. We categorized the loneliness scores from the UCLA questionnaire as loneliness (scores≥12) and nonloneliness (scores<12). We developed decision tree and gradient-boosting models to predict loneliness. We evaluated the models by using leave-one-participant-out cross-validation. Moreover, we discussed the importance of extracted health parameters in our models for loneliness prediction.

RESULTS

The gradient boosting and decision tree models predicted maternal social loneliness with weighted F1-scores of 0.897 and 0.872, respectively. Our results also show that loneliness is highly associated with activity intensity and activity distribution during the day. In addition, resting heart rate (HR) and resting HR variability (HRV) were correlated with loneliness.

CONCLUSIONS

Our results show the potential benefit and feasibility of using passive sensing with a smartwatch to predict maternal loneliness. Our developed machine learning models achieved a high F1-score for loneliness prediction. We also show that intensity of activity, activity pattern, and resting HR and HRV are good predictors of loneliness. These results indicate the intervention opportunities made available by wearable devices and predictive models to improve maternal well-being through early detection of loneliness.

NAPS Fusion: A framework to overcome experimental data limitations to predict human performance and cognitive task outcomes

In the area of human performance and cognitive research, machine learning (ML) problems become increasingly complex due to limitations in the experimental design, resulting in the development of poor predictive models. More specifically, experimental study designs produce very few data instances, have large class imbalances and conflicting ground truth labels, and generate wide data sets due to the diverse amount of sensors. From an ML perspective these problems are further exacerbated in anomaly detection cases where class imbalances occur and there are almost always more features than samples. Typically, dimensionality reduction methods (e.g., PCA, autoencoders) are utilized to handle these issues from wide data sets. However, these dimensionality reduction methods do not always map to a lower dimensional space appropriately, and they capture noise or irrelevant information. In addition, when new sensor modalities are incorporated, the entire ML paradigm has to be remodeled because of new dependencies introduced by the new information. Remodeling these ML paradigms is time-consuming and costly due to lack of modularity in the paradigm design, which is not ideal. Furthermore, human performance research experiments, at times, creates ambiguous class labels because the ground truth data cannot be agreed upon by subject-matter experts annotations, making ML paradigm nearly impossible to model. This work pulls insights from Dempster-Shafer theory (DST), stacking of ML models, and bagging to address uncertainty and ignorance for multi-classification ML problems caused by ambiguous ground truth, low samples, subject-to-subject variability, class imbalances, and wide data sets. Based on these insights, we propose a probabilistic model fusion approach, Naive Adaptive Probabilistic Sensor (NAPS), which combines ML paradigms built around bagging algorithms to overcome these experimental data concerns while maintaining a modular design for future sensor (new feature integration) and conflicting ground truth data. We demonstrate significant overall performance improvements using NAPS (an accuracy of 95.29%) in detecting human task errors (a four class problem) caused by impaired cognitive states and a negligible drop in performance with the case of ambiguous ground truth labels (an accuracy of 93.93%), when compared to other methodologies (an accuracy of 64.91%). This work potentially sets the foundation for other human-centric modeling systems that rely on human state prediction modeling.

Opportunities for Smartphone Sensing in E-Health Research: A Narrative Review

Recent years have seen significant advances in the sensing capabilities of smartphones, enabling them to collect rich contextual information such as location, device usage, and human activity at a given point in time. Combined with widespread user adoption and the ability to gather user data remotely, smartphone-based sensing has become an appealing choice for health research. Numerous studies over the years have demonstrated the promise of using smartphone-based sensing to monitor a range of health conditions, particularly mental health conditions. However, as research is progressing to develop the predictive capabilities of smartphones, it becomes even more crucial to fully understand the capabilities and limitations of using this technology, given its potential impact on human health. To this end, this paper presents a narrative review of smartphone-sensing literature from the past 5 years, to highlight the opportunities and challenges of this approach in healthcare. It provides an overview of the type of health conditions studied, the types of data collected, tools used, and the challenges encountered in using smartphones for healthcare studies, which aims to serve as a guide for researchers wishing to embark on similar research in the future. Our findings highlight the predominance of mental health studies, discuss the opportunities of using standardized sensing approaches and machine-learning advancements, and present the trends of smartphone sensing in healthcare over the years.

Loneliness and Social Isolation Detection Using Passive Sensing Techniques: Scoping Review

BACKGROUND

Loneliness and social isolation are associated with multiple health problems, including depression, functional impairment, and death. Mobile sensing using smartphones and wearable devices, such as fitness trackers or smartwatches, as well as ambient sensors, can be used to acquire data remotely on individuals and their daily routines and behaviors in real time. This has opened new possibilities for the early detection of health and social problems, including loneliness and social isolation.

OBJECTIVE

This scoping review aimed to identify and synthesize recent scientific studies that used passive sensing techniques, such as the use of in-home ambient sensors, smartphones, and wearable device sensors, to collect data on device users' daily routines and behaviors to detect loneliness or social isolation. This review also aimed to examine various aspects of these studies, especially target populations, privacy, and validation issues.

METHODS

A scoping review was undertaken, following the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews). Studies on the topic under investigation were identified through 6 databases (IEEE Xplore, Scopus, ACM, PubMed, Web of Science, and Embase). The identified studies were screened for the type of passive sensing detection methods for loneliness and social isolation, targeted population, reliability of the detection systems, challenges, and limitations of these detection systems.

RESULTS

After conducting the initial search, a total of 40,071 papers were identified. After screening for inclusion and exclusion criteria, 29 (0.07%) studies were included in this scoping review. Most studies (20/29, 69%) used smartphone and wearable technology to detect loneliness or social isolation, and 72% (21/29) of the studies used a validated reference standard to assess the accuracy of passively collected data for detecting loneliness or social isolation.

CONCLUSIONS

Despite the growing use of passive sensing technologies for detecting loneliness and social isolation, some substantial gaps still remain in this domain. A population heterogeneity issue exists among several studies, indicating that different demographic characteristics, such as age and differences in participants' behaviors, can affect loneliness and social isolation. In addition, despite extensive personal data collection, relatively few studies have addressed privacy and ethical issues. This review provides uncertain evidence regarding the use of passive sensing to detect loneliness and social isolation. Future research is needed using robust study designs, measures, and examinations of privacy and ethical concerns.

Integrating Insights About Human Movement Patterns From Digital Data Into Psychological Science

Understanding people’s movement patterns has many important applications, from analyzing habits and social behaviors, to predicting the spread of disease. Information regarding these movements and their locations is now deeply embedded in digital data generated via smartphones, wearable sensors, and social-media interactions. Research has largely used data-driven modeling to detect patterns in people’s movements, but such approaches are often devoid of psychological theory and fail to capitalize on what movement data can convey about associated thoughts, feelings, attitudes, and behavior. This article outlines trends in current research in this area and discusses how psychologists can better address theoretical and methodological challenges in future work while capitalizing on the opportunities that digital movement data present. We argue that combining approaches from psychology and data science will improve researchers’ and policy makers’ abilities to make predictions about individuals’ or groups’ movement patterns. At the same time, an interdisciplinary research agenda will provide greater capacity to advance psychological theory.

Exploring Post COVID-19 Outbreak Intradaily Mobility Pattern Change in College Students: A GPS-Focused Smartphone Sensing Study

With the outbreak of the COVID-19 pandemic in 2020, most colleges and universities move to restrict campus activities, reduce indoor gatherings and move instruction online. These changes required that students adapt and alter their daily routines accordingly. To investigate patterns associated with these behavioral changes, we collected smartphone sensing data using the Beiwe platform from two groups of undergraduate students at a major North American university, one from January to March of 2020 (74 participants), the other from May to August (52 participants), to observe the differences in students' daily life patterns before and after the start of the pandemic. In this paper, we focus on the mobility patterns evidenced by GPS signal tracking from the students' smartphones and report findings using several analytical methods including principal component analysis, circadian rhythm analysis, and predictive modeling of perceived sadness levels using mobility-based digital metrics. Our findings suggest that compared to the pre-COVID group, students in the mid-COVID group generally 1) registered a greater amount of midday movement than movement in the morning (8-10 a.m.) and in the evening (7-9 p.m.), as opposed to the other way around; 2) exhibited significantly less intradaily variability in their daily movement; 3) visited less places and stayed at home more everyday, and; 4) had a significant lower correlation between their mobility patterns and negative mood.

Momentary loneliness among older adults: Contextual differences and their moderation by gender and race/ethnicity

RATIONALE

Studies suggest that loneliness is associated with age. Among older adults, women and Black adults may be at greater risk than men and White adults, respectively. Social and physical contexts are also linked with loneliness. However, little is known about whether and how those of different genders and racial/ethnic groups may experience social and physical contexts differently in terms of their real-time loneliness, and the extent to which these differences may be explained by differential exposure or reactivity to contexts.

OBJECTIVE

We examine (1) how momentary loneliness relates to (a) gender and race/ethnicity and (b) social and physical context; and the extent to which gender and racial/ethnic groups may be (2) differentially exposed to loneliness-related contexts and/or (3) differentially reacting to these contexts.

METHODS

Using multilevel regressions, we analyzed ecological momentary assessments from 342 community-dwelling U.S. older adults from the Chicago Health and Activity Space in Real Time study. In each of three waves of data collection, smartphone "pings" (five per day for 21 days; n = 12,793 EMAs) assessed loneliness, social context (e.g., alone, with a spouse/partner), and location/physical context (e.g., home, at work).

RESULTS

Men consistently reported greater loneliness intensity than women, including after adjusting for momentary physical and social context. Older adults momentarily outside the home and/or not alone were less likely to feel lonely than their counterparts. However, the protective effect of being outside of the home (vs. home) was weaker among women and Black and Hispanic older adults, and the protective effect of being with one or more others (vs. alone) was weaker among women.

CONCLUSIONS

Results are among the first to identify contextual effects on real-time loneliness in older adults and how these associations vary by gender and race/ethnicity. Knowledge regarding momentary variation in loneliness may inform future just-in-time adaptive loneliness interventions in older adulthood.

Multi-modal data collection for measuring health, behavior, and living environment of large-scale participant cohorts

BACKGROUND

As mobile technologies become ever more sensor-rich, portable, and ubiquitous, data captured by smart devices are lending rich insights into users' daily lives with unprecedented comprehensiveness and ecological validity. A number of human-subject studies have been conducted to examine the use of mobile sensing to uncover individual behavioral patterns and health outcomes, yet minimal attention has been placed on measuring living environments together with other human-centered sensing data. Moreover, the participant sample size in most existing studies falls well below a few hundred, leaving questions open about the reliability of findings on the relations between mobile sensing signals and human outcomes.

RESULTS

To address these limitations, we developed a home environment sensor kit for continuous indoor air quality tracking and deployed it in conjunction with smartphones, Fitbits, and ecological momentary assessments in a cohort study of up to 1,584 college student participants per data type for 3 weeks. We propose a conceptual framework that systematically organizes human-centric data modalities by their temporal coverage and spatial freedom. Then we report our study procedure, technologies and methods deployed, and descriptive statistics of the collected data that reflect the participants' mood, sleep, behavior, and living environment.

CONCLUSIONS

We were able to collect from a large participant cohort satisfactorily complete multi-modal sensing and survey data in terms of both data continuity and participant adherence. Our novel data and conceptual development provide important guidance for data collection and hypothesis generation in future human-centered sensing studies.