Momentary predictors of compliance in studies using the experience sampling method

Understanding the Relationship Between Ecological Momentary Assessment Methods, Sensed Behavior, and Responsiveness: Cross-Study Analysis

Background

Ecological momentary assessment (EMA) offers an effective method to collect frequent, real-time data on an individual's well-being. However, challenges exist in response consistency, completeness, and accuracy.

Objective

This study examines EMA response patterns and their relationship with sensed behavior for data collected from diverse studies. We hypothesize that EMA response rate (RR) will vary with prompt time of day, number of questions, and behavior context. In addition, we postulate that response quality will decrease over the study duration and that relationships will exist between EMA responses, participant demographics, behavior context, and study purpose.

Methods

Data from 454 participants in 9 clinical studies were analyzed, comprising 146,753 EMA mobile prompts over study durations ranging from 2 weeks to 16 months. Concurrently, sensor data were collected using smartwatch or smart home sensors. Digital markers, such as activity level, time spent at home, and proximity to activity transitions (change points), were extracted to provide context for the EMA responses. All studies used the same data collection software and EMA interface but varied in participant groups, study length, and the number of EMA questions and tasks. We analyzed RR, completeness, quality, alignment with sensor-observed behavior, impact of study design, and ability to model the series of responses.

Results

The average RR was 79.95%. Of those prompts that received a response, the proportion of fully completed response and task sessions was 88.37%. Participants were most responsive in the evening (82.31%) and on weekdays (80.43%), although results varied by study demographics. While overall RRs were similar for weekday and weekend prompts, older adults were more responsive during the week (an increase of 0.27), whereas younger adults responded less during the week (a decrease of 3.25). RR was negatively correlated with the number of EMA questions (r=-0.433, P<.001). Additional correlations were observed between RR and sensor-detected activity level (r=0.045, P<.001), time spent at home (r=0.174, P<.001), and proximity to change points (r=0.124, P<.001). Response quality showed a decline over time, with careless responses increasing by 0.022 (P<.001) and response variance decreasing by 0.363 (P<.001). The within-study dynamic time warping distance between response sequences averaged 14.141 (SD 11.957), compared with the 33.246 (SD 4.971) between-study average distance. ARIMA (Autoregressive Integrated Moving Average) models fit the aggregated time series with high log-likelihood values, indicating strong model fit with low complexity.

Conclusions

EMA response patterns are significantly influenced by participant demographics and study parameters. Tailoring EMA prompt strategies to specific participant characteristics can improve RRs and quality. Findings from this analysis suggest that timing EMA prompts close to detected activity transitions and minimizing the duration of EMA interactions may improve RR. Similarly, strategies such as gamification may be introduced to maintain participant engagement and retain response variance.

Smartwatch-Based Ecological Momentary Assessment for High-Temporal-Density, Longitudinal Measurement of Alcohol Use (AlcoWatch): Feasibility Evaluation

BACKGROUND

Ecological momentary assessment methods have recently been adapted for use on smartwatches. One particular class of these methods, developed to minimize participant burden and maximize engagement and compliance, is referred to as microinteraction-based ecological momentary assessment (μEMA).

OBJECTIVE

This study explores the feasibility of using these smartwatch-based μEMA methods to capture longitudinal, high-temporal-density self-report data about alcohol consumption in a nonclinical population selected to represent high- and low-socioeconomic position (SEP) groups.

METHODS

A total of 32 participants from the Avon Longitudinal Study of Parents and Children (13 high and 19 low SEP) wore a smartwatch running a custom-developed μEMA app for 3 months between October 2019 and June 2020. Every day over a 12-week period, participants were asked 5 times a day about any alcoholic drinks they had consumed in the previous 2 hours, and the context in which they were consumed. They were also asked if they had missed recording any alcoholic drinks the day before. As a comparison, participants also completed fortnightly online diaries of alcohol consumed using the Timeline Followback (TLFB) method. At the end of the study, participants completed a semistructured interview about their experiences.

RESULTS

The compliance rate for all participants who started the study for the smartwatch μEMA method decreased from around 70% in week 1 to 45% in week 12, compared with the online TLFB method which was flatter at around 50% over the 12 weeks. The compliance for all participants still active for the smartwatch μEMA method was much flatter, around 70% for the whole 12 weeks, while for the online TLFB method, it varied between 50% and 80% over the same period. The completion rate for the smartwatch μEMA method varied around 80% across the 12 weeks. Within high- and low-SEP groups there was considerable variation in compliance and completion at each week of the study for both methods. However, almost all point estimates for both smartwatch μEMA and online TLFB indicated lower levels of engagement for low-SEP participants. All participants scored "experiences of using" the 2 methods equally highly, with "willingness to use again" slightly higher for smartwatch μEMA.

CONCLUSIONS

Our findings demonstrate the acceptability and potential utility of smartwatch μEMA methods for capturing data on alcohol consumption. These methods have the benefits of capturing higher-temporal-density longitudinal data on alcohol consumption, promoting greater participant engagement with less missing data, and potentially being less susceptible to recall errors than established methods such as TLFB. Future studies should explore the factors impacting participant attrition (the biggest reason for reduced engagement), latency issues, and the validity of alcohol data captured with these methods. The consistent pattern of lower engagement among low-SEP participants than high-SEP participants indicates that further work is warranted to explore the impact and causes of these differences.

Samply Stream API: The AI-enhanced method for real-time event data streaming

This manuscript introduces a novel method for conducting behavioral and social research by streaming real-time information to participants and manipulating content for experimental purposes via AI. We present an extension of the Samply software, which facilitates the integration of event-related data with mobile surveys and experiments. To assess the feasibility of this method, we conducted an experiment where news headlines were modified by a Chat-GPT algorithm and streamed to participants via the Samply Stream API and mobile push notifications. Feedback from participants indicated that most did not experience technical problems. There was no significant difference in readability across original, paraphrased, and misinformation-injected news conditions, with only 1.2% of all news items reported as unreadable. Participants reported significantly less familiarity with misinformation-injected news (84% unfamiliarity) compared to original and paraphrased news (73% unfamiliarity), suggesting successful manipulation of information without compromising readability. Dropout and non-response rates were comparable to those in other experience sampling studies. The streaming method offers significant potential for various applications, including public opinion research, healthcare, marketing, and environmental monitoring. By enabling the real-time collection of contextually relevant data, this method has the potential to enhance the external validity of behavioral research and provides a powerful tool for studying human behavior in naturalistic settings.

Applying machine learning to ecological momentary assessment data to identify predictors of loss-of-control eating and overeating severity in adolescents: A preliminary investigation

OBJECTIVE

Several factors (e.g., interpersonal stress, affect) predict loss-of-control (LOC) eating and overeating in adolescents, but most past research has tested predictors separately. We applied machine learning to simultaneously evaluate multiple possible predictors of LOC-eating and overeating severity in pooled and person-specific models.

METHOD

Twenty-eight adolescents (78.57% female, age = 15.87 ± 1.59 years, BMI %ile = 92.71 ± 8.86) who endorsed ≥ two past-month LOC-eating episodes completed a week-long ecological momentary assessment protocol. Pooled models were fit to the aggregated data with elastic-net regularized regression and evaluated using nested cross-validation. Person-specific models were fit and evaluated as proof-of-concept.

RESULTS

Across adolescents, the median out-of-sample R2 of the pooled LOC-eating severity model was .33. The top predictors were between-subjects food craving, sadness, interpersonal conflict, shame, distress, stress (inverse association), and anger (inverse association), and within- and between-subjects wishing relationships were better. The median out-of-sample R2 for pooled overeating severity model was .20. The top predictors were between-person food craving, loneliness, mixed race, and feeling rejected (inverse association), and within-subjects guilt, nervousness, wishing for more friends (inverse association), and feeling scared, annoyed, and rejected (all inverse associations). Person-specific models demonstrated poor fit (median LOC-eating severity R2 = .003, median overeating R2 = -.009); 61% and 36% of adolescents' models performed better than chance for LOC-eating and overeating severity, respectively.

DISCUSSION

Altogether, group-level models may hold utility in predicting LOC-eating and overeating severity, but model performance for person-specific models is variable, and additional research with larger samples over an extended assessment period is needed. Ultimately, a mix of these approaches may improve the identification of momentary predictors of LOC eating and overeating, providing novel and personalized opportunities for intervention.

The potential of experience sampling methods in palliative care

BACKGROUND

Experience sampling methods typically involve multiple self-report assessments per day over consecutive days. Unlike traditional patient-reported outcome measures or interviews, such methods offer the possibility to capture the temporal fluctuations of experiences in daily environments, making them valuable for studying the daily lives of people with advanced illness. Yet, their use in palliative care research is limited.

AIMS

To introduce experience sampling methods to the field of palliative care as a valuable tool for studying the everyday experiences of people with advanced illness, and to present the findings of an experience sampling methods pilot study with people with advanced breast or advanced lung cancer.

EVIDENCE USED TO SUPPORT THE INFORMATION PRESENTED

We draw on published health research using experience sampling methods. We present a newly developed experience sampling methods questionnaire (ESM-AC) and report pilot study findings on the feasibility and acceptability of experience sampling methods among people with advanced breast or lung cancer.

KEY LEARNING POINTS

Experience sampling methods hold potential to uncover the dynamics of everyday experiences of people with advanced illness. The methods offer considerable flexibility and options to answer a variety of research questions, but consideration is required regarding sampling protocols and participant burden. We showed appropriate feasibility and acceptable participant burden of the methods among people with advanced breast or advanced lung cancer.

Uncovering the Missing Pieces: Predictors of Nonresponse in a Mobile Experience Sampling Study on Media Effects Among Youth

Mobile Experience Sampling (MES) is a promising tool for understanding youth digital media use and its effects. Unfortunately, the method suffers from high levels of missing data. Depending on whether the data is randomly or non-randomly missing, it can have severe effects on the validity of findings. For this reason, we investigated predictors of non-response in an MES study on displacement effects of digital media use on adolescents' well-being and academic performance (N = 347). Multilevel binary logistic regression identified significant influencing factors of response odds, such as afternoon beeps and being outside. Importantly, adolescents with poorer school grades were more likely to miss beeps. Because this missingness was related to the outcome variable, modern missing data methods such as multiple imputation should be applied before analyzing the data. Understanding the reasons for non-response can be seen as the first step to preventing, minimizing, and handling missing data in MES studies, ultimately ensuring that the collected data is fully utilized to draw accurate conclusions.

Perceived responsiveness in suicidal ideation: An experience sampling study in psychiatric patients

INTRODUCTION

Perceived responsiveness, or the extent to which one feels understood, validated and cared for by close others, plays a crucial role in people's well-being. Can this interpersonal process also protect people at risk? We assessed whether fluctuations in suicidal ideation were associated with fluctuations in the degree of perceived responsiveness that psychiatric patients (admitted in the context of suicide or indicating suicidal ideation) experienced in daily interactions immediately after discharge.

METHODS

Fifty-seven patients reported on suicidal ideation (5 times a day) and perceived responsiveness (daily) for four consecutive weeks. The effects of established risk factors-thwarted belongingness, perceived burdensomeness, and hopelessness-were assessed as well.

RESULTS

The more patients felt that close others had been responsive to them, the less suicidal ideation they reported. At low levels of thwarted belongingness, perceived burdensomeness, or hopelessness, perceived responsiveness seemed to play a protective role, negatively co-occurring with suicidal ideation. When thwarted belongingness, perceived burdensomeness, and hopelessness were high, perceived responsiveness did not have an effect.

CONCLUSION

Perceived responsiveness could be a protective factor for suicidal ideation for people at risk only when they are experiencing low levels of negative perceptions. When experiencing highly negative perceptions, however, perceived responsiveness seems to matter less.

Complex PTSD symptoms predict positive symptoms of psychosis in the flow of daily life

BACKGROUND

Post-traumatic stress disorder (PTSD) has been shown to predict psychotic symptomology. However, few studies have examined the relative contribution of PTSD compared to broader post-traumatic sequelae in maintaining psychosis. Complex PTSD (cPTSD), operationalized using ICD-11 criteria, includes core PTSD (intrusions, avoidance, hyperarousal) as well as additional "disturbances of self-organisation" (DSO; emotional dysregulation, interpersonal difficulties, negative self-concept) symptoms, more likely to be associated with complex trauma histories. It was hypothesized that DSOs would be associated with positive psychotic symptoms (paranoia, voices, and visions) in daily life, over and above core PTSD symptoms.

METHODS

This study (N = 153) employed a baseline subsample of the Study of Trauma And Recovery (STAR), a clinical sample of participants with comorbid post-traumatic stress and psychosis symptoms. Core PTSD, DSO and psychosis symptoms were assessed up to 10 times per day at quasi-random intervals over six consecutive days using Experience Sampling Methodology.

RESULTS

DSOs within the preceding 90 min predicted paranoia, voices, and visions at subsequent moments. These relationships persisted when controlling for core PTSD symptoms within this timeframe, which were themselves significant. The associations between DSOs and paranoia but not voices or visions, were significantly stronger than those between psychosis and core PTSD symptoms.

CONCLUSIONS

Consistent with an affective pathway to psychosis, the findings suggest that DSOs may be more important than core PTSD symptoms in maintaining psychotic experiences in daily life among people with comorbid psychosis and cPTSD, and indicate the potential importance of addressing broad post-traumatic sequelae in trauma-focused psychosis interventions.

Associations between daily affective experiences, trait and daily rumination on negative and positive affect: a diary study

BACKGROUND

Rumination has mostly been studied in relation to depression, however, it may also occur in response to positive emotions (i.e., positive rumination) and therefore may be a protective factor related to the maintenance of positive mood.

OBJECTIVE

We hypothesized that daily positive and negative affect would be associated with daily positive and negative rumination even after controlling for trait-level rumination.

METHOD

We carried out a diary study with university students (n = 178), where participants had to answer short surveys online about their daily affect and daily rumination every evening for 10 days. We analyzed our data with multilevel regression in R.

RESULTS

Daily positive and negative affect were significantly associated with daily negative and positive rumination, while trait-level rumination scores were not. Daily and trait-level rumination were moderately correlated (r = 0.333-0.440).

CONCLUSIONS

Our findings highlight that daily rumination plays a more significant role in daily emotional experiences than trait rumination across positive and negative valence domains. Daily negative affect appears to be more closely related to higher daily negative rumination than the lack of daily positive rumination, which could be relevant for intervention strategies.

Examining missingness at the momentary level in clinical research using ecological momentary assessment: Implications for suicide research

The use of intensive time sampling methods, such as ecological momentary assessment (EMA), has increased in clinical, and specifically suicide, research during the past decade. While EMA can capture dynamic intraindividual processes, repeated assessments increase participant burden, potentially resulting in low compliance. This study aimed to shed light on study-level and psychological variables, including suicidal ideation (SI), that may predict momentary prompt (i.e., prompt-to-prompt) completion. We combined data from three EMA studies examining mental health difficulties (N = 103; 10,656 prompts; 7144 completed), using multilevel models and machine learning to determine how well we can predict prompt-to-prompt completion and which variables are most important. The two most important variables in prompt-to-prompt completion were hours since the last prompt and time in study. Psychological variables added little predictive validity; similarly, trait-level SI demonstrated a small effect on prompt-to-prompt completion. Our study showed how study-level characteristics can be used to explain prompt-to-prompt compliance rates in EMA research, highlighting the potential for developing adaptive assessment schedules to improve compliance.

Predicting Completion of Ecological Momentary Assessments Among Substance-Using Women Experiencing Intimate Partner Violence

Noncompletion of ecological momentary assessment (EMA) surveys is a common issue and may yield bias in results if not properly handled. Using data observed at scheduled times as well as data retrieved later to fill missing responses, this study aims to investigate predictors of EMA completion, including demographic characteristics, time-related factors, and momentary experiences/behaviors. Data were from a 30-day EMA study including 145 women currently experiencing intimate partner violence (IPV) and using substances. The average rate of EMA completion was initially 51.4% at the scheduled times and increased to 72.6% after incorporating data from later-retrieved surveys. Participants who were younger, had more children, or had lower mean levels of negative affect dysregulation showed lower completion rates. At the momentary survey level, more days into the study and afternoon/evening reports (vs. morning reports) were associated with lower completion; lower levels of negative affect dysregulation, less smoking or alcohol use, and experiencing IPV were linked to lower momentary completion. Implications of the results for handling missing data in EMA are discussed and have important ramifications for future research, practice, and theory.

The relation between stress-induced dopamine release in the ventromedial prefrontal cortex, fronto-striatal functional connectivity, and negative urgency: A multimodal investigation using [18F]Fallypride PET, MRI and experience sampling

Negative urgency (NU), or the tendency to act rashly when stress of negative affect is high, could be the result of an insufficient control of the ventromedial prefrontal cortex (vmPFC) over the striatum, through an impaired dopamine (DA) transmission. Therefore, we investigated in vivo human stress-induced DA release in the vmPFC, its relation with fronto-striatal functional connectivity (FC), and NU in daily life. In total, 12 female healthy participants performed a simultaneous [18 F]fallypride PET and fMRI scan during which stress was induced. Regions displaying stress-induced DA release were identified and used to investigate stress-induced changes in fronto-striatal FC. Additionally, participants enrolled in an experience sampling study, reporting on daily life stress and rash actions over a 12-month-long period. Mixed models explored whether stress-induced DA release and FC moderated NU in daily life. Stress led to a lower FC between the vmPFC and dorsal striatum, but a higher FC between the vmPFC and contralateral ventral striatum. Participants with a higher FC between the vmPFC and dorsal striatum displayed more NU in daily life. A higher stress-induced DA release in the vmPFC was related to a higher stress-induced change in FC between the vmPFC and striatum. Participants with a higher DA release in the vmPFC displayed more NU in daily life. In conclusion, stress could differentially impact fronto-striatal FC whereby the connectivity with the dorsal striatum is especially important for NU in daily life. This could be mediated by a higher, but not a lower, stress-induced DA release in the vmPFC.

Burden and Inattentive Responding in a 12-Month Intensive Longitudinal Study: Interview Study Among Young Adults

BACKGROUND

Intensive longitudinal data (ILD) collection methods have gained popularity in social and behavioral research as a tool to better understand behavior and experiences over time with reduced recall bias. Engaging participants in these studies over multiple months and ensuring high data quality are crucial but challenging due to the potential burden of repeated measurements. It is suspected that participants may engage in inattentive responding (IR) behavior to combat burden, but the processes underlying this behavior are unclear as previous studies have focused on the barriers to compliance rather than the barriers to providing high-quality data.

OBJECTIVE

This study aims to broaden researchers' knowledge about IR during ILD studies using qualitative analysis and uncover the underlying IR processes to aid future hypothesis generation.

METHODS

We explored the process of IR by conducting semistructured qualitative exit interviews with 31 young adult participants (aged 18-29 years) who completed a 12-month ILD health behavior study with daily evening smartphone-based ecological momentary assessment (EMA) surveys and 4-day waves of hourly EMA surveys. The interviews assessed participants' motivations, the impact of time-varying contexts, changes in motivation and response patterns over time, and perceptions of attention check questions (ACQs) to understand participants' response patterns and potential factors leading to IR.

RESULTS

Thematic analysis revealed 5 overarching themes on factors that influence participant engagement: (1) friends and family also had to tolerate the frequent surveys, (2) participants tried to respond to surveys quickly, (3) the repetitive nature of surveys led to neutral responses, (4) ACQs within the surveys helped to combat overly consistent response patterns, and (5) different motivations for answering the surveys may have led to different levels of data quality.

CONCLUSIONS

This study aimed to examine participants' perceptions of the quality of data provided in an ILD study to contribute to the field's understanding of engagement. These findings provide insights into the complex process of IR and participant engagement in ILD studies with EMA. The study identified 5 factors influencing IR that could guide future research to improve EMA survey design. The identified themes offer practical implications for researchers and study designers, including the importance of considering social context, the consideration of dynamic motivations, and the potential benefit of including ACQs as a technique to reduce IR and leveraging the intrinsic motivators of participants. By incorporating these insights, researchers might maximize the scientific value of their multimonth ILD studies through better data collection protocols.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.2196/36666.

How Individuals Shape Their Acoustic Environment: Implications for Hearing Aid Comparison in Ecological Momentary Assessment

OBJECTIVES

When using ecological momentary assessment (EMA) to compare different hearing aid programs, it is usually assumed that for sufficiently long study durations similar situations will be experienced in both programs. However, conscious or subconscious adaptation of situations to one's hearing ability (e.g., asking a conversation partner to speak up, increasing TV volume)-which might be different across the time spent in different hearing aid programs-may challenge this assumption. In the present study, we investigated how test participants modify their acoustic environment and if these modifications depend on the hearing program.

DESIGN

Twenty-nine experienced hearing aid users were provided with hearing aids containing two hearing programs differing in directionality and noise reduction (NR). The hearing programs called NR-on and NR-off changed each day automatically. Over the course of 3 weeks, participants were asked to answer a questionnaire every time they encountered an acoustic situation they modified or would have liked to modify to improve the listening situation. Objective data on sound pressure level and classification of the acoustic situation were collected from the hearing aids. At the beginning of the study participants recollected modifications of the acoustic environments they typically do when using their own hearing aids and reported on the frequency of this behavior.

RESULTS

During the field trial, participants reported on average 2.3 situations per day that they modified or would have liked to modify. Modifications were usually performed quickly after the onset of the situation and significantly improved the pleasantness of the listening situation. While the number of the reported situations did not differ between the programs, modifications increasing the volume of the target signal and increasing the hearing aid volume were more frequent for the NR-on hearing program. Changes in the objective data at the time of the modification were consistent with the reported modifications. Further, the usage time as well as the distribution of the acoustic situations over the entire study period differed between the two hearing programs.

CONCLUSIONS

The large improvement in pleasantness due to the modification might explain the generally positive ratings observed in EMA studies. Furthermore, the results found here suggest that caution is needed when comparing ratings of audiological attributes in EMA, because the different modification behavior across hearing programs may lead to an underestimation of hearing problems and reduced sensitivity when comparing two technologies.

Loneliness in the Digital World: protocol for a co-produced ecological momentary assessment study in adolescents

INTRODUCTION

Loneliness has been identified as an important public health issue, peaking during adolescence. Previous research has suggested that social interaction is a key factor in loneliness, and positive social interaction can act as a protective factor against loneliness. However, it is unclear whether there are differing impacts of in-person and online social interaction on adolescents' loneliness and mental health. Ecological Momentary Assessment (EMA) designs are ideally suited for better understanding these associations.

METHOD AND ANALYSIS

In the 'Loneliness in the Digital World' study, we will use a co-developed EMA design to capture daily social interactions, loneliness and mental health such as positive and negative emotions, depression and anxiety in approximately 200 adolescents aged 12-15 years. We will combine this with comprehensive information gathered from online surveys. Analysing the data using techniques such as dynamic structural equation modelling, we will examine, among other research questions, the associations between online and in-person social interaction and feelings of loneliness. The results can help inform interventions to support adolescents with high levels of loneliness and poor mental health.

ETHICS AND DISSEMINATION

We received the ethics approval for the data collection from The Academic and Clinical Central Office for Research and Development, followed by the College of Medicine and Veterinary Medicine Ethics panel at University of Edinburgh, and finally reviewed by East of Scotland Research Ethics Service. The results will be disseminated through journal publications, conferences and seminar presentations and to relevant stakeholders such as teachers.

Never miss a beep: Using mobile sensing to investigate (non-)compliance in experience sampling studies

Given the increasing number of studies in various disciplines using experience sampling methods, it is important to examine compliance biases because related patterns of missing data could affect the validity of research findings. In the present study, a sample of 592 participants and more than 25,000 observations were used to examine whether participants responded to each specific questionnaire within an experience sampling framework. More than 400 variables from the three categories of person, behavior, and context, collected multi-methodologically via traditional surveys, experience sampling, and mobile sensing, served as predictors. When comparing different linear (logistic and elastic net regression) and non-linear (random forest) machine learning models, we found indication for compliance bias: response behavior was successfully predicted. Follow-up analyses revealed that study-related past behavior, such as previous average experience sampling questionnaire response rate, was most informative for predicting compliance, followed by physical context variables, such as being at home or at work. Based on our findings, we discuss implications for the design of experience sampling studies in applied research and future directions in methodological research addressing experience sampling methodology and missing data.

Investigating Receptivity and Affect Using Machine Learning: Ecological Momentary Assessment and Wearable Sensing Study

BACKGROUND

As mobile health (mHealth) studies become increasingly productive owing to the advancements in wearable and mobile sensor technology, our ability to monitor and model human behavior will be constrained by participant receptivity. Many health constructs are dependent on subjective responses, and without such responses, researchers are left with little to no ground truth to accompany our ever-growing biobehavioral data. This issue can significantly impact the quality of a study, particularly for populations known to exhibit lower compliance rates. To address this challenge, researchers have proposed innovative approaches that use machine learning (ML) and sensor data to modify the timing and delivery of surveys. However, an overarching concern is the potential introduction of biases or unintended influences on participants' responses when implementing new survey delivery methods.

OBJECTIVE

This study aims to demonstrate the potential impact of an ML-based ecological momentary assessment (EMA) delivery system (using receptivity as the predictor variable) on the participants' reported emotional state. We examine the factors that affect participants' receptivity to EMAs in a 10-day wearable and EMA-based emotional state-sensing mHealth study. We study the physiological relationships indicative of receptivity and affect while also analyzing the interaction between the 2 constructs.

METHODS

We collected data from 45 healthy participants wearing 2 devices measuring electrodermal activity, accelerometer, electrocardiography, and skin temperature while answering 10 EMAs daily, containing questions about perceived mood. Owing to the nature of our constructs, we can only obtain ground truth measures for both affect and receptivity during responses. Therefore, we used unsupervised and supervised ML methods to infer affect when a participant did not respond. Our unsupervised method used k-means clustering to determine the relationship between physiology and receptivity and then inferred the emotional state during nonresponses. For the supervised learning method, we primarily used random forest and neural networks to predict the affect of unlabeled data points as well as receptivity.

RESULTS

Our findings showed that using a receptivity model to trigger EMAs decreased the reported negative affect by >3 points or 0.29 SDs in our self-reported affect measure, scored between 13 and 91. The findings also showed a bimodal distribution of our predicted affect during nonresponses. This indicates that this system initiates EMAs more commonly during states of higher positive emotions.

CONCLUSIONS

Our results showed a clear relationship between affect and receptivity. This relationship can affect the efficacy of an mHealth study, particularly those that use an ML algorithm to trigger EMAs. Therefore, we propose that future work should focus on a smart trigger that promotes EMA receptivity without influencing affect during sampled time points.

Illuminating the daily life experiences of adolescents with and without ADHD: protocol for an ecological momentary assessment study

INTRODUCTION

Adolescents with attention-deficit/hyperactivity disorder (ADHD) are at elevated risk of a range of difficulties, among which emotion regulation, peer and co-occurring mental health problems are prominent challenges. To better support adolescents with ADHD, ecologically valid interventions that can be embedded in daily life to target the most proximal antecedents of these challenges are needed. Ecological momentary assessment (EMA) designs are ideally suited to meeting this need.

METHODS AND ANALYSES

In the mental health in the moment ADHD study, we will use an EMA design to capture the daily life experiences of approximately 120 adolescents aged 11-14 years with a clinical diagnosis of ADHD and the same number of age-matched and gender-matched peers without a diagnosis of ADHD. We will combine this with comprehensive information gathered from online surveys. Analysing the data using techniques such as dynamic structural equation modelling, we will examine, among other research questions, the role of emotion regulation and peer problems in mediating the links between characteristics of ADHD and commonly co-occurring outcomes such as anxiety, depression and conduct problems. The results can help inform interventions to support improved peer functioning and emotion regulation for adolescents with ADHD.

ETHICS AND DISSEMINATION

This study received a favourable ethical opinion through the National Health Service ethical review board and the University of Edinburgh PPLS Research Ethics panel. The results will be disseminated through journal publications, conferences and seminar presentations and to relevant stakeholders, such as those with ADHD, their families and clinicians.

Acceptability, usability and feasibility of experienced sampling method in chronic secondary pain syndromes

Background

In chronic pain syndromes, symptoms can fluctuate and change over time. Standard questionnaires cannot register these fluctuations. Nonetheless, the experience sampling method (ESM) is developed to collect momentary measurements of everyday complaints, tracing fluctuations in symptoms and disabling factors over time. Although valuable information can be collected in this way, assessment may also be a burden. This study aimed to investigate the acceptability, usability, and feasibility of ESM in chronic secondary pain syndromes, in a single-center study in the Netherlands.

Methods

A prospective observational study with repeated measurements was conducted in patients with chronic secondary neuropathic and musculoskeletal pain syndromes, including small fiber neuropathy, spinal cord injury, and rheumatoid disorder.

Results

Thirty-four participants were included and filled in the ESM, of whom 19 were diagnosed with small fiber neuropathy, 11 with spinal cord injury, and 4 with a rheumatoid disorder. The mean age was 54.7 ± 13.9 years (range: 23-77) of whom 52.9% were female. In total, 19 participants filled in the general and user-friendliness evaluation about the acceptability and usability of the ESM. The general evaluation showed no influence of ESM on participants' social contacts (mean 1.47, SD 1.12), activities (mean 1.74, SD 1.44), and mood (mean 1.89, SD 1.59). The answers options of ESM were a good representation of the experiences of participants (mean 4.58, SD 1.77). Regarding feasibility, the overall response rate for answering the beep signals of ESM was 44.5% in total. The missing rate per person varied from 13% to 97% with a median of 54.1%.

Conclusion

The general evaluation and the user-friendliness revealed sufficient outcomes in favor of the ESM application. ESM seems a promising measurement tool to use in secondary chronic pain syndromes.

Evaluation of Pressing Issues in Ecological Momentary Assessment

The use of repeated, momentary, real-world assessment methods known as the Experience Sampling Method and Ecological Momentary Assessment (EMA) has been broadly embraced over the last few decades. These methods have extended our assessment reach beyond lengthy retrospective self-reports as they can capture everyday experiences in their immediate context, including affect, behavior, symptoms, and cognitions. In this review we evaluate nine conceptual, methodological, and psychometric issues about EMA with the goal of stimulating conversation and guiding future research on these matters: the extent to which participants are actually reporting momentary experiences, respondents' interpretation of momentary questions, the use of comparison standards in responding, efforts to increase the EMA reporting period beyond the moment to longer periods within a day, training of EMA study participants, concerns about selection bias of respondents, the impact of missing EMA assessments, the reliability of momentary data, and for which purposes EMA might be considered a gold standard for assessment. Resolution of these issues should have far-reaching implications for advancing the field.

Introducing a Real-Time Method for Identifying the Predictors of Noncompliance with Event-Based Reporting of Tobacco Use in Ecological Momentary Assessment

BACKGROUND

Little is known about the factors that bias event-based (i.e., self-initiated) reporting of health behaviors in ecological momentary assessment (EMA) due to the difficulty inherent to tracking failures to self-initiate reports.

PURPOSE

To introduce a real-time method for identifying the predictors of noncompliance with event-based reporting.

METHODS

N = 410 adults who used both cigarettes and e-cigarettes completed a 1-week EMA protocol that combined random reporting of current contexts with event-based reporting of tobacco use. Each random assessment first asked if participants were currently using tobacco and, if so, the assessment converted into a "randomly captured" event report-indicating failure to self-initiate that report. Multilevel modeling tested predictors of failing to complete random reports and failing to self-initiate event reports.

RESULTS

On the person level, male sex, higher average cigarette rate, and higher average cigarette urge each predicted missing random reports. The person-level predictors of failing to self-initiate event reports were older age, higher average cigarette and e-cigarette rates, higher average cigarette urge, and being alone more on average; the moment-level predictors were lower cigarette urge, lower positive affect, alcohol use, and cannabis use. Strikingly, the randomly captured events comprised more of the total EMA reports (28%) than did the self-initiated event reports (24%). These report types were similar across most variables, with some exceptions, such as momentary cannabis use predicting the random capture of tobacco events.

CONCLUSIONS

This study demonstrated a method of identifying predictors of noncompliance with event-based reporting of tobacco use and enhancing the real-time capture of events.

Ecological Momentary Assessment of Heavy Episodic Drinking in the Early Postpartum Period: A Feasibility Study

Background

Postpartum mothers are at heightened risk for heavy episodic drinking (HED). Research with this population is critical to developing acceptable and effective tailored interventions, but new mothers who use alcohol are often reluctant to engage in research due to stigma and fear of child removal. This study examined feasibility of recruitment and ecological momentary assessment (EMA) in early postpartum mothers with histories of HED.

Methods

Participants were recruited via Facebook and Reddit and completed 14 days of EMA surveys. Baseline characteristics, recruitment feasibility, and EMA feasibility and acceptability were examined. Participants attended focus groups to further inform quantitative data.

Results

Reddit yielded a larger proportion of eligible individuals than Facebook, and 86% of the final enroled sample was recruited via Reddit. The average compliance rate of 75% is in line with other studies of similar populations. Half the sample reported alcohol use, and 78% reported the urge to drink at least once, supporting feasibility of EMA for collecting alcohol use data. Participants reported low burden and high acceptability of the study on both quantitative and qualitative measures. Baseline low maternal self-efficacy was associated with greater EMA compliance, and first-time mothers reported lower EMA burden compared to veteran mothers. College graduates, and participants with lower drinking refusal self-efficacy and greater alcohol severity were more likely to report alcohol use on EMA.

Conclusions

Future studies should consider Reddit as a recruitment strategy. Findings generally support feasibility and acceptability of EMA to assess HED in postpartum mothers.

Prompt-level predictors of compliance in an ecological momentary assessment study of young adults' mental health

BACKGROUND

Ecological momentary assessment (EMA) has become a popular method of gathering information about participants as they go about their daily lives. However, participant non-compliance, especially non-random compliance, in EMA is a concern. Better knowledge of the moment-to-moment factors that predict prompt non-response can inform the design of strategies to mitigate it.

METHOD

We used data from a general population young adult (n = 260) EMA study, 'decades-to-minutes' (D2M) and fitted dynamic structural equation models (DSEMs) to explore a range of candidate momentary predictors of missing the next prompt.

RESULTS

We found that higher levels of stress, overall negative affect, and the specific negative affective state of 'upset' at a given prompt predicted a greater likelihood of missing the next prompt. However, no other specific affective states, alcohol use, experiencing social provocations nor aggressive behaviour predicted missing the next prompt.

LIMITATIONS

The primary limitation of the present study was a lack of information on predictors concurrent with missed prompts.

CONCLUSIONS

Findings point to the potential value of gathering information on momentary negative affect (especially feeling upset) and stress to help inform strategies that intervene to prevent application disengagement at optimal moments and to feed into strategies to mitigate bias due to non-random non-response in EMA studies.

The Relation of Mood and Sexual Desire: An Experience Sampling Perspective on the Dual Control Model

We investigated the effect of fluctuations in negative and positive affect on momentary sexual motivation in a sample of women and men in a steady relationship (n = 133). Sexual motivation was regarded as the aggregate of sexual desire, subjective sexual arousal and openness to sexual contact. Experience sampling methodology was used to collect up to 70 measurements per participant over a period of seven consecutive days of sexual motivation, and negative and positive affect. Using multilevel analysis, we investigated cross-level interactions between affect and trait measures as specified in the dual control model (DCM). This model postulates sexually excitatory and inhibitory mechanisms as relatively independent systems that together can explain individual differences in sexual motivation and behavior. Results implicated that any intensification of feelings, positive or negative, was associated with a momentary increase in sexual motivation for participants more prone to sexual excitation. In the lagged analysis, higher preceding negative affect, measured 1-2 h earlier, forecasted an increase in current sexual motivation for participants more prone to sexual excitation. The lagged analysis included the autoregressive effect or inertia of sexual motivation. Inertia reflects the extent to which sexual motivation lingers and persists at similar levels. Our findings showed that sexual motivation levels persisted less in individuals with higher sexual inhibition proneness due to threat of performance failure. This study demonstrated how experience sampling methodology can be used to extend research on associations between mood and sexual motivation and implicates that DCM factors moderate these associations.

Don't Miss the Moment: A Systematic Review of Ecological Momentary Assessment in Suicide Research

Suicide and suicide-related behaviors are prevalent yet notoriously difficult to predict. Specifically, short-term predictors and correlates of suicide risk remain largely unknown. Ecological momentary assessment (EMA) may be used to assess how suicidal thoughts and behaviors (STBs) unfold in real-world contexts. We conducted a systematic literature review of EMA studies in suicide research to assess (1) how EMA has been utilized in the study of STBs (i.e., methodology, findings), and (2) the feasibility, validity and safety of EMA in the study of STBs. We identified 45 articles, detailing 23 studies. Studies mainly focused on examining how known longitudinal predictors of suicidal ideation perform within shorter (hourly, daily) time frames. Recent studies have explored the prospects of digital phenotyping of individuals with suicidal ideation. The results indicate that suicidal ideation fluctuates substantially over time (hours, days), and that individuals with higher mean ideation also have more fluctuations. Higher suicidal ideation instability may represent a phenotypic indicator for increased suicide risk. Few studies succeeded in establishing prospective predictors of suicidal ideation beyond prior ideation itself. Some studies show negative affect, hopelessness and burdensomeness to predict increased ideation within-day, and sleep characteristics to impact next-day ideation. The feasibility of EMA is encouraging: agreement to participate in EMA research was moderate to high (median = 77%), and compliance rates similar to those in other clinical samples (median response rate = 70%). More individuals reported suicidal ideation through EMA than traditional (retrospective) self-report measures. Regarding safety, no evidence was found of systematic reactivity of mood or suicidal ideation to repeated assessments of STBs. In conclusion, suicidal ideation can fluctuate substantially over short periods of time, and EMA is a suitable method for capturing these fluctuations. Some specific predictors of subsequent ideation have been identified, but these findings warrant further replication. While repeated EMA assessments do not appear to result in systematic reactivity in STBs, participant burden and safety remains a consideration when studying high-risk populations. Considerations for designing and reporting on EMA studies in suicide research are discussed.

A Novel Experience Sampling Method Tool Integrating Momentary Assessments of Cognitive Biases: Two Compliance, Usability, and Measurement Reactivity Studies

BACKGROUND

Experience sampling methods (ESMs) are increasingly being used to study ecological emotion dynamics in daily functioning through repeated assessments taken over several days. However, most of these ESM approaches are only based on self-report assessments, and therefore, studies on the ecological trajectories of their underlying mechanisms are scarce (ie, cognitive biases) and require evaluation through experimental tasks. We developed a novel ESM tool that integrates self-report measures of emotion and emotion regulation with a previously validated app-based cognitive task that allows for the assessment of underlying mechanisms during daily functioning.

OBJECTIVE

The objective of the study is to test this new tool and study its usability and the possible factors related to compliance with it in terms of latency and missing responses. Among the compliance predictors, we considered psychological and time-related variables, as well as usability, measurement reactivity, and participants' satisfaction with the tool.

METHODS

We conducted 2 extensive ESM studies-study 1 (N=84; a total of 3 assessments per day for 5 days) and study 2 (N=135; a total of 3 assessments per day for 10 days).

RESULTS

In both studies, participants found the tool highly usable (average usability score >81). By using mixed regression models, we found both common and specific results for the compliance predictors. In both study 1 and study 2, latency was significantly predicted by the day (P<.001 and P=.003, respectively). Participants showed slower responses to the notification as the days of the study progressed. In study 2 but not in study 1, latency was further predicted by individual differences in overload with the use of the app, and missing responses were accounted for by individual differences in stress reactivity to notifications (P=.04). Thus, by using a more extensive design, participants who experienced higher overload during the study were characterized by slower responses to notifications (P=.01), whereas those who experienced higher stress reactivity to the notification system were characterized by higher missing responses.

CONCLUSIONS

The new tool had high levels of usability. Furthermore, the study of compliance is of enormous importance when implementing novel ESM methods, including app-based cognitive tasks. The main predictors of latency and missing responses found across studies, specifically when using extensive ESM protocols (study 2), are methodology-related variables. Future research that integrates cognitive tasks in ESM designs should take these results into consideration by performing accurate estimations of participants' response rates to facilitate the optimal quality of novel eHealth approaches, as in this study.

Contextual Biases in Microinteraction Ecological Momentary Assessment (μEMA) Non-response

Ecological momentary assessment (EMA) is used to gather in-situ self-report on behaviors using mobile devices. Microinteraction EMA (μEMA), is a type of EMA where each survey is only one single question that can be answered with a glanceable microinteraction on a smartwatch. Prior work shows that even when μEMA interrupts far more frequently than smartphone-EMA, μEMA yields higher response rates with lower burden. We examined the contextual biases associated with non-response of μEMA prompts on a smartwatch. Based on prior work on EMA non-response and smartwatch use, we identified 10 potential contextual biases from three categories: temporal (time of the day, parts of waking day, day of the week, and days in study), device use (screen state, charging status, battery mode, and phone usage), and activity (wrist motion and location). We used data from a longitudinal study where 131 participants (Mean age 22.9 years, SD = 3.0) responded to μEMA surveys on a smartwatch for at least six months. Using mixed-effects logistic regression, we found that all temporal, activity/mobility, and device use variables had a statistically significant (p<0.001) association with momentary μEMA non-response. We discuss the implication of these results for future use of context-aware μEMA methodology.

Age-related changes in ongoing thought relate to external context and individual cognition

Understanding how age-related changes in cognition manifest in the real world is an important goal. One means of capturing these changes involves "experience sampling" participant's self-reported thoughts. Research has shown age-related changes in ongoing thought: e.g., older adults have fewer thoughts unrelated to the here-and-now. However, it is currently unclear how these changes reflect cognitive aging or lifestyle changes. 78 younger adults and 35 older adults rated their thought contents along 20 dimensions and the difficulty of their current activity in their daily lives. They also performed cognitive tasks in the laboratory. In a set of exploratory analyses, we found that older adults spent more time thinking positive, wanted thoughts, particularly in demanding contexts, and less time mind wandering about their future selves. Past-related thought related to episodic memory differently in older and younger adults. These findings inform the use of experience sampling to understand cognitive aging.

Daily-Life Social Experiences as a Potential Mediator of the Relationship Between Parenting and Psychopathology in Adolescence

Adolescence is a vulnerable period for psychopathology development, and certain parenting styles are consistent and robust predictors of a broad range of mental health outcomes. The mechanisms through which maladaptive parenting styles affect the development of psychopathology are assumed to be largely social in nature. Yet, the social mechanisms linking parenting to psychopathology are unexplored at arguably the most important level of functioning: daily life. This study aims to identify the associations between three parenting styles, and the experience of daily-life social interactions. Furthermore, we aim to explore the extent to which these parenting styles and altered daily-life social experiences are associated with psychopathology. In this study, we recruited a sample of N = 1,913 adolescents (63.3% girls; mean age = 13.7, age range = 11 to 20) as part of the first wave of the longitudinal cohort study "SIGMA". Parenting styles (psychological control, responsiveness, and autonomy support) and psychopathology symptoms were assessed using a retrospective questionnaire battery. The experienced quality of social interactions in different types of company was assessed using the experience sampling method, ten times per day for 6 days. Direct associations between parenting styles and general quality of daily-life social experiences were tested using a three-level linear model, revealing significant associations between social experiences and different parenting styles. When interaction effects were added to this model, we found that maternal responsiveness and paternal psychological control mainly related to altered qualities of social interactions with parents, while paternal autonomy support was associated with better experiences of non-family social interactions. Finally, an exploratory path analysis highlighted how both paternal autonomy support and altered quality of non-family interactions are uniquely associated with psychopathology levels. These findings demonstrate the general and pervasive effects of maladaptive parenting styles, as parenting seems to broadly affect adolescents' interactions with different types of social partners in everyday life. Moreover, they illustrate a potential mediated relationship in which altered daily-life social interactions could drive the development of psychopathology. A stronger focus may be required on the role of altered day-to-day social experiences in the prevention and potentially, the treatment, of adolescent psychopathology.

Can a social media intervention improve online communication about suicide? A feasibility study examining the acceptability and potential impact of the #chatsafe campaign

There is a need for effective and youth-friendly approaches to suicide prevention, and social media presents a unique opportunity to reach young people. Although there is some evidence to support the delivery of population-wide suicide prevention campaigns, little is known about their capacity to change behaviour, particularly among young people and in the context of social media. Even less is known about the safety and feasibility of using social media for the purpose of suicide prevention. Based on the #chatsafe guidelines, this study examines the acceptability, safety and feasibility of a co-designed social media campaign. It also examines its impact on young people's willingness to intervene against suicide and their perceived self-efficacy, confidence and safety when communicating on social media platforms about suicide. A sample of 189 young people aged 16-25 years completed three questionnaires across a 20-week period (4 weeks pre-intervention, immediately post-intervention, and at 4-week follow up). The intervention took the form of a 12-week social media campaign delivered to participants via direct message. Participants reported finding the intervention acceptable and they also reported improvements in their willingness to intervene against suicide, and their perceived self-efficacy, confidence and safety when communicating on social media about suicide. Findings from this study present a promising picture for the acceptability and potential impact of a universal suicide prevention campaign delivered through social media, and suggest that it can be safe to utilize social media for the purpose of suicide prevention.

How COVID-19 lockdown measures could impact patients with bulimia nervosa: Exploratory results from an ongoing experience sampling method study

OBJECTIVE

The impact of COVID-19 lockdown measures on patients with an eating disorder remains unclear, but preliminary results indicate that some patients could be more vulnerable to experience an increase in eating disorder pathology than others. To provide possible directions for future research, this report explored the impact of the Belgian COVID-19 lockdown measures on patients with bulimia nervosa (BN).

METHODS

The data of 15 female patients with BN from an ongoing experience sampling method study were analyzed. Mixed effects models compared surroundings, social context, negative affect (NA), positive affect (PA) and binge eating before and after the implementation of the lockdown measures.

RESULTS

After the implementation of the lockdown measures, significant changes in surroundings and social context were found as well as an increase in NA and decrease in PA. Patients who experienced a higher binge eating frequency during the lockdown also experienced a stronger change in NA and PA.

CONCLUSIONS

Future research should also look at changes in surroundings, social context, affect and how these interact with factors such as personality traits and coping styles when investigating why some patients are more susceptible to the negative effects of lockdown measures than others.

Evaluation of Hearing Aids in Everyday Life Using Ecological Momentary Assessment: What Situations Are We Missing?

Background Ecological momentary assessment (EMA) is a method to evaluate hearing aids in everyday life that uses repeated smartphone-based questionnaires to assess a situation as it happens. Although being ecologically valid and avoiding memory bias, this method may be prone to selection biases due to questionnaires being skipped or the phone not being carried along in certain situations. Purpose This investigation analyzed which situations are underrepresented in questionnaire responses and physically measured objective EMA data (e.g., sound level), and how such underrepresentation may depend on different triggers. Method In an EMA study, 20 subjects with hearing impairment provided daily information on reasons for missed data, that is, skipped questionnaires or missing connections between their phone and hearing aids. Results Participants often deliberately did not bring the study phone to social situations or skipped questionnaires because they considered it inappropriate, for example, during church service or when engaging in conversation. They answered fewer questions in conversations with multiple partners and were more likely to postpone questionnaires when not in quiet environments. Conclusion Data for social situations will likely be underrepresented in EMA. However, these situations are particularly important for the evaluation of hearing aids, as individuals with hearing impairment often have difficulties communicating in noisy situations. Thus, it is vital to optimize the design of the study to find a balance between avoiding memory bias and enabling subjects to report retrospectively on situations where phone usage may be difficult. The implications for several applications of EMA are discussed. Supplemental Material https://doi.org/10.23641/asha.12746849.