Designing mHealth Apps to Incorporate Evidence-Based Techniques for Prolonging User Engagement

Maintaining user engagement with mobile health (mHealth) apps can be a challenge. Previously, we developed a conceptual model to optimize patient engagement in mHealth apps by incorporating multiple evidence-based methods, including increasing health literacy, enhancing technical competence, and improving feelings about participation in clinical trials. This viewpoint aims to report on a series of exploratory mini-experiments demonstrating the feasibility of testing our previously published engagement conceptual model. We collected data from 6 participants using an app that showed a series of educational videos and obtained additional data via questionnaires to illustrate and pilot the approach. The videos addressed 3 elements shown to relate to engagement in health care app use: increasing health literacy, enhancing technical competence, and improving positive feelings about participation in clinical trials. We measured changes in participants' knowledge and feelings, collected feedback on the videos and content, made revisions based on this feedback, and conducted participant reassessments. The findings support the feasibility of an iterative approach to creating and refining engagement enhancements in mHealth apps. Systematically identifying the key evidence-based elements intended to be included in an app's design and then systematically testing the implantation of each element separately until a satisfactory level of positive impact is achieved is feasible and should be incorporated into standard app design. While mHealth apps have shown promise, participants are more likely to drop out than to be retained. This viewpoint highlights the potential for mHealth researchers to test and refine mHealth apps using approaches to better engage users.

Validating the efficacy and value proposition of mental fitness vocal biomarkers in a psychiatric population: prospective cohort study

Background

The utility of vocal biomarkers for mental health assessment has gained increasing attention. This study aims to further this line of research by introducing a novel vocal scoring system designed to provide mental fitness tracking insights to users in real-world settings.

Methods

A prospective cohort study with 104 outpatient psychiatric participants was conducted to validate the "Mental Fitness Vocal Biomarker" (MFVB) score. The MFVB score was derived from eight vocal features, selected based on literature review. Participants' mental health symptom severity was assessed using the M3 Checklist, which serves as a transdiagnostic tool for measuring depression, anxiety, post-traumatic stress disorder, and bipolar symptoms.

Results

The MFVB demonstrated an ability to stratify individuals by their risk of elevated mental health symptom severity. Continuous observation enhanced the MFVB's efficacy, with risk ratios improving from 1.53 (1.09-2.14, p=0.0138) for single 30-second voice samples to 2.00 (1.21-3.30, p=0.0068) for data aggregated over two weeks. A higher risk ratio of 8.50 (2.31-31.25, p=0.0013) was observed in participants who used the MFVB 5-6 times per week, underscoring the utility of frequent and continuous observation. Participant feedback confirmed the user-friendliness of the application and its perceived benefits.

Conclusions

The MFVB is a promising tool for objective mental health tracking in real-world conditions, with potential to be a cost-effective, scalable, and privacy-preserving adjunct to traditional psychiatric assessments. User feedback suggests that vocal biomarkers can offer personalized insights and support clinical therapy and other beneficial activities that are associated with improved mental health risks and outcomes.

Deep breathing in your hands: designing and assessing a DTx mobile app

Digital Therapeutics (DTx) are experiencing rapid advancements within mobile and mental healthcare sectors, with their ubiquity and enhanced accessibility setting them apart as uniquely effective solutions. In this evolving context, our research focuses on deep breathing, a vital technique in mental health management, aiming to optimize its application in DTx mobile platforms. Based on well-founded theories, we introduced a gamified and affordance-driven design, facilitating intuitive breath control. To enhance user engagement, we deployed the Mel Frequency Cepstral Coefficient (MFCC)-driven personalized machine learning method for accurate biofeedback visualization. To assess our design, we enlisted 70 participants, segregating them into a control and an intervention group. We evaluated Heart Rate Variability (HRV) metrics and collated user experience feedback. A key finding of our research is the stabilization of the Standard Deviation of the NN Interval (SDNN) within Heart Rate Variability (HRV), which is critical for stress reduction and overall health improvement. Our intervention group observed a pronounced stabilization in SDNN, indicating significant stress alleviation compared to the control group. This finding underscores the practical impact of our DTx solution in managing stress and promoting mental health. Furthermore, in the assessment of our intervention cohort, we observed a significant increase in perceived enjoyment, with a notable 22% higher score and 10.69% increase in positive attitudes toward the application compared to the control group. These metrics underscore our DTx solution's effectiveness in improving user engagement and fostering a positive disposition toward digital therapeutic efficacy. Although current technology poses challenges in seamlessly incorporating machine learning into mobile platforms, our model demonstrated superior effectiveness and user experience compared to existing solutions. We believe this result demonstrates the potential of our user-centric machine learning techniques, such as gamified and affordance-based approaches with MFCC, which could contribute significantly to the field of mobile mental healthcare.

Predictors of Use and Drop Out From a Web-Based Cognitive Behavioral Therapy Program and Health Community for Depression and Anxiety in Primary Care Patients: Secondary Analysis of a Randomized Controlled Trial

BACKGROUND

A previously reported study examined the treatment of primary care patients with at least moderate severity depressive or anxiety symptoms via an evidence-based computerized cognitive behavioral therapy (CCBT) program (Beating the Blues) and an online health community (OHC) that included a moderated internet support group. The 2 treatment arms proved to be equally successful at 6-month follow-up.

OBJECTIVE

Although highly promising, e-mental health treatment programs have encountered high rates of noninitiation, poor adherence, and discontinuation. Identifying ways to counter these tendencies is critical for their success. To further explore these issues, this study identified the primary care patient characteristics that increased the chances patients would not initiate the use of an intervention, (ie, not try it even once), initiate use, and go on to discontinue or continue to use an intervention.

METHODS

The study had 3 arms: one received access to CCBT (n=301); another received CCBT plus OHC (n=302), which included a moderated internet support group; and the third received usual care (n=101). Participants in the 2 active intervention arms of the study were grouped together for analyses of CCBT use (n=603) because both arms had access to CCBT, and there were no differences in outcomes between the 2 arms. Analyses of OHC use were based on 302 participants who were randomized to that arm.

RESULTS

Several baseline patient characteristics were associated with failure to initiate the use of CCBT, including having worse physical health (measured by the Short Form Health Survey Physical Components Score, P=.01), more interference from pain (by the Patient-Reported Outcomes Measurement Information System Pain Interference score, P=.048), less formal education (P=.02), and being African American or another US minority group (P=.006). Characteristics associated with failure to initiate use of the OHC were better mental health (by the Short Form Health Survey Mental Components Score, P=.04), lower use of the internet (P=.005), and less formal education (P=.001). Those who initiated the use of the CCBT program but went on to complete less of the program had less formal education (P=.01) and lower severity of anxiety symptoms (P=.03).

CONCLUSIONS

This study found that several patient characteristics predicted whether a patient was likely to not initiate use or discontinue the use of CCBT or OHC. These findings have clear implications for actionable areas that can be targeted during initial and ongoing engagement activities designed to increase patient buy-in, as well as increase subsequent use and the resulting success of eHealth programs.

TRIAL REGISTRATION

ClinicalTrials.gov NCT01482806; https://clinicaltrials.gov/study/NCT01482806.

Users' views on the use of a smartwatch app to collect daily symptom data in individuals with multiple long-term conditions (Multimorbidity): A qualitative study

Introduction

Long-term conditions are a major burden on health systems. One way to facilitate more research and better clinical care among patients with long-term conditions is to collect accurate data on their daily symptoms (patient-generated health data) using wearable technologies. Whilst evidence is growing for the use of wearable technologies in single conditions, there is less evidence of the utility of frequent symptom tracking in those who have more than one condition.

Aims

To explore patient views of the acceptability of collecting daily patient-generated health data for three months using a smartwatch app.

Methods

Watch Your Steps was a longitudinal study which recruited 53 patients to track over 20 symptoms per day for a 90-day period using a study app on smartwatches. Semi-structured interviews were conducted with a sub-sample of 20 participants to explore their experience of engaging with the app.

Results

In a population of older people with multimorbidity, patients were willing and able to engage with a patient-generated health data app on a smartwatch. It was suggested that to maintain engagement over a longer period, more 'real-time' feedback from the app should be available. Participants did not seem to consider the management of more than one condition to be a factor in either engagement or use of the app, but the presence of severe or chronic pain was at times a barrier.

Conclusion

This study has provided preliminary evidence that multimorbidity was not a major barrier to engagement with patient-generated health data via a smartwatch symptom tracking app.

Identifying Behavior Change Techniques in an Artificial Intelligence-Based Fitness App: A Content Analysis

In the field of artificial intelligence-based fitness apps, the effective integration of behavior change techniques (BCTs) is critical for promoting physical activity and improving health outcomes. However, the specific BCTs employed by apps and their impact on user engagement and behavior change are not explored sufficiently. This study investigates the Freeletics fitness app through a mixed-methods approach to evaluate the use of BCTs. In the quantitative analysis, fifteen unique BCTs were identified based on the Behavior Change Technique Taxonomy (V1). In the qualitative analysis, user reviews (n=400) were examined to understand perspectives on the app's effectiveness in promoting behavior change. Goal setting, action planning, self-monitoring of behavior, and social support were among the most prevalent BCTs identified in the Freeletics app, and their effectiveness in enhancing user engagement and promoting behavior change was also highlighted by user reviews. Among the areas of improvement identified in the study were the need for simplifying personalization options and addressing user concerns regarding the specificity of feedback. The study underscores the importance of integrating BCTs effectively within AI-based fitness apps to drive user engagement and facilitate behavior change. It contributes valuable insights into the design and implementation of BCTs in fitness apps and offers recommendations for developers, emphasizing the significance of goal setting, feedback mechanisms, self-monitoring, and social support. By understanding the impact of specific BCTs on user behavior and addressing user concerns, developers can create more effective fitness apps, ultimately promoting healthier lifestyles and positive behavior change.

An Active Model of Research Translation for the General Public: Content Analysis of a YouTube-Based Health Podcast

BACKGROUND

Online health information seeking is changing the way people engage with health care and the health system. Recent changes in practices related to seeking, accessing, and disseminating scientific research, and in particular health information, have enabled a high level of user engagement.

OBJECTIVE

This study aims to examine an innovative model of research translation, The Huberman Lab Podcast (HLP), developed by Andrew Huberman, Professor of Neurobiology and Ophthalmology at the Stanford School of Medicine. The HLP leverages social media to deliver health information translated into specific, actionable practices and health strategies directly to the general public. This research characterizes the HLP as an Active Model of Research Translation and assesses its potential as a framework for replicability and wider adoption.

METHODS

We applied conventional content analysis of the YouTube transcript data and directed content analysis of viewers' YouTube comments to 23 HLP episodes released from January to October 2021, reflecting the time of data analysis. We selected 7 episodes and a welcome video, to describe and identify key characteristics of the HLP model. We analyzed viewer comments for 18 episodes to determine whether viewers found the HLP content valuable, accessible, and easy to implement.

RESULTS

The key HLP features are direct-to-the-consumer, zero-cost, bilingual, and actionable content. We identified 3 main organizing categories and 10 subcategories as the key elements of the HLP: (1) Why: Educate and Empower and Bring Zero Cost to Consumer Information to the General Public; (2) What: Tools and Protocols; Underlying Mechanisms; and Grounded in Science; (3) How: Linear and Iterative Knowledge Building Process; Lecture-Style Sessions; Interactive and Consumer Informed; Easily Accessible; and Building the Community. Analysis of viewers' comments found strong consumer support for the key HLP model elements.

CONCLUSIONS

This Active Model of Research Translation offers a way to synthesize scientific evidence and deliver it directly to end users in the form of actionable tools and education. Timely evidence translation using effective consumer engagement and education techniques appears to improve access and confidence related to health information use and reduces challenges to understanding and applying health information received from health providers. Framing complex content in an approachable manner, engaging the target audience, encouraging participation, and ensuring open access to the content meet current recommendations on innovative practices for leveraging social media or other digital platforms for disseminating science and research findings to the general public, and are likely key contributors to HLP impact and potential for success. The model offers a replicable framework for translating and disseminating scientific evidence. Similar active models of research translation can have implications for accessing health information and implementing health strategies for improved outcomes. Areas for further investigation are specific and measurable impacts on health, usability, and relevance of the model for reaching marginalized and high-risk populations.

Invoking and identifying task-oriented interlocutor confusion in human-robot interaction

Successful conversational interaction with a social robot requires not only an assessment of a user's contribution to an interaction, but also awareness of their emotional and attitudinal states as the interaction unfolds. To this end, our research aims to systematically trigger, but then interpret human behaviors to track different states of potential user confusion in interaction so that systems can be primed to adjust their policies in light of users entering confusion states. In this paper, we present a detailed human-robot interaction study to prompt, investigate, and eventually detect confusion states in users. The study itself employs a Wizard-of-Oz (WoZ) style design with a Pepper robot to prompt confusion states for task-oriented dialogues in a well-defined manner. The data collected from 81 participants includes audio and visual data, from both the robot's perspective and the environment, as well as participant survey data. From these data, we evaluated the correlations of induced confusion conditions with multimodal data, including eye gaze estimation, head pose estimation, facial emotion detection, silence duration time, and user speech analysis-including emotion and pitch analysis. Analysis shows significant differences of participants' behaviors in states of confusion based on these signals, as well as a strong correlation between confusion conditions and participants own self-reported confusion scores. The paper establishes strong correlations between confusion levels and these observable features, and lays the ground or a more complete social and affect oriented strategy for task-oriented human-robot interaction. The contributions of this paper include the methodology applied, dataset, and our systematic analysis.

User Engagement Clusters of an 8-Week Digital Mental Health Intervention Guided by a Relational Agent (Woebot): Exploratory Study

BACKGROUND

With the proliferation of digital mental health interventions (DMHIs) guided by relational agents, little is known about the behavioral, cognitive, and affective engagement components associated with symptom improvement over time. Obtaining a better understanding could lend clues about recommended use for particular subgroups of the population, the potency of different intervention components, and the mechanisms underlying the intervention's success.

OBJECTIVE

This exploratory study applied clustering techniques to a range of engagement indicators, which were mapped to the intervention's active components and the connect, attend, participate, and enact (CAPE) model, to examine the prevalence and characterization of each identified cluster among users of a relational agent-guided DMHI.

METHODS

We invited adults aged 18 years or older who were interested in using digital support to help with mood management or stress reduction through social media to participate in an 8-week DMHI guided by a natural language processing-supported relational agent, Woebot. Users completed assessments of affective and cognitive engagement, working alliance as measured by goal and task working alliance subscale scores, and enactment (ie, application of therapeutic recommendations in real-world settings). The app passively collected data on behavioral engagement (ie, utilization). We applied agglomerative hierarchical clustering analysis to the engagement indicators to identify the number of clusters that provided the best fit to the data collected, characterized the clusters, and then examined associations with baseline demographic and clinical characteristics as well as mental health outcomes at week 8.

RESULTS

Exploratory analyses (n=202) supported 3 clusters: (1) "typical utilizers" (n=81, 40%), who had intermediate levels of behavioral engagement; (2) "early utilizers" (n=58, 29%), who had the nominally highest levels of behavioral engagement in week 1; and (3) "efficient engagers" (n=63, 31%), who had significantly higher levels of affective and cognitive engagement but the lowest level of behavioral engagement. With respect to mental health baseline and outcome measures, efficient engagers had significantly higher levels of baseline resilience (P<.001) and greater declines in depressive symptoms (P=.01) and stress (P=.01) from baseline to week 8 compared to typical utilizers. Significant differences across clusters were found by age, gender identity, race and ethnicity, sexual orientation, education, and insurance coverage. The main analytic findings remained robust in sensitivity analyses.

CONCLUSIONS

There were 3 distinct engagement clusters found, each with distinct baseline demographic and clinical traits and mental health outcomes. Additional research is needed to inform fine-grained recommendations regarding optimal engagement and to determine the best sequence of particular intervention components with known potency. The findings represent an important first step in disentangling the complex interplay between different affective, cognitive, and behavioral engagement indicators and outcomes associated with use of a DMHI incorporating a natural language processing-supported relational agent.

TRIAL REGISTRATION

ClinicalTrials.gov NCT05672745; https://classic.clinicaltrials.gov/ct2/show/NCT05672745.

Examining Engagement and Usability in an Online Discussion Platform for Older Adults: Findings From Pilot Studies

Social media may facilitate older adults' ability to engage socially and explore health information, but it can present difficulties for older adults. Therefore, it is important to explore older adults' experience of usability and user engagement. We conducted two rounds of pilot studies where we used Facebook to engage older adults. We performed a mixed-methods evaluation of user engagement and usability. A directed content analysis of qualitative data from the pilot studies was used to explore engagement and perceived usability, and the Mann-Whitney U test was used to examine differences in feature usage and engagement. We analyzed qualitative data from 13 participants. Qualitative data analysis yielded themes pertaining to three main domains: user engagement , usability , and usability related to aging-related changes . In terms of user engagement and usability, participants in both pilot studies reported positive feedback on felt involvement and endurability, and the second pilot group reported more positive comments regarding perceived usefulness compared with the first pilot group. There was no statistically significant difference in usage over the two studies. The findings of this study suggest opportunities to improve older adults' experience of online discussion platforms. Considering changes that improve perceived aesthetic appeal and focused attention will be helpful.

Seeing eye to eye: trustworthy embodiment for task-based conversational agents

Smart speakers and conversational agents have been accepted into our homes for a number of tasks such as playing music, interfacing with the internet of things, and more recently, general chit-chat. However, they have been less readily accepted in our workplaces. This may be due to data privacy and security concerns that exist with commercially available smart speakers. However, one of the reasons for this may be that a smart speaker is simply too abstract and does not portray the social cues associated with a trustworthy work colleague. Here, we present an in-depth mixed method study, in which we investigate this question of embodiment in a serious task-based work scenario of a first responder team. We explore the concepts of trust, engagement, cognitive load, and human performance using a humanoid head style robot, a commercially available smart speaker, and a specially developed dialogue manager. Studying the effect of embodiment on trust, being a highly subjective and multi-faceted phenomena, is clearly challenging, and our results indicate that potentially, the robot, with its anthropomorphic facial features, expressions, and eye gaze, was trusted more than the smart speaker. In addition, we found that embodying a conversational agent helped increase task engagement and performance compared to the smart speaker. This study indicates that embodiment could potentially be useful for transitioning conversational agents into the workplace, and further in situ, "in the wild" experiments with domain workers could be conducted to confirm this.

Developing A Mobile App With a Human-Centered Design Lens to Improve Access to Mental Health Care (Mentallys Project): Protocol for an Initial Co-Design Process

BACKGROUND

Co-design is one of the human-centered design approaches that allows end users to significantly and positively impact the design of mental health technologies. It is a promising approach to foster user acceptance and engagement in digital mental health solutions. Surprisingly, there is a lack of understanding of what co-design is in this field. In this paper, co-design is approached as a cocreation process involving persons with a lived experience of mental health problems, health professionals, and design experts who lead and facilitate the overall creative process.

OBJECTIVE

This paper describes an initial co-design research protocol for the development of a mobile app that aims to improve access to mental health care. It highlights the characteristics of a co-design approach in e-mental health rooted in human-centered design and led by design experts alongside health experts. The paper focuses on the first steps (phase 1) of the co-design process of the ongoing Mentallys project.

METHODS

This Mentallys project will be located in Montréal (Quebec, Canada). The method approach will be based on the "method stories," depicting the "making of" this project and reflecting adjustments needed to the protocol throughout the project in specific situations. Phase 1 of the process will focus on the desirability of the app. Targeted participants will include people with a lived experience of mental health problems, peer support workers and clinicians, and 3 facilitators (all design experts or researchers). Web-based sessions will be organized because of the COVID-19 pandemic, using Miro (RealtimeBoard Inc) and Zoom (Zoom Video Communications, Inc). Data collection will be based on the comments, thoughts, and new ideas of participants around the imaginary prototypes. Thematic analysis will be carried out after each session to inform a new version of the prototype.

RESULTS

We conducted 2 stages in phase 1 of the process. During stage 1, we explored ideas through group co-design workshops (divergent thinking). Six co-design workshops were held: 2 with only clinicians (n=7), 2 with peer support workers (n=5) and people with a lived experience of mental health problems (n=2), and 2 with all of them (n=14). A total of 6 facilitators participated in conducting activities in subgroups. During stage 2, ideas were refined through 10 dyad co-design sessions (convergent thinking). Stage 2 involved 3 participants (n=3) and 1 facilitator. Thematic analysis was performed after stage 1, while analytic questioning is being performed for stage 2. Both stages allowed several iterations of the prototypes.

CONCLUSIONS

The design of the co-design process, the leadership of the design expertise throughout the process, and the different forms of co-design activities are key elements in this project. We highly recommend that health researchers partner with professional designers or design researchers who are familiar with co-design.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/47220.

Users' Concerns About Endometriosis on Social Media: Sentiment Analysis and Topic Modeling Study

BACKGROUND

Endometriosis is a debilitating and difficult-to-diagnose gynecological disease. Owing to limited information and awareness, women often rely on social media platforms as a support system to engage in discussions regarding their disease-related concerns.

OBJECTIVE

This study aimed to apply computational techniques to social media posts to identify discussion topics about endometriosis and to identify themes that require more attention from health care professionals and researchers. We also aimed to explore whether, amid the challenging nature of the disease, there are themes within the endometriosis community that gather posts with positive sentiments.

METHODS

We retrospectively extracted posts from the subreddits r/Endo and r/endometriosis from January 2011 to April 2022. We analyzed 45,693 Reddit posts using sentiment analysis and topic modeling-based methods in machine learning.

RESULTS

Since 2011, the number of posts and comments has increased steadily. The posts were categorized into 11 categories, and the highest number of posts were related to either asking for information (Question); sharing the experiences (Rant/Vent); or diagnosing and treating endometriosis, especially surgery (Surgery related). Sentiment analysis revealed that 92.09% (42,077/45,693) of posts were associated with negative sentiments, only 2.3% (1053/45,693) expressed positive feelings, and there were no categories with more positive than negative posts. Topic modeling revealed 27 major topics, and the most popular topics were Surgery, Questions/Advice, Diagnosis, and Pain. The Survey/Research topic, which brought together most research-related posts, was the last in terms of posts.

CONCLUSIONS

Our study shows that posts on social media platforms can provide insights into the concerns of women with endometriosis symptoms. The analysis of the posts confirmed that women with endometriosis have to face negative emotions and pain daily. The large number of posts related to asking questions shows that women do not receive sufficient information from physicians and need community support to cope with the disease. Health care professionals should pay more attention to the symptoms and diagnosis of endometriosis, discuss these topics with patients to reduce their dissatisfaction with doctors, and contribute more to the overall well-being of women with endometriosis. Researchers should also become more involved in social media and share new science-based knowledge regarding endometriosis.

Data-to-music sonification and user engagement

The process of transforming data into sounds for auditory display provides unique user experiences and new perspectives for analyzing and interpreting data. A research study for data transformation to sounds based on musical elements, called data-to-music sonification, reveals how musical characteristics can serve analytical purposes with enhanced user engagement. An existing user engagement scale has been applied to measure engagement levels in three conditions within melodic, rhythmic, and chordal contexts. This article reports findings from a user engagement study with musical traits and states the benefits and challenges of using musical characteristics in sonifications. The results can guide the design of future sonifications of multivariable data.

Workload, Usability, and Engagement with a Mobile App Supporting Video Observation of Methadone Take-Home Dosing: Usability Study

BACKGROUND

Methadone, a cornerstone of opioid use disorder treatments for many decades, is an essential tool for combatting the opioid epidemic. However, requirements for observing methadone dosing in person through direct observed therapy (DOT) impose significant barriers for many patients. Digital technology can facilitate remote DOT, which could reduce barriers to methadone treatment. Currently, there are limited data on the usability of such technology among patients and counselors in methadone treatment settings.

OBJECTIVE

The primary objective of this study was to assess the workload, usability, and engagement of a video-based DOT mobile app for patients with opioid use disorder receiving methadone treatment. The secondary objective was to assess the workload, usability, and engagement of the provider-facing app portal used by counselors.

METHODS

Patients (n=12) and counselors (n=3) who previously tried video DOT for methadone through a smartphone app in an opioid treatment program participated in usability testing sessions. Participants completed essential tasks for video DOT, then provided ratings of workload (NASA Task Load Index), usability (modified System Usability Scale), and engagement (modified Engagement Scale) with the core features of the video DOT program.

RESULTS

Patients and counselors reported low mental, physical, and temporal demands, successful performance, low effort, and low frustration associated with activities. Patients reported high usability (mean 85, SD 9.5) and engagement (mean 3.8, SD 1.1); counselors reported moderate usability (mean 43.3, SD 17.7) and engagement (mean 2.81, SD 0.63).

CONCLUSIONS

A mobile health app that facilitates video-based DOT for methadone required a low workload for patients and counselors and was highly usable for patients in an opioid treatment program; however, there are opportunities to improve usability and engagement for the counselor-facing portal.

Human-to-Computer Interactivity Features Incorporated Into Behavioral Health mHealth Apps: Systematic Search

BACKGROUND

While there are thousands of behavioral health apps available to consumers, users often quickly discontinue their use, which limits their therapeutic value. By varying the types and number of ways that users can interact with behavioral health mobile health apps, developers may be able to support greater therapeutic engagement and increase app stickiness.

OBJECTIVE

The main objective of this analysis was to systematically characterize the types of user interactions that are available in behavioral health apps and then examine if greater interactivity was associated with greater user satisfaction, as measured by app metrics.

METHODS

Using a modified PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analysis) methodology, we searched several different app clearinghouse websites and identified 76 behavioral health apps that included some type of interactivity. We then filtered the results to ensure we were examining behavioral health apps and further refined our search to include apps that identified one or more of the following terms: peer or therapist forum, discussion, feedback, professional, licensed, buddy, friend, artificial intelligence, chatbot, counselor, therapist, provider, mentor, bot, coach, message, comment, chat room, community, games, care team, connect, share, and support in the app descriptions. In the final group of 34 apps, we examined the presence of 6 types of human-machine interactivities: human-to-human with peers, human-to-human with providers, human-to-artificial intelligence, human-to-algorithms, human-to-data, and novel interactive smartphone modalities. We also downloaded information on app user ratings and visibility, as well as reviewed other key app features.

RESULTS

We found that on average, the 34 apps reviewed included 2.53 (SD 1.05; range 1-5) features of interactivity. The most common types of interactivities were human-to-data (n=34, 100%), followed by human-to-algorithm (n=15, 44.2%). The least common type of interactivity was human-artificial intelligence (n=7, 20.5%). There were no significant associations between the total number of app interactivity features and user ratings or app visibility. We found that a full range of therapeutic interactivity features were not used in behavioral health apps.

CONCLUSIONS

Ideally, app developers would do well to include more interactivity features in behavioral health apps in order to fully use the capabilities of smartphone technologies and increase app stickiness. Theoretically, increased user engagement would occur by using multiple types of user interactivity, thereby maximizing the benefits that a person would receive when using a mobile health app.

Avatar and virtual agent-assisted telecare for patients in their homes: A scoping review

INTRODUCTION

Telecare can be an effective way to deliver healthcare to patients' homes. Avatar or virtual agent-equipped technologies have the potential to increase user engagement and adherence to telecare. This study aimed to identify telecare interventions assisted by avatars/virtual agents, reflect the concepts of telecare and give an overview on its outcomes.

METHODS

A scoping review guided by the PRISMA-ScR checklist was conducted. MEDLINE, CINAHL, PsycINFO and grey literature were searched through 12 July 2022. Studies were included if patients were remotely cared for by healthcare professionals and received telecare interventions assisted by avatars/virtual agents in their homes. Studies underwent quality appraisal, and were synthesized along the dimensions 'study characteristics', 'intervention' and 'outcomes'.

RESULTS

Out of 535 records screened, 14 studies were included, reporting the effects of avatar/virtual agent-assisted telecare interventions, tailored to specific patient groups. Telecare interventions mainly focused on teletherapy and telemonitoring. Telecare services were rehabilitative, preventive, palliative, promotive and curative. Modes of communication were asynchronous, synchronous or a mix of both. Tasks of the implemented avatars/virtual agents comprised delivering health interventions, monitoring, assessment, guidance and strengthening agency. Telecare interventions led to improved clinical outcomes and higher adherence. Most studies reported sufficient system usability and high satisfaction among participants.

CONCLUSIONS

Telecare interventions were overall target group related and integrated in a service model. This combined with the use of avatars and virtual agents leads to improved adherence to telecare in the home setting. Further studies could account for relatives' experiences with telecare.

Minimal Effect of Messaging on Engagement in a Digital Anxiety Intervention

This study evaluated the effectiveness of different recruitment messages for encouraging enrollment in a digital mental health intervention (DMHI) for anxiety among 1,600 anxious patients in a large healthcare system. Patients were randomly assigned to receive a standard message, or one of five messages designed to encourage enrollment: Three messages offered varying financial incentives, one message offered coaching, and one message provided consumer testimonials. Patients could then click a link in the message to visit the DMHI website, enroll, and start the first session. We examined the effects of message features and message length (short vs. long) on rates of site clicks, enrollment, and starting the first session. We also tested whether demographic and clinical factors derived from patients' electronic health records were associated with rates of enrollment and starting the first session to understand the characteristics of patients most likely to use DMHIs in this setting. Across messages, 19.4% of patients clicked a link to visit the DMHI website, but none of the messages were significantly associated with rates of site clicks, enrollment, or starting the first session. Females (vs. males) had a greater probability of enrollment. No other demographic or clinical variables were significantly associated with enrollment or starting the first session. Findings provide guidance for resource allocation decisions in larger scale DMHI implementations in healthcare settings.

Situating Meditation Apps Within the Ecosystem of Meditation Practice: Population-Based Survey Study

BACKGROUND

Meditation apps have the potential to increase access to evidence-based strategies to promote mental health. However, it is currently unclear how meditation apps are situated within the broader landscape of meditation practice and what factors may influence engagement with them.

OBJECTIVE

This study aimed to clarify the prevalence and correlates of meditation app use in a population-based sample of individuals with lifetime exposure to meditation in the United States. In addition, we sought to identify the concerns and desired features of meditation apps among those with lifetime exposure to meditation.

METHODS

A total of 953 participants completed an initial screening survey. Of these 953 participants, 434 (45.5%) reported lifetime exposure to meditation and completed a follow-up survey (434/470, 92.3% response rate) assessing their meditation app use, anxiety, depression, loneliness, initial motivation for meditation, and concerns about and desired features of meditation apps.

RESULTS

Almost half (434/953, 45.5%) of the participants who completed the screening survey reported lifetime exposure to meditation. Among those with lifetime exposure to meditation (ie, meditators), more than half (255/434, 58.8%) had used meditation apps at least once in their lives, and 21.7% (94/434) used meditation apps weekly or daily (ie, active users). Younger age, higher anxiety, and a mental health motivation for practicing meditation were associated with lifetime exposure to meditation apps. Among meditators, those with lifetime exposure to meditation apps were more likely to report concerns about apps, including concerns regarding the cost and effectiveness of apps, time required for use, technical issues with apps, and app user-friendliness. Meditators who used meditation apps weekly or daily (ie, active users) were younger, less likely to be men and non-Latinx White individuals and have lower income, and more likely to have an initial spiritual motivation for meditation. Active users reported more concerns regarding usability and technical problems and were less likely to report disinterest in apps. Headspace and Calm were the most frequently used apps. Tips and reminders for practice, encouragement of "mini" practices, and mental health content were the most desired features. Participants were less interested in social features (eg, the ability to communicate with other users or teachers).

CONCLUSIONS

Meditation apps are commonly used by meditators in the United States, with a higher use among certain demographic groups. Future studies may increase user engagement in meditation apps by addressing concerns (eg, cost and effectiveness) and incorporating desired features (eg, tips and reminders for practice).

An Evidence-Based Serious Game App for Public Education on Antibiotic Use and Antimicrobial Resistance: Protocol of a Randomized Controlled Trial

BACKGROUND

The misuse and overuse of antibiotics contribute to the acceleration of antimicrobial resistance (AMR), but public knowledge on appropriate antibiotic use and AMR remained low despite ongoing health promotion efforts. App gamification has gained traction in recent years for health promotion and to affect change in health behaviors. Hence, we developed an evidence-based serious game app "SteWARdS Antibiotic Defence" to educate the public on appropriate antibiotic use and AMR and address knowledge gaps.

OBJECTIVE

We aim to evaluate the effectiveness of the "SteWARdS Antibiotic Defence" app in improving the knowledge of, attitude toward, and perception (KAP) of appropriate antibiotic use and AMR among the public. The primary objective is to assess the changes in KAP of antibiotic use and AMR in our participants, while the secondary objectives are to assess the extent of user engagement with the app and the level of user satisfaction in using the app.

METHODS

Our study is a parallel 2-armed randomized controlled trial with a 1:1 allocation. We plan to recruit 400 participants (patients or their caregivers) aged 18-65 years from government-funded primary care clinics in Singapore. Participants are randomized in blocks of 4 and into the intervention or control group. Participants in the intervention group are required to download the "SteWARdS Antibiotic Defence" app on their smartphones and complete a game quest within 2 weeks. Users will learn about appropriate antibiotic use and effective methods to recover from uncomplicated upper respiratory tract infections by interacting with the nonplayer characters and playing 3 minigames in the app. The control group will not receive any intervention.

RESULTS

The primary study outcome is the change in participants' KAP toward antibiotic use and AMR 6-10 weeks post intervention or 6-10 weeks from baseline for the control group (web-based survey). We will also assess the knowledge level of participants immediately after the participant completes the game quest (in the app). The secondary study outcomes are the user engagement level (tracked by the app) and satisfaction level of playing the game (via the immediate postgame survey). The satisfaction survey will also collect participants' feedback on the game app.

CONCLUSIONS

Our proposed study provides a unique opportunity to assess the effectiveness of a serious game app in public health education. We anticipate possible ceiling effects and selection bias in our study and have planned to perform subgroup analyses to adjust for confounding factors. The app intervention will benefit a larger population if it is proven to be effective and acceptable to users.

TRIAL REGISTRATION

ClinicalTrials.gov NCT05445414; https://clinicaltrials.gov/ct2/show/NCT05445414.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/45833.

COVID-19 Response Resource Engagement and User Characteristics of the Wichealth Web-Based Nutrition Education System: Comparative Cross-sectional Study

BACKGROUND

In response to the COVID-19 pandemic, Wichealth launched 4 information resources on the site's user landing dashboard page. These resources were used consistently during the period in which they were available (April 1, 2020, through October 31, 2021); however, only 9% (n=50,888) of Wichealth users eligible for inclusion in the study accessed at least one resource. User engagement with emergency response resources within the context of a web-based health educational tool has not been well investigated due to a paucity of opportunities and a lack of the ability to evaluate relevant users at scale.

OBJECTIVE

This investigation was carried out to understand if user characteristics and behaviors measured by the Wichealth web-based education system are associated with a participant's motivation, or lack thereof, to engage with the added COVID-19 resources.

METHODS

Sociodemographic characteristics were gathered from Wichealth users with at least one lesson completed and a complete user profile to identify which factors increase the likelihood of user access of any of the Wichealth COVID-19 response resources during the 19-month period between April 1, 2020, and October 31, 2021. A logistic regression analysis was conducted to determine the relative importance of all factors on the likelihood of a user accessing the COVID-19 resources.

RESULTS

A total of 50,888 unique Wichealth users included in the study accessed the COVID-19 response resources 66,849 times during the time period. During the same period, 510,939 unique Wichealth users completed at least one lesson about how to engage in healthy behaviors with respect to parent-child feeding but did not access any COVID-19 resources. Therefore, only 9% of Wichealth users who completed a lesson during the time when COVID-19 response resources were available accessed any of the information in those resources. Users of the Spanish language Wichealth version, older users, those less educated, and users with prior Wichealth lesson engagement demonstrated the greatest likelihood of COVID-19 resource use.

CONCLUSIONS

This investigation presents findings that demonstrate significant differences between Wichealth users that opted to access COVID-19-specific resources and those who chose not to during their web-based educational session. Reaching users of a web-based educational system with supplemental information may require multiple strategies to increase coverage and ensure the widest possible distribution.

Engagement With Gamification Elements in a Smoking Cessation App and Short-term Smoking Abstinence: Quantitative Assessment

BACKGROUND

Gamification in smoking cessation apps has been found to improve cognitive outcomes associated with higher odds of quitting. Although some research has shown that gamification can also positively impact behavioral outcomes such as smoking cessation, studies have largely focused on physical activity and mental health. Only a few studies have explored the effects of gamification on smoking cessation outcomes, of which the majority have adopted qualitative methodologies and/or assessed engagement with apps using self-report.

OBJECTIVE

This study aimed to explore levels of user engagement with gamification features in a smoking cessation app via in-app metrics. Specifically, the objective of this paper was to investigate whether higher engagement with gamification features is associated with the likelihood of quitting in the short term.

METHODS

Data from a larger online study that recruited smokers seeking to quit were analyzed to address the objectives presented in this paper. The study took place between June 2019 and July 2020, and participants were primarily recruited via social media posts. Participants who met the eligibility criteria used 1 of 2 mobile apps for smoking cessation. In-app metrics shared by the developer of one of the smoking cessation apps, called Kwit, were used to assess engagement with gamification features. Out of 58 participants who used the Kwit app, 14 were excluded due to missing data or low engagement with the app (ie, not opening the app once a week). For the remaining 44 participants, mean (SD) values were calculated for engagement with the app using in-app metrics. A logistic regression model was used to investigate the association between engagement with gamification and 7-day smoking abstinence.

RESULTS

In total, data from 44 participants who used the Kwit app were analyzed. The majority of participants were male, married, and employed. Almost 30% (n=13) of participants self-reported successful 7-day abstinence at the end of the study. On average, the Kwit app was opened almost 31 (SD 39) times during the 4-week study period, with the diary feature used the most often (mean 22.8, SD 49.3). Moreover, it was found that each additional level unlocked was associated with approximately 22% higher odds of achieving 7-day abstinence after controlling for other factors such as age and gender (odds ratio 1.22, 95% CI 1.01-1.47).

CONCLUSIONS

This study highlights the likely positive effects of certain gamification elements such as levels and achievements on short-term smoking abstinence. Although more robust research with a larger sample size is needed, this research highlights the important role that gamification features integrated into mobile apps can play in facilitating and supporting health behavior change.

Evaluating a Digital Mental Health Intervention (Wysa) for Workers' Compensation Claimants: Pilot Feasibility Study

OBJECTIVE

This study examines the feasibility and acceptability of an AI-led digital mental health intervention in a Workers' Compensation (WC) program, Wysa for Return to Work.

METHODS

Self-reported demographic data and responses to psychosocial screening questions were analyzed alongside participants' app usage through which four key outcomes were measured: recruitment rate, onboarding rate, retention, and engagement.

RESULTS

The data demonstrated a high need for psychosocial interventions among injured workers, especially women, young adults, and those with high severity injuries. Those with more psychosocial risk factors had a higher rate of onboarding, retention, and engagement, and those with severe injuries had higher retention.

CONCLUSIONS

Our study concluded that Wysa for Return to Work, the AI-led digital mental health intervention that delivers a recovery program using a digital conversational agent, is feasible and acceptable for a return-to-work population.

DECENT: A sociotechnical approach for developing mobile health apps in underserved settings

Objective

Despite the fact that user engagement is critical to the efficacy of mobile health (mHealth) interventions in the Global South, many of these interventions lack user engagement features. This is because sociotechnical aspects of such initiatives are frequently ignored during the design, development, and implementation stages. This research highlighted the importance of considering sociotechnical factors when developing mHealth apps. The intended users for the mHealth technologies in this study are care professionals.

Materials and Methods

Five semi-structured interviews and a pilot interview were conducted to identify user engagement facilitators and barriers. The interview data were analysed using NVivo. The Capability, Opportunity, Motivation - Behaviour (COM-B) model is then used to map the facilitators and barriers to mHealth app engagement, allowing researchers to better understand how users engage/disengage with mHealth apps.

Results and Discussion

Capability facilitators included features that assist users in learning more about the app (e.g. a user manual and statistical data) as well as features that assist users in developing a routine. The lack of app skills and cognitive overload limit capability. While social connectedness and offline functionality were identified as facilitators of user engagement, non-user-friendly design and cultural dimensions were identified as barriers. Early user engagement and rewards were identified as motivational facilitators that influence user engagement. Furthermore, perceived non-utility and a lack of encouragement were identified as motivational barriers to engagement.

Conclusion

Several factors were discovered across all COM-B model components that could be used to develop more engaging mHealth apps. Adopting a techno-centric approach that ignores sociotechnical factors can reduce user engagement. The design process engagement enhancement system (DECENT) framework was proposed based on the findings.

Enhancing equitable engagement for digital health promotion: Lessons from evaluating a childrearing app in Indonesia

Part of the appeal of digital health interventions, including mHealth, is the potential for greater reach in places where conventional health promotion is hampered by geographical, financial or social barriers. Yet, 'engagement' - typically understood as user experience and interactions with technology - remains a persistent challenge, particularly in places where technology access or familiarity with technology is limited. We undertook an evaluation of a childrearing app to promote socioemotional and cognitive development in early childhood across the world. In this article, we present findings from qualitative research on app rollout in Indonesia, the first of numerous low- and middle-income countries targeted by the app. We draw on systems theory and complexity thinking to broaden the lens of 'engagement' beyond individual users to encompass collective systems (families and communities), exploring how the intervention was harnessed to meet local contextual needs. The qualitative research involved semi-structured interviews, workshops and audio diaries with 57 diverse stakeholders, including Indonesian parents, caregivers, and collaborators involved in funding, development, and dissemination of the app. We observed the importance of social connection, sense-making, and interactive learning for enhancing engagement with the app and its messages. Enthusiastic users, strongly linked across community networks (e.g. kindergarten teachers), improvised dissemination strategies to facilitate uptake. Interactive learning that tapped into familiar social structures (e.g. intergenerational hierarchies) was crucial for engagement. Understanding ways the app failed to tap into structures of social connection served to highlight the need to embed strategies to support collective engagement.

Knowledge platform affordances and knowledge collaboration performance: The mediating effect of user engagement

Knowledge collaboration is the result of knowledge transfer and social interaction among users on knowledge platforms, and its formation mechanism has attracted much attention. Based on the affordance theory, this paper introduces user engagement as a mediating variable to study the relationship between knowledge platform affordances and knowledge collaboration performance. Data collected from 361 valid questionnaires from the Zhihu platform were analyzed by using SPSS 26.0 and Amos 24.0. The results show that knowledge platform affordances have a direct and positive influence on knowledge collaboration performance as well as an indirect influence through user engagement. Expressly, user engagement undertakes three intermediary paths between knowledge platform affordances and knowledge collaboration performance: knowledge affordances-conscious participation-knowledge collaboration performance, knowledge affordances-enthusiasm-knowledge collaboration performance, and social affordances-social interaction-knowledge collaboration performance. This paper explores the formation process of knowledge collaboration performance by synthesizing affordance and user engagement theories. It clarifies the fundamental role of knowledge affordances in stimulating users' conscious participation and enthusiasm, and the critical role of social affordances in stimulating social interaction. Therefore, this paper further enriches theories of knowledge collaboration and knowledge platform affordances and provides a practical reference for the strategic optimization and development of knowledge platforms.

Exploring the effect of use contexts on user engagement toward tourism short video platforms

With tourism short video platforms' increasingly fierce competition, retaining existing users and improving engagement has taken on greater theoretical and practical significance. Based on the self-system model of motivational development, this study, involving an empirical analysis of 252 user data samples, establishes a research model to determine how the use contexts affect users' psychological process and finally lead to behavioral engagement. In particular, four use contexts of tourism short video platforms are proposed-namely, information acquisition, leisure and entertainment, attention obtainment, and social interaction. Different use contexts differ in the degree to which they satisfy users' three psychological needs, thus influence their attitudinal engagement and behavioral engagement. The research results can provide theoretical and practical references on how to improve user engagement toward tourism short video platforms.

Long-term usage of a commercial mHealth app: A "multiple-lives" perspective

Background

Emerging evidence suggests that individuals use mHealth apps in multiple disjointed ways in the real-world-individuals, for example, may engage, take breaks, and re-engage with these apps. To our knowledge, very few studies have adopted this 'multiple-live' perspective to analyze long-term usage of a physical activity (PA) app. This study aimed to examine the duration of use, as well as the frequency, length, and timing of streaks (uninterrupted periods of use) and breaks (uninterrupted periods of non-use) within a popular commercial PA app called Carrot Rewards over 12 months. We also examined sociodemographic correlates of usage.

Method

This retrospective observational study analyzed data from 41,207 Carrot Rewards users participating in the "Steps" walking program from June/July 2016 to June/July 2017. We measured four usage indicators: duration of use, frequency and length of streaks and breaks, time to first break, and time to resume second streak. We also extracted information regarding participants' age, gender, province, and proxy indicators of socioeconomic status derived from census data. We used descriptive statistics to summarize usage patterns, Kaplan-Meier curves to illustrate the time to first break and time to resume second streak. We used linear regressions and Cox Proportional Hazard regression models to examine sociodemographic correlates of usage.

Results

Over 60% of the participants used Carrot Rewards for ≥6 months and 29% used it for 12 months (mean = 32.59 ± 18.435 weeks). The frequency of streaks and breaks ranged from 1 to 9 (mean = 1.61 ± 1.04 times). The mean streak and break length were 20.22 ± 18.26 and 16.14 ± 15.74 weeks, respectively. The median time to first break was 18 weeks across gender groups and provinces; the median time for participants to resume the second streak was between 12 and 32 weeks. Being female, older, and living in a community with greater post-secondary education levels were associated with increased usage.

Conclusion

This study provides empirical evidence that long-term mHealth app usage is possible. In this context, it was common for users to take breaks and re-engage with Carrot Rewards. When designing and evaluating PA apps, therefore, interventionists should consider the 'multiple-lives' perspective described here, as well as the impact of gender and age.

Exploring the Black Box of an mHealth Intervention (LIFE4YOUth): A Qualitative Process and Outcome Evaluation of End-User Engagement

The effectiveness of mHealth interventions rely on whether the content successfully activate mechanisms necessary for behavior change. These mechanisms may be affected by end-users' experience of the intervention content. The aim of this study was to explore how the content of a novel mHealth intervention (LIFE4YOUth) was understood, interpreted, and applied by high school students, and the consequences of engaging with the content. Qualitative content analysis was used inductively and deductively to analyze interview data (n = 16) based on think-aloud techniques with Swedish high school students aged 16-19 years. Theoretical constructs from social cognitive theory framed the deductive analysis. The analysis resulted in four categories which describe central activities of intervention engagement among end-users: defining, considering, centralizing, and personalizing. End-users engaged in these activities to different degrees as illustrated by four typologies: Literal, Vague, Rigid, and Creative engagement. Most informants knew about the risks and benefits of health behaviors, but engagement with intervention content generally increased informants' awareness. In conclusion, this study provides in-depth knowledge on the cognitive process when engaging with mHealth content and suggests that deliberate and flexible engagement most likely deepens end-users' understanding of why and how health behavior change can be managed.

Platform Effects on Public Health Communication: A Comparative and National Study of Message Design and Audience Engagement Across Twitter and Facebook

BACKGROUND

Public health agencies widely adopt social media for health and risk communication. Moreover, different platforms have different affordances, which may impact the quality and nature of the messaging and how the public engages with the content. However, these platform effects are not often compared in studies of health and risk communication and not previously for the COVID-19 pandemic.

OBJECTIVE

This study measures the potential media effects of Twitter and Facebook on public health message design and engagement by comparing message elements and audience engagement in COVID-19-related posts by local, state, and federal public health agencies in the United States during the pandemic, to advance theories of public health messaging on social media and provide recommendations for tailored social media communication strategies.

METHODS

We retrieved all COVID-19-related posts from major US federal agencies related to health and infectious disease, all major state public health agencies, and selected local public health departments on Twitter and Facebook. A total of 100,785 posts related to COVID-19, from 179 different accounts of 96 agencies, were retrieved for the entire year of 2020. We adopted a framework of social media message elements to analyze the posts across Facebook and Twitter. For manual content analysis, we subsampled 1677 posts. We calculated the prevalence of various message elements across the platforms and assessed the statistical significance of differences. We also calculated and assessed the association between message elements with normalized measures of shares and likes for both Facebook and Twitter.

RESULTS

Distributions of message elements were largely similar across both sites. However, political figures (P<.001), experts (P=.01), and nonpolitical personalities (P=.01) were significantly more present on Facebook posts compared to Twitter. Infographics (P<.001), surveillance information (P<.001), and certain multimedia elements (eg, hyperlinks, P<.001) were more prevalent on Twitter. In general, Facebook posts received more (normalized) likes (0.19%) and (normalized) shares (0.22%) compared to Twitter likes (0.08%) and shares (0.05%). Elements with greater engagement on Facebook included expressives and collectives, whereas posts related to policy were more engaged with on Twitter. Science information (eg, scientific explanations) comprised 8.5% (73/851) of Facebook and 9.4% (78/826) of Twitter posts. Correctives of misinformation only appeared in 1.2% (11/851) of Facebook and 1.4% (12/826) of Twitter posts.

CONCLUSIONS

In general, we find a data and policy orientation for Twitter messages and users and a local and personal orientation for Facebook, although also many similarities across platforms. Message elements that impact engagement are similar across platforms but with some notable distinctions. This study provides novel evidence for differences in COVID-19 public health messaging across social media sites, advancing knowledge of public health communication on social media and recommendations for health and risk communication strategies on these online platforms.

Understanding Engagement With Forensic Smartphone Apps: The Service Design Engagement Model

Justice services have begun to integrate the use of mobile applications into treatment, support, and rehabilitative programs for forensic clients. One such application that been adopted to support forensic clients is "eRecovery": a smartphone application that provides clients recovering from a substance addiction with support for managing relapse. In this article, we report on evaluation findings from a trial of eRecovery in an Australian Community Justice Centre, and reflect on several issues relating to fostering and sustaining client engagement with similar applications within forensic and justice settings. We propose the Service Design Engagement Model to organize, visualize, and describe the stages and factors important to adoption, appropriation, and on-going routine use of the software by forensic clients. The model recognizes the role of contextual and environmental factors in supporting users through the early stages of engagement, and the importance of user agency in longer-term engagement with therapeutic apps.

Accessing the Impact Mechanism of Sense of Virtual Community on User Engagement

Although research has begun to explore the influence patterns of sense of virtual community, there is limited research on how sense of virtual community affects educational virtual community user engagement. Based on the educational virtual community context, this study constructs a theoretical model with moderation and mediation to explore the mediation mechanism of sense of virtual community affecting user engagement and its boundary conditions. In this study, the data collected from 377 users are analyzed by structural equation modeling. The research findings found that not only effective commitment has a mediating role between sense of virtual community and user engagement, but also perceived support has a moderating role in the process of effective commitment's influence on user engagement. This study examines the practical effects of sense of virtual community in the context of educational virtual community use and reveals the mechanism of the effect of sense of virtual community on user engagement.

Clinician collaboration to improve clinical decision support: the Clickbusters initiative

OBJECTIVE

We describe the Clickbusters initiative implemented at Vanderbilt University Medical Center (VUMC), which was designed to improve safety and quality and reduce burnout through the optimization of clinical decision support (CDS) alerts.

MATERIALS AND METHODS

We developed a 10-step Clickbusting process and implemented a program that included a curriculum, CDS alert inventory, oversight process, and gamification. We carried out two 3-month rounds of the Clickbusters program at VUMC. We completed descriptive analyses of the changes made to alerts during the process, and of alert firing rates before and after the program.

RESULTS

Prior to Clickbusters, VUMC had 419 CDS alerts in production, with 488 425 firings (42 982 interruptive) each week. After 2 rounds, the Clickbusters program resulted in detailed, comprehensive reviews of 84 CDS alerts and reduced the number of weekly alert firings by more than 70 000 (15.43%). In addition to the direct improvements in CDS, the initiative also increased user engagement and involvement in CDS.

CONCLUSIONS

At VUMC, the Clickbusters program was successful in optimizing CDS alerts by reducing alert firings and resulting clicks. The program also involved more users in the process of evaluating and improving CDS and helped build a culture of continuous evaluation and improvement of clinical content in the electronic health record.

PEGASO e-Diary: User Engagement and Dietary Behavior Change of a Mobile Food Record for Adolescents

Background

Obesity amongst children and adolescents is becoming a major health problem globally and mobile food records can play a crucial role in promoting healthy dietary habits.

Objective

To describe the methodology for the implementation of the e-Diary mobile food record, to assess its capability in promoting healthy eating habits, to evaluate the factors associated with its usage and engagement.

Methods

This is a descriptive study that compared the characteristics of participants engaged in the e-Diary, which was part of the PEGASO project in which an app to provide proactive health promotion was given to 365 students at 4 European sites enrolled during October to December 2016: England (UK), Scotland (UK), Lombardy (Italy), and Catalonia (Spain). The e-Diary tracked the users' dietary habits in terms of food groups, dietary indexes, and 6 dietary target behaviors relating to consumption of: fruit; vegetable; breakfast; sugar-sweetened beverages; fast-food; and snacks. The e-Diary provided also personalized suggestions for the next meal and gamification.

Results

The e-Diary was used for 6 months by 357 adolescents (53.8% females). The study showed that females used the e-Diary much more than males (aOR 3.8, 95% CI 1.6-8.8). Participants aged 14 years were more engaged in the e-Diary than older age groups (aOR 5.1, 95% CI 1.4-18.8) as were those with a very good/excellent self-perceived health status compared to their peers with fair/poor health perception (aOR 4.2, 95% CI 1.3-13.3). Compared to the intervention sites, those living in Catalonia (aOR 13.2 95% CI 2.5-68.8) were more engaged. In terms of behavior change, a significant positive correlation between fruit (p < 0.0001) and vegetables (p = 0.0087) intake was observed in association with increased engagement in the e-Diary. Similarly, adolescents who used the app for more than 2 weeks had significantly higher odds of not skipping breakfast over the study period (aOR 2.5, 95% CI 1.0-6.3).

Conclusions

The users highly engaged with the e-Diary were associated with improved dietary behaviors: increased consumption of fruit and vegetables and reduced skipping of breakfast. Although the overall usage of the e-Diary was high during the first weeks, it declined thereafter. Future applications should foster user engagement, particularly targeting adolescents at high risk.

Clinical Trial Registration

https://www.clinicaltrials.gov/, identifier: NCT02930148.

Text Messaging Intervention for Mental Wellness in American Indian and Alaska Native Teens and Young Adults (BRAVE Study): Analysis of User Engagement Patterns

Background

Many American Indian and Alaska Native (AI/AN or Native) communities express concern about high rates of suicide and poor mental health. Technology-based health interventions that nurture resilience, coping skills, connectedness, and help-seeking skills may be an effective strategy for promoting health and wellbeing among AI/AN youth. The Northwest Portland Area Indian Health Board designed the BRAVE intervention for AI/AN youth. BRAVE is delivered via SMS text messaging and includes role model videos, mental wellness strategies, links to culturally relevant resources, and social support from family and friends.

Objective

The aim of this study is to explore system data from the BRAVE intervention to determine patterns of user engagement and differences in psychosocial outcomes based on the number of clicks on BRAVE content.

Methods

The BRAVE study included 1030 AI/AN teens and young adults nationwide (15 to 24 years old). The message series in the BRAVE and STEM study arms included 3 to 5 SMS text messages per week, featuring 1 role model video and 1 image per week. Messages were sent out via Mobile Commons (Upland Software Inc), a mobile messaging provider that supports text, picture, and video SMS.

Results

Of the 509 participants in the original BRAVE analysis, 270 had sufficient data to analyze user engagement, with at least 1 trackable click on a study SMS text message. Of the 270, 184 (68.1%) were female, 50 (18.5%) were male, and 36 (13.3%) selected another gender category. The average participant was 20.6 years old, with a minimum and maximum age of 15 and 26 years. Most participants had relatively low engagement measured by the number of clicks (median 2; mean 3.4), although others clicked message content as many as 49 times. Users engaged most frequently with the YouTube-based content (viewing 1 of 7 role model videos), with 64.8% (175/270) of total clicks coming from the role model videos, and earlier episodes receiving the highest number of clicks. Most baseline psychosocial measures were not significantly associated with the number of links clicked. However, help-seeking behavior was highly significant (P<.001), with a rate ratio of 0.82 (0.73, 0.92), indicating that each 1-unit increase in help-seeking score at baseline was associated with an 18% decrease in the expected number of study content clicks.

Conclusions

This is the first study to set initial standards for assessing user engagement in an mHealth intervention. Our work underscores the feasibility of exploring the impact of engagement on intended outcomes, allowing for more precise exploration of the dose-response relationship that may be realized through these low-touch interventions that offer promising potential for reaching high numbers of program participants.

Trial Registration

ClinicalTrials.gov NCT04979481; https://clinicaltrials.gov/ct2/show/NCT04979481

Changes in Use of a Leisure Activity Mobile App for Children With Disabilities During the COVID-19 Pandemic: Retrospective Study

BACKGROUND

Participation in leisure activities is essential for child development and a human right as per the United Nations Convention on the Rights of the Child. Children with disabilities face several restrictions when participating in leisure activities as compared to same age peers without disabilities. Access to information about accessible, inclusive leisure activities is one of the barriers limiting participation, and one potential health promotion strategy is to provide access to information to increase participation. The Jooay App is a mobile app listing such activities in Canada and Australia. With the COVID-19 global pandemic and subsequent public health measures, most community-based facilities providing the activities listed on Jooay were closed. The app therefore started listing online activities offered with the expectation of continuing to provide information for families and understanding the extent to which users relied on the mobile app as a tool to identify new safe leisure opportunities.

OBJECTIVE

This study aims to describe the engagement of the Jooay app before and during COVID-19, and to estimate the extent to which the listing of online activities was related to the engagement of the Jooay app.

METHODS

We conducted a retrospective study comparing Jooay app use between March 2020 and February 2021 to the engagement between March 2019 and February 2020 by Jooay users. Spearman rank correlations were carried out to identify associations between the activities listed and the users' engagement from May 2020 to February 2021.

RESULTS

Active engagement with the Jooay app from March 2020 to February 2021 dropped by an average of 135 engagements (64.2%) compared to engagements in 2019-2020. The largest monthly drop in engagement was observed in May 2020 by 239 engagements (88.8%). There was a strong positive correlation between the number of active users and the number of online activities listed on the app (r_s=0.900).

CONCLUSIONS

The engagement with the Jooay App presented an expected decrease during the first wave of the COVID-19 pandemic. The addition of online adapted leisure activities to the app's listings during the pandemic increased app use. Access to information about inclusive activities is a barrier for children with disabilities to engage in leisure. Mobile health solutions can be responsive to contextual factors and consider the social determinants of health such as socioeconomic and public health emergency issues that can impact the participation of vulnerable populations such as children with disabilities and help eliminate barriers to participation. The provision of online leisure opportunities during the pandemic could facilitate participation in these activities during the pandemic and beyond, which is essential and beneficial for the physical and mental well-being of children with disabilities and their families.

Digitally Supported Lifestyle Intervention to Prevent Type 2 Diabetes Through Healthy Habits: Secondary Analysis of Long-Term User Engagement Trajectories in a Randomized Controlled Trial

BACKGROUND

Digital health interventions may offer a scalable way to prevent type 2 diabetes (T2D) with minimal burden on health care systems by providing early support for healthy behaviors among adults at increased risk for T2D. However, ensuring continued engagement with digital solutions is a challenge impacting the expected effectiveness.

OBJECTIVE

We aimed to investigate the longitudinal usage patterns of a digital healthy habit formation intervention, BitHabit, and the associations with changes in T2D risk factors.

METHODS

This is a secondary analysis of the StopDia (Stop Diabetes) study, an unblinded parallel 1-year randomized controlled trial evaluating the effectiveness of the BitHabit app alone or together with face-to-face group coaching in comparison with routine care in Finland in 2017-2019 among community-dwelling adults (aged 18 to 74 years) at an increased risk of T2D. We used longitudinal data on usage from 1926 participants randomized to the digital intervention arms. Latent class growth models were applied to identify user engagement trajectories with the app during the study. Predictors for trajectory membership were examined with multinomial logistic regression models. Analysis of covariance was used to investigate the association between trajectories and 12-month changes in T2D risk factors.

RESULTS

More than half (1022/1926, 53.1%) of the participants continued to use the app throughout the 12-month intervention. The following 4 user engagement trajectories were identified: terminated usage (904/1926, 46.9%), weekly usage (731/1926, 38.0%), twice weekly usage (208/1926, 10.8%), and daily usage (83/1926, 4.3%). Active app use during the first month, higher net promoter score after the first 1 to 2 months of use, older age, and better quality of diet at baseline increased the odds of belonging to the continued usage trajectories. Compared with other trajectories, daily usage was associated with a higher increase in diet quality and a more pronounced decrease in BMI and waist circumference at 12 months.

CONCLUSIONS

Distinct long-term usage trajectories of the BitHabit app were identified, and individual predictors for belonging to different trajectory groups were found. These findings highlight the need for being able to identify individuals likely to disengage from interventions early on, and could be used to inform the development of future adaptive interventions.

TRIAL REGISTRATION

ClinicalTrials.gov NCT03156478; https://clinicaltrials.gov/ct2/show/NCT03156478.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.1186/s12889-019-6574-y.

Key Challenges and Opportunities for Cloud Technology in Health Care: Semistructured Interview Study

Background

The use of cloud computing (involving storage and processing of data on the internet) in health care has increasingly been highlighted as having great potential in facilitating data-driven innovations. Although some provider organizations are reaping the benefits of using cloud providers to store and process their data, others are lagging behind.

Objective

We aim to explore the existing challenges and barriers to the use of cloud computing in health care settings and investigate how perceived risks can be addressed.

Methods

We conducted a qualitative case study of cloud computing in health care settings, interviewing a range of individuals with perspectives on supply, implementation, adoption, and integration of cloud technology. Data were collected through a series of in-depth semistructured interviews exploring current applications, implementation approaches, challenges encountered, and visions for the future. The interviews were transcribed and thematically analyzed using NVivo 12 (QSR International). We coded the data based on a sociotechnical coding framework developed in related work.

Results

We interviewed 23 individuals between September 2020 and November 2020, including professionals working across major cloud providers, health care provider organizations, innovators, small and medium-sized software vendors, and academic institutions. The participants were united by a common vision of a cloud-enabled ecosystem of applications and by drivers surrounding data-driven innovation. The identified barriers to progress included the cost of data migration and skill gaps to implement cloud technologies within provider organizations, the cultural shift required to move to externally hosted services, a lack of user pull as many benefits were not visible to those providing frontline care, and a lack of interoperability standards and central regulations.

Conclusions

Implementations need to be viewed as a digitally enabled transformation of services, driven by skill development, organizational change management, and user engagement, to facilitate the implementation and exploitation of cloud-based infrastructures and to maximize returns on investment.

An Artificial Intelligence Chatbot for Young People's Sexual and Reproductive Health in India (SnehAI): Instrumental Case Study

Background

Leveraging artificial intelligence (AI)–driven apps for health education and promotion can help in the accomplishment of several United Nations sustainable development goals. SnehAI, developed by the Population Foundation of India, is the first Hinglish (Hindi + English) AI chatbot, deliberately designed for social and behavioral changes in India. It provides a private, nonjudgmental, and safe space to spur conversations about taboo topics (such as safe sex and family planning) and offers accurate, relatable, and trustworthy information and resources.

Objective

This study aims to use the Gibson theory of affordances to examine SnehAI and offer scholarly guidance on how AI chatbots can be used to educate adolescents and young adults, promote sexual and reproductive health, and advocate for the health entitlements of women and girls in India.

Methods

We adopted an instrumental case study approach that allowed us to explore SnehAI from the perspectives of technology design, program implementation, and user engagement. We also used a mix of qualitative insights and quantitative analytics data to triangulate our findings.

Results

SnehAI demonstrated strong evidence across fifteen functional affordances: accessibility, multimodality, nonlinearity, compellability, queriosity, editability, visibility, interactivity, customizability, trackability, scalability, glocalizability, inclusivity, connectivity, and actionability. SnehAI also effectively engaged its users, especially young men, with 8.2 million messages exchanged across a 5-month period. Almost half of the incoming user messages were texts of deeply personal questions and concerns about sexual and reproductive health, as well as allied topics. Overall, SnehAI successfully presented itself as a trusted friend and mentor; the curated content was both entertaining and educational, and the natural language processing system worked effectively to personalize the chatbot response and optimize user experience.

Conclusions

SnehAI represents an innovative, engaging, and educational intervention that enables vulnerable and hard-to-reach population groups to talk and learn about sensitive and important issues. SnehAI is a powerful testimonial of the vital potential that lies in AI technologies for social good.

Factors Affecting Physicians' Credibility on Twitter When Sharing Health Information: Online Experimental Study

Background

Largely absent from research on how users appraise the credibility of professionals as sources for the information they find on social media is work investigating factors shaping credibility within a specific profession, such as physicians.

Objective

We address debates about how physicians can show their credibility on social media depending on whether they employ a formal or casual appearance in their profile picture. Using prominence-interpretation theory, we posit that formal appearance will affect perceived credibility based on users' social context-specifically, whether they have a regular health care provider.

Methods

For this experiment, we recruited 205 social media users using Amazon Mechanical Turk. We asked participants if they had a regular health care provider and then randomly assigned them to read 1 of 3 Twitter posts that varied only in the profile picture of the physician offering health advice. Next, we tasked participants with assessing the credibility of the physician and their likelihood of engaging with the tweet and the physician on Twitter. We used path analysis to assess whether participants having a regular health care provider impacted how the profile picture affected their ratings of the physician's credibility and their likelihood to engage with the tweet and physician on Twitter.

Results

We found that the profile picture of a physician posting health advice in either formal or casual attire did not elicit significant differences in credibility, with ratings comparable to those having no profile image. Among participants assigned the formal appearance condition, those with a regular provider rated the physician higher on a credibility than those without, which led to stronger intentions to engage with the tweet and physician.

Conclusions

The findings add to existing research by showing how the social context of information seeking on social media shapes the credibility of a given professional. Practical implications for professionals engaging with the public on social media and combating false information include moving past debates about casual versus formal appearances and toward identifying ways to segment audiences based on factors like their backgrounds (eg, experiences with health care providers).

Development of Goal Management Training+ for Methamphetamine Use Disorder Through Collaborative Design

BACKGROUND

Methamphetamine use disorder (MUD) is associated with executive dysfunctions, which are linked with poorer treatment outcomes including earlier drop out and relapse. However, current treatments for MUD do not address executive functions. Goal Management Training (GMT) is an evidence-based cognitive remediation program for executive dysfunction, although required modifications to enhance its relevance and application within addiction treatment settings. This study aimed to (1) tailor GMT to the key cognitive deficits and typical treatment duration of MUD; (2) explore consumers' (people with MUD) engagement with the revised program; (3) implement a prototype of the program with consumers; and (4) present the manualized standard administration to clinical service providers.

METHODS

We followed the Medical Research Council Complex Interventions Framework and employed an evidence- and person-based intervention development process. We used a four-phased approach and collaborated with neuropsychology experts, design researchers in healthcare, consumers with MUD, and clinical service providers. Each aim was addressed in a separate study phase; including content refinement and review with neuropsychology experts (phase 1), intervention design and collaboration with consumers (phase 2), prototype development and review with consumers (phase 3), and final program modifications and review with clinical stakeholders (phase 4).

RESULTS

Findings from phase 1 indicated support for targeting four cognitive processes (attention, impulse control, goal setting, and decision-making). Key feedback included the need to help habitualize cognitive strategies and to guide consumers in applying these strategies in emotionally salient situations. Findings from phases 2 and 3 indicated consumer support for the program strategies and materials but highlighted the need to further enhance the personal relevance of specific content and journal activities. Findings from phase 4 provided clinicians support for the revised program but indicated an opportunity to minimize unintended effects. We present the intervention materials for the final revised program, Goal Management Training⁺ (GMT⁺), in line with TIDieR guidelines.

CONCLUSIONS

GMT⁺ targets key cognitive processes and is sensitive to the clinical needs of people with MUD. Our intervention development process was important for informing the active ingredients and materials for GMT⁺, and indicated initial consumer and provider acceptability prior to conducting a clinical trial.

Estimating the impact of engagement with digital health interventions on patient outcomes in randomized trials

OBJECTIVE

Guidance is needed on studying engagement and treatment effects in digital health interventions, including levels required for benefit. We evaluated multiple analytic approaches for understanding the association between engagement and clinical outcomes.

MATERIALS AND METHODS

We defined engagement as intervention participants' response rate to interactive text messages, and considered moderation, standard regression, mediation, and a modified instrumental variable (IV) analysis to investigate the relationship between engagement and clinical outcomes. We applied each approach to two randomized controlled trials featuring text message content in the intervention: REACH (Rapid Encouragement/Education and Communications for Health), which targeted diabetes, and VERB (Vanderbilt Emergency Room Bundle), which targeted hypertension.

RESULTS

In REACH, the treatment effect on hemoglobin A1c was estimated to be -0.73% (95% CI: [-1.29, -0.21]; P = 0.008), and in VERB, the treatment effect on systolic blood pressure was estimated to be -10.1 mmHg (95% CI: [-17.7, -2.8]; P = 0.007). Only the IV analyses suggested an effect of engagement on outcomes; the difference in treatment effects between engagers and non-engagers was -0.29% to -0.51% in the REACH study and -1.08 to -3.25 mmHg in the VERB study.

DISCUSSION

Standard regression and mediation have less power than a modified IV analysis, but the IV approach requires specification of assumptions. This is the first review of the strengths and limitations of various approaches to evaluating the impact of engagement on outcomes.

CONCLUSIONS

Understanding the role of engagement in digital health interventions can help reveal when and how these interventions achieve desired outcomes.

On the Use of Large Interactive Displays to Support Collaborative Engagement and Visual Exploratory Tasks

Large interactive displays can provide suitable workspaces for learners to conduct collaborative learning tasks with visual information in co-located settings. In this research, we explored the use of these displays to support collaborative engagement and exploratory tasks with visual representations. Our investigation looked at the effect of four factors (number of virtual workspaces within the display, number of displays, position arrangement of the collaborators, and collaborative modes of interaction) on learners’ knowledge acquisition, engagement level, and task performance. To this end, a user study was conducted with 72 participants divided into 6 groups using an interactive tool developed to support the collaborative exploration of 3D visual structures. The results of this study showed that learners with one shared workspace and one single display can achieve better user performance and engagement levels. In addition, the back-to-back position with learners sharing their view and control of the workspaces was the most favorable. It also led to improved learning outcomes and engagement levels during the collaboration process.

Perceptions of Factors Influencing Engagement With Health and Well-being Apps in the United Kingdom: Qualitative Interview Study

BACKGROUND

Digital health devices, such as health and well-being smartphone apps, could offer an accessible and cost-effective way to deliver health and well-being interventions. A key component of the effectiveness of health and well-being apps is user engagement. However, engagement with health and well-being apps is typically poor. Previous studies have identified a list of factors that could influence engagement; however, most of these studies were conducted on a particular population or for an app targeting a particular behavior. An understanding of the factors that influence engagement with a wide range of health and well-being apps can inform the design and the development of more engaging apps in general.

OBJECTIVE

The aim of this study is to explore user experiences of and reasons for engaging and not engaging with a wide range of health and well-being apps.

METHODS

A sample of adults in the United Kingdom (N=17) interested in using a health or well-being app participated in a semistructured interview to explore experiences of engaging and not engaging with these apps. Participants were recruited via social media platforms. Data were analyzed with the framework approach, informed by the Capability, Opportunity, Motivation-Behaviour (COM-B) model and the Theoretical Domains Framework, which are 2 widely used frameworks that incorporate a comprehensive set of behavioral influences.

RESULTS

Factors that influence the capability of participants included available user guidance, statistical and health information, reduced cognitive load, well-designed reminders, self-monitoring features, features that help establish a routine, features that offer a safety net, and stepping-stone app characteristics. Tailoring, peer support, and embedded professional support were identified as important factors that enhance user opportunities for engagement with health and well-being apps. Feedback, rewards, encouragement, goal setting, action planning, self-confidence, and commitment were judged to be the motivation factors that affect engagement with health and well-being apps.

CONCLUSIONS

Multiple factors were identified across all components of the COM-B model that may be valuable for the development of more engaging health and well-being apps. Engagement appears to be influenced primarily by features that provide user guidance, promote minimal cognitive load, support self-monitoring (capability), provide embedded social support (opportunity), and provide goal setting with action planning (motivation). This research provides recommendations for policy makers, industry, health care providers, and app developers for increasing effective engagement.

Integration of Web Analytics Into Graduate Medical Education: Usability Study

Background

Web analytics is the measurement, collection, analysis, and reporting of website and web application usage data. While common in the e-commerce arena, web analytics is underutilized in graduate medical education (GME).

Objective

The University of Arkansas for Medical Sciences Department of Surgery website was revamped with input from in-house surgeons in August 2017. This study investigated the use of web analytics to gauge the impact of our department’s website redesign project.

Methods

Google Analytics software was used to measure website performance before and after implementation of the new website. Eight-month matched periods were compared. Factors tracked included total users, new users, total sessions, sessions per user, pages per session, average session duration, total page views, and bounce rate (the percentage of visitors who visit a site and then leave [ie, bounce] without continuing to another page on the same site).

Results

Analysis using a nonpaired Student t test demonstrated a statistically significant increase for total page views (before vs after: 33,065 vs 81,852; P<.001) and decrease for bounce rate (before vs after: 50.70% vs 0.23%; P<.001). Total users, new users, total sessions, sessions per user, and pages per session showed improvement. The average session duration was unchanged. Subgroup analysis showed that after the main page, the next 3 most frequently visited pages were related to GME programs in our department.

Conclusions

Web analytics is a practical measure of a website’s efficacy. Our data suggest that a modern website significantly improves user engagement. An up-to-date website is essential for contemporary GME recruitment, will likely enhance engagement of residency applicants with GME programs, and warrants further investigation.

Engagement in Digital Mental Health Interventions: Can Monetary Incentives Help?

Digital mental health interventions (DMHI) are scalable and cost-effective strategies for increasing access to mental health care; however, dropout rates associated with digital interventions are high, particularly for open-access digital interventions. While some studies have focused on predictors of dropout from digital mental health programs, few studies have focused on engagement features that might improve engagement. In this perspective article, we discuss whether monetary incentives (MI) are one avenue to increasing user engagement in DMHI. We begin by reviewing the literature on the effects of MI for behavior change in health domains (e.g., dietary behaviors, substance use, and medication adherence). Then, drawing on a pilot study we conducted to test the effects of different levels of MI on usage and improvement in subjective well-being among users of a DMHI (Happify), we discuss the potential applications of MI for DMHI, the potential drawbacks of financial incentives in this context, and open questions for future research.

Engagement with consumer smartwatches for tracking symptoms of individuals living with multiple long-term conditions (multimorbidity): A longitudinal observational study

Introduction

People living with multiple long-term conditions (multimorbidity) (MLTC-M) experience an accumulating combination of different symptoms. It has been suggested that these symptoms can be tracked longitudinally using consumer technology, such as smartphones and wearable devices.

Aim

The aim of this study was to investigate longitudinal user engagement with a smartwatch application, collecting survey questions and active tasks over 90 days, in people living with MLTC-M.

Methods

‘Watch Your Steps’ was a prospective observational study, administering multiple questions and active tasks over 90 days. Adults with more than one clinician-diagnosed long-term conditions were loaned Fossil® Sport smartwatches, pre-loaded with the study app. Around 20 questions were prompted per day.

Daily completion rates were calculated to describe engagement patterns over time, and to explore how these varied by patient characteristics and question type.

Results

Fifty three people with MLTC-M took part in the study. Around half were male ( = 26; 49%) and the majority had a white ethnic background (n = 45; 85%). About a third of participants engaged with the smartwatch app nearly every day. The overall completion rate of symptom questions was 45% inter-quartile range (IQR 23–67%) across all study participants. Older patients and those with greater MLTC-M were more engaged, although engagement was not significantly different between genders.

Conclusion

It was feasible for people living with MLTC-M to report multiple symptoms per day over 3 months. User engagement appeared as good as other mobile health studies that recruited people with single health conditions, despite the higher daily data entry burden.

Responsible innovation in health and health system sustainability: Insights from health innovators' views and practices

Although healthcare managers make increasingly difficult decisions about health innovations, the way they may interact with innovators to foster health system sustainability remains underexplored. Drawing on the Responsible Innovation in Health (RIH) framework, this paper analyses interviews (n=37) with Canadian and Brazilian innovators to identify: how they operationalize inclusive design processes; what influences the responsiveness of their innovation to system-level challenges; and how they consider the level and intensity of care required by their innovation. Our qualitative findings indicate that innovators seek to: 1) engage stakeholders at an early ideation stage through context-specific methods combining both formal and informal strategies; 2) address specific system-level benefits but often struggle with the positioning of their solution within the health system; and 3) mitigate staff shortages in specialized care, increase general practitioners’ capacity or patients and informal caregivers’ autonomy. These findings provide empirical insights on how healthcare managers can promote and organize collaborative processes that harness innovation towards more sustainable health systems. By adopting a RIH-oriented managerial role, they can set in place more inclusive design processes, articulate key system-level challenges, and help innovators adjust the level and intensity of care required by their innovation.

Characteristics of and User Engagement With Antivaping Posts on Instagram: Observational Study

BACKGROUND

Although government agencies acknowledge that messages about the adverse health effects of e-cigarette use should be promoted on social media, effectively delivering those health messages is challenging. Instagram is one of the most popular social media platforms among US youth and young adults, and it has been used to educate the public about the potential harm of vaping through antivaping posts.

OBJECTIVE

We aim to analyze the characteristics of and user engagement with antivaping posts on Instagram to inform future message development and information delivery.

METHODS

A total of 11,322 Instagram posts were collected from November 18, 2019, to January 2, 2020, by using antivaping hashtags including #novape, #novaping, #stopvaping, #dontvape, #antivaping, #quitvaping, #antivape, #stopjuuling, #dontvapeonthepizza, and #escapethevape. Among those posts, 1025 posts were randomly selected and 500 antivaping posts were further identified by hand coding. The image type, image content, and account type of antivaping posts were hand coded, the text information in the caption was explored by topic modeling, and the user engagement of each category was compared.

RESULTS

Analyses found that antivaping images of the educational/warning type were the most common (253/500; 50.6%). The average likes of the educational/warning type (15 likes/post) were significantly lower than the catchphrase image type (these emphasized a slogan such as "athletesdontvape" in the image; 32.5 likes/post; P<.001). The majority of the antivaping posts contained the image content element text (n=332, 66.4%), followed by the image content element people/person (n=110, 22%). The images containing people/person elements (32.8 likes/post) had more likes than the images containing other elements (13.8-21.1 likes/post). The captions of the antivaping Instagram posts covered topics including "lung health," "teen vaping," "stop vaping," and "vaping death cases." Among the 500 antivaping Instagram posts, while most posts were from the antivaping community (n=177, 35.4%) and personal account types (n=182, 36.4%), the antivaping community account type had the highest average number of posts (1.69 posts/account). However, there was no difference in the number of likes among different account types.

CONCLUSIONS

Multiple features of antivaping Instagram posts may be related to user engagement and perception. This study identified the critical elements associated with high user engagement, which could be used to design antivaping posts to deliver health-related information more efficiently.