Evaluating a Digital Mental Health Intervention (Wysa) for Workers' Compensation Claimants: Pilot Feasibility Study

OBJECTIVE

This study examines the feasibility and acceptability of an AI-led digital mental health intervention in a Workers' Compensation (WC) program, Wysa for Return to Work.

METHODS

Self-reported demographic data and responses to psychosocial screening questions were analyzed alongside participants' app usage through which four key outcomes were measured: recruitment rate, onboarding rate, retention, and engagement.

RESULTS

The data demonstrated a high need for psychosocial interventions among injured workers, especially women, young adults, and those with high severity injuries. Those with more psychosocial risk factors had a higher rate of onboarding, retention, and engagement, and those with severe injuries had higher retention.

CONCLUSIONS

Our study concluded that Wysa for Return to Work, the AI-led digital mental health intervention that delivers a recovery program using a digital conversational agent, is feasible and acceptable for a return-to-work population.

DECENT: A sociotechnical approach for developing mobile health apps in underserved settings

Objective

Despite the fact that user engagement is critical to the efficacy of mobile health (mHealth) interventions in the Global South, many of these interventions lack user engagement features. This is because sociotechnical aspects of such initiatives are frequently ignored during the design, development, and implementation stages. This research highlighted the importance of considering sociotechnical factors when developing mHealth apps. The intended users for the mHealth technologies in this study are care professionals.

Materials and Methods

Five semi-structured interviews and a pilot interview were conducted to identify user engagement facilitators and barriers. The interview data were analysed using NVivo. The Capability, Opportunity, Motivation - Behaviour (COM-B) model is then used to map the facilitators and barriers to mHealth app engagement, allowing researchers to better understand how users engage/disengage with mHealth apps.

Results and Discussion

Capability facilitators included features that assist users in learning more about the app (e.g. a user manual and statistical data) as well as features that assist users in developing a routine. The lack of app skills and cognitive overload limit capability. While social connectedness and offline functionality were identified as facilitators of user engagement, non-user-friendly design and cultural dimensions were identified as barriers. Early user engagement and rewards were identified as motivational facilitators that influence user engagement. Furthermore, perceived non-utility and a lack of encouragement were identified as motivational barriers to engagement.

Conclusion

Several factors were discovered across all COM-B model components that could be used to develop more engaging mHealth apps. Adopting a techno-centric approach that ignores sociotechnical factors can reduce user engagement. The design process engagement enhancement system (DECENT) framework was proposed based on the findings.

Enhancing equitable engagement for digital health promotion: Lessons from evaluating a childrearing app in Indonesia

Part of the appeal of digital health interventions, including mHealth, is the potential for greater reach in places where conventional health promotion is hampered by geographical, financial or social barriers. Yet, 'engagement' - typically understood as user experience and interactions with technology - remains a persistent challenge, particularly in places where technology access or familiarity with technology is limited. We undertook an evaluation of a childrearing app to promote socioemotional and cognitive development in early childhood across the world. In this article, we present findings from qualitative research on app rollout in Indonesia, the first of numerous low- and middle-income countries targeted by the app. We draw on systems theory and complexity thinking to broaden the lens of 'engagement' beyond individual users to encompass collective systems (families and communities), exploring how the intervention was harnessed to meet local contextual needs. The qualitative research involved semi-structured interviews, workshops and audio diaries with 57 diverse stakeholders, including Indonesian parents, caregivers, and collaborators involved in funding, development, and dissemination of the app. We observed the importance of social connection, sense-making, and interactive learning for enhancing engagement with the app and its messages. Enthusiastic users, strongly linked across community networks (e.g. kindergarten teachers), improvised dissemination strategies to facilitate uptake. Interactive learning that tapped into familiar social structures (e.g. intergenerational hierarchies) was crucial for engagement. Understanding ways the app failed to tap into structures of social connection served to highlight the need to embed strategies to support collective engagement.

Knowledge platform affordances and knowledge collaboration performance: The mediating effect of user engagement

Knowledge collaboration is the result of knowledge transfer and social interaction among users on knowledge platforms, and its formation mechanism has attracted much attention. Based on the affordance theory, this paper introduces user engagement as a mediating variable to study the relationship between knowledge platform affordances and knowledge collaboration performance. Data collected from 361 valid questionnaires from the Zhihu platform were analyzed by using SPSS 26.0 and Amos 24.0. The results show that knowledge platform affordances have a direct and positive influence on knowledge collaboration performance as well as an indirect influence through user engagement. Expressly, user engagement undertakes three intermediary paths between knowledge platform affordances and knowledge collaboration performance: knowledge affordances-conscious participation-knowledge collaboration performance, knowledge affordances-enthusiasm-knowledge collaboration performance, and social affordances-social interaction-knowledge collaboration performance. This paper explores the formation process of knowledge collaboration performance by synthesizing affordance and user engagement theories. It clarifies the fundamental role of knowledge affordances in stimulating users' conscious participation and enthusiasm, and the critical role of social affordances in stimulating social interaction. Therefore, this paper further enriches theories of knowledge collaboration and knowledge platform affordances and provides a practical reference for the strategic optimization and development of knowledge platforms.

Exploring the effect of use contexts on user engagement toward tourism short video platforms

With tourism short video platforms' increasingly fierce competition, retaining existing users and improving engagement has taken on greater theoretical and practical significance. Based on the self-system model of motivational development, this study, involving an empirical analysis of 252 user data samples, establishes a research model to determine how the use contexts affect users' psychological process and finally lead to behavioral engagement. In particular, four use contexts of tourism short video platforms are proposed-namely, information acquisition, leisure and entertainment, attention obtainment, and social interaction. Different use contexts differ in the degree to which they satisfy users' three psychological needs, thus influence their attitudinal engagement and behavioral engagement. The research results can provide theoretical and practical references on how to improve user engagement toward tourism short video platforms.

Long-term usage of a commercial mHealth app: A "multiple-lives" perspective

Background

Emerging evidence suggests that individuals use mHealth apps in multiple disjointed ways in the real-world-individuals, for example, may engage, take breaks, and re-engage with these apps. To our knowledge, very few studies have adopted this 'multiple-live' perspective to analyze long-term usage of a physical activity (PA) app. This study aimed to examine the duration of use, as well as the frequency, length, and timing of streaks (uninterrupted periods of use) and breaks (uninterrupted periods of non-use) within a popular commercial PA app called Carrot Rewards over 12 months. We also examined sociodemographic correlates of usage.

Method

This retrospective observational study analyzed data from 41,207 Carrot Rewards users participating in the "Steps" walking program from June/July 2016 to June/July 2017. We measured four usage indicators: duration of use, frequency and length of streaks and breaks, time to first break, and time to resume second streak. We also extracted information regarding participants' age, gender, province, and proxy indicators of socioeconomic status derived from census data. We used descriptive statistics to summarize usage patterns, Kaplan-Meier curves to illustrate the time to first break and time to resume second streak. We used linear regressions and Cox Proportional Hazard regression models to examine sociodemographic correlates of usage.

Results

Over 60% of the participants used Carrot Rewards for ≥6 months and 29% used it for 12 months (mean = 32.59 ± 18.435 weeks). The frequency of streaks and breaks ranged from 1 to 9 (mean = 1.61 ± 1.04 times). The mean streak and break length were 20.22 ± 18.26 and 16.14 ± 15.74 weeks, respectively. The median time to first break was 18 weeks across gender groups and provinces; the median time for participants to resume the second streak was between 12 and 32 weeks. Being female, older, and living in a community with greater post-secondary education levels were associated with increased usage.

Conclusion

This study provides empirical evidence that long-term mHealth app usage is possible. In this context, it was common for users to take breaks and re-engage with Carrot Rewards. When designing and evaluating PA apps, therefore, interventionists should consider the 'multiple-lives' perspective described here, as well as the impact of gender and age.

Exploring the Black Box of an mHealth Intervention (LIFE4YOUth): A Qualitative Process and Outcome Evaluation of End-User Engagement

The effectiveness of mHealth interventions rely on whether the content successfully activate mechanisms necessary for behavior change. These mechanisms may be affected by end-users' experience of the intervention content. The aim of this study was to explore how the content of a novel mHealth intervention (LIFE4YOUth) was understood, interpreted, and applied by high school students, and the consequences of engaging with the content. Qualitative content analysis was used inductively and deductively to analyze interview data (n = 16) based on think-aloud techniques with Swedish high school students aged 16-19 years. Theoretical constructs from social cognitive theory framed the deductive analysis. The analysis resulted in four categories which describe central activities of intervention engagement among end-users: defining, considering, centralizing, and personalizing. End-users engaged in these activities to different degrees as illustrated by four typologies: Literal, Vague, Rigid, and Creative engagement. Most informants knew about the risks and benefits of health behaviors, but engagement with intervention content generally increased informants' awareness. In conclusion, this study provides in-depth knowledge on the cognitive process when engaging with mHealth content and suggests that deliberate and flexible engagement most likely deepens end-users' understanding of why and how health behavior change can be managed.

Platform Effects on Public Health Communication: A Comparative and National Study of Message Design and Audience Engagement Across Twitter and Facebook

BACKGROUND

Public health agencies widely adopt social media for health and risk communication. Moreover, different platforms have different affordances, which may impact the quality and nature of the messaging and how the public engages with the content. However, these platform effects are not often compared in studies of health and risk communication and not previously for the COVID-19 pandemic.

OBJECTIVE

This study measures the potential media effects of Twitter and Facebook on public health message design and engagement by comparing message elements and audience engagement in COVID-19-related posts by local, state, and federal public health agencies in the United States during the pandemic, to advance theories of public health messaging on social media and provide recommendations for tailored social media communication strategies.

METHODS

We retrieved all COVID-19-related posts from major US federal agencies related to health and infectious disease, all major state public health agencies, and selected local public health departments on Twitter and Facebook. A total of 100,785 posts related to COVID-19, from 179 different accounts of 96 agencies, were retrieved for the entire year of 2020. We adopted a framework of social media message elements to analyze the posts across Facebook and Twitter. For manual content analysis, we subsampled 1677 posts. We calculated the prevalence of various message elements across the platforms and assessed the statistical significance of differences. We also calculated and assessed the association between message elements with normalized measures of shares and likes for both Facebook and Twitter.

RESULTS

Distributions of message elements were largely similar across both sites. However, political figures (P<.001), experts (P=.01), and nonpolitical personalities (P=.01) were significantly more present on Facebook posts compared to Twitter. Infographics (P<.001), surveillance information (P<.001), and certain multimedia elements (eg, hyperlinks, P<.001) were more prevalent on Twitter. In general, Facebook posts received more (normalized) likes (0.19%) and (normalized) shares (0.22%) compared to Twitter likes (0.08%) and shares (0.05%). Elements with greater engagement on Facebook included expressives and collectives, whereas posts related to policy were more engaged with on Twitter. Science information (eg, scientific explanations) comprised 8.5% (73/851) of Facebook and 9.4% (78/826) of Twitter posts. Correctives of misinformation only appeared in 1.2% (11/851) of Facebook and 1.4% (12/826) of Twitter posts.

CONCLUSIONS

In general, we find a data and policy orientation for Twitter messages and users and a local and personal orientation for Facebook, although also many similarities across platforms. Message elements that impact engagement are similar across platforms but with some notable distinctions. This study provides novel evidence for differences in COVID-19 public health messaging across social media sites, advancing knowledge of public health communication on social media and recommendations for health and risk communication strategies on these online platforms.

Understanding Engagement With Forensic Smartphone Apps: The Service Design Engagement Model

Justice services have begun to integrate the use of mobile applications into treatment, support, and rehabilitative programs for forensic clients. One such application that been adopted to support forensic clients is "eRecovery": a smartphone application that provides clients recovering from a substance addiction with support for managing relapse. In this article, we report on evaluation findings from a trial of eRecovery in an Australian Community Justice Centre, and reflect on several issues relating to fostering and sustaining client engagement with similar applications within forensic and justice settings. We propose the Service Design Engagement Model to organize, visualize, and describe the stages and factors important to adoption, appropriation, and on-going routine use of the software by forensic clients. The model recognizes the role of contextual and environmental factors in supporting users through the early stages of engagement, and the importance of user agency in longer-term engagement with therapeutic apps.

Accessing the Impact Mechanism of Sense of Virtual Community on User Engagement

Although research has begun to explore the influence patterns of sense of virtual community, there is limited research on how sense of virtual community affects educational virtual community user engagement. Based on the educational virtual community context, this study constructs a theoretical model with moderation and mediation to explore the mediation mechanism of sense of virtual community affecting user engagement and its boundary conditions. In this study, the data collected from 377 users are analyzed by structural equation modeling. The research findings found that not only effective commitment has a mediating role between sense of virtual community and user engagement, but also perceived support has a moderating role in the process of effective commitment's influence on user engagement. This study examines the practical effects of sense of virtual community in the context of educational virtual community use and reveals the mechanism of the effect of sense of virtual community on user engagement.

Clinician collaboration to improve clinical decision support: the Clickbusters initiative

OBJECTIVE

We describe the Clickbusters initiative implemented at Vanderbilt University Medical Center (VUMC), which was designed to improve safety and quality and reduce burnout through the optimization of clinical decision support (CDS) alerts.

MATERIALS AND METHODS

We developed a 10-step Clickbusting process and implemented a program that included a curriculum, CDS alert inventory, oversight process, and gamification. We carried out two 3-month rounds of the Clickbusters program at VUMC. We completed descriptive analyses of the changes made to alerts during the process, and of alert firing rates before and after the program.

RESULTS

Prior to Clickbusters, VUMC had 419 CDS alerts in production, with 488 425 firings (42 982 interruptive) each week. After 2 rounds, the Clickbusters program resulted in detailed, comprehensive reviews of 84 CDS alerts and reduced the number of weekly alert firings by more than 70 000 (15.43%). In addition to the direct improvements in CDS, the initiative also increased user engagement and involvement in CDS.

CONCLUSIONS

At VUMC, the Clickbusters program was successful in optimizing CDS alerts by reducing alert firings and resulting clicks. The program also involved more users in the process of evaluating and improving CDS and helped build a culture of continuous evaluation and improvement of clinical content in the electronic health record.

PEGASO e-Diary: User Engagement and Dietary Behavior Change of a Mobile Food Record for Adolescents

Background

Obesity amongst children and adolescents is becoming a major health problem globally and mobile food records can play a crucial role in promoting healthy dietary habits.

Objective

To describe the methodology for the implementation of the e-Diary mobile food record, to assess its capability in promoting healthy eating habits, to evaluate the factors associated with its usage and engagement.

Methods

This is a descriptive study that compared the characteristics of participants engaged in the e-Diary, which was part of the PEGASO project in which an app to provide proactive health promotion was given to 365 students at 4 European sites enrolled during October to December 2016: England (UK), Scotland (UK), Lombardy (Italy), and Catalonia (Spain). The e-Diary tracked the users' dietary habits in terms of food groups, dietary indexes, and 6 dietary target behaviors relating to consumption of: fruit; vegetable; breakfast; sugar-sweetened beverages; fast-food; and snacks. The e-Diary provided also personalized suggestions for the next meal and gamification.

Results

The e-Diary was used for 6 months by 357 adolescents (53.8% females). The study showed that females used the e-Diary much more than males (aOR 3.8, 95% CI 1.6-8.8). Participants aged 14 years were more engaged in the e-Diary than older age groups (aOR 5.1, 95% CI 1.4-18.8) as were those with a very good/excellent self-perceived health status compared to their peers with fair/poor health perception (aOR 4.2, 95% CI 1.3-13.3). Compared to the intervention sites, those living in Catalonia (aOR 13.2 95% CI 2.5-68.8) were more engaged. In terms of behavior change, a significant positive correlation between fruit (p < 0.0001) and vegetables (p = 0.0087) intake was observed in association with increased engagement in the e-Diary. Similarly, adolescents who used the app for more than 2 weeks had significantly higher odds of not skipping breakfast over the study period (aOR 2.5, 95% CI 1.0-6.3).

Conclusions

The users highly engaged with the e-Diary were associated with improved dietary behaviors: increased consumption of fruit and vegetables and reduced skipping of breakfast. Although the overall usage of the e-Diary was high during the first weeks, it declined thereafter. Future applications should foster user engagement, particularly targeting adolescents at high risk.

Clinical Trial Registration

https://www.clinicaltrials.gov/, identifier: NCT02930148.

Text Messaging Intervention for Mental Wellness in American Indian and Alaska Native Teens and Young Adults (BRAVE Study): Analysis of User Engagement Patterns

Background

Many American Indian and Alaska Native (AI/AN or Native) communities express concern about high rates of suicide and poor mental health. Technology-based health interventions that nurture resilience, coping skills, connectedness, and help-seeking skills may be an effective strategy for promoting health and wellbeing among AI/AN youth. The Northwest Portland Area Indian Health Board designed the BRAVE intervention for AI/AN youth. BRAVE is delivered via SMS text messaging and includes role model videos, mental wellness strategies, links to culturally relevant resources, and social support from family and friends.

Objective

The aim of this study is to explore system data from the BRAVE intervention to determine patterns of user engagement and differences in psychosocial outcomes based on the number of clicks on BRAVE content.

Methods

The BRAVE study included 1030 AI/AN teens and young adults nationwide (15 to 24 years old). The message series in the BRAVE and STEM study arms included 3 to 5 SMS text messages per week, featuring 1 role model video and 1 image per week. Messages were sent out via Mobile Commons (Upland Software Inc), a mobile messaging provider that supports text, picture, and video SMS.

Results

Of the 509 participants in the original BRAVE analysis, 270 had sufficient data to analyze user engagement, with at least 1 trackable click on a study SMS text message. Of the 270, 184 (68.1%) were female, 50 (18.5%) were male, and 36 (13.3%) selected another gender category. The average participant was 20.6 years old, with a minimum and maximum age of 15 and 26 years. Most participants had relatively low engagement measured by the number of clicks (median 2; mean 3.4), although others clicked message content as many as 49 times. Users engaged most frequently with the YouTube-based content (viewing 1 of 7 role model videos), with 64.8% (175/270) of total clicks coming from the role model videos, and earlier episodes receiving the highest number of clicks. Most baseline psychosocial measures were not significantly associated with the number of links clicked. However, help-seeking behavior was highly significant (P<.001), with a rate ratio of 0.82 (0.73, 0.92), indicating that each 1-unit increase in help-seeking score at baseline was associated with an 18% decrease in the expected number of study content clicks.

Conclusions

This is the first study to set initial standards for assessing user engagement in an mHealth intervention. Our work underscores the feasibility of exploring the impact of engagement on intended outcomes, allowing for more precise exploration of the dose-response relationship that may be realized through these low-touch interventions that offer promising potential for reaching high numbers of program participants.

Trial Registration

ClinicalTrials.gov NCT04979481; https://clinicaltrials.gov/ct2/show/NCT04979481

Changes in Use of a Leisure Activity Mobile App for Children With Disabilities During the COVID-19 Pandemic: Retrospective Study

BACKGROUND

Participation in leisure activities is essential for child development and a human right as per the United Nations Convention on the Rights of the Child. Children with disabilities face several restrictions when participating in leisure activities as compared to same age peers without disabilities. Access to information about accessible, inclusive leisure activities is one of the barriers limiting participation, and one potential health promotion strategy is to provide access to information to increase participation. The Jooay App is a mobile app listing such activities in Canada and Australia. With the COVID-19 global pandemic and subsequent public health measures, most community-based facilities providing the activities listed on Jooay were closed. The app therefore started listing online activities offered with the expectation of continuing to provide information for families and understanding the extent to which users relied on the mobile app as a tool to identify new safe leisure opportunities.

OBJECTIVE

This study aims to describe the engagement of the Jooay app before and during COVID-19, and to estimate the extent to which the listing of online activities was related to the engagement of the Jooay app.

METHODS

We conducted a retrospective study comparing Jooay app use between March 2020 and February 2021 to the engagement between March 2019 and February 2020 by Jooay users. Spearman rank correlations were carried out to identify associations between the activities listed and the users' engagement from May 2020 to February 2021.

RESULTS

Active engagement with the Jooay app from March 2020 to February 2021 dropped by an average of 135 engagements (64.2%) compared to engagements in 2019-2020. The largest monthly drop in engagement was observed in May 2020 by 239 engagements (88.8%). There was a strong positive correlation between the number of active users and the number of online activities listed on the app (r_s=0.900).

CONCLUSIONS

The engagement with the Jooay App presented an expected decrease during the first wave of the COVID-19 pandemic. The addition of online adapted leisure activities to the app's listings during the pandemic increased app use. Access to information about inclusive activities is a barrier for children with disabilities to engage in leisure. Mobile health solutions can be responsive to contextual factors and consider the social determinants of health such as socioeconomic and public health emergency issues that can impact the participation of vulnerable populations such as children with disabilities and help eliminate barriers to participation. The provision of online leisure opportunities during the pandemic could facilitate participation in these activities during the pandemic and beyond, which is essential and beneficial for the physical and mental well-being of children with disabilities and their families.

Digitally Supported Lifestyle Intervention to Prevent Type 2 Diabetes Through Healthy Habits: Secondary Analysis of Long-Term User Engagement Trajectories in a Randomized Controlled Trial

BACKGROUND

Digital health interventions may offer a scalable way to prevent type 2 diabetes (T2D) with minimal burden on health care systems by providing early support for healthy behaviors among adults at increased risk for T2D. However, ensuring continued engagement with digital solutions is a challenge impacting the expected effectiveness.

OBJECTIVE

We aimed to investigate the longitudinal usage patterns of a digital healthy habit formation intervention, BitHabit, and the associations with changes in T2D risk factors.

METHODS

This is a secondary analysis of the StopDia (Stop Diabetes) study, an unblinded parallel 1-year randomized controlled trial evaluating the effectiveness of the BitHabit app alone or together with face-to-face group coaching in comparison with routine care in Finland in 2017-2019 among community-dwelling adults (aged 18 to 74 years) at an increased risk of T2D. We used longitudinal data on usage from 1926 participants randomized to the digital intervention arms. Latent class growth models were applied to identify user engagement trajectories with the app during the study. Predictors for trajectory membership were examined with multinomial logistic regression models. Analysis of covariance was used to investigate the association between trajectories and 12-month changes in T2D risk factors.

RESULTS

More than half (1022/1926, 53.1%) of the participants continued to use the app throughout the 12-month intervention. The following 4 user engagement trajectories were identified: terminated usage (904/1926, 46.9%), weekly usage (731/1926, 38.0%), twice weekly usage (208/1926, 10.8%), and daily usage (83/1926, 4.3%). Active app use during the first month, higher net promoter score after the first 1 to 2 months of use, older age, and better quality of diet at baseline increased the odds of belonging to the continued usage trajectories. Compared with other trajectories, daily usage was associated with a higher increase in diet quality and a more pronounced decrease in BMI and waist circumference at 12 months.

CONCLUSIONS

Distinct long-term usage trajectories of the BitHabit app were identified, and individual predictors for belonging to different trajectory groups were found. These findings highlight the need for being able to identify individuals likely to disengage from interventions early on, and could be used to inform the development of future adaptive interventions.

TRIAL REGISTRATION

ClinicalTrials.gov NCT03156478; https://clinicaltrials.gov/ct2/show/NCT03156478.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.1186/s12889-019-6574-y.

Key Challenges and Opportunities for Cloud Technology in Health Care: Semistructured Interview Study

Background

The use of cloud computing (involving storage and processing of data on the internet) in health care has increasingly been highlighted as having great potential in facilitating data-driven innovations. Although some provider organizations are reaping the benefits of using cloud providers to store and process their data, others are lagging behind.

Objective

We aim to explore the existing challenges and barriers to the use of cloud computing in health care settings and investigate how perceived risks can be addressed.

Methods

We conducted a qualitative case study of cloud computing in health care settings, interviewing a range of individuals with perspectives on supply, implementation, adoption, and integration of cloud technology. Data were collected through a series of in-depth semistructured interviews exploring current applications, implementation approaches, challenges encountered, and visions for the future. The interviews were transcribed and thematically analyzed using NVivo 12 (QSR International). We coded the data based on a sociotechnical coding framework developed in related work.

Results

We interviewed 23 individuals between September 2020 and November 2020, including professionals working across major cloud providers, health care provider organizations, innovators, small and medium-sized software vendors, and academic institutions. The participants were united by a common vision of a cloud-enabled ecosystem of applications and by drivers surrounding data-driven innovation. The identified barriers to progress included the cost of data migration and skill gaps to implement cloud technologies within provider organizations, the cultural shift required to move to externally hosted services, a lack of user pull as many benefits were not visible to those providing frontline care, and a lack of interoperability standards and central regulations.

Conclusions

Implementations need to be viewed as a digitally enabled transformation of services, driven by skill development, organizational change management, and user engagement, to facilitate the implementation and exploitation of cloud-based infrastructures and to maximize returns on investment.

An Artificial Intelligence Chatbot for Young People's Sexual and Reproductive Health in India (SnehAI): Instrumental Case Study

Background

Leveraging artificial intelligence (AI)–driven apps for health education and promotion can help in the accomplishment of several United Nations sustainable development goals. SnehAI, developed by the Population Foundation of India, is the first Hinglish (Hindi + English) AI chatbot, deliberately designed for social and behavioral changes in India. It provides a private, nonjudgmental, and safe space to spur conversations about taboo topics (such as safe sex and family planning) and offers accurate, relatable, and trustworthy information and resources.

Objective

This study aims to use the Gibson theory of affordances to examine SnehAI and offer scholarly guidance on how AI chatbots can be used to educate adolescents and young adults, promote sexual and reproductive health, and advocate for the health entitlements of women and girls in India.

Methods

We adopted an instrumental case study approach that allowed us to explore SnehAI from the perspectives of technology design, program implementation, and user engagement. We also used a mix of qualitative insights and quantitative analytics data to triangulate our findings.

Results

SnehAI demonstrated strong evidence across fifteen functional affordances: accessibility, multimodality, nonlinearity, compellability, queriosity, editability, visibility, interactivity, customizability, trackability, scalability, glocalizability, inclusivity, connectivity, and actionability. SnehAI also effectively engaged its users, especially young men, with 8.2 million messages exchanged across a 5-month period. Almost half of the incoming user messages were texts of deeply personal questions and concerns about sexual and reproductive health, as well as allied topics. Overall, SnehAI successfully presented itself as a trusted friend and mentor; the curated content was both entertaining and educational, and the natural language processing system worked effectively to personalize the chatbot response and optimize user experience.

Conclusions

SnehAI represents an innovative, engaging, and educational intervention that enables vulnerable and hard-to-reach population groups to talk and learn about sensitive and important issues. SnehAI is a powerful testimonial of the vital potential that lies in AI technologies for social good.

Factors Affecting Physicians' Credibility on Twitter When Sharing Health Information: Online Experimental Study

Background

Largely absent from research on how users appraise the credibility of professionals as sources for the information they find on social media is work investigating factors shaping credibility within a specific profession, such as physicians.

Objective

We address debates about how physicians can show their credibility on social media depending on whether they employ a formal or casual appearance in their profile picture. Using prominence-interpretation theory, we posit that formal appearance will affect perceived credibility based on users' social context-specifically, whether they have a regular health care provider.

Methods

For this experiment, we recruited 205 social media users using Amazon Mechanical Turk. We asked participants if they had a regular health care provider and then randomly assigned them to read 1 of 3 Twitter posts that varied only in the profile picture of the physician offering health advice. Next, we tasked participants with assessing the credibility of the physician and their likelihood of engaging with the tweet and the physician on Twitter. We used path analysis to assess whether participants having a regular health care provider impacted how the profile picture affected their ratings of the physician's credibility and their likelihood to engage with the tweet and physician on Twitter.

Results

We found that the profile picture of a physician posting health advice in either formal or casual attire did not elicit significant differences in credibility, with ratings comparable to those having no profile image. Among participants assigned the formal appearance condition, those with a regular provider rated the physician higher on a credibility than those without, which led to stronger intentions to engage with the tweet and physician.

Conclusions

The findings add to existing research by showing how the social context of information seeking on social media shapes the credibility of a given professional. Practical implications for professionals engaging with the public on social media and combating false information include moving past debates about casual versus formal appearances and toward identifying ways to segment audiences based on factors like their backgrounds (eg, experiences with health care providers).

Development of Goal Management Training+ for Methamphetamine Use Disorder Through Collaborative Design

BACKGROUND

Methamphetamine use disorder (MUD) is associated with executive dysfunctions, which are linked with poorer treatment outcomes including earlier drop out and relapse. However, current treatments for MUD do not address executive functions. Goal Management Training (GMT) is an evidence-based cognitive remediation program for executive dysfunction, although required modifications to enhance its relevance and application within addiction treatment settings. This study aimed to (1) tailor GMT to the key cognitive deficits and typical treatment duration of MUD; (2) explore consumers' (people with MUD) engagement with the revised program; (3) implement a prototype of the program with consumers; and (4) present the manualized standard administration to clinical service providers.

METHODS

We followed the Medical Research Council Complex Interventions Framework and employed an evidence- and person-based intervention development process. We used a four-phased approach and collaborated with neuropsychology experts, design researchers in healthcare, consumers with MUD, and clinical service providers. Each aim was addressed in a separate study phase; including content refinement and review with neuropsychology experts (phase 1), intervention design and collaboration with consumers (phase 2), prototype development and review with consumers (phase 3), and final program modifications and review with clinical stakeholders (phase 4).

RESULTS

Findings from phase 1 indicated support for targeting four cognitive processes (attention, impulse control, goal setting, and decision-making). Key feedback included the need to help habitualize cognitive strategies and to guide consumers in applying these strategies in emotionally salient situations. Findings from phases 2 and 3 indicated consumer support for the program strategies and materials but highlighted the need to further enhance the personal relevance of specific content and journal activities. Findings from phase 4 provided clinicians support for the revised program but indicated an opportunity to minimize unintended effects. We present the intervention materials for the final revised program, Goal Management Training⁺ (GMT⁺), in line with TIDieR guidelines.

CONCLUSIONS

GMT⁺ targets key cognitive processes and is sensitive to the clinical needs of people with MUD. Our intervention development process was important for informing the active ingredients and materials for GMT⁺, and indicated initial consumer and provider acceptability prior to conducting a clinical trial.

Estimating the impact of engagement with digital health interventions on patient outcomes in randomized trials

OBJECTIVE

Guidance is needed on studying engagement and treatment effects in digital health interventions, including levels required for benefit. We evaluated multiple analytic approaches for understanding the association between engagement and clinical outcomes.

MATERIALS AND METHODS

We defined engagement as intervention participants' response rate to interactive text messages, and considered moderation, standard regression, mediation, and a modified instrumental variable (IV) analysis to investigate the relationship between engagement and clinical outcomes. We applied each approach to two randomized controlled trials featuring text message content in the intervention: REACH (Rapid Encouragement/Education and Communications for Health), which targeted diabetes, and VERB (Vanderbilt Emergency Room Bundle), which targeted hypertension.

RESULTS

In REACH, the treatment effect on hemoglobin A1c was estimated to be -0.73% (95% CI: [-1.29, -0.21]; P = 0.008), and in VERB, the treatment effect on systolic blood pressure was estimated to be -10.1 mmHg (95% CI: [-17.7, -2.8]; P = 0.007). Only the IV analyses suggested an effect of engagement on outcomes; the difference in treatment effects between engagers and non-engagers was -0.29% to -0.51% in the REACH study and -1.08 to -3.25 mmHg in the VERB study.

DISCUSSION

Standard regression and mediation have less power than a modified IV analysis, but the IV approach requires specification of assumptions. This is the first review of the strengths and limitations of various approaches to evaluating the impact of engagement on outcomes.

CONCLUSIONS

Understanding the role of engagement in digital health interventions can help reveal when and how these interventions achieve desired outcomes.

On the Use of Large Interactive Displays to Support Collaborative Engagement and Visual Exploratory Tasks

Large interactive displays can provide suitable workspaces for learners to conduct collaborative learning tasks with visual information in co-located settings. In this research, we explored the use of these displays to support collaborative engagement and exploratory tasks with visual representations. Our investigation looked at the effect of four factors (number of virtual workspaces within the display, number of displays, position arrangement of the collaborators, and collaborative modes of interaction) on learners’ knowledge acquisition, engagement level, and task performance. To this end, a user study was conducted with 72 participants divided into 6 groups using an interactive tool developed to support the collaborative exploration of 3D visual structures. The results of this study showed that learners with one shared workspace and one single display can achieve better user performance and engagement levels. In addition, the back-to-back position with learners sharing their view and control of the workspaces was the most favorable. It also led to improved learning outcomes and engagement levels during the collaboration process.

Perceptions of Factors Influencing Engagement With Health and Well-being Apps in the United Kingdom: Qualitative Interview Study

BACKGROUND

Digital health devices, such as health and well-being smartphone apps, could offer an accessible and cost-effective way to deliver health and well-being interventions. A key component of the effectiveness of health and well-being apps is user engagement. However, engagement with health and well-being apps is typically poor. Previous studies have identified a list of factors that could influence engagement; however, most of these studies were conducted on a particular population or for an app targeting a particular behavior. An understanding of the factors that influence engagement with a wide range of health and well-being apps can inform the design and the development of more engaging apps in general.

OBJECTIVE

The aim of this study is to explore user experiences of and reasons for engaging and not engaging with a wide range of health and well-being apps.

METHODS

A sample of adults in the United Kingdom (N=17) interested in using a health or well-being app participated in a semistructured interview to explore experiences of engaging and not engaging with these apps. Participants were recruited via social media platforms. Data were analyzed with the framework approach, informed by the Capability, Opportunity, Motivation-Behaviour (COM-B) model and the Theoretical Domains Framework, which are 2 widely used frameworks that incorporate a comprehensive set of behavioral influences.

RESULTS

Factors that influence the capability of participants included available user guidance, statistical and health information, reduced cognitive load, well-designed reminders, self-monitoring features, features that help establish a routine, features that offer a safety net, and stepping-stone app characteristics. Tailoring, peer support, and embedded professional support were identified as important factors that enhance user opportunities for engagement with health and well-being apps. Feedback, rewards, encouragement, goal setting, action planning, self-confidence, and commitment were judged to be the motivation factors that affect engagement with health and well-being apps.

CONCLUSIONS

Multiple factors were identified across all components of the COM-B model that may be valuable for the development of more engaging health and well-being apps. Engagement appears to be influenced primarily by features that provide user guidance, promote minimal cognitive load, support self-monitoring (capability), provide embedded social support (opportunity), and provide goal setting with action planning (motivation). This research provides recommendations for policy makers, industry, health care providers, and app developers for increasing effective engagement.

Integration of Web Analytics Into Graduate Medical Education: Usability Study

Background

Web analytics is the measurement, collection, analysis, and reporting of website and web application usage data. While common in the e-commerce arena, web analytics is underutilized in graduate medical education (GME).

Objective

The University of Arkansas for Medical Sciences Department of Surgery website was revamped with input from in-house surgeons in August 2017. This study investigated the use of web analytics to gauge the impact of our department’s website redesign project.

Methods

Google Analytics software was used to measure website performance before and after implementation of the new website. Eight-month matched periods were compared. Factors tracked included total users, new users, total sessions, sessions per user, pages per session, average session duration, total page views, and bounce rate (the percentage of visitors who visit a site and then leave [ie, bounce] without continuing to another page on the same site).

Results

Analysis using a nonpaired Student t test demonstrated a statistically significant increase for total page views (before vs after: 33,065 vs 81,852; P<.001) and decrease for bounce rate (before vs after: 50.70% vs 0.23%; P<.001). Total users, new users, total sessions, sessions per user, and pages per session showed improvement. The average session duration was unchanged. Subgroup analysis showed that after the main page, the next 3 most frequently visited pages were related to GME programs in our department.

Conclusions

Web analytics is a practical measure of a website’s efficacy. Our data suggest that a modern website significantly improves user engagement. An up-to-date website is essential for contemporary GME recruitment, will likely enhance engagement of residency applicants with GME programs, and warrants further investigation.

Engagement in Digital Mental Health Interventions: Can Monetary Incentives Help?

Digital mental health interventions (DMHI) are scalable and cost-effective strategies for increasing access to mental health care; however, dropout rates associated with digital interventions are high, particularly for open-access digital interventions. While some studies have focused on predictors of dropout from digital mental health programs, few studies have focused on engagement features that might improve engagement. In this perspective article, we discuss whether monetary incentives (MI) are one avenue to increasing user engagement in DMHI. We begin by reviewing the literature on the effects of MI for behavior change in health domains (e.g., dietary behaviors, substance use, and medication adherence). Then, drawing on a pilot study we conducted to test the effects of different levels of MI on usage and improvement in subjective well-being among users of a DMHI (Happify), we discuss the potential applications of MI for DMHI, the potential drawbacks of financial incentives in this context, and open questions for future research.