Participatory Development and Pilot Testing of an Adolescent Health Promotion Chatbot

A necessary conversation to develop chatbots for HIV studies: qualitative findings from research staff, community advisory board members, and study participants

Chatbots increase business productivity by handling customer conversations instead of human agents. Similar rationale applies to use chatbots in the healthcare sector, especially for health coaches who converse with clients. Chatbots are nascent in healthcare. Study findings have been mixed in terms of engagement and their impact on outcomes. Questions remain as to chatbot acceptability with coaches and other providers; studies have focused on clients.To clarify perceived benefits of chatbots in HIV interventions we conducted virtual focus groups with 13 research staff, eight community advisory board members, and seven young adults who were HIV intervention trial participants (clients). Our HIV healthcare context is important. Clients represent a promising age demographic for chatbot uptake. They are a marginalized population warranting consideration to avoid technology that limits healthcare access.Focus group participants expressed the value of chatbots for HIV research staff and clients. Staff discussed how chatbot functions, such as automated appointment scheduling and service referrals, could reduce workloads while clients discussed the after-hours convenience of these functions. Participants also emphasized that chatbots should provide relatable conversation, reliable functionality, and would not be appropriate for all clients. Our findings underscore the need to further examine appropriate chatbot functionality in HIV interventions.

Evaluation framework for conversational agents with artificial intelligence in health interventions: a systematic scoping review

OBJECTIVES

Conversational agents (CAs) with emerging artificial intelligence present new opportunities to assist in health interventions but are difficult to evaluate, deterring their applications in the real world. We aimed to synthesize existing evidence and knowledge and outline an evaluation framework for CA interventions.

MATERIALS AND METHODS

We conducted a systematic scoping review to investigate designs and outcome measures used in the studies that evaluated CAs for health interventions. We then nested the results into an overarching digital health framework proposed by the World Health Organization (WHO).

RESULTS

The review included 81 studies evaluating CAs in experimental (n = 59), observational (n = 15) trials, and other research designs (n = 7). Most studies (n = 72, 89%) were published in the past 5 years. The proposed CA-evaluation framework includes 4 evaluation stages: (1) feasibility/usability, (2) efficacy, (3) effectiveness, and (4) implementation, aligning with WHO's stepwise evaluation strategy. Across these stages, this article presents the essential evidence of different study designs (n = 8), sample sizes, and main evaluation categories (n = 7) with subcategories (n = 40). The main evaluation categories included (1) functionality, (2) safety and information quality, (3) user experience, (4) clinical and health outcomes, (5) costs and cost benefits, (6) usage, adherence, and uptake, and (7) user characteristics for implementation research. Furthermore, the framework highlighted the essential evaluation areas (potential primary outcomes) and gaps across the evaluation stages.

DISCUSSION AND CONCLUSION

This review presents a new framework with practical design details to support the evaluation of CA interventions in healthcare research.

PROTOCOL REGISTRATION

The Open Science Framework (https://osf.io/9hq2v) on March 22, 2021.

A mobile healthy lifestyle intervention to promote mental health in adolescence: a mixed-methods evaluation

BACKGROUND

A healthy lifestyle may improve mental health. It is yet not known whether and how a mobile intervention can be of help in achieving this in adolescents. This study investigated the effectiveness and perceived underlying mechanisms of the mobile health (mHealth) intervention #LIFEGOALS to promote healthy lifestyles and mental health. #LIFEGOALS is an evidence-based app with activity tracker, including self-regulation techniques, gamification elements, a support chatbot, and health narrative videos.

METHODS

A quasi-randomized controlled trial (N = 279) with 12-week intervention period and process evaluation interviews (n = 13) took place during the COVID-19 pandemic. Adolescents (12-15y) from the general population were allocated at school-level to the intervention (n = 184) or to a no-intervention group (n = 95). Health-related quality of life (HRQoL), psychological well-being, mood, self-perception, peer support, resilience, depressed feelings, sleep quality and breakfast frequency were assessed via a web-based survey; physical activity, sedentary time, and sleep routine via Axivity accelerometers. Multilevel generalized linear models were fitted to investigate intervention effects and moderation by pandemic-related measures. Interviews were coded using thematic analysis.

RESULTS

Non-usage attrition was high: 18% of the participants in the intervention group never used the app. An additional 30% stopped usage by the second week. Beneficial intervention effects were found for physical activity (χ21 = 4.36, P = .04), sedentary behavior (χ21 = 6.44, P = .01), sleep quality (χ21 = 6.11, P = .01), and mood (χ21 = 2.30, P = .02). However, effects on activity-related behavior were only present for adolescents having normal sports access, and effects on mood only for adolescents with full in-school education. HRQoL (χ22 = 14.72, P < .001), mood (χ21 = 6.03, P = .01), and peer support (χ21 = 13.69, P < .001) worsened in adolescents with pandemic-induced remote-education. Interviewees reported that the reward system, self-regulation guidance, and increased health awareness had contributed to their behavior change. They also pointed to the importance of social factors, quality of technology and autonomy for mHealth effectiveness.

CONCLUSIONS

#LIFEGOALS showed mixed results on health behaviors and mental health. The findings highlight the role of contextual factors for mHealth promotion in adolescence, and provide suggestions to optimize support by a chatbot and narrative episodes.

TRIAL REGISTRATION

ClinicalTrials.gov [NCT04719858], registered on 22/01/2021.

Development of a chatbot for school violence prevention among elementary school students in South Korea: a methodological study

PURPOSE

This study develops a chatbot for school violence prevention (C-SVP) among elementary school students.

METHODS

Among the analysis, design, development, implementation, and evaluation (ADDIE) models, ADD phases were applied to develop a C-SVP. Students' learning needs were identified by constructing content with a design that attracted their attention. Subsequently, a formative evaluation was conducted on the developed C-SVP to test its applicability by ten elementary school students targeting the 5th and 6th grades.

RESULTS

The chatbot was designed using KakaoTalk and named "School Guardian Angel." The formative evaluation revealed that the developed C-SVP was easily accessible and useful for elementary school students.

CONCLUSION

The developed C-SVP is expected to be effective in preventing violence among elementary school students. However, further research involving children of various age groups is required.

Feasibility and Acceptability of Chatbots for Nutrition and Physical Activity Health Promotion Among Adolescents: Systematic Scoping Review With Adolescent Consultation

BACKGROUND

Reducing lifestyle risk behaviors among adolescents depends on access to age-appropriate health promotion information. Chatbots-computer programs designed to simulate conversations with human users-have the potential to deliver health information to adolescents to improve their lifestyle behaviors and support behavior change, but research on the feasibility and acceptability of chatbots in the adolescent population is unknown.

OBJECTIVE

This systematic scoping review aims to evaluate the feasibility and acceptability of chatbots in nutrition and physical activity interventions among adolescents. A secondary aim is to consult adolescents to identify features of chatbots that are acceptable and feasible.

METHODS

We searched 6 electronic databases from March to April 2022 (MEDLINE, Embase, Joanna Briggs Institute, the Cumulative Index to Nursing and Allied Health, the Association for Computing Machinery library, and the IT database Institute of Electrical and Electronics Engineers). Peer-reviewed studies were included that were conducted in the adolescent population (10-19 years old) without any chronic disease, except obesity or type 2 diabetes, and assessed chatbots used nutrition or physical activity interventions or both that encouraged individuals to meet dietary or physical activity guidelines and support positive behavior change. Studies were screened by 2 independent reviewers, with any queries resolved by a third reviewer. Data were extracted into tables and collated in a narrative summary. Gray literature searches were also undertaken. Results of the scoping review were presented to a diverse youth advisory group (N=16, 13-18 years old) to gain insights into this topic beyond what is published in the literature.

RESULTS

The search identified 5558 papers, with 5 (0.1%) studies describing 5 chatbots meeting the inclusion criteria. The 5 chatbots were supported by mobile apps using a combination of the following features: personalized feedback, conversational agents, gamification, and monitoring of behavior change. Of the 5 studies, 2 (40.0%) studies focused on nutrition, 2 (40.0%) studies focused on physical activity, and 1 (20.0%) focused on both nutrition and physical activity. Feasibility and acceptability varied across the 5 studies, with usage rates above 50% in 3 (60.0%) studies. In addition, 3 (60.0%) studies reported health-related outcomes, with only 1 (20.0%) study showing promising effects of the intervention. Adolescents presented novel concerns around the use of chatbots in nutrition and physical activity interventions, including ethical concerns and the use of false or misleading information.

CONCLUSIONS

Limited research is available on chatbots in adolescent nutrition and physical activity interventions, finding insufficient evidence on the acceptability and feasibility of chatbots in the adolescent population. Similarly, adolescent consultation identified issues in the design features that have not been mentioned in the published literature. Therefore, chatbot codesign with adolescents may help ensure that such technology is feasible and acceptable to an adolescent population.

Developing and Testing a Chatbot to Integrate HIV Education Into Family Planning Clinic Waiting Areas in Lusaka, Zambia

BACKGROUND

To maximize protection against both unintended pregnancy and HIV, it is important that family planning (FP) services integrate HIV counseling, both to support method choice and identify potential HIV services of interest, such as pre-exposure prophylaxis (PrEP). However, FP providers often lack sufficient time and knowledge to address HIV vulnerability with clients. To potentially offload some of the initial HIV counseling burden from FP providers, we developed and tested a chatbot that provided information about HIV and dual protection to FP clients in waiting areas of FP clinics in Lusaka, Zambia.

CHATBOT DEVELOPMENT

We drafted a scripted conversation and tested it in English in formative workshops with Zambian women between the ages of 15 and 49 years. After translating the content to Bemba and Nyanja, we conducted a second round of workshops to validate the translations, before uploading the content into the chatbot platform.

CHATBOT USER TEST

Thirty volunteers tested the chatbot in 3 Lusaka FP clinics, completing an exit survey to provide feedback. A large majority (83%) said they learned new HIV information from the chatbot. Twenty (67%) learned about PrEP for the first time through the chat. Most (96%) reported discussing HIV with the provider, after engaging with the chatbot. In response to an open-ended question, several testers volunteered that they wanted to learn more about PrEP.

CONCLUSIONS

Pre-consultation waiting-area time is an underutilized opportunity to impart HIV information to FP clients, thereby preparing them to discuss their dual HIV and pregnancy prevention needs when they see their providers. FP clients expressed particular interest in learning more about PrEP, underscoring the importance of integrating HIV into FP services.

How Far Can Conversational Agents Contribute to IBD Patient Health Care—A Review of the Literature

Modern societies are facing health and healthcare challenges as never seen before. The digital world in which we are living today considers digital health interventions such as “internet-delivered” therapy (e-Therapy) or mobile apps as an integrated part of healthcare systems. Digital transformation in health care requires the active involvement of patients as the central part of healthcare interventions. In the case of chronic health conditions, such as inflammatory bowel disease (IBD), it is believed that the adoption of new digital tools helps to maintain and extend the health and care of patients, optimizing the course of the treatment of the disease. The study goal was to undertake a literature review associating the use of chatbot technology with IBD patients' health care. This study intends to support digital product developments, mainly chatbot for IBD or other chronic diseases. The work was carried out through two literature review phases. The first one was based on a systematic approach and the second was a scoping review focused only on Frontiers Journals. This review followed a planned protocol for search and selection strategy that was created by a research team discussion. Chatbot technology for chronic disease self-management can have high acceptance and usability levels. The more interaction with a chatbot, the more patients are able to increase their self-care practice, but there is a challenge. The chatbot ontology to personalize the communication still needed to have strong guidelines helping other researchers to define which Electronic Medical Records (EMRs) should be used in the chatbots to improve the user satisfaction, engagement, and dialog quality. The literature review showed us both evidence and success of these tools in other health disorders. Some of them revealed a huge potential for conversational agents as a part of digital health interventions.

Nonusage Attrition of Adolescents in an mHealth Promotion Intervention and the Role of Socioeconomic Status: Secondary Analysis of a 2-Arm Cluster-Controlled Trial

Background

Mobile health (mHealth) interventions may help adolescents adopt healthy lifestyles. However, attrition in these interventions is high. Overall, there is a lack of research on nonusage attrition in adolescents, particularly regarding the role of socioeconomic status (SES).

Objective

The aim of this study was to focus on the role of SES in the following three research questions (RQs): When do adolescents stop using an mHealth intervention (RQ1)? Why do they report nonusage attrition (RQ2)? Which intervention components (ie, self-regulation component, narrative, and chatbot) prevent nonusage attrition among adolescents (RQ3)?

Methods

A total of 186 Flemish adolescents (aged 12-15 years) participated in a 12-week mHealth program. Log data were monitored to measure nonusage attrition and usage duration for the 3 intervention components. A web-based questionnaire was administered to assess reasons for attrition. A survival analysis was conducted to estimate the time to attrition and determine whether this differed according to SES (RQ1). Descriptive statistics were performed to map the attrition reasons, and Fisher exact tests were used to determine if these reasons differed depending on the educational track (RQ2). Mixed effects Cox proportional hazard regression models were used to estimate the associations between the use duration of the 3 components during the first week and attrition. An interaction term was added to the regression models to determine whether associations differed by the educational track (RQ3).

Results

After 12 weeks, 95.7% (178/186) of the participants stopped using the app. 30.1% (56/186) of the adolescents only opened the app on the installation day, and 44.1% (82/186) stopped using the app in the first week. Attrition at any given time during the intervention period was higher for adolescents from the nonacademic educational track compared with those from the academic track. The other SES indicators (family affluence and perceived financial situation) did not explain attrition. The most common reasons for nonusage attrition among participants were perceiving that the app did not lead to behavior change, not liking the app, thinking that they already had a sufficiently healthy lifestyle, using other apps, and not being motivated by the environment. Attrition reasons did not differ depending on the educational track. More time spent in the self-regulation and narrative components during the first week was associated with lower attrition, whereas chatbot use duration was not associated with attrition rates. No moderating effects of SES were observed in the latter association.

Conclusions

Nonusage attrition was high, especially among adolescents in the nonacademic educational track. The reported reasons for attrition were diverse, with no statistical differences according to the educational level. The duration of the use of the self-regulation and narrative components during the first week may prevent attrition for both educational tracks.

Trial Registration

ClinicalTrials.gov NCT04719858; http://clinicaltrials.gov/ct2/show/NCT04719858