Emotional Reactions and Likelihood of Response to Questions Designed for a Mental Health Chatbot Among Adolescents: Experimental Study

Effectiveness of mental health chatbots in depression and anxiety for adolescents and young adults: a meta-analysis of randomized controlled trials

BACKGROUND

The mental health chatbot is dedicated to providing assistance to individuals grappling with the complexities of depression and anxiety.

OBJECTIVE

The study aimed to evaluate the effectiveness of the mental health chatbot in alleviating symptoms of depression and anxiety among adolescents and young adults.

METHODS

A systematic review framework was employed with a protocol pre-registered on Prospero (CRD42023418877). Databases were systematically searched, including PubMed, ACM Digital Library, Embase, Cochrane and IEEE. Data synthesis was conducted narratively, and meta-analysis was performed by pooling data from the original studies.

RESULTS

Ten randomized controlled trials focused on an acute population, mainly females and university students. Chatbots designed for daily conversations and mood monitoring, using cognitive behavioral therapy techniques, showed efficacy in treating depression (95% CI = -1.09 to -0.23; p = .003). However, it is essential to highlight that these interventions utilizing chatbots for mental health were not found to be efficacious in managing symptoms of anxiety (95% CI = -0.56 to 0.4; p = .74).

CONCLUSIONS

Evidence supports the effectiveness of mental health chatbots in treating depression, but further exploration and refinement are needed to optimize their efficacy in managing anxiety.

Current Landscape and Future Directions for Mental Health Conversational Agents for Youth: Scoping Review

BACKGROUND

Conversational agents (CAs; chatbots) are systems with the ability to interact with users using natural human dialogue. They are increasingly used to support interactive knowledge discovery of sensitive topics such as mental health topics. While much of the research on CAs for mental health has focused on adult populations, the insights from such research may not apply to CAs for youth.

OBJECTIVE

This study aimed to comprehensively evaluate the state-of-the-art research on mental health CAs for youth.

METHODS

Following PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines, we identified 39 peer-reviewed studies specific to mental health CAs designed for youth across 4 databases, including ProQuest, Scopus, Web of Science, and PubMed. We conducted a scoping review of the literature to evaluate the characteristics of research on mental health CAs designed for youth, the design and computational considerations of mental health CAs for youth, and the evaluation outcomes reported in the research on mental health CAs for youth.

RESULTS

We found that many mental health CAs (11/39, 28%) were designed as older peers to provide therapeutic or educational content to promote youth mental well-being. All CAs were designed based on expert knowledge, with a few that incorporated inputs from youth. The technical maturity of CAs was in its infancy, focusing on building prototypes with rule-based models to deliver prewritten content, with limited safety features to respond to imminent risk. Research findings suggest that while youth appreciate the 24/7 availability of friendly or empathetic conversation on sensitive topics with CAs, they found the content provided by CAs to be limited. Finally, we found that most (35/39, 90%) of the reviewed studies did not address the ethical aspects of mental health CAs, while youth were concerned about the privacy and confidentiality of their sensitive conversation data.

CONCLUSIONS

Our study highlights the need for researchers to continue to work together to align evidence-based research on mental health CAs for youth with lessons learned on how to best deliver these technologies to youth. Our review brings to light mental health CAs needing further development and evaluation. The new trend of large language model-based CAs can make such technologies more feasible. However, the privacy and safety of the systems should be prioritized. Although preliminary evidence shows positive trends in mental health CAs, long-term evaluative research with larger sample sizes and robust research designs is needed to validate their efficacy. More importantly, collaboration between youth and clinical experts is essential from the early design stages through to the final evaluation to develop safe, effective, and youth-centered mental health chatbots. Finally, best practices for risk mitigation and ethical development of CAs with and for youth are needed to promote their mental well-being.

Development of Chatbot-Based Oral Health Care for Young Children and Evaluation of its Effectiveness, Usability, and Acceptability: Mixed Methods Study

Background

Chatbots are increasingly accepted in public health for their ability to replicate human-like communication and provide scalable, 24/7 services. The high prevalence of dental caries in children underscores the need for early and effective intervention.

Objective

This study aimed to develop the 30-Day FunDee chatbot and evaluate its effectiveness, usability, and acceptability in delivering oral health education to caregivers of children aged 6 to 36 months.

Methods

The chatbot was created using the artificial intelligence (AI) chatbot behavior change model, integrating behavioral change theories into content designed for 3-5 minutes of daily use over 30 days. A pre-post experimental study was conducted from December 2021 to February 2022 in Hat Yai District, Songkhla Province, and Maelan District, Pattani Province, Thailand. Fifty-eight caregivers completed a web-based structured questionnaire at baseline and 2 months post baseline to evaluate knowledge, protection motivation theory-based perceptions, and tooth-brushing practices. Usability was assessed via chatbot logfiles and a web-based questionnaire at 2 months post baseline. Acceptability was evaluated through three methods: (1) open-ended chatbot interactions on day 30, (2) a web-based structured questionnaire at 2 months post baseline, and (3) semistructured telephone interviews with 15 participants 2 weeks post intervention. Participants for interviews were stratified by adherence levels and randomly selected from Hatyai and Maelan districts. All self-reported variables were measured on a 5-point Likert scale (1=lowest, 5=highest).

Results

The chatbot was successfully developed based on the 4 components of the AI chatbot behavior change model. Participants had a mean age of 34.5 (SD 8.6) years. The frequency of tooth brushing among caregivers significantly improved, increasing from 72.4% at baseline to 93.1% two months post baseline (P=.006). Protection motivation theory-based perceptions also showed significant improvement, with mean scores rising from 4.0 (SD 0.6) at baseline to 4.5 (SD 0.6) two months post baseline (P<.001). The chatbot received high ratings for satisfaction (4.7/5, SD 0.6) and usability (4.7/5, SD 0.5). Participants engaged with the chatbot for an average of 24.7 (SD 7.2) days out of 30. Caregivers praised the chatbot's content quality, empathetic communication, and multimedia design, but noted the intervention's lengthy duration and messaging system as limitations.

Conclusions

The 30-Day FunDee chatbot effectively enhanced caregivers' perceptions of oral health care and improved tooth-brushing practices for children aged 6-36 months. High user satisfaction and engagement demonstrate its potential as an innovative tool for oral health education. These findings warrant further validation through large-scale, randomized controlled trials.

eHealth Communication Intervention to Promote Human Papillomavirus Vaccination Among Middle-School Girls: Development and Usability Study

BACKGROUND

As the age of initiating sexual intercourse has gradually decreased among South Korean adolescents, earlier vaccination of adolescents for human papillomavirus (HPV) is necessary before their exposure to HPV. Health communication includes "cues to action" that lead to preventive health behaviors, and recently, social networking services, which operate with fewer time and space constraints, have been used in various studies as a form of eHealth communication.

OBJECTIVE

This study aims to investigate the feasibility and usability of an eHealth communication intervention for HPV vaccination in middle-school girls aimed at the girls and their mothers.

METHODS

The eHealth communication intervention for HPV vaccination was developed using a 6-step intervention mapping process: needs assessments, setting program outcomes, selection of a theory-based method and practical strategies, development of the intervention, implementation plan, and testing the validity of the intervention.

RESULTS

A review of 10 studies identified effective health communication messages, delivery methods, and theories for HPV vaccination among adolescents. Barriers including low knowledge, perceived threat, and the inconvenience of taking 2 doses of the vaccine were identified through focus groups, suggesting a need for youth-friendly and easy-to-understand information for adolescents delivered via mobile phones. The expected outcomes and the performance objectives are specifically tailored to reflect the vaccination intention. Behavior change techniques were applied using trusted sources and a health belief model. Health messages delivered through a KakaoTalk chatbot improved awareness and self-efficacy. Quality control was ensured with the use of a log system. The experts' chatbot usability average score was 80.13 (SD 8.15) and the average score of girls was 84.06 (SD 7.61).

CONCLUSIONS

Future studies need to verify the effectiveness of health communication strategies in promoting HPV vaccination and the effectiveness of scientific intervention using a chatbot as a delivery method for the intervention.

A necessary conversation to develop chatbots for HIV studies: qualitative findings from research staff, community advisory board members, and study participants

Chatbots increase business productivity by handling customer conversations instead of human agents. Similar rationale applies to use chatbots in the healthcare sector, especially for health coaches who converse with clients. Chatbots are nascent in healthcare. Study findings have been mixed in terms of engagement and their impact on outcomes. Questions remain as to chatbot acceptability with coaches and other providers; studies have focused on clients.To clarify perceived benefits of chatbots in HIV interventions we conducted virtual focus groups with 13 research staff, eight community advisory board members, and seven young adults who were HIV intervention trial participants (clients). Our HIV healthcare context is important. Clients represent a promising age demographic for chatbot uptake. They are a marginalized population warranting consideration to avoid technology that limits healthcare access.Focus group participants expressed the value of chatbots for HIV research staff and clients. Staff discussed how chatbot functions, such as automated appointment scheduling and service referrals, could reduce workloads while clients discussed the after-hours convenience of these functions. Participants also emphasized that chatbots should provide relatable conversation, reliable functionality, and would not be appropriate for all clients. Our findings underscore the need to further examine appropriate chatbot functionality in HIV interventions.

Promoting mental health in children and adolescents through digital technology: a systematic review and meta-analysis

Background

The increasing prevalence of mental health issues among children and adolescents has prompted a growing number of researchers and practitioners to explore digital technology interventions, which offer convenience, diversity, and proven effectiveness in addressing such problems. However, the existing literature reveals a significant gap in comprehensive reviews that consolidate findings and discuss the potential of digital technologies in enhancing mental health.

Methods

To clarify the latest research progress on digital technology to promote mental health in the past decade (2013-2023), we conducted two studies: a systematic review and meta-analysis. The systematic review is based on 59 empirical studies identified from three screening phases, with basic information, types of technologies, types of mental health issues as key points of analysis for synthesis and comparison. The meta-analysis is conducted with 10 qualified experimental studies to determine the overall effect size of digital technology interventions and possible moderating factors.

Results

The results revealed that (1) there is an upward trend in relevant research, comprising mostly experimental and quasi-experimental designs; (2) the common mental health issues include depression, anxiety, bullying, lack of social emotional competence, and mental issues related to COVID-19; (3) among the various technological interventions, mobile applications (apps) have been used most frequently in the diagnosis and treatment of mental issues, followed by virtual reality, serious games, and telemedicine services; and (4) the meta-analysis results indicated that digital technology interventions have a moderate and significant effect size (g = 0.43) for promoting mental health.

Conclusion

Based on these findings, this study provides guidance for future practice and research on the promotion of adolescent mental health through digital technology.

Systematic review registration

https://inplasy.com/inplasy-2023-12-0004/, doi: 10.37766/inplasy2023.12.0004.

An Integrative Survey on Mental Health Conversational Agents to Bridge Computer Science and Medical Perspectives

Mental health conversational agents (a.k.a. chatbots) are widely studied for their potential to offer accessible support to those experiencing mental health challenges. Previous surveys on the topic primarily consider papers published in either computer science or medicine, leading to a divide in understanding and hindering the sharing of beneficial knowledge between both domains. To bridge this gap, we conduct a comprehensive literature review using the PRISMA framework, reviewing 534 papers published in both computer science and medicine. Our systematic review reveals 136 key papers on building mental health-related conversational agents with diverse characteristics of modeling and experimental design techniques. We find that computer science papers focus on LLM techniques and evaluating response quality using automated metrics with little attention to the application while medical papers use rule-based conversational agents and outcome metrics to measure the health outcomes of participants. Based on our findings on transparency, ethics, and cultural heterogeneity in this review, we provide a few recommendations to help bridge the disciplinary divide and enable the cross-disciplinary development of mental health conversational agents.

Assessing Mood With the Identifying Depression Early in Adolescence Chatbot (IDEABot): Development and Implementation Study

BACKGROUND

Mental health status assessment is mostly limited to clinical or research settings, but recent technological advances provide new opportunities for measurement using more ecological approaches. Leveraging apps already in use by individuals on their smartphones, such as chatbots, could be a useful approach to capture subjective reports of mood in the moment.

OBJECTIVE

This study aimed to describe the development and implementation of the Identifying Depression Early in Adolescence Chatbot (IDEABot), a WhatsApp-based tool designed for collecting intensive longitudinal data on adolescents' mood.

METHODS

The IDEABot was developed to collect data from Brazilian adolescents via WhatsApp as part of the Identifying Depression Early in Adolescence Risk Stratified Cohort (IDEA-RiSCo) study. It supports the administration and collection of self-reported structured items or questionnaires and audio responses. The development explored WhatsApp's default features, such as emojis and recorded audio messages, and focused on scripting relevant and acceptable conversations. The IDEABot supports 5 types of interactions: textual and audio questions, administration of a version of the Short Mood and Feelings Questionnaire, unprompted interactions, and a snooze function. Six adolescents (n=4, 67% male participants and n=2, 33% female participants) aged 16 to 18 years tested the initial version of the IDEABot and were engaged to codevelop the final version of the app. The IDEABot was subsequently used for data collection in the second- and third-year follow-ups of the IDEA-RiSCo study.

RESULTS

The adolescents assessed the initial version of the IDEABot as enjoyable and made suggestions for improvements that were subsequently implemented. The IDEABot's final version follows a structured script with the choice of answer based on exact text matches throughout 15 days. The implementation of the IDEABot in 2 waves of the IDEA-RiSCo sample (140 and 132 eligible adolescents in the second- and third-year follow-ups, respectively) evidenced adequate engagement indicators, with good acceptance for using the tool (113/140, 80.7% and 122/132, 92.4% for second- and third-year follow-up use, respectively), low attrition (only 1/113, 0.9% and 1/122, 0.8%, respectively, failed to engage in the protocol after initial interaction), and high compliance in terms of the proportion of responses in relation to the total number of elicited prompts (12.8, SD 3.5; 91% out of 14 possible interactions and 10.57, SD 3.4; 76% out of 14 possible interactions, respectively).

CONCLUSIONS

The IDEABot is a frugal app that leverages an existing app already in daily use by our target population. It follows a simple rule-based approach that can be easily tested and implemented in diverse settings and possibly diminishes the burden of intensive data collection for participants by repurposing WhatsApp. In this context, the IDEABot appears as an acceptable and potentially scalable tool for gathering momentary information that can enhance our understanding of mood fluctuations and development.

"A Great Way to Start the Conversation": Evidence for the Use of an Adolescent Mental Health Chatbot Navigator for Youth at Risk of HIV and Other STIs

Chatbot use is increasing for mobile health interventions on sensitive and stigmatized topics like mental health because of their anonymity and privacy. This anonymity provides acceptability to sexual and gendered minority youth (ages 16-24) at increased risk of HIV and other STIs with poor mental health due to higher levels of stigma, discrimination, and social isolation. This study evaluates the usability of Tabatha-YYC, a pilot chatbot navigator created to link these youth to mental health resources. Tabatha-YYC was developed using a Youth Advisory Board (n = 7). The final design underwent user testing (n = 20) through a think-aloud protocol, semi-structured interview, and a brief survey post-exposure which included the Health Information Technology Usability Evaluation Scale. The chatbot was found to be an acceptable mental health navigator by participants. This study provides important design methodology considerations and key insights into chatbot design preferences of youth at risk of STIs seeking mental health resources.

A Relational Agent Intervention for Adolescents Seeking Mental Health Treatment: Protocol for a Randomized Controlled Trial

BACKGROUND

Unmet pediatric mental health (MH) needs are growing as rates of pediatric depression and anxiety dramatically increase. Access to care is limited by multiple factors, including a shortage of clinicians trained in developmentally specific, evidence-based services. Novel approaches to MH care delivery, including technology-leveraged and readily accessible options, need to be evaluated in service of expanding evidence-based services to youths and their families. Preliminary evidence supports the use of Woebot, a relational agent that digitally delivers guided cognitive behavioral therapy (CBT) through a mobile app, for adults with MH concerns. However, no studies have evaluated the feasibility and acceptability of such app-delivered relational agents specifically for adolescents with depression and/or anxiety within an outpatient MH clinic, nor compared them to other MH support services.

OBJECTIVE

This paper describes the protocol for a randomized controlled trial evaluating the feasibility and acceptability of an investigational device, Woebot for Adolescents (W-GenZD), within an outpatient MH clinic for youths presenting with depression and/or anxiety. The study's secondary aim will compare the clinical outcomes of self-reported depressive symptoms with W-GenZD and a telehealth-delivered CBT-based skills group (CBT-group). Tertiary aims will evaluate additional clinical outcomes and therapeutic alliance between adolescents in W-GenZD and the CBT-group.

METHODS

Participants include youths aged 13-17 years with depression and/or anxiety seeking care from an outpatient MH clinic at a children's hospital. Eligible youths will have no recent safety concerns or complex comorbid clinical diagnoses; have no concurrent individual therapy; and, if on medications, are on stable doses, based on clinical screening and as well as study-specific criteria.

RESULTS

Recruitment began in May 2022. As of December 8, 2022, we have randomized 133 participants.

CONCLUSIONS

Establishing the feasibility and acceptability of W-GenZD within an outpatient MH clinical setting will add to the field's current understanding of the utility and implementation considerations of this MH care service modality. The study will also evaluate the noninferiority of W-GenZD against the CBT-group. Findings may also have implications for patients, families, and providers looking for additional MH support options for adolescents seeking help for their depression and/or anxiety. Such options expand the menu of supports for youths with lower-intensity needs as well as possibly reduce waitlists and optimize clinician deployment toward more severe cases.

TRIAL REGISTRATION

ClinicalTrials.gov NCT05372913; https://clinicaltrials.gov/ct2/show/NCT05372913.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/44940.

COVID-19 Vaccine Equity and Access: Case Study for Health Care Chatbots

BACKGROUND

Disparities in COVID-19 information and vaccine access have emerged during the pandemic. Individuals from historically excluded communities (eg, Black and Latin American) experience disproportionately negative health outcomes related to COVID-19. Community gaps in COVID-19 education, social, and health care services (including vaccines) should be prioritized as a critical effort to end the pandemic. Misinformation created by the politicization of COVID-19 and related public health measures has magnified the pandemic's challenges, including access to health care, vaccination and testing efforts, as well as personal protective equipment. Information and Communication Technology (ICT) has been demonstrated to reduce the gaps of marginalization in education and access among communities. Chatbots are an increasingly present example of ICTs, particularly in health care and in relation to the COVID-19 pandemic.

OBJECTIVE

This project aimed to (1) follow an inclusive and theoretically driven design process to develop and test a COVID-19 information ICT bilingual (English and Spanish) chatbot tool named "Ana" and (2) characterize and evaluate user experiences of these innovative technologies.

METHODS

Ana was developed following a multitheoretical framework, and the project team was comprised of public health experts, behavioral scientists, community members, and medical team. A total of 7 iterations of ß chatbots were tested, and a total of 22 ß testers participated in this process. Content was curated primarily to provide users with factual answers to common questions about COVID-19. To ensure relevance of the content, topics were driven by community concerns and questions, as ascertained through research. Ana's repository of educational content was based on national and international organizations as well as interdisciplinary experts. In the context of this development and pilot project, we identified an evaluation framework to explore reach, engagement, and satisfaction.

RESULTS

A total of 626 community members used Ana from August 2021 to March 2022. Among those participants, 346 used the English version, with an average of 43 users per month; and 280 participants used the Spanish version, with an average of 40 users monthly. Across all users, 63.87% (n=221) of English users and 22.14% (n=62) of Spanish users returned to use Ana at least once; 18.49% (n=64) among the English version users and 18.57% (n=52) among the Spanish version users reported their ranking. Positive ranking comprised the "smiley" and "loved" emojis, and negative ranking comprised the "neutral," "sad," and "mad" emojis. When comparing negative and positive experiences, the latter was higher across Ana's platforms (English: n=41, 64.06%; Spanish: n=41, 77.35%) versus the former (English: n=23, 35.93%; Spanish: n=12, 22.64%).

CONCLUSIONS

This pilot project demonstrated the feasibility and capacity of an innovative ICT to share COVID-19 information within diverse communities. Creating a chatbot like Ana with bilingual content contributed to an equitable approach to address the lack of accessible COVID-19-related information.

Chatbots to Support Mental Wellbeing of People Living in Rural Areas: Can User Groups Contribute to Co-design?

Digital technologies such as chatbots can be used in the field of mental health. In particular, chatbots can be used to support citizens living in sparsely populated areas who face problems such as poor access to mental health services, lack of 24/7 support, barriers to engagement, lack of age appropriate support and reductions in health budgets. The aim of this study was to establish if user groups can design content for a chatbot to support the mental wellbeing of individuals in rural areas. University students and staff, mental health professionals and mental health service users (N = 78 total) were recruited to workshops across Northern Ireland, Ireland, Scotland, Finland and Sweden. The findings revealed that participants wanted a positive chatbot that was able to listen, support, inform and build a rapport with users. Gamification could be used within the chatbot to increase user engagement and retention. Content within the chatbot could include validated mental health scales and appropriate response triggers, such as signposting to external resources should the user disclose potentially harmful information or suicidal intent. Overall, the workshop participants identified user needs which can be transformed into chatbot requirements. Responsible design of mental healthcare chatbots should consider what users want or need, but also what chatbot features artificial intelligence can competently facilitate and which features mental health professionals would endorse.

Pharmacogenomic Clinical Decision Support: A Review, How-to Guide, and Future Vision

Clinical decision support (CDS) is an essential part of any pharmacogenomics (PGx) implementation. Increasingly, institutions have implemented CDS tools in the clinical setting to bring PGx data into patient care, and several have published their experiences with these implementations. However, barriers remain that limit the ability of some programs to create CDS tools to fit their PGx needs. Therefore, the purpose of this review is to summarize the types, functions, and limitations of PGx CDS currently in practice. Then, we provide an approachable step‐by‐step how‐to guide with a case example to help implementers bring PGx to the front lines of care regardless of their setting. Particular focus is paid to the five “rights” of CDS as a core around designing PGx CDS tools. Finally, we conclude with a discussion of opportunities and areas of growth for PGx CDS.