Perceptions and Opinions of Patients About Mental Health Chatbots: Scoping Review

Rates and correlates of study enrolment and use of a chatbot aimed to promote mental health services use for eating disorders following online screening

OBJECTIVE

We developed a chatbot aimed to facilitate mental health services use for eating disorders (EDs) and offered the opportunity to enrol in a research study and use the chatbot to all adult respondents to a publicly available online ED screen who screened positive for clinical/subclinical EDs and reported not currently being in treatment. We examined the rates and correlates of enrolment in the study and uptake of the chatbot.

METHOD

Following screening, eligible respondents (≥18 years, screened positive for a clinical/subclinical ED, not in treatment for an ED) were shown the study opportunity. Chi-square tests and logistic regressions explored differences in demographics, ED symptoms, suicidality, weight, and probable ED diagnoses between those who enroled and engaged with the chatbot versus those who did not.

RESULTS

6747 respondents were shown the opportunity (80.0% of all adult screens). 3.0% enroled, of whom 90.2% subsequently used the chatbot. Enrolment and chatbot uptake were more common among respondents aged ≥25 years old versus those aged 18-24 and less common among respondents who reported engaging in regular dietary restriction.

CONCLUSIONS

Overall enrolment was low, yet uptake was high among those that enroled and did not differ across most demographics and symptom presentations. Future directions include evaluating respondents' attitudes towards treatment-promoting tools and removing barriers to uptake.

Feasibility, usability, and acceptability of MobileCoach-Teen: A smartphone app-based preventative intervention for risky adolescent drinking behavior

BACKGROUND

Older adolescence (ages 15-18) is a critical period for experimentation with substance use, especially alcohol. Adolescent drinking poses hazards to physical and mental health, amplifies risk associated with other activities typically initiated during this life stage (e.g., driving, sexual activity), and is associated with adverse outcomes in adolescence and adulthood. Existing preventative interventions are expensive and have questionable long-term efficacy. Digital interventions may represent an accessible and personalized approach to providing preventative intervention content to youth.

METHODS

This study recruited 29 adolescents aged 16-18 (M = 17.24, SD = 0.74) for a pilot feasibility trial of the MobileCoach-Teen (MC-Teen) smartphone app-based intervention. The study team randomized participants to receive either the alcohol intervention (MC-Teen) or attention control pseudo-intervention (MC-Fit). MC-Teen participants received 12 weeks of content adapted from a prior Swiss-based trial of a preventative alcohol intervention. Participants provided qualitative and quantitative feedback at baseline, via six biweekly surveys during and post-intervention.

RESULTS

Both groups rated the application as easy to download (M = 4.31, SD = 0.93; 5-point Likert). All participants completed the baseline survey in less than the estimated time of 10 min (M = 7:42, SD = 2:15) and rated the survey as easy to complete (M = 4.69, SD = 0.60; 5-point Likert). MC-Teen participants favorably assessed application user experience, message user experience, and digital working alliance with application. Qualitative themes included a desire for increased rate/amount and diversity of content, greater representation via coach options, user interface/user experience improvements, and additional features.

CONCLUSION

The MC-Teen intervention is feasible and acceptable based on a pilot feasibility trial with a sample of U.S. adolescents.

Usability and Acceptability of a Conversational Agent Health Education App (Nthabi) for Young Women in Lesotho: Quantitative Study

BACKGROUND

Young women in Lesotho face myriad sexual and reproductive health problems. There is little time to provide health education to women in low-resource settings with critical shortages of human resources for health.

OBJECTIVE

This study aims to determine the acceptability and usability of a conversational agent system, the Nthabi health promotion app, which was culturally adapted for use in Lesotho.

METHODS

We conducted a descriptive quantitative study, using a 22-item Likert scale survey to assess the perceptions of the usability and acceptability of 172 young women aged 18-28 years in rural districts of Lesotho, who used the system on either smartphones or tablets for up to 6 weeks. Descriptive statistics were used to calculate the averages and frequencies of the variables. χ2 tests were used to determine any associations among variables.

RESULTS

A total of 138 participants were enrolled and completed the survey. The mean age was 22 years, most were unmarried, 56 (40.6%) participants had completed high school, 39 (28.3%) participants were unemployed, and 88 (63.8%) participants were students. Respondents believed the app was helpful, with 134 (97.1%) participants strongly agreeing or agreeing that the app was "effective in helping them make decisions" and "could quickly improve health education and counselling." In addition, 136 (98.5%) participants strongly agreed or agreed that the app was "simple to use," 130 (94.2 %) participants reported that Nthabi could "easily repeat words that were not well understood," and 128 (92.7%) participants reported that the app "could quickly load the information on the screen." Respondents were generally satisfied with the app, with 132 (95.6%) participants strongly agreeing or agreeing that the health education content delivered by the app was "well organised and delivered in a timely way," while 133 (96.4%) participants "enjoyed using the interface." They were satisfied with the cultural adaptation, with 133 (96.4%) participants strongly agreeing or agreeing that the app was "culturally appropriate and that it could be easily shared with a family or community members." They also reported that Nthabi was worthwhile, with 127 (92%) participants reporting that they strongly agreed or agreed that they were "satisfied with the application and intended to continue using it," while 135 (97.8%) participants would "encourage others to use it." Participants aged 18-24 years (vs those aged 25-28 years) agreed that the "Nthabi app was simple to use" (106/106, 100% vs 30/32, 98.8%; P=.01), and agreed that "the educational content was well organised and delivered in a timely way" (104/106, 98.1% vs 28/32, 87.5%; P=.01).

CONCLUSIONS

These results support further study of conversational agent systems as alternatives to traditional face-to-face provision of health education services in Lesotho, where there are critical shortages of human resources for health.

TRIAL REGISTRATION

ClinicalTrials.gov NCT04354168; https://www.clinicaltrials.gov/study/NCT04354168.

The Effects of Artificial Intelligence Chatbots on Women's Health: A Systematic Review and Meta-Analysis

PURPOSE

This systematic review and meta-analysis aimed to investigate the effects of artificial intelligence chatbot interventions on health outcomes in women.

METHODS

Ten relevant studies published between 2019 and 2023 were extracted from the PubMed, Cochrane Library, EMBASE, CINAHL, and RISS databases in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. This review focused on experimental studies concerning chatbot interventions in women's health. The literature was assessed using the ROB 2 quality appraisal checklist, and the results were visualized with a risk-of-bias visualization program.

RESULTS

This review encompassed seven randomized controlled trials and three single-group experimental studies. Chatbots were effective in addressing anxiety, depression, distress, healthy relationships, cancer self-care behavior, preconception intentions, risk perception in eating disorders, and gender attitudes. Chatbot users experienced benefits in terms of internalization, acceptability, feasibility, and interaction. A meta-analysis of three studies revealed significant effects in reducing anxiety (I2 = 0%, Q = 8.10, p < 0.017), with an effect size of -0.30 (95% CI, -0.42 to -0.18).

CONCLUSIONS

Artificial intelligence chatbot interventions had positive effects on physical, physiological, and cognitive health outcomes. Using chatbots may represent pivotal nursing interventions for female populations to improve health status and support women socially as a form of digital therapy.

Implementation of Chatbot Technology in Health Care: Protocol for a Bibliometric Analysis

BACKGROUND

Chatbots have the potential to increase people's access to quality health care. However, the implementation of chatbot technology in the health care system is unclear due to the scarce analysis of publications on the adoption of chatbot in health and medical settings.

OBJECTIVE

This paper presents a protocol of a bibliometric analysis aimed at offering the public insights into the current state and emerging trends in research related to the use of chatbot technology for promoting health.

METHODS

In this bibliometric analysis, we will select published papers from the databases of CINAHL, IEEE Xplore, PubMed, Scopus, and Web of Science that pertain to chatbot technology and its applications in health care. Our search strategy includes keywords such as "chatbot," "virtual agent," "virtual assistant," "conversational agent," "conversational AI," "interactive agent," "health," and "healthcare." Five researchers who are AI engineers and clinicians will independently review the titles and abstracts of selected papers to determine their eligibility for a full-text review. The corresponding author (ZN) will serve as a mediator to address any discrepancies and disputes among the 5 reviewers. Our analysis will encompass various publication patterns of chatbot research, including the number of annual publications, their geographic or institutional distribution, and the number of annual grants supporting chatbot research, and further summarize the methodologies used in the development of health-related chatbots, along with their features and applications in health care settings. Software tool VOSViewer (version 1.6.19; Leiden University) will be used to construct and visualize bibliometric networks.

RESULTS

The preparation for the bibliometric analysis began on December 3, 2021, when the research team started the process of familiarizing themselves with the software tools that may be used in this analysis, VOSViewer and CiteSpace, during which they consulted 3 librarians at the Yale University regarding search terms and tentative results. Tentative searches on the aforementioned databases yielded a total of 2340 papers. The official search phase started on July 27, 2023. Our goal is to complete the screening of papers and the analysis by February 15, 2024.

CONCLUSIONS

Artificial intelligence chatbots, such as ChatGPT (OpenAI Inc), have sparked numerous discussions within the health care industry regarding their impact on human health. Chatbot technology holds substantial promise for advancing health care systems worldwide. However, developing a sophisticated chatbot capable of precise interaction with health care consumers, delivering personalized care, and providing accurate health-related information and knowledge remain considerable challenges. This bibliometric analysis seeks to fill the knowledge gap in the existing literature on health-related chatbots, entailing their applications, the software used in their development, and their preferred functionalities among users.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

PRR1-10.2196/54349.

Assessing the Effectiveness of ChatGPT in Delivering Mental Health Support: A Qualitative Study

Background

Artificial Intelligence (AI) applications are widely researched for their potential in effectively improving the healthcare operations and disease management. However, the research trend shows that these applications also have significant negative implications on the service delivery.

Purpose

To assess the use of ChatGPT for mental health support.

Methods

Due to the novelty and unfamiliarity of the ChatGPT technology, a quasi-experimental design was chosen for this study. Outpatients from a public hospital were included in the sample. A two-week experiment followed by semi-structured interviews was conducted in which participants used ChatGPT for mental health support. Semi-structured interviews were conducted with 24 individuals with mental health conditions.

Results

Eight positive factors (psychoeducation, emotional support, goal setting and motivation, referral and resource information, self-assessment and monitoring, cognitive behavioral therapy, crisis interventions, and psychotherapeutic exercises) and four negative factors (ethical and legal considerations, accuracy and reliability, limited assessment capabilities, and cultural and linguistic considerations) were associated with the use of ChatGPT for mental health support.

Conclusion

It is important to carefully consider the ethical, reliability, accuracy, and legal challenges and develop appropriate strategies to mitigate them in order to ensure safe and effective use of AI-based applications like ChatGPT in mental health support.

Loneliness and suicide mitigation for students using GPT3-enabled chatbots

Mental health is a crisis for learners globally, and digital support is increasingly seen as a critical resource. Concurrently, Intelligent Social Agents receive exponentially more engagement than other conversational systems, but their use in digital therapy provision is nascent. A survey of 1006 student users of the Intelligent Social Agent, Replika, investigated participants' loneliness, perceived social support, use patterns, and beliefs about Replika. We found participants were more lonely than typical student populations but still perceived high social support. Many used Replika in multiple, overlapping ways-as a friend, a therapist, and an intellectual mirror. Many also held overlapping and often conflicting beliefs about Replika-calling it a machine, an intelligence, and a human. Critically, 3% reported that Replika halted their suicidal ideation. A comparative analysis of this group with the wider participant population is provided.

Rosie, a Health Education Question-and-Answer Chatbot for New Mothers: Randomized Pilot Study

BACKGROUND

Stark disparities exist in maternal and child outcomes and there is a need to provide timely and accurate health information.

OBJECTIVE

In this pilot study, we assessed the feasibility and acceptability of a health chatbot for new mothers of color.

METHODS

Rosie, a question-and-answer chatbot, was developed as a mobile app and is available to answer questions about pregnancy, parenting, and child development. From January 9, 2023, to February 9, 2023, participants were recruited using social media posts and through engagement with community organizations. Inclusion criteria included being aged ≥14 years, being a woman of color, and either being currently pregnant or having given birth within the past 6 months. Participants were randomly assigned to the Rosie treatment group (15/29, 52% received the Rosie app) or control group (14/29, 48% received a children's book each month) for 3 months. Those assigned to the treatment group could ask Rosie questions and receive an immediate response generated from Rosie's knowledgebase. Upon detection of a possible health emergency, Rosie sends emergency resources and relevant hotline information. In addition, a study staff member, who is a clinical social worker, reaches out to the participant within 24 hours to follow up. Preintervention and postintervention tests were completed to qualitatively and quantitatively evaluate Rosie and describe changes across key health outcomes, including postpartum depression and the frequency of emergency room visits. These measurements were used to inform the clinical trial's sample size calculations.

RESULTS

Of 41 individuals who were screened and eligible, 31 (76%) enrolled and 29 (71%) were retained in the study. More than 87% (13/15) of Rosie treatment group members reported using Rosie daily (5/15, 33%) or weekly (8/15, 53%) across the 3-month study period. Most users reported that Rosie was easy to use (14/15, 93%) and provided responses quickly (13/15, 87%). The remaining issues identified included crashing of the app (8/15, 53%), and users were not satisfied with some of Rosie's answers (12/15, 80%). Mothers in both the Rosie treatment group and control group experienced a decline in depression scores from pretest to posttest periods, but the decline was statistically significant only among treatment group mothers (P=.008). In addition, a low proportion of treatment group infants had emergency room visits (1/11, 9%) compared with control group members (3/13, 23%). Nonetheless, no between-group differences reached statistical significance at P<.05.

CONCLUSIONS

Rosie was found to be an acceptable, feasible, and appropriate intervention for ethnic and racial minority pregnant women and mothers of infants owing to the chatbot's ability to provide a personalized, flexible tool to increase the timeliness and accessibility of high-quality health information to individuals during a period of elevated health risks for the mother and child.

TRIAL REGISTRATION

ClinicalTrials.gov NCT06053515; https://clinicaltrials.gov/study/NCT06053515.

A text-based conversational agent for asthma support: Mixed-methods feasibility study

Objective

Millions of people in the UK have asthma, yet 70% do not access basic care, leading to the largest number of asthma-related deaths in Europe. Chatbots may extend the reach of asthma support and provide a bridge to traditional healthcare. This study evaluates 'Brisa', a chatbot designed to improve asthma patients' self-assessment and self-management.

Methods

We recruited 150 adults with an asthma diagnosis to test our chatbot. Participants were recruited over three waves through social media and a research recruitment platform. Eligible participants had access to 'Brisa' via a WhatsApp or website version for 28 days and completed entry and exit questionnaires to evaluate user experience and asthma control. Weekly symptom tracking, user interaction metrics, satisfaction measures, and qualitative feedback were utilised to evaluate the chatbot's usability and potential effectiveness, focusing on changes in asthma control and self-reported behavioural improvements.

Results

74% of participants engaged with 'Brisa' at least once. High task completion rates were observed: asthma attack risk assessment (86%), voice recording submission (83%) and asthma control tracking (95.5%). Post use, an 8% improvement in asthma control was reported. User satisfaction surveys indicated positive feedback on helpfulness (80%), privacy (87%), trustworthiness (80%) and functionality (84%) but highlighted a need for improved conversational depth and personalisation.

Conclusions

The study indicates that chatbots are effective for asthma support, demonstrated by the high usage of features like risk assessment and control tracking, as well as a statistically significant improvement in asthma control. However, lower satisfaction in conversational flexibility highlights rising expectations for chatbot fluency, influenced by advanced models like ChatGPT. Future health-focused chatbots must balance conversational capability with accuracy and safety to maintain engagement and effectiveness.

Patient Acceptability of Symptom Screening and Patient Education Using a Chatbot for Autoimmune Inflammatory Diseases: Survey Study

BACKGROUND

Chatbots have the potential to enhance health care interaction, satisfaction, and service delivery. However, data regarding their acceptance across diverse patient populations are limited. In-depth studies on the reception of chatbots by patients with chronic autoimmune inflammatory diseases are lacking, although such studies are vital for facilitating the effective integration of chatbots in rheumatology care.

OBJECTIVE

We aim to assess patient perceptions and acceptance of a chatbot designed for autoimmune inflammatory rheumatic diseases (AIIRDs).

METHODS

We administered a comprehensive survey in an outpatient setting at a top-tier rheumatology referral center. The target cohort included patients who interacted with a chatbot explicitly tailored to facilitate diagnosis and obtain information on AIIRDs. Following the RE-AIM (Reach, Effectiveness, Adoption, Implementation and Maintenance) framework, the survey was designed to gauge the effectiveness, user acceptability, and implementation of the chatbot.

RESULTS

Between June and October 2022, we received survey responses from 200 patients, with an equal number of 100 initial consultations and 100 follow-up (FU) visits. The mean scores on a 5-point acceptability scale ranged from 4.01 (SD 0.63) to 4.41 (SD 0.54), indicating consistently high ratings across the different aspects of chatbot performance. Multivariate regression analysis indicated that having a FU visit was significantly associated with a greater willingness to reuse the chatbot for symptom determination (P=.01). Further, patients' comfort with chatbot diagnosis increased significantly after meeting physicians (P<.001). We observed no significant differences in chatbot acceptance according to sex, education level, or diagnosis category.

CONCLUSIONS

This study underscores that chatbots tailored to AIIRDs have a favorable reception. The inclination of FU patients to engage with the chatbot signifies the possible influence of past clinical encounters and physician affirmation on its use. Although further exploration is required to refine their integration, the prevalent positive perceptions suggest that chatbots have the potential to strengthen the bridge between patients and health care providers, thus enhancing the delivery of rheumatology care to various cohorts.

Evaluation of the Current State of Chatbots for Digital Health: Scoping Review

BACKGROUND

Chatbots have become ubiquitous in our daily lives, enabling natural language conversations with users through various modes of communication. Chatbots have the potential to play a significant role in promoting health and well-being. As the number of studies and available products related to chatbots continues to rise, there is a critical need to assess product features to enhance the design of chatbots that effectively promote health and behavioral change.

OBJECTIVE

This scoping review aims to provide a comprehensive assessment of the current state of health-related chatbots, including the chatbots' characteristics and features, user backgrounds, communication models, relational building capacity, personalization, interaction, responses to suicidal thoughts, and users' in-app experiences during chatbot use. Through this analysis, we seek to identify gaps in the current research, guide future directions, and enhance the design of health-focused chatbots.

METHODS

Following the scoping review methodology by Arksey and O'Malley and guided by the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) checklist, this study used a two-pronged approach to identify relevant chatbots: (1) searching the iOS and Android App Stores and (2) reviewing scientific literature through a search strategy designed by a librarian. Overall, 36 chatbots were selected based on predefined criteria from both sources. These chatbots were systematically evaluated using a comprehensive framework developed for this study, including chatbot characteristics, user backgrounds, building relational capacity, personalization, interaction models, responses to critical situations, and user experiences. Ten coauthors were responsible for downloading and testing the chatbots, coding their features, and evaluating their performance in simulated conversations. The testing of all chatbot apps was limited to their free-to-use features.

RESULTS

This review provides an overview of the diversity of health-related chatbots, encompassing categories such as mental health support, physical activity promotion, and behavior change interventions. Chatbots use text, animations, speech, images, and emojis for communication. The findings highlight variations in conversational capabilities, including empathy, humor, and personalization. Notably, concerns regarding safety, particularly in addressing suicidal thoughts, were evident. Approximately 44% (16/36) of the chatbots effectively addressed suicidal thoughts. User experiences and behavioral outcomes demonstrated the potential of chatbots in health interventions, but evidence remains limited.

CONCLUSIONS

This scoping review underscores the significance of chatbots in health-related applications and offers insights into their features, functionalities, and user experiences. This study contributes to advancing the understanding of chatbots' role in digital health interventions, thus paving the way for more effective and user-centric health promotion strategies. This study informs future research directions, emphasizing the need for rigorous randomized control trials, standardized evaluation metrics, and user-centered design to unlock the full potential of chatbots in enhancing health and well-being. Future research should focus on addressing limitations, exploring real-world user experiences, and implementing robust data security and privacy measures.

An Integrative Survey on Mental Health Conversational Agents to Bridge Computer Science and Medical Perspectives

Mental health conversational agents (a.k.a. chatbots) are widely studied for their potential to offer accessible support to those experiencing mental health challenges. Previous surveys on the topic primarily consider papers published in either computer science or medicine, leading to a divide in understanding and hindering the sharing of beneficial knowledge between both domains. To bridge this gap, we conduct a comprehensive literature review using the PRISMA framework, reviewing 534 papers published in both computer science and medicine. Our systematic review reveals 136 key papers on building mental health-related conversational agents with diverse characteristics of modeling and experimental design techniques. We find that computer science papers focus on LLM techniques and evaluating response quality using automated metrics with little attention to the application while medical papers use rule-based conversational agents and outcome metrics to measure the health outcomes of participants. Based on our findings on transparency, ethics, and cultural heterogeneity in this review, we provide a few recommendations to help bridge the disciplinary divide and enable the cross-disciplinary development of mental health conversational agents.

ChatGPT: promise and challenges for deployment in low- and middle-income countries

In low- and middle-income countries (LMICs), the fields of medicine and public health grapple with numerous challenges that continue to hinder patients' access to healthcare services. ChatGPT, a publicly accessible chatbot, has emerged as a potential tool in aiding public health efforts in LMICs. This viewpoint details the potential benefits of employing ChatGPT in LMICs to improve medicine and public health encompassing a broad spectrum of domains ranging from health literacy, screening, triaging, remote healthcare support, mental health support, multilingual capabilities, healthcare communication and documentation, medical training and education, and support for healthcare professionals. Additionally, we also share potential concerns and limitations associated with the use of ChatGPT and provide a balanced discussion on the opportunities and challenges of using ChatGPT in LMICs.

Your robot therapist is not your therapist: understanding the role of AI-powered mental health chatbots

Artificial intelligence (AI)-powered chatbots have the potential to substantially increase access to affordable and effective mental health services by supplementing the work of clinicians. Their 24/7 availability and accessibility through a mobile phone allow individuals to obtain help whenever and wherever needed, overcoming financial and logistical barriers. Although psychological AI chatbots have the ability to make significant improvements in providing mental health care services, they do not come without ethical and technical challenges. Some major concerns include providing inadequate or harmful support, exploiting vulnerable populations, and potentially producing discriminatory advice due to algorithmic bias. However, it is not always obvious for users to fully understand the nature of the relationship they have with chatbots. There can be significant misunderstandings about the exact purpose of the chatbot, particularly in terms of care expectations, ability to adapt to the particularities of users and responsiveness in terms of the needs and resources/treatments that can be offered. Hence, it is imperative that users are aware of the limited therapeutic relationship they can enjoy when interacting with mental health chatbots. Ignorance or misunderstanding of such limitations or of the role of psychological AI chatbots may lead to a therapeutic misconception (TM) where the user would underestimate the restrictions of such technologies and overestimate their ability to provide actual therapeutic support and guidance. TM raises major ethical concerns that can exacerbate one's mental health contributing to the global mental health crisis. This paper will explore the various ways in which TM can occur particularly through inaccurate marketing of these chatbots, forming a digital therapeutic alliance with them, receiving harmful advice due to bias in the design and algorithm, and the chatbots inability to foster autonomy with patients.

Utilizing User Preferences in Designing the AGILE (Accelerating Access to Gender-Based Violence Information and Services Leveraging on Technology Enhanced) Chatbot

INTRODUCTION

Technology advancements have enhanced artificial intelligence, leading to a user shift towards virtual assistants, but a human-centered approach is needed to assess for acceptability and effectiveness. The AGILE chatbot is designed in Kenya with features to redefine the response towards gender-based violence (GBV) among vulnerable populations, including adolescents, young women and men, and sexual and gender minorities, to offer accurate and reliable information among users.

METHODS

We conducted an exploratory qualitative study through focus group discussions (FGDs) targeting 150 participants sampled from vulnerable categories; adolescent girls and boys, young women, young men, and sexual and gender minorities. The FGDs included multiple inquiries to assess knowledge and prior interaction with intelligent conversational assistants to inform the user-centric development of a decision-supportive chatbot and a pilot of the chatbot prototype. Each focus group comprised 9-10 members, and the discussions lasted about two hours to gain qualitative user insights and experiences. We used thematic analysis and drew on grounded theory to analyze the data.

RESULTS

The analysis resulted in 14 salient themes composed of sexual violence, physical violence, emotional violence, intimate partner violence, female genital mutilation, sexual reproductive health, mental health, help-seeking behaviors/where to seek support, who to talk to, and what information they would like, features of the chatbot, access of chatbot, abuse and HIV, family and community conflicts, and information for self-care.

CONCLUSION

Adopting a human-centered approach in designing an effective chatbot with as many human features as possible is crucial in increasing utilization, addressing the gaps presented by marginalized/vulnerable populations, and reducing the current GBV epidemic by moving prevention and response services closer to people in need.

AI-Powered Mental Health Virtual Assistants' Acceptance: An Empirical Study on Influencing Factors Among Generations X, Y, and Z

STUDY PURPOSE

This study aims to analyze various influencing factors among generations X (Gen X), Y (Gen Y), and Z (Gen Z) of artificial intelligence (AI)-powered mental health virtual assistants.

METHODS

A cross-sectional survey design was adopted in this study. The study sample consisted of outpatients diagnosed with various mental health illnesses, such as anxiety, depression, schizophrenia, and behavioral disorders. A survey questionnaire was designed based on the factors (performance expectancy, effort expectancy, social influence, facilitating conditions, and behavioural intention) identified from the unified theory of acceptance and use of the technology model. Ethical approval was received from the Ethics Committee at Imam Abdulrahman Bin Faisal University, Saudi Arabia.

RESULTS

A total of 506 patients participated in the study, with over 80% having moderate to high experience in using mental health AI assistants. The ANOVA results for performance expectancy (PE), effort expectancy (EE), social influence (SI), facilitating conditions (FC), and behavioral intentions (BI) indicate that there are statistically significant differences (p < 0.05) between the Gen X, Gen Y, and Gen Z participants.

CONCLUSION

The findings underscore the significance of considering generational differences in attitudes and perceptions, with Gen Y and Gen Z demonstrating more positive attitudes and stronger intentions to use AI mental health virtual assistants, while Gen X appears to be more cautious.

A review of the explainability and safety of conversational agents for mental health to identify avenues for improvement

Virtual Mental Health Assistants (VMHAs) continuously evolve to support the overloaded global healthcare system, which receives approximately 60 million primary care visits and 6 million emergency room visits annually. These systems, developed by clinical psychologists, psychiatrists, and AI researchers, are designed to aid in Cognitive Behavioral Therapy (CBT). The main focus of VMHAs is to provide relevant information to mental health professionals (MHPs) and engage in meaningful conversations to support individuals with mental health conditions. However, certain gaps prevent VMHAs from fully delivering on their promise during active communications. One of the gaps is their inability to explain their decisions to patients and MHPs, making conversations less trustworthy. Additionally, VMHAs can be vulnerable in providing unsafe responses to patient queries, further undermining their reliability. In this review, we assess the current state of VMHAs on the grounds of user-level explainability and safety, a set of desired properties for the broader adoption of VMHAs. This includes the examination of ChatGPT, a conversation agent developed on AI-driven models: GPT3.5 and GPT-4, that has been proposed for use in providing mental health services. By harnessing the collaborative and impactful contributions of AI, natural language processing, and the mental health professionals (MHPs) community, the review identifies opportunities for technological progress in VMHAs to ensure their capabilities include explainable and safe behaviors. It also emphasizes the importance of measures to guarantee that these advancements align with the promise of fostering trustworthy conversations.

Mapping loneliness through social intelligence analysis: a step towards creating global loneliness map

OBJECTIVES

Loneliness is a prevalent global public health concern with complex dynamics requiring further exploration. This study aims to enhance understanding of loneliness dynamics through building towards a global loneliness map using social intelligence analysis.

SETTINGS AND DESIGN

This paper presents a proof of concept for the global loneliness map, using data collected in October 2022. Twitter posts containing keywords such as 'lonely', 'loneliness', 'alone', 'solitude' and 'isolation' were gathered, resulting in 841 796 tweets from the USA. City-specific data were extracted from these tweets to construct a loneliness map for the country. Sentiment analysis using the valence aware dictionary for sentiment reasoning tool was employed to differentiate metaphorical expressions from meaningful correlations between loneliness and socioeconomic and emotional factors.

MEASURES AND RESULTS

The sentiment analysis encompassed the USA dataset and city-wise subsets, identifying negative sentiment tweets. Psychosocial linguistic features of these negative tweets were analysed to reveal significant connections between loneliness, socioeconomic aspects and emotional themes. Word clouds depicted topic variations between positively and negatively toned tweets. A frequency list of correlated topics within broader socioeconomic and emotional categories was generated from negative sentiment tweets. Additionally, a comprehensive table displayed top correlated topics for each city.

CONCLUSIONS

Leveraging social media data provide insights into the multifaceted nature of loneliness. Given its subjectivity, loneliness experiences exhibit variability. This study serves as a proof of concept for an extensive global loneliness map, holding implications for global public health strategies and policy development. Understanding loneliness dynamics on a larger scale can facilitate targeted interventions and support.

Factors Influencing the Acceptability, Acceptance, and Adoption of Conversational Agents in Health Care: Integrative Review

BACKGROUND

Conversational agents (CAs), also known as chatbots, are digital dialog systems that enable people to have a text-based, speech-based, or nonverbal conversation with a computer or another machine based on natural language via an interface. The use of CAs offers new opportunities and various benefits for health care. However, they are not yet ubiquitous in daily practice. Nevertheless, research regarding the implementation of CAs in health care has grown tremendously in recent years.

OBJECTIVE

This review aims to present a synthesis of the factors that facilitate or hinder the implementation of CAs from the perspectives of patients and health care professionals. Specifically, it focuses on the early implementation outcomes of acceptability, acceptance, and adoption as cornerstones of later implementation success.

METHODS

We performed an integrative review. To identify relevant literature, a broad literature search was conducted in June 2021 with no date limits and using all fields in PubMed, Cochrane Library, Web of Science, LIVIVO, and PsycINFO. To keep the review current, another search was conducted in March 2022. To identify as many eligible primary sources as possible, we used a snowballing approach by searching reference lists and conducted a hand search. Factors influencing the acceptability, acceptance, and adoption of CAs in health care were coded through parallel deductive and inductive approaches, which were informed by current technology acceptance and adoption models. Finally, the factors were synthesized in a thematic map.

RESULTS

Overall, 76 studies were included in this review. We identified influencing factors related to 4 core Unified Theory of Acceptance and Use of Technology (UTAUT) and Unified Theory of Acceptance and Use of Technology 2 (UTAUT2) factors (performance expectancy, effort expectancy, facilitating conditions, and hedonic motivation), with most studies underlining the relevance of performance and effort expectancy. To meet the particularities of the health care context, we redefined the UTAUT2 factors social influence, habit, and price value. We identified 6 other influencing factors: perceived risk, trust, anthropomorphism, health issue, working alliance, and user characteristics. Overall, we identified 10 factors influencing acceptability, acceptance, and adoption among health care professionals (performance expectancy, effort expectancy, facilitating conditions, social influence, price value, perceived risk, trust, anthropomorphism, working alliance, and user characteristics) and 13 factors influencing acceptability, acceptance, and adoption among patients (additionally hedonic motivation, habit, and health issue).

CONCLUSIONS

This review shows manifold factors influencing the acceptability, acceptance, and adoption of CAs in health care. Knowledge of these factors is fundamental for implementation planning. Therefore, the findings of this review can serve as a basis for future studies to develop appropriate implementation strategies. Furthermore, this review provides an empirical test of current technology acceptance and adoption models and identifies areas where additional research is necessary.

TRIAL REGISTRATION

PROSPERO CRD42022343690; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=343690.

Scope, Characteristics, Behavior Change Techniques, and Quality of Conversational Agents for Mental Health and Well-Being: Systematic Assessment of Apps

BACKGROUND

Mental disorders cause substantial health-related burden worldwide. Mobile health interventions are increasingly being used to promote mental health and well-being, as they could improve access to treatment and reduce associated costs. Behavior change is an important feature of interventions aimed at improving mental health and well-being. There is a need to discern the active components that can promote behavior change in such interventions and ultimately improve users' mental health.

OBJECTIVE

This study systematically identified mental health conversational agents (CAs) currently available in app stores and assessed the behavior change techniques (BCTs) used. We further described their main features, technical aspects, and quality in terms of engagement, functionality, esthetics, and information using the Mobile Application Rating Scale.

METHODS

The search, selection, and assessment of apps were adapted from a systematic review methodology and included a search, 2 rounds of selection, and an evaluation following predefined criteria. We conducted a systematic app search of Apple's App Store and Google Play using 42matters. Apps with CAs in English that uploaded or updated from January 2020 and provided interventions aimed at improving mental health and well-being and the assessment or management of mental disorders were tested by at least 2 reviewers. The BCT taxonomy v1, a comprehensive list of 93 BCTs, was used to identify the specific behavior change components in CAs.

RESULTS

We found 18 app-based mental health CAs. Most CAs had <1000 user ratings on both app stores (12/18, 67%) and targeted several conditions such as stress, anxiety, and depression (13/18, 72%). All CAs addressed >1 mental disorder. Most CAs (14/18, 78%) used cognitive behavioral therapy (CBT). Half (9/18, 50%) of the CAs identified were rule based (ie, only offered predetermined answers) and the other half (9/18, 50%) were artificial intelligence enhanced (ie, included open-ended questions). CAs used 48 different BCTs and included on average 15 (SD 8.77; range 4-30) BCTs. The most common BCTs were 3.3 "Social support (emotional)," 4.1 "Instructions for how to perform a behavior," 11.2 "Reduce negative emotions," and 6.1 "Demonstration of the behavior." One-third (5/14, 36%) of the CAs claiming to be CBT based did not include core CBT concepts.

CONCLUSIONS

Mental health CAs mostly targeted various mental health issues such as stress, anxiety, and depression, reflecting a broad intervention focus. The most common BCTs identified serve to promote the self-management of mental disorders with few therapeutic elements. CA developers should consider the quality of information, user confidentiality, access, and emergency management when designing mental health CAs. Future research should assess the role of artificial intelligence in promoting behavior change within CAs and determine the choice of BCTs in evidence-based psychotherapies to enable systematic, consistent, and transparent development and evaluation of effective digital mental health interventions.

Insights of Parents and Parents-To-Be in Using Chatbots to Improve Their Preconception, Pregnancy, and Postpartum Health: A Mixed Studies Review

INTRODUCTION

Chatbots, which are also known as conversational or virtual agents, are digital programs that can interact with humans using voice, text, or animation. They have shown promise in providing preconception, pregnancy, and postpartum care. This review aims to consolidate the insights of parents and parents-to-be in using chatbots to improve their preconception, pregnancy, and postpartum health.

METHODS

Seven electronic databases were searched from their inception dates until April 2022 (PubMed, Embase, CINAHL, PsycINFO, Web of Science, Scopus, and ProQuest Dissertations and Theses Global) for relevant studies. English language primary studies that were conducted on parents or parents-to-be aged ≥18 years old who had undergone interventions involving the use of any type of chatbot were included in this review. The quality of included studies was appraised using the Mixed Methods Appraisal Tool. A convergent qualitative synthesis design for mixed studies reviews was used to synthesize the findings, and results were thematically analyzed.

RESULTS

Fifteen studies met the inclusion criteria: quantitative (n = 11), qualitative (n = 1), and mixed method (n = 3). Three main themes were identified: (1) welcoming a new health resource, (2) obstacles blocking the way, and (3) moving toward a digital health era.

DISCUSSION

Parents and parents-to-be appreciated the informational, socioemotional, and psychological support provided by chatbots. Recommendations for technological improvements in the functionality of the chatbots were made that include training sessions for health care providers to familiarize them with this new digital technology. Multidisciplinary chatbot development teams could also be established to develop more comprehensive chatbot-delivered health programs for more diverse populations.

Providing Self-Led Mental Health Support Through an Artificial Intelligence-Powered Chat Bot (Leora) to Meet the Demand of Mental Health Care

Digital mental health services are becoming increasingly valuable for addressing the global public health burden of mental ill-health. There is significant demand for scalable and effective web-based mental health services. Artificial intelligence (AI) has the potential to improve mental health through the deployment of chatbots. These chatbots can provide round-the-clock support and triage individuals who are reluctant to access traditional health care due to stigma. The aim of this viewpoint paper is to consider the feasibility of AI-powered platforms to support mental well-being. The Leora model is considered a model with the potential to provide mental health support. Leora is a conversational agent that uses AI to engage in conversations with users about their mental health and provide support for minimal-to-mild symptoms of anxiety and depression. The tool is designed to be accessible, personalized, and discreet, offering strategies for promoting well-being and acting as a web-based self-care coach. Across all AI-powered mental health services, there are several challenges in the ethical development and deployment of AI in mental health treatment, including trust and transparency, bias and health inequity, and the potential for negative consequences. To ensure the effective and ethical use of AI in mental health care, researchers must carefully consider these challenges and engage with key stakeholders to provide high-quality mental health support. Validation of the Leora platform through rigorous user testing will be the next step in ensuring the model is effective.

Effectiveness and Acceptability of Conversational Agents for Smoking Cessation: A Systematic Review and Meta-analysis

INTRODUCTION

Conversational agents (CAs; computer programs that use artificial intelligence to simulate a conversation with users through natural language) have evolved considerably in recent years to support healthcare by providing autonomous, interactive, and accessible services, making them potentially useful for supporting smoking cessation. We performed a systematic review and meta-analysis to provide an overarching evaluation of their effectiveness and acceptability to inform future development and adoption.

AIMS AND METHODS

PsycInfo, Web of Science, ACM Digital Library, IEEE Xplore, Medline, EMBASE, Communication and Mass Media Complete, and CINAHL Complete were searched for studies examining the use of CAs for smoking cessation. Data from eligible studies were extracted and used for random-effects meta-analyses.

RESULTS

The search yielded 1245 publications with 13 studies eligible for systematic review (total N = 8236) and six studies for random-effects meta-analyses. All studies reported positive effects on cessation-related outcomes. A meta-analysis with randomized controlled trials reporting on abstinence yielded a sample-weighted odds ratio of 1.66 (95% CI = 1.33% to 2.07%, p < .001), favoring CAs over comparison groups. A narrative synthesis of all included studies showed overall high acceptability, while some barriers were identified from user feedback. Overall, included studies were diverse in design with mixed quality, and evidence of publication bias was identified. A lack of theoretical foundations was noted, as well as a clear need for relational communication in future designs.

CONCLUSIONS

The effectiveness and acceptability of CAs for smoking cessation are promising. However, standardization of reporting and designing of the agents is warranted for a more comprehensive evaluation.

IMPLICATIONS

This is the first systematic review to provide insight into the use of CAs to support smoking cessation. Our findings demonstrated initial promise in the effectiveness and user acceptability of these agents. We also identified a lack of theoretical and methodological limitations to improve future study design and intervention delivery.

Waiting for a digital therapist: three challenges on the path to psychotherapy delivered by artificial intelligence

Growing demand for broadly accessible mental health care, together with the rapid development of new technologies, trigger discussions about the feasibility of psychotherapeutic interventions based on interactions with Conversational Artificial Intelligence (CAI). Many authors argue that while currently available CAI can be a useful supplement for human-delivered psychotherapy, it is not yet capable of delivering fully fledged psychotherapy on its own. The goal of this paper is to investigate what are the most important obstacles on our way to developing CAI systems capable of delivering psychotherapy in the future. To this end, we formulate and discuss three challenges central to this quest. Firstly, we might not be able to develop effective AI-based psychotherapy unless we deepen our understanding of what makes human-delivered psychotherapy effective. Secondly, assuming that it requires building a therapeutic relationship, it is not clear whether psychotherapy can be delivered by non-human agents. Thirdly, conducting psychotherapy might be a problem too complicated for narrow AI, i.e., AI proficient in dealing with only relatively simple and well-delineated tasks. If this is the case, we should not expect CAI to be capable of delivering fully-fledged psychotherapy until the so-called "general" or "human-like" AI is developed. While we believe that all these challenges can ultimately be overcome, we think that being mindful of them is crucial to ensure well-balanced and steady progress on our path to AI-based psychotherapy.

'Parentbot - A Digital healthcare Assistant (PDA)': A mobile application-based perinatal intervention for parents: Development study

OBJECTIVE

To describe the development procedure of a mobile application-based parenting support program with integrated chatbot features entitled Parentbot - a Digital healthcare Assistant (PDA) for multi-racial Singaporean parents across the perinatal period.

METHODS

The PDA development process was guided by the combined information systems research framework with design thinking modes, and Tuckman's model of team development. A user acceptability testing (UAT) process was conducted among 11 adults of child-bearing age. Feedback was obtained using a custom-made evaluation form and the 26-item User Experience Questionnaire.

RESULTS

The combined information systems research framework with design thinking modes helped researchers to successfully create a PDA prototype based on end-users' needs. Results from the UAT process indicated that the PDA provided participants with an overall positive user experience. Feedback gathered from UAT participants was used to enhance the PDA.

CONCLUSION

Although the effectiveness of the PDA in improving parental outcomes during the perinatal period is still being evaluated, this paper highlights the key details of developing a mobile application-based parenting intervention which future studies could learn from.

PRACTICE IMPLICATIONS

Having carefully planned timelines with margins of delays, extra funds to resolve technical issues, team cohesion, and an experienced leader can facilitate intervention development.

"A Great Way to Start the Conversation": Evidence for the Use of an Adolescent Mental Health Chatbot Navigator for Youth at Risk of HIV and Other STIs

Chatbot use is increasing for mobile health interventions on sensitive and stigmatized topics like mental health because of their anonymity and privacy. This anonymity provides acceptability to sexual and gendered minority youth (ages 16-24) at increased risk of HIV and other STIs with poor mental health due to higher levels of stigma, discrimination, and social isolation. This study evaluates the usability of Tabatha-YYC, a pilot chatbot navigator created to link these youth to mental health resources. Tabatha-YYC was developed using a Youth Advisory Board (n = 7). The final design underwent user testing (n = 20) through a think-aloud protocol, semi-structured interview, and a brief survey post-exposure which included the Health Information Technology Usability Evaluation Scale. The chatbot was found to be an acceptable mental health navigator by participants. This study provides important design methodology considerations and key insights into chatbot design preferences of youth at risk of STIs seeking mental health resources.

A chatbot for mental health support: exploring the impact of Emohaa on reducing mental distress in China

Introduction

The growing demand for mental health support has highlighted the importance of conversational agents as human supporters worldwide and in China. These agents could increase availability and reduce the relative costs of mental health support. The provided support can be divided into two main types: cognitive and emotional. Existing work on this topic mainly focuses on constructing agents that adopt Cognitive Behavioral Therapy (CBT) principles. Such agents operate based on pre-defined templates and exercises to provide cognitive support. However, research on emotional support using such agents is limited. In addition, most of the constructed agents operate in English, highlighting the importance of conducting such studies in China. To this end, we introduce Emohaa, a conversational agent that provides cognitive support through CBT-Bot exercises and guided conversations. It also emotionally supports users through ES-Bot, enabling them to vent their emotional problems. In this study, we analyze the effectiveness of Emohaa in reducing symptoms of mental distress.

Methods and Results

Following the RCT design, the current study randomly assigned participants into three groups: Emohaa (CBT-Bot), Emohaa (Full), and control. With both Intention-To-Treat (N=247) and PerProtocol (N=134) analyses, the results demonstrated that compared to the control group, participants who used two types of Emohaa experienced considerably more significant improvements in symptoms of mental distress, including depression (F[2,244]=6.26, p=0.002), negative affect (F[2,244]=6.09, p=0.003), and insomnia (F[2,244]=3.69, p=0.026).

Discussion

Based on the obtained results and participants' satisfaction with the platform, we concluded that Emohaa is a practical and effective tool for reducing mental distress.

Exploring the Perceptions of mHealth Interventions for the Prevention of Common Mental Disorders in University Students in Singapore: Qualitative Study

BACKGROUND

Mental health interventions delivered through mobile health (mHealth) technologies can increase the access to mental health services, especially among university students. The development of mHealth intervention is complex and needs to be context sensitive. There is currently limited evidence on the perceptions, needs, and barriers related to these interventions in the Southeast Asian context.

OBJECTIVE

This qualitative study aimed to explore the perception of university students and mental health supporters in Singapore about mental health services, campaigns, and mHealth interventions with a focus on conversational agent interventions for the prevention of common mental disorders such as anxiety and depression.

METHODS

We conducted 6 web-based focus group discussions with 30 university students and one-to-one web-based interviews with 11 mental health supporters consisting of faculty members tasked with student pastoral care, a mental health first aider, counselors, psychologists, a clinical psychologist, and a psychiatrist. The qualitative analysis followed a reflexive thematic analysis framework.

RESULTS

The following 6 main themes were identified: a healthy lifestyle as students, access to mental health services, the role of mental health promotion campaigns, preferred mHealth engagement features, factors that influence the adoption of mHealth interventions, and cultural relevance of mHealth interventions. The interpretation of our findings shows that students were reluctant to use mental health services because of the fear of stigma and a possible lack of confidentiality.

CONCLUSIONS

Study participants viewed mHealth interventions for mental health as part of a blended intervention. They also felt that future mental health mHealth interventions should be more personalized and capable of managing adverse events such as suicidal ideation.

Reinventing mental health care in youth through mobile approaches: Current status and future steps

In this perspective, we aim to bring together research on mobile assessments and interventions in the context of mental health care in youth. After the COVID-19 pandemic, one out of five young people is experiencing mental health problems worldwide. New ways to face this burden are now needed. Young people search for low-burden services in terms of costs and time, paired with high flexibility and easy accessibility. Mobile applications meet these principles by providing new ways to inform, monitor, educate, and enable self-help, thus reinventing mental health care in youth. In this perspective, we explore the existing literature reviews on mobile assessments and interventions in youth through data collected passively (e.g., digital phenotyping) and actively (e.g., using Ecological Momentary Assessments—EMAs). The richness of such approaches relies on assessing mental health dynamically by extending beyond the confines of traditional methods and diagnostic criteria, and the integration of sensor data from multiple channels, thus allowing the cross-validation of symptoms through multiple information. However, we also acknowledge the promises and pitfalls of such approaches, including the problem of interpreting small effects combined with different data sources and the real benefits in terms of outcome prediction when compared to gold-standard methods. We also explore a new promising and complementary approach, using chatbots and conversational agents, that encourages interaction while tracing health and providing interventions. Finally, we suggest that it is important to continue to move beyond the ill-being framework by giving more importance to intervention fostering well-being, e.g., using positive psychology.

AI-based chatbot micro-intervention for parents: Meaningful engagement, learning, and efficacy

Introduction

Mental health issues have been on the rise among children and adolescents, and digital parenting programs have shown promising outcomes. However, there is limited research on the potential efficacy of utilizing chatbots to promote parental skills. This study aimed to understand whether parents learn from a parenting chatbot micro intervention, to assess the overall efficacy of the intervention, and to explore the user characteristics of the participants, including parental busyness, assumptions about parenting, and qualitative engagement with the chatbot.

Methods

A sample of 170 parents with at least one child between 2-11 years old were recruited. A randomized control trial was conducted. Participants in the experimental group accessed a 15-min intervention that taught how to utilize positive attention and praise to promote positive behaviors in their children, while the control group remained on a waiting list.

Results

Results showed that participants engaged with a brief AI-based chatbot intervention and were able to learn effective praising skills. Although scores moved in the expected direction, there were no significant differences by condition in the praising knowledge reported by parents, perceived changes in disruptive behaviors, or parenting self-efficacy, from pre-intervention to 24-hour follow-up.

Discussion

The results provided insight to understand how parents engaged with the chatbot and suggests that, in general, brief, self-guided, digital interventions can promote learning in parents. It is possible that a higher dose of intervention may be needed to obtain a therapeutic change in parents. Further research implications on chatbots for parenting skills are discussed.

User experience with a parenting chatbot micro intervention

Background

The use of chatbots to address mental health conditions have become increasingly popular in recent years. However, few studies aimed to teach parenting skills through chatbots, and there are no reports on parental user experience. Aim: This study aimed to assess the user experience of a parenting chatbot micro intervention to teach how to praise children in a Spanish-speaking country.

Methods

A sample of 89 parents were assigned to the chatbot micro intervention as part of a randomized controlled trial study. Completion rates, engagement, satisfaction, net promoter score, and acceptability were analyzed.

Results

66.3% of the participants completed the intervention. Participants exchanged an average of 49.8 messages (SD = 1.53), provided an average satisfaction score of 4.19 (SD = .79), and reported that they would recommend the chatbot to other parents (net promoter score = 4.63/5; SD = .66). Acceptability level was high (ease of use = 4.66 [SD = .73]; comfortability = 4.76 [SD = .46]; lack of technical problems = 4.69 [SD = .59]; interactivity = 4.51 [SD = .77]; usefulness for everyday life = 4.75 [SD = .54]).

Conclusions

Overall, users completed the intervention at a high rate, engaged with the chatbot, were satisfied, would recommend it to others, and reported a high level of acceptability. Chatbots have the potential to teach parenting skills however research on the efficacy of parenting chatbot interventions is needed.

An artificial intelligence-based chatbot for prostate cancer education: Design and patient evaluation study

Introduction

Artificial intelligence (AI) is increasingly used in healthcare. AI-based chatbots can act as automated conversational agents, capable of promoting health and providing education at any time. The objective of this study was to develop and evaluate a user-friendly medical chatbot (prostate cancer communication assistant (PROSCA)) for provisioning patient information about early detection of prostate cancer (PC).

Methods

The chatbot was developed to provide information on prostate diseases, diagnostic tests for PC detection, stages, and treatment options. Ten men aged 49 to 81 years with suspicion of PC were enrolled in this study. Nine of ten patients used the chatbot during the evaluation period and filled out the questionnaires on usage and usability, perceived benefits, and potential for improvement.

Results

The chatbot was straightforward to use, with 78% of users not needing any assistance during usage. In total, 89% of the chatbot users in the study experienced a clear to moderate increase in knowledge about PC through the chatbot. All study participants who tested the chatbot would like to re-use a medical chatbot in the future and support the use of chatbots in the clinical routine.

Conclusions

Through the introduction of the chatbot PROSCA, we created and evaluated an innovative evidence-based health information tool in the field of PC, allowing targeted support for doctor-patient communication and offering great potential in raising awareness, patient education, and support. Our study revealed that a medical chatbot in the field of early PC detection is readily accepted and benefits patients as an additional informative tool.

User Experience of COVID-19 Chatbots: Scoping Review

BACKGROUND

The COVID-19 pandemic has had global impacts and caused some health systems to experience substantial pressure. The need for accurate health information has been felt widely. Chatbots have great potential to reach people with authoritative information, and a number of chatbots have been quickly developed to disseminate information about COVID-19. However, little is known about user experiences of and perspectives on these tools.

OBJECTIVE

This study aimed to describe what is known about the user experience and user uptake of COVID-19 chatbots.

METHODS

A scoping review was carried out in June 2021 using keywords to cover the literature concerning chatbots, user engagement, and COVID-19. The search strategy included databases covering health, communication, marketing, and the COVID-19 pandemic specifically, including MEDLINE Ovid, Embase, CINAHL, ACM Digital Library, Emerald, and EBSCO. Studies that assessed the design, marketing, and user features of COVID-19 chatbots or those that explored user perspectives and experience were included. We excluded papers that were not related to COVID-19; did not include any reporting on user perspectives, experience, or the general use of chatbot features or marketing; or where a version was not available in English. The authors independently screened results for inclusion, using both backward and forward citation checking of the included papers. A thematic analysis was carried out with the included papers.

RESULTS

A total of 517 papers were sourced from the literature, and 10 were included in the final review. Our scoping review identified a number of factors impacting adoption and engagement including content, trust, digital ability, and acceptability. The papers included discussions about chatbots developed for COVID-19 screening and general COVID-19 information, as well as studies investigating user perceptions and opinions on COVID-19 chatbots.

CONCLUSIONS

The COVID-19 pandemic presented a unique and specific challenge for digital health interventions. Design and implementation were required at a rapid speed as digital health service adoption accelerated globally. Chatbots for COVID-19 have been developed quickly as the pandemic has challenged health systems. There is a need for more comprehensive and routine reporting of factors impacting adoption and engagement. This paper has shown both the potential of chatbots to reach users in an emergency and the need to better understand how users engage and what they want.

Evaluation of chatbot-delivered interventions for self-management of depression: Content analysis

BACKGROUND

Conversational agents (CAs) or chatbots are increasingly used for depression, anxiety, and wellbeing management. CAs are considered acceptable and helpful. However, little is known about the adequacy of CA responses. This study assessed the structure, content, and user-customization of mental health CA dialogues with users with depression or at risk of suicide.

METHODS

We used content analysis to examine the dialogues of CAs previously included in three assessments of mental health apps (depression education, self-guided cognitive behavioural therapy, and suicide prevention) performed between 2019 and 2020. Two standardized user personas with depression were developed to interact with the CA. All conversations were saved as screenshots, transcribed verbatim, and coded inductively.

RESULTS

Nine CAs were included. Seven CAs (78%) had Android and iOS versions; five CAs (56%) had at least 500,000 downloads. The analysis generated eight categories: self-introduction, personalization, appropriateness of CA responses, conveying empathy, guiding users through mood-boosting activities, mood monitoring, suicide risk management, and others. CAs could engage in empathic, non-judgemental conversations with users, offer support, and guide psychotherapeutic exercises.

LIMITATIONS

CA evaluations were performed using standardized personas, not real-world users. CAs were included for evaluation only if retrieved in the search strategies associated with the previous assessment studies.

CONCLUSION

Assessed CAs offered anonymous, empathic, non-judgemental interactions that align with evidence for face-to-face psychotherapy. CAs from app stores are not suited to provide comprehensive suicide risk management. Further research should evaluate the effectiveness of CA-led interventions in mental health care and in enhancing suicide risk management strategies.

Experiences and perceptions of final-year nursing students of using a chatbot in a simulated emergency situation: A qualitative study

AIM

The aim of this study is to explore the experiences and perceptions of final-year nursing students on the acceptability and feasibility of using a chatbot for clinical decision-making and patient safety.

BACKGROUND

The effective and inclusive use of new technologies such as conversational agents or chatbots could support nurses in increasing evidence-based care and decreasing low-quality services.

METHODS

A descriptive qualitative study was used through focus group interviews. The data analysis was conducted using a thematic analysis.

RESULTS

This study included 114 participants. After our data analysis, two main themes emerged: (i) experiences in the use of a chatbot service for clinical decision-making and and (ii) integrating conversational agents into the organizational safety culture.

CONCLUSIONS

The findings of our study provide preliminary support for the acceptability and feasibility of adopting SafeBot, a chatbot for clinical decision-making and patient safety. Our results revealed substantial recommendations for refining navigation, layout and content, as well as useful insights to support its acceptance in real nursing practice.

IMPLICATIONS FOR NURSING MANAGEMENT

Leaders and managers may well see artificial intelligence-based conversational agents like SafeBot as a potential solution in modern nursing practice for effective problem-solving resolution, innovative staffing and nursing care delivery models at the bedside and criteria for measuring and ensure quality and patient safety.

Virtual conversational agents versus online forms: Patient experience and preferences for health data collection

Objective

Virtual conversational agents, or chatbots, have emerged as a novel approach to health data collection. However, research on patient perceptions of chatbots in comparison to traditional online forms is sparse. This study aimed to compare and assess the experience of completing a health assessment using a chatbot vs. an online form.

Methods

A counterbalanced, within-subject experimental design was used with participants recruited via Amazon Mechanical Turk (mTurk). Participants completed a standardized health assessment using a chatbot (i.e., Dokbot) and an online form (i.e., REDCap), each followed by usability and experience questionnaires. To address poor data quality and preserve integrity of mTurk responses, we employed a thorough data cleaning process informed by previous literature. Quantitative (descriptive and inferential statistics) and qualitative (thematic analysis and complex coding query) approaches were used for analysis.

Results

A total of 391 participants were recruited, 185 of whom were excluded, resulting in a final sample size of 206 individuals. Most participants (69.9%) preferred the chatbot over the online form. Average Net Promoter Score was higher for the chatbot (NPS = 24) than the online form (NPS = 13) at a statistically significant level. System Usability Scale scores were also higher for the chatbot (i.e. 69.7 vs. 67.7), but this difference was not statistically significant. The chatbot took longer to complete but was perceived as conversational, interactive, and intuitive. The online form received favorable comments for its familiar survey-like interface.

Conclusion

Our findings demonstrate that a chatbot provided superior engagement, intuitiveness, and interactivity despite increased completion time compared to online forms. Knowledge of patient preferences and barriers will inform future design and development of recommendations and best practice for chatbots for healthcare data collection.

Designing, Developing, Evaluating, and Implementing a Smartphone-Delivered, Rule-Based Conversational Agent (DISCOVER): Development of a Conceptual Framework

BACKGROUND

Conversational agents (CAs), also known as chatbots, are computer programs that simulate human conversations by using predetermined rule-based responses or artificial intelligence algorithms. They are increasingly used in health care, particularly via smartphones. There is, at present, no conceptual framework guiding the development of smartphone-based, rule-based CAs in health care. To fill this gap, we propose structured and tailored guidance for their design, development, evaluation, and implementation.

OBJECTIVE

The aim of this study was to develop a conceptual framework for the design, evaluation, and implementation of smartphone-delivered, rule-based, goal-oriented, and text-based CAs for health care.

METHODS

We followed the approach by Jabareen, which was based on the grounded theory method, to develop this conceptual framework. We performed 2 literature reviews focusing on health care CAs and conceptual frameworks for the development of mobile health interventions. We identified, named, categorized, integrated, and synthesized the information retrieved from the literature reviews to develop the conceptual framework. We then applied this framework by developing a CA and testing it in a feasibility study.

RESULTS

The Designing, Developing, Evaluating, and Implementing a Smartphone-Delivered, Rule-Based Conversational Agent (DISCOVER) conceptual framework includes 8 iterative steps grouped into 3 stages, as follows: design, comprising defining the goal, creating an identity, assembling the team, and selecting the delivery interface; development, including developing the content and building the conversation flow; and the evaluation and implementation of the CA. They were complemented by 2 cross-cutting considerations-user-centered design and privacy and security-that were relevant at all stages. This conceptual framework was successfully applied in the development of a CA to support lifestyle changes and prevent type 2 diabetes.

CONCLUSIONS

Drawing on published evidence, the DISCOVER conceptual framework provides a step-by-step guide for developing rule-based, smartphone-delivered CAs. Further evaluation of this framework in diverse health care areas and settings and for a variety of users is needed to demonstrate its validity. Future research should aim to explore the use of CAs to deliver health care interventions, including behavior change and potential privacy and safety concerns.

Comparing Professional and Consumer Ratings of Mental Health Apps: Mixed Methods Study

Background

As the number of mental health apps has grown, increasing efforts have been focused on establishing quality tailored reviews. These reviews prioritize clinician and academic views rather than the views of those who use them, particularly those with lived experiences of mental health problems. Given that the COVID-19 pandemic has increased reliance on web-based and mobile mental health support, understanding the views of those with mental health conditions is of increasing importance.

Objective

This study aimed to understand the opinions of people with mental health problems on mental health apps and how they differ from established ratings by professionals.

Methods

A mixed methods study was conducted using a web-based survey administered between December 2020 and April 2021, assessing 11 mental health apps. We recruited individuals who had experienced mental health problems to download and use 3 apps for 3 days and complete a survey. The survey consisted of the One Mind PsyberGuide Consumer Review Questionnaire and 2 items from the Mobile App Rating Scale (star and recommendation ratings from 1 to 5). The consumer review questionnaire contained a series of open-ended questions, which were thematically analyzed and using a predefined protocol, converted into binary (positive or negative) ratings, and compared with app ratings by professionals and star ratings from app stores.

Results

We found low agreement between the participants’ and professionals’ ratings. More than half of the app ratings showed disagreement between participants and professionals (198/372, 53.2%). Compared with participants, professionals gave the apps higher star ratings (3.58 vs 4.56) and were more likely to recommend the apps to others (3.44 vs 4.39). Participants’ star ratings were weakly positively correlated with app store ratings (r=0.32, P=.01). Thematic analysis found 11 themes, including issues of user experience, ease of use and interactivity, privacy concerns, customization, and integration with daily life. Participants particularly valued certain aspects of mental health apps, which appear to be overlooked by professional reviewers. These included functions such as the ability to track and measure mental health and providing general mental health education. The cost of apps was among the most important factors for participants. Although this is already considered by professionals, this information is not always easily accessible.

Conclusions

As reviews on app stores and by professionals differ from those by people with lived experiences of mental health problems, these alone are not sufficient to provide people with mental health problems with the information they desire when choosing a mental health app. App rating measures must include the perspectives of mental health service users to ensure ratings represent their priorities. Additional work should be done to incorporate the features most important to mental health service users into mental health apps.

Assessing the Impact of Conversational Artificial Intelligence in the Treatment of Stress and Anxiety in Aging Adults: Randomized Controlled Trial

BACKGROUND

While mental health applications are increasingly becoming available for large populations of users, there is a lack of controlled trials on the impacts of such applications. Artificial intelligence (AI)-empowered agents have been evaluated when assisting adults with cognitive impairments; however, few applications are available for aging adults who are still actively working. These adults often have high stress levels related to changes in their work places, and related symptoms eventually affect their quality of life.

OBJECTIVE

We aimed to evaluate the contribution of TEO (Therapy Empowerment Opportunity), a mobile personal health care agent with conversational AI. TEO promotes mental health and well-being by engaging patients in conversations to recollect the details of events that increased their anxiety and by providing therapeutic exercises and suggestions.

METHODS

The study was based on a protocolized intervention for stress and anxiety management. Participants with stress symptoms and mild-to-moderate anxiety received an 8-week cognitive behavioral therapy (CBT) intervention delivered remotely. A group of participants also interacted with the agent TEO. The participants were active workers aged over 55 years. The experimental groups were as follows: group 1, traditional therapy; group 2, traditional therapy and mobile health (mHealth) agent; group 3, mHealth agent; and group 4, no treatment (assigned to a waiting list). Symptoms related to stress (anxiety, physical disease, and depression) were assessed prior to treatment (T1), at the end (T2), and 3 months after treatment (T3), using standardized psychological questionnaires. Moreover, the Patient Health Questionnaire-8 and General Anxiety Disorders-7 scales were administered before the intervention (T1), at mid-term (T2), at the end of the intervention (T3), and after 3 months (T4). At the end of the intervention, participants in groups 1, 2, and 3 filled in a satisfaction questionnaire.

RESULTS

Despite randomization, statistically significant differences between groups were present at T1. Group 4 showed lower levels of anxiety and depression compared with group 1, and lower levels of stress compared with group 2. Comparisons between groups at T2 and T3 did not show significant differences in outcomes. Analyses conducted within groups showed significant differences between times in group 2, with greater improvements in the levels of stress and scores related to overall well-being. A general worsening trend between T2 and T3 was detected in all groups, with a significant increase in stress levels in group 2. Group 2 reported higher levels of perceived usefulness and satisfaction.

CONCLUSIONS

No statistically significant differences could be observed between participants who used the mHealth app alone or within the traditional CBT setting. However, the results indicated significant differences within the groups that received treatment and a stable tendency toward improvement, which was limited to individual perceptions of stress-related symptoms.

TRIAL REGISTRATION

ClinicalTrials.gov NCT04809090; https://clinicaltrials.gov/ct2/show/NCT04809090.

Development and usability testing of a chatbot to promote mental health services use among individuals with eating disorders following screening

OBJECTIVE

A significant gap exists between those who need and those who receive care for eating disorders (EDs). Novel solutions are needed to encourage service use and address treatment barriers. This study developed and evaluated the usability of a chatbot designed for pairing with online ED screening. The tool aimed to promote mental health service utilization by improving motivation for treatment and self-efficacy among individuals with EDs.

METHODS

A chatbot prototype, Alex, was designed using decision trees and theoretically-informed components: psychoeducation, motivational interviewing, personalized recommendations, and repeated administration. Usability testing was conducted over four iterative cycles, with user feedback informing refinements to the next iteration. Post-testing, participants (N= 21) completed the System Usability Scale (SUS), the Usefulness, Satisfaction, and Ease of Use Questionnaire (USE), and a semi-structured interview.

RESULTS

Interview feedback detailed chatbot aspects participants enjoyed and aspects necessitating improvement. Feedback converged on four themes: user experience, chatbot qualities, chatbot content, and ease of use. Following refinements, users described Alex as humanlike, supportive, and encouraging. Content was perceived as novel and personally relevant. USE scores across domains were generally above average (~5 out of 7), and SUS scores indicated "good" to "excellent" usability across cycles, with the final iteration receiving the highest average score.

DISCUSSION

Overall, participants generally reflected positively on interactions with Alex, including the initial version. Refinements between cycles further improved user experiences. This study provides preliminary evidence of the feasibility and acceptance of a chatbot designed to promote motivation for and use of services among individuals with EDs.

PUBLIC SIGNIFICANCE

Low rates of service utilization and treatment have been observed among individuals following online eating disorder screening. Tools are needed, including scalable, digital options, that can be easily paired with screening, to improve motivation for addressing eating disorders and promote service utilization.

Thematic Analysis on User Reviews for Depression and Anxiety Chatbot Apps: Machine Learning Approach

BACKGROUND

Anxiety and depression are among the most commonly prevalent mental health disorders worldwide. Chatbot apps can play an important role in relieving anxiety and depression. Users' reviews of chatbot apps are considered an important source of data for exploring users' opinions and satisfaction.

OBJECTIVE

This study aims to explore users' opinions, satisfaction, and attitudes toward anxiety and depression chatbot apps by conducting a thematic analysis of users' reviews of 11 anxiety and depression chatbot apps collected from the Google Play Store and Apple App Store. In addition, we propose a workflow to provide a methodological approach for future analysis of app review comments.

METHODS

We analyzed 205,581 user review comments from chatbots designed for users with anxiety and depression symptoms. Using scraper tools and Google Play Scraper and App Store Scraper Python libraries, we extracted the text and metadata. The reviews were divided into positive and negative meta-themes based on users' rating per review. We analyzed the reviews using word frequencies of bigrams and words in pairs. A topic modeling technique, latent Dirichlet allocation, was applied to identify topics in the reviews and analyzed to detect themes and subthemes.

RESULTS

Thematic analysis was conducted on 5 topics for each sentimental set. Reviews were categorized as positive or negative. For positive reviews, the main themes were confidence and affirmation building, adequate analysis, and consultation, caring as a friend, and ease of use. For negative reviews, the results revealed the following themes: usability issues, update issues, privacy, and noncreative conversations.

CONCLUSIONS

Using a machine learning approach, we were able to analyze ≥200,000 comments and categorize them into themes, allowing us to observe users' expectations effectively despite some negative factors. A methodological workflow is provided for the future analysis of review comments.

A Cognitive Behavior Therapy Chatbot (Otis) for Health Anxiety Management: A Mixed-Methods Pilot Study (Preprint)

Background

An increase in health anxiety was observed during the COVID-19 pandemic. However, due to physical distancing restrictions and a strained mental health system, people were unable to access support to manage health anxiety. Chatbots are emerging as an interactive means to deliver psychological interventions in a scalable manner and provide an opportunity for novel therapy delivery to large groups of people including those who might struggle to access traditional therapies.

Objective

The aim of this mixed methods pilot study was to investigate the feasibility, acceptability, engagement, and effectiveness of a cognitive behavioral therapy (CBT)–based chatbot (Otis) as an early health anxiety management intervention for adults in New Zealand during the COVID-19 pandemic.

Methods

Users were asked to complete a 14-day program run by Otis, a primarily decision tree–based chatbot on Facebook Messenger. Health anxiety, general anxiety, intolerance of uncertainty, personal well-being, and quality of life were measured pre-intervention, postintervention, and at a 12-week follow-up. Paired samples t tests and 1-way ANOVAs were conducted to investigate the associated changes in the outcomes over time. Semistructured interviews and written responses in the self-report questionnaires and Facebook Messenger were thematically analyzed.

Results

The trial was completed by 29 participants who provided outcome measures at both postintervention and follow-up. Although an average decrease in health anxiety did not reach significance at postintervention (P=.55) or follow-up (P=.08), qualitative analysis demonstrated that participants perceived benefiting from the intervention. Significant improvement in general anxiety, personal well-being, and quality of life was associated with the use of Otis at postintervention and follow-up. Anthropomorphism, Otis’ appearance, and delivery of content facilitated the use of Otis. Technical difficulties and high performance and effort expectancy were, in contrast, barriers to acceptance and engagement of Otis.

Conclusions

Otis may be a feasible, acceptable, and engaging means of delivering CBT to improve anxiety management, quality of life, and personal well-being but might not significantly reduce health anxiety.

The effectiveness of serious games in improving processing speed among elderly people with cognitive impairment: A systematic review and meta-analysis (Preprint).. [DOI: 10.2196/preprints.36754]

BACKGROUND

Cognitive processing speed is known to decline by age. Processing speed refers to the time that an individual takes from receiving a stimulus to reacting to it. Serious games, which are video games employed for training and educational purposes, have the potential in improving processing speed. Numerous systematic reviews have summarized the evidence about the effectiveness of serious games in improving processing speed, but they are undermined by some limitations.

OBJECTIVE

This study aims to pool the evidence about the effectiveness of serious games in improving processing speed among elderly people with cognitive impairment.

METHODS

A systematic review of randomized controlled trials (RCTs) was undertaken. Two search sources were used in this review: 8 electronic databases as well as backward and forward reference list checking. Two reviewers independently checked the eligibility of the studies, extracted data from the included studies, and appraised the risk of bias and quality of the evidence. Evidence from the included studies was synthesized using a narrative and statistical approach (i.e., meta-analysis), as appropriate.

RESULTS

Out of 548 publications identified, 16 RCTs eventually met all eligibility criteria. Very low-quality evidence from 8 RCTs and 6 RCTs showed no statistically significant effect of serious games on the processing speed as compared with no or passive interventions groups (P=0.77) and conventional exercises (P=0.58), respectively. A subgroup analysis showed that both types of serious games (cognitive training games (P=0.26) and exergames (P=0.88)) are as effective as conventional exercises in improving processing speed.

CONCLUSIONS

There is no superiority of serious games over no or passive interventions and conventional exercises in improving processing speed among older adults with cognitive impairment. Yet, our findings remain inconclusive due to the low quality of the evidence, the small sample size in most included studies, and the paucity of studies included in the meta-analyses. Therefore, until more robust evidence is published, serious games should be offered or used as an adjunct to existing interventions. Further trials should be undertaken to investigate the effect of serious games that target specifically processing speed rather than cognitive abilities in general.

The effectiveness of serious games on the cognitive processing speed among elderly people with cognitive impairment: A systematic review and meta-analysis (Preprint)

Background

Human cognitive processing speed is known to decline with age. Human cognitive processing speed refers to the time that an individual takes from receiving a stimulus to reacting to it. Serious games, which are video games used for training and educational purposes, have the potential to improve processing speed. Numerous systematic reviews have summarized the evidence regarding the effectiveness of serious games in improving processing speed, but they are undermined by some limitations.

Objective

This study aimed to examine the effectiveness of serious games on the cognitive processing speed of an older adult population living with cognitive impairment.

Methods

A systematic review of randomized controlled trials (RCTs) was conducted. Two search sources were used in this review: 8 electronic databases and backward and forward reference list checking. A total of 2 reviewers independently checked the eligibility of the studies, extracted data from the included studies, and appraised the risk of bias and quality of the evidence. Evidence from the included studies was synthesized using a narrative and statistical approach (ie, meta-analysis), as appropriate.

Results

Of the 548 publications identified, 16 (2.9%) RCTs eventually met all eligibility criteria. Very-low-quality evidence from 50% (8/16) and 38% (6/16) of the RCTs showed no statistically significant effect of serious games on processing speed compared with no or passive intervention groups (P=.77) and conventional exercises (P=.58), respectively. A subgroup analysis showed that both types of serious games (cognitive training games: P=.26; exergames: P=.88) were as effective as conventional exercises in improving processing speed.

Conclusions

There is no superiority of serious games over no or passive interventions and conventional exercises in improving processing speed among older adults with cognitive impairment. However, our findings remain inconclusive because of the low quality of the evidence, the small sample size in most of the included studies, and the paucity of studies included in the meta-analyses. Therefore, until more robust evidence is published, serious games should be offered or used as an adjunct to existing interventions. Further trials should be undertaken to investigate the effect of serious games that specifically target processing speed rather than cognitive abilities in general.

Trial Registration

PROSPERO CRD42022301667; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=301667

Digital mental health intervention for orthopedic patients with symptoms of depression and/or anxiety: Pilot feasibility study

BACKGROUND

Symptoms of depression and anxiety commonly coexist with chronic musculoskeletal pain, and when this occurs, standard orthopedic treatment is less effective. Nevertheless, addressing mental health is not yet a routine part of standard orthopedic treatment, in part due to access related barriers. Digital mental health intervention offers the potential to be a scalable resource that could feasibly be incorporated into orthopedic care.

OBJECTIVE

The primary purpose of this study was to assess the feasibility of introducing a digital mental health intervention (Wysa) within an outpatient orthopedic setting to patients who endorse coexisting symptoms of depression and/or anxiety. The secondary purpose was to perform a preliminary effectiveness analysis of the intervention.

METHODS

In this single-arm, prospective cohort study, participants included adult patients (18 years and older) who presented to a non-surgical orthopedic specialist at a single tertiary care academic center for evaluation of a musculoskeletal condition and who self-reported symptoms of depression and/or anxiety (Patient-Reported Outcomes Measurement Information System (PROMIS) Depression and/or Anxiety score ≥ 55). Enrollment was performed face-to-face by a research coordinator immediately after the participant's encounter with an orthopedic clinician. Participants were provided two months of access to a mobile app called Wysa, which is an established, multi-component digital mental health intervention that uses chatbot technology and text-based access to human counselors to deliver cognitive behavioral therapy, mindfulness training, and sleep tools, among other features. For this study, Wysa access also included novel, behavioral activation based features specifically developed for users with chronic pain. Primary feasibility outcomes included the study recruitment rate, retention rate, and engagement rate with Wysa (defined as engaging with a therapeutic Wysa tool at least once during the study period). Secondary effectiveness outcomes were between-group differences in mean longitudinal PROMIS mental and physical health score changes at two-month follow-up between high Wysa users and low Wysa users, defined by a median split.

RESULTS

The recruitment rate was 61/208 (29%), retention rate was 51/61 (84%), and engagement rate was 44/61 (72%). Compared to low users, high Wysa users achieved greater improvement in PROMIS Anxiety (between-group difference -4.2 points [95% CI -8.1 to -0.2], P=.044) at two-month follow-up. Between-group differences in PROMIS Depression (-3.2 points [-7.5 to 1.2], P=.15) and Pain Interference (-2.3 points [-6.3 to 1.7], P=.26) favored high users but did not meet statistical significance. Improvements in PROMIS Physical Function were comparable between groups.

CONCLUSIONS

Delivery of a digital mental health intervention within the context of orthopedic care is feasible and demonstrates potential to improve mental health and pain-related impairment to a clinically meaningful degree. Participants' engagement rates exceeded industry standards, and additional opportunities to improve recruitment and retention were identified. Further pilot study followed by a definitive, randomized controlled trial is warranted.

CLINICALTRIAL

ClinicalTrials.gov NCT202005219.

Effectiveness of Serious Games for Improving Executive Functions Among Older Adults With Cognitive Impairment: Systematic Review and Meta-analysis (Preprint).. [DOI: 10.2196/preprints.36123]

BACKGROUND

Executive functions are one of the known cognitive abilities that decline with age. They are the high-order cognitive processes that enable an individual to concentrate, plan, and take action. Serious games, which are games developed for specific purposes other than entertainment, could play a positive role in improving executive functions. Several systematic reviews have pooled the evidence about the effectiveness of serious games in improving executive functions; however, they are limited by some weaknesses.

OBJECTIVE

This study aims to investigate the effectiveness of serious games for improving executive functions among older adults with cognitive impairment.

METHODS

A systematic review of randomized controlled trials (RCTs) was conducted. To retrieve relevant studies, 8 electronic databases were searched. Further, reference lists of the included studies and relevant reviews were screened, and we checked studies that cited our included studies. Two reviewers independently checked the eligibility of the studies, extracted data from the included studies, assessed the risk of bias, and appraised the quality of the evidence. We used a narrative and statistical approach, as appropriate, to synthesize results of the included studies.

RESULTS

Of 548 publications identified, 16 RCTs were eventually included in this review. Of the 16 studies, 14 studies were included in 6 meta-analyses. Our meta-analyses showed that serious games are as effective as no or passive interventions at improving executive functions (<i>P</i>=.29). Surprisingly, conventional exercises were more effective than serious games at improving executive functions (<i>P</i>=.03). Our subgroup analysis showed that both types of serious games (cognitive training games, <i>P</i>=.08; exergames, <i>P</i>=.16) are as effective as conventional exercises at improving executive functions. No difference was found between adaptive serious games and nonadaptive serious games for improving executive functions (<i>P</i>=.59).

CONCLUSIONS

Serious games are not superior to no or passive interventions and conventional exercises at improving executive functions among older adults with cognitive impairment. However, our findings remain inconclusive due to the low quality of the evidence, the small sample size in most included studies, and the paucity of studies included in the meta-analyses. Accordingly, until more robust evidence is available, serious games should not be offered by health care providers nor used by patients for improving executive functions among older adults with cognitive impairment. Further reviews are needed to assess the long-term effect of serious games on specific executive functions or other cognitive abilities among people from different age groups with or without cognitive impairment.

CLINICALTRIAL

PROSPERO CRD42021272757; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=272757

The effectiveness of serious games in improving executive functions among older adults with cognitive impairment: A systematic review and meta-analysis (Preprint)

Background

Executive functions are one of the known cognitive abilities that decline with age. They are the high-order cognitive processes that enable an individual to concentrate, plan, and take action. Serious games, which are games developed for specific purposes other than entertainment, could play a positive role in improving executive functions. Several systematic reviews have pooled the evidence about the effectiveness of serious games in improving executive functions; however, they are limited by some weaknesses.

Objective

This study aims to investigate the effectiveness of serious games for improving executive functions among older adults with cognitive impairment.

Methods

A systematic review of randomized controlled trials (RCTs) was conducted. To retrieve relevant studies, 8 electronic databases were searched. Further, reference lists of the included studies and relevant reviews were screened, and we checked studies that cited our included studies. Two reviewers independently checked the eligibility of the studies, extracted data from the included studies, assessed the risk of bias, and appraised the quality of the evidence. We used a narrative and statistical approach, as appropriate, to synthesize results of the included studies.

Results

Of 548 publications identified, 16 RCTs were eventually included in this review. Of the 16 studies, 14 studies were included in 6 meta-analyses. Our meta-analyses showed that serious games are as effective as no or passive interventions at improving executive functions (P=.29). Surprisingly, conventional exercises were more effective than serious games at improving executive functions (P=.03). Our subgroup analysis showed that both types of serious games (cognitive training games, P=.08; exergames, P=.16) are as effective as conventional exercises at improving executive functions. No difference was found between adaptive serious games and nonadaptive serious games for improving executive functions (P=.59).

Conclusions

Serious games are not superior to no or passive interventions and conventional exercises at improving executive functions among older adults with cognitive impairment. However, our findings remain inconclusive due to the low quality of the evidence, the small sample size in most included studies, and the paucity of studies included in the meta-analyses. Accordingly, until more robust evidence is available, serious games should not be offered by health care providers nor used by patients for improving executive functions among older adults with cognitive impairment. Further reviews are needed to assess the long-term effect of serious games on specific executive functions or other cognitive abilities among people from different age groups with or without cognitive impairment.

Trial Registration

PROSPERO CRD42021272757; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=272757