Health-focused conversational agents in person-centered care: a review of apps

Controversy in Hypertension: Pro-Side of the Argument Using Artificial Intelligence for Hypertension Diagnosis and Management

Hypertension presents the largest modifiable public health challenge due to its high prevalence, its intimate relationship to cardiovascular diseases, and its complex pathogenesis and pathophysiology. Low awareness of blood pressure elevation and suboptimal hypertension diagnosis serve as the major hurdles in effective hypertension management. Advances in artificial intelligence in hypertension have permitted the integrative analysis of large data sets including omics, clinical (with novel sensor and wearable technologies), health-related, social, behavioral, and environmental sources, and hold transformative potential in achieving large-scale, data-driven approaches toward personalized diagnosis, treatment, and long-term management. However, although the emerging artificial intelligence science may advance the concept of precision hypertension in discovery, drug targeting and development, patient care, and management, its clinical adoption at scale today is lacking. Recognizing that clinical implementation of artificial intelligence-based solutions need evidence generation, this opinion statement examines a clinician-centric perspective of the state-of-art in using artificial intelligence in the management of hypertension and puts forward recommendations toward equitable precision hypertension care.

The evolving field of digital mental health: current evidence and implementation issues for smartphone apps, generative artificial intelligence, and virtual reality

The expanding domain of digital mental health is transitioning beyond traditional telehealth to incorporate smartphone apps, virtual reality, and generative artificial intelligence, including large language models. While industry setbacks and methodological critiques have highlighted gaps in evidence and challenges in scaling these technologies, emerging solutions rooted in co-design, rigorous evaluation, and implementation science offer promising pathways forward. This paper underscores the dual necessity of advancing the scientific foundations of digital mental health and increasing its real-world applicability through five themes. First, we discuss recent technological advances in digital phenotyping, virtual reality, and generative artificial intelligence. Progress in this latter area, specifically designed to create new outputs such as conversations and images, holds unique potential for the mental health field. Given the spread of smartphone apps, we then evaluate the evidence supporting their utility across various mental health contexts, including well-being, depression, anxiety, schizophrenia, eating disorders, and substance use disorders. This broad view of the field highlights the need for a new generation of more rigorous, placebo-controlled, and real-world studies. We subsequently explore engagement challenges that hamper all digital mental health tools, and propose solutions, including human support, digital navigators, just-in-time adaptive interventions, and personalized approaches. We then analyze implementation issues, emphasizing clinician engagement, service integration, and scalable delivery models. We finally consider the need to ensure that innovations work for all people and thus can bridge digital health disparities, reviewing the evidence on tailoring digital tools for historically marginalized populations and low- and middle-income countries. Regarding digital mental health innovations as tools to augment and extend care, we conclude that smartphone apps, virtual reality, and large language models can positively impact mental health care if deployed correctly.

Using Large Language Models to Enhance Exercise Recommendations and Physical Activity in Clinical and Healthy Populations: Scoping Review

Background

Regular exercise recommendations (ERs) and physical activity (PA) are crucial for the prevention and management of chronic diseases. However, creating effective exercise programs demand substantial time and specialized expertise from both medical and sports professionals. Large language models (LLMs), such as ChatGPT, offer a promising solution by helping create personalized ERs. While LLMs show potential, their use in exercise planning remains in its early stages and requires further exploration.

objectives

This study aims to systematically review and classify the applications of LLMs in ERs and PA. It also seeks to identify existing gaps and provide insights into future research directions for optimizing LLM integration in personalized health interventions.

Methods

A scoping review methodology was used to identify studies related to LLM applications in ERs and PA. Literature searches were conducted in Web of Science, PubMed, IEEE, and arXiv for English language papers published up to March 21, 2024. Keywords included LLMs, chatbots, ERs, PA, fitness plan, and related terms. Two independent reviewers (XL and CH) screened and selected studies based on predefined inclusion criteria. Thematic analysis was used to synthesize findings, which were presented narratively.

Results

An initial search identified 598 papers, of which 1.8% (11/598) of studies were included after screening and applying selection criteria. Of these, ChatGPT-based models were used in 55% (6/11) of the studies. In addition, 73% (8/11) of the studies used expert evaluations and user feedback to assess model usability, and 45% (5/11) of the studies used experimental designs to evaluate LLM interventions in ERs and PA. Key findings indicated that LLMs can generate tailored ERs, save time in clinical practice, and enhance safety by incorporating patient-specific data. They also increased engagement and supported behavior change. This made PA guidance more accessible, especially in remote or underserved communities.

Conclusions

This review highlights the promising applications of LLMs in ERs and PA but emphasizes that they remain a supplement to human expertise. Expert validation is essential to ensure safety and mitigate risks. Future research should prioritize pilot testing, clinician training programs, and large-scale clinical trials to enhance feasibility, transparency, and ethical integration.

Initial Requirements for the Prototyping of an App for a Psychosocial Rehabilitation Project: An Integrative Review

The Psychosocial Rehabilitation Project (PRP) is a tool designed to structure and organize mental health care, guided by the theoretical and practical principles of Psychosocial Rehabilitation (PR). This article aims to identify the initial requirements for the prototyping of a "Psychosocial Rehabilitation Project App". To achieve this, an integrative review was conducted with the research question: what initial requirements are important to compose the prototype of the "Psychosocial Rehabilitation Project App" in mental health? In the search process, 834 articles were identified and exported to the online systematic review application Rayyan QCRI, resulting in 36 eligible articles for this study, along with one app. The reading of this material allowed the elicitation of three themes: privacy and data protection policy; design; and software and programming. The prototyping of the "Psychosocial Rehabilitation Project App" should prioritize data security and protection, simplicity in design, and the integration of technological resources that facilitate the management, construction, monitoring, and evaluation of psychosocial rehabilitation projects by mental health professionals.

The real ethical issues with AI for clinical psychiatry

This article explores the ethical issues arising from ordinary AI applications currently used in mental health care, rather than speculative future scenarios. AI tools are already in use for a variety of purposes, including data collection for screening and intake, documentation, decision support, non-clinical support, and, in limited cases, adjunctive treatment. After reviewing the range of and distinctions between those applications, including when those distinctions become blurred, the article discusses selected ethical considerations. The use of AI in psychiatry raises issues related to reflective practice, the seductive allure of AI, varieties of bias, data security, and liability. These examples highlight how seemingly simple AI applications can still present significant ethical implications, suggesting practical considerations for clinicians, professional organizations, treatment organizations, training programs, and policymakers.

Transforming Hypertension Diagnosis and Management in The Era of Artificial Intelligence: A 2023 National Heart, Lung, and Blood Institute (NHLBI) Workshop Report

Hypertension is among the most important risk factors for cardiovascular disease, chronic kidney disease, and dementia. The artificial intelligence (AI) field is advancing quickly, and there has been little discussion on how AI could be leveraged for improving the diagnosis and management of hypertension. AI technologies, including machine learning tools, could alter the way we diagnose and manage hypertension, with potential impacts for improving individual and population health. The development of successful AI tools in public health and health care systems requires diverse types of expertise with collaborative relationships between clinicians, engineers, and data scientists. Unbiased data sources, management, and analyses remain a foundational challenge. From a diagnostic standpoint, machine learning tools may improve the measurement of blood pressure and be useful in the prediction of incident hypertension. To advance the management of hypertension, machine learning tools may be useful to find personalized treatments for patients using analytics to predict response to antihypertension medications and the risk for hypertension-related complications. However, there are real-world implementation challenges to using AI tools in hypertension. Herein, we summarize key findings from a diverse group of stakeholders who participated in a workshop held by the National Heart, Lung, and Blood Institute in March 2023. Workshop participants presented information on communication gaps between clinical medicine, data science, and engineering in health care; novel approaches to estimating BP, hypertension risk, and BP control; and real-world implementation challenges and issues.

Individual differences in views toward healthcare conversational agents: A cross-sectional survey study

Background and Objective

To date, there has been limited research on people's attitudes and design preferences with respect to conversational agents (CAs) that are used for healthcare. Individual differences in attitudes and design preferences have received particularly little attention. The purpose of this study was to gain greater insight into this topic.

Methods

We recruited American and Canadian residents through the online research platform Prolific. Participants completed a cross-sectional survey assessing demographic, personality, and health factors, as well as attitudes and design preferences with respect to healthcare CAs. Hierarchical regressions were used to determine demographic, personality, and health predictors of attitudes and design preferences.

Results

A total of 227 participants (116 women; M age = 39.92 years, SD = 12.94) were included in the analysis. Participants tended to report slightly positive attitudes toward healthcare CAs, with more positive attitudes among American residents and people with lower income, lower education levels, and higher levels of the personality factor conscientiousness. In general, participants preferred CAs that use text communication, have unrestricted language input, are disembodied, and simulate health professionals in their presentation. CAs that use text communication were preferred to a greater degree among people with higher levels of digital health literacy, and disembodied CAs were preferred to a greater degree among people with lower levels of conscientiousness.

Conclusion

The results of this study provide insight into people's attitudes and design preferences with respect to healthcare CAs. This information will help guide developers on how to better design and market CAs for the health sector, which may increase people's adoption and use of these programs.

A behaviourally informed chatbot increases vaccination rates in Argentina more than a one-way reminder

Maintaining COVID-19 vaccine demand was key to ending the global health emergency. To help do this, many governments used chatbots that provided personalized information guiding people on where, when and how to get vaccinated. We designed and tested a WhatsApp chatbot to understand whether two-way interactive messaging incorporating behaviourally informed functionalities could perform better than one-way message reminders. We ran a large-scale preregistered randomized controlled trial with 249,705 participants in Argentina, measuring vaccinations using Ministry of Health records. The behaviourally informed chatbot more than tripled COVID-19 vaccine uptake compared with the control group (a 1.6 percentage point increase (95% confidence interval, (1.36 pp, 1.77 pp)) and nearly doubled uptake compared with the one-way message reminder (a 1 percentage point increase (95% confidence interval, (0.83 pp, 1.17 pp)). Communications tools designed with behaviourally informed functionalities that simplify the vaccine user journey can increase vaccination more than traditional message reminders and may have applications to other health behaviours.

Task-Specific Transformer-Based Language Models in Health Care: Scoping Review

BACKGROUND

Transformer-based language models have shown great potential to revolutionize health care by advancing clinical decision support, patient interaction, and disease prediction. However, despite their rapid development, the implementation of transformer-based language models in health care settings remains limited. This is partly due to the lack of a comprehensive review, which hinders a systematic understanding of their applications and limitations. Without clear guidelines and consolidated information, both researchers and physicians face difficulties in using these models effectively, resulting in inefficient research efforts and slow integration into clinical workflows.

OBJECTIVE

This scoping review addresses this gap by examining studies on medical transformer-based language models and categorizing them into 6 tasks: dialogue generation, question answering, summarization, text classification, sentiment analysis, and named entity recognition.

METHODS

We conducted a scoping review following the Cochrane scoping review protocol. A comprehensive literature search was performed across databases, including Google Scholar and PubMed, covering publications from January 2017 to September 2024. Studies involving transformer-derived models in medical tasks were included. Data were categorized into 6 key tasks.

RESULTS

Our key findings revealed both advancements and critical challenges in applying transformer-based models to health care tasks. For example, models like MedPIR involving dialogue generation show promise but face privacy and ethical concerns, while question-answering models like BioBERT improve accuracy but struggle with the complexity of medical terminology. The BioBERTSum summarization model aids clinicians by condensing medical texts but needs better handling of long sequences.

CONCLUSIONS

This review attempted to provide a consolidated understanding of the role of transformer-based language models in health care and to guide future research directions. By addressing current challenges and exploring the potential for real-world applications, we envision significant improvements in health care informatics. Addressing the identified challenges and implementing proposed solutions can enable transformer-based language models to significantly improve health care delivery and patient outcomes. Our review provides valuable insights for future research and practical applications, setting the stage for transformative advancements in medical informatics.

Development and Refinement of a Chatbot for Birthing Individuals and Newborn Caregivers: Mixed Methods Study

BACKGROUND

The 42 days after delivery ("fourth trimester") are a high-risk period for birthing individuals and newborns, especially those who are racially and ethnically marginalized due to structural racism.

OBJECTIVE

To fill a gap in the critical "fourth trimester," we developed 2 ruled-based chatbots-one for birthing individuals and one for newborn caregivers-that provided trusted information about postbirth warning signs and newborn care and connected patients with health care providers.

METHODS

A total of 4370 individuals received the newborn chatbot outreach between September 1, 2022, and December 31, 2023, and 3497 individuals received the postpartum chatbot outreach between November 16, 2022, and December 31, 2023. We conducted surveys and interviews in English and Spanish to understand the acceptability and usability of the chatbot and identify areas for improvement. We sampled from hospital discharge lists that distributed the chatbot, stratified by prenatal care location, age, type of insurance, and racial and ethnic group. We analyzed quantitative results using descriptive analyses in SPSS (IBM Corp) and qualitative results using deductive coding in Dedoose (SocioCultural Research Consultants).

RESULTS

Overall, 2748 (63%) individuals opened the newborn chatbot messaging, and 2244 (64%) individuals opened the postpartum chatbot messaging. A total of 100 patients engaged with the chatbot and provided survey feedback; of those, 40% (n=40) identified as Black, 27% (n=27) identified as Hispanic/Latina, and 18% (n=18) completed the survey in Spanish. Payer distribution was 55% (n=55) for individuals with public insurance, 39% (n=39) for those with commercial insurance, and 2% (n=2) for uninsured individuals. The majority of surveyed participants indicated that chatbot messaging was timely and easy to use (n=80, 80%) and found the reminders to schedule the newborn visit (n=59, 59%) and postpartum visit (n=66, 66%) useful. Across 23 interviews (n=14, 61% Black; n=4, 17% Hispanic/Latina; n=2, 9% in Spanish; n=11, 48% public insurance), 78% (n=18) of interviewees engaged with the chatbot. Interviewees provided positive feedback on usability and content and recommendations for improving the outreach messages.

CONCLUSIONS

Chatbots are a promising strategy to reach birthing individuals and newborn caregivers with information about postpartum recovery and newborn care, but intentional outreach and engagement strategies are needed to optimize interaction. Future work should measure the chatbot's impact on health outcomes and reduce disparities.

Realizing the potential of social determinants data in EHR systems: A scoping review of approaches for screening, linkage, extraction, analysis, and interventions

Background

Social determinants of health (SDoH), such as socioeconomics and neighborhoods, strongly influence health outcomes. However, the current state of standardized SDoH data in electronic health records (EHRs) is lacking, a significant barrier to research and care quality.

Methods

We conducted a PubMed search using "SDOH" and "EHR" Medical Subject Headings terms, analyzing included articles across five domains: 1) SDoH screening and assessment approaches, 2) SDoH data collection and documentation, 3) Use of natural language processing (NLP) for extracting SDoH, 4) SDoH data and health outcomes, and 5) SDoH-driven interventions.

Results

Of 685 articles identified, 324 underwent full review. Key findings include implementation of tailored screening instruments, census and claims data linkage for contextual SDoH profiles, NLP systems extracting SDoH from notes, associations between SDoH and healthcare utilization and chronic disease control, and integrated care management programs. However, variability across data sources, tools, and outcomes underscores the need for standardization.

Discussion

Despite progress in identifying patient social needs, further development of standards, predictive models, and coordinated interventions is critical for SDoH-EHR integration. Additional database searches could strengthen this scoping review. Ultimately, widespread capture, analysis, and translation of multidimensional SDoH data into clinical care is essential for promoting health equity.

Health Professionals' Views on the Use of Conversational Agents for Health Care: Qualitative Descriptive Study

BACKGROUND

In recent years, there has been an increase in the use of conversational agents for health promotion and service delivery. To date, health professionals' views on the use of this technology have received limited attention in the literature.

OBJECTIVE

The purpose of this study was to gain a better understanding of how health professionals view the use of conversational agents for health care.

METHODS

Physicians, nurses, and regulated mental health professionals were recruited using various web-based methods. Participants were interviewed individually using the Zoom (Zoom Video Communications, Inc) videoconferencing platform. Interview questions focused on the potential benefits and risks of using conversational agents for health care, as well as the best way to integrate conversational agents into the health care system. Interviews were transcribed verbatim and uploaded to NVivo (version 12; QSR International, Inc) for thematic analysis.

RESULTS

A total of 24 health professionals participated in the study (19 women, 5 men; mean age 42.75, SD 10.71 years). Participants said that the use of conversational agents for health care could have certain benefits, such as greater access to care for patients or clients and workload support for health professionals. They also discussed potential drawbacks, such as an added burden on health professionals (eg, program familiarization) and the limited capabilities of these programs. Participants said that conversational agents could be used for routine or basic tasks, such as screening and assessment, providing information and education, and supporting individuals between appointments. They also said that health professionals should have some oversight in terms of the development and implementation of these programs.

CONCLUSIONS

The results of this study provide insight into health professionals' views on the use of conversational agents for health care, particularly in terms of the benefits and drawbacks of these programs and how they should be integrated into the health care system. These collective findings offer useful information and guidance to stakeholders who have an interest in the development and implementation of this technology.

Depiction of conversational agents as health professionals: a scoping review

OBJECTIVE

The purpose of this scoping review was to examine the depiction of conversational agents as health professionals. We identified the professional characteristics that are used with these depictions and determined the prevalence of these characteristics among conversational agents that are used for health care.

INTRODUCTION

The depiction of conversational agents as health professionals has implications for both the users and the developers of these programs. For this reason, it is important to know more about these depictions and how they are implemented in practical settings.

INCLUSION CRITERIA

This review included scholarly literature on conversational agents that are used for health care. It focused on conversational agents designed for patients and health seekers, not health professionals or trainees. Conversational agents that address physical and/or mental health care were considered, as were programs that promote healthy behaviors.

METHODS

This review was conducted in accordance with JBI methodology for scoping reviews. The databases searched included MEDLINE (PubMed), Embase, CINAHL with Full Text (EBSCOhost), Scopus, Web of Science, ACM Guide to Computing Literature (Association for Computing Machinery Digital Library), and IEEE Xplore (IEEE). The main database search was conducted in June 2021, and an updated search was conducted in January 2022. Extracted data included characteristics of the report, basic characteristics of the conversational agent, and professional characteristics of the conversational agent. Extracted data were summarized using descriptive statistics. Results are presented in a narrative summary and accompanying tables.

RESULTS

A total of 38 health-related conversational agents were identified across 41 reports. Six of these conversational agents (15.8%) had professional characteristics. Four conversational agents (10.5%) had a professional appearance in which they displayed the clothing and accessories of health professionals and appeared in professional settings. One conversational agent (2.6%) had a professional title (Dr), and 4 conversational agents (10.5%) were described as having professional roles. Professional characteristics were more common among embodied vs disembodied conversational agents.

CONCLUSIONS

The results of this review show that the depiction of conversational agents as health professionals is not particularly common, although it does occur. More discussion is needed on the potential ethical and legal issues surrounding the depiction of conversational agents as health professionals. Future research should examine the impact of these depictions, as well as people's attitudes toward them, to better inform recommendations for practice.

Dialogue agents for artificial intelligence-based conversational systems for cognitively disabled: a systematic review

PURPOSE

We present a systematic literature review of dialogue agents for Artificial Intelligence (AI) and agent-based conversational systems dealing with cognitive disability of aged and impaired people including dementia and Parkinson's disease. We analyze current applications, gaps, and challenges in the existing research body, and provide guidelines and recommendations for their future development and use.

MATERIALS AND METHODS

We perform this study by applying Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) criteria. We performed a systematic search using relevant databases (ACM Digital Library, Google Scholar, IEEE Xplore, PubMed, and Scopus).

RESULTS

This study identified 468 articles on the use of conversational agents in healthcare. We finally selected 124 articles based on their objectives and content as directly related to our main topic.

CONCLUSION

We identified the main challenges in the field and analyzed the typical examples of the application of conversational agents in the healthcare domain, the desired characteristics of conversational agents, and chatbot support for aged people and people with cognitive disabilities. Our results contribute to a discussion on conversational health agents and emphasize current knowledge gaps and challenges for future research.IMPLICATIONS FOR REHABILITATIONA systematic literature review of dialogue agents for artificial intelligence and agent-based conversational systems dealing with cognitive disability of aged and impaired people.Main challenges and desired characteristics of the conversational agents, and chatbot support for aged people and people with cognitive disability.Current knowledge gaps and challenges for remote healthcare and rehabilitation.Guidelines and recommendations for future development and use of conversational systems.

Beyond rating scales: With targeted evaluation, large language models are poised for psychological assessment

In this narrative review, we survey recent empirical evaluations of AI-based language assessments and present a case for the technology of large language models to be poised for changing standardized psychological assessment. Artificial intelligence has been undergoing a purported "paradigm shift" initiated by new machine learning models, large language models (e.g., BERT, LAMMA, and that behind ChatGPT). These models have led to unprecedented accuracy over most computerized language processing tasks, from web searches to automatic machine translation and question answering, while their dialogue-based forms, like ChatGPT have captured the interest of over a million users. The success of the large language model is mostly attributed to its capability to numerically represent words in their context, long a weakness of previous attempts to automate psychological assessment from language. While potential applications for automated therapy are beginning to be studied on the heels of chatGPT's success, here we present evidence that suggests, with thorough validation of targeted deployment scenarios, that AI's newest technology can move mental health assessment away from rating scales and to instead use how people naturally communicate, in language.

Is ChatGPT a trusted source of information for total hip and knee arthroplasty patients?

Aims

While internet search engines have been the primary information source for patients' questions, artificial intelligence large language models like ChatGPT are trending towards becoming the new primary source. The purpose of this study was to determine if ChatGPT can answer patient questions about total hip (THA) and knee arthroplasty (TKA) with consistent accuracy, comprehensiveness, and easy readability.

Methods

We posed the 20 most Google-searched questions about THA and TKA, plus ten additional postoperative questions, to ChatGPT. Each question was asked twice to evaluate for consistency in quality. Following each response, we responded with, "Please explain so it is easier to understand," to evaluate ChatGPT's ability to reduce response reading grade level, measured as Flesch-Kincaid Grade Level (FKGL). Five resident physicians rated the 120 responses on 1 to 5 accuracy and comprehensiveness scales. Additionally, they answered a "yes" or "no" question regarding acceptability. Mean scores were calculated for each question, and responses were deemed acceptable if ≥ four raters answered "yes."

Results

The mean accuracy and comprehensiveness scores were 4.26 (95% confidence interval (CI) 4.19 to 4.33) and 3.79 (95% CI 3.69 to 3.89), respectively. Out of all the responses, 59.2% (71/120; 95% CI 50.0% to 67.7%) were acceptable. ChatGPT was consistent when asked the same question twice, giving no significant difference in accuracy (t = 0.821; p = 0.415), comprehensiveness (t = 1.387; p = 0.171), acceptability (χ2 = 1.832; p = 0.176), and FKGL (t = 0.264; p = 0.793). There was a significantly lower FKGL (t = 2.204; p = 0.029) for easier responses (11.14; 95% CI 10.57 to 11.71) than original responses (12.15; 95% CI 11.45 to 12.85).

Conclusion

ChatGPT answered THA and TKA patient questions with accuracy comparable to previous reports of websites, with adequate comprehensiveness, but with limited acceptability as the sole information source. ChatGPT has potential for answering patient questions about THA and TKA, but needs improvement.

Realizing the Potential of Social Determinants Data: A Scoping Review of Approaches for Screening, Linkage, Extraction, Analysis and Interventions

Background

Social determinants of health (SDoH) like socioeconomics and neighborhoods strongly influence outcomes, yet standardized SDoH data is lacking in electronic health records (EHR), limiting research and care quality.

Methods

We searched PubMed using keywords "SDOH" and "EHR", underwent title/abstract and full-text screening. Included records were analyzed under five domains: 1) SDoH screening and assessment approaches, 2) SDoH data collection and documentation, 3) Use of natural language processing (NLP) for extracting SDoH, 4) SDoH data and health outcomes, and 5) SDoH-driven interventions.

Results

We identified 685 articles, of which 324 underwent full review. Key findings include tailored screening instruments implemented across settings, census and claims data linkage providing contextual SDoH profiles, rule-based and neural network systems extracting SDoH from notes using NLP, connections found between SDoH data and healthcare utilization/chronic disease control, and integrated care management programs executed. However, considerable variability persists across data sources, tools, and outcomes.

Discussion

Despite progress identifying patient social needs, further development of standards, predictive models, and coordinated interventions is critical to fulfill the potential of SDoH-EHR integration. Additional database searches could strengthen this scoping review. Ultimately widespread capture, analysis, and translation of multidimensional SDoH data into clinical care is essential for promoting health equity.

Demographic and clinical characteristics associated with anxiety and depressive symptom outcomes in users of a digital mental health intervention incorporating a relational agent

BACKGROUND

Digital mental health interventions (DMHIs) may reduce treatment access issues for those experiencing depressive and/or anxiety symptoms. DMHIs that incorporate relational agents may offer unique ways to engage and respond to users and to potentially help reduce provider burden. This study tested Woebot for Mood & Anxiety (W-MA-02), a DMHI that employs Woebot, a relational agent that incorporates elements of several evidence-based psychotherapies, among those with baseline clinical levels of depressive or anxiety symptoms. Changes in self-reported depressive and anxiety symptoms over 8 weeks were measured, along with the association between each of these outcomes and demographic and clinical characteristics.

METHODS

This exploratory, single-arm, 8-week study of 256 adults yielded non-mutually exclusive subsamples with either clinical levels of depressive or anxiety symptoms at baseline. Week 8 Patient Health Questionnaire-8 (PHQ-8) changes were measured in the depressive subsample (PHQ-8 ≥ 10). Week 8 Generalized Anxiety Disorder-7 (GAD-7) changes were measured in the anxiety subsample (GAD-7 ≥ 10). Demographic and clinical characteristics were examined in association with symptom changes via bivariate and multiple regression models adjusted for W-MA-02 utilization. Characteristics included age, sex at birth, race/ethnicity, marital status, education, sexual orientation, employment status, health insurance, baseline levels of depressive and anxiety symptoms, and concurrent psychotherapeutic or psychotropic medication treatments during the study.

RESULTS

Both the depressive and anxiety subsamples were predominantly female, educated, non-Hispanic white, and averaged 38 and 37 years of age, respectively. The depressive subsample had significant reductions in depressive symptoms at Week 8 (mean change =-7.28, SD = 5.91, Cohen's d = -1.23, p < 0.01); the anxiety subsample had significant reductions in anxiety symptoms at Week 8 (mean change = -7.45, SD = 5.99, Cohen's d = -1.24, p < 0.01). No significant associations were found between sex at birth, age, employment status, educational background and Week 8 symptom changes. Significant associations between depressive and anxiety symptom outcomes and sexual orientation, marital status, concurrent mental health treatment, and baseline symptom severity were found.

CONCLUSIONS

The present study suggests early promise for W-MA-02 as an intervention for depression and/or anxiety symptoms. Although exploratory in nature, this study revealed potential user characteristics associated with outcomes that can be investigated in future studies.

TRIAL REGISTRATION

This study was retrospectively registered on ClinicalTrials.gov (#NCT05672745) on January 5th, 2023.

Loneliness and suicide mitigation for students using GPT3-enabled chatbots

Mental health is a crisis for learners globally, and digital support is increasingly seen as a critical resource. Concurrently, Intelligent Social Agents receive exponentially more engagement than other conversational systems, but their use in digital therapy provision is nascent. A survey of 1006 student users of the Intelligent Social Agent, Replika, investigated participants' loneliness, perceived social support, use patterns, and beliefs about Replika. We found participants were more lonely than typical student populations but still perceived high social support. Many used Replika in multiple, overlapping ways-as a friend, a therapist, and an intellectual mirror. Many also held overlapping and often conflicting beliefs about Replika-calling it a machine, an intelligence, and a human. Critically, 3% reported that Replika halted their suicidal ideation. A comparative analysis of this group with the wider participant population is provided.

A text-based conversational agent for asthma support: Mixed-methods feasibility study

Objective

Millions of people in the UK have asthma, yet 70% do not access basic care, leading to the largest number of asthma-related deaths in Europe. Chatbots may extend the reach of asthma support and provide a bridge to traditional healthcare. This study evaluates 'Brisa', a chatbot designed to improve asthma patients' self-assessment and self-management.

Methods

We recruited 150 adults with an asthma diagnosis to test our chatbot. Participants were recruited over three waves through social media and a research recruitment platform. Eligible participants had access to 'Brisa' via a WhatsApp or website version for 28 days and completed entry and exit questionnaires to evaluate user experience and asthma control. Weekly symptom tracking, user interaction metrics, satisfaction measures, and qualitative feedback were utilised to evaluate the chatbot's usability and potential effectiveness, focusing on changes in asthma control and self-reported behavioural improvements.

Results

74% of participants engaged with 'Brisa' at least once. High task completion rates were observed: asthma attack risk assessment (86%), voice recording submission (83%) and asthma control tracking (95.5%). Post use, an 8% improvement in asthma control was reported. User satisfaction surveys indicated positive feedback on helpfulness (80%), privacy (87%), trustworthiness (80%) and functionality (84%) but highlighted a need for improved conversational depth and personalisation.

Conclusions

The study indicates that chatbots are effective for asthma support, demonstrated by the high usage of features like risk assessment and control tracking, as well as a statistically significant improvement in asthma control. However, lower satisfaction in conversational flexibility highlights rising expectations for chatbot fluency, influenced by advanced models like ChatGPT. Future health-focused chatbots must balance conversational capability with accuracy and safety to maintain engagement and effectiveness.

Digital technology and mental health during the COVID-19 pandemic: a narrative review with a focus on depression, anxiety, stress, and trauma

The sudden appearance and devastating effects of the COVID-19 pandemic resulted in the need for multiple adaptive changes in societies, business operations and healthcare systems across the world. This review describes the development and increased use of digital technologies such as chat bots, electronic diaries, online questionnaires and even video gameplay to maintain effective treatment standards for individuals with mental health conditions such as depression, anxiety and post-traumatic stress syndrome. We describe how these approaches have been applied to help meet the challenges of the pandemic in delivering mental healthcare solutions. The main focus of this narrative review is on describing how these digital platforms have been used in diagnostics, patient monitoring and as a treatment option for the general public, as well as for frontline medical staff suffering with mental health issues.

Existing Barriers Faced by and Future Design Recommendations for Direct-to-Consumer Health Care Artificial Intelligence Apps: Scoping Review

BACKGROUND

Direct-to-consumer (DTC) health care artificial intelligence (AI) apps hold the potential to bridge the spatial and temporal disparities in health care resources, but they also come with individual and societal risks due to AI errors. Furthermore, the manner in which consumers interact directly with health care AI is reshaping traditional physician-patient relationships. However, the academic community lacks a systematic comprehension of the research overview for such apps.

OBJECTIVE

This paper systematically delineated and analyzed the characteristics of included studies, identified existing barriers and design recommendations for DTC health care AI apps mentioned in the literature and also provided a reference for future design and development.

METHODS

This scoping review followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews guidelines and was conducted according to Arksey and O'Malley's 5-stage framework. Peer-reviewed papers on DTC health care AI apps published until March 27, 2023, in Web of Science, Scopus, the ACM Digital Library, IEEE Xplore, PubMed, and Google Scholar were included. The papers were analyzed using Braun and Clarke's reflective thematic analysis approach.

RESULTS

Of the 2898 papers retrieved, 32 (1.1%) covering this emerging field were included. The included papers were recently published (2018-2023), and most (23/32, 72%) were from developed countries. The medical field was mostly general practice (8/32, 25%). In terms of users and functionalities, some apps were designed solely for single-consumer groups (24/32, 75%), offering disease diagnosis (14/32, 44%), health self-management (8/32, 25%), and health care information inquiry (4/32, 13%). Other apps connected to physicians (5/32, 16%), family members (1/32, 3%), nursing staff (1/32, 3%), and health care departments (2/32, 6%), generally to alert these groups to abnormal conditions of consumer users. In addition, 8 barriers and 6 design recommendations related to DTC health care AI apps were identified. Some more subtle obstacles that are particularly worth noting and corresponding design recommendations in consumer-facing health care AI systems, including enhancing human-centered explainability, establishing calibrated trust and addressing overtrust, demonstrating empathy in AI, improving the specialization of consumer-grade products, and expanding the diversity of the test population, were further discussed.

CONCLUSIONS

The booming DTC health care AI apps present both risks and opportunities, which highlights the need to explore their current status. This paper systematically summarized and sorted the characteristics of the included studies, identified existing barriers faced by, and made future design recommendations for such apps. To the best of our knowledge, this is the first study to systematically summarize and categorize academic research on these apps. Future studies conducting the design and development of such systems could refer to the results of this study, which is crucial to improve the health care services provided by DTC health care AI apps.

An Integrative Survey on Mental Health Conversational Agents to Bridge Computer Science and Medical Perspectives

Mental health conversational agents (a.k.a. chatbots) are widely studied for their potential to offer accessible support to those experiencing mental health challenges. Previous surveys on the topic primarily consider papers published in either computer science or medicine, leading to a divide in understanding and hindering the sharing of beneficial knowledge between both domains. To bridge this gap, we conduct a comprehensive literature review using the PRISMA framework, reviewing 534 papers published in both computer science and medicine. Our systematic review reveals 136 key papers on building mental health-related conversational agents with diverse characteristics of modeling and experimental design techniques. We find that computer science papers focus on LLM techniques and evaluating response quality using automated metrics with little attention to the application while medical papers use rule-based conversational agents and outcome metrics to measure the health outcomes of participants. Based on our findings on transparency, ethics, and cultural heterogeneity in this review, we provide a few recommendations to help bridge the disciplinary divide and enable the cross-disciplinary development of mental health conversational agents.

Acceptability of a Pain History Assessment and Education Chatbot (Dolores) Across Age Groups in Populations With Chronic Pain: Development and Pilot Testing

BACKGROUND

The delivery of education on pain neuroscience and the evidence for different treatment approaches has become a key component of contemporary persistent pain management. Chatbots, or more formally conversation agents, are increasingly being used in health care settings due to their versatility in providing interactive and individualized approaches to both capture and deliver information. Research focused on the acceptability of diverse chatbot formats can assist in developing a better understanding of the educational needs of target populations.

OBJECTIVE

This study aims to detail the development and initial pilot testing of a multimodality pain education chatbot (Dolores) that can be used across different age groups and investigate whether acceptability and feedback were comparable across age groups following pilot testing.

METHODS

Following an initial design phase involving software engineers (n=2) and expert clinicians (n=6), a total of 60 individuals with chronic pain who attended an outpatient clinic at 1 of 2 pain centers in Australia were recruited for pilot testing. The 60 individuals consisted of 20 (33%) adolescents (aged 10-18 years), 20 (33%) young adults (aged 19-35 years), and 20 (33%) adults (aged >35 years) with persistent pain. Participants spent 20 to 30 minutes completing interactive chatbot activities that enabled the Dolores app to gather a pain history and provide education about pain and pain treatments. After the chatbot activities, participants completed a custom-made feedback questionnaire measuring the acceptability constructs pertaining to health education chatbots. To determine the effect of age group on the acceptability ratings and feedback provided, a series of binomial logistic regression models and cumulative odds ordinal logistic regression models with proportional odds were generated.

RESULTS

Overall, acceptability was high for the following constructs: engagement, perceived value, usability, accuracy, responsiveness, adoption intention, esthetics, and overall quality. The effect of age group on all acceptability ratings was small and not statistically significant. An analysis of open-ended question responses revealed that major frustrations with the app were related to Dolores' speech, which was explored further through a comparative analysis. With respect to providing negative feedback about Dolores' speech, a logistic regression model showed that the effect of age group was statistically significant (χ22=11.7; P=.003) and explained 27.1% of the variance (Nagelkerke R2). Adults and young adults were less likely to comment on Dolores' speech compared with adolescent participants (odds ratio 0.20, 95% CI 0.05-0.84 and odds ratio 0.05, 95% CI 0.01-0.43, respectively). Comments were related to both speech rate (too slow) and quality (unpleasant and robotic).

CONCLUSIONS

This study provides support for the acceptability of pain history and education chatbots across different age groups. Chatbot acceptability for adolescent cohorts may be improved by enabling the self-selection of speech characteristics such as rate and personable tone.

Conversational AI and Vaccine Communication: Systematic Review of the Evidence

BACKGROUND

Since the mid-2010s, use of conversational artificial intelligence (AI; chatbots) in health care has expanded significantly, especially in the context of increased burdens on health systems and restrictions on in-person consultations with health care providers during the COVID-19 pandemic. One emerging use for conversational AI is to capture evolving questions and communicate information about vaccines and vaccination.

OBJECTIVE

The objective of this systematic review was to examine documented uses and evidence on the effectiveness of conversational AI for vaccine communication.

METHODS

This systematic review was conducted following the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. PubMed, Web of Science, PsycINFO, MEDLINE, Scopus, CINAHL Complete, Cochrane Library, Embase, Epistemonikos, Global Health, Global Index Medicus, Academic Search Complete, and the University of London library database were searched for papers on the use of conversational AI for vaccine communication. The inclusion criteria were studies that included (1) documented instances of conversational AI being used for the purpose of vaccine communication and (2) evaluation data on the impact and effectiveness of the intervention.

RESULTS

After duplicates were removed, the review identified 496 unique records, which were then screened by title and abstract, of which 38 were identified for full-text review. Seven fit the inclusion criteria and were assessed and summarized in the findings of this review. Overall, vaccine chatbots deployed to date have been relatively simple in their design and have mainly been used to provide factual information to users in response to their questions about vaccines. Additionally, chatbots have been used for vaccination scheduling, appointment reminders, debunking misinformation, and, in some cases, for vaccine counseling and persuasion. Available evidence suggests that chatbots can have a positive effect on vaccine attitudes; however, studies were typically exploratory in nature, and some lacked a control group or had very small sample sizes.

CONCLUSIONS

The review found evidence of potential benefits from conversational AI for vaccine communication. Factors that may contribute to the effectiveness of vaccine chatbots include their ability to provide credible and personalized information in real time, the familiarity and accessibility of the chatbot platform, and the extent to which interactions with the chatbot feel "natural" to users. However, evaluations have focused on the short-term, direct effects of chatbots on their users. The potential longer-term and societal impacts of conversational AI have yet to be analyzed. In addition, existing studies do not adequately address how ethics apply in the field of conversational AI around vaccines. In a context where further digitalization of vaccine communication can be anticipated, additional high-quality research will be required across all these areas.

These Aren't The Droids You Are Looking for: Promises and Challenges for the Intersection of Affective Science and Robotics/AI

AI research focused on interactions with humans, particularly in the form of robots or virtual agents, has expanded in the last two decades to include concepts related to affective processes. Affective computing is an emerging field that deals with issues such as how the diagnosis of affective states of users can be used to improve such interactions, also with a view to demonstrate affective behavior towards the user. This type of research often is based on two beliefs: (1) artificial emotional intelligence will improve human computer interaction (or more specifically human robot interaction), and (2) we understand the role of affective behavior in human interaction sufficiently to tell artificial systems what to do. However, within affective science the focus of research is often to test a particular assumption, such as "smiles affect liking." Such focus does not provide the information necessary to synthesize affective behavior in long dynamic and real-time interactions. In consequence, theories do not play a large role in the development of artificial affective systems by engineers, but self-learning systems develop their behavior out of large corpora of recorded interactions. The status quo is characterized by measurement issues, theoretical lacunae regarding prevalence and functions of affective behavior in interaction, and underpowered studies that cannot provide the solid empirical foundation for further theoretical developments. This contribution will highlight some of these challenges and point towards next steps to create a rapprochement between engineers and affective scientists with a view to improving theory and solid applications.

Digital transformation of mental health services

This paper makes a case for digital mental health and provides insights into how digital technologies can enhance (but not replace) existing mental health services. We describe digital mental health by presenting a suite of digital technologies (from digital interventions to the application of artificial intelligence). We discuss the benefits of digital mental health, for example, a digital intervention can be an accessible stepping-stone to receiving support. The paper does, however, present less-discussed benefits with new concepts such as 'poly-digital', where many different apps/features (e.g. a sleep app, mood logging app and a mindfulness app, etc.) can each address different factors of wellbeing, perhaps resulting in an aggregation of marginal gains. Another benefit is that digital mental health offers the ability to collect high-resolution real-world client data and provide client monitoring outside of therapy sessions. These data can be collected using digital phenotyping and ecological momentary assessment techniques (i.e. repeated mood or scale measures via an app). This allows digital mental health tools and real-world data to inform therapists and enrich face-to-face sessions. This can be referred to as blended care/adjunctive therapy where service users can engage in 'channel switching' between digital and non-digital (face-to-face) interventions providing a more integrated service. This digital integration can be referred to as a kind of 'digital glue' that helps join up the in-person sessions with the real world. The paper presents the challenges, for example, the majority of mental health apps are maybe of inadequate quality and there is a lack of user retention. There are also ethical challenges, for example, with the perceived 'over-promotion' of screen-time and the perceived reduction in care when replacing humans with 'computers', and the trap of 'technological solutionism' whereby technology can be naively presumed to solve all problems. Finally, we argue for the need to take an evidence-based, systems thinking and co-production approach in the form of stakeholder-centred design when developing digital mental health services based on technologies. The main contribution of this paper is the integration of ideas from many different disciplines as well as the framework for blended care using 'channel switching' to showcase how digital data and technology can enrich physical services. Another contribution is the emergence of 'poly-digital' and a discussion on the challenges of digital mental health, specifically 'digital ethics'.

An Artificial Therapist (Manage Your Life Online) to Support the Mental Health of Youth: Co-Design and Case Series

BACKGROUND

The prevalence of child and adolescent mental health issues is increasing faster than the number of services available, leading to a shortfall. Mental health chatbots are a highly scalable method to address this gap. Manage Your Life Online (MYLO) is an artificially intelligent chatbot that emulates the method of levels therapy. Method of levels is a therapy that uses curious questioning to support the sustained awareness and exploration of current problems.

OBJECTIVE

This study aimed to assess the feasibility and acceptability of a co-designed interface for MYLO in young people aged 16 to 24 years with mental health problems.

METHODS

An iterative co-design phase occurred over 4 months, in which feedback was elicited from a group of young people (n=7) with lived experiences of mental health issues. This resulted in the development of a progressive web application version of MYLO that could be used on mobile phones. We conducted a case series to assess the feasibility and acceptability of MYLO in 13 young people over 2 weeks. During this time, the participants tested MYLO and completed surveys including clinical outcomes and acceptability measures. We then conducted focus groups and interviews and used thematic analysis to obtain feedback on MYLO and identify recommendations for further improvements.

RESULTS

Most participants were positive about their experience of using MYLO and would recommend MYLO to others. The participants enjoyed the simplicity of the interface, found it easy to use, and rated it as acceptable using the System Usability Scale. Inspection of the use data found evidence that MYLO can learn and adapt its questioning in response to user input. We found a large effect size for the decrease in participants' problem-related distress and a medium effect size for the increase in their self-reported tendency to resolve goal conflicts (the proposed mechanism of change) in the testing phase. Some patients also experienced a reliable change in their clinical outcome measures over the 2 weeks.

CONCLUSIONS

We established the feasibility and acceptability of MYLO. The initial outcomes suggest that MYLO has the potential to support the mental health of young people and help them resolve their own problems. We aim to establish whether the use of MYLO leads to a meaningful reduction in participants' symptoms of depression and anxiety and whether these are maintained over time by conducting a randomized controlled evaluation trial.

Conducting a systematic review and evaluation of commercially available mobile applications (apps) on a health-related topic: the TECH approach and a step-by-step methodological guide

OBJECTIVES

To provide an overview of the methodological considerations for conducting commercial smartphone health app reviews (mHealth reviews), with the aim of systematising the process and supporting high-quality evaluations of mHealth apps.

DESIGN

Synthesis of our research team's experiences of conducting and publishing various reviews of mHealth apps available on app stores and hand-searching the top medical informatics journals (eg, The Lancet Digital Health, npj Digital Medicine, Journal of Biomedical Informatics and the Journal of the American Medical Informatics Association) over the last five years (2018-2022) to identify other app reviews to contribute to the discussion of this method and supporting framework for developing a research (review) question and determining the eligibility criteria.

RESULTS

We present seven steps to support rigour in conducting reviews of health apps available on the app market: (1) writing a research question or aims, (2) conducting scoping searches and developing the protocol, (3) determining the eligibility criteria using the TECH framework, (4) conducting the final search and screening of health apps, (5) data extraction, (6) quality, functionality and other assessments and (7) analysis and synthesis of findings. We introduce the novel TECH approach to developing review questions and the eligibility criteria, which considers the Target user, Evaluation focus, Connectedness and the Health domain. Patient and public involvement and engagement opportunities are acknowledged, including co-developing the protocol and undertaking quality or usability assessments.

CONCLUSION

Commercial mHealth app reviews can provide important insights into the health app market, including the availability of apps and their quality and functionality. We have outlined seven key steps for conducting rigorous health app reviews in addition to the TECH acronym, which can support researchers in writing research questions and determining the eligibility criteria. Future work will include a collaborative effort to develop reporting guidelines and a quality appraisal tool to ensure transparency and quality in systematic app reviews.

Trialling a Large Language Model (ChatGPT) in General Practice With the Applied Knowledge Test: Observational Study Demonstrating Opportunities and Limitations in Primary Care

BACKGROUND

Large language models exhibiting human-level performance in specialized tasks are emerging; examples include Generative Pretrained Transformer 3.5, which underlies the processing of ChatGPT. Rigorous trials are required to understand the capabilities of emerging technology, so that innovation can be directed to benefit patients and practitioners.

OBJECTIVE

Here, we evaluated the strengths and weaknesses of ChatGPT in primary care using the Membership of the Royal College of General Practitioners Applied Knowledge Test (AKT) as a medium.

METHODS

AKT questions were sourced from a web-based question bank and 2 AKT practice papers. In total, 674 unique AKT questions were inputted to ChatGPT, with the model's answers recorded and compared to correct answers provided by the Royal College of General Practitioners. Each question was inputted twice in separate ChatGPT sessions, with answers on repeated trials compared to gauge consistency. Subject difficulty was gauged by referring to examiners' reports from 2018 to 2022. Novel explanations from ChatGPT-defined as information provided that was not inputted within the question or multiple answer choices-were recorded. Performance was analyzed with respect to subject, difficulty, question source, and novel model outputs to explore ChatGPT's strengths and weaknesses.

RESULTS

Average overall performance of ChatGPT was 60.17%, which is below the mean passing mark in the last 2 years (70.42%). Accuracy differed between sources (P=.04 and .06). ChatGPT's performance varied with subject category (P=.02 and .02), but variation did not correlate with difficulty (Spearman ρ=-0.241 and -0.238; P=.19 and .20). The proclivity of ChatGPT to provide novel explanations did not affect accuracy (P>.99 and .23).

CONCLUSIONS

Large language models are approaching human expert-level performance, although further development is required to match the performance of qualified primary care physicians in the AKT. Validated high-performance models may serve as assistants or autonomous clinical tools to ameliorate the general practice workforce crisis.

Assessing the Feasibility of a Text-Based Conversational Agent for Asthma Support: Protocol for a Mixed Methods Observational Study

BACKGROUND

Despite efforts, the UK death rate from asthma is the highest in Europe, and 65% of people with asthma in the United Kingdom do not receive the professional care they are entitled to. Experts have recommended the use of digital innovations to help address the issues of poor outcomes and lack of care access. An automated SMS text messaging-based conversational agent (ie, chatbot) created to provide access to asthma support in a familiar format via a mobile phone has the potential to help people with asthma across demographics and at scale. Such a chatbot could help improve the accuracy of self-assessed risk, improve asthma self-management, increase access to professional care, and ultimately reduce asthma attacks and emergencies.

OBJECTIVE

The aims of this study are to determine the feasibility and usability of a text-based conversational agent that processes a patient's text responses and short sample voice recordings to calculate an estimate of their risk for an asthma exacerbation and then offers follow-up information for lowering risk and improving asthma control; assess the levels of engagement for different groups of users, particularly those who do not access professional services and those with poor asthma control; and assess the extent to which users of the chatbot perceive it as helpful for improving their understanding and self-management of their condition.

METHODS

We will recruit 300 adults through four channels for broad reach: Facebook, YouGov, Asthma + Lung UK social media, and the website Healthily (a health self-management app). Participants will be screened, and those who meet inclusion criteria (adults diagnosed with asthma and who use WhatsApp) will be provided with a link to access the conversational agent through WhatsApp on their mobile phones. Participants will be sent scheduled and randomly timed messages to invite them to engage in dialogue about their asthma risk during the period of study. After a data collection period (28 days), participants will respond to questionnaire items related to the quality of the interaction. A pre- and postquestionnaire will measure asthma control before and after the intervention.

RESULTS

This study was funded in March 2021 and started in January 2022. We developed a prototype conversational agent, which was iteratively improved with feedback from people with asthma, asthma nurses, and specialist doctors. Fortnightly reviews of iterations by the clinical team began in September 2022 and are ongoing. This feasibility study will start recruitment in January 2023. The anticipated completion of the study is July 2023. A future randomized controlled trial will depend on the outcomes of this study and funding.

CONCLUSIONS

This feasibility study will inform a follow-up pilot and larger randomized controlled trial to assess the impact of a conversational agent on asthma outcomes, self-management, behavior change, and access to care.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

PRR1-10.2196/42965.

Digital mental health: challenges and next steps

Digital innovations in mental health offer great potential, but present unique challenges. Using a consensus development panel approach, an expert, international, cross-disciplinary panel met to provide a framework to conceptualise digital mental health innovations, research into mechanisms and effectiveness and approaches for clinical implementation. Key questions and outputs from the group were agreed by consensus, and are presented and discussed in the text and supported by case examples in an accompanying appendix. A number of key themes emerged. (1) Digital approaches may work best across traditional diagnostic systems: we do not have effective ontologies of mental illness and transdiagnostic/symptom-based approaches may be more fruitful. (2) Approaches in clinical implementation of digital tools/interventions need to be creative and require organisational change: not only do clinicians and patients need training and education to be more confident and skilled in using digital technologies to support shared care decision-making, but traditional roles need to be extended, with clinicians working alongside digital navigators and non-clinicians who are delivering protocolised treatments. (3) Designing appropriate studies to measure the effectiveness of implementation is also key: including digital data raises unique ethical issues, and measurement of potential harms is only just beginning. (4) Accessibility and codesign are needed to ensure innovations are long lasting. (5) Standardised guidelines for reporting would ensure effective synthesis of the evidence to inform clinical implementation. COVID-19 and the transition to virtual consultations have shown us the potential for digital innovations to improve access and quality of care in mental health: now is the ideal time to act.

Usability and Credibility of a COVID-19 Vaccine Chatbot for Young Adults and Health Workers in the United States: Formative Mixed Methods Study

BACKGROUND

The COVID-19 pandemic raised novel challenges in communicating reliable, continually changing health information to a broad and sometimes skeptical public, particularly around COVID-19 vaccines, which, despite being comprehensively studied, were the subject of viral misinformation. Chatbots are a promising technology to reach and engage populations during the pandemic. To inform and communicate effectively with users, chatbots must be highly usable and credible.

OBJECTIVE

We sought to understand how young adults and health workers in the United States assessed the usability and credibility of a web-based chatbot called Vira, created by the Johns Hopkins Bloomberg School of Public Health and IBM Research using natural language processing technology. Using a mixed method approach, we sought to rapidly improve Vira's user experience to support vaccine decision-making during the peak of the COVID-19 pandemic.

METHODS

We recruited racially and ethnically diverse young people and health workers, with both groups from urban areas of the United States. We used the validated Chatbot Usability Questionnaire to understand the tool's navigation, precision, and persona. We also conducted 11 interviews with health workers and young people to understand the user experience, whether they perceived the chatbot as confidential and trustworthy, and how they would use the chatbot. We coded and categorized emerging themes to understand the determining factors for participants' assessment of chatbot usability and credibility.

RESULTS

In all, 58 participants completed a web-based usability questionnaire and 11 completed in-depth interviews. Most questionnaire respondents said the chatbot was "easy to navigate" (51/58, 88%) and "very easy to use" (50/58, 86%), and many (45/58, 78%) said its responses were relevant. The mean Chatbot Usability Questionnaire score was 70.2 (SD 12.1) and scores ranged from 40.6 to 95.3. Interview participants felt the chatbot achieved high usability due to its strong functionality, performance, and perceived confidentiality and that the chatbot could attain high credibility with a redesign of its cartoonish visual persona. Young people said they would use the chatbot to discuss vaccination with hesitant friends or family members, whereas health workers used or anticipated using the chatbot to support community outreach, save time, and stay up to date.

CONCLUSIONS

This formative study conducted during the pandemic's peak provided user feedback for an iterative redesign of Vira. Using a mixed method approach provided multidimensional feedback, identifying how the chatbot worked well-being easy to use, answering questions appropriately, and using credible branding-while offering tangible steps to improve the product's visual design. Future studies should evaluate how chatbots support personal health decision-making, particularly in the context of a public health emergency, and whether such outreach tools can reduce staff burnout. Randomized studies should also be conducted to measure how chatbots countering health misinformation affect user knowledge, attitudes, and behavior.

Cocreation of a conversational agent to help patients adhere to their varenicline treatment: A study protocol

Objective

Varenicline is the most efficacious approved smoking cessation medication, making it one of the most cost-effective clinical interventions for reducing tobacco-related morbidity and mortality. Adhering to varenicline is strongly associated with smoking cessation. Healthbots have the potential to help people adhere to their medications by scaling up evidence-based behavioral interventions. In this protocol, we outline how we will follow the UK's Medical Research Council's guidance to codesign a theory-informed, evidence-based, and patient-centered healthbot to help people adhere to varenicline.

Methods

The study will utilize the Discover, Design and Build, and Test framework and will include three phases: (a) a rapid review and interviews with 20 patients and 20 healthcare providers to understand barriers and facilitators to varenicline adherence (Discover phase); (b) Wizard of Oz test to design the healthbot and get a sense of the questions that chatbot has to be able to answer (Design phase); and (c) building, training, and beta-testing the healthbot (Building and Testing phases) where the Nonadoption, Abandonment, Scale-up, Spread, and Sustainability framework will be used to develop the healthbot using the simplest sensible solution, and 20 participants will beta test the healthbot. We will use the Capability, Opportunity, Motivation-Behavior (COM-B) model of behavior change and its associated framework, the Theoretical Domains Framework, to organize the findings.

Conclusions

The present approach will enable us to systematically identify the most appropriate features for the healthbot based on a well-established behavioral theory, the latest scientific evidence, and end users' and healthcare providers' knowledge.

Interactive Conversational Agents for Health Promotion, Prevention, and Care: Protocol for a Mixed Methods Systematic Scoping Review

BACKGROUND

Interactive conversational agents, also known as "chatbots," are computer programs that use natural language processing to engage in conversations with humans to provide or collect information. Although the literature on the development and use of chatbots for health interventions is growing, important knowledge gaps remain, such as identifying design aspects relevant to health care and functions to offer transparency in decision-making automation.

OBJECTIVE

This paper presents the protocol for a scoping review that aims to identify and categorize the interactive conversational agents currently used in health care.

METHODS

A mixed methods systematic scoping review will be conducted according to the Arksey and O'Malley framework and the guidance of Peters et al for systematic scoping reviews. A specific search strategy will be formulated for 5 of the most relevant databases to identify studies published in the last 20 years. Two reviewers will independently apply the inclusion criteria using the full texts and extract data. We will use structured narrative summaries of main themes to present a portrait of the current scope of available interactive conversational agents targeting health promotion, prevention, and care. We will also summarize the differences and similarities between these conversational agents.

RESULTS

The search strategy and screening steps were completed in March 2022. Data extraction and analysis started in May 2022, and the results are expected to be published in October 2022.

CONCLUSIONS

This fundamental knowledge will be useful for the development of interactive conversational agents adapted to specific groups in vulnerable situations in health care and community settings.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/40265.

Using Chatbot as an Alternative Approach for In-Person Tooth Brushing Training During the COVID-19 Pandemic

Background

It is recommended that caregivers receive oral health education and in-person training to improve toothbrushing for young children. To strengthen oral health education before COVID-19, the 21-Day FunDee chatbot with in-person toothbrushing training for caregivers was used. During the pandemic, practical experience was difficult to implement. Therefore, the 30-Day FunDee chatbot was created to extend the coverage of chatbots from 21 days to 30 days by incorporating more videos on toothbrushing demonstrations and dialogue. This was a secondary data comparison of 2 chatbots in similar rural areas of Pattani province: Maikan district (Study I) and Maelan district (Study II).

Objective

This study aimed to evaluate the effectiveness and usability of 2 chatbots, 21-Day FunDee (Study I) and 30-Day FunDee (Study II), based on the protection motivation theory (PMT). This study explored the feasibility of using the 30-Day FunDee chatbot to increase toothbrushing behaviors for caregivers in oral hygiene care for children aged 6 months to 36 months without in-person training during the COVID-19 pandemic.

Methods

A pre-post design was used in both studies. The effectiveness was evaluated among caregivers in terms of oral hygiene practices, knowledge, and oral health care perceptions based on PMT. In Study I, participants received in-person training and a 21-day chatbot course during October 2018 to February 2019. In Study II, participants received only daily chatbot programming for 30 days during December 2021 to February 2022. Data were gathered at baseline of each study and at 30 days and 60 days after the start of Study I and Study II, respectively. After completing their interventions, the chatbot's usability was assessed using open-ended questions. Study I evaluated the plaque score, whereas Study II included an in-depth interview. The 2 studies were compared to determine the feasibility of using the 30-Day FunDee chatbot as an alternative to in-person training.

Results

There were 71 pairs of participants: 37 in Study I and 34 in Study II. Both chatbots significantly improved overall knowledge (Study I: P<.001; Study II: P=.001), overall oral health care perceptions based on PMT (Study I: P<.001; Study II: P<.001), and toothbrushing for children by caregivers (Study I: P=.02; Study II: P=.04). Only Study I had statistically significant differences in toothbrushing at least twice a day (P=.002) and perceived vulnerability (P=.003). The highest overall chatbot satisfaction was 9.2 (SD 0.9) in Study I and 8.6 (SD 1.2) in Study II. In Study I, plaque levels differed significantly (P<.001).

Conclusions

This was the first study using a chatbot in oral health education. We established the effectiveness and usability of 2 chatbot programs for promoting oral hygiene care of young children by caregivers. The 30-Day FunDee chatbot showed the possibility of improving toothbrushing skills without requiring in-person training.

Trial Registration

Thai Clinical Trials Registry TCTR20191223005; http://www.thaiclinicaltrials.org/show/TCTR20191223005 and TCTR20210927004; https://www.thaiclinicaltrials.org/show/TCTR20210927004