Assessment of users' acceptability of a mobile-based embodied conversational agent for the prevention and detection of suicidal behaviour

Conversational assessment using artificial intelligence is as clinically useful as depression scales and preferred by users

BACKGROUND

Depression is prevalent, chronic, and burdensome. Due to limited screening access, depression often remains undiagnosed. Artificial intelligence (AI) models based on spoken responses to interview questions may offer an effective, efficient alternative to other screening methods.

OBJECTIVE

The primary aim was to use a demographically diverse sample to validate an AI model, previously trained on human-administered interviews, on novel bot-administered interviews, and to check for algorithmic biases related to age, sex, race, and ethnicity.

METHODS

Using the Aiberry app, adults recruited via social media (N = 393) completed a brief bot-administered interview and a depression self-report form. An AI model was used to predict form scores based on interview responses alone. For all meaningful discrepancies between model inference and form score, clinicians performed a masked review to determine which one they preferred.

RESULTS

There was strong concurrent validity between the model predictions and raw self-report scores (r = 0.73, MAE = 3.3). 90 % of AI predictions either agreed with self-report or with clinical expert opinion when AI contradicted self-report. There was no differential model performance across age, sex, race, or ethnicity.

LIMITATIONS

Limitations include access restrictions (English-speaking ability and access to smartphone or computer with broadband internet) and potential self-selection of participants more favorably predisposed toward AI technology.

CONCLUSION

The Aiberry model made accurate predictions of depression severity based on remotely collected spoken responses to a bot-administered interview. This study shows promising results for the use of AI as a mental health screening tool on par with self-report measures.

Conversational agents for depression screening: A systematic review

OBJECTIVE

This work explores the advances in conversational agents aimed at the detection of mental health disorders, and specifically the screening of depression. The focus is put on those based on voice interaction, but other approaches are also tackled, such as text-based interaction or embodied avatars.

METHODS

PRISMA was selected as the systematic methodology for the analysis of existing literature, which was retrieved from Scopus, PubMed, IEEE Xplore, APA PsycINFO, Cochrane, and Web of Science. Relevant research addresses the detection of depression using conversational agents, and the selection criteria utilized include their effectiveness, usability, personalization, and psychometric properties.

RESULTS

Of the 993 references initially retrieved, 36 were finally included in our work. The analysis of these studies allowed us to identify 30 conversational agents that claim to detect depression, specifically or in combination with other disorders such as anxiety or stress disorders. As a general approach, screening was implemented in the conversational agents taking as a reference standardized or psychometrically validated clinical tests, which were also utilized as a golden standard for their validation. The implementation of questionnaires such as Patient Health Questionnaire or the Beck Depression Inventory, which are used in 65% of the articles analyzed, stand out.

CONCLUSIONS

The usefulness of intelligent conversational agents allows screening to be administered to different types of profiles, such as patients (33% of relevant proposals) and caregivers (11%), although in many cases a target profile is not clearly of (66% of solutions analyzed). This study found 30 standalone conversational agents, but some proposals were explored that combine several approaches for a more enriching data acquisition. The interaction implemented in most relevant conversational agents is text-based, although the evolution is clearly towards voice integration, which in turns enhances their psychometric characteristics, as voice interaction is perceived as more natural and less invasive.

Embodied Conversational Agents Providing Motivational Interviewing to Improve Health-Related Behaviors: Scoping Review

BACKGROUND

Embodied conversational agents (ECAs) are advanced human-like interfaces that engage users in natural face-to-face conversations and interactions. These traits position ECAs as innovative tools for delivering interventions for promoting health-related behavior adoption. This includes motivational interviewing (MI), a therapeutic approach that combines brief interventions with motivational techniques to encourage the adoption of healthier behaviors.

OBJECTIVE

This study aims to identify the health issues addressed by ECAs delivering MI interventions, explore the key characteristics of these ECAs (eg, appearance, dialogue mechanism, emotional model), analyze the implementation of MI principles and techniques within ECAs, and examine the evaluation methods and primary outcomes of studies that use ECAs providing MI interventions.

METHODS

We conducted a scoping review following the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews) methodology. Our systematic search covered the PubMed, Scopus, IEEE Xplore, ACM Digital, and PsycINFO databases for papers published between January 2008 and December 2022. We included papers describing ECAs developed for delivering MI interventions targeting health-related behaviors and excluded articles that did not describe ECAs with human appearances and without the necessary evaluation or MI explanation. In a multistage process, 3 independent reviewers performed screening and data extraction, and the collected data were synthesized using a narrative approach.

RESULTS

The initial search identified 404 articles, of which 3.5% (n=14) were included in the review. ECAs primarily focused on reducing alcohol use (n=5, 36%), took on female representations (n=9, 64%), and gave limited consideration to user ethnicity (n=9, 64%). Most of them used rules-driven dialogue mechanisms (n=13, 93%), include emotional behavior to convey empathy (n=8, 57%) but without an automatic recognition of user emotions (n=12, 86%). Regarding MI implementation, of 14 studies, 3 (21%) covered all MI principles, 4 (29%) included all processes, and none covered all techniques. Most studies (8/14, 57%) conducted acceptability, usability, and user experience assessments, whereas a smaller proportion (4/14, 29%) used randomized controlled trials to evaluate behavior changes. Overall, the studies reported positive results regarding acceptability, usability, and user experience and showed promising outcomes in changes in attitudes, beliefs, motivation, and behavior.

CONCLUSIONS

This study revealed significant advancements in the use of ECAs for delivering MI interventions aimed at promoting healthier behaviors over the past 15 years. However, this review emphasizes the need for a more in-depth exploration of ECA characteristics. In addition, there is a need for the enhanced integration of MI principles, processes, and techniques into ECAs. Although acceptability and usability have received considerable attention, there is a compelling argument for placing a stronger emphasis on assessing changes in attitudes, beliefs, motivation, and behavior. Consequently, inclusion of more randomized controlled trials is essential for comprehensive intervention evaluations.

Factors Influencing the Acceptability, Acceptance, and Adoption of Conversational Agents in Health Care: Integrative Review

BACKGROUND

Conversational agents (CAs), also known as chatbots, are digital dialog systems that enable people to have a text-based, speech-based, or nonverbal conversation with a computer or another machine based on natural language via an interface. The use of CAs offers new opportunities and various benefits for health care. However, they are not yet ubiquitous in daily practice. Nevertheless, research regarding the implementation of CAs in health care has grown tremendously in recent years.

OBJECTIVE

This review aims to present a synthesis of the factors that facilitate or hinder the implementation of CAs from the perspectives of patients and health care professionals. Specifically, it focuses on the early implementation outcomes of acceptability, acceptance, and adoption as cornerstones of later implementation success.

METHODS

We performed an integrative review. To identify relevant literature, a broad literature search was conducted in June 2021 with no date limits and using all fields in PubMed, Cochrane Library, Web of Science, LIVIVO, and PsycINFO. To keep the review current, another search was conducted in March 2022. To identify as many eligible primary sources as possible, we used a snowballing approach by searching reference lists and conducted a hand search. Factors influencing the acceptability, acceptance, and adoption of CAs in health care were coded through parallel deductive and inductive approaches, which were informed by current technology acceptance and adoption models. Finally, the factors were synthesized in a thematic map.

RESULTS

Overall, 76 studies were included in this review. We identified influencing factors related to 4 core Unified Theory of Acceptance and Use of Technology (UTAUT) and Unified Theory of Acceptance and Use of Technology 2 (UTAUT2) factors (performance expectancy, effort expectancy, facilitating conditions, and hedonic motivation), with most studies underlining the relevance of performance and effort expectancy. To meet the particularities of the health care context, we redefined the UTAUT2 factors social influence, habit, and price value. We identified 6 other influencing factors: perceived risk, trust, anthropomorphism, health issue, working alliance, and user characteristics. Overall, we identified 10 factors influencing acceptability, acceptance, and adoption among health care professionals (performance expectancy, effort expectancy, facilitating conditions, social influence, price value, perceived risk, trust, anthropomorphism, working alliance, and user characteristics) and 13 factors influencing acceptability, acceptance, and adoption among patients (additionally hedonic motivation, habit, and health issue).

CONCLUSIONS

This review shows manifold factors influencing the acceptability, acceptance, and adoption of CAs in health care. Knowledge of these factors is fundamental for implementation planning. Therefore, the findings of this review can serve as a basis for future studies to develop appropriate implementation strategies. Furthermore, this review provides an empirical test of current technology acceptance and adoption models and identifies areas where additional research is necessary.

TRIAL REGISTRATION

PROSPERO CRD42022343690; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=343690.

A Computational Platform to Support the Detection, Follow-up, and Epidemiological Surveillance of Mental Health and Substance Use Disorders: Protocol for a Development and Evaluation Study

BACKGROUND

According to the World Health Organization, approximately 15% of the global population is affected by mental health or substance use disorders. These conditions contribute significantly to the global disease burden, which has worsened because of the direct and indirect effects of COVID-19. In Mexico, a quarter of the population between the ages of 18 and 65 years who reside in urban areas present a mental health condition. The presence of a mental or substance abuse disorder is behind a significant percentage of suicidal behaviors in Mexico, where only 1 in 5 of those who have these disorders receive any treatment.

OBJECTIVE

This study aims to develop, deploy, and evaluate a computational platform to support the early detection and intervention of mental and substance use disorders in secondary and high schools as well as primary care units. The platform also aims to facilitate monitoring, treatment, and epidemiological surveillance ultimately helping specialized health units at the secondary level of care.

METHODS

The development and evaluation of the proposed computational platform will run during 3 stages. In stage 1, the identification of the functional and user requirements and the implementation of the modules to support the screening, follow-up, treatment, and epidemiological surveillance will be performed. In stage 2, the initial deployment of the screening module will be carried out in a set of secondary and high schools, as well as the deployment of the modules to support the follow-up, treatment, and epidemiological surveillance processes in primary and secondary care health units. In parallel, during stage 2, patient applications to support early interventions and continuous monitoring will also be developed. Finally, during stage 3, the deployment of the complete platform will be performed jointly with a quantitative and qualitative evaluation.

RESULTS

The screening process has started, and 6 schools have been currently enrolled. As of February 2023, a total of 1501 students have undergone screening, and the referral of those students presenting a risk in mental health or substance use to primary care units has also started. The development, deployment, and evaluation of all the modules of the proposed platform are expected to be completed by late 2024.

CONCLUSIONS

The expected results of this study are to impact a better integration between the different levels of health care, from early detection to follow-up and epidemiological surveillance of mental and substance use disorders contributing to reducing the gap in the attention to these problems in the community.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/44607.

A survey on agents applications in healthcare: Opportunities, challenges and trends

BACKGROUND AND OBJECTIVE

The agent abstraction is a powerful one, developed decades ago to represent crucial aspects of artificial intelligence research. The meaning has transformed over the years and now there are different nuances across research communities. At its core, an agent is an autonomous computational entity capable of sensing, acting, and capturing interactions with other agents and its environment. This review examines how agent-based techniques have been implemented and evaluated in a specific and very important domain, i.e. healthcare research.

METHODS

We survey key areas of agent-based research in healthcare, e.g. individual and collective behaviours, communicable and non-communicable diseases, and social epidemiology. We propose a systematic search and critical review of relevant recent works, introduced by an exploratory network analysis.

RESULTS

Network analysis enables to devise out 5 main research clusters, the most active authors, and 4 main research topics.

CONCLUSIONS

Our findings support discussion of some future directions for increasing the value of agent-based approaches in healthcare.

Comparing the Feasibility and Acceptability of a Virtual Human, Teletherapy, and an e-Manual in Delivering a Stress Management Intervention to Distressed Adult Women: Pilot Study

BACKGROUND

Virtual humans (VHs), teletherapy, and self-guided e-manuals may increase the accessibility of psychological interventions. However, there is limited research on how these technologies compare in terms of their feasibility and acceptability in delivering stress management interventions.

OBJECTIVE

We conducted a preliminary comparison of the feasibility and acceptability of a VH, teletherapy, and an e-manual at delivering 1 module of cognitive behavioral stress management (CBSM) to evaluate the feasibility of the trial methodology in preparation for a future randomized controlled trial (RCT).

METHODS

A pilot RCT was conducted with a parallel, mixed design. A community sample of distressed adult women were randomly allocated to receive 1 session of CBSM involving training in cognitive and behavioral techniques by a VH, teletherapy, or an e-manual plus homework over 2 weeks. Data were collected on the feasibility of the intervention technologies (technical support and homework access), trial methods (recruitment methods, questionnaire completion, and methodological difficulty observations), intervention acceptability (intervention completion, self-report ratings, therapist rapport, and trust), and acceptability of the trial methods (self-report ratings and observations). Qualitative data in the form of written responses to open-ended questions were collected to enrich and clarify the findings on intervention acceptability.

RESULTS

Overall, 38 participants' data were analyzed. A VH (n=12), teletherapy (n=12), and an e-manual (n=14) were found to be feasible and acceptable for delivering 1 session of CBSM to distressed adult women based on the overall quantitative and qualitative findings. Technical difficulties were minimal and did not affect intervention completion, and no significant differences were found between the conditions (P=.31). The methodology was feasible, although improvements were identified for a future trial. All conditions achieved good satisfaction and perceived engagement ratings, and no significant group differences were found (P>.40). Participants had similar willingness to recommend each technology (P=.64). There was a nonsignificant trend toward participants feeling more open to using the VH and e-manual from home than teletherapy (P=.10). Rapport (P<.001) and trust (P=.048) were greater with the human teletherapist than with the VH. The qualitative findings enriched the quantitative results by revealing the unique strengths and limitations of each technology that may have influenced acceptability.

CONCLUSIONS

A VH, teletherapy, and a self-guided e-manual were found to be feasible and acceptable methods of delivering 1 session of a stress management intervention to a community sample of adult women. The technologies were found to have unique strengths and limitations that may affect which works best for whom and in what circumstances. Future research should test additional CBSM modules for delivery by these technologies and conduct a larger RCT to compare their feasibility, acceptability, and effectiveness when delivering a longer home-based stress management program.

TRIAL REGISTRATION

Australian New Zealand Clinical Trials Registry ACTRN12620000859987; https://www.anzctr.org.au/Trial/Registration/TrialReview.aspx?id=380114&isReview=true.

Developing, Implementing, and Evaluating an Artificial Intelligence-Guided Mental Health Resource Navigation Chatbot for Health Care Workers and Their Families During and Following the COVID-19 Pandemic: Protocol for a Cross-sectional Study

Background

Approximately 1 in 3 Canadians will experience an addiction or mental health challenge at some point in their lifetime. Unfortunately, there are multiple barriers to accessing mental health care, including system fragmentation, episodic care, long wait times, and insufficient support for health system navigation. In addition, stigma may further reduce an individual’s likelihood of seeking support. Digital technologies present new and exciting opportunities to bridge significant gaps in mental health care service provision, reduce barriers pertaining to stigma, and improve health outcomes for patients and mental health system integration and efficiency. Chatbots (ie, software systems that use artificial intelligence to carry out conversations with people) may be explored to support those in need of information or access to services and present the opportunity to address gaps in traditional, fragmented, or episodic mental health system structures on demand with personalized attention. The recent COVID-19 pandemic has exacerbated even further the need for mental health support among Canadians and called attention to the inefficiencies of our system. As health care workers and their families are at an even greater risk of mental illness and psychological distress during the COVID-19 pandemic, this technology will be first piloted with the goal of supporting this vulnerable group.

Objective

This pilot study seeks to evaluate the effectiveness of the Mental Health Intelligent Information Resource Assistant in supporting health care workers and their families in the Canadian provinces of Alberta and Nova Scotia with the provision of appropriate information on mental health issues, services, and programs based on personalized needs.

Methods

The effectiveness of the technology will be assessed via voluntary follow-up surveys and an analysis of client interactions and engagement with the chatbot. Client satisfaction with the chatbot will also be assessed.

Results

This project was initiated on April 1, 2021. Ethics approval was granted on August 12, 2021, by the University of Alberta Health Research Board (PRO00109148) and on April 21, 2022, by the Nova Scotia Health Authority Research Ethics Board (1027474). Data collection is anticipated to take place from May 2, 2022, to May 2, 2023. Publication of preliminary results will be sought in spring or summer 2022, with a more comprehensive evaluation completed by spring 2023 following the collection of a larger data set.

Conclusions

Our findings can be incorporated into public policy and planning around mental health system navigation by Canadian mental health care providers—from large public health authorities to small community-based, not-for-profit organizations. This may serve to support the development of an additional touch point, or point of entry, for individuals to access the appropriate services or care when they need them, wherever they are.

International Registered Report Identifier (IRRID)

PRR1-10.2196/33717

Chatbots to Support Young Adults’ Mental Health: an Exploratory Study of Acceptability

Despite the prevalence of mental health conditions, stigma, lack of awareness and limited resources impede access to care, creating a need to improve mental health support. The recent surge in scientific and commercial interest in conversational agents and their potential to improve diagnosis and treatment seems a potentially fruitful area in this respect, particularly for young adults who widely use such systems in other contexts. Yet, there is little research that considers the acceptability of conversational agents in mental health. This study, therefore, presents three research activities that explore whether conversational agents and, in particular, chatbots can be an acceptable solution in mental healthcare for young adults. First, a survey of young adults (in a university setting) provides an understanding of the landscape of mental health in this age group and of their views around mental health technology, including chatbots. Second, a literature review synthesises current evidence relating to the acceptability of mental health conversational agents and points to future research priorities. Third, interviews with counsellors who work with young adults, supported by a chatbot prototype and user-centred design techniques, reveal the perceived benefits and potential roles of mental health chatbots from the perspective of mental health professionals, while suggesting preconditions for the acceptability of the technology. Taken together, these research activities: provide evidence that chatbots are an acceptable solution to offering mental health support for young adults; identify specific challenges relating to both the technology and environment; and argue for the application of user-centred approaches during development of mental health chatbots and more systematic and rigorous evaluations of the resulting solutions.

Human Support in App-Based Cognitive Behavioral Therapies for Emotional Disorders: Scoping Review

BACKGROUND

Smartphone app-based therapies offer clear promise for reducing the gap in available mental health care for people at risk for or people with mental illness. To this end, as smartphone ownership has become widespread, app-based therapies have become increasingly common. However, the research on app-based therapies is lagging behind. In particular, although experts suggest that human support may be critical for increasing engagement and effectiveness, we have little systematic knowledge about the role that human support plays in app-based therapy. It is critical to address these open questions to optimally design and scale these interventions.

OBJECTIVE

The purpose of this study is to provide a scoping review of the use of human support or coaching in app-based cognitive behavioral therapy for emotional disorders, identify critical knowledge gaps, and offer recommendations for future research. Cognitive behavioral therapy is the most well-researched treatment for a wide range of concerns and is understood to be particularly well suited to digital implementations, given its structured, skill-based approach.

METHODS

We conducted systematic searches of 3 databases (PubMed, PsycINFO, and Embase). Broadly, eligible articles described a cognitive behavioral intervention delivered via smartphone app whose primary target was an emotional disorder or problem and included some level of human involvement or support (coaching). All records were reviewed by 2 authors. Information regarding the qualifications and training of coaches, stated purpose and content of the coaching, method and frequency of communication with users, and relationship between coaching and outcomes was recorded.

RESULTS

Of the 2940 titles returned by the searches, 64 (2.18%) were eligible for inclusion. This review found significant heterogeneity across all of the dimensions of coaching considered as well as considerable missing information in the published articles. Moreover, few studies had qualitatively or quantitatively evaluated how the level of coaching impacts treatment engagement or outcomes. Although users tend to self-report that coaching improves their engagement and outcomes, there is limited and mixed supporting quantitative evidence at present.

CONCLUSIONS

Digital mental health is a young but rapidly expanding field with great potential to improve the reach of evidence-based care. Researchers across the reviewed articles offered numerous approaches to encouraging and guiding users. However, with the relative infancy of these treatment approaches, this review found that the field has yet to develop standards or consensus for implementing coaching protocols, let alone those for measuring and reporting on the impact. We conclude that coaching remains a significant hole in the growing digital mental health literature and lay out recommendations for future data collection, reporting, experimentation, and analysis.

Implementation of Cognitive Behavioral Therapy in e-Mental Health Apps: Literature Review

Background

To address the matter of limited resources for treating individuals with mental disorders, e–mental health has gained interest in recent years. More specifically, mobile health (mHealth) apps have been suggested as electronic mental health interventions accompanied by cognitive behavioral therapy (CBT).

Objective

This study aims to identify the therapeutic aspects of CBT that have been implemented in existing mHealth apps and the technologies used. From these, we aim to derive research gaps that should be addressed in the future.

Methods

Three databases were screened for studies on mHealth apps in the context of mental disorders that implement techniques of CBT: PubMed, IEEE Xplore, and ACM Digital Library. The studies were independently selected by 2 reviewers, who then extracted data from the included studies. Data on CBT techniques and their technical implementation in mHealth apps were synthesized narratively.

Results

Of the 530 retrieved citations, 34 (6.4%) studies were included in this review. mHealth apps for CBT exploit two groups of technologies: technologies that implement CBT techniques for cognitive restructuring, behavioral activation, and problem solving (exposure is not yet realized in mHealth apps) and technologies that aim to increase user experience, adherence, and engagement. The synergy of these technologies enables patients to self-manage and self-monitor their mental state and access relevant information on their mental illness, which helps them cope with mental health problems and allows self-treatment.

Conclusions

There are CBT techniques that can be implemented in mHealth apps. Additional research is needed on the efficacy of the mHealth interventions and their side effects, including inequalities because of the digital divide, addictive internet behavior, lack of trust in mHealth, anonymity issues, risks and biases for user groups and social contexts, and ethical implications. Further research is also required to integrate and test psychological theories to improve the impact of mHealth and adherence to the e–mental health interventions.

Evaluation Metrics for Health Chatbots: A Delphi Study

BACKGROUND

 In recent years, an increasing number of health chatbots has been published in app stores and described in research literature. Given the sensitive data they are processing and the care settings for which they are developed, evaluation is essential to avoid harm to users. However, evaluations of those systems are reported inconsistently and without using a standardized set of evaluation metrics. Missing standards in health chatbot evaluation prevent comparisons of systems, and this may hamper acceptability since their reliability is unclear.

OBJECTIVES

 The objective of this paper is to make an important step toward developing a health-specific chatbot evaluation framework by finding consensus on relevant metrics.

METHODS

 We used an adapted Delphi study design to verify and select potential metrics that we retrieved initially from a scoping review. We invited researchers, health professionals, and health informaticians to score each metric for inclusion in the final evaluation framework, over three survey rounds. We distinguished metrics scored relevant with high, moderate, and low consensus. The initial set of metrics comprised 26 metrics (categorized as global metrics, metrics related to response generation, response understanding and aesthetics).

RESULTS

 Twenty-eight experts joined the first round and 22 (75%) persisted to the third round. Twenty-four metrics achieved high consensus and three metrics achieved moderate consensus. The core set for our framework comprises mainly global metrics (e.g., ease of use, security content accuracy), metrics related to response generation (e.g., appropriateness of responses), and related to response understanding. Metrics on aesthetics (font type and size, color) are less well agreed upon-only moderate or low consensus was achieved for those metrics.

CONCLUSION

 The results indicate that experts largely agree on metrics and that the consensus set is broad. This implies that health chatbot evaluation must be multifaceted to ensure acceptability.

Security, privacy, and healthcare-related conversational agents: a scoping review

Health chatbots interview patients and collect health data. This process makes demands on data security and data privacy. To identify how and to what extent security and privacy are considered in current health chatbots. We conducted a scoping review by searching three bibliographic databases (PubMed, ACM Digital Library, IEEExplore) for papers reporting on chatbots in healthcare. We extracted which, how, and where data is stored by health chatbots and identified which external services have access to the data. Out of 1026 retrieved papers, we included 70 studies in the qualitative synthesis. Most papers report on chatbots that collect and process personal health data, usually in the context of mental health coaching applications. The majority did not provide any information regarding security or privacy aspects. We were able to determine limitations in literature and identified concrete challenges, including data access and usage of (third-party) services, data storage, data security methods, use case peculiarities and data privacy, as well as legal requirements. Data privacy and security in health chatbots are still underresearched and related information is underrepresented in scientific literature. By addressing the five key challenges in future, the transfer of theoretical solutions into practice can be facilitated.

The growing field of digital psychiatry: current evidence and the future of apps, social media, chatbots, and virtual reality

As the COVID-19 pandemic has largely increased the utilization of telehealth, mobile mental health technologies - such as smartphone apps, vir-tual reality, chatbots, and social media - have also gained attention. These digital health technologies offer the potential of accessible and scalable interventions that can augment traditional care. In this paper, we provide a comprehensive update on the overall field of digital psychiatry, covering three areas. First, we outline the relevance of recent technological advances to mental health research and care, by detailing how smartphones, social media, artificial intelligence and virtual reality present new opportunities for "digital phenotyping" and remote intervention. Second, we review the current evidence for the use of these new technological approaches across different mental health contexts, covering their emerging efficacy in self-management of psychological well-being and early intervention, along with more nascent research supporting their use in clinical management of long-term psychiatric conditions - including major depression; anxiety, bipolar and psychotic disorders; and eating and substance use disorders - as well as in child and adolescent mental health care. Third, we discuss the most pressing challenges and opportunities towards real-world implementation, using the Integrated Promoting Action on Research Implementation in Health Services (i-PARIHS) framework to explain how the innovations themselves, the recipients of these innovations, and the context surrounding innovations all must be considered to facilitate their adoption and use in mental health care systems. We conclude that the new technological capabilities of smartphones, artificial intelligence, social media and virtual reality are already changing mental health care in unforeseen and exciting ways, each accompanied by an early but promising evidence base. We point out that further efforts towards strengthening implementation are needed, and detail the key issues at the patient, provider and policy levels which must now be addressed for digital health technologies to truly improve mental health research and treatment in the future.

Changes to the Psychiatric Chatbot Landscape: A Systematic Review of Conversational Agents in Serious Mental Illness: Changements du paysage psychiatrique des chatbots: une revue systématique des agents conversationnels dans la maladie mentale sérieuse

OBJECTIVE

The need for digital tools in mental health is clear, with insufficient access to mental health services. Conversational agents, also known as chatbots or voice assistants, are digital tools capable of holding natural language conversations. Since our last review in 2018, many new conversational agents and research have emerged, and we aimed to reassess the conversational agent landscape in this updated systematic review.

METHODS

A systematic literature search was conducted in January 2020 using the PubMed, Embase, PsychINFO, and Cochrane databases. Studies included were those that involved a conversational agent assessing serious mental illness: major depressive disorder, schizophrenia spectrum disorders, bipolar disorder, or anxiety disorder.

RESULTS

Of the 247 references identified from selected databases, 7 studies met inclusion criteria. Overall, there were generally positive experiences with conversational agents in regard to diagnostic quality, therapeutic efficacy, or acceptability. There continues to be, however, a lack of standard measures that allow ease of comparison of studies in this space. There were several populations that lacked representation such as the pediatric population and those with schizophrenia or bipolar disorder. While comparing 2018 to 2020 research offers useful insight into changes and growth, the high degree of heterogeneity between all studies in this space makes direct comparison challenging.

CONCLUSIONS

This review revealed few but generally positive outcomes regarding conversational agents' diagnostic quality, therapeutic efficacy, and acceptability, which may augment mental health care. Despite this increase in research activity, there continues to be a lack of standard measures for evaluating conversational agents as well as several neglected populations. We recommend that the standardization of conversational agent studies should include patient adherence and engagement, therapeutic efficacy, and clinician perspectives.

The emergence of digital mental health in low-income and middle-income countries: A review of recent advances and implications for the treatment and prevention of mental disorders

In low-income and middle-income countries (LMICs), emerging digital mental health interventions should be accompanied by regular and comprehensive assessment of available scientific evidence. This review aims to support efforts to monitor progress in digital mental health research, ensuring new evidence can guide researchers, clinicians, policymakers and program managers positioned to adopt and implement these digitally-enabled treatments. In accordance with PRISMA guidelines, an electronic database search from 2016 to 2020 yielded 37 digital intervention studies for detection, diagnosis, prevention, treatment, and/or management of a broad range of mental disorders in 13 LMICs. This date range was selected to update previous reviews. Most studies involved online interventions and many reported feasibility and acceptability, reflected by participant satisfaction or program adherence. About half the studies (N = 23) reported clinical benefits based on changes in mental health. For depression and mood disorders, some digital interventions showed improvements in depressive symptoms, quality of life, treatment adherence, and recovery. However, sample sizes were small and studies focused primarily on adults. Further limiting generalizability was the lack of consistency in clinical assessment and measurement tools between studies. No studies reported worsening symptoms, negative acceptability or dissatisfaction with digital interventions, suggesting possible publication bias. While digital interventions show promise, it remains difficult to conclude that digital interventions are effective from these studies, as it is prudent to exercise caution before drawing conclusions about clinical effectiveness. This review reflects continued growth in digital mental health research in LMICs and further highlights the need for rigorous evaluation of effectiveness and cost-effectiveness.

Technical Aspects of Developing Chatbots for Medical Applications: Scoping Review

Background

Chatbots are applications that can conduct natural language conversations with users. In the medical field, chatbots have been developed and used to serve different purposes. They provide patients with timely information that can be critical in some scenarios, such as access to mental health resources. Since the development of the first chatbot, ELIZA, in the late 1960s, much effort has followed to produce chatbots for various health purposes developed in different ways.

Objective

This study aimed to explore the technical aspects and development methodologies associated with chatbots used in the medical field to explain the best methods of development and support chatbot development researchers on their future work.

Methods

We searched for relevant articles in 8 literature databases (IEEE, ACM, Springer, ScienceDirect, Embase, MEDLINE, PsycINFO, and Google Scholar). We also performed forward and backward reference checking of the selected articles. Study selection was performed by one reviewer, and 50% of the selected studies were randomly checked by a second reviewer. A narrative approach was used for result synthesis. Chatbots were classified based on the different technical aspects of their development. The main chatbot components were identified in addition to the different techniques for implementing each module.

Results

The original search returned 2481 publications, of which we identified 45 studies that matched our inclusion and exclusion criteria. The most common language of communication between users and chatbots was English (n=23). We identified 4 main modules: text understanding module, dialog management module, database layer, and text generation module. The most common technique for developing text understanding and dialogue management is the pattern matching method (n=18 and n=25, respectively). The most common text generation is fixed output (n=36). Very few studies relied on generating original output. Most studies kept a medical knowledge base to be used by the chatbot for different purposes throughout the conversations. A few studies kept conversation scripts and collected user data and previous conversations.

Conclusions

Many chatbots have been developed for medical use, at an increasing rate. There is a recent, apparent shift in adopting machine learning–based approaches for developing chatbot systems. Further research can be conducted to link clinical outcomes to different chatbot development techniques and technical characteristics.

Scientific Publication Patterns of Mobile Technologies and Apps for Posttraumatic Stress Disorder Treatment: Bibliometric Co-Word Analysis

Background

Mobile apps are viewed as a promising opportunity to provide support for patients who have posttraumatic stress disorder (PTSD). The development of mobile technologies and apps shows similar trends in PTSD treatment. Therefore, this emerging research field has received substantial attention. Consequently, various research settings are planned for current and further studies.

Objective

The aim of this study was to explore the scientific patterns of research domains related to mobile apps and other technologies for PTSD treatment in scholarly publications, and to suggest further studies for this emerging research field.

Methods

We conducted a bibliometric analysis to identify publication patterns, most important keywords, trends for topicality, and text analysis, along with construction of a word cloud for papers published in the last decade (2010 to 2019). Research questions were formulated based on the relevant literature. In particular, we concentrated on highly ranked sources. Based on the proven bibliometric approach, the data were ultimately retrieved from the Web of Science Core Collection (Clarivate Analytics).

Results

A total of 64 studies were found concerning the research domains. The vast majority of the papers were written in the English language (63/64, 98%) with the remaining article (1/64, 2%) written in French. The articles were written by 323 authors/coauthors from 11 different countries, with the United States predominating, followed by England, Canada, Italy, the Netherlands, Australia, France, Germany, Mexico, Sweden, and Vietnam. The most common publication type was peer-reviewed journal articles (48/64, 75%), followed by reviews (8/64, 13%), meeting abstracts (5/64, 8%), news items (2/64, 3%), and a proceeding (1/64, 2%). There was a mean of 6.4 papers published per year over the study period. There was a 100% increase in the number of publications published from 2016 to 2019 with a mean of 13.33 papers published per year during this latter period.

Conclusions

Although the number of papers on mobile technologies for PTSD was quite low in the early period, there has been an overall increase in this research domain in recent years (2016-2019). Overall, these findings indicate that mobile health tools in combination with traditional treatment for mental disorders among veterans increase the efficiency of health interventions, including reducing PTSD symptoms, improving quality of life, conducting intervention evaluation, and monitoring of improvements. Mobile apps and technologies can be used as supportive tools in managing pain, anger, stress, and sleep disturbance. These findings therefore provide a useful overview of the publication trends on research domains that can inform further studies and highlight potential gaps in this field.

Artificial Intelligence-Based Conversational Agents for Chronic Conditions: Systematic Literature Review

BACKGROUND

A rising number of conversational agents or chatbots are equipped with artificial intelligence (AI) architecture. They are increasingly prevalent in health care applications such as those providing education and support to patients with chronic diseases, one of the leading causes of death in the 21st century. AI-based chatbots enable more effective and frequent interactions with such patients.

OBJECTIVE

The goal of this systematic literature review is to review the characteristics, health care conditions, and AI architectures of AI-based conversational agents designed specifically for chronic diseases.

METHODS

We conducted a systematic literature review using PubMed MEDLINE, EMBASE, PyscInfo, CINAHL, ACM Digital Library, ScienceDirect, and Web of Science. We applied a predefined search strategy using the terms "conversational agent," "healthcare," "artificial intelligence," and their synonyms. We updated the search results using Google alerts, and screened reference lists for other relevant articles. We included primary research studies that involved the prevention, treatment, or rehabilitation of chronic diseases, involved a conversational agent, and included any kind of AI architecture. Two independent reviewers conducted screening and data extraction, and Cohen kappa was used to measure interrater agreement.A narrative approach was applied for data synthesis.

RESULTS

The literature search found 2052 articles, out of which 10 papers met the inclusion criteria. The small number of identified studies together with the prevalence of quasi-experimental studies (n=7) and prevailing prototype nature of the chatbots (n=7) revealed the immaturity of the field. The reported chatbots addressed a broad variety of chronic diseases (n=6), showcasing a tendency to develop specialized conversational agents for individual chronic conditions. However, there lacks comparison of these chatbots within and between chronic diseases. In addition, the reported evaluation measures were not standardized, and the addressed health goals showed a large range. Together, these study characteristics complicated comparability and open room for future research. While natural language processing represented the most used AI technique (n=7) and the majority of conversational agents allowed for multimodal interaction (n=6), the identified studies demonstrated broad heterogeneity, lack of depth of reported AI techniques and systems, and inconsistent usage of taxonomy of the underlying AI software, further aggravating comparability and generalizability of study results.

CONCLUSIONS

The literature on AI-based conversational agents for chronic conditions is scarce and mostly consists of quasi-experimental studies with chatbots in prototype stage that use natural language processing and allow for multimodal user interaction. Future research could profit from evidence-based evaluation of the AI-based conversational agents and comparison thereof within and between different chronic health conditions. Besides increased comparability, the quality of chatbots developed for specific chronic conditions and their subsequent impact on the target patients could be enhanced by more structured development and standardized evaluation processes.

Perceived Usefulness, Satisfaction, Ease of Use and Potential of a Virtual Companion to Support the Care Provision for Older Adults

This article reports a study aiming to determine the perceptions of older adults needing formal care about the usefulness, satisfaction, and ease of use of CaMeLi, a virtual companion based on an embodied conversational agent, and the perceptions of formal caregivers about the potential of virtual companions to support care provision. An observational study involving older adults needing formal care was conducted to assess CaMeLi using a multi-method approach (i.e., an auto-reported questionnaire—the Usefulness, Satisfaction, and Ease of use questionnaire; a scale for the usability assessment based on the opinion of observers—the International Classification of Functioning Disability and Health-based Usability Scale; and critical incident registration). Moreover, a focus group was conducted to collect data regarding the perceived utility of virtual companions to support care provision. The observational study was conducted with 46 participants with an average age of 63.6 years, and the results were associated with a high level of usefulness, satisfaction, and ease of use of CaMeLi. Furthermore, the focus group composed of four care providers considered virtual companions a promising solution to support care provision and to prevent loneliness and social isolation. The results of both the observational study and the focus group revealed good perceptions regarding the role of virtual companions to support the care provision for older adults.

Acceptability of an Embodied Conversational Agent for Type 2 Diabetes Self-Management Education and Support via a Smartphone App: Mixed Methods Study

BACKGROUND

Embodied conversational agents (ECAs) are increasingly used in health care apps; however, their acceptability in type 2 diabetes (T2D) self-management apps has not yet been investigated.

OBJECTIVE

This study aimed to evaluate the acceptability of the ECA (Laura) used to deliver diabetes self-management education and support in the My Diabetes Coach (MDC) app.

METHODS

A sequential mixed methods design was applied. Adults with T2D allocated to the intervention arm of the MDC trial used the MDC app over a period of 12 months. At 6 months, they completed questions assessing their interaction with, and attitudes toward, the ECA. In-depth qualitative interviews were conducted with a subsample of the participants from the intervention arm to explore their experiences of using the ECA. The interview questions included the participants' perceptions of Laura, including their initial impression of her (and how this changed over time), her personality, and human character. The quantitative and qualitative data were interpreted using integrated synthesis.

RESULTS

Of the 93 intervention participants, 44 (47%) were women; the mean (SD) age of the participants was 55 (SD 10) years and the baseline glycated hemoglobin A1c level was 7.3% (SD 1.5%). Overall, 66 of the 93 participants (71%) provided survey responses. Of these, most described Laura as being helpful (57/66, 86%), friendly (57/66, 86%), competent (56/66, 85%), trustworthy (48/66, 73%), and likable (40/66, 61%). Some described Laura as not real (18/66, 27%), boring (26/66, 39%), and annoying (20/66, 30%). Participants reported that interacting with Laura made them feel more motivated (29/66, 44%), comfortable (24/66, 36%), confident (14/66, 21%), happy (11/66, 17%), and hopeful (8/66, 12%). Furthermore, 20% (13/66) of the participants were frustrated by their interaction with Laura, and 17% (11/66) of the participants reported that interacting with Laura made them feel guilty. A total of 4 themes emerged from the qualitative data (N=19): (1) perceived role: a friendly coach rather than a health professional; (2) perceived support: emotional and motivational support; (3) embodiment preference acceptability of a human-like character; and (4) room for improvement: need for greater congruence between Laura's words and actions.

CONCLUSIONS

These findings suggest that an ECA is an acceptable means to deliver T2D self-management education and support. A human-like character providing ongoing, friendly, nonjudgmental, emotional, and motivational support is well received. Nevertheless, the ECA can be improved by increasing congruence between its verbal and nonverbal communication and accommodating user preferences.

TRIAL REGISTRATION

Australian New Zealand Clinical Trials Registry CTRN12614001229662; https://tinyurl.com/yxshn6pd.

Technical Metrics Used to Evaluate Health Care Chatbots: Scoping Review

BACKGROUND

Dialog agents (chatbots) have a long history of application in health care, where they have been used for tasks such as supporting patient self-management and providing counseling. Their use is expected to grow with increasing demands on health systems and improving artificial intelligence (AI) capability. Approaches to the evaluation of health care chatbots, however, appear to be diverse and haphazard, resulting in a potential barrier to the advancement of the field.

OBJECTIVE

This study aims to identify the technical (nonclinical) metrics used by previous studies to evaluate health care chatbots.

METHODS

Studies were identified by searching 7 bibliographic databases (eg, MEDLINE and PsycINFO) in addition to conducting backward and forward reference list checking of the included studies and relevant reviews. The studies were independently selected by two reviewers who then extracted data from the included studies. Extracted data were synthesized narratively by grouping the identified metrics into categories based on the aspect of chatbots that the metrics evaluated.

RESULTS

Of the 1498 citations retrieved, 65 studies were included in this review. Chatbots were evaluated using 27 technical metrics, which were related to chatbots as a whole (eg, usability, classifier performance, speed), response generation (eg, comprehensibility, realism, repetitiveness), response understanding (eg, chatbot understanding as assessed by users, word error rate, concept error rate), and esthetics (eg, appearance of the virtual agent, background color, and content).

CONCLUSIONS

The technical metrics of health chatbot studies were diverse, with survey designs and global usability metrics dominating. The lack of standardization and paucity of objective measures make it difficult to compare the performance of health chatbots and could inhibit advancement of the field. We suggest that researchers more frequently include metrics computed from conversation logs. In addition, we recommend the development of a framework of technical metrics with recommendations for specific circumstances for their inclusion in chatbot studies.

Technical Aspects of Developing Chatbots for Medical Applications: Scoping Review (Preprint).. [DOI: 10.2196/preprints.19127] [Citation(s) in RCA: 0] [Impact Index Per Article: 0]

BACKGROUND

Chatbots are applications that can conduct natural language conversations with users. In the medical field, chatbots have been developed and used to serve different purposes. They provide patients with timely information that can be critical in some scenarios, such as access to mental health resources. Since the development of the first chatbot, ELIZA, in the late 1960s, much effort has followed to produce chatbots for various health purposes developed in different ways.

OBJECTIVE

This study aimed to explore the technical aspects and development methodologies associated with chatbots used in the medical field to explain the best methods of development and support chatbot development researchers on their future work.

METHODS

We searched for relevant articles in 8 literature databases (IEEE, ACM, Springer, ScienceDirect, Embase, MEDLINE, PsycINFO, and Google Scholar). We also performed forward and backward reference checking of the selected articles. Study selection was performed by one reviewer, and 50% of the selected studies were randomly checked by a second reviewer. A narrative approach was used for result synthesis. Chatbots were classified based on the different technical aspects of their development. The main chatbot components were identified in addition to the different techniques for implementing each module.

RESULTS

The original search returned 2481 publications, of which we identified 45 studies that matched our inclusion and exclusion criteria. The most common language of communication between users and chatbots was English (n=23). We identified 4 main modules: text understanding module, dialog management module, database layer, and text generation module. The most common technique for developing text understanding and dialogue management is the pattern matching method (n=18 and n=25, respectively). The most common text generation is fixed output (n=36). Very few studies relied on generating original output. Most studies kept a medical knowledge base to be used by the chatbot for different purposes throughout the conversations. A few studies kept conversation scripts and collected user data and previous conversations.

CONCLUSIONS

Many chatbots have been developed for medical use, at an increasing rate. There is a recent, apparent shift in adopting machine learning–based approaches for developing chatbot systems. Further research can be conducted to link clinical outcomes to different chatbot development techniques and technical characteristics.

Technical Metrics Used to Evaluate Health Care Chatbots: Scoping Review (Preprint).. [DOI: 10.2196/preprints.18301] [Citation(s) in RCA: 0] [Impact Index Per Article: 0]

BACKGROUND

Dialog agents (chatbots) have a long history of application in health care, where they have been used for tasks such as supporting patient self-management and providing counseling. Their use is expected to grow with increasing demands on health systems and improving artificial intelligence (AI) capability. Approaches to the evaluation of health care chatbots, however, appear to be diverse and haphazard, resulting in a potential barrier to the advancement of the field.

OBJECTIVE

This study aims to identify the technical (nonclinical) metrics used by previous studies to evaluate health care chatbots.

METHODS

Studies were identified by searching 7 bibliographic databases (eg, MEDLINE and PsycINFO) in addition to conducting backward and forward reference list checking of the included studies and relevant reviews. The studies were independently selected by two reviewers who then extracted data from the included studies. Extracted data were synthesized narratively by grouping the identified metrics into categories based on the aspect of chatbots that the metrics evaluated.

RESULTS

Of the 1498 citations retrieved, 65 studies were included in this review. Chatbots were evaluated using 27 technical metrics, which were related to chatbots as a whole (eg, usability, classifier performance, speed), response generation (eg, comprehensibility, realism, repetitiveness), response understanding (eg, chatbot understanding as assessed by users, word error rate, concept error rate), and esthetics (eg, appearance of the virtual agent, background color, and content).

CONCLUSIONS

The technical metrics of health chatbot studies were diverse, with survey designs and global usability metrics dominating. The lack of standardization and paucity of objective measures make it difficult to compare the performance of health chatbots and could inhibit advancement of the field. We suggest that researchers more frequently include metrics computed from conversation logs. In addition, we recommend the development of a framework of technical metrics with recommendations for specific circumstances for their inclusion in chatbot studies.

[Effect of antibiotics on D-serotonin-reactive structures]

Experiments on the isolated organs showed that ampicillin and levomycetin have pronounced D-antiserotoninergic effects; antagonism of antibodies and serotonin was found to be of competitive type. At an increase in levomycetin dosage D-antiserotoninergic effect was followed by the spasmolytic effect. Kefzol and benzylpenicillin failed to show any D-antiserotonin-ergic properties.