Automatic generation of spoken dialogue from medical plans and ontologies

Dialogue agents for artificial intelligence-based conversational systems for cognitively disabled: a systematic review

PURPOSE

We present a systematic literature review of dialogue agents for Artificial Intelligence (AI) and agent-based conversational systems dealing with cognitive disability of aged and impaired people including dementia and Parkinson's disease. We analyze current applications, gaps, and challenges in the existing research body, and provide guidelines and recommendations for their future development and use.

MATERIALS AND METHODS

We perform this study by applying Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) criteria. We performed a systematic search using relevant databases (ACM Digital Library, Google Scholar, IEEE Xplore, PubMed, and Scopus).

RESULTS

This study identified 468 articles on the use of conversational agents in healthcare. We finally selected 124 articles based on their objectives and content as directly related to our main topic.

CONCLUSION

We identified the main challenges in the field and analyzed the typical examples of the application of conversational agents in the healthcare domain, the desired characteristics of conversational agents, and chatbot support for aged people and people with cognitive disabilities. Our results contribute to a discussion on conversational health agents and emphasize current knowledge gaps and challenges for future research.IMPLICATIONS FOR REHABILITATIONA systematic literature review of dialogue agents for artificial intelligence and agent-based conversational systems dealing with cognitive disability of aged and impaired people.Main challenges and desired characteristics of the conversational agents, and chatbot support for aged people and people with cognitive disability.Current knowledge gaps and challenges for remote healthcare and rehabilitation.Guidelines and recommendations for future development and use of conversational systems.

Theory of radiologist interaction with instant messaging decision support tools: A sequential-explanatory study

Radiology specific clinical decision support systems (CDSS) and artificial intelligence are poorly integrated into the radiologist workflow. Current research and development efforts of radiology CDSS focus on 4 main interventions, based around exam centric time points-after image acquisition, intra-report support, post-report analysis, and radiology workflow adjacent. We review the literature surrounding CDSS tools in these time points, requirements for CDSS workflow augmentation, and technologies that support clinician to computer workflow augmentation. We develop a theory of radiologist-decision tool interaction using a sequential explanatory study design. The study consists of 2 phases, the first a quantitative survey and the second a qualitative interview study. The phase 1 survey identifies differences between average users and radiologist users in software interventions using the User Acceptance of Information Technology: Toward a Unified View (UTAUT) framework. Phase 2 semi-structured interviews provide narratives on why these differences are found. To build this theory, we propose a novel solution called Radibot-a conversational agent capable of engaging clinicians with CDSS as an assistant using existing instant messaging systems supporting hospital communications. This work contributes an understanding of how radiologist-users differ from the average user and can be utilized by software developers to increase satisfaction of CDSS tools within radiology.

Framework for Guiding the Development of High-Quality Conversational Agents in Healthcare

Evaluating conversational agents (CAs) that are supposed to be applied in healthcare settings and ensuring their quality is essential to avoid patient harm and ensure efficacy of the CA-delivered intervention. However, a guideline for a standardized quality assessment of health CAs is still missing. The objective of this work is to describe a framework that provides guidance for development and evaluation of health CAs. In previous work, consensus on categories for evaluating health CAs has been found. In this work, we identify concrete metrics, heuristics, and checklists for these evaluation categories to form a framework. We focus on a specific type of health CA, namely rule-based systems that are based on written input and output, have a simple personality without any kind of embodiment. First, we identified relevant metrics, heuristics, and checklists to be linked to the evaluation categories through a literature search. Second, five experts judged the metrics regarding their relevance to be considered within evaluation and development of health CAs. The final framework considers nine aspects from a general perspective, five aspects from a response understanding perspective, one aspect from a response generation perspective, and three aspects from an aesthetics perspective. Existing tools and heuristics specifically designed for evaluating CAs were linked to these evaluation aspects (e.g., Bot usability scale, design heuristics for CAs); tools related to mHealth evaluation were adapted when necessary (e.g., aspects from the ISO technical specification for mHealth Apps). The resulting framework comprises aspects to be considered not only as part of a system evaluation, but already during the development. In particular, aspects related to accessibility or security have to be addressed in the design phase (e.g., which input and output options are provided to ensure accessibility?) and have to be verified after the implementation phase. As a next step, transfer of the framework to other types of health CAs has to be studied. The framework has to be validated by applying it during health CA design and development.

Effect of Speech Recognition on Problem Solving and Recall in Consumer Digital Health Tasks: Controlled Laboratory Experiment

Background

Recent advances in natural language processing and artificial intelligence have led to widespread adoption of speech recognition technologies. In consumer health applications, speech recognition is usually applied to support interactions with conversational agents for data collection, decision support, and patient monitoring. However, little is known about the use of speech recognition in consumer health applications and few studies have evaluated the efficacy of conversational agents in the hands of consumers. In other consumer-facing tools, cognitive load has been observed to be an important factor affecting the use of speech recognition technologies in tasks involving problem solving and recall. Users find it more difficult to think and speak at the same time when compared to typing, pointing, and clicking. However, the effects of speech recognition on cognitive load when performing health tasks has not yet been explored.

Objective

The aim of this study was to evaluate the use of speech recognition for documentation in consumer digital health tasks involving problem solving and recall.

Methods

Fifty university staff and students were recruited to undertake four documentation tasks with a simulated conversational agent in a computer laboratory. The tasks varied in complexity determined by the amount of problem solving and recall required (simple and complex) and the input modality (speech recognition vs keyboard and mouse). Cognitive load, task completion time, error rate, and usability were measured.

Results

Compared to using a keyboard and mouse, speech recognition significantly increased the cognitive load for complex tasks (Z=–4.08, P<.001) and simple tasks (Z=–2.24, P=.03). Complex tasks took significantly longer to complete (Z=–2.52, P=.01) and speech recognition was found to be overall less usable than a keyboard and mouse (Z=–3.30, P=.001). However, there was no effect on errors.

Conclusions

Use of a keyboard and mouse was preferable to speech recognition for complex tasks involving problem solving and recall. Further studies using a broader variety of consumer digital health tasks of varying complexity are needed to investigate the contexts in which use of speech recognition is most appropriate. The effects of cognitive load on task performance and its significance also need to be investigated.

Medical Instructed Real-Time Assistant for Patient with Glaucoma and Diabetic Conditions

Virtual assistants are involved in the daily activities of humans such as managing calendars, making appointments, and providing wake-up calls. They provide a conversational service to customers around-the-clock and make their daily life manageable. With this emerging trend, many well-known companies launched their own virtual assistants that manage the daily routine activities of customers. In the healthcare sector, virtual medical assistants also provide a list of relevant diseases linked to a specific symptom. Due to low accuracy and uncertainty, these generated recommendations are untrusted and may lead to hypochondriasis. In this study, we proposed a Medical Instructed Real-time Assistant (MIRA) that listens to the user’s chief complaint and predicts a specific disease. Instead of informing about the medical condition, the user is referred to a nearby appropriate medical specialist. We designed an architecture for MIRA that considers the limitations of existing virtual medical assistants such as weak authentication, lack of understanding multiple intent statements about a specific medical condition, and uncertain diagnosis recommendations. To implement the designed architecture, we collected the chief complaints along with the dialogue corpora of real patients. Then, we manually validated these data under the supervision of medical specialists. We then used these data for natural language understanding, disease identification, and appropriate response generation. For the prototype version of MIRA, we considered the cases of glaucoma (eye disease) and diabetes (an autoimmune disease) only. The performance measure of MIRA was evaluated in terms of accuracy (89%), precision (90%), sensitivity (89.8%), specificity (94.9%), and F-measure (89.8%). The task completion was calculated using Cohen’s Kappa ( k = 0.848 ) that categorizes MIRA as ‘Almost Perfect’. Furthermore, the voice-based authentication identifies the user effectively and prevent against masquerading attack. Simultaneously, the user experience shows relatively good results in all aspects based on the User Experience Questionnaire (UEQ) benchmark data. The experimental results show that MIRA efficiently predicts a disease based on chief complaints and supports the user in decision making.

Conceiving an application ontology to model patient human papillomavirus vaccine counseling for dialogue management

Background

In the United States and parts of the world, the human papillomavirus vaccine uptake is below the prescribed coverage rate for the population. Some research have noted that dialogue that communicates the risks and benefits, as well as patient concerns, can improve the uptake levels. In this paper, we introduce an application ontology for health information dialogue called Patient Health Information Dialogue Ontology for patient-level human papillomavirus vaccine counseling and potentially for any health-related counseling.

Results

The ontology’s class level hierarchy is segmented into 4 basic levels - Discussion, Goal, Utterance, and Speech Task. The ontology also defines core low-level utterance interaction for communicating human papillomavirus health information. We discuss the design of the ontology and the execution of the utterance interaction.

Conclusion

With an ontology that represents patient-centric dialogue to communicate health information, we have an application-driven model that formalizes the structure for the communication of health information, and a reusable scaffold that can be integrated for software agents. Our next step will to be develop the software engine that will utilize the ontology and automate the dialogue interaction of a software agent.

Conversational agents in healthcare: a systematic review

Objective

Our objective was to review the characteristics, current applications, and evaluation measures of conversational agents with unconstrained natural language input capabilities used for health-related purposes.

Methods

We searched PubMed, Embase, CINAHL, PsycInfo, and ACM Digital using a predefined search strategy. Studies were included if they focused on consumers or healthcare professionals; involved a conversational agent using any unconstrained natural language input; and reported evaluation measures resulting from user interaction with the system. Studies were screened by independent reviewers and Cohen’s kappa measured inter-coder agreement.

Results

The database search retrieved 1513 citations; 17 articles (14 different conversational agents) met the inclusion criteria. Dialogue management strategies were mostly finite-state and frame-based (6 and 7 conversational agents, respectively); agent-based strategies were present in one type of system. Two studies were randomized controlled trials (RCTs), 1 was cross-sectional, and the remaining were quasi-experimental. Half of the conversational agents supported consumers with health tasks such as self-care. The only RCT evaluating the efficacy of a conversational agent found a significant effect in reducing depression symptoms (effect size d = 0.44, p = .04). Patient safety was rarely evaluated in the included studies.

Conclusions

The use of conversational agents with unconstrained natural language input capabilities for health-related purposes is an emerging field of research, where the few published studies were mainly quasi-experimental, and rarely evaluated efficacy or safety. Future studies would benefit from more robust experimental designs and standardized reporting.

Protocol Registration

The protocol for this systematic review is registered at PROSPERO with the number CRD42017065917.

Conversational Interfaces for Health: Bibliometric Analysis of Grants, Publications, and Patents

BACKGROUND

Conversational interfaces (CIs) in different modalities have been developed for health purposes, such as health behavioral intervention, patient self-management, and clinical decision support. Despite growing research evidence supporting CIs' potential, CI-related research is still in its infancy. There is a lack of systematic investigation that goes beyond publication review and presents the state of the art from perspectives of funding agencies, academia, and industry by incorporating CI-related public funding and patent activities.

OBJECTIVE

This study aimed to use data systematically extracted from multiple sources (ie, grant, publication, and patent databases) to investigate the development, research, and fund application of health-related CIs and associated stakeholders (ie, countries, organizations, and collaborators).

METHODS

A multifaceted search query was executed to retrieve records from 9 databases. Bibliometric analysis, social network analysis, and term co-occurrence analysis were conducted on the screened records.

RESULTS

This review included 42 funded projects, 428 research publications, and 162 patents. The total dollar amount of grants awarded was US $30,297,932, of which US $13,513,473 was awarded by US funding agencies and US $16,784,459 was funded by the Europe Commission. The top 3 funding agencies in the United States were the National Science Foundation, National Institutes of Health, and Agency for Healthcare Research and Quality. Boston Medical Center was awarded the largest combined grant size (US $2,246,437) for 4 projects. The authors of the publications were from 58 countries and 566 organizations; the top 3 most productive organizations were Northeastern University (United States), Universiti Teknologi MARA (Malaysia), and the French National Center for Scientific Research (CNRS; France). US researchers produced 114 publications. Although 82.0% (464/566) of the organizations engaged in interorganizational collaboration, 2 organizational research-collaboration clusters were observed with Northeastern University and CNRS as the central nodes. About 112 organizations from the United States and China filed 87.7% patents. IBM filed most patents (N=17). Only 5 patents were co-owned by different organizations, and there was no across-country collaboration on patenting activity. The terms patient, child, elderly, and robot were frequently discussed in the 3 record types. The terms related to mental and chronic issues were discussed mainly in grants and publications. The terms regarding multimodal interactions were widely mentioned as users' communication modes with CIs in the identified records.

CONCLUSIONS

Our findings provided an overview of the countries, organizations, and topic terms in funded projects, as well as the authorship, collaboration, content, and related information of research publications and patents. There is a lack of broad cross-sector partnerships among grant agencies, academia, and industry, particularly in the United States. Our results suggest a need to improve collaboration among public and private sectors and health care organizations in research and patent activities.

The implementation of a haptic feedback virtual reality simulation clinic with dynamic patient interaction and communication for medical imaging students

INTRODUCTION

An immersive virtual reality (VR) simulation clinic with dynamic patient interaction and communication was developed to facilitate the training of medical radiation science students. The software "CETSOL VR Clinic" was integrated into the Medical Imaging programme at Monash University in 2016 in order to benchmark student experiences against existing simulation techniques (Shaderware™).

METHODS

An iterative approach to development, based on two cycles of user feedback, was used to develop and refine the simulated clinical environment. This environment uses realistic 3D models, embedded clinical scenarios, dynamic communication, 3D hand gesture interaction, gaze and positional stereoscopic tracking and online user capabilities using the Unity™ game and physics engines. Students' perceptions of educational enhancement of their positioning skills following the use of the simulation tools were analysed via a 5-point Likert scale questionnaire.

RESULTS

Student perception scores indicated a significant difference between simulation modalities in favour of the immersive CETSOL VR Clinic, χ2 (4, N = 92) = 9.5, P-value <0.001.

CONCLUSION

Student perception scores on improvement of their clinical and technical skills were higher for the hand-positioning tasks performed with the CETSOL VR Clinic™ than with the comparative benchmark simulation that did not provide dynamic patient interaction and communication.

Ontology based intercultural patient practitioner assistive communications from qualitative gap analysis

Purpose

– There is an increasing interest in using information and communication technologies to support health services. But the adoption and development of even basic ICT communications services in many health services is limited, leaving enormous gaps in the broad understanding of its role in health care delivery. The purpose of this paper is to address a specific (intercultural) area of healthcare communications consumer disadvantage; and it examines the potential for ICT exploitation through the lens of a conceptual framework. The opportunity to pursue a new solutions pathway has been amplified in recent times through the development of computer-based ontologies and the resultant knowledge from ontologist activity and consequential research publishing.

Design/methodology/approach

– A specific intercultural area of patient disadvantage arises from variations in meaning and understanding of patient and clinician words, phrases and non-verbal expression. Collection and localization of data concepts, their attributes and individual instances were gathered from an Aboriginal trainee nurse focus group and from a qualitative gap analysis (QGA) of 130 criteria-selected sources of literature. These concepts, their relationships and semantic interpretations populate the computer ontology. The ontology mapping involves two domains, namely, Aboriginal English (AE) and Type II diabetes care guidelines. This is preparatory to development of the Patient Practitioner Assistive Communications (PPAC) system for Aboriginal rural and remote patient primary care.

Findings

– The combined QGA and focus group output reported has served to illustrate the call for three important drivers of change. First, there is no evidence to contradict the hypothesis that patient-practitioner interview encounters for many Australian Aboriginal patients and wellbeing outcomes are unsatisfactory at best. Second, there is a potent need for cultural competence knowledge and practice uptake on the part of health care providers; and third, the key contributory component to determine success or failures within healthcare for ethnic minorities is communication. Communication, however, can only be of value in health care if in practice it supports shared cognition; and mutual cognition is rarely achievable when biopsychosocial and other cultural worldview differences go unchallenged.

Research limitations/implications

– There has been no direct engagement with remote Aboriginal communities in this work to date. The authors have initially been able to rely upon a cohort of both Indigenous and non-Indigenous people with relevant cultural expertise and extended family relationships. Among these advisers are health care practitioners, academics, trainers, Aboriginal education researchers and workshop attendees. It must therefore be acknowledged that as is the case with the QGA, the majority of the concept data is from third parties. The authors have also discovered that urban influences and cultural sensitivities tend to reduce the extent of, and opportunity to, witness AE usage, thereby limiting the ability to capture more examples of code-switching. Although the PPAC system concept is qualitatively well developed, pending future work planned for rural and remote community engagement the authors presently regard the work as mostly allied to a hypothesis on ontology-driven communications. The concept data population of the AE home talk/health talk ontology has not yet reached a quantitative critical mass to justify application design model engineering and real-world testing.

Originality/value

– Computer ontologies avail us of the opportunity to use assistive communications technology applications as a dynamic support system to elevate the pragmatic experience of health care consultations for both patients and practitioners. The human-machine interactive development and use of such applications is required just to keep pace with increasing demand for healthcare and the growing health knowledge transfer environment. In an age when the worldwide web, communications devices and social media avail us of opportunities to confront the barriers described the authors have begun the first construction of a merged schema for two domains that already have a seemingly intractable negative connection. Through the ontology discipline of building syntactically and semantically robust and accessible concepts; explicit conceptual relationships; and annotative context-oriented guidance; the authors are working towards addressing health literacy and wellbeing outcome deficiencies of benefit to the broader communities of disadvantage patients.

Developing VISO: Vaccine Information Statement Ontology for patient education

Objective

To construct a comprehensive vaccine information ontology that can support personal health information applications using patient-consumer lexicon, and lead to outcomes that can improve patient education.

Methods

The authors composed the Vaccine Information Statement Ontology (VISO) using the web ontology language (OWL). We started with 6 Vaccine Information Statement (VIS) documents collected from the Centers for Disease Control and Prevention (CDC) website. Important and relevant selections from the documents were recorded, and knowledge triples were derived. Based on the collection of knowledge triples, the meta-level formalization of the vaccine information domain was developed. Relevant instances and their relationships were created to represent vaccine domain knowledge

Results

The initial iteration of the VISO was realized, based on the 6 Vaccine Information Statements and coded into OWL2 with Protégé. The ontology consisted of 132 concepts (classes and subclasses) with 33 types of relationships between the concepts. The total number of instances from classes totaled at 460, along with 429 knowledge triples in total. Semiotic-based metric scoring was applied to evaluate quality of the ontology.

Adapting research-tested computerized tailored interventions for broader dissemination and implementation

This paper focuses on the process for adapting existing legacy computerized tailored intervention (CTI) programs and implications for future development of CTI to ensure that interventions can be disseminated and implemented in different settings. A significant amount of work is required to adapt existing CTI for new research applications and public health interventions. Most new CTI are still developed from scratch, with minimal re-use of software or message content, even when there are considerable overlaps in functionality. This is largely a function of the substantial technical, organizational, and content-based barriers to adapting and disseminating CTI. CTI developers should thus consider dissemination and re-use early in the design phase of their systems. This is not intended to be a step-by-step guide on how to adopt or disseminate research-tested CTI, but rather a discussion that highlights issues to be considered for adapting and disseminating evidence-based CTI.

A reusable framework for health counseling dialogue systems based on a behavioral medicine ontology

Automated approaches to promoting health behavior change, such as exercise, diet, and medication adherence promotion, have the potential for significant positive impact on society. We describe a theory-driven computational model of dialogue that simulates a human health counselor who is helping his or her clients to change via a series of conversations over time. Applications built using this model can be used to change the health behavior of patients and consumers at low cost over a wide range of media including the web and the phone. The model is implemented using an OWL ontology of health behavior change concepts and a public standard task modeling language (ANSI/CEA-2018). We demonstrate the power of modeling dialogue using an ontology and task model by showing how an exercise promotion system developed in the framework was re-purposed for diet promotion with 98% reuse of the abstract models. Evaluations of these two systems are presented, demonstrating high levels of fidelity to best practices in health behavior change counseling.