Communication training for pharmacy students with standard patients using artificial intelligence

Evaluation framework for conversational AI agents in pharmacy education: A scoping review of key characteristics and outcome measures

BACKGROUND

Innovative and scalable training solutions, such as conversational artificial intelligence agents (CAIAs), are essential to meet the complex demands of pharmacy education and practice.

OBJECTIVES

To explore and synthesize the key characteristics and outcome measures of CAIAs in pharmacy education and propose an evaluation framework for their use in pharmacy education.

METHODS

A scoping review identified studies published between 2020 and 2025. An evaluation framework was developed to capture key CAIA characteristics and outcome measures using the World Health Organization's digital health framework.

RESULTS

Of 961 studies screened, six met inclusion criteria. Five studies originated from English-speaking countries. CAIAs supported training in communication skills (n = 3), human resource management (n = 2), and HIV care (n = 1). Learner interaction with CAIAs was variable and often poorly described. Common features included scenario-based learning, immediate, real-time, automated feedback, interactive-learning, and modalities. Frequently evaluated outcomes were functionality (n = 5), user experience (n = 4), cost-benefit (n = 3), and user characteristics (n = 3). Educational outcome measures of confidence, knowledge and skills were included in three studies. All studies used text-based interaction; two included audiovisual elements, one combined text and voice, and two relied solely on text. Most (n = 5) involved single-user formats. Three studies were in evaluation stage one (feasibility/usability), two were in stage two (effectiveness) and one in stage three (efficacy). Outcomes demonstrated a low uptake for CAIAs, but indicate increased learner confidence, knowledge and communication skills. Eleven educational features were added to our evaluation framework and three educational outcome categories.

CONCLUSION

This review contributed to the development of a framework to guide the design and evaluation of CAIAs in pharmacy education. CAIAs have been introduced in pharmacy education but remain in the early stages of adoption. Further research is needed to validate their effectiveness and expand their use in pharmacy as well test the use of the framework in other healthcare disciplines.

Virtual Simulation Tools for Communication Skills Training in Health Care Professionals: Literature Review

Background

Quality clinical care is supported by effective patient-centered communication. Health care professionals can improve their communication skills through simulation-based training, but our knowledge about virtual simulation and its effectiveness and use in training health professionals and students is still growing rapidly.

Objective

The objective of this study was to review the current academic literature to identify and evaluate the virtual simulation tools used to train communication skills in health care students and professionals.

Methods

This review was carried out in June 2023 by collecting data from the MEDLINE/PubMed and Web of Science electronic databases. Once applicable studies were identified, we recorded data related to type of technology used, learning objectives, degree of learning autonomy, outcomes, and other details.

Results

We found 35 articles that had developed and/or applied a virtual environment for training communication skills aimed at patients, in which 24 different learning tools were identified. Most had been developed to independently train communication skills in English, either generally or in the specific context of medical history (anamnesis) interviews. Many of these tools used a virtual patient that looked like a person and had the ability to vocally respond. Almost half of the tools analyzed allowed the person being trained to respond orally using natural language. Of note, not all these studies described the technology they had used in detail.

Conclusions

Many different learning tools with very heterogeneous characteristics are being used for the purposes of communication skills training. Continued research will still be required to develop virtual tools that include the most advanced features to achieve high-fidelity simulation training.

Applications of Artificial Intelligence for Nonpsychomotor Skills Training in Health Professions Education: A Scoping Review

PURPOSE

This study explores uses of artificial intelligence (AI) in health professions education for nonpsychomotor skills training at undergraduate, postgraduate, and continuing health professions education levels for education program development, delivery, and evaluation.

METHOD

This scoping review was conducted in 5 stages: (1) planning and research, (2) search strategy, (3) screening and selection, (4) review and recording data, and (5) synthesis. Seven bibliographic databases were searched using terms for artificial intelligence and continuing health professional education to capture articles that used AI for the purposes of nonpsychomotor skills training for health professions education and involved health care professionals and/or trainees. Databases were searched for articles published from January 1, 2001, to March 26, 2024. The original searches were performed on July 26, 2021, and again on March 26, 2024. Two reviewers independently screened, reviewed, and extracted data. Data extraction was performed using Kern's 6-step curriculum development framework to guide analysis.

RESULTS

In total, 9,914 studies related to AI in health professions education for nonpsychomotor skills training were screened. Of these, 103 studies were identified that met the inclusion criteria. Of these 103 studies, 52 (50%) were cohort studies. The most common learner population was health care professional students (67 studies [65%]). Most studies (76 [74%]) were set in nonclinical settings. Sixty-eight studies (66%) fit under step 6 of Kern's criteria (evaluation and assessment), illustrating that AI is predominantly being used for the purposes of evaluation and assessment of learners and programs.

CONCLUSIONS

Most studies in the literature illustrate that AI is being applied in a nonpsychomotor context to evaluate health professional education programs and assess learners. Additional opportunities to use AI in curriculum design and implementation could include identification of learning needs for training, personalizing learning with AI principles, and evaluating health care professional education programs.

Keeping pace in the age of innovation: The perspective of Dutch pharmaceutical science students on the position of machine learning training in an undergraduate curriculum

BACKGROUND

Over the years, approaches of the pharmaceutical industry to discover and develop drugs have changed rapidly due to new scientific trends. Among others, they have started to explore Machine Learning (ML), a subset of Artificial Intelligence (AI), as a promising tool to generate new hypotheses regarding drug candidate selections for clinical trials and to predict adverse side effects. Despite these recent developments, the possibilities of ML in pharmaceutical sciences have so far hardly penetrated the training of pharmaceutical science students. 1, 2 Therefore, as part of an elective course, an introductory module on ML was developed at Utrecht University, Department of Pharmaceutical Sciences.

OBJECTIVE

The aim of this study was to assess student' views on the module set-up, and their perspectives on ML within pharmaceutical science curricula.

METHODS

Semi-structured interviews over three years were conducted with 15 students participating in the module.

RESULTS

The students valued the well-designed and effective delivered module. They were personally motivated to learn more about ML in a future master or research internship. The students now perceive a lack of possibilities for ML training in pharmaceutical sciences education and indicate the value of incorporating ML opportunities for their future career.

CONCLUSION

Integrating ML training into pharmaceutical sciences curricula is needed to keep future drug researchers up to date with drug research advancements, enhancing their skills, academic development, and career prospects.

Generative artificial intelligence (Gen-AI) in pharmacy education: Utilization and implications for academic integrity: A scoping review

Introduction

Generative artificial intelligence (Gen-AI), exemplified by the widely adopted ChatGPT, has garnered significant attention in recent years. Its application spans various health education domains, including pharmacy, where its potential benefits and drawbacks have become increasingly apparent. Despite the growing adoption of Gen-AIsuch as ChatGPT in pharmacy education, there remains a critical need to assess and mitigate associated risks. This review exploresthe literature and potential strategies for mitigating risks associated with the integration of Gen-AI in pharmacy education.

Aim

To conduct a scoping review to identify implications of Gen-AI in pharmacy education, identify its use and emerging evidence, with a particular focus on strategies which mitigate potential risks to academic integrity.

Methods

A scoping review strategy was employed in accordance with the PRISMA-ScR guidelines. Databases searched includedPubMed, ERIC [Education Resources Information Center], Scopus and ProQuestfrom August 2023 to 20 February 2024 and included all relevant records from 1 January 2000 to 20 February 2024 relating specifically to LLM use within pharmacy education. A grey literature search was also conducted due to the emerging nature of this topic. Policies, procedures, and documents from institutions such as universities and colleges, including standards, guidelines, and policy documents, were hand searched and reviewed in their most updated form. These documents were not published in the scientific literature or indexed in academic search engines.

Results

Articles (n = 12) were derived from the scientific data bases and Records (n = 9) derived from the grey literature. Potential use and benefits of Gen-AI within pharmacy education were identified in all included published articles however there was a paucity of published articles related the degree of consideration to the potential risks to academic integrity. Grey literature recordsheld the largest proportion of risk mitigation strategies largely focusing on increased academic and student education and training relating to the ethical use of Gen-AI as well considerations for redesigning of current assessments likely to be a risk for Gen-AI use to academic integrity.

Conclusion

Drawing upon existing literature, this review highlights the importance of evidence-based approaches to address the challenges posed by Gen-AI such as ChatGPT in pharmacy education settings. Additionally, whilst mitigation strategies are suggested, primarily drawn from the grey literature, there is a paucity of traditionally published scientific literature outlining strategies for the practical and ethical implementation of Gen-AI within pharmacy education. Further research related to the responsible and ethical use of Gen-AIin pharmacy curricula; and studies related to strategies adopted to mitigate risks to academic integrity would be beneficial.

What's been trending with OSCEs in pharmacy education over the last 20 years? A bibliometric review and content analysis

BACKGROUND

Objective structured clinical examinations (OSCEs) remain an integral part of pharmacy education. This study aimed to characterize key researchers, areas, and themes in pharmacy education OSCEs using a bibliometric review with content analysis.

METHODS

A bibliometric review was conducted on literature from over 23 years from January 2000 to May 2023. Articles focusing on any type of OSCE research in pharmacy education in both undergraduate and postgraduate sectors were included. Articles were excluded if they were not original articles or not published in English. A summative content analysis was also conducted to identify key topics.

RESULTS

A total of 192 articles were included in the analysis. There were 242 institutions that contributed to the OSCE literature in pharmacy education, with the leading country being Canada. Most OSCE research came from developed countries and were descriptive studies based on single institution data. The top themes emerging from content analysis were student perceptions on OSCE station styles (n = 98), staff perception (n = 19), grade assessment of OSCEs (n = 145), interprofessional education (n = 11), standardized patients (n = 12), and rubric development and standard setting (n = 8).

IMPLICATIONS

There has been a growth in virtual OSCEs, interprofessional OSCEs, and artificial intelligence OSCEs. Communication rubrics and minimizing assessor variability are still trending research areas. There is scope to conduct more research on evaluating specific types of OSCEs, when best to hold an OSCE, and comparing OSCEs to other assessments.

Harnessing the potential of large language models in medical education: promise and pitfalls

OBJECTIVES

To provide balanced consideration of the opportunities and challenges associated with integrating Large Language Models (LLMs) throughout the medical school continuum.

PROCESS

Narrative review of published literature contextualized by current reports of LLM application in medical education.

CONCLUSIONS

LLMs like OpenAI's ChatGPT can potentially revolutionize traditional teaching methodologies. LLMs offer several potential advantages to students, including direct access to vast information, facilitation of personalized learning experiences, and enhancement of clinical skills development. For faculty and instructors, LLMs can facilitate innovative approaches to teaching complex medical concepts and fostering student engagement. Notable challenges of LLMs integration include the risk of fostering academic misconduct, inadvertent overreliance on AI, potential dilution of critical thinking skills, concerns regarding the accuracy and reliability of LLM-generated content, and the possible implications on teaching staff.

A scoping review of artificial intelligence within pharmacy education

OBJECTIVES

This scoping review aimed to summarize the available literature on the use of artificial intelligence (AI) in pharmacy education and identify gaps where additional research is needed.

FINDINGS

Seven studies specifically addressing the use of AI in pharmacy education were identified. Of these 7 studies, 5 focused on AI use in the context of teaching and learning, 1 on the prediction of academic performance for admissions, and the final study focused on using AI text generation to elucidate the benefits and limitations of ChatGPT use in pharmacy education.

SUMMARY

There are currently a limited number of available publications that describe AI use in pharmacy education. Several challenges exist regarding the use of AI in pharmacy education, including the need for faculty expertise and time, limited generalizability of tools, limited outcomes data, and several legal and ethical concerns. As AI use increases and implementation becomes more standardized, opportunities will be created for the inclusion of AI in pharmacy education.