A scoping review of artificial intelligence within pharmacy education

Artificial intelligence (AI) performance on pharmacy skills laboratory course assignments

OBJECTIVE

To compare pharmacy student scores to scores of artificial intelligence (AI)-generated results of three common platforms on pharmacy skills laboratory assignments.

METHODS

Pharmacy skills laboratory course assignments were completed by four fourth-year pharmacy student investigators with three free AI platforms: ChatGPT, Copilot, and Gemini. Assignments evaluated were calculations, patient case vignettes, in-depth patient cases, drug information questions, and a reflection activity. Course coordinators graded the AI-generated submissions. Descriptive statistics were utilized to summarize AI scores and compare averages to recent pharmacy student cohorts. Interrater reliability for the four student investigators completing the assignments was assessed.

RESULTS

Fourteen skills laboratory assignments were completed utilizing three different AI platforms (ChatGPT, Copilot, and Gemini) by four fourth-year student investigators (n = 168 AI-generated submissions). Copilot was unable to complete 12; therefore, 156 AI-generated submissions were graded by the faculty course coordinators for accuracy and scored from 0 to 100 %. Pharmacy student cohort scores were higher than the average AI scores for all of the skills laboratory assignments except for two in-depth patient cases completed with ChatGPT.

CONCLUSION

Pharmacy students on average performed better on most skills laboratory assignments than three commonly used artificial intelligence platforms. Teaching students the strengths and weaknesses of utilizing AI in the classroom is essential.

Pharmacy faculty and students perceptions of artificial intelligence: A National Survey

INTRODUCTION

This study explores the perceptions, familiarity, and utilization of artificial intelligence (AI) among pharmacy faculty and students across the United States. By identifying key gaps in AI education and training, it highlights the need for structured curricular integration to prepare future pharmacists for an evolving digital healthcare landscape.

METHODS

A 19-item Qualtrics™ survey was created to assess perceptions of AI use among pharmacy faculty and students and distributed utilizing publicly available contacts at schools of pharmacy and intern lists. The electronic survey was open from September 5th to November 22nd 2023. Responses were analyzed for trends and compared between faculty and student responses across four sub-domains.

RESULTS

A total of 235 pharmacy faculty and 405 pharmacy students completed the survey. Responses indicated high familiarity with AI in both groups but found differences in training. Both groups identified ethical considerations and training as major barriers to AI integration. Faculty were less likely to trust AI responses than students despite reporting similar rates of incorrect information. Students were more concerned than faculty about AI reducing pharmacy jobs, particularly in community and health-system settings.

DISCUSSION

This study highlights the need for intentional AI training tailored to pharmacy students, aiming to bridge the knowledge gap and equip them with the skills to navigate an AI-driven future. The inconsistency in how AI is addressed within the curriculum and the lack of established ethical guidelines display the need for clear and consistent institutional policies and professional guidance.

Comparison of a generative large language model to pharmacy student performance on therapeutics examinations

OBJECTIVE

To compare the performance of a generative language model (ChatGPT-3.5) to pharmacy students on therapeutics examinations.

METHODS

Questions were drawn from two pharmacotherapeutics courses in a 4-year PharmD program. Questions were classified as case based or non-case based and application or recall. Questions were entered into ChatGPT version 3.5 and responses were scored. ChatGPT's score for each exam was calculated by dividing the number of correct responses by the total number of questions. The mean composite score for ChatGPT was calculated by adding individual scores from each exam and dividing by the number of exams. The mean composite score for the students was calculated by dividing the sum of the mean class performance on each exam divided by the number of exams. Chi-square was used to identify factors associated with incorrect responses from ChatGPT.

RESULTS

The mean composite score across 6 exams for ChatGPT was 53 (SD = 19.2) compared to 82 (SD = 4) for the pharmacy students (p = 0.0048). ChatGPT answered 51 % of questions correctly. ChatGPT was less likely to answer application-based questions correctly compared to recall-based questions (44 % vs 80 %) and less likely to answer case-based questions correctly compared to non-case-based questions (45 % vs 74 %).

CONCLUSION

ChatGPT scored lower than the average grade for pharmacy students and was less likely to answer application-based and case-based questions correctly. These findings provide valuable insight into how this technology will perform which can help to inform best practices for item development and helps highlight the limitations of this technology.

Modern artificial intelligence and large language models in graduate medical education: a scoping review of attitudes, applications & practice

BACKGROUND

Artificial intelligence (AI) holds transformative potential for graduate medical education (GME), yet, a comprehensive exploration of AI's applications, perceptions, and limitations in GME is lacking.

OBJECTIVE

To map the current literature on AI in GME, identifying prevailing perceptions, applications, and research gaps to inform future research, policy discussions, and educational practices through a scoping review.

METHODS

Following the Joanna Briggs Institute guidelines and the PRISMA-ScR checklist a comprehensive search of multiple databases up to February 2024 was performed to include studies addressing AI interventions in GME.

RESULTS

Out of 1734 citations, 102 studies met the inclusion criteria, conducted across 16 countries, predominantly from North America (72), Asia (14), and Europe (6). Radiology had the highest number of publications (21), followed by general surgery (11) and emergency medicine (8). The majority of studies were published in 2023. Several key thematic areas emerged from the literature. Initially, perceptions of AI in graduate medical education (GME) were mixed, but have increasingly shifted toward a more favorable outlook, particularly as the benefits of AI integration in education become more apparent. In assessments, AI demonstrated the ability to differentiate between skill levels and offer meaningful feedback. It has also been effective in evaluating narrative comments to assess resident performance. In the domain of recruitment, AI tools have been applied to analyze letters of recommendation, applications, and personal statements, helping identify potential biases and improve equity in candidate selection. Furthermore, large language models consistently outperformed average candidates on board certification and in-training examinations, indicating their potential utility in standardized assessments. Finally, AI tools showed promise in enhancing clinical decision-making by supporting trainees with improved diagnostic accuracy and efficiency.

CONCLUSIONS

This scoping review provides a comprehensive overview of applications and limitations of AI in GME but is limited with potential biases, study heterogeneity, and evolving nature of AI.

Exploring the landscape of pharmacology education in Health Professions Programs: From historical perspectives to current approaches to teaching

Although health care professionals have been providing care as part of organized medical systems for millennia, therapeutics in its current sense only emerged in the nineteenth century. Teaching was conducted primarily using a lecture-based format. The Therapeutic Revolution of the 1930s heralded an explosion in the number and types of therapies available. As therapy has evolved so has teaching. Didactic teaching has, in many cases, been replaced by active learning and the health professions curriculum has shifted from an instructor-centered and discipline-based to a learner-centered, competence-based model. Pharmacology as a stand-alone discipline has largely been integrated into systems based or other modes of teaching. Assessments have also evolved from traditional examination formats that emphasized rote knowledge memorization to other assessment formats such as objective structured clinical examinations that emphasize evaluation of skills and attitudes. It has been challenging to define the best modalities given the wide variances in health professions education and the structure of health care systems internationally. Nonetheless, International collaboration efforts have been crucial to define core competencies which can then be used to guide curricular development. Challenges facing educators also include teaching ethical conduct of prescribing and how Artificial Intelligence (AI) can be used in both teaching and evaluation, suggesting the need for on-going dialogue, continuing professional development and research in these important areas.

[Digital learning methods in pharmacy]

With the outbreak of the SARS-CoV‑2 pandemic in March 2020 and the associated restrictions on teaching, digital learning methods were increasingly used at many universities. Digital learning methods generally include fully or partially digitized learning elements such as lecture recordings, open learning materials, or e‑portfolios. Fully or partially digitized learning formats include game-based learning, the inverted classroom, mobile learning, the use of social media, online peer and collaborative learning, and adaptive learning. Digitized realities are created in the context of simulation-based learning and in augmented and virtual reality. Online-based event formats and online degree programs are characterized by an almost exclusive proportion of internet-based learning phases.The extent to which digital learning methods are used in pharmacy courses in Germany is explained in this article using selected practical examples. The selected examples include the creation of an audio podcast to assess the performance of a clinical chemistry internship as a form of digital learning element, the use of a digital analysis tool to carry out medication analyses as an example of mobile learning, a blended learning concept to teach the basics of clinical pharmacy, an online concept of virtual bedside teaching, and a game-like simulation for dispensing medicines. The inclusion of artificial intelligence can be helpful in the development and implementation of digital learning offerings. However, a sufficiently high quality and critical approach must be guaranteed.

Exploring Generative Artificial Intelligence to Enhance Reflective Writing in Pharmacy Education

The integration of generative artificial intelligence (AI) holds the potential to impact teaching and learning. In this commentary, we explore the opportunity for AI to enhance reflective writing (RW) among student pharmacists. AI-guided RW has the potential to strengthen students' reflective capacity, deepen their autobiographical memory, and develop their self-confidence. This commentary presents examples of how AI can be utilized to enrich RW and includes a sample prompt aimed at facilitating student self-reflection. We explore how integrating AI-facilitated RW assignments into the pharmacy curriculum can help students develop detailed examples for self-reflection and gain exposure to the potential uses of AI in their professional development and career advancement.

Will AI reshape or deform pharmacy education?

The integration of artificial intelligence (AI) into pharmacy education offers transformative opportunities but also introduces significant challenges. This commentary explores whether AI will reshape or deform pharmacy education by analyzing its effects on personalized learning, complex concept comprehension, simulation-based clinical training, interprofessional education, and administrative efficiency. While AI-driven tools provide adaptive learning experiences, immersive visualizations, and streamlined administrative processes, concerns persist about overreliance on technology, skill atrophy, ethical and legal challenges, erosion of humanistic skills, inequities stemming from the digital divide, and faculty preparedness. To address these risks while harnessing AI's potential, a balanced approach is essential. Key strategies include integrating AI into curricula alongside traditional teaching methods, fostering digital literacy and critical thinking, enhancing humanistic education, supporting faculty development, ensuring equitable access, and establishing ethical frameworks. By thoughtfully implementing these strategies, pharmacy educators can prepare students to thrive in an AI-driven healthcare landscape while preserving core professional competencies.

Students' perception of the use of artificial intelligence (AI) in pharmacy school

INTRODUCTION

The increasing adoption of artificial intelligence (AI) among college students, particularly in pharmacy education, raises ethical concerns and prompts debates on responsible usage. The promise of the potential to reduce workload is met with concerns of accuracy issues, algorithmic bias, and the lack of AI education and training. This study aims to understand pharmacy students' perspectives on the use of AI in pharmacy education.

METHODS

This study used an anonymous 14-question survey distributed among second, third, and fourth-year pharmacy students at four schools of pharmacy in the United States.

RESULTS

A total of 171 responses were analyzed. Demographic information included institution, class identification (P2, P3, P4), and age range. Regarding the use of AI, 43% of respondents were unaware of limitations of AI tools. Many respondents (45%) had used AI tools to complete assignments, while 42% considered it academic dishonesty. Fifty-six percent believed AI tools could be used ethically. Student perspectives on AI were varied but many expressed that it will be integral to pharmacy education and future practice.

CONCLUSIONS

This study highlights the nuances of AI usage among pharmacy students. Despite limited education and training on AI, students utilized tools for various tasks. This survey provides evidence that pharmacy students are exploring the use of AI and would likely benefit from education on using AI as a supplement to critical thinking.

Student Pharmacists' Perceptions of Artificial Intelligence and Machine Learning in Pharmacy Practice and Pharmacy Education

OBJECTIVE

This study explored student pharmacists' perceptions and attitudes regarding artificial intelligence (AI) and machine learning (ML) in pharmacy practice. Due to AI/ML's promising prospects, understanding students' current awareness, comprehension, and hopes for their use in this field is essential.

METHODS

In April 2024, a Zoom focus group discussion was conducted with 6 student pharmacists using a self-developed interview guide. The guide included questions about the benefits, challenges, and ethical considerations of implementing AI/ML in pharmacy practice and education. The participants' demographic information was collected through a questionnaire. The research team conducted a thematic analysis of the discussion transcript. The results generated by a team member using NVivo were compared with those generated by ChatGPT, and all discrepancies were addressed.

RESULTS

Student pharmacists displayed a generally positive attitude toward the implementation of AI/ML in pharmacy practice but lacked knowledge about AI/ML applications. Participants recognized several advantages of AI/ML implementation in pharmacy practice, including improved accuracy and time-saving for pharmacists. Some identified challenges were alert fatigue, AI/ML-generated errors, and the potential obstacle to person-centered care. The study participants expressed their interest in learning about AI/ML and their desire to integrate these technologies into pharmacy education.

CONCLUSION

The demand for integrating AI/ML into pharmacy practice is increasing. Student and professional pharmacists need additional AI/ML training to equip them with knowledge and practical skills. Collaboration between pharmacists, institutions, and AI/ML companies is essential to address barriers and advance AI/ML implementation in the pharmacy field.

Generative artificial intelligence (Gen-AI) in pharmacy education: Utilization and implications for academic integrity: A scoping review

Introduction

Generative artificial intelligence (Gen-AI), exemplified by the widely adopted ChatGPT, has garnered significant attention in recent years. Its application spans various health education domains, including pharmacy, where its potential benefits and drawbacks have become increasingly apparent. Despite the growing adoption of Gen-AIsuch as ChatGPT in pharmacy education, there remains a critical need to assess and mitigate associated risks. This review exploresthe literature and potential strategies for mitigating risks associated with the integration of Gen-AI in pharmacy education.

Aim

To conduct a scoping review to identify implications of Gen-AI in pharmacy education, identify its use and emerging evidence, with a particular focus on strategies which mitigate potential risks to academic integrity.

Methods

A scoping review strategy was employed in accordance with the PRISMA-ScR guidelines. Databases searched includedPubMed, ERIC [Education Resources Information Center], Scopus and ProQuestfrom August 2023 to 20 February 2024 and included all relevant records from 1 January 2000 to 20 February 2024 relating specifically to LLM use within pharmacy education. A grey literature search was also conducted due to the emerging nature of this topic. Policies, procedures, and documents from institutions such as universities and colleges, including standards, guidelines, and policy documents, were hand searched and reviewed in their most updated form. These documents were not published in the scientific literature or indexed in academic search engines.

Results

Articles (n = 12) were derived from the scientific data bases and Records (n = 9) derived from the grey literature. Potential use and benefits of Gen-AI within pharmacy education were identified in all included published articles however there was a paucity of published articles related the degree of consideration to the potential risks to academic integrity. Grey literature recordsheld the largest proportion of risk mitigation strategies largely focusing on increased academic and student education and training relating to the ethical use of Gen-AI as well considerations for redesigning of current assessments likely to be a risk for Gen-AI use to academic integrity.

Conclusion

Drawing upon existing literature, this review highlights the importance of evidence-based approaches to address the challenges posed by Gen-AI such as ChatGPT in pharmacy education settings. Additionally, whilst mitigation strategies are suggested, primarily drawn from the grey literature, there is a paucity of traditionally published scientific literature outlining strategies for the practical and ethical implementation of Gen-AI within pharmacy education. Further research related to the responsible and ethical use of Gen-AIin pharmacy curricula; and studies related to strategies adopted to mitigate risks to academic integrity would be beneficial.

Advancing reliability and efficiency of urban communication: Unmanned aerial vehicles, intelligent reflection surfaces, and deep learning techniques

Unmanned aerial vehicles (UAVs) have garnered attention for their potential to improve wireless communication networks by establishing line-of-sight (LoS) connections. However, urban environments pose challenges such as tall buildings and trees, impacting communication pathways. Intelligent reflection surfaces (IRSs) offer a solution by creating virtual LoS routes through signal reflection, enhancing reliability and coverage. This paper presents a three-dimensional dynamic channel model for UAV-assisted communication systems with IRSs. Additionally, it proposes a novel channel-tracking approach using deep learning and artificial intelligence techniques, comprising preliminary estimation with a deep neural network and continuous monitoring with a Stacked Bidirectional Long and Short-Term Memory (Bi-LSTM) model. Simulation results demonstrate faster convergence and superior performance compared to benchmarks, highlighting the effectiveness of integrating IRSs into UAV-enabled communication for enhanced reliability and efficiency.

ChatGPT vs pharmacy students in the pharmacotherapy time-limit test: A comparative study in Thailand

OBJECTIVES

ChatGPT is an innovative artificial intelligence designed to enhance human activities and serve as a potent tool for information retrieval. This study aimed to evaluate the performance and limitation of ChatGPT on fourth-year pharmacy student examination.

METHODS

This cross-sectional study was conducted on February 2023 at the Faculty of Pharmacy, Chiang Mai University, Thailand. The exam contained 16 multiple-choice questions and 2 short-answer questions, focusing on classification and medical management of shock and electrolyte disorders.

RESULTS

Out of the 18 questions, ChatGPT provided 44% (8 out of 18) correct responses. In contrast, the students provided a higher accuracy rate with 66% (12 out of 18) correctly answered questions. The findings of this study underscore that while AI exhibits proficiency, it encounters limitations when confronted with specific queries derived from practical scenarios, on the contrary with pharmacy students who possess the liberty to explore and collaborate, mirroring real-world scenarios.

CONCLUSIONS

Users must exercise caution regarding its reliability, and interpretations of AI-generated answers should be approached judiciously due to potential restrictions in multi-step analysis and reliance on outdated data. Future advancements in AI models, with refinements and tailored enhancements, offer the potential for improved performance.

Considerations for conducting a scoping review in pharmacy education

Interrogating the literature is among the first steps a researcher undertakes when actuating a research project or also when any scholar might seek to know what has been done in an area, best practices for conducting a certain activity, or simply to seek answers for a question ranging from one's own personal curiosity to those that might affect departmental or institutional guidance. Decisions on the type of review process to undertake is one that is not taken lightly. This methods commentary outlines the reasons for conducting a scoping review versus a systematic review for topics related to pharmacy education. Considerations for conducting the scoping review are outlined including considerations for writing a protocol prior to conducting a scoping review, to potential platforms to use for transparency of sharing data, processes related to guidelines for data extraction and types of search strategies utilized.

Art or Artifact: Evaluating the Accuracy, Appeal, and Educational Value of AI-Generated Imagery in DALL·E 3 for Illustrating Congenital Heart Diseases

Artificial Intelligence (AI), particularly AI-Generated Imagery, has the potential to impact medical and patient education. This research explores the use of AI-generated imagery, from text-to-images, in medical education, focusing on congenital heart diseases (CHD). Utilizing ChatGPT's DALL·E 3, the research aims to assess the accuracy and educational value of AI-created images for 20 common CHDs. In this study, we utilized DALL·E 3 to generate a comprehensive set of 110 images, comprising ten images depicting the normal human heart and five images for each of the 20 common CHDs. The generated images were evaluated by a diverse group of 33 healthcare professionals. This cohort included cardiology experts, pediatricians, non-pediatric faculty members, trainees (medical students, interns, pediatric residents), and pediatric nurses. Utilizing a structured framework, these professionals assessed each image for anatomical accuracy, the usefulness of in-picture text, its appeal to medical professionals, and the image's potential applicability in medical presentations. Each item was assessed on a Likert scale of three. The assessments produced a total of 3630 images' assessments. Most AI-generated cardiac images were rated poorly as follows: 80.8% of images were rated as anatomically incorrect or fabricated, 85.2% rated to have incorrect text labels, 78.1% rated as not usable for medical education. The nurses and medical interns were found to have a more positive perception about the AI-generated cardiac images compared to the faculty members, pediatricians, and cardiology experts. Complex congenital anomalies were found to be significantly more predicted to anatomical fabrication compared to simple cardiac anomalies. There were significant challenges identified in image generation. Based on our findings, we recommend a vigilant approach towards the use of AI-generated imagery in medical education at present, underscoring the imperative for thorough validation and the importance of collaboration across disciplines. While we advise against its immediate integration until further validations are conducted, the study advocates for future AI-models to be fine-tuned with accurate medical data, enhancing their reliability and educational utility.

An Artificial Intelligence-Supported Medicinal Chemistry Project: An Example for Incorporating Artificial Intelligence Within the Pharmacy Curriculum

OBJECTIVE

This study aims to integrate and use AI to teach core concepts in a medicinal chemistry course and to increase the familiarity of pharmacy students with AI in pharmacy practice and drug development. Artificial intelligence (AI) is a multidisciplinary science that aims to build software tools that mimic human intelligence. AI is revolutionizing pharmaceutical research and patient care. Hence, it is important to include AI in pharmacy education to prepare a competent workforce of pharmacists with skills in this area.

METHODS

AI principles were introduced in a required medicinal chemistry course for first-year pharmacy students. An AI software, KNIME, was used to examine structure-activity relationships for 5 drugs. Students completed a data sheet that required comprehension of molecular structures and drug-protein interactions. These data were then used to make predictions for molecules with novel substituents using AI. The familiarity of students with AI was surveyed before and after this activity.

RESULTS

There was an increase in the number of students indicating familiarity with use of AI in pharmacy (before vs after: 25.3% vs 74.5%). The introduction of AI stimulated interest in the course content (> 60% of students indicated increased interest in medicinal chemistry) without compromising the learning outcomes. Almost 70% of students agreed that more AI should be taught in the PharmD curriculum.

CONCLUSION

This is a successful and transferable example of integrating AI in pharmacy education without changing the main learning objectives of a course. This approach is likely to stimulate student interest in AI applications in pharmacy.

Search Engines and Generative Artificial Intelligence Integration: Public Health Risks and Recommendations to Safeguard Consumers Online

BACKGROUND

The online pharmacy market is growing, with legitimate online pharmacies offering advantages such as convenience and accessibility. However, this increased demand has attracted malicious actors into this space, leading to the proliferation of illegal vendors that use deceptive techniques to rank higher in search results and pose serious public health risks by dispensing substandard or falsified medicines. Search engine providers have started integrating generative artificial intelligence (AI) into search engine interfaces, which could revolutionize search by delivering more personalized results through a user-friendly experience. However, improper integration of these new technologies carries potential risks and could further exacerbate the risks posed by illicit online pharmacies by inadvertently directing users to illegal vendors.

OBJECTIVE

The role of generative AI integration in reshaping search engine results, particularly related to online pharmacies, has not yet been studied. Our objective was to identify, determine the prevalence of, and characterize illegal online pharmacy recommendations within the AI-generated search results and recommendations.

METHODS

We conducted a comparative assessment of AI-generated recommendations from Google's Search Generative Experience (SGE) and Microsoft Bing's Chat, focusing on popular and well-known medicines representing multiple therapeutic categories including controlled substances. Websites were individually examined to determine legitimacy, and known illegal vendors were identified by cross-referencing with the National Association of Boards of Pharmacy and LegitScript databases.

RESULTS

Of the 262 websites recommended in the AI-generated search results, 47.33% (124/262) belonged to active online pharmacies, with 31.29% (82/262) leading to legitimate ones. However, 19.04% (24/126) of Bing Chat's and 13.23% (18/136) of Google SGE's recommendations directed users to illegal vendors, including for controlled substances. The proportion of illegal pharmacies varied by drug and search engine. A significant difference was observed in the distribution of illegal websites between search engines. The prevalence of links leading to illegal online pharmacies selling prescription medications was significantly higher (P=.001) in Bing Chat (21/86, 24%) compared to Google SGE (6/92, 6%). Regarding the suggestions for controlled substances, suggestions generated by Google led to a significantly higher number of rogue sellers (12/44, 27%; P=.02) compared to Bing (3/40, 7%).

CONCLUSIONS

While the integration of generative AI into search engines offers promising potential, it also poses significant risks. This is the first study to shed light on the vulnerabilities within these platforms while highlighting the potential public health implications associated with their inadvertent promotion of illegal pharmacies. We found a concerning proportion of AI-generated recommendations that led to illegal online pharmacies, which could not only potentially increase their traffic but also further exacerbate existing public health risks. Rigorous oversight and proper safeguards are urgently needed in generative search to mitigate consumer risks, making sure to actively guide users to verified pharmacies and prioritize legitimate sources while excluding illegal vendors from recommendations.