Targeting Assessment for Learning within Pharmacy Education

A Pharmacy Drug Knowledge Assessment Pilot: Who Will Fly Farthest and What Downs the Plane?

OBJECTIVE

To evaluate the effectiveness of a sequenced drug knowledge pilot in third professional year students in a capstone course.

METHODS

A three-phase drug knowledge pilot was conducted in spring 2022. Students completed a total of thirteen assessments, including nine low-stakes quizzes, three formative tests, and a final summative comprehensive exam. Results from the previous year's cohort (historical control) who only completed a summative comprehensive exam were compared to the pilot (test group) results to assess effectiveness. The faculty spent over 300 h developing content for the test group.

RESULTS

The pilot group had a mean score of 80.9% on the final competency exam, which was one percent lower than the control group who had a less rigorous intervention. A sub-analysis was conducted that removed the students who failed (<73%) the final competency exam, and no significant difference in the exam score was found. One practice drug exam was found to be moderately correlated and significant (r = 0.62) with the final knowledge exam performance in the control. The number of attempts on the low-stakes assessments had a low correlation with the final exam score in the test group compared to the control (r = 0.24).

CONCLUSION

The results of this study suggest a need to further investigate the best practices for knowledge-based drug characteristic assessments.

Evaluation of the Potential of National Sharing of a Unified Progress Test Among Colleges of Pharmacy in the Kingdom of Saudi Arabia

BACKGROUND

With the expansion in pharmacy education in Saudi Arabia, there is a pressing need to maintain quality assurance in pharmacy programs using several tools. The progress test is a formative assessment tool that can serve to provide information to all stakeholders. This study evaluated the results of a unified progress test that was shared among 15 colleges of pharmacy.

METHODS

The progress test was composed of 100 MCQs where 30% of which cover basic pharmaceutical sciences and 70% cover pharmacy practice. The questions were collected from all the 15 colleges of pharmacy participated in the test. The test was administered online to all undergraduate students in the professional programs of these colleges.

RESULTS

The overall attendance rate was 80% from the total number of students enrolled in the participating colleges. Mean scores of students in basic pharmaceutical sciences were relatively higher than in pharmacy practice. The assessment results of the students in the unified program learning outcomes among colleges were higher in the domains of knowledge and skills compared to competence domain. There was a significant increment in the mean scores of the students as they progress through the years of the professional program. No correlation was found between the mean scores in the test and the cumulative grade point average (cGPA) of all students regardless of their level.

CONCLUSION

The results indicated growth and maintenance of the gained knowledge and skills by the students as they progress through the years of the professional program with consistency in the results among the participating colleges. Sharing a unified test was effective as a valuable tool for the colleges of pharmacy for the purposes of benchmarking and improving the curricula. In addition, it could serve to evaluate learning of students and harmonize knowledge and skills gained by students at different institutions.

Implementation and Use of Workplace-Based Assessment in Clinical Learning Environments: A Scoping Review

PURPOSE

Workplace-based assessment (WBA) serves a critical role in supporting competency-based medical education (CBME) by providing assessment data to inform competency decisions and support learning. Many WBA systems have been developed, but little is known about how to effectively implement WBA. Filling this gap is important for creating suitable and beneficial assessment processes that support large-scale use of CBME. As a step toward filling this gap, the authors describe what is known about WBA implementation and use to identify knowledge gaps and future directions.

METHOD

The authors used Arksey and O'Malley's 6-stage scoping review framework to conduct the review, including: (1) identifying the research question; (2) identifying relevant studies; (3) study selection; (4) charting the data; (5) collating, summarizing, and reporting the results; and (6) consulting with relevant stakeholders.

RESULTS

In 2019-2020, the authors searched and screened 726 papers for eligibility using defined inclusion and exclusion criteria. One hundred sixty-three met inclusion criteria. The authors identified 5 themes in their analysis: (1) Many WBA tools and programs have been implemented, and barriers are common across fields and specialties; (2) Theoretical perspectives emphasize the need for data-driven implementation strategies; (3) User perceptions of WBA vary and are often dependent on implementation factors; (4) Technology solutions could provide useful tools to support WBA; and (5) Many areas of future research and innovation remain.

CONCLUSIONS

Knowledge of WBA as an implemented practice to support CBME remains constrained. To remove these constraints, future research should aim to generate generalizable knowledge on WBA implementation and use, address implementation factors, and investigate remaining knowledge gaps.

Developing Core Competencies for Pharmacy Graduates: The Lebanese Experience

BACKGROUND

In the absence of a national qualification framework for health professionals, the Lebanese Order of Pharmacists (OPL) took the initiative of developing a pharmacy competency framework to standardize pharmacy education in all universities. The objective of this manuscript is to describe the steps that led to the Lebanese Pharmacy Core Competencies Framework.

METHODS

Based on a primer developed by a group of academic instructors, following international reference documents from different parts of the world, the Academic board appointed by the OPL, reviewed, streamlined, and validated the suggested primer using the Delphi technique over three rounds, to come up with the final framework.

RESULTS

The framework encompassed competencies distributed over 7 domains: fundamental knowledge, professional practice, personal skills, supply of medications, safe and rational use of medications, pharmaceutical public health competencies, and organization and management competencies. Its structure was similar to that of the International Pharmaceutical Federation (FIP), with cultural and legal adaptations and the addition of some behaviors related to OPL specific practice and research projects.

CONCLUSION

This work describes the elaboration of the Lebanese Pharmacy Core Competencies Framework, a project considered to be an essential step forward for the profession and a first step toward providing information on human resource planning and professional development of the pharmacy workforce. Instead of adapting existing international educational standards from resource-rich contexts, which may not meet the actual needs of Lebanon, universities can draw our findings to develop relevant programs targeting the skills needed for health workers in Lebanon.

Use of End-of-Class Quizzes to Promote Pharmacy Student Self-Reflection, Motivate Students to Improve Study Habits, and to Improve Performance on Summative Examinations

Underperforming students are often unaware of deficiencies requiring improvement until after poor performance on summative exams. The goal of the current study was to determine whether inclusion of individual end-of-class formative quizzes, which comprise of higher level Bloom's questions, could encourage students to reflect on and address deficiencies and improve academic performance. Ninety-seven out of 123 first-year pharmacy students (79%) enrolled in a Biochemistry and Cell & Molecular Biology course participated in a single-blinded, randomized, controlled, crossover study. Paired t-test analyses demonstrated that that implementation of individual end-of-class formative quizzes resulted in significantly higher summative exam scores for below average students (p = 0.029). Notably, inclusion of quizzes significantly improved performance on higher Bloom's questions for these students (p = 0.006). Analysis of surveys completed by students prior to summative exam indicate that the formative end-of-class quizzes helped students identify deficiencies (89%) and making them feel compelled to study more (83%) and attend review sessions (61%). Many students indicated that quizzes increased stress levels (45%). Our collective data indicate that quizzes can improve summative exam performance for below average first year pharmacy students, and improve self-reflection and student motivation to study. However, the impact on student stress levels should be considered.

An Elective Course in Lesbian, Gay, Bisexual, and Transgender Health and Practice Issues

Objective. To design, implement and assess a lesbian, gay, bisexual, and transgender (LGBT) health and practice elective course for second- and third-year Doctor of Pharmacy (PharmD) students. Methods. The course focused on health promotion, health care barriers, disease prevention, and treatment throughout an LGBT person's lifespan. The course included topic discussions, reading assignments, various active-learning activities, an objective structured clinical examination (OSCE) with a transgender person, and guest speakers from the LGBT community. Five quizzes were administered during the course that were mapped to specific session learning objectives and course learning outcomes. Students completed an anonymous pre- and post-course survey on the seven course learning outcomes to assess their knowledge and skills regarding the health of LGBT people. Results. Students exhibited significant learning with improvement in the seven course learning outcomes. The two most improved course learning outcomes were the medications used for LGBT people and summarizing health care resources available to LGBT people. The content of student portfolios included general themes of discrimination, health care access problems, advocacy, inclusive pharmacy environments, and desire to be a better practitioner. More than 91% of the students actively engaged the guest speakers from the LGBT community. Student performance on quizzes and in the OSCE activity was excellent. The capstone presentations covered a variety of topics including LGBT in Islam. Conclusion. Students demonstrated knowledge of the unique health care issues among the LGBT community. This elective course provides a framework for other pharmacy programs to incorporate LGBT health topics into the curriculum and to engage with their local LGBT community.

Current Trends and Opportunities for Competency Assessment in Pharmacy Education-A Literature Review

An increasing emphasis on health professional competency in recent times has been matched by an increased prevalence of competency-based education models. Assessments can generate information on competence, and authentic, practice-based assessment methods are critical. Assessment reform has emerged as an academic response to the demands of the pharmacy profession and the need to equip graduates with the necessary knowledge, skills and attributes to face the challenges of the modern workforce. The objective of this review was to identify and appraise the range of assessment methods used in entry-level pharmacy education and examine current trends in health professional assessment. The initial search located 2854 articles. After screening, 36 sources were included in the review, 13 primary research studies, 12 non-experimental pharmacy research papers, and 11 standards and guidelines from the grey literature. Primary research studies were critically appraised using the Medical Education Research Study Quality Instrument (MERSQI). This review identified three areas in pharmacy practice assessment which provide opportunities for expansion and improvement of assessment approaches: (1) integrated approaches to performance assessment; (2) simulation-based assessment approaches, and; (3) collection of validity evidence to support assessment decisions. Competency-based assessment shows great potential for expanded use in pharmacy, but there is a need for further research and development to ensure its appropriate and effective use.