Current Trends and Opportunities for Competency Assessment in Pharmacy Education-A Literature Review

Competency-based education in pharmacy- Challenges, opportunities, and the path forward

Competency-based education (CBE) is an essential framework for preparing pharmacy graduates to meet evolving healthcare demands. While existing competency frameworks emphasize foundational knowledge, patient-centered care, and interprofessional collaboration, they often lack structured mechanisms for advanced specialization and career-specific skill development. This paper advocates for a paradigm shift in pharmacy education, transitioning from a current competency "I-shaped" skill model, which prioritizes clinical expertise, to a "comb-shaped" model that fosters broad, interdisciplinary competencies. Integrating customizable modules can restate pharmacy curricula to support lifelong learning, global mobility, and adaptability to emerging professional roles. Drawing insights from competency-based models like the Association of Chartered Certified Accountants (ACCA) competency model and the Royal College of Physicians and Surgeons of Canada Competence by Design (CBD) model, this paper proposes a modular, flexible approach that balances core competencies with specialized career pathways. A structured approach is outlined to support curriculum designers in implementing adaptive frameworks that enhance specialization opportunities while ensuring the retention of fundamental professional standards. By embracing this modular and competency-driven approach, pharmacy education can better equip graduates for diverse career trajectories in research, industry, regulatory affairs, digital health, and clinical practice, thus reinforcing its relevance in an evolving healthcare landscape.

Pharmacists' perceptions of the impact of Objective Structured Clinical Examination on professional development and competence: a cross-sectional study in Qatar

OBJECTIVES

The Objective Structured Clinical Examination (OSCE) is a key tool in healthcare education, assessing clinical skills and competencies. This study explores OSCE's impact on pharmacists' competence, confidence, and practice readiness, focusing on its role in professional development and long-term practice improvement.

METHODS

A cross-sectional survey was conducted among Qatar University pharmacy graduates with at least a year of practice experience. The survey included 10 items on a 4-point Likert scale to capture quantitative data, along with open-ended questions for more detailed reflections and qualitative insights. Quantitative data were analyzed using SPSS, and qualitative data were thematically analyzed through an inductive approach, with two researchers independently coded responses and identifying themes.

KEY FINDINGS

Out of 75 eligible pharmacists, 46 responded (61.3% response rate). Over 85% reported that OSCE improved communication and interprofessional collaboration skills, and 80% felt it helped apply essential professional skills effectively. However, opinions varied on making OSCE a licensure requirement. Six main themes emerged: (i) Enhanced Interprofessional Collaboration Skills, (ii) Application of Professional Skills, (iii) Practical Application of Pharmacy Education, (iv) Adaptability for Evolving Healthcare Challenges, (v) Enhancement of Patient-Facing Communication Skills, and (vi) Identification of Personal Strengths and Weaknesses.

CONCLUSIONS

This study demonstrates that OSCEs enhance pharmacists' clinical skills, communication, and practice readiness. Pharmacy programs should continue integrating OSCEs alongside other assessments for comprehensive evaluation. Policymakers are encouraged to explore OSCEs for licensure, ensuring standardization and evaluator training. Future research should assess OSCE's long-term impact on pharmacists' career development and patient outcomes.

Entrustable professional activities-based objective structured clinical examinations in a pharmacy curriculum

BACKGROUND

The integration of Objective Structured Clinical Examinations (OSCEs) within the professional pharmacy program, contributes to assessing the readiness of pharmacy students for Advanced Pharmacy Practice Experiences (APPEs) and real-world practice.

METHODS

In a study conducted at an Accreditation Council for Pharmacy Education (ACPE)-accredited Doctor of Pharmacy professional degree program, 69 students in their second professional year (P2) were engaged in OSCEs. These comprised 3 stations: best possible medication history, patient education, and healthcare provider communication. These stations were aligned with Entrustable Professional Activities (EPAs) and Ability Statements (AS). The assessment aimed to evaluate pharmacy students' competencies in key areas such as ethical and legal behaviors, general communication skills, and interprofessional collaboration.

RESULTS

The formulation of the OSCE stations highlighted the importance of aligning the learning objectives of the different stations with EPAs and AS. The evaluation of students' ethical and legal behaviors, the interprofessional general communication, and collaboration showed average scores of 82.6%, 88.3%, 89.3%, respectively. Student performance on communication-related statements exceeded 80% in all 3 stations. A significant difference (p < 0.0001) was found between the scores of the observer and the SP evaluator in stations 1 and 2 while comparable results (p = 0.426) were shown between the observer and the HCP evaluator in station 3. Additionally, a discrepancy among the observers' assessments was detected across the 3 stations. The study shed light on challenges encountered during OSCEs implementation, including faculty involvement, resource constraints, and the necessity for consistent evaluation criteria.

CONCLUSIONS

This study highlights the importance of refining OSCEs to align with EPAs and AS, ensuring a reliable assessment of pharmacy students' clinical competencies and their preparedness for professional practice. It emphasizes the ongoing efforts needed to enhance the structure, content, and delivery of OSCEs in pharmacy education. The findings serve as a catalyst for addressing identified challenges and advancing the effectiveness of OSCEs in accurately evaluating students' clinical readiness.

Establishing competency based internship program through participatory action research in a private pharmacy college in the Eastern Province of Saudi Arabia

Objectives

To use Participatory Action Research (PAR) methodology to develop a competency-based training (CBT) program for Bachelor of Pharmacy interns at Mohammed Al-Mana College for Medical Sciences (MACHS), Dammam, Saudi Arabia, based on the International Pharmaceutical Federation (FIP) Global Competency Framework.

Methods

The MACHS Pharmacy Department Training Unit developed a competency-based training (CBT) framework over 6 cohorts of interns based on the FIP Global Competency Framework using the Participatory Action Research (PAR) methodology. Assignments were set throughout the training period to support competency development. Assessment methods used for the evaluation included student portfolio, site preceptor evaluation and the college-based assessments. End of training and baseline results were compared to determine the effectiveness of CBT in terms of improvement of skills. Problems were identified and action plans developed, to be implemented on the following cohort. Successful completion of CBT required a total score of 80%. The students who could not pass the assessment were given a chance to improve their weak competencies and retake the assessment.

Results

Since its implementation, five cohorts have been trained through CBT. Only 12% of interns passed the training in first attempt in the first cohort. This passing percentage dramatically increased to 75-100% in the consecutive cohorts where students scored better in the portfolio, and site preceptor evaluation as compared to the college-based assessment. Students' feedback towards the assignments was positive.

Conclusion

Participatory Action Research was found to be an effective approach towards developing a competency-based training program for Pharmacy interns. More FIP competencies and evaluation strategies will be added to the internship program in the future. Furthermore, a national approach towards implementation of CBT should be used to ensure the uniformity of competency of pharmacists across the kingdom.

Is it Time for Competency-Based Education to Move Forward in Pharmacy Education?

Competency-based education is rapidly emerging as a paradigmatic shift in health professions education, as we grapple with the realities of ever-changing and increasing demands of society and health systems. While pharmacy educators are becoming more familiar with this paradigm, colleagues in medical education have been exploring models and methods of competency-based education for many years, and their experiences can be illuminating for us. The persistent question that drives continuous quality improvement in pharmacy education and the development of initiatives within American Association of Colleges of Pharmacy might be stated as "Is there a better (more effective, more efficient) way to prepare pharmacists (future and current) to meet the medication-related needs of the public?"

The development of a marker system for Pharmacists' Behavioural Skills

OBJECTIVES

Pharmacists increasingly require complex behavioural skills to fulfil enhanced roles within healthcare teams. Behavioural marker systems are used to assess behavioural (or non-technical) skills during immersive simulation. This study aimed to develop a marker system for pharmacists' behavioural skills in patient-focussed care scenarios, and to investigate its content validity.

METHODS

Literature describing existing marker systems and the requisite behavioural skills of pharmacists were presented to two expert panels, alongside video examples of pharmacists in patient-focussed care simulations. The expert panels used this information to develop a new behavioural marker system. A third expert panel assessed the content validity, and the item- and scale-content validity indices were calculated.

KEY FINDINGS

The resulting tool contains four categories, each with three or four skill elements: situation awareness (gathering information; recognising and understanding information; anticipating, preparing and planning), decision-making and prioritisation (identifying options; prioritising; dealing with uncertainty; implementing or reviewing decisions), collaborative working (involving the patient; information sharing; leadership or followership), self-awareness (role awareness; speaking up; escalating care; coping with stress). The scale-content validity index was 0.95 (ideal) and the only item below the acceptable cut-off was 'leadership or followership' (0.7).

CONCLUSIONS

This tool is the first marker system designed to assess the behavioural skills of pharmacists in patient-focussed care scenarios. There is evidence of good content validity. It is hoped that once validated, the Pharmacists' Behavioural Skills marker system will enable pharmacy educators to provide individualised and meaningful feedback on simulation participants' behavioural skills.

Academic resilience in UK pharmacy education - a pilot study applying love and break up letters methodology

INTRODUCTION

Academic resilience is seen as a positive attribute that supports academic attainment and protects against attrition and burnout. Studies have reported that UK pharmacy students have lower academic resilience and wellbeing than the general UK student population but the reasons for this have not been established. This study pilots the use of a novel methodology, love and break-up letter methodology (LBM), to explore these issues focusing on the lived experience of pharmacy students.

METHOD

Final year undergraduate pharmacy study were purposely recruited to the study. Employing LBM, each participant was invited to write reflective love and break-up letters to their academic resilience in higher education during a focus group. Letters and transcripts of subsequent focus group discussion on the feelings and ideas expressed in the letters were thematically analysed.

RESULTS

Three meta-themes were identified within the data; the curriculum as gas lighting; the curriculum as abusive; and the curriculum as controlling. Students described how the curriculum diminishes academic resilience by working against their sense of agency and self-esteem. A constant threat of failure emerged as defining the student lived experience; students felt controlled by a curriculum with negative impacts on wellbeing and perseverance.

DISCUSSION

This is the first study to use LBM to explore academic resilience in UK pharmacy students. The results provide evidence that some students view the pharmacy curriculum as a source of relentless adversity that is responsible for promoting a hidden negative connection between students and their education. Further study is required to determine if the results can be generalised across the UK pharmacy student body to explain why UK pharmacy students have lower academic resilience than other UK university students and the steps needed to improve academic resilience in UK pharmacy students.

Development, Validation, and Reliability of a P1 Objective Structured Clinical Examination Assessing the National EPAs

OBJECTIVE

To document the performance of first-year pharmacy students on a revised objective structured clinical examination (OSCE) based on national entrustable professional activities, identify risk factors for poor performance, and assess its validity and reliability.

METHODS

A working group developed the OSCE to verify students' progress toward readiness for advanced pharmacy practice experiences at the L1 level of entrustment (ready for thoughtful observation) on the national entrustable professional activities, with stations cross-mapped to the Accreditation Council for Pharmacy Education educational outcomes. Baseline characteristics and academic performance were used to investigate risk factors for poor performance and validity, respectively, by comparing students who were successful on the first attempt with those who were not. Reliability was evaluated using re-grading by a blinded, independent grader, and analyzed using Cohen's kappa.

RESULTS

A total of 65 students completed the OSCE. Of these, 33 (50.8%) successfully completed all stations on first attempt, and 32 (49.2%) had to re-attempt at least 1 station. Successful students had higher Health Sciences Reasoning Test scores (mean difference 5, 95% CI 2-9). First professional year grade point average was higher for students who passed all stations on first attempt (mean difference 0.4 on a 4-point scale, 95% CI 0.1-0.7). When evaluated in a multiple logistic regression, no differences were statistically significant between groups. Most kappa values were above 0.4 (range 0.404-0.708), suggesting moderate to substantial reliability.

CONCLUSION

Though predictors of poor performance were not identified when accounting for covariates, the OSCE was found to have good validity and reliability.

A Pharmacy Drug Knowledge Assessment Pilot: Who Will Fly Farthest and What Downs the Plane?

OBJECTIVE

To evaluate the effectiveness of a sequenced drug knowledge pilot in third professional year students in a capstone course.

METHODS

A three-phase drug knowledge pilot was conducted in spring 2022. Students completed a total of thirteen assessments, including nine low-stakes quizzes, three formative tests, and a final summative comprehensive exam. Results from the previous year's cohort (historical control) who only completed a summative comprehensive exam were compared to the pilot (test group) results to assess effectiveness. The faculty spent over 300 h developing content for the test group.

RESULTS

The pilot group had a mean score of 80.9% on the final competency exam, which was one percent lower than the control group who had a less rigorous intervention. A sub-analysis was conducted that removed the students who failed (<73%) the final competency exam, and no significant difference in the exam score was found. One practice drug exam was found to be moderately correlated and significant (r = 0.62) with the final knowledge exam performance in the control. The number of attempts on the low-stakes assessments had a low correlation with the final exam score in the test group compared to the control (r = 0.24).

CONCLUSION

The results of this study suggest a need to further investigate the best practices for knowledge-based drug characteristic assessments.

Objective structured clinical examination as a competency assessment tool of students' readiness for advanced pharmacy practice experiences in South Korea: a pilot study

BACKGROUND

The assessment of pharmacy students' readiness to begin the education of an advanced pharmacy practice experience (APPE) in clinical pharmacy settings continues to gain increasing attention. This study aimed to develop an objective structured clinical examination (OSCE) in the core domains acquired through an introductory pharmacy practice experience (IPPE), for evaluating its appropriateness as a tool of assessing clinical pharmacist competency for APPEs in Korean pharmacy students throughout a pilot study.

METHODS

OSCE's core competency domains and case scenarios were developed through a literature review, ideation by researchers, and external experts' consensus by a Delphi method. A prospective single-arm pilot test was conducted to implement the OSCE for Korean pharmacy students who completed a 60-h course of in-class simulation IPPE. Their competencies were assessed by four assessors in each OSCE station with a pass-fail grading system accompanied by a scoring rubric.

RESULTS

OSCE competency areas including patient counseling, provision of drug information, over-the-counter (OTC) counseling, and pharmaceutical care services were developed with four interactive and one non-interactive cases. Twenty pharmacy students participated in the OSCE pilot test, and their competencies were evaluated by 20 assessors. The performance rate was the lowest in the area of patient counseling for a respiratory inhaler (32.1%) and the highest (79.7%) in OTC counseling for constipation. The students had an average performance rate of 60.4% in their communication skills. Most participants agreed on the appropriateness, necessity, and effectiveness of the OSCE in evaluating pharmacy students' clinical performance and communication skills.

CONCLUSIONS

The OSCE model can be used to assess pharmacy students' readiness for off-campus clinical pharmacy practice experience. Our pilot study suggests the necessity of conducting an OSCE domain-based adjustment of difficulty levels, and strengthening simulation-based IPPE education.

Motivation, satisfaction, and future career intentions of pharmacy students: A cross-sectional preliminary analysis

INTRODUCTION

In recent years, the pharmacy profession has undergone many changes that impacted pharmacy educational programs and students' needs and experiences. In Lebanon, no previous studies have addressed students' perspectives of pharmacy studies and their future aspirations.

METHODS

A cross-sectional survey was randomly sent to all pharmacy students at the main campus of the Lebanese International University. Questions included demographics, students' reasons for choosing pharmacy, satisfaction regarding choice of pharmacy, and future career plans.

RESULTS

Pharmacy as a study major was the first choice for 61% of respondents, and reasons for this choice were family recommendations (87%), friends' encouragement (26%), and schoolteachers' endorsements (20%). Looking for a job with various career opportunities, flexible working hours, security, good salary, and wishing to provide public service and improve health were the main incentives to join the pharmacy program. Concerning their satisfaction, 88% were proud of studying pharmacy and 66% would choose to study pharmacy if they were to select their program of study again. As for career intentions, 68% were considering community pharmacy, 66% hospital pharmacy, and 60% academic pharmacy.

CONCLUSIONS

While the attitudes, perceptions, and career intentions of the respondents were variable, the majority were aspired by family or societal factors to study pharmacy, were satisfied by their choice, and intended to work in the community pharmacy sector. Pharmacy curricula should be regularly assessed to meet students' expectations and society's needs.

Impact of a gamified simulation on pharmacy students' self-assessed competencies

INTRODUCTION

Competency-based pharmacist education develops robust professional identities and prepares graduates for future practice to ensure optimal patient outcomes. An extended gamified simulation was developed as a capstone activity for a new Australian Bachelor of Pharmacy (BPharm) program. The simulation was designed to consolidate students' knowledge, skills, and behaviors from prior learning. This research aimed to explore whether participation in an extended gamified simulation could influence pharmacy students' perceptions of their professional competencies.

METHODS

Data were collected over three years to compare a superseded Master of Pharmacy (MPharm) program with an incoming BPharm program. Final year students were invited to self-assess their professional competencies at the start and end of their final semester of study, using a digital self-assessment tool which replicated Australia's National Competency Standards Framework for Pharmacists. Participants rated their own competency against the 26 competency standards across five domains on a five-point Likert scale (not at all competent to very competent). This provided pre- and post-data to compare the simulation (BPharm intervention) and a traditional semester (MPharm comparison), in addition to final course grades.

RESULTS

From 2016 to 2019, 85 (90.4%) of 94 intervention and 50 (83.3%) of 60 comparison students completed the self-assessment of professional competencies. Participation in the gamified simulation significantly improved students' pharmacotherapeutics grades and pre-post change scores for seven of the 26 competency standards, two of the five domains, and all domains combined of the National Competency Standards.

CONCLUSIONS

An extended, gamified simulation enhances the development of pharmacy students' self-assessed professional competencies.

Assessment of an intensive education program for pharmacists on treatment of tobacco use disorder using an objective structured clinical examination: a randomized controlled trial

BACKGROUND

Tobacco use is one of the major public health threats globally. Community pharmacists are uniquely positioned to offer tobacco cessation services owing to their easy accessibility by the public. To prepare Qatar community pharmacists to develop the competencies and skills required to offer smoking cessation services, an intensive tobacco control education program was designed and implemented. The study aimed to assess the impact of the tobacco education program on the pharmacists' skills and competence.

METHODS

A random sample of community pharmacists in Qatar was chosen for participation in the program. Consenting participants were randomly assigned to either intervention or control groups. The intervention group received an intensive education program on treatment of tobacco-use disorder, while a short didactic session on a non-tobacco-related topic was delivered to the control group. The pharmacists' tobacco cessation skills and competencies were assessed using an Objective Structured Clinical Examination (OSCE).

RESULTS

A total of 54 and 32 community pharmacists in the intervention group and the control group, respectively, completed the OSCE. The intensive tobacco education group achieved significantly higher total scores than the control group in all the OSCE cases. Specifically, the mean total scores for the intervention group were 15.2, 15.3, 14.2, 14.6, 16.3, and 15.2 compared to 8.8, 6.2, 7.7, 9.2, 8.3, and 11.3 for the control group (p < 0.001) for cases one to six respectively.

CONCLUSION

The study demonstrated that an intensive tobacco cessation education program can improve pharmacists' tobacco cessation skills and increase their tobacco cessation counseling abilities.

TRIAL REGISTRATION

Clinical Trials NCT03518476 ( https://clinicaltrials.gov/ct2/show/NCT03518476 ) Registration date: May 8, 2018.

Comparing trained student peers versus paid actors as standardized patients for simulated patient prescription counseling

Background

Simulation can be a useful tool for teaching and assessing clinical skills, but can also be costly and faculty-time intensive. It is defined as a technique to create an activity to portray a real experience for purpose of practicing or evaluating. Simulations can use standardized patients (SPs), which can be paid actors (PASPs), staff and faculty, manikins, volunteers, or students from higher level cohorts, also known as advanced class standardized patients (ACSPs).

Objective

The objective of this study was to conduct a multifaceted analysis comparing ACSPs and PASPs, based on student performance in the assessment, student preference of SP type, and SP performance as an actor.

Methods

ACSPs and PASPs were used in a summative prescription counseling role play. For the evaluation, students counseled a SP about a new prescription medication and answered questions about taking an over-the-counter product with the new medication. The interaction was recorded and evaluated by faculty using a previously developed rubric. SP performance was evaluated by faculty using a separate rubric to determine how well the patient role was performed. A pre- and post-evaluation survey was completed by student pharmacists to gather student preferences about SPs and confidence in their counseling skills. Data were evaluated using a paired t-test.

Results

One hundred sixty-seven student pharmacists completed the summative prescription counseling evaluation. Student pharmacists performed well overall with minimal differences between SP types. Students preferred PASPs to role play the patient but felt that the actor type did not affect their performance.

Conclusions

ACSPs performed the role of the SP well for a summative prescription counseling session without impacting student performance compared to PASPs and with reduced cost. However, students preferred PASPs, and PASPs were better at role playing the patient.

Using Technology to Enhance Teaching and Learning in Pharmacy Education

It was a privilege to serve as guest editors in Pharmacy for the Special Issue 'Technology-Enhanced Pharmacy Teaching and Learning Strategies' [...].

Impact of a novel project management course sequence on innovative thinking in pharmacy students

BACKGROUND

As healthcare continues to become more complex, pharmacist innovators have worked to advance the profession and expand the role of the pharmacist on the healthcare team. Accreditation standards for schools of pharmacy recognize the importance of developing future pharmacist innovators capable of making positive change in the profession, but there are limited resources available on how to best instill innovative thinking in student pharmacists.

EDUCATIONAL ACTIVITY

A two-semester elective course sequence was created for third-year doctor of pharmacy students requiring completion of a longitudinal quality improvement project at a partnering health system. Students collaborated with key stakeholders to design a project plan and charter, identify deliverables, and deliver project results. Innovative thinking was assessed using a mixed methods approach including questionnaires with forced choice and open response items, focus group data, and semi-structured interviews. Each questionnaire item mapped specifically to an element of a validated model for employee innovation. From the beginning to the end of the course sequence, there were significant improvements in student-perceived project management self-efficacy and innovative thinking.

CRITICAL ANALYSIS OF THE EDUCATIONAL ACTIVITY

Student learning outcomes and the course structure mapped closely with a validated model of innovative behavior, demonstrating the effectiveness of utilizing project management to instill innovative thinking in student pharmacists. These findings support the concept that innovative thinking can be taught in pharmacy didactic curricula by situating students in the environment of real-world pharmacy practice.

Communicating definitive uncertainty: Teaching pharmacy students to say "I don't know"

BACKGROUND AND PURPOSE

Communicating uncertainty is an art requiring practice. The purpose of this study was to compare pedagogies for the instruction of pharmacy students in communicating definitive uncertainty.

EDUCATIONAL ACTIVITY AND SETTING

A case scenario featuring a busy physician asking a question without a definitive answer was directed to the pharmacy student using two pedagogies: (1) in-person standardized client and (2) virtual written case. Students provided self-assessments of their confidence in communicating uncertainty after completing the case utilizing a survey containing both rating scale questions and open-ended questions. Self-confidence within-group differences were compared using Wilcoxon signed-rank tests and between-group differences were compared using Mann-Whitney U tests. Responses to open-ended questions were descriptively analyzed for themes using qualitative assessment methods.

FINDINGS

Both the in-person standardized client (70 to 81, P ≤ .001) and the virtual written case (74 to 85, P ≤ .001) significantly increased students' self-rated confidence to verbalize "I don't know" to a healthcare provider. No significant differences were observed between the pedagogies. However, students who participated in the virtual written case mentioned a desire for "additional practice opportunities" more frequently than students who participated in the in-person standardized client.

SUMMARY

In-person standardized client and virtual written case are effective methods for increasing pharmacy student comfort with communicating definitive uncertainty. Further research is needed to instruct pharmacists in uncertainty communication.

Inter-rater Reliability of Preceptors on Clinical Pharmacy Competency Evaluation

Background::

Clinical competence of pharmacy students is better evaluated at their practice sites compared to the classroom. A clinical pharmacy competency evaluation rubric like that of the American College of Clinical Pharmacy (ACCP) is an effective assessment tool for clinical skills and can be used to show item reliability. The preceptors should be trained on how to use the rubrics as many inherent factors could influence inter-rater reliability.

Objective::

To evaluate inter-rater reliability among preceptors on evaluating clinical competence of pharmacy students, before and after a group discussion intervention.

Materials and Methods:

In this quasi-experimental study in a United Arab Emirates teaching hospital, Seven clinical pharmacy preceptors rated the clinical pharmacy competencies of ten recent PharmD graduates referring to their portfolios and preceptorship. Clinical pharmacy competencies were adopted from ACCP and mildly modified to be relevant for the local settings.

Results::

Inter-rater reliability (Cronbach's Alpha) among preceptors was reasonable being practitioners at a single site for 2-4 years. At domain level, inter-rater reliability ranged from 0.79 - 0.93 before intervention and 0.94 - 0.99 after intervention. No inter-rater reliability was observed in relation to certain competency elements ranging from 0.31 - 0.61 before the intervention, but improved to 0.79 - 0.97 after the intervention. Intra-class correlation coefficient improved among all individual preceptors being reliable with each other after group discussion though some had no reliability with each other before group discussion.

Conclusion::

Group discussion among preceptors at the training site was found to be effective in improving inter-rater reliability on all elements of the clinical pharmacy competency evaluation. Removing a preceptor from the analysis did not affect inter-rater reliability after group discussion.

Developing a consensus-based scoring rubric to enhance practice-based assessment of student nurses' clinical competence: A Delphi study

BACKGROUND

Concerns about reliability and validity of practice-based assessment of professional competencies are frequently reported in the literature. Difficulty in understanding competency statements or distinguishing different achievement levels has been found to be a major factor.

OBJECTIVES

To develop a consensus-based scoring rubric based on stakeholders' interpretations of level descriptors for student nurses' professional values competencies.

DESIGN

Two rounds of Classic e-Delphi.

SETTINGS

This study was conducted in a London based university using Bristol Online Survey website as a host.

PARTICIPANTS

100 stakeholders with vested interests in undergraduate pre-registration nurse education were purposefully invited to participate.

METHOD

Round one collected free-text interpretations of the United Kingdom Nursing and Midwifery Council professional values competency statements. Round two used a Likert scale questionnaire to measure the level of agreement to the level descriptor statements generated through round one. Responses were analysed through content analysis in round one and consensus measure in round two. A threshold of 70% agreement to determine consensus was set in advance.

RESULTS

In round one, 47 participants provided their interpretations of the competency statements. In round two, 51 participants completed the questionnaire. All 24 items achieved a strong consensus with 86%-100% of participants agreeing or strongly agreeing with the statements.

CONCLUSIONS

A Delphi study was successfully used to develop a consensus-based scoring rubric with clearly stated descriptors for professional values competency statements. This scoring rubric holds the potential to enhance practice-based assessment across all healthcare professional disciplines.

Transitioning Competency-Based Communication Assessments to the Online Platform: Examples and Student Outcomes

In light of the COVID-19 pandemic, pharmacy education has shifted from in-person teaching and assessments to the virtual environment. Many education programs had previously adopted objective structured clinical examinations (OSCEs) to assess communication abilities in-person with standardized patients (SPs). However, comparative student outcome data between virtual and in-person methods as well as guidance on how to conduct communication-based OSCEs in the virtual environment is lacking. The University of Colorado Skaggs School of Pharmacy and Pharmaceutical Sciences (SSPPS) describes its methods of conducting two types of communication-based OSCEs (patient counseling and gathering a medical history). Student performance data from the two virtual OSCEs in 2020 was compared to results from two 2019 in-person OSCEs using Mann Whitney U Tests. The 2020 cohort scored significantly higher than the 2019 cohort in all variables (i.e., using effective verbal and non-verbal communication, using patient friendly education, organizing the encounter, demonstrating empathy, establishing trust, professionalism) and in overall score. However, the effect size for these findings indicate the differences between performances are generally small and more likely due to changes in grading patterns due to the pandemic.

Cognitive load theory: Implications for assessment in pharmacy education

The concept of mental workload is well studied from a learner's perspective but has yet to be better understood from the perspective of an assessor. Mental workload is largely associated with cognitive load theory, which describes three different types of load. Intrinsic load deals with the complexity of the task, extraneous load describes distractors to the task at hand, and germane load focuses on the development of schemas in working memory for future recall. Studies from medical education show that all three types of load are relevant when considering rater -based assessment (e.g. Objective Structured Clinical Examinations (OSCEs), or experiential training). Assessments with high intrinsic and extraneous load may interfere with assessors' attention and working memory and result in poorer quality assessment. Reducing these loads within assessment tasks should therefore be a priority for pharmacy educators. This commentary aims to provide a theoretical overview of mental workload in assessment, outline research findings from the medical education context, and propose strategies to be considered for reducing mental workload in rater-based assessments relevant to pharmacy education. Suggestions for future research are also addressed.

The Pharmacy Game-GIMMICS® a Simulation Game for Competency-Based Education

The profile of the profession of pharmacists has profoundly changed over the last decades. Pharmacy education has moved towards competency-based education. The pharmacy game, called GIMMICS^®, developed at the University of Groningen, is unique in combining simulation with serious gaming to teach a wide range of competencies. In this article, we describe the learning goals, the assessment methods, the teaching tools, and the students’ view of the pharmacy game. The learning goals are to train the competencies of collaboration, leadership, communication, and pharmaceutical expertise. The core of the game is the simulation of community pharmacy practice activities, such as patient counseling, processing of prescriptions, and collaboration with other health professionals. Students are assessed individually and as a pharmacy team. The pharmacy team, with the largest number of patients wins the game. Student evaluations show that they value the course. Currently, seven universities from around the globe have adopted the pharmacy game in their curriculum, adjusting the course to their country’s pharmacy practice and educational system.

Developing a validity argument for a simulation-based model of entrustment in dispensing skills assessment framework

INTRODUCTION

Integrated assessment of multiple competencies at once, including entrustable professional activity (EPA) based assessment, is emerging as an effective approach to competency-based evaluation of health professionals. However, there is an absence of validated assessment frameworks in entry level pharmacy education. We aimed to develop an assessment framework and establish a validity argument, containing multiple sources of evidence, for use in the integrated assessment of pharmacy student's competency in all aspects of the supply of prescribed medicine(s).

METHODS

A two-phase prospective study was conducted. Phase 1 involved development and content validation of the Model of Entrustment in Dispensing Skills (MEDS) assessment framework using a literature review, a think-aloud study, and expert consultation. In phase 2, a pilot study was conducted with faculty and expert assessors to test the framework. Subsequent analysis involved psychometric evaluation of rating scales and usability testing.

RESULTS

Validity evidence was collected and organised across the two study phases. The MEDS framework had good evidence of content validity supported by the rigorous development and consultation process, as well as case sampling, with 88% of national practice-based competencies represented across the two simulations. Reliability coefficients were high and acceptable, supporting strong agreement across domains, students, and simulations as well as a strong correlation between the EPA and total score (spearman correlation rho 0.725, P < .001).

CONCLUSIONS

This study describes a valid and rigorous approach for the implementation and interpretation of an integrated simulation-based assessment tool for determining pharmacy student's progress towards entrustment for independent medication supply practice.

Alignment of CanMEDS-based Undergraduate and Postgraduate Pharmacy Curricula in The Netherlands

In this article the design of three master programs (MSc in Pharmacy) and two postgraduate specialization programs for community or hospital pharmacist is described. After a preceding BSc in Pharmacy, these programs cover the full pharmacy education capacity for pharmacists in primary and secondary health care in the Netherlands. All programs use the CanMEDS framework, adapted to pharmacy education and specialization, which facilitates the horizontal integration of pharmacists’ professional development with other health care professions in the country. Moreover, it is illustrated that crossing the boundary from formal (university) education to experiential (workplace) education is eased by a gradual change in time spent in these two educational environments and by the use of comparable monitoring, feedback, and authentic assessment instruments. A reflection on the curricula, based on the principles of the Integrative Pedagogy Model and the Self-determination Theory, suggests that the alignment of these educational programs facilitates the development of professional expertise and professional identity of Dutch pharmacists.

Evaluation of a program of pharmaceutical counseling for French patients on oral anticoagulant therapy

Background Oral anticoagulants are widely used for treatment and prevention of thromboembolic diseases. We set up a pharmaceutical counseling program for both direct oral anticoagulant and vitamin K antagonist drugs in our hospital in 2015. Objective Evaluate patient satisfaction and the evolution of their knowledge throughout the pharmaceutical counseling program on anticoagulants and identify knowledge variability factors. Setting Cardiology Inpatient Unit from the University Antoine Béclère Hospital, France. Methods Evaluation was based on data collection of patients surveyed between 2015 and 2018. Inpatients in the cardiology department on oral anticoagulants were eligible. The learning process was designed to enhance patient knowledge and understanding based on 10 cognitive or self-management skills, relating to the optimization of oral anticoagulant therapy management. It consisted in 2 face-to-face interviews during hospitalization and 2 additional phone interviews one and six months after discharge. The median patient score was evaluated at each step of the process as well as the mean score for each item from the global population. A sub-analysis was run on the less well-acquired skills in order to identify risk factors for limited knowledge. The association between those factors and the level of knowledge (score ≥ 7 or < 7) was assessed using Chi square test followed by multivariate analysis. Main outcome measure Patient knowledge of anticoagulation therapy depending on specific factors. Results Of the 880 patients eligible for pharmaceutical counseling, 319 entered the process and 102 completed it. Median knowledge scores were 8/10 and 9/10 after the first and the final interviews respectively with a significant improvement (p = 0.0003). The least well-acquired items at each step were surveillance and under-dosing management. The sub-analysis showed the use of vitamin K antagonist to be linked to an enhanced understanding related to treatment surveillance (p = 0.029). Patients suffering from atrial fibrillation were found to have a worse understanding of under-dosing management (p = 0.013). Finally, patients evaluated the process as helpful and suitable for their conditions. Conclusion Pharmaceutical counseling is appropriate for patients, improving and maintaining knowledge of oral anticoagulants. Our evaluation highlights the need to focus on patient-specific profiles to reach a satisfactory level of knowledge.

Mastery learning in a bachelor's of nursing program: the Roseman University of Health Sciences experience

BACKGROUND

Roseman University of Health Sciences (RUHS) developed and delivers a mastery learning curriculum designed for students to acquire the knowledge and skills to become competent nurses. Despite a trend in nursing education to adopt competency-based education (CBE) models, there is little in the nursing literature about programs based on a mastery model. The aim of this study is to describe an undergraduate nursing program built on a mastery learning model and to report on program outcome measures.

METHODS

The 18-month BSN nursing program is divided into blocks, varying in length and focusing on a single subject. Students must demonstrate mastery, defined as ≥90% on an assessment, to pass a block. Recognizing the critical nature of health care, educators seek methods to assure that practitioners become competent to perform the services they provide.Program outcomes reported include comparisons to national standards and RUHS student exit survey data.

RESULTS

From 2013 to 2017 the RUHS College of Nursing students' pass rates ranged from 82 to 97% for the National Council Licensure Examination exam compared to national pass rates between 81.8-84.5% during the same time frame. The program completion rate ranged from 86 to 100% and employment rates exceeded accreditation standards. Students reported overall satisfaction with their education as 4.38 and with the block system as 4.74 (5 point Likert scale).

CONCLUSIONS

Roseman University's mastery learning model appears successful as measured by high levels of student satisfaction, outcomes on exams, and degree completion when compared to national averages. The results suggest that other nursing and health profession's programs can develop a successful mastery based learning model.