Entrustable Professional Activities for Pharmacy Practice

Responding to the real problem of sustainable resuscitation skills with real assessment. Mixed-methods evaluation of an authentic assessment programme

INTRODUCTION

The retention of resuscitation skills is a widespread concern, with a rapid decay in competence frequently following training. Meanwhile, training programmes continue to be disconnected with real-world expectations and assessment designs remain in conflict with the evidence for sustainable learning. This study aimed to evaluate a programmatic assessment pedagogy which employed entrustment decision and the principles of authentic and sustainable assessment (SA).

METHODS

We conducted a prospective sequential explanatory mixed methods study to understand and address the sustainable learning challenges faced by final-year undergraduate paramedic students. We introduced a programme of five authentic assessments based on actual resuscitation cases, each integrating contextual elements that featured in these real-life events. The student-tutor consensus assessment (STCA) tool was configured to accommodate an entrustment scale framework. Each test produced dual student led and assessor scores. Students and assessors were surveyed about their experiences with the assessment methodologies and asked to evaluate the programme using the Ottawa Good Assessment Criteria.

RESULTS

Eighty-four students participated in five assessments, generating dual assessor-only and student-led results. There was a reported mean score increase of 9% across the five tests and an 18% reduction in borderline or below scores. No statistical significance was observed among the scores from eight assessors across 420 unique tests. The mean student consensus remained above 91% in all 420 tests. Both student and assessor participant groups expressed broad agreement that the Ottawa criteria were well-represented in the design, and they shared their preference for the authentic methodology over traditional approaches.

CONCLUSION

In addition to confirming local sustainability issues, this study has highlighted the validity concerns that exist with conventional resuscitation training designs. We have successfully demonstrated an alternative pedagogy which responds to these concerns, and which embodies the principles of SA, quality in assessment practice, and the real-world expectations of professionals.

Determining the Optimal Length of Clinical Education Experiences: Surveying Doctor of Physical Therapy Academic and Clinical Faculty

INTRODUCTION/REVIEW OF LITERATURE

There is wide variability in the length of full-time clinical education experiences (CEEs) among Doctor of Physical Therapy (DPT) programs. We investigated academic and clinical faculty perspectives on the optimal length and level of agreement on the length, timing, and offering of full-time CEEs.

SUBJECTS

A survey assessing perspectives on the optimal length and level of agreement for the length, timing, and offering of full-time CEEs was completed by 100 academic and 240 clinical faculty.

METHODS

Differences between academic and clinical faculty responses on the optimal length and level of agreement were assessed with a 2-sample test of proportions and 2-sample t-test.

RESULTS

Participants believed that the length of full-time CEEs (in weeks) should be single CEE 10.6 (SD 2.3); first CEE 7.8 (SD 2.1); terminal CEE 12.5 (SD 5.4); integrated CEE 9.6 (SD 1.9); and total CEEs 37.2 (SD 6.9). There were significant differences (MD = -3.0; 95% confidence interval [CI] -4.6 to -1.6) between academic and clinical faculty perspectives for the total length of CEEs. No significant differences between academic and clinical faculty perspectives for the length of single, first, terminal, or integrated CEEs. There were significant differences between academic and clinical faculty perspectives for the optimal length of inpatient (MD = 1.1; 95% CI 0.6-1.6) and specialty (MD = 0.8; 95% CI 0.2-1.3) settings. There were significant differences between academic and clinical faculty level of agreement for 8 of 15 items.

DISCUSSION AND CONCLUSION

There was consistency between academic and clinical faculty regarding the perceived optimal length of full-time CEEs. Academic and clinical faculty perspectives for the optimal length of terminal full-time CEEs (12.5 weeks) were different than those for the national average (21.8 weeks) length of terminal full-time CEEs. Our study provides evidence to support DPT programs' clinical education curriculum decisions regarding the length of full-time CEEs to optimize students learning and maturation.

Pharmacy faculty experiences with student academic entitlement: a multinational study from the Arab world

Academic Entitlement (AE) is the expectation by students to receive high grades or preferential treatment without significant effort. Exploring AE from faculty perspective has not been investigated in Arab colleges of pharmacy. The aim of this study was to explore experiences and perceptions towards student AE among pharmacy faculty in the Arab World. A cross-sectional, self-administered, anonymous, electronic survey was sent to pharmacy faculty across pharmacy colleges in Arab countries. The survey collected demographic data, an AE measure including 17 items reflecting seven AE components, and faculty perceptions and perceived reasons for AE. A total of 345 responses were collected. The AE level was moderate (46.05 ±7.29), and the highest scores among its components were for customer service expectation (62%) and responsibility avoidance (59%). In multiple linear regression, AE showed positive significant association with faculty in clinical pharmacy departments and those having fewer years of experience. Most common complaints heard by faculty from students were requests to turn in assignments late (90%), while the most common communication issues faculty faced with students were unprofessional verbal communication (58%) and unprofessional messages on social media (57%). Poor admission criteria (40%) and existence of multiple private colleges of pharmacy (37%) were the most common perceived reasons for AE by participating faculty. This study reveals moderate AE experienced by pharmacy faculty in the Arab World, as well as common complaints, communication issues, and AE reasons. In collaboration with other stakeholders, faculty play an important role in indicating expectations from students regarding AE, and research is warranted to check if such interventions reduce AE among pharmacy students.

You'll have to call the attending: Impact of a longitudinal, "real-time" case-based infectious diseases elective on entrustable professional activities to enhance APPE readiness

BACKGROUND AND PURPOSE

Transitioning from the didactic to experiential setting is challenging for student pharmacists, perhaps due to lack of experiences providing "real-time" clinician interaction. We describe findings from a semester-long infectious diseases (ID) didactic elective that utilized a national cohort of preceptors and faculty across the United States to mimic clinician interaction and "real-time" ID management of various disease states. The mechanics of this elective provide a framework for others to implement to enhance advanced pharmacy practice experience (APPE) readiness.

EDUCATION ACTIVITY AND SETTING

Students enrolled in an ID elective course at a school of pharmacy participated in "real-time" acute care scenarios. They assisted in multidisciplinary management of a patient's infection, mimicking "rounds" on an APPE, via interaction with external pharmacist volunteers (playing the roles of other healthcare personnel). Additionally, students formally presented and discussed their cases within the class, further promoting learning while optimizing presentation skills. Pharmacist volunteers were surveyed to assess student performances as measured by four entrustable professional activities (EPAs).

FINDINGS

A total of 48 volunteer opportunities occurred during two course offerings. Results from 43 surveys were analyzed (90% response rate). Of those responses, 22/24 (92%) played the role of attending physician, and 19/24 (79%) played the role of technician. Volunteers agreed that students met the four EPAs evaluated (agreement was 85-100%).

SUMMARY

This semester-long elective provided "real-time" experience and feedback for pre-APPE students to enhance APPE readiness and reinforce EPAs. Students are likely to benefit from mimicked intra-professional interaction and augmented critical thinking skills that could be adapted to various disease states within pharmacy curricula.

Entrustable professional activities-based objective structured clinical examinations in a pharmacy curriculum

BACKGROUND

The integration of Objective Structured Clinical Examinations (OSCEs) within the professional pharmacy program, contributes to assessing the readiness of pharmacy students for Advanced Pharmacy Practice Experiences (APPEs) and real-world practice.

METHODS

In a study conducted at an Accreditation Council for Pharmacy Education (ACPE)-accredited Doctor of Pharmacy professional degree program, 69 students in their second professional year (P2) were engaged in OSCEs. These comprised 3 stations: best possible medication history, patient education, and healthcare provider communication. These stations were aligned with Entrustable Professional Activities (EPAs) and Ability Statements (AS). The assessment aimed to evaluate pharmacy students' competencies in key areas such as ethical and legal behaviors, general communication skills, and interprofessional collaboration.

RESULTS

The formulation of the OSCE stations highlighted the importance of aligning the learning objectives of the different stations with EPAs and AS. The evaluation of students' ethical and legal behaviors, the interprofessional general communication, and collaboration showed average scores of 82.6%, 88.3%, 89.3%, respectively. Student performance on communication-related statements exceeded 80% in all 3 stations. A significant difference (p < 0.0001) was found between the scores of the observer and the SP evaluator in stations 1 and 2 while comparable results (p = 0.426) were shown between the observer and the HCP evaluator in station 3. Additionally, a discrepancy among the observers' assessments was detected across the 3 stations. The study shed light on challenges encountered during OSCEs implementation, including faculty involvement, resource constraints, and the necessity for consistent evaluation criteria.

CONCLUSIONS

This study highlights the importance of refining OSCEs to align with EPAs and AS, ensuring a reliable assessment of pharmacy students' clinical competencies and their preparedness for professional practice. It emphasizes the ongoing efforts needed to enhance the structure, content, and delivery of OSCEs in pharmacy education. The findings serve as a catalyst for addressing identified challenges and advancing the effectiveness of OSCEs in accurately evaluating students' clinical readiness.

Examining Competency-Based Education Through the Lens of Implementation Science: A Scoping Review

OBJECTIVES

This study aimed to identify evidence for the implementation and assessment of competency-based education (CBE) in health professions curricula using an implementation science framework.

FINDINGS

Using the PRISMA framework, a systematic review of the literature applying a prespecified and piloted search strategy from 2017 to the present in PubMed and CINAHL was performed. References identified from the search strategy were imported into Covidence for title and abstract screening and full-text review by 2 researchers. A third researcher resolved discrepancies. Data were extracted and synthesized to identify key elements from the article related to implementation science, with a quality appraisal. A total of 25 studies out of 304 initially identified records were included. The studies covered a broad range of health professions and countries. Key findings were limited use of implementation science elements, including variability in CBE implementation, limited fidelity assessment, and partial examination of the process continuum. Programs with a more robust implementation approach have a team-based strategy to lead, implement, and support CBE. Motivation and training of faculty are also key components of successful CBE implementation.

SUMMARY

Competency-based education is implemented differently across institutions, with variation among programs in their choice of elements of implementation science used. Further research is needed to examine CBE from an implementation science perspective and address remaining questions.

Less Points, Less Stress, Better Learning in a Skills Laboratory Course Using Alternative Grading

OBJECTIVE

The objective of this study was to evaluate the impact of alternative grading strategies on students' knowledge and confidence in a redesigned pharmacy skills laboratory course.

METHODS

Alternative grading strategies including revise-and-resubmit assignments, purposeful peer and instructor feedback, and self-reflection were used to partially replace traditional grading strategies. Pre-post knowledge and confidence surveys were administered to 3 consecutive cohorts of students. Cohort 1 (N = 65) was exposed to traditional assessment strategies and was used as the control arm. Cohorts 2 (N = 59) and 3 (N = 51) were exposed to alternative grading strategies and were used as the intervention arms. Cohort 3 completed an additional postsurvey to evaluate students' perceptions of alternative grading strategies.

RESULTS

Students enrolled in a pharmacy skills laboratory course at a single institution demonstrated increased pre-to-post knowledge and confidence when assessed using both traditional and alternative grading strategies. Students in the alternative grading cohort entered the course with statistically significant lower levels of knowledge of institutional pharmacy-related topics but exited the course with higher levels of knowledge of institutional pharmacy-related topics as compared to the traditionally graded cohort.

CONCLUSION

Findings from the evaluation of a two-year course redesign indicated that transitioning from a traditional to an alternative grading strategy resulted in similar or higher learning outputs and a significant increase in students' confidence in institutional pharmacy practice skills and abilities.

Development and Refinement of a Matrix Competency Framework, with Associated Entrustable Professional Activities, to Support Initial Pharmacy Education in Kuwait

The development of competency frameworks serves as the foundation for the development of competency-based education. It is vital to develop a country-specific framework to address the specific needs of the local population for pharmacy services. This study aimed to describe the development process of a competency framework for undergraduate pharmacy education in Kuwait with a unique matrix structure. The process started with the development of guiding principles for curriculum revision and implementation, as well as the identification of global educational outcomes. This process was followed by: (A) a needs assessment with key stakeholders; (B) development of the initial competency framework; and (C) refinement of the framework. Qualitative data were thematically analyzed to identify the main competency domains that students need to perform the identified entrustable professional activities (EPAs). Five population needs were identified by the needs assessment, with 17 EPAs suggested to fulfill those needs. In addition, 11 competency domains were identified. The initial competency framework was created as a 3 × 8 matrix, with 3 professional and 8 transversal competency domains. Refinement of the framework resulted in the removal of redundancies and the development of a global behavior competency profile. The development of a matrix competency framework and associated EPAs for Kuwait serves as a foundation for preparing pharmacists to fulfill local population needs and expanding the scope of practice in the country.

Development and Validation of Entrustable Professional Activities for Provisionally Registered (Intern) Pharmacists in Australia

OBJECTIVE

The purpose of this study was to describe the development and validation of entrustable professional activities (EPAs) for a pharmacy intern training program in Australia.

METHODS

Performance outcomes with EPAs suggested as an assessment in the intern year were identified and EPAs were developed. The EQual rubric, designed for EPAs in graduate medical education settings, was used to validate the EPAs. Eight subject matter experts were recruited to validate 14 EPAs using prespecified cutoffs in the EQual rubric to determine the degree of alignment with established key domains of the EPA construct. Descriptive statistics for the EQual rubric scores were calculated for each of the 14 EPAs overall and for each of the 3 domains. Any EPA falling below the prespecified cut score required revisions and a second round of validation.

RESULTS

Eight responses were recorded for the first round of validation. Seven EPAs were below the discrete units of work cutoff (4.17); 2 of these EPAs were also below the EPA curricular role cutoff (4.00). All scores described in the EPAs as entrustable, essential, and important tasks of the profession were above the cutoff (4.00). Seven subject matter experts responded to the second round of validation. All 7 revised EPAs scored above the EQual rubric's overall cutoff of 4.07.

CONCLUSION

Fourteen EPAs were validated for use with provisionally registered (intern) pharmacists. To our knowledge, this is the first time that the EQual rubric has been used in the evaluation and validation of pharmacy EPAs.

Evaluating the Quality of the Core Entrustable Professional Activities for New Pharmacy Graduates

This study aimed to evaluate the quality of the American Association of Colleges of Pharmacy Core Entrustable Professional Activities (Core EPAs) for New Pharmacy Graduates according to standards outlined in competency-based education literature utilizing the Queen's EPA Quality (EQual) rubric. A cohort of pharmacists with EPA expertise rated Core EPA quality with the EQual rubric and provided recommendations for revisions. A generalizability study determined the reliability of the EQual ratings with pharmacist users. Nine pharmacists responded (4.4%). Most EPAs (9/15) did not reach the overall cut-off score, indicating low quality. EPAs 1 through 5 and EPA 14 (fulfill a medication order) were deemed high quality. EPA 12 (use evidence-based information to advance patient care) scored the lowest at 3.47 (SEM 0.29). EPA 14 scored the highest at 4.60 (SEM 0.14). EPA 15 (create a written plan for continuous professional development) was the only EPA to fail to reach the cut-off across all EQual domains. EPAs in the Patient Care Provider Domain received significantly higher ratings than other EPAs. On average, three respondents recommended revision for each. Most comments aligned with the EPA's EQual rubric performance. The generalizability study analysis revealed excellent reliability (G = 0.80). Determining EPA quality utilizing objective measurement tools should drive EPA development and revisions to more accurately reflect the roles, responsibilities, and expectations of pharmacists on the healthcare team.

The AACP Academic Affairs Committee's Final 2022 Curricular Outcomes and Entrustable Professional Activities (COEPA) for Pharmacy Graduates to Replace 2013 CAPE and 2016 EPAs

The American Association of Colleges of Pharmacy (AACP) Academic Affairs Committee was charged with revising both the 2013 Center for the Advancement of Pharmacy Education (CAPE) Educational Outcomes (EOs) and the 2016 Entrustable Professional Activities (EPAs). The Committee changed the document name from the CAPE outcomes to COEPA, (Curricular Outcomes and Entrustable Professional Activities) since the EOs and EPAs would now be housed together. A draft of the COEPA EOs and EPAs was released at the AACP July 2022 Annual meeting. After receiving additional stakeholder feedback during and after the meeting, the Committee made additional revisions. The final COEPA document was submitted to and approved by the AACP Board of Directors in November 2022. This COEPA document contains the final version of the 2022 EOs and EPAs. The revised EOs have been reduced to 3 domains and 12 subdomains (from 4 domains and 15 subdomains previously in CAPE 2013) and the revised EPAs have been reduced from 15 to 13 activities.

The AACP Academic Affairs Committee's Guidance for Use of the Curricular Outcomes and Entrustable Professional Activities (COEPA) for Pharmacy Graduates

The 2021-2023 American Association of Colleges of Pharmacy Academic Affairs Committee (AAC) was charged with and completed the revision of the 2013 Center for the Advancement of Pharmacy Education Outcomes and the 2016 Entrustable Professional Activity (EPA) statements for new pharmacy graduates. This work resulted in a new combined document, the Curricular Outcomes and Entrustable Professional Activities (COEPA) that was unanimously approved by the American Association of Colleges of Pharmacy Board of Directors and was published in the Journal. The AAC was also charged with providing stakeholders with guidance about how to use the new COEPA document. To achieve this charge, the AAC created example objectives for all 12 Educational Outcomes (EOs) and example tasks for all 13 EPAs. Although programs are asked to retain the EO domains, subdomains, one-word descriptors, and descriptions, unless they are adding more EOs or increasing the taxonomy level of a description, colleges and schools of pharmacy can expand or edit the example objectives and example tasks to meet local needs, as these are not designed to be prescriptive. This guidance document is published separately from the COEPA EOs and EPAs to reinforce the message that the example objectives and tasks are modifiable.

An overview of Instructional approaches and decision-making strategies to curtail curricular overload

OBJECTIVES

Curriculum overload often occurs when content is kept in the curriculum that may no longer be necessary to prepare students for professional practice. The overload becomes compounded by the addition of new content from the ever-changing professional practice needs and updates to accreditation standards. Challenges may occur when programs must first determine the "level" of proficiency a new graduate should attain and then determine the appropriate breadth and depth of educational outcomes in relation to proficiency, while examining what content should be retained from past curricula. Thus, the purpose of this manuscript is to summarize institutional approaches for making content delivery more effective and efficient with the goal of curtailing curriculum expansion.

FINDINGS

Four key elements were consistently identified in the literature as important considerations to address curriculum overload - 1) communication and coordination among faculty, 2) incorporation of active learning strategies, 3) effective utilization of technology, and 4) minimizing faculty and student workload and cognitive burden.

SUMMARY

Each pharmacy program will need to take an individualized approach in addressing curriculum overload; however, consideration of the aforementioned key elements can assist in making these decisions. With increased student engagement in the classroom, intentional design to reduce content and student workload, enhanced communication among faculty, and appropriate technology utilization, curriculum overload can be addressed at every level of pharmacy education.

Academic resilience in UK pharmacy education - a pilot study applying love and break up letters methodology

INTRODUCTION

Academic resilience is seen as a positive attribute that supports academic attainment and protects against attrition and burnout. Studies have reported that UK pharmacy students have lower academic resilience and wellbeing than the general UK student population but the reasons for this have not been established. This study pilots the use of a novel methodology, love and break-up letter methodology (LBM), to explore these issues focusing on the lived experience of pharmacy students.

METHOD

Final year undergraduate pharmacy study were purposely recruited to the study. Employing LBM, each participant was invited to write reflective love and break-up letters to their academic resilience in higher education during a focus group. Letters and transcripts of subsequent focus group discussion on the feelings and ideas expressed in the letters were thematically analysed.

RESULTS

Three meta-themes were identified within the data; the curriculum as gas lighting; the curriculum as abusive; and the curriculum as controlling. Students described how the curriculum diminishes academic resilience by working against their sense of agency and self-esteem. A constant threat of failure emerged as defining the student lived experience; students felt controlled by a curriculum with negative impacts on wellbeing and perseverance.

DISCUSSION

This is the first study to use LBM to explore academic resilience in UK pharmacy students. The results provide evidence that some students view the pharmacy curriculum as a source of relentless adversity that is responsible for promoting a hidden negative connection between students and their education. Further study is required to determine if the results can be generalised across the UK pharmacy student body to explain why UK pharmacy students have lower academic resilience than other UK university students and the steps needed to improve academic resilience in UK pharmacy students.

Community pharmacies and the empowerment of self-care in the United States

Background

As part of self-care, patients prevent diseases or conditions, maintain, or enhance their current health status, and address current health challenges. As a key member of the healthcare team, the role of the pharmacist in relation to self-care is important, given the ease of access to pharmacists in a variety of care settings.

Objectives

To describe the structure and function of self-care within community pharmacies throughout the United States (U.S.).

Methods

The literature within the U.S. was reviewed related to self-care definitions and practice, the role and training of the pharmacist, and challenges and opportunities for pharmacists.

Results

Within the U.S., self-care is broader than simply products found in the "over-the-counter" (OTC) section of pharmacies; it includes pharmacists involvement in disease prevention, evaluation and maintenance of current health status, and assistance in managing health challenges. There is growing recognition of the significance of pharmacists in aiding patients in self-care, due to publication of outcomes related to pharmacist-provided self-care support. Accreditation standards require student pharmacists to develop knowledge and competency related to the broadest definition of self-care; resources exist related to curricular content and student evaluation and assessment support. The evolving role of the pharmacist in relation to self-care has presented challenges in terms of recognition and renumeration for services and workload issues. Efforts are currently underway to address these challenges.

Conclusions

Spiraling health care costs, poor health outcomes, and continued health disparities indicate a need to better support U.S. patients on their health care journey, which often begins with self-care. There is a plethora of opportunities for pharmacists to advocate for expanding roles related to self-care, including participation in national efforts to recognize the outcomes of pharmacists in self-care.

A consortium's approach to developing an EPA-based community IPPE curriculum

INTRODUCTION

Currently, there are limited data on the use of entrustable professional activities (EPAs) within introductory pharmacy practice experiences (IPPEs). The objective of this study was to identify supporting EPA tasks community IPPE students should perform at the "Competent with Support" level to prepare them for advanced pharmacy practice experiences (APPEs).

METHODS

The Southeastern Pharmacy Experiential Education Consortium utilized a modified Delphi process to incorporate EPAs into community IPPE curricula to mirror the consortium's community APPE curricula. Community IPPE and APPE preceptors (N = 140) were invited to participate in focus groups and two surveys to identify and build consensus on EPA-based activities community IPPE students should perform to prepare them for APPEs. The primary outcome was development of an EPA-based community IPPE curriculum.

RESULTS

Nine preceptors (6.43%) participated in a focus group, 34 preceptors completed survey one (24.29%), and 20 preceptors completed survey two (14.29%). The initial list of 62 tasks for 14 EPAs was tailored to reflect an IPPE student skill set. Survey consensus led to a community IPPE curricula with 12 required EPAs and 54 tasks (40 required and 14 suggested).

CONCLUSIONS

The modified Delphi process provided a mechanism for preceptor collaboration with experiential programs to build consensus on community IPPE curricula redesigned around EPAs and supporting tasks. A unified IPPE curriculum adds value to colleges and schools of pharmacy with shared preceptors by improving continuity of experience, expectations, and evaluation of student learners and allows for targeted regional preceptor development.

Advancing Systems Citizenship in Colleges and Schools of Pharmacy

The pharmacy academy works collectively to serve the educational needs of diverse stakeholders by promulgating expectations for professional programs to achieve standards for both practice and professional development. Building systems thinking into the learning process, with its associative benefits to postgraduate preparation and lifelong practice, offers a pathway to achieve this educational mission. The concept of systems citizenship has been suggested as a process for helping health professional students incorporate a meaningful professional identity and responsibly seek out an understanding of the connections between patients, communities, and the larger institutions and environments that affect each. Drawing on the discipline of systems thinking, the student and pharmacist learn to be effective locally by thinking globally. Systems thinking, a basis for effective citizenship, is a proactive and shared approach to problem-solving that integrates professional identity with the goal of closing gaps in care. Pharmacy colleges/schools provide an opportune forum for educating professional students and postgraduates with the knowledge, skills, and abilities critical to becoming valuable and contributing systems citizens.

Which Entrustment-Supervision Scale is Right for Pharmacy Education?

Entrustable professional activities (EPAs) are an outgrowth of the competency-based educational model to support workplace-based learner assessments and evaluation. A learner's performance of EPAs is assessed by the degree of provided entrustment and required supervision rather than by a score, percentage, or letter grade typically assigned in traditional academic coursework. Entrustment-supervision (ES) scales are used to document learner progression and steer learner development over time. The purpose of this article is to critique various ES tools in health professions education for utilization within an EPA framework for learner assessment in workplace-based settings and to determine which will best suit pharmacy education. Exploring the advantages and disadvantages across all types of ES scales is a critical step in determining the most useful ES tool for use within a specific pharmacy institution and across the Academy. An ES scale with the traditional 5 levels, a prospective assessment frame, and increased stratification at lower levels should be recommended by the Academy and utilized in workplace-based settings for formative and summative assessment to provide more valid assessment of learners, support the ideal of life-long learning, and give more meaning for pharmacy faculty and learners within assessment.

Qualitative analysis of advanced pharmacy practice experience goals

INTRODUCTION

This study identified advanced pharmacy practice experience (APPE) activities students wish to pursue via a qualitative analysis of goal setting. It also aimed to understand student goal setting alignment with core Entrustable Professional Activities (EPAs).

METHODS

SMART (specific, measurable, achievable, realistic/relevant, time-bound) goal data were reviewed, analyzed, and coded to develop an inductive thematic analysis. Overall frequency of SMART goal coding themes was analyzed using descriptive statistics, along with frequency of themes based on rotation type. Relative percentages of each rotation type were compared. A crosswalk between themes and EPA was created post-coding.

RESULTS

Several themes among student SMART APPE goals from one complete academic year were identified (n = 79). A total of 1690 SMART goals were categorized into 22 themes. The top five most frequently utilized themes represent 60.6% of all goals, and included "Assess, Select, Recommend Therapy" (19%); "Patient Communication" (15.4%); "Foundational Knowledge" (12.4%); "Interprofessional Collaboration" (7.9%); and "Workflow, Roles and Responsibilities" (6%). The core EPA Patient Care Provider Domain was most common and accounted for 46.4% of SMART goals. The remainder included Information Master (18.8%); Practice Manager (16.1%); Interprofessional Education Team Member (7.9%); Population Health Promoter (6.6%); and Self-Developer (3.6%).

CONCLUSIONS

Our analysis revealed that students set goals in alignment with the profession's primary focus of providing direct patient care however some EPAs were infrequently citied by students in their goal setting process which may mean there is less focus on these important skills.

A Review of Entrustable Professional Activities in Pharmacy Education

Objective. To review the published literature describing how entrustable professional activities (EPAs) are currently used in health professions education with a focus on pharmacy education.Findings. English-language searches in the databases CINAHL Plus and MEDLINE were conducted for January 2011 through March 2021, which identified 21 publications for inclusion in this review. The following themes were identified: frameworks for the development of EPAs, the implementation and evaluation of EPAs for workplace learning, and gaps in knowledge and future directions for EPAs.Summary. Currently, no standardized approach exists for developing EPA frameworks for health disciplines. Also, the ways that EPAs are implemented and evaluated as an educational tool in practice settings are inconsistent. An opportunity exists to further establish the development, implementation, and evaluation of EPAs in all clinical practice settings. Much can be learned from other health professions to ensure successful translation of EPAs into pharmacy education.

Improving Knowledge of Top 200 Medications Through Retrieval Practice, Content Alignment, and Autonomous Learning

Objective. To determine the impact of the holistic redesign of top 200 medications learning activities within a Doctor of Pharmacy (PharmD) curriculum by comparing student performances on a comprehensive examination before and after the redesign.Methods. During a curricular revision at The Ohio State University College of Pharmacy that began with the class of 2020, learning activities involving the top 200 medications were implemented that involved repeated retrieval and mastery concepts, alignment with therapeutic coursework, and autonomous learning regarding the top 200 medications. A high-stakes comprehensive top 200 medications examination was administered to students at the end of their third professional year both before and after implementation of these activities. The difference in the percentage of students who achieved a satisfactory score on the comprehensive examination was compared between cohorts prior to and following the curricular redesign.Results. The study analyzed results from 134, 130, and 120 students from three PharmD classes (one before and two after the redesign of top 200 medications activities). Following the redesign, a higher percentage of students achieved a satisfactory score of 85% on the examination (class of 2020: 116/130, 89.2%; class of 2022: 107/120, 89.2%) compared to before the redesign (class of 2019: 88/134, 65.7%).Conclusion. The combination of repeated retrieval and mastery, alignment with therapeutic coursework, and development of autonomous learning can significantly increase student knowledge and retention of top 200 medications.

The Need for Quality Assessment of Entrustable Professional Activities in Pharmacy Education

Entrustable professional activities (EPAs) are specific tasks that a professional is entrusted to perform autonomously and, together, they define the scope of a profession. There are specific attributes that a task must possess both structurally and conceptually to be classified as an EPA. A high-quality EPA must be an observable, measurable, and professional task that requires training to execute, is fit for entrustment, and can be performed independently. In 2017, the American Association of Colleges of Pharmacy (AACP) defined 15 core EPAs expected of a pharmacy learner upon graduation (Core EPAs). Despite acceptance and implementation by pharmacy schools across the country, the Core EPAs have not been evaluated using an objective assessment tool to ensure that they meet specific EPA quality standards outlined in the literature. This article describes existing objective assessment tools for EPA quality and highlights the importance of ensuring high-quality EPAs in pharmacy education, which would be an important step for the AACP Academic Affairs Committee to take to further develop EPAs for implementation.

An Online, Multi-institutional Debate on Diabetes Medications by Advanced Pharmacy Practice Experience Students

Objective. The purpose of this study was to evaluate changes in advanced pharmacy practice experience (APPE) students' knowledge and to measure student perceptions of an online, multi-institutional debate curriculum as an alternative to a journal club to improve critical thinking skills related to diabetes medications.Methods. All APPE students assigned to four faculty (n=37) at three different colleges during the 2020-2021 academic year received instruction on diabetes medication classes and their cardiovascular outcome trials. Students debated via the Lincoln-Douglas format whether the preferred second-line therapies for patients with type 2 diabetes mellitus are either glucagon-like peptide-1 receptor agonists or sodium-glucose cotransporter-2 inhibitors. Matched pre- and post-APPE knowledge scores were measured using a seven-item assessment tool. A 22-item post-debate survey measured student perceptions of the activity.Results. Pre- and post-APPE knowledge scores were compared in 32 students, yielding an 86% response rate. Knowledge scores improved 32% (59% pre vs 87% post). Thirty-three students completed the perceptions survey, yielding an 89% response rate. Students reported that the debate activity was beneficial (100%) and rated it more effective than a journal club at improving critical thinking skills and knowledge retention.Conclusion. Preliminary results suggest that incorporating an online, multi-institutional debate as an alternative to journal clubs during APPE rotations was well received. Further research is warranted on the impact of the multi-institutional debate and how to best deliver it during the APPEs of a pharmacy curriculum.

Relationship between advanced pharmacy practice experience grading schemes and pharmacy residency match rates

DISCLAIMER

In an effort to expedite the publication of articles, AJHP is posting manuscripts online as soon as possible after acceptance. Accepted manuscripts have been peer-reviewed and copyedited, but are posted online before technical formatting and author proofing. These manuscripts are not the final version of record and will be replaced with the final article (formatted per AJHP style and proofed by the authors) at a later time.

PURPOSE

To determine the relationship of advanced pharmacy practice experience (APPE) grading schemes and other pharmacy program variables (ie, program age and funding) with pharmacy residency match rates.

SUMMARY

A 12-question survey was disseminated to experiential administrators of pharmacy programs in October 2018. Respondents identified their program's APPE grading scheme (pass/fail, letter grades, or other) and associated pros and cons. Responding programs were categorized by age and funding status. Survey responses were correlated with the American Society of Health-System Pharmacists residency match rates for 2016 through 2018. Data were analyzed using descriptive statistics and logistic regression models as well as by attributes via thematic analysis. Most pharmacy programs (62%) reported using letter grades for APPEs compared to pass/fail (30%) or other (8%) schemes. Pharmacy programs using pass/fail grading were more likely to have students match to postgraduate year 1 (PGY1) (P < 0.001) and postgraduate year 2 (PGY2) (P = 0.0074) residencies. Older pharmacy programs for each grading scheme were more likely to have higher match rates; however, for PGY1 match rates, older programs utilizing letter grades correlated to lower match rates than those utilizing pass/fail grading (P < 0.0001). Likewise, both public and private pharmacy programs using pass/fail grading had higher PGY1 match rates than those using letter grades (P = 0.0006 and P = 0.0014).

CONCLUSION

Pass/fail grading in APPEs does not hinder PGY1 or PGY2 residency placement compared to other grading schemes both overall and in combination with certain pharmacy program variables. Grading scheme strengths and weaknesses should be considered when deciding on optimal assessment strategies for APPEs and when evaluating candidates for residencies.

O'Mara and Lemon reply

In an effort to expedite the publication of articles, AJHP is posting manuscripts online as soon as possible after acceptance. Accepted manuscripts have been peer-reviewed and copyedited, but are posted online before technical formatting and author proofing. These manuscripts are not the final version of record and will be replaced with the final article (formatted per AJHP style and proofed by the authors) at a later time.

Evaluating a Skills Laboratory Curriculum to Determine the Essential Skills for Pharmacy Students to Learn

The profession of pharmacy is complex and diverse, encompassing a wide range of necessary practice skills. Decisions regarding which skills are essential for students are made by faculty of skills-based laboratory courses, who also determine how these skills are taught and assessed. This commentary aims to further the discussion regarding how the essential skills taught and assessed in skills laboratory curricula are identified, with the intention of preparing student pharmacists for both current and future pharmacy practice. Guidance on essential skills is provided by various organizations and documents, but ultimately each institution has the autonomy to decide what to teach. As such, this piece discusses the importance of frequent curriculum evaluation to determine how to train pharmacists of the future so they will practice at the top of their license rather than simply meet competency with historical skills.

Applying Entrustable professional activities to prescription verification activities in two skills-based laboratory courses

BACKGROUND AND PURPOSE

Prescription verification is a practice-ready expectation for pharmacy graduates. Entrustable professional activities (EPAs) should be applied to practice-ready skills-based assessments. This manuscript describes the technique of two different institutions in assessing prescription verification aligned to the Practice Manager domain of the Core EPAs as defined by the Academic Affairs Standing Committee 2015-2016 report.

EDUCATIONAL ACTIVITY AND SETTING

Virginia Commonwealth University School of Pharmacy (VCU) and University of Maryland Eastern Shore (UMES) School of Pharmacy and Health Professions describe their two methods of evaluating prescription verification with the EPA Practice Manager domain. Each program performed the activities in first-year skills-based laboratory courses.

FINDINGS

Fulfillment of a medication order was framed into law, medication label, and dispensing accuracy. Both institutions' assessments were high-stakes assessments that included errors. Overall, the majority of both programs' students passed with the institution-specified level of entrustment on their first attempt, with 75.9% to 77.5% of VCU students per each assessment and 74.5% for UMES. The lowest performance, Level 1, assessment scores for the first attempt were 10% for VCU and 2% for UMES, requiring repetition of the assessment for those students. All students achieved a minimum of Level 3 during the reassessments.

SUMMARY

Different assessment techniques may arise from program design and institutional resources. However, it is important to ensure EPAs are met for all students in prescription verification. Faculty should consider their own verification activities and level of trust expected for students to meet prior to pharmacy practice experiences.

Embarking as "Captain of the Ship" for the Curriculum Committee

This commentary examines the curriculum chair's responsibilities and discusses considerations when assuming this role, using a "captain of the ship" metaphor. From knowing the crew to managing a diverse set of responsibilities, the path to becoming an effective chair is challenging and each captain's stripe must be earned. Advice is provided to assist with understanding the curriculum and governance processes, as well as the chair's various roles and professional development. The need for leadership and management is also emphasized.

Addressing practical challenges with entrustable professional activities in ambulatory care experiential education

BACKGROUND AND PURPOSE

Entrustable professional activities (EPAs) have been adopted and endorsed by the American Association of Colleges of Pharmacy as an assessment strategy. Application of EPAs in ambulatory care advanced pharmacy practice experiences (APPEs) has demonstrated potential internal grading consistency challenges and warrants further guidance.

EDUCATIONAL ACTIVITY AND SETTING

Within a required ambulatory care APPE, a rubric was developed using purposeful and convenience sampling of preceptors, faculty, and students to concretely delineate expected EPA level progression from week one to week four based on setting specific activities.

FINDINGS

The rubric was perceived to be accurate and useful as a tool for improving expectations of progression in the respective roles on ambulatory care APPEs.

SUMMARY

Specifying performance of common clinical activities by weekly EPA level progression within an ambulatory care APPE was perceived to address some of the challenges in APPE grading consistency. Research for determining appropriate progression within an APPE and progression context within graduate and post-graduate training is needed. As more pharmacy training programs implement EPAs, collaboration in applying core EPAs into experiential education will be needed. This example of a setting specific progression rubric may serve as a useful tool for preceptors and students during their APPEs.

Implementing interactive hiring activity in pharmacy management education

BACKGROUND AND PURPOSE

A major element of management is the process of hiring employees. The purpose of this brief is to describe the implementation of a human resources learning activity asking students to choose an employee to hire, describe student responses, and explore student perspectives of the activity.

EDUCATIONAL ACTIVITY AND SETTING

An interactive hiring activity was presented to third-year students (n = 386) at two different institutions, with varied classroom settings. Students were provided with candidates' curricula vitae and then completed a student-led interview using scripted questions presented in a PowerPoint format. The activity asked students to hire one candidate from five qualified candidates of varying backgrounds and traits. A Qualtrics survey collected students' individual hiring choice, hiring rationale, participant demographic information, and activity perceptions. Demographics and candidate choice were assessed using descriptive analysis. Open-ended answers were analyzed for frequency of terms, followed by a latent class analysis.

FINDINGS

Over 90% of students (n = 386) completed the post-activity survey. Demographic data between student cohorts varied, with only student age similar. Hired candidates differed by institution with one college expressing preference for a specific candidate and the other demonstrating a greater diversity of candidate preferences. Influencing factors on student choice were numerous. Overall, students reported a favorable experience with the activity.

SUMMARY

This management hiring activity allowed students to engage in the practice-ready skills of hiring pharmacy technicians in an interactive classroom activity. This activity allowed the educators involved to develop hypotheses about student preferences in hiring for future study.

Development of an Entrustment-Supervision Assessment Tool for Pharmacy Experiential Education Using Stakeholder Focus Groups

Objective To devise a pharmacy-specific, expanded entrustable professional activities (EPA) entrustment-supervision scale that would frame preceptor ratings in a prospective and retrospective manner for use in experiential learning settings.Methods A series of focus group sessions were conducted to solicit expert opinion on how to develop and refine two entrustment-supervision scales. Purposive sampling was used to identify experts from different professional groups (physicians, pharmacy experiential administrators, and pharmacy practice faculty) who had extensive knowledge regarding EPAs and at least one publication related to EPAs. Panelists were invited to participate via email. Three focus sessions were conducted via videoconferencing between June and September 2019. The primary outcome was development of a pharmacy-specific EPA entrustment-supervision assessment tool. Secondary outcomes were individual entrustment-supervision statements across five levels of the entrustment-supervision scale.Results The focus group consisted of four pharmacy practice faculty, two experiential administrators, and one academic physician. Four concepts emerged from the focus group discussion: need for more granularity in entrustment-supervision scales; limitations due to differences in licensure requirements across the United States; present and ongoing use of expanded scales by schools and colleges of pharmacy; and uncertainty regarding how to rate entrustment-supervision when a student exhibits unprofessional behavior.Conclusion A pharmacy-specific, expanded EPA entrustment-supervision scale will be useful to support longitudinal assessment of learners in experiential settings where an EPA framework is utilized. Determining when to use a prospective versus retrospective perspective requires further evaluation.

Creating a National Standard for Prelicensure Clinical Evaluation in Nursing

BACKGROUND

Measuring clinical outcomes in prelicensure students is desired by key stakeholders but is fraught with challenges. There are currently no standardized, psychometrically validated clinical-outcome measures available for prelicensure nursing programs, requiring each program to design its own measures.

METHOD

We conducted a review of the potential antecedents of this gap, as well as models for standardized clinical outcome measures and recommendations from within health-professions education.

RESULTS

There are benefits to pursuing a standardized clinical outcome tool, including an improved student learning experience and unifying our communication regarding graduate nurse preparation to valued stakeholders.

CONCLUSION

Nursing education has a unique opportunity to emulate our colleagues in medicine and pharmacy by working at the national level to create a standardized tool using current psychometric methods for development and validation. [J Nurs Educ. 2021;60(12):686-689.].

Development of the Proposed 13 Entrustable Professional Activities for Physician Assistant Graduates

ABSTRACT

Competency-based training has not consistently produced medical professionals who are ready to practice. To address this situation, entrustable professional activities (EPAs) have been introduced as a means to assess medical trainee readiness for their next phase of training or practice. Thirteen EPAs were developed by the Physician Assistant Education Association (PAEA) Presidents Commission in 2016 to indicate what activities they thought future physician assistant (PA) students should be expected to perform at the completion of their PA education. The 13 EPAs as proposed by the PAEA Presidents Commission were adopted by the Mayo Clinic PA program and further developed through multiple steps, including (1) Discovery and Alignment; (2) EPA Development; (3) Curriculum Development; (4) Assessment and Rubric Development, and (5) Implementation.

The recommended description of an entrustable professional activity: AMEE Guide No. 140

Entrustable professional activities (EPAs) have received much attention in the literature since they were first proposed in 2005. Useful guidelines, workshops, courses, and conferences have supported faculty in developing programs and designing assessment procedures using EPAs and entrustment decision-making. Yet, the need for clarification remains, particularly as more programs make the step from design to implementation.Well-written EPAs provide a natural construct to establish the outcome of training. To be useful, EPAs require more than a suitable title. This AMEE Guide elaborates eight sections of a full EPA description, and provides explanations and justifications for each. These sections are: title; specification and limitations; risks in case of failure; most relevant competency domains; knowledge, skills, attitudes and experiences; information sources to assess progress and support summative entrustment; entrustment/supervision level expected at which stage of training; and time period to expiration if not practiced.Constructing fully elaborated EPAs creates a shared mental model amongst learners and programs, informs competency-based curriculum design, directs ad-hoc and formal entrustment decision-making, and provides standards for certifying bodies and boundaries for scope of practice. The framework intends to support curricular leaders looking to adopt new EPAs, or revise and define established EPAs for competency-based education.

Development of a Vertically Integrated Pharmacy Degree

Whilst curriculum revision is commonplace, whole degree transformation is less so. In this paper we discuss the rationale, design and implementation of a unique pharmacy program by a research-intensive faculty. The new Monash pharmacy curriculum, which had its first intake in 2017, was built using a range of key innovations that aimed to produce graduates that demonstrate key conceptual understanding and all the skills required to deliver world-best patient outcomes. The key elements of the re-design are outlined and include the process and principles developed, as well as key features such as a student-centred individualised program of development arranged around specific, authentic tasks for each skill and earlier enhanced experiential placements where students become proficient in entrustable professional activities. It is hoped the dissemination of this process, as well as the lessons learnt in the process, will be useful to others looking to transform a health curriculum.

A Scoping Review of the Pharmacy Curriculum Outcomes Assessment Literature

Objective. To identify themes, gaps, and sources of evidence from the literature regarding the Pharmacy Curriculum Outcomes Assessment (PCOA) to inform practice and additional areas for research in pharmacy education.Findings. Nineteen articles describing the administration and use of PCOA were identified. Since PCOA was made a curricular requirement by the Accreditation Council of Pharmacy Education in 2016, the focus of literature related to the PCOA has shifted from administration practices (four articles published before 2016 vs two articles published since) to determining models that may predict student performance on the assessment (two vs five articles) or how the examination might be used to predict future performance (one vs seven articles), especially on the North American Pharmacist Licensing Examination. While there is a growing body of literature focused on the PCOA's utility for measuring performance, few variables have been consistently used.Summary. This review found no studies with objectives that aligned with the initial intended use of the PCOA as defined by the National Association of Boards of Pharmacy, which included tracking individual student performance throughout the curriculum, benchmarking programs against other programs, and evaluating whether a program was meeting their desired outcomes. Additionally, no consensus across the Academy was found as to the proper use of the PCOA, and a paucity of literature exists regarding how the PCOA informs schools and colleges about the effectiveness of their curriculum. There is a need for the Academy to establish a uniform application for the PCOA in pharmacy schools, assess the resources that programs need to administer this required assessment, and determine the utility of the PCOA to measure curricular effectiveness and/or student performance.

Educating Pharmacy Students About Underserved Populations Using Patient Speakers and Simulation Activities

Objective. To assess the impact of the Patient Voices series on Doctor of Pharmacy (PharmD) students.Methods. A series of patient speakers and integrated simulation activities focused on underserved populations, otherwise known as the Patient Voices series, was embedded into a pharmacy skills laboratory curriculum. First-year PharmD students' self-ratings of confidence were compared on pre- and post-course surveys. Using evaluations from first-year introductory pharmacy practice experiences (IPPEs), student self-evaluation data were compared to preceptor evaluations of student performance. Open-ended responses to course evaluations from first- and second-year PharmD students and student reflections from third-year PharmD students were assessed using conventional content analysis to identify and characterize student perceptions.Results. Significant increases were observed in first-year students' confidence to show empathy (mean, 4.2 to 4.7) and to interact with patients from underserved communities (mean, 2.2 to 4.2). Preceptor ratings of students' empathy were consistent with the students' self-rated abilities, while students' self-ratings on cultural sensitivity were higher than the preceptors' ratings. Qualitative analysis of course evaluation surveys and reflections revealed common themes identified by students, such as understanding different perspectives, increased empathy for patients, and the value of including this content in the curriculum.Conclusion. Student confidence to interact with patients from a variety of underserved populations increased following introduction of the Patient Voices series into the PharmD curriculum. Students perceived the series to be a valuable learning experience.

Exploring Multiple Perspectives on Pharmacy Students' Readiness for Advanced Pharmacy Practice Experiences

Objective. This study aimed to enrich the Academy's understanding of pharmacy students' readiness for advanced pharmacy practice experiences (APPEs) by exploring the perspectives of three primary stakeholders: APPE students, APPE preceptors, and APPE faculty site directors.Methods. A descriptive qualitative study of APPE readiness was conducted using workplace learning as a guiding conceptual framework. Data were collected between March and September 2019 through semi-structured focus groups and interviews with students (five groups), preceptors (four groups), and faculty site directors (one group, two individual interviews). The data were analyzed using directed content analysis.Results. Participants described APPE readiness as a multifaceted construct comprised of four themes: learner characteristics, participation in workplace activities, relationship-building, and workplace practices to orient and support students. While all participants addressed each category, faculty site directors and preceptors tended to focus on learner characteristics, while students emphasized their participation in the workplace and relationship building.Conclusion. Knowledge is widely recognized as a requirement for APPE readiness. This study identified learner characteristics, workplace participation, and relational skills as additional requirements. Some of these criteria are challenging to assess prior to APPEs, which makes orienting students both prior to and at the start of APPEs particularly important to support readiness. Thus, a comprehensive review of APPE readiness might also include assessing the readiness of workplaces, administrators, and preceptors for APPE students.

Entrustability levels of general internal medicine residents

BACKGROUND

Entrustable professional activities (EPAs) are those activities that a health professional can perform without direct supervision in a defined environment. Bridging the gap between competencies and learning objectives, EPAs have made assessing the performances of health professional more realistic. The main objective of the present study was developing and customizing EPAs for Iranian Internal Medicine Residency Programs.

RESULTS

After reviewing the publications, residency curricula and logbooks, and collecting experts' ideas, the initial list of EPAs was developed. Then, in a focus group, the list was refined, the entrustability level of each residency year was determined, and finally, the EPA-competency cross-tab was established, and in the next step, through a one- round Delphi, the results were validated. Twenty-eight EPAs were developed. Some of them were definitely suitable for the higher levels of residency, such that they had to be accomplished under direct supervision until the end of the program. On the other hand, some of EPAs were those that residents, even from the first year, are expected to perform independently or under indirect supervision. Most of the EPAs cover a wide range of competencies.

CONCLUSION

Determining the entrustability level of each residency year in each EPA as well as the competency- EPA matrix has crucial effect on the quality of the graduates. It seems that our findings are applicable in developing countries like Iran.

Validity Evidence for a Pharmacists' Patient Care Process Self-Efficacy Scale Among Pharmacy Students

Objective. To further refine and examine the validity of an instrument for assessing pharmacy students' self-efficacy in implementing the Pharmacists' Patient Care Process (PPCP) components.Methods. An instrument was developed and pilot tested in spring 2018 at one college of pharmacy. In spring 2019, a modified version of the instrument, the PPCP Self-Efficacy Scale (PPCP-SES), was administered to third professional year (P3) pharmacy students at seven institutions. Self-efficacy items were based on Bandura's theory of self-efficacy, and students were asked to rate each item on a continuous scale (0-100). Data analyses included descriptive statistics, reliability analysis, and confirmatory factor analysis (CFA).Results. Three hundred P3 students completed the PPCP-SES. The domain-specific Cronbach alpha coefficients were: collect, α=.89; assess, α=.92; plan, α=.95; implement, α=.96; and follow-up, α=.95. Based on the factor analysis results, three items were removed. Model fit statistics indicated the overall instrument had moderate goodness of fit.Conclusion. Results indicate the PPCP-SES demonstrated initial evidence of validity for use by pharmacy faculty members to identify students' self-efficacy related to implementing components of the PPCP. Future research is needed to examine validity evidence in other student populations and among practicing pharmacists.

A Six-Semester Integrated Pharmacy Practice Course Based on Entrustable Professional Activities

Objective. To design and assess the outcomes of a longitudinal, integrated pharmacy course.Methods. A one-credit course, Applications of Pharmacy Practice, was built into a revised curriculum. The course was offered during the first six semesters of the Doctor of Pharmacy curriculum and met once every three weeks for a total of five sessions per semester. Each session integrated curricular material from all courses taught during a given semester into an individual, case-based assessment and a team activity. Team activities were evaluated using rubrics developed based on the Core Entrustable Professional Activities for New Pharmacy Graduates.Results. First-year students were automatically enrolled in the inaugural offerings of the course in fall 2018 and spring 2019, and the majority of students achieved final course grades of 70% to 89% for each semester. Students reported that the course helped them to develop critical-thinking skills and to keep up with semester material. Although the majority of students believed the addition of the course to the curriculum was beneficial, some felt the expectation for them to keep up with all course material was excessive.Conclusion. The addition of a longitudinal integrated course, Applications of Pharmacy Practice, appeared to benefit students, helping them integrate material from all of their courses during the semester. This integration of clinical, administrative, and pharmaceutical sciences material reduced compartmentalization of knowledge. Future studies should investigate the impact of this course on student success and performance on standardized assessments.

Unpacking student learning from an early experience with the Pharmacists' Patient Care Process

BACKGROUND

To illuminate learning, a case study approach was used to examine early, authentic experiences within Pharmacists' Patient Care Process (PPCP)-focused practices.

EDUCATIONAL ACTIVITY

Six students were matched with five practitioners and spent five half-days in a primary care clinic in a PPCP-committed health system. Students practiced interviewing, determining the patient's medication experience, and formulating the beginnings of the assessment, as well as observing and debriefing on the completion of the process by a practitioner mentor. The Five R Model was used to prompt student learning reflection. In addition, instructors examined students' work for evidence of transformative learning and observations were captured using forms of reflective practice and collaborative debriefing.

CRITICAL ANALYSIS OF THE EDUCATIONAL ACTIVITY

Reflection performance ratings varied; however, there was strong evidence of transformative learning for all students. Specifically, most student reflections demonstrated a focus on elaborating on existing frames of reference. The most prevalent indicator of transformative learning was exploration of options for new roles, relationships, and actions. The codes from instructors' observations revealed five categories of learning evidence, with the strongest in the patient centeredness category. The process of reviewing student work products, documenting instructor observations, and collaborative debriefing resulted in insights for curricular improvement and explanations for learning difficulties. Further work is needed in understanding student experiential learning intentions and their influence on learning and reflection. Additionally, further research should explore the value of longitudinal assessment of reflection and the value of assessing student work products using criteria beyond traditional reflection criteria.

Using Best-Worst Choice Methodology in a Survey of Pharmacists Regarding Pharmacy Practice Skills Teaching

Objective. To conduct a survey of practicing pharmacists in which best-worst choice methodology was used to prioritize pharmacy practice skills for inclusion in a pharmacy curriculum in New Zealand.Methods. A literature search and review of pharmacy curricula were conducted, and the findings were used to develop a best-worst choice survey instrument regarding inclusion of pharmacy practice skills in the pharmacy curriculum. The survey was sent to registered pharmacists and intern pharmacists in New Zealand. Participants were asked to prioritize 16 skills in terms of their importance and relevance to pharmacy practice.Results. Of the 3836 pharmacists invited to participate in the survey, 388 completed the questionnaire. Comprehensive chronic disease management, specialty medications, and medicines use review were the top three prioritized skills. Injections, independent prescribing, and specialty compounding were the skills ranked as having the lowest priority. The pharmacists' gender, age, practice setting, and ethnicity all influenced their skill prioritization. The pharmacists emphasized skills required in their current practice but deemphasized some skills that were emerging professional responsibilities.Conclusion. If curricular reform is to include new skills that are largely unfamiliar to or deemed unimportant by practicing pharmacists, quality assurance of students' experiential education will be needed. Furthermore, preceptor education about changing expectations for pharmacy graduates' skill sets must be adequately developed and implemented to ensure that preceptors provide students with opportunities to practice the full range of skills they will need in practice and provide them with accurate assessment and helpful feedback.

A Curriculum Crosswalk of the Core Entrustable Professional Activities for New Pharmacy Graduates

Objective. To cross reference the core entrustable professional activities (EPAs) to a complete set of educational guidance documents for the Doctor of Pharmacy (PharmD) curriculum to create a map for pharmacy educators.Methods. The Mapping EPAs Task Force consisted of nine members who first worked independently and then together in small working groups to map five assigned educational guidance documents (eg, Center for the Advancement of Pharmacy Education [CAPE] Outcomes, Accreditation Council for Pharmacy Education [ACPE] Standards 1-4, and the Essential Elements for Core Advanced Pharmacy Practice Experiences [APPEs]) to the Core Entrustable Professional Activities for New Pharmacy Graduates. Four working groups completed the mapping process during phases 1 and 2, which was followed by an independent quality assurance review and consensus in phase 3.Results. All 15 core EPA statements were mapped to one or more of the educational documents. One item from the CAPE Outcomes could not be mapped to a core EPA statement. The first five EPA statements mapped directly to the five elements of the Pharmacists' Patient Care Process: collect, assess, plan, implement, and follow-up: monitor and evaluate.Conclusion. This comprehensive EPA map is the first curriculum crosswalk that encompasses a complete set of educational guidance documents including the Essential Elements for Core APPEs for the Doctor of Pharmacy curriculum. If adopted by the Academy, this curriculum crosswalk will provide pharmacy schools with a common interpretation of important educational guidance documents; serve as the foundation for curricular development, revision, and assessment; and ensure student pharmacists are prepared to enter the pharmacy profession.

Evaluating Practice Readiness of Advanced Pharmacy Practice Experience Students Using the Core Entrustable Professional Activities

Objective. To evaluate levels of entrustability and practice readiness in advanced pharmacy practice experience (APPE) students using a pilot instrument designed to assess their competency in performing the entrustable professional activities (EPAs) expected of new pharmacy graduates. Methods. A pilot instrument was developed directly from EPAs to measure entrustability levels on a scale of one to five. Five APPE preceptors from several different practice areas participated. Fourth-year students used the instrument to self-evaluate their knowledge and skills at the beginning, midpoint, and end of the APPE. The preceptors evaluated students using the same instrument at APPE midpoint and end. The instrument had assigned weights for each EPA and entrustability level for a score of 100 if all items were marked five. If a rating of non-applicable was chosen, score adjustments were made. All students in the graduating class of 2018 were invited to perform a self-evaluation at the end of the fourth (APPE) year using the same instrument that study participants used. Results. Twenty-eight students and five preceptors completed evaluations during the APPE year. Overall scores from both preceptor evaluations of students and student self-evaluations increased significantly from pre-APPE to midpoint to final. Student self-evaluations were only slightly higher than preceptor evaluations. The mean (SD) preceptor scores for students and student self-assessment scores at the end of each APPE were 85.4% (7.1) and 87.2% (10.3), respectfully. One practice manager EPA and three population health EPAs were considered to not be applicable by preceptors on ≥50% of evaluations. Approximately 94% of all graduating students completed the year-end self-evaluations, with a mean (SD) score of 89% (8.6) and no EPAs marked as not applicable. Conclusion. Pharmacy students' proficiency in EPA improved during individual APPEs. According to preceptors, students' greatest improvement in entrustability was in educating patients and colleagues regarding appropriate use of medications and collecting information to identify medication-related problems.

Developing Physical Assessment Skills in Pharmacy Students through Participation in a Creative Movement Workshop: An Interdisciplinary Study between Pharmacy and Dance

The role a pharmacist plays in the care of patients is continually changing and expanding. Most recently, there is movement towards including pharmacists in the physical assessment of patients. We developed a creative movement workshop with the purpose of increasing students' levels of comfort with touch, ability to interpret non-verbal mannerisms, to increase empathy for the patient, and to increase student comfort in conducting physical exams. In this interventional study, surveys were administered to third year pharmacy students, before and after the creative movement workshop, in order to assess participant's change in level of comfort with a variety of behaviors needed to conduct effective physical assessment. The two hour workshop involved: partner stretching, mirroring, and creative spatial exploration between bodies. The 11-item survey evaluated students' perceptions on touch, nonverbal communication, and sharing personal space. Our results showed that the level of comfort improved for the ability to give touch (p = 0.001), the ability to receive touch (p = 0.002), and the ability to share personal space (p = 0.001). Participants commented that the workshop increased their understanding of how important confidence is when performing physical assessment and reported an increased appreciation for how much non-verbal mannerisms can communicate to another. This study explores how an interdisciplinary workshop between pharmacy and dance has the potential to increase student effectiveness as future healthcare professionals, by targeting skills not often focused on within traditional pharmacy curriculums, including: sharing personal space, displaying empathy, and providing a comforting and confident touch.