Identifying criteria for the assessment of pharmacy students' communication skills with patients

Analysis of pharmacy student communication self-evaluation skills during standardized patient encounters

INTRODUCTION

The objective of this study was to evaluate pharmacy students' communication self-evaluation skills by comparing student self-evaluations with those completed by course graders and standardized patients (SP).

METHODS

As part of a required communications course, third-year pharmacy students completed a medication counseling encounter with a SP during a midpoint encounter and final assessment. Students' communication skills were evaluated by course graders and SPs. Students used the same assessment rubric to also complete self-evaluations immediately after each event and after reviewing a recording of their midpoint encounter. Agreement among student, SP, and course grader ratings on individual items were examined using the kappa statistic.

RESULTS

A total of 206 students completed the midpoint encounter, and 208 completed the final assessment. Agreement between students' and SPs' evaluations was high during both the midpoint and final encounters, with >90% agreement on 14 out of 17 items at the midpoint encounter and 16 out of 17 during the final assessment. There were diffiences in scoring for the use of verbal distractors/fillers, with SPs' evaluations showing more favorable scores when differences existed [at midpoint 69 (33.7%) SP vs. 7 (3.4%) student; at final 31 (15%) SP vs. 3 (1.5%) student]. Agreement improved from the midpoint to final assessment (62.9% vs. 83.7%, respectively).

CONCLUSIONS

Self-evaluation of communication skills may be an acceptable alternative to faculty or SP evaluations, when appropriate for the purpose of the activity, based on the high agreement observed among communication skills evaluations completed by students, course graders, and SPs.

A comparison of medication counseling evaluation tools utilized in colleges of pharmacy to three recognized guidance documents

INTRODUCTION

A universal tool to assess proficiency in patient counseling has not been developed. The objectives of this project were to assess current practices in evaluating patient counseling at colleges of pharmacy and compare the results with three nationally recognized reference standards: the Omnibus Budget Reconciliation Act of 1990, the American Pharmacists Association Academy of Student Pharmacists National Patient Counseling Competition Evaluation Form, and the American Society of Health-System Pharmacists Guidelines on Pharmacist Conducted Patient Education and Counseling.

METHODS

A cover letter and questionnaire were sent to all members of the American Association of Colleges of Pharmacy Laboratory Instructors Special Interest Group with an invitation to submit an evaluation tool. Descriptive statistics were used to examine the questionnaire items. An inductive approach was used to analyze the evaluation tools, which was completed independently by two members and then compared to build consensus to establish common content categories.

RESULTS

Five major categories emerged with variability in content. More than 75% of tools included an introduction, almost 59% included collecting information, most included drug information (94%) and counseling points (88%), and most included verification of understanding (94%) and communication skills (94%).

CONCLUSION

Although there was variability, most evaluation tools incorporated aspects of all three guidance documents. A limitation of this study was that data may be subject to non-response bias, as the response rate was limited to 17.8%. Future work should focus on creating and validating a standardized evaluation tool to ultimately promote consistency among student pharmacists.

Striving for interrater reliability with a patient counseling assessment rubric: Lessons learned

BACKGROUND

Providing effective patient counseling is an essential pharmacist skill to ensure patients understand how to take medications, prevent medication-related errors, and meet requirements of federal law. This study sought to develop a new patient counseling assessment rubric to minimize interrater variability, deliver a consistent summative competency assessment, and provide students with formative, actionable feedback.

IMPACT

A first attempt to achieve statistically significant interrater reliability was not successful due to incorporation of too many variables into study design and the subjective nature of patient counseling. After reducing study variables (number of different medications, number of evaluators, and number of videos) and consulting a statistician, a second attempt was made to analyze interrater reliability for the rubric. However, even with variables minimized, this attempt did not lead to statistically significant agreement.

RECOMMENDATIONS

The faculty team identified four recommendations (Omnibus Budget Reconciliation Act of 1990): conduct a norming session for graders prior to the assessment (Rantucci, 2006), conduct a post-hoc analysis after grading to reduce interrater variability and increase consistency (Taitel et al., 2012), simplify the rubric to reduce subjectivity and clarify the intent of rubric elements, and (Saranagam et al., 2013) rubrics can be utilized differently in separate courses to target specific learning objectives.

DISCUSSION

Although the goal of creating a rubric with statistically significant interrater reliability was not achieved, we did learn important lessons about evaluating student pharmacist performance with less subjectivity and more consistency. The authors hope the results and lessons learned will be valuable to our colleagues at other institutions as patient counseling content and rubrics are developed.

Determining Indicators of High-Quality Application Activities for Team-Based Learning

Objective. To determine the indicators of quality for application activities in pharmacy team-based learning (TBL). Methods. A modified Delphi process was conducted with pharmacy TBL experts. Twenty-three experts met the inclusion criteria, including having at least four years of TBL experience, designing at least eight TBL sessions, training others to use TBL, and authoring a peer-reviewed TBL pharmacy paper. In round 1, panelists responded to five open-ended questions about their successful TBL applications activities, including satisfaction with the activity and methods for creating positive student outcomes. In round 2, panelists indicated their level of agreement with the round 1 quality indicators using a four-point Likert rating. Consensus was set at 80% strongly agree/agree. In an open comment period, panelists provided suggestions to help expand the indicator descriptions. Indicators were verified based on TBL and the education literature. Results. Twenty panelists (87% of those eligible) responded in round 1 and 17 (85% participation) in round 2. Sixteen quality indicators were identified in round 1, with 14 achieving consensus in round 2. "Uses authentic pharmacy challenges or situations" (88% strongly agree/agree) and "incorporates or provides effective feedback to groups" (88% strongly agree/agree) met consensus. However, "has multiple right answers" (76% strongly agree/agree) and "incorporates elements from school specific emphases (eg, faith, underserved)" (53% strongly agree/agree) did not reach consensus. Conclusions. These indicators can assist faculty members in designing application activities to provide high-quality TBL exercises that promote deep thinking and engaged classroom discussion. The indicators could also guide faculty development and quality improvement efforts, such as peer review of application activities.

Current Trends and Opportunities for Competency Assessment in Pharmacy Education-A Literature Review

An increasing emphasis on health professional competency in recent times has been matched by an increased prevalence of competency-based education models. Assessments can generate information on competence, and authentic, practice-based assessment methods are critical. Assessment reform has emerged as an academic response to the demands of the pharmacy profession and the need to equip graduates with the necessary knowledge, skills and attributes to face the challenges of the modern workforce. The objective of this review was to identify and appraise the range of assessment methods used in entry-level pharmacy education and examine current trends in health professional assessment. The initial search located 2854 articles. After screening, 36 sources were included in the review, 13 primary research studies, 12 non-experimental pharmacy research papers, and 11 standards and guidelines from the grey literature. Primary research studies were critically appraised using the Medical Education Research Study Quality Instrument (MERSQI). This review identified three areas in pharmacy practice assessment which provide opportunities for expansion and improvement of assessment approaches: (1) integrated approaches to performance assessment; (2) simulation-based assessment approaches, and; (3) collection of validity evidence to support assessment decisions. Competency-based assessment shows great potential for expanded use in pharmacy, but there is a need for further research and development to ensure its appropriate and effective use.

Suggested pharmacy practice laboratory activities to align with pre-APPE domains in the Doctor of Pharmacy curriculum

OUR SITUATION

The Accreditation Council for Pharmacy Education outlines ability statements that pharmacy students should be able to demonstrate prior to beginning their Advanced Pharmacy Practice Experiences (APPEs). Practice laboratory courses offer extensive opportunities for students to participate in activities and assessments that enable them to meet the objectives outlined in the Pre-APPE Core Domains in Standards 2016. This review identifies selected published literature, activities, and assessment methods that can be adapted and implemented in practice laboratory courses to help achieve the abilities outlined within the Pre-APPE Core Domains.

METHODOLOGICAL LITERATURE REVIEW

The Medline database and journals related to pharmacy education were searched to identify activities and assessments for each domain. Search terms for each core domain were extracted from the domain titles, ability statements, and performance competencies and coupled with "laboratory" or "lab." "Pharmacy" was also added as a search term when searching the Medline database. Preference was given to example activities published in the last 15 years. Abstracts and activities based on author experience were also included.

OUR RECOMMENDATIONS AND THEIR APPLICATIONS

Specific examples of how activities and assessments can be included in practice laboratories to develop or refresh skills identified in the pre-APPE core domains were described.

POTENTIAL IMPACT

The practice laboratory setting is an ideal place for students to learn and practice the skills necessary to demonstrate readiness for APPEs. This paper serves as a resource for instructors, curriculum committees, or pharmacy programs looking for ideas to expand specific training or develop particular skill areas.

Identifying Shared Values for School-Affiliated Student Organizations

Objective. To identify shared values for student organizations. Methods. A three-round Delphi approach was utilized to identify and prioritize shared values among student organization leadership. In round 1, student leaders selected 15 values from a list of 36 organizational values and were given an opportunity to include up to five suggestions not incorporated within the original list. Student leaders narrowed the 15 values to 12 in round 2. The top 12 priorities were ranked in round 3 and participants were invited to write a brief statement regarding their perspectives of the results. Results. Twelve shared values were identified and ranked: professional development, improving leadership of your members, advancing the role of pharmacy, planning quality events, networking, improving the academic experience for peers, community service, learning from pharmacy shadowing/speakers, social outlet, recruitment/gaining student membership, attracting students to events, and gaining national/local attention or awards. Conclusion. This study contributes to the small but growing body of literature concerning student organizations in pharmacy education and provides a foundation by which this work could be advanced. Given the importance of student organizations in promoting student development, identifying strategies for supporting and facilitating the effectiveness of these groups is critical for optimizing student outcomes and institutional effectiveness.

Design and Validation of Patient-Centered Communication Tools (PaCT) to Measure Students' Communication Skills

Objective. To develop a comprehensive instrument specific to student pharmacist-patient communication skills, and to determine face, content, construct, concurrent, and predictive validity and reliability of the instrument. Methods. A multi-step approach was used to create and validate an instrument, including the use of external experts for face and content validity, students for construct validity, comparisons to other rubrics for concurrent validity, comparisons to other coursework for predictive validity, and extensive reliability and inter-rater reliability testing with trained faculty assessors. Results. Patient-centered Communication Tools (PaCT) achieved face and content validity and performed well with multiple correlation tests with significant findings for reliability testing and when compared to an alternate rubric. Conclusion. PaCT is a useful instrument for assessing student pharmacist communication skills with patients.

Enhancing Student Communication Skills Through Arabic Language Competency and Simulated Patient Assessments

Objective. To assess student communication and patient management skill with introduction of Arabic and use of simulated patient assessments to a communication and counseling course. Design. Five, 3-hour tutorials (clinical skill laboratory) were added to the course covering: listening and empathic responding, non-verbal communications, interviewing skills, assertiveness, counseling in special situations: conflict, anger, worry or rushed situations, and professional decision making. Arabic content was introduced to the course to enhance Arabic communications and competence among students. Simulated patient assessment was used to evaluate student skills. Students' feedback about course changes was evaluated. Assessment. The course now covers a wider content and Arabic language. Students' scores were similar in the assessment and other assessments within the course and between Arabic and English groups. Students favorably rated the changes in the course and provided constructive feedback on content usefulness and adequacy. Conclusion. Expanding the course to include Arabic language and content and simulated patient assessments enhanced student communication skills.

Training and Validation of Standardized Patients for Assessing Communication and Counseling Skills of Pharmacy Students: A Pilot Study

OBJECTIVE

The objective of this study is to describe the process of training valid simulated patients (SPs) for assessing communication and counseling skills of pharmacy students' performance.

METHODS

This is a cross-sectional and correlational study. Psychometric properties of checklist and SPs' portrayals and their filling of the checklist regarding assessing pharmacy students were assessed. Five SPs who were working in the simulated patient's pool were volunteered to take part in the project, which one of the SPs failed. Three scenarios, along with corresponding checklists, were developed based on the usual medications of different diseases consisting of asthma, respiratory infections, and osteoporosis. The SPs' role-play performance was video-recorded and rated independently by two experts according to an observational rating scale to assess validity. The role-play was repeated after 1 week with the same scenario and the same doctor, to assess test-retest reliability. The inter-rater agreement between SPs and experts was determined by calculating the intraclass correlation coefficient and kappa coefficient.

FINDINGS

The four eligible SPs were all women, with an average age of 37 years. The correlation between mean scores of raters and mean scores of SPs was 0.91 and 0.85, respectively. The Pearson's correlation between mean scores of raters with SPs was 0.75. The checklists' reliability, Cronbach's alpha, was calculated to be 0.72. The measured weighted Cohen's kappa for the ratings of by each SP, and the gold standard was between 0.53 and 0.57, indicating a moderate agreement. The inter-rater reliability kappa coefficient between raters was 0.75 (P = 0.01).

CONCLUSION

The authors have demonstrated the technique of using standardized patients to evaluate communication and counseling skills of pharmacy students. The findings indicated that trained SPs can be used as an effective tool to assess pharmacy students' communication and counseling skills.

How to use the nominal group and Delphi techniques

Introduction The Nominal Group Technique (NGT) and Delphi Technique are consensus methods used in research that is directed at problem-solving, idea-generation, or determining priorities. While consensus methods are commonly used in health services literature, few studies in pharmacy practice use these methods. This paper provides an overview of the NGT and Delphi technique, including the steps involved and the types of research questions best suited to each method, with examples from the pharmacy literature. Methodology The NGT entails face-to-face discussion in small groups, and provides a prompt result for researchers. The classic NGT involves four key stages: silent generation, round robin, clarification and voting (ranking). Variations have occurred in relation to generating ideas, and how 'consensus' is obtained from participants. The Delphi technique uses a multistage self-completed questionnaire with individual feedback, to determine consensus from a larger group of 'experts.' Questionnaires have been mailed, or more recently, e-mailed to participants. When to use The NGT has been used to explore consumer and stakeholder views, while the Delphi technique is commonly used to develop guidelines with health professionals. Method choice is influenced by various factors, including the research question, the perception of consensus required, and associated practicalities such as time and geography. Limitations The NGT requires participants to personally attend a meeting. This may prove difficult to organise and geography may limit attendance. The Delphi technique can take weeks or months to conclude, especially if multiple rounds are required, and may be complex for lay people to complete.

How to use the nominal group and Delphi techniques

Introduction The Nominal Group Technique (NGT) and Delphi Technique are consensus methods used in research that is directed at problem-solving, idea-generation, or determining priorities. While consensus methods are commonly used in health services literature, few studies in pharmacy practice use these methods. This paper provides an overview of the NGT and Delphi technique, including the steps involved and the types of research questions best suited to each method, with examples from the pharmacy literature. Methodology The NGT entails face-to-face discussion in small groups, and provides a prompt result for researchers. The classic NGT involves four key stages: silent generation, round robin, clarification and voting (ranking). Variations have occurred in relation to generating ideas, and how ‘consensus’ is obtained from participants. The Delphi technique uses a multistage self-completed questionnaire with individual feedback, to determine consensus from a larger group of ‘experts.’ Questionnaires have been mailed, or more recently, e-mailed to participants. When to use The NGT has been used to explore consumer and stakeholder views, while the Delphi technique is commonly used to develop guidelines with health professionals. Method choice is influenced by various factors, including the research question, the perception of consensus required, and associated practicalities such as time and geography. Limitations The NGT requires participants to personally attend a meeting. This may prove difficult to organise and geography may limit attendance. The Delphi technique can take weeks or months to conclude, especially if multiple rounds are required, and may be complex for lay people to complete.

Performance and Perceptions: Evaluation of Pharmacy Students' Consultation via Telepharmacy

Background: Pharmacy education standards highlight the importance of effective communication skills and the use of technology to provide patient care. As technology evolves, pharmacists have opportunities to communicate in different and broader ways. Objective: The objectives of this study were 3-fold: to evaluate student ability to counsel via telepharmacy, to determine if there is a difference in students' abilities to counsel face-to-face or via telepharmacy, and to determine students' perceptions regarding patient consultation via telepharmacy. Methods: Professional pharmacy students completed a pharmaceutical care laboratory activity focused on communication via telepharmacy. Comparisons were made between students' ability to provide patient consultation via telepharmacy and face-to-face utilizing a faculty-developed rubric. Students also completed a questionnaire on their perception of utilizing telepharmacy technology to provide patient consultation. Results: Eighty-two second-year professional pharmacy students participated in the study. Results showed students are able to successfully provide patient consultation via telepharmacy without prior practice; however, there was a statistically significant difference between students' ability to counsel face-to-face and via telepharmacy (P < .001). Overall, students were more successful at providing face-to-face consultation than via telepharmacy, and students who were first assessed on their ability to counsel face-to-face perceived a greater difference between telepharmacy and face-to-face consultation (P < .05). Conclusion: Student-perceived differences between the 2 means of consultation and demonstrated a difference in their ability to counsel via telepharmacy and face-to-face. It appears that, when evaluating the need to teach professional pharmacy students how to provide patient consultation via telepharmacy, additional exposure to telepharmacy technology could be beneficial by enhancing student comfort and proficiency.

Developing pharmacy student communication skills through role-playing and active learning

OBJECTIVE

To evaluate the impact on pharmacy students of a communication course, which used role-playing to develop active-learning skills.

DESIGN

Students role-playing pharmacists in patient care scenarios were critiqued by students and pharmacist faculty members. Grading was performed using the rubric inspired by Bruce Berger's Communication Skills for Pharmacists. Written skills were evaluated using student written critique questionnaires. Students completed precourse and postcourse self-assessment surveys. Preceptor evaluations were analyzed for course impact.

ASSESSMENT

Students demonstrated improvement in oral skills based on role-play scores (45.87/50) after practice sessions. The average score based on the student questionnaire was 9.31/10. Gain was demonstrated in all defined course objectives. Impact on introductory pharmacy practice experience (IPPE) communication objectives was insignificant. Student evaluations for course and teaching strategy reflected a high average.

CONCLUSION

Study results demonstrated improvement in oral and written communication skills that may help improve interprofessional teamwork between pharmacists and other health care providers.

Communications training in pharmacy education, 1995-2010

The role of the pharmacist as a "communicator" of information and advice between patients, other healthcare practitioners, and the community is recognized as a vital component of the responsibilities of a practicing pharmacist. Pharmacy education is changing to reflect this, although the difficulty is in designing a curriculum that is capable of equipping students with the necessary knowledge and skills, using activities that are effective in promoting communication competency. The objective of this review was to identify published, peer-reviewed articles concerning communication training in pharmacy education programs, and describe which communication skills the structured learning activities aimed to improve and how these learning activities were assessed. A systematic literature search was conducted and the articles found were analyzed and divided into categories based on specific communication skills taught and type of learning activity used. Oral interpersonal communication skills targeted at patients were the most common skill-type described, followed by clinical writing skills. Common teaching methods included simulated and standardized patient interactions and pharmacy practice experience courses. Most educational interventions were assessed by subjective measures. Many interventions were described as fragments, in isolation of other learning activities that took place in a course, which impedes complete analysis of study results. To succeed in communication training, integration between different learning activities and progression within pharmacy educations are important.

The influence of a patient-counseling course on the communication apprehension, outcome expectations, and self-efficacy of first-year pharmacy students

OBJECTIVES

To evaluate first-year doctor of pharmacy (PharmD) students' communication apprehension, outcome expectations, and self-efficacy for communication over the duration of a 15-week patient-counseling course.

DESIGN

First-year PharmD students (n=94) were asked to complete a 47-item, self-administered questionnaire on 3 occasions over the duration of the Nonprescription Drugs/Patient-Counseling course during the fall 2009 and 2010 semesters.

ASSESSMENT

Eighty-seven of 94 students completed the survey instrument across data collection periods. There were significant reductions in total communication apprehension scores and in the communication apprehension subscores for meetings and public speaking, and significant increases in self-efficacy over time. No differences were found for outcome expectations of communication scores or the subscores for interpersonal conversations and group discussion.

CONCLUSIONS

Communication apprehension may be decreased and self-efficacy for communication increased in first-year PharmD students through a 15-week Nonprescription Drugs/Patient-Counseling course using small-group practice sessions, case studies, and role-play exercises in conjunction with classroom lectures.

Virtual patients in pharmacy education

A review of the literature relating to the use of virtual patients in teaching pharmaceutical care to pharmacy students was conducted. Only 7 articles met the inclusion criteria for the review and 4 of the studies were conducted in North America. Few articles identified by the review used virtual patient technology that was true-to-life and/or validated.

Benchmarks for effective primary care-based nursing services for adults with depression: a Delphi study

AIM

This paper is a report of a study conducted to identify and gain consensus on appropriate benchmarks for effective primary care-based nursing services for adults with depression.

BACKGROUND

Worldwide evidence suggests that between 5% and 16% of the population have a diagnosis of depression. Most of their care and treatment takes place in primary care. In recent years, primary care nurses, including community mental health nurses, have become more involved in the identification and management of patients with depression; however, there are no appropriate benchmarks to guide, develop and support their practice.

METHOD

In 2006, a three-round electronic Delphi survey was completed by a United Kingdom multi-professional expert panel (n = 67).

FINDINGS

Round 1 generated 1216 statements relating to structures (such as training and protocols), processes (such as access and screening) and outcomes (such as patient satisfaction and treatments). Content analysis was used to collapse statements into 140 benchmarks. Seventy-three benchmarks achieved consensus during subsequent rounds. Of these, 45 (61%) were related to structures, 18 (25%) to processes and 10 (14%) to outcomes.

CONCLUSION

Multi-professional primary care staff have similar views about the appropriate benchmarks for care of adults with depression. These benchmarks could serve as a foundation for depression improvement initiatives in primary care and ongoing research into depression management by nurses.

Developing communication skills in pharmacy: a systematic review of the use of simulated patient methods

OBJECTIVE

To review the literature relating to the use of simulated patient methods to enhance communication skills of pharmacists.

METHODOLOGY

We searched Embase, Lilacs, Medline, Scielo, and Scopus databases between 1980 and 2008, using "communication skills", "patient counseling" and "pharmacist" as keywords. This search was then further refined by using "simulated patients", "pseudo-customer", "standardized patients", and "mystery shoppers" as additional keywords.

RESULTS

The initial search identified 241 published studies. Once further refined, 15 studies met inclusion criteria.

CONCLUSION

The majority of studies had an assessment focus aimed at documenting counseling behavior of practicing pharmacists, rather than an educational focus aimed at equipping pharmacists with effective communication skills. In instances where simulated patient methods were used for educational purposes, little regard was given to the role of performance and corrective feedback in shaping communication behavior of pharmacists. The majority of studies failed to describe the competencies and skills being investigated in relation to communication in the practice of pharmacy.

PRACTICE IMPLICATIONS

Simulated patient methods provide pharmacy educators with a tool for implementing communication skills in the practice of pharmacy and will serve as a basis for implementing communication skills development programs at the College of Pharmacy of the Federal University of Sergipe in Brazil.

The impact of a standardized patient program on student learning of communication skills

OBJECTIVE

Determine the quantitative and qualitative value of a lecture-laboratory course with standardized patients on student communication skills.

METHODS

A blinded retrospective analysis was conducted on the counseling tapes of 127 students who took a lecture-laboratory course with standardized patients. A Communication Skills Assessment Form (CSAF) was used to evaluate baseline, midpoint and final tapes. Descriptive statistics and repeated measures analysis of variance were used to compare tapes. Students and standardized patients completed written survey instruments evaluating the laboratory at the end of the semester.

RESULTS

Students had significant and progressively higher scores on the assessment across baseline, midpoint and final time points (p < or = 0.001). Students had significantly higher final assessment scores across all subsections than at baseline (p < or = 0.01). Students and standardized patients were favorable towards the laboratories and made useful recommendations.

CONCLUSIONS

A lecture-laboratory course with standardized patients had a significant impact on student communication skills across time and was well received by students and standardized patients.