Transitioning Competency-Based Communication Assessments to the Online Platform: Examples and Student Outcomes

Online medical student OSCE examinations during the first three years of the COVID-19 pandemic compared to three years pre-pandemic: An Australian experience in psychiatry and addiction medicine

PURPOSE

We have evaluated the final-year Psychiatry and Addiction Medicine (PAM) summative Objective Structured Clinical Examination (OSCE) examinations in a four-year graduate medical degree program, for the previous three years as a baseline comparator, and during three years of the COVID-19 pandemic (2020-2022).

METHODS

A de-identified analysis of medical student summative OSCE examination performance, and comparative review for the 3 years before, and for each year of the pandemic.

RESULTS

Internal reliability in test scores as measured by R-squared remained the same or increased following the start of the pandemic. There was a significant increase in mean test scores after the start of the pandemic compared to pre-pandemic for combined OSCE scores for all final-year disciplines, as well as for the PAM role-play OSCEs, but not for the PAM mental state examination OSCEs.

CONCLUSIONS

Changing to online OSCEs during the pandemic was related to an increase in scores for some but not all domains of the tests. This is in line with a nascent body of literature on medical teaching and examination following the start of the pandemic. Further research is needed to optimise teaching and examination in a post-pandemic medical school environment.

Evaluation of Clinical Communication in Pharmacy Undergraduates in Brazil: A Multicentric Study

OBJECTIVE

To assess the clinical communication characteristics of pharmacy undergraduates, estimate differences in this specific competency, and produce recommendations for further education and training.

METHODS

Analysis of communication competence between 110 pharmacy students in the last graduation year from each of the 5 Brazilian regions and a simulated patient with complaints of mild allergic rhinitis passive of resolution with non-prescription medicines. The simulated appointment was recorded, and the video was analyzed using the 2 main elements: biomedical/task-focused and socio-emotional exchange of the Roter Interaction Analysis System.

RESULTS

The total of utterances/speech from the pharmacist to the patient was 183.4; there was a statistically significant difference according to the Brazilian region. In the consultation, the frequency with which pharmacy students returned to the segment was evaluated, with a total mean clinical history segment 2 of mean 5.60; in segment 4, which is the counseling phase, an average of 4.80. In the task codes and the socio-emotional codes, there was a statistically significant difference between the codes when compared by region. We compare by sex because it is said that women talk more than men. There was a statistically significant difference in socio-emotional code and biomedical/focused and task being higher for women.

CONCLUSION

The level of communication competence of students should be that desired for graduation, in all regions. There seems to be a difference between training and level of competence. Considering gender, although the consultation time is similar, it appears that the quality of communication is higher for women.

What's been trending with OSCEs in pharmacy education over the last 20 years? A bibliometric review and content analysis

BACKGROUND

Objective structured clinical examinations (OSCEs) remain an integral part of pharmacy education. This study aimed to characterize key researchers, areas, and themes in pharmacy education OSCEs using a bibliometric review with content analysis.

METHODS

A bibliometric review was conducted on literature from over 23 years from January 2000 to May 2023. Articles focusing on any type of OSCE research in pharmacy education in both undergraduate and postgraduate sectors were included. Articles were excluded if they were not original articles or not published in English. A summative content analysis was also conducted to identify key topics.

RESULTS

A total of 192 articles were included in the analysis. There were 242 institutions that contributed to the OSCE literature in pharmacy education, with the leading country being Canada. Most OSCE research came from developed countries and were descriptive studies based on single institution data. The top themes emerging from content analysis were student perceptions on OSCE station styles (n = 98), staff perception (n = 19), grade assessment of OSCEs (n = 145), interprofessional education (n = 11), standardized patients (n = 12), and rubric development and standard setting (n = 8).

IMPLICATIONS

There has been a growth in virtual OSCEs, interprofessional OSCEs, and artificial intelligence OSCEs. Communication rubrics and minimizing assessor variability are still trending research areas. There is scope to conduct more research on evaluating specific types of OSCEs, when best to hold an OSCE, and comparing OSCEs to other assessments.

Development of a communication-based virtual patient counseling objective structured clinical examination (OSCE) for first year student pharmacists

Objective

This text seeks to describe a framework for delivering and executing a virtual patient counseling OSCE and compare student performance to the in-person alternative.

Methods

A communication-based virtual patient counseling objective structured clinical examination (OSCE) was created to fulfill the educational outcomes of a previously held in-person OSCE in response to the COVID-19 pandemic. The virtual nature of this OSCE simulated a telehealth encounter, which has increased in utilization since the beginning of the COVID-19 pandemic. This OSCE was offered twice in one semester - once as a formative assessment and once as a summative assessment. Student performance was mapped to learning outcomes and compared the previous year's in person performance.

Results

The described virtual framework for executing the OSCE successfully decreased the time required and saw <1% change in overall performance from students when compared to the previous year in person which was not statistically significant.

Conclusion

This framework for a virtual communication-based OSCE reduces time with a negligible impact on student performance compared to the in person alternative.

Innovation

This work describes a telehealth virtual patient counseling model to replace the traditional patient counseling OSCE in pharmacy education with comparable outcomes.

Comparability of Objective Structured Clinical Examinations (OSCEs) and Written Tests for Assessing Medical School Students' Competencies: A Scoping Review

Objective Structured Clinical Examinations (OSCEs) and written tests are commonly used to assess health professional students, but it remains unclear whether the additional human resources and expenses required for OSCEs, both in-person and online, are worthwhile for assessing competencies. This scoping review summarized literature identified by searching MEDLINE and EMBASE comparing 1) OSCEs and written tests and 2) in-person and online OSCEs, for assessing health professional trainees' competencies. For Q1, 21 studies satisfied inclusion criteria. The most examined health profession was medical trainees (19, 90.5%), the comparison was most frequently OSCEs versus multiple-choice questions (MCQs) (18, 85.7%), and 18 (87.5%) examined the same competency domain. Most (77.5%) total score correlation coefficients between testing methods were weak (r < 0.40). For Q2, 13 articles were included. In-person and online OSCEs were most used for medical trainees (9, 69.2%), checklists were the most prevalent evaluation scheme (7, 63.6%), and 14/17 overall score comparisons were not statistically significantly different. Generally low correlations exist between MCQ and OSCE scores, providing insufficient evidence as to whether OSCEs provide sufficient value to be worth their additional cost. Online OSCEs may be a viable alternative to in-person OSCEs for certain competencies where technical challenges can be met.

Student Performance on an Objective Structured Clinical Exam Delivered Both Virtually and In-Person

OBJECTIVE

Passing a milestone objective structured clinical examination (OSCE) is a graduation requirement for the University of Waterloo Pharmacy students. In January 2021, the milestone OSCE was offered concurrently both virtually and in-person, with students being able to choose their desired format. The purpose of this study was to compare student performance between the 2 formats and to identify factors that may have predicted student choice of format.

METHODS

Objective structured clinical examination scores for in-person and virtual exam-takers were compared using 2-tailed independent t tests with Bonferroni correction. Pass rates were compared using χ2 analysis. Prior academic performance variables were analyzed to identify predictors of the chosen exam format. Student and exam personnel surveys were used to capture OSCE feedback.

RESULTS

A total of 67 students (56%) participated in the in-person OSCE, and 52 students (44%) participated virtually. There were no significant differences in overall exam averages or pass rates between the 2 groups. However, virtual exam-takers scored lower in 2 of 7 cases. Previous academic performance did not predict the choice of exam format. Feedback surveys indicated that the exam organization was perceived as a strength regardless of format, but in-person students felt more prepared for the exam than virtual exam-takers with technical challenges and difficulty navigating station resources being noted as barriers in the virtual offering.

CONCLUSION

Virtual and in-person administration of a milestone OSCE resulted in similar student performance, with slightly lower performance on 2 individual case scores with virtual delivery. These results may inform the future development of virtual OSCEs.

Comparison of Students' Self-Assessment and Simulated Patient Assessment in a Patient Counseling Evaluation and Perceived Importance of Communication Skills

The primary objective of this study was to compare students' self-assessment ratings with simulated patient (SP) assessment ratings of communication skills in a patient counseling Objective Structured Clinical Exam (OSCE). The secondary objective was to evaluate student perceptions of the importance of communication skills in the practice of pharmacy as well as the impact of a virtual OSCE format. First-year pharmacy students completed an OSCE focused on self-care product counseling. The evaluation was graded using a rubric covering both verbal and non-verbal communication. Students who completed the course were provided a 15-question, post-evaluation survey with questions related to self-assessment of communication skills and perceptions of the importance of communication skills. Of the 138 students in the course, 68 completed the optional post-assessment survey (49% response rate). There were no statistically significant differences between the ratings by students and SPs for the four communication elements included in the self-assessment. Most of the students recognized the importance of communication skills, including developing rapport and trust. Recognition of the importance of communication skills to future practice as a pharmacist positively correlated with performance on the evaluation (r² = 0.5409, p-value = 0.0007). Student self-assessment is an effective and cost-effective mode of feedback for practice experiences as an alternative to the use of SPs.

Educating pharmacy students through a pandemic: Reflecting on our COVID-19 experience

The impact of the COVID-19 pandemic on pharmacy education worldwide has been immense, affecting students, educators and regulatory agencies. Pharmacy programmes have had to rapidly adapt in their delivery of education, maintaining standards while also ensuring the safety of all stakeholders. In this commentary, we describe the challenges, compromises and solutions adopted by our institution throughout the pandemic, the lessons learnt, adaptive measures taken, and strategies to develop and future-proof our curricula.

COVID-19-Driven Improvements and Innovations in Pharmacy Education: A Scoping Review

The COVID-19 pandemic led to many colleges of pharmacy having to make major changes relating to their infrastructure and delivery of their curriculum within a very short time frame, including the transition of many components to an online setting. This scoping review sought to summarize what is known about the impact of COVID-19 on pharmacy education and the effectiveness of adaptation strategies which were put in place. PubMed, Web of Science, OVID Medline, and MedEdPortal were searched to identify pharmacy education-related articles published since the beginning of the COVID-19 pandemic. For article inclusion, the following criteria had to be met: described original research, related directly to PharmD or PharmBS education, related to the impact of COVID-19 on pharmacy education, and was available in English. Out of a total of 813 articles, 50 primary research articles were selected for inclusion. Our review of these identified four domains relating to the impact of COVID-19 on pharmacy education and/or effectiveness of adaptation strategies: (1) lab-based courses and activities (including interprofessional education activities), (2) experiential education, (3) didactic education, and (4) student well-being. The key research findings are summarized and discussed. While the COVID-19 pandemic has clearly brought many challenges to pharmacy education, it has also led to key improvements and innovations.

Perceptions of Pharmacy Students on the E-Learning Strategies Adopted during the COVID-19 Pandemic: A Systematic Review

BACKGROUND

E-learning strategies were globally adopted by academies because of the COVID-19 pandemic. The characterization of students' perception of online learning is fundamental to design appropriate models for pharmacy curricula. The study aim was to carry out a systematic review about the perception of pharmacy students on the e-learning strategies adopted during the COVID-19 pandemic.

METHODS

The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) checklist was followed. PICOS criteria were applied. Five databases were screened: PubMed, Cochrane Library, DOAJ-Directory of Open Access Journals, SciELO-Scientific Electronic Library Online and b-on-Online Library of knowledge (Biblioteca do conhecimento online).

KEYWORDS

"pharmacy and (distant or remote or e-learning or online or zoom or education or training or digital) and (COVID or SARS and (student or undergraduate) and (opinion or satisfaction or perception or attitude)".

RESULTS

23 out of 176 papers were selected (28 duplicated and 125 excluded). Selected papers were classified, as follows: studies exclusively involving pharmacy students (n = 8); studies simultaneously involving pharmacy students and other healthcare students (n = 6); and studies related to the involvement of pharmacy students in specific courses (n = 9).

CONCLUSIONS

In general, the perception of pharmacy students on e-learning strategies adopted during the COVID-19 pandemic was positive. However, an expressive proportion of undergraduates reported negative issues about online education, which seems to support the necessity of optimizing e-learning strategies in the future.

Students' and Examiners' Experiences of Their First Virtual Pharmacy Objective Structured Clinical Examination (OSCE) in Australia during the COVID-19 Pandemic

Objective Structured Clinical Examinations (OSCEs) are routinely used in healthcare education programs. Traditionally, students undertake OSCEs as face-to-face interactions to assess competency in soft skills. Due to physical distancing restrictions during COVID-19, alternative methods were required. This study utilized a mixed-method design (online survey and interviews) to evaluate second-year pharmacy students' and examiners' experiences of their first virtual OSCEs in Australia. A total of 196 students completed their first virtual OSCE in June 2020 of which 190 students completed the online survey. However, out of the 190 students, only 88% (n = 167) consented to the use of the data from their online survey. A further 10 students and 12 examiners were interviewed. Fifty-five students (33%) who participated in the online survey strongly agreed or agreed that they preferred the virtual experience to face-to-face OSCEs while 44% (n = 73) neither agreed nor disagreed. Only 20% (n = 33) felt more anxious with the virtual OSCEs. Additionally, thematic analysis found non-verbal communication as a barrier during the OSCE. Positive aspects about virtual OSCEs included flexibility, decreased levels of anxiety and relevance with emerging telehealth practice. The need for remote online delivery of assessments saw innovative ways of undertaking OSCEs and an opportunity to mimic telehealth. While students and examiners embraced the virtual OSCE process, face-to-face OSCEs were still considered important and irreplaceable. Future opportunities for OSCEs to be delivered both face-to-face and virtually should be considered.

Disadvantages of using Traditional, Interactive and Distance Learning Methods when Studying the Specialty "Pharmacy"

<p>Interactive and remote technologies are being actively introduced into the training process in the specialty “Pharmacy.” In addition to ease of use, they have several advantages and disadvantages for all stakeholders. The purpose of the work is to analyze the disadvantages of traditional, interactive, and remote methods of teaching in the specialty “Pharmacy” at the Far Eastern State Medical University. This is a sociological survey of students of the Faculty of Pharmacy and Biomedicine (higher education), as well as the Medical-Pharmaceutical College (secondary education). Statistical processing was performed using the non-parametric Mann-Whitney, Kruskal–Wallis, Spearman’s rank correlation coefficient. Traditional teaching methods are characterized by the inconvenience of tight time frames. During on-line classes, it is more difficult to use an individual methodology for the student and change the teaching method. The most characteristic shortcomings of distance learning include the lack of motivation and the possibility of violations (writing off, forgery, etc.) by the student. Despite the geographical remoteness of the regions, material, and technical difficulties in switching to distance learning, as a rule, did not arise. The lack of an individual approach and the subjectivity of student assessments in distance and traditional teaching methods are due to the high occupancy of groups and their frequent combination (10-25 people) during the class. The disadvantage of remote methods is the complexity of regular control by the teacher, the lack of secure personality identification systems when sending written works and testing on-line, and the low level of consciousness of future specialists. According to the results of the questionnaire, the least number of shortcomings have interactive teaching methods, implying the active participation of the student himself and an individual approach.</p>

Using Technology to Enhance Teaching and Learning in Pharmacy Education

It was a privilege to serve as guest editors in Pharmacy for the Special Issue 'Technology-Enhanced Pharmacy Teaching and Learning Strategies' [...].