Malaysian pharmacy students' perspectives on the virtual objective structured clinical examination during the coronavirus disease 2019 pandemic

What's been trending with OSCEs in pharmacy education over the last 20 years? A bibliometric review and content analysis

BACKGROUND

Objective structured clinical examinations (OSCEs) remain an integral part of pharmacy education. This study aimed to characterize key researchers, areas, and themes in pharmacy education OSCEs using a bibliometric review with content analysis.

METHODS

A bibliometric review was conducted on literature from over 23 years from January 2000 to May 2023. Articles focusing on any type of OSCE research in pharmacy education in both undergraduate and postgraduate sectors were included. Articles were excluded if they were not original articles or not published in English. A summative content analysis was also conducted to identify key topics.

RESULTS

A total of 192 articles were included in the analysis. There were 242 institutions that contributed to the OSCE literature in pharmacy education, with the leading country being Canada. Most OSCE research came from developed countries and were descriptive studies based on single institution data. The top themes emerging from content analysis were student perceptions on OSCE station styles (n = 98), staff perception (n = 19), grade assessment of OSCEs (n = 145), interprofessional education (n = 11), standardized patients (n = 12), and rubric development and standard setting (n = 8).

IMPLICATIONS

There has been a growth in virtual OSCEs, interprofessional OSCEs, and artificial intelligence OSCEs. Communication rubrics and minimizing assessor variability are still trending research areas. There is scope to conduct more research on evaluating specific types of OSCEs, when best to hold an OSCE, and comparing OSCEs to other assessments.

A Telehealth Module and Virtual Objective Structured Clinical Examination of Health Literacy in Pharmacy Education

OBJECTIVE

To implement an online health literacy (HL) module and evaluate its impact through virtual Objective Structured Clinical Examination (OSCE) on students' abilities to manage patients with low HL.

METHODS

Students virtually engaged in several learning activities related to HL, including practicing using HL assessment tools, designing an information booklet suitable for low HL patients, using readability formulas to lower text readability level to ≤ sixth grade, role-playing scenarios related to HL, and undergoing a virtual OSCE. Student performance on course assessments was tested using Spearman's rank-order. Students evaluated their OSCE experience in terms of case content, virtual assessment, and logistics; and effectiveness of the HL module and confidence related to HL.

RESULTS

A total of 90 students completed the virtual OSCE; the mean score was 8.8 out of 10 (88%), which correlated with similar course assessments. The average score was 3.46 out of 3.7 for the "gathering information" domain (recognizing risk factors and behaviors, assessing HL and adherence, etc.), and 4.06 out of 4.9 for the "patient management" domain (counseling patients on taking medications, focusing on repeating key messages, offering adherence interventions, etc.). Students responded positively regarding case content and virtual assessment, but less favorably regarding logistics. Feedback on effectiveness of the HL module and confidence in managing patients with low HL was positive.

CONCLUSION

An HL module delivered online was effective in improving student knowledge, abilities, and confidence related to HL. Students achieved high scores in all items in the virtual OSCE, and considered it a measure of communication and clinical skills that is comparable to traditional in-room assessments.

Shifting to Authentic Assessments? A Systematic Review of Student Perceptions of High-Fidelity Assessments in Pharmacy

OBJECTIVE

Despite becoming increasingly popular, there is no comprehensive review on high-fidelity assessments in pharmacy education that has a focus on the perceptions and experiences of students. This systematic review investigates the student acceptability of high-fidelity simulation for summative assessments in pharmacy education and provides recommendations regarding high-fidelity simulation practices.

FINDINGS

The search yielded 37 studies in total. The articles fell into 3 distinct categories: objective structured clinical examinations (N = 25); face-to-face simulation assessments (N = 9), and augmented reality assessments (N = 3). Most high-fidelity assessments were well received with most students agreeing they had a vital role in assessing the application of clinical knowledge, even though they are stressful. Students prefer high-fidelity assessments to be face-to-face rather than online and also prefer when simulated patients whom they are unfamiliar with are used. Students also expressed a need to be well-prepared for the assessment regarding the logistics of the exam and technology use.

SUMMARY

High-fidelity simulation is likely to be increasingly important in the assessment of the knowledge and skills of pharmacy students and student perception is an important factor to consider when developing such assessments. Reducing stress associated with high-fidelity assessments could include familiarizing students with task logistics or technology used prior to the assessment, using external simulated patients, and having face-to-face assessments and practice sessions.

Evaluation of grit and its associated factors among undergraduate pharmacy students from 14 Asian and Middle Eastern countries amid the COVID-19 pandemic

Introduction

Grit is proposed as an essential trait for academic achievement. Thus, evaluating its current status and the associated factors could aid academic support planning.

Objective

The present study aimed to assess grit level and its related factors among undergraduate pharmacy students from 14 countries amid the COVID-19 pandemic.

Methods

A cross-sectional survey-based study was conducted among pharmacy students from 14 countries in Asia and the Middle East. A 31-item questionnaire was developed, validated, and pilot-tested, including the validated short scale for grit assessment. The data was collected between 1 February and 15 April 2022. Descriptive and inferential statistics were employed as appropriate.

Results

A total of 2665 responses were received, mainly from females (68.7 %), living in urban areas (69.2 %) and studying at private universities (59.1 %). The average grit score on a scale of 5 was 3.15 ± 0.54. The responses revealed higher favourable responses to items on the perseverance of efforts (34.9 % to 54 %) compared to items on the consistency of interests (26.5 % to 31.1 %). Students who did not exercise (AOR: 0.47, 95 %CI: 0.33-0.67) or exercised irregularly (AOR: 0.64, 95 %CI: 0.45-0.90) were less likely to have higher grit scores than those who exercised regularly. Additionally, students who did not receive COVID-19 vaccination (AOR: 0.50, 95 %CI: 0.36-0.71) or received only one dose (AOR: 0.67, 95 %CI: 0.46-0.99) were less likely to have higher grit scores than those who received their booster vaccination. Interestingly, students who chose the pharmacy program as their only available or reasonable choice (AOR: 0.33, 95 %CI: 0.17-0.62) and students from public universities (AOR: 0.82, 95 %CI: 0.68-0.98) were less likely to have higher grit scores. On the other hand, students who did not face educational challenges with online learning (AOR: 1.19, 95 %CI: 1.003-1.416) and students with excellent (AOR: 2.28, 95 %CI: 1.57-3.31) and very good (AOR: 2.16, 95 %CI: 1.53-3.04) academic performance were more likely to have higher grit scores.

Conclusion

The findings revealed moderate grit levels. Higher grit levels were thought to be associated with several personal, lifestyle and academic factors. Further interventions to support students' grit attributes are required, particularly concerning the consistency of interests.

Supporting pharmacy students' preparation for an entry-to-practice OSCE using video cases

BACKGROUND AND PURPOSE

Objective structured clinical examinations (OSCEs) are widely used, including in licensure examinations. OSCEs assess diverse skills in a structured manner, but can be stressful for students and have a significant organisational burden for faculty. Case-based video OSCE preparation resources were developed for students preparing for a licensure OSCE. The study aimed to examine student engagement, compare scores awarded by students to performances at specific competence standards with faculty scores, and examine usability, usefulness, and acceptability of the video cases.

EDUCATIONAL ACTIVITY AND SETTING

Final year pharmacy students (n = 149) enrolled in an integrated master of pharmacy programme in Ireland were invited to participate. Six sets of recorded OSCE-based video cases were developed, comprising multiple recordings of the same case scenario, with each pitched at a different level of performance. Students watched and scored the video cases. Usability, usefulness, and acceptability were evaluated via questionnaire.

FINDINGS

One or more video cases were accessed by 70.5% of students. Score ratings awarded by the students, when compared to faculty ratings, showed an overall trend towards inter-rater agreement between students and faculty. Students felt positively in terms of the usability, usefulness, and acceptability of the videos.

SUMMARY

Video cases designed to support OSCE preparation for a high-stakes national pharmacy licensure examination were widely used by students and were perceived to be usable, useful, and acceptable. Such video cases may be a feasible alternative to additional mock OSCEs to support student preparation.

Experience with an OSCE anamnesis station via Zoom: Feasibility, acceptance and challenges from the perspective of students, simulated patients and examiners during the COVID-19 pandemic

Aim: Assessments of practical clinical competencies pose a challenge during the COVID-19 pandemic. Reports about OSCE stations administered online show that, despite technical feasibility and acceptance, there is a lingering desire for in-person assessments. Barriers and challenges must therefore also be identified in regard to the future integration of digital competencies into the curriculum. Based on a study investigating the feasibility and acceptance of an online OSCE anamnesis station and the descriptions given by students, simulated patients and examiners of the challenges and limitations, we make recommendations for necessary future adaptations to anamnesis training and testing in the context of telemedicine. Method: We surveyed students after completion of an OSCE anamnesis station, adapted to the telemedical setting, that was administered as an alternative assessment to 149 students via Zoom^®. Using semi-structured interviews, we analyzed the resulting challenges and limitations as seen by all of the participants. Results: We confirm the existence of good technical and organizational feasibility, positive learning experiences through feedback, the acquisition of clinical competencies, and a high acceptance of this format as an alternative assessment during the pandemic. Using the semi-structured interviews, it was also possible to analyze additional categories that identify necessary adaptations of this type of format. Conclusion: Adaptation of the content-based training for all of the participants and a targeted revision of the checklists, e.g., regarding communication techniques in a telemedicine setting, is required due to the effects of the online format on communication and interactions between students and simulated patients.

COVID-19-Driven Improvements and Innovations in Pharmacy Education: A Scoping Review

The COVID-19 pandemic led to many colleges of pharmacy having to make major changes relating to their infrastructure and delivery of their curriculum within a very short time frame, including the transition of many components to an online setting. This scoping review sought to summarize what is known about the impact of COVID-19 on pharmacy education and the effectiveness of adaptation strategies which were put in place. PubMed, Web of Science, OVID Medline, and MedEdPortal were searched to identify pharmacy education-related articles published since the beginning of the COVID-19 pandemic. For article inclusion, the following criteria had to be met: described original research, related directly to PharmD or PharmBS education, related to the impact of COVID-19 on pharmacy education, and was available in English. Out of a total of 813 articles, 50 primary research articles were selected for inclusion. Our review of these identified four domains relating to the impact of COVID-19 on pharmacy education and/or effectiveness of adaptation strategies: (1) lab-based courses and activities (including interprofessional education activities), (2) experiential education, (3) didactic education, and (4) student well-being. The key research findings are summarized and discussed. While the COVID-19 pandemic has clearly brought many challenges to pharmacy education, it has also led to key improvements and innovations.

Perceptions of Pharmacy Students on the E-Learning Strategies Adopted during the COVID-19 Pandemic: A Systematic Review

BACKGROUND

E-learning strategies were globally adopted by academies because of the COVID-19 pandemic. The characterization of students' perception of online learning is fundamental to design appropriate models for pharmacy curricula. The study aim was to carry out a systematic review about the perception of pharmacy students on the e-learning strategies adopted during the COVID-19 pandemic.

METHODS

The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) checklist was followed. PICOS criteria were applied. Five databases were screened: PubMed, Cochrane Library, DOAJ-Directory of Open Access Journals, SciELO-Scientific Electronic Library Online and b-on-Online Library of knowledge (Biblioteca do conhecimento online).

KEYWORDS

"pharmacy and (distant or remote or e-learning or online or zoom or education or training or digital) and (COVID or SARS and (student or undergraduate) and (opinion or satisfaction or perception or attitude)".

RESULTS

23 out of 176 papers were selected (28 duplicated and 125 excluded). Selected papers were classified, as follows: studies exclusively involving pharmacy students (n = 8); studies simultaneously involving pharmacy students and other healthcare students (n = 6); and studies related to the involvement of pharmacy students in specific courses (n = 9).

CONCLUSIONS

In general, the perception of pharmacy students on e-learning strategies adopted during the COVID-19 pandemic was positive. However, an expressive proportion of undergraduates reported negative issues about online education, which seems to support the necessity of optimizing e-learning strategies in the future.

Students’ and Examiners’ Experiences of Their First Virtual Pharmacy Objective Structured Clinical Examination (OSCE) in Australia during the COVID-19 Pandemic

Objective Structured Clinical Examinations (OSCEs) are routinely used in healthcare education programs. Traditionally, students undertake OSCEs as face-to-face interactions to assess competency in soft skills. Due to physical distancing restrictions during COVID-19, alternative methods were required. This study utilized a mixed-method design (online survey and interviews) to evaluate second-year pharmacy students’ and examiners’ experiences of their first virtual OSCEs in Australia. A total of 196 students completed their first virtual OSCE in June 2020 of which 190 students completed the online survey. However, out of the 190 students, only 88% (n = 167) consented to the use of the data from their online survey. A further 10 students and 12 examiners were interviewed. Fifty-five students (33%) who participated in the online survey strongly agreed or agreed that they preferred the virtual experience to face-to-face OSCEs while 44% (n = 73) neither agreed nor disagreed. Only 20% (n = 33) felt more anxious with the virtual OSCEs. Additionally, thematic analysis found non-verbal communication as a barrier during the OSCE. Positive aspects about virtual OSCEs included flexibility, decreased levels of anxiety and relevance with emerging telehealth practice. The need for remote online delivery of assessments saw innovative ways of undertaking OSCEs and an opportunity to mimic telehealth. While students and examiners embraced the virtual OSCE process, face-to-face OSCEs were still considered important and irreplaceable. Future opportunities for OSCEs to be delivered both face-to-face and virtually should be considered.

Evaluation of the Utility of Online Objective Structured Clinical Examination Conducted During the COVID-19 Pandemic

BACKGROUND

The COVID-19 pandemic led to profound restrictions on the face-to-face learning and assessment in all educational institutions, particularly the medical schools. The College of Medicine and Medical Sciences of the Arabian Gulf University (CMMS-AGU) conducted the final exams, both theoretical and clinical components, for its MD students online. This study was conducted to evaluate the utility of online clinical exams held at CMMS-AGU.

METHODS

This is a cross-sectional, mixed method study that included samples from final year medical students, examiners, and heads of clinical departments. Data were collected through surveys, structured interviews, documents' review, and calculation of online examination's psychometrics. Descriptive statistics were used. Quantitative data were presented in the form of means and standard deviations. Responses of heads of clinical departments in the structured interview were transcribed and analyzed thematically based on three pre-established themes.

RESULTS

Quantitative and qualitative data on the utility (validity, reliability, acceptability, educational impact, and cost and feasibility) of online objective structured clinical examination (OSCE) were collected. Content validity of the online clinical examination was established through high mean scores of content representativeness, which was confirmed by the heads of clinical departments regarding the proper coverage of clinical skills. Criterion validity was established through a high correlation between clinical and theoretical exam results (r = 0.75). Reliability of the exam was established through an acceptable Cronbach's alpha value (0.70 to 0.78) over the four days of the examinations. The examinations were perceived as highly acceptable by both students and examiners. High educational impact was inferred from students' responses and review of documents. The examination was found to be feasible and of reasonable cost.

CONCLUSION

Online OSCE might be a good alternative of conventional clinical assessments in times of crises and impossibility of having in-person contact between students, examiners, and patients. An important major drawback is still present in such initiatives, which is the inability to assess students' physical examination skills.

Applying the Student Response System in the Online Dermatologic Video Curriculum on Medical Students' Interaction and Learning Outcomes during the COVID-19 Pandemic

Background

The coronavirus disease 2019 (COVID-19) pandemic impacted medical education worldwide. Online lecture is increasingly prevalent in higher education, but students' completion rate is quite low.

Aims

This study aimed to determine the effectiveness of the student response system (SRS) in the online dermatologic video curriculum on medical students.

Methods

A prospective study was conducted on 176 undergraduate fourth-year medical students. The online video lecture was integrated with SRS.

Results

A total of 173 students completed the pre-test, and the attendance rate (pre-test/total) was 98.3%. A total of 142 students completed the post-test, and the completion rate (post-test/pre-test) was 82.8%. The post-test score (83.69 ± 4.34) was found to be significantly higher than that of the pre-test (62.69 ± 6.08, P =0.0002). A total of 138 students completed the questionnaire, and 92% of students opined that SRS was easy to operate. 86% of students agreed with the fact that the use of SRS could increase their learning performance by interacting with teachers. In the open-ended question, students stated that SRS offered opportunities for student-faculty interaction, allowed them to get immediate feedback, and promote active participation.

Conclusions

These results highlight that the integration of SRS in the online video curriculum increases students' completion rates and learning outcomes. Moreover, the SRS is easy to operate for the students and enhances student-faculty interaction. The SRS may be adopted in online learning during this challenging time.

Empirical analysis comparing the tele-objective structured clinical examination (teleOSCE) and the in-person assessment in Australia

PURPOSE

It aimed to compare the use of the tele objective structured clinical examination (teleOSCE) with in-person assessment in high-stakes clinical examination so as to determine the impact of the teleOSCE on the assessment undertaken. Discussion follows regarding what skills and domains can effectively be assessed in a teleOSCE.

METHODS

This study is a retrospective observational analysis. It compares the results achieved by final year medical students in their clinical examination, assessed using the teleOSCE in 2020 (n=285), with those who were examined using the traditional in-person format in 2019 (n=280). The study was undertaken at the University of New South Wales, Australia.

RESULTS

In the domain of physical examination, students in 2020 scored 0.277 points higher than those in 2019 (mean difference -0.277, P<0.001, effect size 0.332). Across all other domains, there was no significant difference in mean scores between 2019 and 2020.

CONCLUSION

The teleOSCE does not negatively impact assessment in clinical examination in all domains except physical examination. If the teleOSCE is the future of clinical skills examination, assessment of physical examination will require concomitant workplace-based assessment.