[Changes in the Objective Structured Clinical Examination (OSCE) of University Schools of Medicine during COVID-19. Experience with a computer-based case simulation OSCE (CCS-OSCE)◊]

A comparison of the American Board of Anesthesiology's in-person and virtual objective structured clinical examinations

BACKGROUND

The American Board of Anesthesiology's Objective Structured Clinical Examination (OSCE), as a component of its initial certification process, had been administered in-person in a dedicated assessment center since its launch in 2018 until March 2020. Due to the COVID-19 pandemic, a virtual format of the exam was piloted in December 2020 and was administered in 2021. This study aimed to compare candidate performance, examiner grading severity, and scenario difficulty between these two formats of the OSCE.

METHODS

The Many-Facet Rasch Model was utilized to estimate candidate performance, examiner grading severity, and scenario difficulty for the in-person and virtual OSCEs separately. The virtual OSCE was equated to the in-person OSCE by common examiners and common scenarios. Independent-samples t-test was used to compare candidate performance, and partially overlapping samples t-tests were applied to compare examiner grading severity and scenario difficulty between the in-person and virtual OSCEs.

RESULTS

The in-person (n = 3235) and virtual (n = 2934) first-time candidates were comparable in age, sex, race/ethnicity, and whether U.S. medical school graduates. The virtual scenarios (n = 35, mean [0.21] ± SD [0.38] in logits) were more difficult than the in-person scenarios (n = 93, 0.00 ± 0.69, Welch's partially overlapping samples t-test, p = 0.01); there were no statistically significant differences in examiner severity (n = 390, -0.01 ± 0.82 vs. n = 304, -0.02 ± 0.93, Welch's partially overlapping samples t-test, p = 0.81) or candidate performance (2.19 ± 0.93 vs. 2.18 ± 0.92, Welch's independent samples t-test, p = 0.83) between the in-person and virtual OSCEs.

CONCLUSIONS

Our retrospective analyses of first-time OSCEs found comparable candidate performance and examiner grading severity between the in-person and virtual formats, despite the virtual scenarios being more difficult than the in-person scenarios. These results provided assurance that the virtual OSCE functioned reasonably well in a high-stakes setting.

Impact of a peer-to-peer escape room activity in the learning of Human Physiology of medical students from the university of Málaga

Escape room's popularity has raised over the past years among young adults. It creates a distended competitive environment, where participants collaborate to achieve a common objective through teamwork. We decided to apply this format as a teaching method for medical students at the University of Malaga, Spain. A peer-to-peer physiological cardiorespiratory escape room was designed by intern undergraduate students, collaborating within the Department of Human Physiology. This activity integrated the contents of the Human Physiology syllabus, which were organized into four stages that culminated in a final medical case. Intern students oversaw the design, promotion, preparation and execution of the activity, and were in charge of conducting the evaluation and follow up. The escape room was done in mid-December, after all theoretical and practical contents had been delivered, for four consecutive years, improving from each year's experience. The target group for this activity were second year medical students, who were asked to team up freely in groups of four to six students before the start of the activity. The students in each group cooperated with each other while trying to solve the different puzzles and questions in each stage of the escape room. After the activity, the results of the final evaluation exam of these participants were compared against non-participants, who served as a control group. Qualitative feedback was also received from the participants via a special survey that was designed for this task. Results between 2020 and 2023 (three last activities) show that the final mark of the participants was significantly higher than in non-participants (6.39 ± 0.14 vs. 5.04 ± 0.2; p < 0.0007). The global exam mark also increased in the participants (5.43 ± 0.10 vs. 4.44 ± 0.15; p < 0.0007). A significant difference was observed in the performance in cardiovascular (p < 0.0007) and respiratory-related questions (p < 0.0007), which was substantial in the participants. The qualitative feedback received from the participants was mainly positive, indicating an overall acceptance of the format by the students. We conclude that escape room format with a peer-to-peer structure is an efficient teaching tool for medical students performed by medical students in the field of Human Physiology.

Medical studies in times of a pandemic - concepts of digital teaching for Orthopaedics and Trauma at german universities

BACKGROUND

Due to the Covid-19 pandemic, on-site classroom teaching became limited at most German medical universities. This caused a sudden demand for digital teaching concepts. How the transfer from classroom to digital teaching or digitally assisted teaching was conducted was decided by each university and/or department individually. As a surgical discipline, Orthopaedics and Trauma have a particular focus on hands-on teaching as well as direct contact to patients. Therefore, specific challenges in designing digital teaching concepts were expected to arise. Aim of this study was to evaluate medical teaching at German universities one year into the pandemic as well as to identify potentials and pitfalls in order to develop possible optimization approaches.

METHODS

A questionnaire with 17 items was designed and sent to the professors in charge of organising the teaching in Orthopaedics and Trauma at each medical university. A differentiation between Orthopaedics and Trauma was not made to allow a general overview. We collected the answers and conducted a qualitative analysis.

RESULTS

We received 24 replies. Each university reported a substantial reduction of their classroom teaching and efforts to transfer their teaching to digital formats. Three sites were able to switch to digital teaching concepts completely, whereas others tried to enable classroom and bedside teaching at least for students of higher edcuational levels. The online platforms used varied depending on the university as well as the format it was supposed to support.

CONCLUSION

One year into the pandemic significant differences concerning the proportions of classroom and digital teaching for Orthopaedics and Trauma can be observed. Simultaneously huge differences in concepts used to create digital teaching are present. Since a complete suspense of classroom teaching was never mandatory, several universities developed hygiene concepts to enable hands-on and bedside teaching. Despite these differences, some similarities were observed: the lack of time and personnel to generate adequate teaching material was reported as the leading challenge by all participants of this study.

The objective structured clinical examination (OSCE): Main aspects and the role of imaging

The objective structured clinical examination (OSCE) is a format of examination that enables students to be evaluated in a uniform, standardized, reliable, and objective way. It is carried out in different clinical stations that simulate real clinical situations and scenarios. Numerous universities in Spain and other countries employ this approach for the final examination for medical school students. This update describes the organization, design, and fundamentals for the OSCE, proposing that radiology should form part of multidisciplinary OSCEs to the extent that it forms part of clinical practice. Moreover, it is interesting and opportune to introduce the OSCE in undergraduate and postgraduate training in radiology. Online platforms enable bidimensional OSCEs that are cost-effective in terms of staff, resources, and physical space, although this approach has certain limitations. Virtual world technologies make it possible to reproduce OSCE stations in three-dimensional scenarios; recent experiences in radiology have shown that this approach interests and motivates students and is widely accepted by them.

Feasibility of large-scale eOSCES: the simultaneous evaluation of 500 medical students during a mock examination

The COVID-19 pandemic has led health schools to cancel many on-site training and exams. Teachers were looking for the best option to carry out online OSCEs, and Zoom was the obvious choice since many schools have used it to pursue education purposes.

METHODS

We conducted a feasibility study during the 2020-2021 college year divided into six pilot phases and the large-scale eOSCEs on Zoom on June 30th, 2021. We developed a specific application allowing us to mass create Zoom meetings and built an entire organization, including a technical support system (an SOS room and catching-up rooms) and teachers' training sessions. We assessed satisfaction via an online survey.

RESULTS

On June 30th, 531/794 fifth-year medical students (67%) participated in a large-scale mock exam distributed in 135 Zoom meeting rooms with the mobilization of 298 teachers who either participated in the Zoom meetings as standardized patients (N =135, 45%) or examiners (N =135, 45%) or as supervisors in the catching-up rooms (N =16, 6%) or the SOS room (N =12, 4%). In addition, 32/270 teachers (12%) experienced difficulties connecting to their Zoom meetings and sought the help of an SOS room member. Furthermore, 40/531 students (7%) were either late to their station or had technical difficulties and declared those issues online and were welcomed in one of the catching-up rooms to perform their eOSCE stations. Additionally, 518/531 students (98%) completed the entire circuit of three stations, and 225/531 students (42%) answered the online survey. Among them, 194/225 (86%) found eOSCES helpful for training and expressed their satisfaction with this experience.

CONCLUSION

Organizing large-scale eOSCEs on Zoom is feasible with the appropriate tools. In addition, eOCSEs should be considered complementary to on-site OSCEs and to train medical students in telemedicine.

Adaptations in clinical examinations of medical students in response to the COVID-19 pandemic: a systematic review

INTRODUCTION

Clinical examinations (assessments) are integral to ensuring that medical students can treat patients safely and effectively. The COVID-19 pandemic disrupted traditional formats of clinical examinations. This prompted Medical Schools to adapt their approaches to conducting these examinations to make them suitable for delivery in the pandemic. This systematic review aims to identify the approaches that Medical Schools, internationally, adopted in adapting their clinical examinations of medical students in response to the COVID-19 pandemic.

METHODS

Three databases and four key medical education journals were systematically searched up to 22 October 2021; a grey literature search was also undertaken. Two reviewers independently screened at title, abstract stage and full text stage against predefined eligibility criteria. Discrepancies were resolved by discussion and involvement of senior authors. Risk of bias assessment was performed using an adapted version of a pre-existing risk of bias assessment tool for medical education developments. Results were summarised in a narrative synthesis.

RESULTS

A total of 36 studies were included, which documented the approaches of 48 Medical Schools in 17 countries. Approaches were categorised into in-person clinical examinations (22 studies) or online clinical examinations (14 studies). Authors of studies reporting in-person clinical examinations described deploying enhanced infection control measures along with modified patient participation. Authors of studies reporting online clinical examinations described using online software to create online examination circuits. All authors reported that adapted examinations were feasible, scores were comparable to previous years' student cohorts, and participant feedback was positive. Risk of bias assessment highlighted heterogeneity in reporting of the clinical examinations.

CONCLUSIONS

This review identified two broad approaches to adapting clinical examinations in the pandemic: in-person and online. Authors reported it was feasible to conduct clinical examinations in the pandemic where medical educators are given sufficient time and resources to carefully plan and introduce suitable adaptations. However, the risk of bias assessment identified few studies with high reporting quality, which highlights the need for a common framework for reporting of medical education developments to enhance reproducibility across wider contexts. Our review provides medical educators with the opportunity to reflect on past practises and facilitate the design and planning of future examinations.

The virtual Clinical Assessment of Skills and Competence: the impact and challenges of a digitised final examination

The COVID-19 pandemic has affected how clinical examinations are conducted, resulting in the Royal College of Psychiatrists delivering the Clinical Assessment of Skills and Competence virtually. Although this pragmatic step has allowed for progression of training, it has come at the cost of a significantly altered examination experience. This article aims to explore the fairness of such an examination, the difference in trainee experience, and the use of telemedicine to consider what might be lost as well as gained at a time when medical education and delivery of healthcare are moving toward the digitised frontier.