Rapid transition to online assessment: Practical steps and unanticipated advantages

Medical students' perception of changes in assessments implemented during the COVID-19 pandemic

BACKGROUND

COVID-19 posed many challenges to medical education in the United Kingdom (UK). This includes implementing assessments during 4 months of national lockdowns within a 2-year period, where in-person education was prohibited. This study aimed to identify medical school assessment formats emerging during COVID-19 restrictions, investigate medical students' perspectives on these and identify influencing factors.

METHODS

The study consisted of two phases: a questionnaire asking medical students about assessment changes they experienced, satisfaction with these changes and preference regarding different assessments that emerged. The second phase involved semi-structured interviews with medical students across the UK to provide a deeper contextualized understanding of the complex factors influencing their perspectives.

RESULTS

In the questionnaire responses, open-book assessments had the highest satisfaction, and were the preferred option indicated. Furthermore, in the case of assessment cancellation, an increase in weighting of future assessments was preferred over increase in weighting of past assessments. Students were also satisfied with formative or pass-fail assessments. Interview analyses indicate that although cancellation or replacement of summative assessments with formative assessments reduced heightened anxiety from additional COVID-19 stressors, students worried about possible future knowledge gaps resulting from reduced motivation for assessment-related study. Students' satisfaction level was also affected by timeliness of communication from universities regarding changes, and student involvement in the decision-making processes. Perceived fairness and standardisation of test-taking conditions were ranked as the most important factors influencing student satisfaction, followed closely by familiarity with the format. In contrast, technical issues, lack of transparency about changes, perceived unfairness around invigilation, and uncertainty around changes in assessment format and weighting contributed to dissatisfaction.

CONCLUSIONS

Online open-book assessments were seen as the most ideal amongst all participants, and students who experienced these were the most satisfied with their assessment change. They were perceived as most fair and authentic compared to real-life medical training. We seek to inform educators about student perceptions of successful assessment strategies under COVID-19 restrictions and provide evidence to allow debate on ongoing assessment reform and innovation. While this work looks specifically at assessment changes during COVID-19, understanding factors affecting student perception of assessment is applicable to examinations beyond COVID-19.

Harnessing the disruption on medical trainee education due to COVID-19 in New South Wales, Australia

The coronavirus disease (COVID-19) pandemic has caused disruption and uncertainty for junior medical doctor training and education. This has compounded the existing stress experienced by this cohort. However, by choosing appropriate educational models, as well as using novel educational approaches and advancing our online technology capabilities, we may be able to provide acceptable and even, superior solutions for educational training moving forward, as well as promote trainee wellbeing during these uncertain times.

How far has the digitization of medical teaching progressed in times of COVID-19? A multinational survey among medical students and lecturers in German-speaking central Europe

BACKGROUND

To ensure successful medical education despite the COVID-19 pandemic, the demand for online instruction has substantially increased. Fast and efficient teaching in a digital format poses a great challenge for medical students and lecturers as well as the universities.

OBJECTIVE

The aim of this study is to capture the readiness of medical students and faculty members to participate in rapidly- evolving online education.

METHODS

This cross-sectional study is based on two questionnaires distributed among medical students and associate deans for education in Germany, Austria and Switzerland. Questions included decision- making questions, categorical questions, and open-ended questions, all addressing the frequency and format of the digital education offered, the perceived quality of digital education, and medical student satisfaction with digital education. Questions about missing content and areas for improvement from the perspectives of medical students were included. The associate deans were asked for their opinions about the impact of the pandemic on teaching, the organizational setup and implementation of digital education by universities, and plans for future initiatives.

RESULTS

Three thousand and thirty medical students (m = 752 and f = 2245) from 53 universities participated in the study. The study showed that 92% of students were affected by the pandemic, and 19% of the students viewed the changes as entirely negative. 97% of the medical students were able to participate in digital courses, but only 4% were able to learn exclusively online. For 77% of the medical students, digital offerings accounted for over 80% of the education offered. In terms of content, medical students complained about a lack of practical teaching, such as contact with patients, lecturers, fellow medical students, and a poor perceived quality of teaching due to dubbing, frequent changeover of seminars, problem-oriented learning groups and in-person teaching, a lack of interaction possibilities and a lack of technical equipment, such as lecturers' knowledge and server capacities, at the universities. Overall, almost half of the medical students (42%) rated the implementation of digital teaching at their universities as good or very good. Forty-one of the 53 associate deans responded to the questionnaire, and 35 felt medical education was influenced by the pandemic. The associate deans (80%; 33/41) felt that the digitalization of medical education was negatively influenced by the pandemic. Only 44% (18/41) felt that their universities were well or very well positioned for digital teaching. All the associate deans believe that digital teaching in medicine will continue after the pandemic.

CONCLUSIONS

In the German-speaking world, the rapid conversion of medical teaching to a digital format has been well implemented in many cases. The perceived quality of the implementation of digital education still lacks practical relevance and the use of new digital media, such as learning games, VR, and online question time. The digital format of medical education will likely continue beyond the COVID-19 pandemic.

Harnessing the disruption on medical trainee education due to COVID-19 in New South Wales, Australia

The coronavirus disease (COVID-19) pandemic has caused disruption and uncertainty for junior medical doctor training and education. This has compounded the existing stress experienced by this cohort. However, by choosing appropriate educational models, as well as using novel educational approaches and advancing our online technology capabilities, we may be able to provide acceptable and even, superior solutions for educational training moving forward, as well as promote trainee wellbeing during these uncertain times.

Immediate consequences and solutions used to maintain medical education during the COVID-19 pandemic for residents and medical students: a restricted review

BACKGROUND

The COVID-19 outbreak has dramatically impacted medical education, both bedside and academic teaching had to be adapted to comply with the reorganisation of care and social distancing measures.

OBJECTIVES

To overview the impact of the pandemic on medical education, including the pedagogical responses adopted and their assessment by medical students and residents.

MATERIAL AND METHODS

This restricted systematic review was performed using Rayyan QCRI, to select observational or interventional articles and field experience reports assessing the impact of the COVID-19 pandemic on medical education for medical students and residents. Study design, study population, geographical origin, use of an educational tools (including softwares and social media), their type and assessment, were recorded. For studies evaluating a specific tool the Medical Education Research Study Quality Instrument (MERSQI) was used to assess study quality.

RESULTS

The literature search identified 1480 references and 60 articles were selected. Most articles focused on residents (41/60; 69%), and half (30/60; 50%) involved surgical specialties. Online courses were the most frequently used pedagogical tool (52/60; 88%). Simulation tools were used more frequently in articles involving surgical specialties (15/29; 52%) compared with medical specialties (2/14; 12%) (p=0.01). Only four studies reported the assessment of pedagogical tools by medical students, their MERSQI scores ranged from 5.5/18 to 9.0/18.

CONCLUSION

Medical education was highly impacted by the COVID-19 pandemic particularly in surgical specialties. Online courses were the most frequently attempted solution to cope with social distancing constraints. Medical students' assessment of pedagogical tools was mostly positive, but the methodological quality of those studies was limited.

Assessment of factual recall and higher-order cognitive domains in an open-book medical school examination

Open-book examinations (OBEs) will likely become increasingly important assessment tools. We investigated how access to open-book resources affected questions testing factual recall, which might be easy to look-up, versus questions testing higher-order cognitive domains. Few studies have investigated OBEs using modern Internet resources or as summative assessments. We compared performance on an examination conducted as a traditional closed-book exam (CBE) in 2019 (N = 320) and a remote OBE with free access to Internet resources in 2020 (N = 337) due to COVID-19. This summative, end-of-year assessment focused on basic science for second-year medical students. We categorized questions by Bloom's taxonomy ('Remember', versus 'Understand/Apply'). We predicted higher performance on the OBE, driven by higher performance on 'Remember' questions. We used an item-centric analysis by using performance per item over all examinees as the outcome variable in logistic regression, with terms 'Open-Book, 'Bloom Category' and their interaction. Performance was higher on OBE questions than CBE questions (OR 2.2, 95% CI: 2.14-2.39), and higher on 'Remember' than 'Understand/Apply' questions (OR 1.13, 95% CI: 1.09-1.19). The difference in performance between 'Remember' and 'Understand/Apply' questions was greater in the OBE than the CBE ('Open-Book' * 'Bloom Category' interaction: OR 1.2, 95% CI: 1.19-1.37). Access to open-book resources had a greater effect on performance on factual recall questions than higher-order questions, though performance was higher in the OBE overall. OBE design must consider how searching for information affects performance, particularly on questions measuring different domains of knowledge.

Are remote clinical assessments a feasible and acceptable method of assessment? A systematic review

The COVID-19 pandemic presented an enormous and immediate challenge to assessing clinical skills in healthcare professionals. Many institutions were unable to deliver established face-to-face assessment methods such as Objective Structured Clinical Examinations (OSCEs). Assessors needed to rapidly institute alternative assessment methods to ensure that candidates met the clinical competences required for progression. Using a systematic review, we aimed to evaluate the feasibility, and acceptability of remote methods of clinical skills assessment, including remote structured clinical assessments and the submission of video recordings. We searched for studies reporting on Remote Clinical Assessments or its variants in MEDLINE, Embase and The Cochrane library from 2000 to March 2021. Twenty eight studies were included in the review; 20 studies related to remote structured clinical examinations or OSCEs and 8 reported the use of video submissions. The participants of the different studies included medical students, nursing students, dental students and doctors in training. A variety of different online platforms were utilised including Zoom, Skype, webcams, and Adobe Connect online. The studies found that delivery of remote clinical assessments is possible and provides an alternative method of assessing many clinical skills, but most also acknowledge limitations and challenges. They are acceptable to both candidates and examiners, and where measured, show moderate agreement with on-site clinical assessments. Current evidence is based on studies with low methodological quality and for the most part, small sample sizes.

The Impact of the COVID-19 Pandemic on Medical Interns' Education, Training, and Mental Health: A Cross-Sectional Study

Background and objectives

The novel coronavirus disease 2019 (COVID-19) pandemic has challenged healthcare systems worldwide. Various studies have revealed the negative impact of the pandemic on the education and mental health of medical students and residents. In this study, we aimed to explore the effects of the COVID-19 pandemic on medical interns’ educational experience, clinical practice, and mental health. We also engage in a discussion on the compensatory methods that have been adopted to improve medical interns’ learning processes during the ongoing pandemic.

Methods

This cross-sectional survey-based study was conducted at the King Saud University Medical City (KSUMC) in Riyadh, Kingdom of Saudi Arabia (KSA) from March to October 2020. The participants consisted of medical interns. The survey collected information on participants’ demographics, training, and the educational and psychological impact of the pandemic.

Results

The survey was distributed to 480 medical interns, of whom 345 (71.8%) participated. Most of our sample (75.6%) believed that the COVID-19 pandemic has negatively impacted their cumulative experience. Regarding the pandemic’s impact on training and education, it was found that the emergency department rotation was the most affected rotation (60.3%). The majority of the participants (55.9%) believed that all tracks have received equal education and the volume of patients seen by interns decreased by 91.9%. As for the compensatory methods, 73.3% reported the utilization of remote platforms. Regarding the pandemic’s mental health impact, 36% believed that it has affected their mental health, where increased stress levels were noticed in 47.8% of participants. Statistical significance was found in both Patient Health Questionnaire-9 (PHQ-9) and General Anxiety Disorder-7 (GAD-7) scores, which indicated that during the pandemic, 20.6% of the participants suffered from depression, and 13.9% had moderate to severe anxiety.

Conclusions

This study highlights the negative educational and psychological impact of COVID-19 on medical interns. With emerging infectious diseases on the rise, recognizing the impact of COVID-19 on medical interns is vital to improving interns’ educational experiences and mental health during future crises.

Emerging from emergency pandemic pedagogy: A survey of anatomical educators in the United Kingdom and Ireland

It is critical that academic opinion of pandemic pedagogy is comprehensively quantified in order to inform future practices. Thus, this study examines how anatomists in the United Kingdom (UK) and Republic of Ireland (ROI) perceive the teaching adaptations made in response to COVID-19, and how these adaptations have impacted their experiences teaching, their online work environment and community. Data was collected via a questionnaire from 24 anatomists across 15 universities in the UK (11) and ROI (4). With regards to teaching, 95.6% of academics have upskilled in new technologies to meet the demands of distance teaching. Academics (95.8%) preferred face-to-face delivery of practical sessions. Most universities (80.0%) reported that practical sessions will continue in a new form that ensures social distancing. However, 50.0% of academics are uncertain if these adaptations will improve student learning. Many anatomists believe that the new adaptations may hinder student-student (66.7%) and student-tutor (45.8%) interactions. Regarding assessment, 52.6% of academics preferred traditional methods to online. Remote online assessment was difficult to protect against collusion, but provided time saving opportunities for academics. Finally, in terms of working environment, 83.3% of academics stated that their workload increased; 54.2% preferred working on site rather than remotely and 79.2% think that staff interactions are better when working on site. These results demonstrate a widespread concern amongst anatomists regarding the pandemic-induced adaptations to teaching, assessment and working environment. However, important opportunities were also identified that could ultimately serve to benefit students and educators alike.

Virtual exams: has COVID-19 provided the impetus to change assessment methods in medicine?

AIMS

The ongoing COVID-19 pandemic has disrupted and delayed medical and surgical examinations where attendance is required in person. Our article aims to outline the validity of online assessment, the range of benefits to both candidate and assessor, and the challenges to its implementation. In addition, we propose pragmatic suggestions for its introduction into medical assessment.

METHODS

We reviewed the literature concerning the present status of online medical and surgical assessment to establish the perceived benefits, limitations, and potential problems with this method of assessment.

RESULTS

Global experience with online, remote virtual examination has been largely successful with many benefits conferred to the trainee, and both an economic and logistical advantage conferred to the assessor or organization. Advances in online examination software and remote proctoring are overcoming practical caveats including candidate authentication, cheating prevention, cybersecurity, and IT failure.

CONCLUSION

Virtual assessment provides benefits to both trainee and assessor in medical and surgical examinations and may also result in cost savings. Virtual assessment is likely to be increasingly used in the post-COVID world and we present recommendations for the continued adoption of virtual examination. It is, however, currently unable to completely replace clinical assessment of trainees. Cite this article: Bone Jt Open 2021;2(2):111-118.

A Near-Peer Educational Model for Online, Interactive Learning in Emergency Medicine

Introduction

The COVID-19 pandemic led to a large disruption in the clinical education of medical students, particularly in-person clinical activities. To address the resulting challenges faced by students interested in emergency medicine (EM), we proposed and held a peer-led, online learning course for rising fourth-year medical students.

Methods

A total of 61 medical students participated in an eight-lecture EM course. Students were evaluated through pre- and post-course assessments designed to ascertain perceived comfort with learning objectives and overall course feedback. Pre- and post-lecture assignments were also used to increase student learning.

Results

Mean confidence improved in every learning objective after the course. Favored participation methods were three-person call-outs, polling, and using the “chat” function. Resident participation was valued for “real-life” examples and clinical pearls.

Conclusion

This interactive model for online EM education can be an effective format for dissemination when in-person education may not be available.