Medical students' perception of the reliability, usefulness and feasibility of unproctored online formative assessment tests

Efficacy of Google Form-based MCQ tests for formative assessment in medical biochemistry education

BACKGROUND

Biochemistry, being a vast and complex subject, can be challenging for Phase I MBBS students to comprehend and retain. Embracing rapidly evolving technology can facilitate a more accessible learning experience. In this study, we investigated the effectiveness of using Google Form-based multiple-choice question (MCQ) tests as a formative assessment tool after each biochemistry lecture series. The aim was to assess the improvement and gather feedback of Phase I MBBS students on the utility of this assessment tool.

MATERIALS AND METHODS

This educational prospective longitudinal study was conducted by the Department of Biochemistry at a university-affiliated medical college and tertiary care hospital. The study included 150 Phase I MBBS students as participants. Google Form-based MCQ tests were implemented as educational interventions after each lecture series during the study period. The study compared the internal assessment (IA) MCQ marks of students before and after the implementation of the intervention. In addition, feedback questionnaires were collected from the students.

RESULTS

There was a significant improvement in students' scores between the first IA (mean ± standard deviation [SD], 8.16 ± 3.08) and second IA (mean ± SD, 17.64 ± 2.02) (P < 0.0001). According to students' feedback, 149 out of 150 (99.3%) students found the use of Google Form-based MCQ tests as a formative assessment tool in the teaching-learning process to be highly beneficial and motivated them to engage in their biochemistry studies.

CONCLUSION

With the shift toward competency-based medical education (CBME) in India, it is crucial for educators to embrace novel teaching-learning and evaluation approaches. Our study highlighted the efficacy of employing Google Form-based MCQ tests in enhancing students' comprehension of the biochemistry subject, evaluating their scores and improving the overall quality of learning. Through this mode of assessment, teachers were able to provide targeted feedback on areas that required improvement, thereby enhancing the learning experience.

Collaborative Assessment in the Gross Anatomy Lab

Purpose

In 2018, the Michigan State University College of Human Medicine incorporated two-stage examinations into the gross anatomy curriculum. Multiple studies have investigated two-stage examinations and have largely reported positive findings. Here, we used a mixed-methods approach to further investigate the feasibility and student perceptions of the two-stage examination in the context of a medical school curriculum that emphasizes longitudinal group-based learning and formative assessments.

Methods

Three student cohorts were assessed with a formative two-stage examination at the end of their first-year anatomy experience. Data for the quantitative analysis included examination scores from the individual and group portions of the two-stage examination. For the qualitative stage of this project, we utilized a constructivist grounded theory methodology in which data, including both post-examination survey results and one-on-one semi-structured student interviews, were transcribed (interviews), coded, inductively and iteratively reviewed, and thematically interpreted.

Results

Survey and interview results revealed an overwhelmingly positive perception of the collaborative assessment experience. Student comments demonstrated educational value in the immediate feedback provided by this examination format and suggested that collaboration during the examination transformed the assessment into a learning experience.

Conclusions

While two-stage examinations have the potential to positively transform an assessment into a learning experience, we also identified complex relationships between content knowledge and anxiety that may affect student perceptions. In addition, examination logistics (e.g., curricular timing) have the potential to negatively affect student perceptions, indicating that faculty should consider these factors when implementing collaborative assessments into their curriculum.

Perception and preference of pharmacy students toward e-learning during the COVID-19 pandemic

This study is designed to evaluate students' knowledge and perceptions about the online learning of pharmacy curricula in Iraq during the COVID-19 quarantine. A descriptive, cross-sectional study involving 278 pharmacy students was done between October 2020 and June 2021. About 42.44% preferred face-face lecturers over other modes of delivery for lectures in the pharmacy curriculum. Most participants preferred both active learning and face-face lectures. The results show that 72.66% of responders chose to stay at home as one of the privileges of e-learning. However, the main barriers that associated with e-learning were lack of patient involvement and some technical issues regarding IT equipment, (74.82%) and (62.23%), respectively. E-learning is seen as a lack of social presence, less social contact, and synchronization of connections. However, e-learning undoubtedly benefits students in several ways. Online learning is an essential podium for students to achieve their studies in periods of crisis.

Formative assessments during COVID-19 pandemic: an observational study on performance and experiences of medical students

Background: Because of COVID-19, the 2020 written medical examinations were replaced by mandatory formative online assessments. This study aimed to determine students' performance, self-assessment of performance, and perception about the switch from a summative to a formative approach. Methods: Medical students from year 2 to 5 (n=648) were included. They could repeat each test once or twice. They rated their performance after each attempt and were then given their score. Detailed feedback was given at the end of the session. An online survey determined medical students' perception about the reorganization of education. Two items concerned the switch from summative to formative assessments Results: Formative assessments involved 2385 examinees totaling 3197 attempts. Among examinees, 30.8% made at least 2 attempts. Scores increased significantly at the second attempt (median 9.4, IQR 10.8), and duration decreased (median -31.0, IQR 48.0). More than half of examinees (54.6%) underestimated their score, female students more often than male. Low performers overestimated, while high performers underestimated their scores. Students approved of the switch to formative assessments. Stress was lessened but motivation for learning decreased. Conclusions: Medical students' better scores at a second attempt support a benefit of detailed feedback, learning time and re-test opportunity on performance. Decreased learning motivation and a minority of students repeating the formative assessments point to the positive influence of summative assessment on learning.

Design of online formative assessment of nursing humanities curriculum during the COVID-19 pandemic: A teaching practice research

BACKGROUND

Under the digital transformation trend nursing education, online formative assessment (OFA) provides a new opportunity. However, the OFA of nursing humanities course lacks design and practice, and faces the challenge of enhancing effective communication between teachers and students, student participation and autonomous learning.

OBJECTIVES

To enhance the reliability of OFA in nursing humanities courses and provide practical experience for online teaching in the nursing profession.

DESIGN

A quantitative research approach was used.

SETTING

This study was conducted in a comprehensive university in China.

PARTICIPANTS

We conducted teaching practice on 185 nursing undergraduates, with 89 students in the experimental group, and 96 students in the control group.

METHODS

In the 2020-2021 multicultural nursing course, student learning outcomes and questionnaires were analyzed through the online learning tool Superstar Learning, student feedback and satisfaction questionnaires, and descriptive analysis and independent sample t-tests were conducted using SPSS 25.0 software.

RESULTS

The OFA of students using Superstar Learning differed in learning performance and time to receive feedback from teachers between the experimental and control groups, and both groups had higher satisfaction levels. The experimental group's instructional design contained a synchronous classroom discussion module with better participation.

CONCLUSIONS

During the COVID-19 pandemic, the use of online learning tools can support the implementation of OFA, build an environment where teachers and students participate together, have a positive impact on the continuous updating of teachers' teaching programs and students' learning outcomes. Simultaneous classroom discussions are expected to be an effective way to improve the reliability of OFA. Our instructional design, provides best practice suggestions for future online teaching and learning.

Online practical assessment in biochemistry designed on Bloom's taxonomy to assess the student's competency: Prior exposure of the students to online tools beneficial

Online assessments are needed during the prevailing pandemic situation to continue educational activities while ensuring safety. After conducting the online practical assessment (OPrA) in Biochemistry, we analyzed the students' responses. The blueprint of the OPrA was prepared by the faculty, referring to the various levels and domains of Bloom's taxonomy. Four components were chosen for the online assessment: digital spotters, enumerating the steps of objective structured practical examination, interpretation of quantitative estimation, and case discussion. Each faculty assessed about 12-13 students in separate breakout rooms over 15-20 min on all four components. Feedback on the conduct of the examination was collected from the students and faculty anonymously and analyzed. Out of the 200 students who attended the online assessment, only one scored less than 50%, majority of them scored between 71% and 90%. Under the individual exercises, the average score of students in "Spotters" was 9.8 out of 10; in "OSPE," 8.7 out of 10; in "Quantitative experiments," 15.2 out of 20 and in "Case discussion," 22.4 out of 30. Around 20% had previous experience attending the OPrA. They differed in their opinion from the rest of the students on five aspects; time allotted for the assessment (p value = 0.02, χ²  = 5.07), students using unfair means during the online viva (p value = 0.02, χ²  = 5.57), their computing skills (p value = 0.001, χ²  = 19.82), their performance (p value = 0.001, χ²  = 8.84), and overall conduct of the examination (p value = 0.001, χ²  = 15.55). OPrA tools may be designed referring to Bloom's taxonomy, and prior exposure to the online tools may benefit the students.

Understanding the Functional Components of Technology-Enhanced Learning Environment in Medical Education: A Scoping Review

Increasing use of technology in medical education has caused concerns to medical teachers pertaining to the quality of digital learning environments. Thus, this review aimed to unearth the functional components of effective technology-enhanced learning environment in the undergraduate medical education context. The revised Arksey and O'Malley protocol was utilized that include identification of research question and relevant studies, selection of studies, data charting and collection, and collating, summarizing, and reporting results after consultation. We discovered nine components with 25 subcomponents of 74 functional elements found to be present in effective online learning environments. The nine components include cognitive enhancement, content curation, digital capability, technological usability, pedagogical practices, learner characteristics, learning facilitator, social representations, and institutional support. There is an interplay between these components, influencing each other in online learning platforms. A technology-enhanced learning in medical education (TELEMEd) model is proposed which can be used as a framework for evaluating online learning environment in medical education.

Supplementary Information

The online version contains supplementary material available at 10.1007/s40670-023-01747-6.

Using live stream technology to conduct workplace observation assessment of trainee dental nurses: an evaluation of effectiveness and user experience

AIM/OBJECTIVE

This study evaluates the effectiveness and users' experience of using live stream technology to conduct workplace observation assessments of trainee dental nurses. Information on the usability, accessibility, and general satisfaction of this technological technique were collected.

MATERIALS AND METHODS

This nationwide cross-sectional survey was conducted in Scotland and included one focus group and three online questionnaires with qualitative and quantitative questions. The quantitative responses were described using standard descriptive analysis, while the quantitative data were investigated using thematic analysis.

RESULTS

Eighty-one trainee dental nurses, 35 clinicians and 19 assessors participated in this study. Live stream observation was generally well received by the trainee dental nurses and clinicians, who thought that it had helped increase their confidence to perform practical skills. The assessors also stated that overall satisfaction was high, and that live stream observation met their expectations for efficacy. However, several technical challenges, such as network issues were brought up by responders.

CONCLUSION

This study provides evidence that workplace observation assessments can be performed in the future by using live stream technology. However, additional investigation and comparison will aid in determining the most effective way of using this approach and providing feedback to promote learning among dental trainees.

Formative assessments during COVID-19 pandemic: an observational study on performance and experiences of medical students

Background: Because of COVID-19, the 2020 written medical examinations were replaced by mandatory formative online assessments. This study aimed to determine students’ performance, self-assessment of performance, and perception about the switch from a summative to a formative approach. Methods: Medical students from year 2 to 5 (n=648) were included. They could repeat each test once or twice. They rated their performance after each attempt and were then given their score. Detailed feedback was given at the end of the session. An online survey determined medical students’ perception about the reorganization of education. Two items concerned the switch from summative to formative assessments Results: Formative assessments involved 2385 examinees totaling 3197 attempts. Among examinees, 30.8% made at least 2 attempts. Scores increased significantly at the second attempt (median 9.4, IQR 10.8), and duration decreased (median -31.0, IQR 48.0). More than half of examinees (54.6%) underestimated their score, female students more often than male. Low performers overestimated, while high performers underestimated their scores. Students approved of the switch to formative assessments. Stress was lessened but motivation for learning decreased. Conclusions: Medical students’ better scores at a second attempt support a benefit of detailed feedback, learning time and re-test opportunity on performance. Decreased learning motivation and a minority of students repeating the formative assessments point to the positive influence of summative assessment on learning.

Medical student exam performance and perceptions of a COVID-19 pandemic-appropriate pre-clerkship medical physiology and pathophysiology curriculum

BACKGROUND

Medical schools were compelled to abruptly transition pre-clerkship curricula to remote learning formats due to the emergence of the Coronavirus Disease 2019 (COVID-19) pandemic. We evaluated student perceptions of remote learning, exam performance, and utilization of third-party learning resources to assess the implementation of a newly developed pandemic-appropriate physiology curriculum.

METHODS

This was an observational study based on a survey conducted in the Spring of 2021 at the University of California, Irvine, School of Medicine (UCISOM). This study aimed to assess first (MS1) and second year (MS2) medical students' perceptions of satisfaction, support, academic performance, and connectedness before and during the COVID-19 pandemic. The MS1 class began medical school during the first year of the COVID-19 pandemic, whereas the MS2 class did so prior to the start of the pandemic. A survey instrument was developed and validated to identify the impact remote learning had on student self-perceptions of the Medical Physiology and Pathophysiology course. Surveys were distributed to all students and responses were collected on a voluntary basis. Exam scores on a customized National Board of Medical Examiners (NBME) physiology shelf exam were also compared to objectively identify how the remote curriculum during the pandemic impacted academic performance.

RESULTS

Of 204 students enrolled, 74 responses were analyzed, with 42 MS1 (40% of MS1s) and 32 MS2 (31% of MS2s) responses. Overall, MS1s and MS2s were satisfied with the curriculum they received (95 and 97% respectively) and the school's support of their concerns (86 and 100% respectively). Notably, only 50% of MS1s felt connected to their peers, compared to 94% of MS2s. Lecture attendance and self-perception of their academic performance were similar between both classes. Interestingly, the intra-pandemic class's NBME exam average in 2020 (60.2% ± 8.9, n = 104) was significantly higher than the pre-pandemic class average in 2019 (56.8% ± 11.3, n = 100). Both classes primarily used course materials over third-party learning resources. An additional set of survey questions distributed only to the MS1 class found that the majority of MS1s reported minimal barriers with regards to accessibility, including internet connectivity, study-conducive environments, and balancing family commitments. Overall, pre-clerkship medical students had positive perceptions of the newly developed pandemic-appropriate physiology curriculum.

CONCLUSIONS

Changes to the pre-clerkship physiology curriculum during the COVID-19 pandemic were met with overall satisfaction from the students and an increase in NBME scores. More attention to student connectedness is needed to improve how remote learning can be best optimized into future curricula development.

With the New Challenges of COVID- 19, Can We Depend Entirely on Online Learning for Higher Education in the Medical Schools?

BACKGROUND

Due to the COVID-19 pandemic, medical education has gone online. Human anatomy is a three-dimensional subject that requires a clear understanding of the relationships between structures through the study of human cadavers, microscopic samples, and models.

OBJECTIVE

This study aimed to examine the attitudes and perceptions of students regarding teaching anatomy before and after the lockdown period and ways of improving the challenges they faced.

METHODS

This cross-sectional study was conducted at AL Imam University School of Medicine. Structured surveys were designed to understand students' attitudes and opinions towards offline versus online teaching of anatomy and suggestions to improve the online teaching process.

RESULTS

The mean attitude scores for the conventional lectures, seminars, and practical sessions were significantly higher. Students expressed positive attitudes towards the three teaching modules. Students agreed that themes related to computer skills and technical Internet infrastructure (71.55%) and administrative procedures (61.74%) are essential for improving the online teaching process.

CONCLUSION

Conventional learning is the main target of student learning. Improvements in computer skills, technical Internet infrastructure, learning resources, staff communication, examinations, and development of blended learning will enrich the learning process, especially during the impending challenges.

Scoping review of published research on medical education in India during the Covid-19 pandemic

Background Medical educators in India made rapid adjustments to maintain continuity and integrity of medical education in the midst of disruption caused by the Covid-19 pandemic. However, there are concerns regarding achievement of competence by undergraduate medical students due to inadequate clinical exposure. We explored the focus of initiatives from medical educators in India by a scoping review of published articles on developments in medical education during the pandemic to map concepts, main sources and the literature available in PubMed. Methods We did this scoping review of published articles in PubMed database in four steps: (i) identification of research questions; (ii) identification of relevant studies; (iii) selection of studies meeting inclusion and exclusion criteria, and charting of data; and (iv) collating the summary and reporting of results. Manual content analysis was done to derive frequencies of variables. Results Of the 52 articles identified, 22 met the requirements. Most studies (68.2%) were published in 2020. Half of the studies were conducted among undergraduate students and the remaining among postgraduates (27.3%), faculty (18.2%) and interns (4.5%). All the studies were evaluations at Kirkpatrick level-1 (18; 81.8%) and level-2 (4; 18.2%). Most of the studies (9, 41%) focused on exploration of perspectives about online learning among students and faculty, 9 (27.3%) on teaching- learning, 4 (18.2%) on formative assessment and 3 (13.6%) on summative assessment. Conclusions Most studies were evaluations at Kirkpatrick level-1 and level-2 among undergraduate medical students with a focus on conceptual understanding.

Assessment of the effectiveness of BOPPPS-based hybrid teaching model in physiology education

BACKGROUND

Online teaching has become increasingly common in higher education of the post-pandemic era. While a traditional face-to-face lecture or offline teaching remains very important and necessary for students to learn the medical knowledge systematically, guided by the BOPPPS teaching model, combination of online and offline learning approaches has become an unavoidable trend for maximizing teaching efficiency. However, in physiological education, the effectiveness of combined online teaching and offline teaching models remains poorly assessed. The present study aims at providing an assessment to the hybrid teaching model.

METHODS

The study was performed among undergraduate medical students of Class 2017 ~ 2019 in the Physiology course in Harbin Medical University during 2018-2020. Based on established offline teaching model with BOPPPS components in 2018, we incorporated online teaching contents into it to form a hybrid BOPPPS teaching model (HBOPPPS, in brief), preliminarily in 2019 and completely in 2020. HBOPPPS effectiveness was assessed through comparing the final examination scores of both objective (multi-choice and single answer questions) and subjective (short and long essays) questions between classes taught with different modalities.

RESULTS

The final examination score of students in Class 2019 (83.9 ± 0.5) who were taught with the HBOPPPS was significantly higher than that in Class 2017 (81.1 ± 0.6) taught with offline BOPPPS and in Class 2018 (82.0 ± 0.5) taught with immature HBOPPPS. The difference mainly attributed to the increase in average subjective scores (41.6 ± 0.3 in Class 2019, 41.4 ± 0.3 in Class 2018, and 38.2 ± 0.4 in Class 2017). In the questionnaire about the HBOPPPS among students in Class 2019, 86.2% responded positively and 79.4% perceived improvement in their learning ability. In addition, 73.5% of the students appreciated the reproducibility of learning content and 54.2% valued the flexibility of HBOPPPS. Lastly, 61.7% of the students preferred the HBOPPPS relative to BOPPPS in future learning.

CONCLUSIONS

HBOPPPS is likely a more effective teaching model and useful for enhancing effectiveness of Physiology teaching. This is attributable to the reproducibility and flexibility as well as the increased learning initiatives.

Students' and Teachers' Perspective on the Implementation of Online Medical Education in China: A Qualitative Study

OBJECTIVE

Due to the quarantine measures during the outbreak of COVID-19, medical schools in China had to shift to online education overnight. Researchers wanted to survey the online medical education status quo and find out the underlying challenges and probable solutions in terms of online medical education to better understand improving and developing medical education in China.

METHODS

Researchers distributed self-administered and piloted surveys regarding the implementation of online medical education on WeChat and Wenjuanxing. Researchers distributed three phases of online surveys to the undergraduate medical students, and one online survey to the medical teachers on phase three.

RESULTS

Online medical education was carried out smoothly in China. Seventy-six percent of the students were satisfied with online medical education, and it hardly changed after a month. Courses with the quiz and live-stream courses were rare compared with courses incorporated with other elements. Most parents would remind their children to study online at home. At first, the biggest challenge of learning online was the lagging platform. Nevertheless, as time went by, the major challenge became the learning motivation. Most students thought it necessary to re-teach face-to-face after online education, while most teachers did not think so.

CONCLUSION

Generally speaking, online medical education was satisfying for both students and teachers, although online courses' diversity with different forms and elements could be enhanced. Infrastructure construction should be considered first for the schools intended to promote online medical education. If the infrastructure were ready, learning motivation would become the biggest challenge for online medical education. Online medical education efficacy evaluation tools need to be developed in the future to narrow the discrepancy of the evaluation between teachers and students.

Distance Learning in Surgical Education

BACKGROUND

Medical education has traditionally relied on in-person-based curriculums in medical school and residency . However, due to the COVID19 pandemic, medical schools and residency programs have been forced to rapidly transition to virtual platforms for learning. Surgical education poses a particular challenge, as virtual platforms cannot adequately replace hands-on learning of surgical skills. In this review, we will discuss the various ways in which virtual learning has been employed in surgical education and how it may be used to enhance learning of medical students and residents in the future.

METHODS

We conducted a comprehensive literature search to identify articles published regarding medical school and surgical residency curriculum changes after COVID19.

RESULTS

Over the past year, several surgery departments have piloted programs using virtual learning modules, live online lectures and training workshops, and remote streaming into the OR to supplement more traditional in-person learning. Overall, these programs have received positive feedback from participating medical students and residents, suggesting that virtual and online tools may be helpful in supplementing surgical education. However, several programs also noted the possibility for significant disparities in learning due to variable access to internet and availability of newer technologies.

CONCLUSION

Going forward, distance learning will play an important role in surgical education to further enhance learning of medical students and residents in a field with rapid technological advancements.