The impact of proctored online exams on the educational experience

E-learning in medical education: a perspective of pre-clinical medical students from a lower-middle income country

BACKGROUND

Many of the educational institutions in developed countries have shifted to online learning. While transition from traditional to electronic learning (e-learning) has remained a great challenge in low-middle income countries, where limited resources for teaching and learning are important factors. Medical education involves not only lecturing but also deep understanding through laboratories and patient exposure. The debate about the effectiveness of e-learning in medical education is still in contradiction due to its limitations. This cross-sectional survey was conducted to assess pre-clinical undergraduate medical students' perception of their first online learning in a lower-middle income country.

METHODOLOGY

The survey was conducted among the students who had participated in online learning during COVID-19 for at least a year. A total of 824 preclinical medical students who completed the survey from public and private medical universities in Sindh, Pakistan were included in the study. We used a validated online-based questionnaire, distributed through E-mail and social media platforms to assess the perception of students regarding their first online learning experience.

RESULTS

The response rate of the survey was 87.9%. The mean age of students was 20.7 ± 3.8 years. 392/824 (47%) were males and 57% were females. Our study indicated that 613/824 (75%) of students were experiencing online learning for the very first time while 631/824 (77%) were facing technical issues like internet accessibility and lack of IT-related skills. 381/824 (46%) were not satisfied with the institute's readiness for online teaching. However, 79% (654/824) of participants were of the idea that traditional learning is more effective in developing their practical skills as compared to e-learning. Of note, 668/824 (81%) showed overall dissatisfaction with e-learning.

CONCLUSION

Based on our study findings, we concluded that most students have a negative perception of e-learning. Difficulty in connectivity, electricity issues, less interaction with colleagues and teachers, and issues with the structure of online courses were the most frequently reported problems by the students.

Global perspectives of the impact of the COVID-19 pandemic on learning science in higher education

The COVID-19 pandemic required higher education institutions to rapidly transition to Emergency Remote Instruction (ERI) with little preparation. Discussions are now underway globally to learn the lessons of COVID-19 and to use this knowledge to shape the future of learning science in higher education. In this study, we examined the experiences of instructors and students to ERI in three universities across three continents-America, Europe, and Australia. We measured the instructional strategies used by instructors including assessment types, and interaction opportunities during and outside class schedules. We also measured the learning challenges experienced by students including planning, distractions, technology, learning resources, their views on educational quality and what characterized quality interactions during ERI. Our findings suggest that most instructional strategies used by instructors changed little during ERI, although the nature of instructor and student interactions during class relied more heavily on technology. Students reported significant learning challenges which included distractions from their physical and social media environments and access to technology. Both instructors and students reported that interactions with each other and their peers were concerningly low, albeit similar to pre COVID-19 pandemic levels. There were differences in the perceptions of instructors and students on whether instructor-student interactions were better or worse online. Common among all universities, there was a large proportion of students reporting mental health and work-related stress. Lessons to be learned from the COVID-19 pandemic include ensuring more support for instructors to implement effective and equitable pedagogies and an increased recognition of the importance of practicals, and the social, interactive and hands-on aspects of learning science in higher education. We predict that the incorporation of active learning pedagogies and strategies which increase student engagement and foster a sense of belonging will be ongoing global challenges for learning science in a post COVID-19 campus.

An automated online proctoring system using attentive-net to assess student mischievous behavior

In recent years, the pandemic situation has forced the education system to shift from traditional teaching to online teaching or blended learning. The ability to monitor remote online examinations efficiently is a limiting factor to the scalability of this stage of online evaluation in the education system. Human Proctoring is the most used common approach by either asking learners to take a test in the examination centers or by monitoring visually asking learners to switch on their camera. However, these methods require huge labor, effort, infrastructure, and hardware. This paper presents an automated AI-based proctoring system- 'Attentive system' for online evaluation by capturing the live video of the examinee. Our Attentive system includes four components to estimate the malpractices such as face detection, multiple person detection, face spoofing, and head pose estimation. Attentive Net detects the faces and draws bounding boxes along with confidences. Attentive Net also checks the alignment of the face using the rotation matrix of Affine Transformation. The face net algorithm is combined with Attentive-Net to extract landmarks and facial features. The process for identifying spoofed faces is initiated only for aligned faces by using a shallow CNN Liveness net. The head pose of the examiner is estimated by using the SolvePnp equation, to check if he/she is seeking help from others. Crime Investigation and Prevention Lab (CIPL) datasets and customized datasets with various types of malpractices are used to evaluate our proposed system. Extensive Experimental results demonstrate that our method is more accurate, reliable and robust for proctoring system that can be practically implemented in real time environment as Automated proctoring System. An improved accuracy of 0.87 is reported by authors with the combination of Attentive Net, Liveness net and head pose estimation.

Modeling and analysis of barriers to ethics in online assessment by TISM and fuzzy MICMAC analysis

The pandemic of COVID-19 has altered the world canvas forever. The education sector, too, has been impacted by the same. There has been a phenomenal rise in e-platforms for teaching, learning, and evaluation. Teachers and students had to train themselves overnight to embrace the changing dynamics of the education sector. The change has been marked with challenges. In this new education landscape, online exams have occupied center stage. While the idea of giving exams from any part of the world welcomes freedom, it also raises concerns among faculty and students about academic integrity. Thus, as online studies and online assessment continue to expand, the paper aims to identify the factors responsible for unethical practices in online assessment. The paper further identifies the association between the identified factors. The paper proposes a four-level model that focuses on the lack of training for both faculty and students, interpersonal barriers, technological barriers, time management, personal ethics, and design of assessment as underlying reasons for unethical behavior in online assessments. The paper further explains the linkages using fuzzy MICMAC analysis. The results have both practical and social implications. Understanding the factors and their relationship with each other can help the instructors and administrators in their decision-making process regarding online evaluations and formulate policies that would instill strong ethical values, such as academic integrity and honesty, in their students throughout their academic journey.

Students' Performance in Face-to-Face, Online, and Hybrid Methods of Teaching and Assessment in Anatomy

In recent times, online teaching and assessment have provided a great opportunity to explore better methods in medical education. There are inconsistent views concerning the effectiveness of online assessment. Hence, the present study aimed to evaluate online teaching and assessment methods in relation to face-to-face methods by comparing students' performances. The students' assessment results in two basic anatomy courses, which are part of the Doctor of Medicine and Biomedical Sciences programs at Sultan Qaboos University, were analysed. We compared the students' mean scores and coefficient of variance in the multiple-choice written exams and the objective structured practical exams during the spring semesters of 2019, 2020, and 2021, containing face-to-face teaching and exams, partial online teaching and online exams, and online teaching and both proctored online and face-to-face exams, respectively. The sudden transition to online teaching and assessment halfway through the semester resulted in higher means and a lower coefficient of variance among students' scores in both theory and practical exams. However, when the fully adopted online method of teaching and assessment was employed, the mean scores decreased, and the coefficient of variance increased to figures close to those witnessed before the pandemic, when teaching and assessment were face-to-face. This trend applied to both the Doctor of Medicine and Biomedical Sciences programs' anatomy courses. The results indicate that online assessment of theoretical and practical anatomical knowledge is comparable to that of face-to-face assessment. However, proper planning and preparedness are mandatory to achieve the desired outcomes.

Changes in Academic Performance after Transitioning to Remote Proctoring: A Before-After Evaluation

Remote proctoring is often used to ensure testing integrity in a distance education environment but may impact academic performance. This quasi-experimental study aimed to evaluate changes in examination scores after transitioning to remote proctoring during the COVID-19 pandemic. Student pharmacists (n = 384) served as their own controls in this before-after analysis of examination scores with in-person versus remote proctoring. To assess differences in examination scores among students with varying levels of testing anxiety, students were classified into low, moderate, or high testing anxiety groups based on their Cognitive Test Anxiety Scale–Second Edition (CTAS-2) score. Students were also stratified into two groups based on their cumulative grade point average (GPA). After transitioning to remote proctoring, examination scores significantly decreased for first-year (P1) students but significantly increased for second-year (P2) students. When stratified by CTAS-2 score, no significant difference in examination scores was found. When stratified by GPA, no significant difference in examination scores was found for P1 students, but a significant improvement was noted for P2 students with remote proctoring. The results of this study indicate that examination scores do not consistently improve or decline after introducing remote proctoring even when considering a student’s GPA and level of testing anxiety.

Use of Alternative Methodologies in Veterinary Medicine Learning and Acceptance of Students

Different university degrees focus on students acquiring theoretical and practical knowledge, aiming to develop their professional activity in the future. However, the usual study plans often forget other skills that will be very useful for the correct performance of their professional activity. In the case of veterinarians, these can range from dialogue with farmers to the unification of knowledge, so that they can provide a simple and effective solution to the different questions that may arise throughout their work activity. On the other hand, the perception of the world and the ways of acquiring knowledge have been changing over the years. Currently, our students require new ways of being presented with the information and knowledge that they should acquire, using, in most cases, new technologies. The present study was carried out with two cases. First, we used gamification through role-play as an alternative methodology to generate a method to unify the knowledge acquired in the subject and, mainly, to acquire skills such as the transfer of this acquired knowledge to other classes and situations. The second case aims to verify if the use of new technologies, specifically the use of interactive videos, can improve the acceptance of students and their training. A total of 2 h of videos were recorded, and 31 min and 42 s of that footage were ultimately used. A special edition and some specific illustrations and designs were made for this work, taking care of the format–background relationship. The results obtained show that these alternative-learning methodologies could be applied to many subjects, so that students, in a playful and relaxed way, are able to unify all the knowledge they are acquiring in their training as veterinarians, preparing them to face the exercise of their future professional activity with greater ease and safety. Finally, we provide the degree of acceptance of these new learning methodologies by students.

Rethinking Assessment: The Future of Examinations in Higher Education

The global higher education landscape is significantly impacted as a result of the COVID-19 pandemic and the majority of the universities now follow an online or hybrid mode of delivery. This presents substantial challenges for universities, particularly to conduct examinations, as traditionally most exams were conducted physically on campus. During the first wave of the pandemic, many universities had no option and were forced to move online in a very short period of time, causing universities also to conduct exams online without transforming pedagogy and the structure/s of closed-book exams. Inevitably, in non-proctored and unregulated examinations, this allowed room for students to collaborate and share material during online exams without being noticed by an invigilator as in the case of physical exams. Online exams, also leave room for students to find information online which made preventing plagiarism a significant challenge. This paper investigates the practices used in both closed-book and open-book exams and identifies the challenges associated with the transition to online exams. It also identifies potential ways forward for future online exams, while minimizing opportunities for students to collaborate, plagiarise and use online material. The findings of this study reveal that online examinations affect teachers and students differently: while teachers have mixed feelings about online exams, students are anxious about their grades and the technical hassle they experience in online exams. While viva has emerged as a popular form of alternative assessment, students still feel the need of returning to physical exams. None of the teachers who participated in this study discussed a psychosocial approach to education and exams in this pandemic. We conclude this study on the note that there is a need for the collaboration of social scientists, psychologists, psychosocial specialists, educationists, and humanities scholars/humanists for better educational policy and pedagogical practices during the pandemic.

Remote versus on-site proctored exam: comparing student results in a cross-sectional study

BACKGROUND

The COVID-19 pandemic has profoundly affected assessment practices in medical education necessitating distancing from the traditional classroom. However, safeguarding academic integrity is of particular importance for high-stakes medical exams. We utilised remote proctoring to administer safely and reliably a proficiency-test for admission to the Advanced Master of General Practice (AMGP). We compared exam results of the remote proctored exam group to those of the on-site proctored exam group.

METHODS

A cross-sectional design was adopted with candidates applying for admission to the AMGP. We developed and applied a proctoring software operating on three levels to register suspicious events: recording actions, analysing behaviour, and live supervision. We performed a Mann-Whitney U test to compare exam results from the remote proctored to the on-site proctored group. To get more insight into candidates' perceptions about proctoring, a post-test questionnaire was administered. An exploratory factor analysis was performed to explore quantitative data, while qualitative data were thematically analysed.

RESULTS

In total, 472 (79%) candidates took the proficiency-test using the proctoring software, while 121 (20%) were on-site with live supervision. The results indicated that the proctoring type does not influence exam results. Out of 472 candidates, 304 filled in the post-test questionnaire. Two factors were extracted from the analysis and identified as candidates' appreciation of proctoring and as emotional distress because of proctoring. Four themes were identified in the thematic analysis providing more insight on candidates' emotional well-being.

CONCLUSIONS

A comparison of exam results revealed that remote proctoring could be a viable solution for administering high-stakes medical exams. With regards to candidates' educational experience, remote proctoring was met with mixed feelings. Potential privacy issues and increased test anxiety should be taken into consideration when choosing a proctoring protocol. Future research should explore generalizability of these results utilising other proctoring systems in medical education and in other educational settings.

Challenges of unrestricted assignment-based examinations (ABE) and restricted open-book examinations (OBE) during Covid-19 pandemic in India: An experimental comparison

COVID‐19 pandemic has affected every sphere of life specially the education sector observing a paradigm shift in the nature of pedagogy from offline face‐to‐face to online‐virtual mode of learning. The biggest challenge in online‐learning was the conduction of online examination for student's assessment specially in Indian context where digital divide is rampant. Thus, present study examines and compares the challenges faced by the students in two most widely accepted modes of examination by Indian universities and institutes of higher learning, that is, take home/unrestricted/assignment‐based examination (ABE) and highly time restricted/open‐book examination (OBE). Primary data was collected through questionnaires prepared by using Google forms to measure adaptability, satisfaction, and challenges using 5‐point Likert's scale. Cronbach's α test was performed on question items to check the reliability and internal consistency of the items. χ² test has been applied in order to check whether there is a statistically significant relationship between the gender and place of residence in the acceptability of ABE and OBE. The findings suggest that both modes of examination have their own challenges largely governed by the digital and economic divide. The acceptance level of ABE and OBE is not associated with gender. However, we found the level of acceptance association of ABE with the place of residence of the students but not with OBE.

Efficiency, Usability, and Outcomes of Proctored Next-Level Exams for Proficiency Testing in Primary Care Education: Observational Study

Background

The COVID-19 pandemic has affected education and assessment programs and has resulted in complex planning. Therefore, we organized the proficiency test for admission to the Family Medicine program as a proctored exam. To prevent fraud, we developed a web-based supervisor app for tracking and tracing candidates’ behaviors.

Objective

We aimed to assess the efficiency and usability of the proctored exam procedure and to analyze the procedure’s impact on exam scores.

Methods

The application operated on the following three levels to register events: the recording of actions, analyses of behavior, and live supervision. Each suspicious event was given a score. To assess efficiency, we logged the technical issues and the interventions. To test usability, we counted the number of suspicious students and behaviors. To analyze the impact that the supervisor app had on students’ exam outcomes, we compared the scores of the proctored group and those of the on-campus group. Candidates were free to register for off-campus participation or on-campus participation.

Results

Of the 593 candidates who subscribed to the exam, 472 (79.6%) used the supervisor app and 121 (20.4%) were on campus. The test results of both groups were comparable. We registered 15 technical issues that occurred off campus. Further, 2 candidates experienced a negative impact on their exams due to technical issues. The application detected 22 candidates with a suspicion rating of >1. Suspicion ratings mainly increased due to background noise. All events occurred without fraudulent intent.

Conclusions

This pilot observational study demonstrated that a supervisor app that records and registers behavior was able to detect suspicious events without having an impact on exams. Background noise was the most critical event. There was no fraud detected. A supervisor app that registers and records behavior to prevent fraud during exams was efficient and did not affect exam outcomes. In future research, a controlled study design should be used to compare the cost-benefit balance between the complex interventions of the supervisor app and candidates’ awareness of being monitored via a safe browser plug-in for exams.

Preferences and Scores of Different Types of Exams during COVID-19 Pandemic in Faculty of Veterinary Medicine in Spain: A Cross-Sectional Study of Paper and E-exams

The World Health Organization (WHO) officially declared the novel coronavirus (COVID-19) as a pandemic on 11 March 2020, and educational institutions have had to modify most of their activities (face-to-face activities were suspended). This situation forced academic institutions to modify the evaluation format of students. The use of proctoring systems quickly became widespread, although some controversies arose. The two main discussions regarding these systems are the integrity of the assessment and the capacity of the students to adapt to this new assessment method, without changes in theirs scores. To elucidate two controversies, we have analyzed the preferences and the scores obtained from a trial of 660 scores from 332 students of the third grade of Veterinary Medicine. The experiment involved three modalities of exam: an online format from home using the Respondus Lockdown Browser system (Modality 1), online in person using the Respondus Lockdown Browser system with the supervision of a teacher (Modality 2), or paper format in person with the supervision of a teacher (Modality 3). The results obtained showed that the students preferred Modality 1 (online at home with Respondus Lockdown Browser system). No statistical differences between the scores obtained by students were found between the three modalities analyzed. The proctoring system is a good method to adjudicate exams in higher education institutions, and the scores of students are similar to those obtained through traditional evaluation and control systems.

A Systematic Review on AI-based Proctoring Systems: Past, Present and Future

There have been giant leaps in the field of education in the past 1-2 years.. Schools and colleges are transitioning online to provide more resources to their students. The COVID-19 pandemic has provided students more opportunities to learn and improve themselves at their own pace. Online proctoring services (part of assessment) are also on the rise, and AI-based proctoring systems (henceforth called as AIPS) have taken the market by storm. Online proctoring systems (henceforth called as OPS), in general, makes use of online tools to maintain the sanctity of the examination. While most of this software uses various modules, the sensitive information they collect raises concerns among the student community. There are various psychological, cultural and technological parameters need to be considered while developing AIPS. This paper systematically reviews existing AI and non-AI-based proctoring systems. Through the systematic search on Scopus, Web of Science and ERIC repositories, 43 paper were listed out from the year 2015 to 2021. We addressed 4 primary research questions which were focusing on existing architecture of AIPS, Parameters to be considered for AIPS, trends and Issues in AIPS and Future of AIPS. Our 360-degree analysis on OPS and AIPS reveals that security issues associated with AIPS are multiplying and are a cause of legitimate concern. Major issues include Security and Privacy concerns, ethical concerns, Trust in AI-based technology, lack of training among usage of technology, cost and many more. It is difficult to know whether the benefits of these Online Proctoring technologies outweigh their risks. The most reasonable conclusion we can reach in the present is that the ethical justification of these technologies and their various capabilities requires us to rigorously ensure that a balance is struck between the concerns with the possible benefits to the best of our abilities. To the best of our knowledge, there is no such analysis on AIPS and OPS. Our work further addresses the issues in AIPS in human and technological aspect. It also lists out key points and new technologies that have only recently been introduced but could significantly impact online education and OPS in the years to come.

E-proctored exams during the COVID-19 pandemic: A close understanding

Researchers have focused on evaluating and exploring the online examination experience during the COVID-19 pandemic. However, understanding the perceptions of using an e-proctoring tool within the online examination experience is still limited. This study explores the first unique experience for students' attitudes and concerns using an e-proctoring tool in their final exams during the COVID-19 pandemic. It also highlights the e-tools' impact on students' performances to guide educational institutions towards appropriate practices going forward, especially as the pandemic is expected to have far-reaching consequences. A mixed-methods analysis was used to examine heterogeneous sources of data including self-reported data and officially documented data. The data was analyzed by a qualitative analysis of the focus group and quantitative analyses of the survey questions and exam attempts. In June 2020, students participated in a focus group to elaborate on their attitudes and concerns pertaining to their e-proctoring experience. Based on the preliminary outcomes, a survey was developed and distributed to a purposive sample (n = 106) of students from information technology majors who had taken at least one e-proctored exam during the COVID-19 pandemic. Finally, 21 online exams with 815 total attempts were analyzed to assess how well students performed under an e-proctored test. The study's findings shed light on students' perceptions of their e-proctoring experience, including their predominant concerns over privacy and various environmental and psychological factors. The research also highlights challenges in implementing the e-proctoring tool as well as its impact on students' performance.

Best Practices on Examination Construction, Administration, and Feedback

Examinations are typically used in higher education to objectively assess student learning, and they are also used as a frequent assessment tool in the Doctor of Pharmacy curriculum. This paper describes best practices and provides examples for faculty to build reliable and valid examinations, ensure examination security and deter academic misconduct, and enhance student learning and achievement of course objectives. Colleges and schools of pharmacy can incorporate these concepts into comprehensive examination policies and focus faculty development efforts on improving the examination purpose, design, and experience for both faculty and students.