High- and low-achieving international medical students' perceptions of the factors influencing their academic performance at Chinese universities

China has become an attractive destination for international medical students, particularly those from developing countries in Asia and Africa. These students are known to face difficulties in adapting to Chinese medical universities and they tend to score poorly on subsequent examinations to enter the medical profession in their home or in a third country. To date, limited research concerning the factors that affect the academic performance of this group of students in China has been conducted. In particular, there have been very few comparisons between high and low achievers to identify the factors that high- and low-achieving international medical students in all years of study perceive as affecting their academic performance. A qualitative research approach was implemented at two medical universities. Semi-structured interviews were conducted with 18 high achievers and 22 low achievers in their second, third, fourth, fifth, sixth, and post-graduation years. Interviews were audio-recorded, transcribed, and coded using a thematic analysis approach.A few key overlapping factors were identified: social support, learning motivation and interest, exam preparation and strategies, time management, and coping. The high achievers manipulated these strategies successfully while the low achievers did not. The high achievers demonstrated efficient learning methods (efficient use of class time, good note-taking habits, preview and review strategies, expanded learning, self-study and group learning, deep learning, choosing an appropriate study location, and focusing on basics and concepts). They also achieved a good balance between studying and leisure. The low achievers mentioned learning challenges, health issues, English language barriers, and problems with online classes. This study allowed us to identify important factors that international medical students at Chinese institutions perceive to influence their performance. The findings contribute to the literature on the topic and provide a solid basis for practice and policy development.

Performance of ChatGPT and Bard in self-assessment questions for nephrology board renewal

BACKGROUND

Large language models (LLMs) have impacted advances in artificial intelligence. While LLMs have demonstrated high performance in general medical examinations, their performance in specialized areas such as nephrology is unclear. This study aimed to evaluate ChatGPT and Bard in their potential nephrology applications.

METHODS

Ninety-nine questions from the Self-Assessment Questions for Nephrology Board Renewal from 2018 to 2022 were presented to two versions of ChatGPT (GPT-3.5 and GPT-4) and Bard. We calculated the correct answer rates for the five years, each year, and question categories and checked whether they exceeded the pass criterion. The correct answer rates were compared with those of the nephrology residents.

RESULTS

The overall correct answer rates for GPT-3.5, GPT-4, and Bard were 31.3% (31/99), 54.5% (54/99), and 32.3% (32/99), respectively, thus GPT-4 significantly outperformed GPT-3.5 (p < 0.01) and Bard (p < 0.01). GPT-4 passed in three years, barely meeting the minimum threshold in two. GPT-4 demonstrated significantly higher performance in problem-solving, clinical, and non-image questions than GPT-3.5 and Bard. GPT-4's performance was between third- and fourth-year nephrology residents.

CONCLUSIONS

GPT-4 outperformed GPT-3.5 and Bard and met the Nephrology Board renewal standards in specific years, albeit marginally. These results highlight LLMs' potential and limitations in nephrology. As LLMs advance, nephrologists should understand their performance for future applications.

Evaluating The Role of ChatGPT as a Study Aid in Medical Education in Surgery

OBJECTIVE

Our aim was to assess how ChatGPT compares to Google search in assisting medical students during their surgery clerkships.

DESIGN

We conducted a crossover study where participants were asked to complete 2 standardized assessments on different general surgery topics before and after they used either Google search or ChatGPT.

SETTING

The study was conducted at the Perelman School of Medicine at the University of Pennsylvania (PSOM) in Philadelphia, Pennsylvania.

PARTICIPANTS

19 third-year medical students participated in our study.

RESULTS

The baseline (preintervention) performance of participants on both quizzes did not differ between the Google search and ChatGPT groups (p = 0.728). Students overall performed better postintervention and the difference in test scores was statistically significant for both the Google group (p < 0.001) and the ChatGPT group (p = 0.01). The mean percent increase in test scores pre- and postintervention was higher in the Google group at 11% vs. 10% in the ChatGPT group, but this difference was not statistically significant (p = 0.87). Similarly, there was no statistically significant difference in postintervention scores on both assessments between the 2 groups (p = 0.508). Postassessment surveys revealed that all students (100%) have known about ChatGPT before, and 47% have previously used it for various purposes. On a scale of 1 to 10 with 1 being the lowest and 10 being the highest, the feasibility of ChatGPT and its usefulness in finding answers were rated as 8.4 and 6.6 on average, respectively. When asked to rate the likelihood of using ChatGPT in their surgery rotation, the answers ranged between 1 and 3 ("Unlikely" 47%), 4 to 6 ("intermediate" 26%), and 7 to 10 ("likely" 26%).

CONCLUSION

Our results show that even though ChatGPT was comparable to Google search in finding answers pertaining to surgery questions, many students were reluctant to use ChatGPT for learning purposes during their surgery clerkship.

Virtual self-care simulations for third-year pharmacy skills laboratory courses in three institutions

BACKGROUND AND PURPOSE

This study was designed to determine whether a virtual, self-care activity improved knowledge and confidence in third-year student pharmacists.

EDUCATIONAL ACTIVITY AND SETTING

Third-year student pharmacists (n = 386) from three institutions participated in the virtual self-care simulation during their respective practice laboratory course. A pre- and post-assessment collected 10 knowledge and five confidence questions, self-reported on 0-100 scale, mapped to learning outcomes and pharmacy standards. Responses for participants who provided consent and had linked assessments were analyzed. Additionally, students participated in a perception assessment following the simulation with the post-assessment. Each knowledge question was scored as binary (correct/incorrect), presented as percentage, and significance identified with a McNemar's test. Total knowledge score and confidence changes were presented as means with standard deviations and significance with a paired t-test. Student perceptions were presented as frequencies and percentages.

FINDINGS

Total knowledge assessment demonstrated a significant improvement (p < 0.001) for the entire cohort of 198 study participants. Upon additional analysis, a single institution led the cohort to significant increase, with variable improvement and significance for each individual question. Confidence improved for the entire cohort of students and at each institution individually. The students perceived the virtual self-care activity favorably.

SUMMARY

The third-year student virtual self-care activity improved knowledge and confidence with varying significance between institutions. Future studies will focus on the impact of continued reinforcement of self-care activities on student growth in knowledge and confidence.

[Communication training in otorhinolaryngology education : Comparison of an online and a classroom-based training course]

BACKGROUND

Communication skills are among the most important key qualifications of the medical profession. To what extent these can also be acquired online in medical education in otolaryngology is investigated in this study.

OBJECTIVE

A voluntary online training for the teaching of communication skills was compared with a corresponding face-to-face format. The question of the extent to which acceptance of the two formats and students' self-assessment of their communicative skills differed was investigated.

MATERIALS AND METHODS

In the online training, students were prepared for the topic asynchronously via a video. Thereafter, they were able to conduct consultations with simulation patients online and synchronously. The face-to-face training was comparable in structure and duration and took place in an earlier semester. The acceptance of both seminars was assessed by a questionnaire with 19 items on a five-point Likert scale. Self-assessment of communication skills was measured by a 10-cm visual analog scale pre/post with 16 items.

RESULTS

Both formats achieved high acceptance with an average score (M) of 2.08 (standard deviation, SD = 0.54) for the online format and M = 1.97 (SD = 0.48) for the face-to-face event. Students' self-assessments of communication skills showed a twofold increase in the online group (M = 1.54, SD = 0.94) compared to the face-to-face group (M = 0.75, SD = 0.87).

CONCLUSION

This study shows that teaching communication skills in the online format was well accepted and resulted in significant changes in students' self-assessment of communication skills.

[Interactive electronic visualization formats in student teaching]

BACKGROUND

In the context of contact restrictions, conventional teaching is currently in need of optimization and expansion. The range of digital teaching formats in student training is very heterogeneous and their effectiveness uncertain. This study aims to investigate the extent to which an electronic ward round can be used as an alternative to the conventional ENT attendance practical course, and whether the use of electronic teaching formats has an influence on the quality of teaching.

MATERIALS AND METHODS

Instead of regular attendance practicals, bedside teaching took place once a week in real time as a video stream via tablet. A total of 43 students in the seventh semester (winter semester 2020/2021) were included in the prospective study. Evaluation forms were used to examine the subjective didactic value of different visualization formats for the students. Examination results from previous years were used for comparison.

RESULTS

The majority of students reported knowledge gain from the electronic rounds (93.02%) and that they were a good alternative to the traditional attendance clerkship (69.77%). The quality of the video and audio transmission as well as the comprehensibility of the case studies presented were consistently rated as good to very good. The students' examination results tended to be slightly worse in the test group compared to the control students of previous years.

CONCLUSION

Integration of innovative interactive visualization options into teaching shows promising prospects as a supplement to conventional face-to-face teaching. The results of this study can contribute to the further expansion of digital teaching. Scaling up this model could be considered especially in countries with limited availability of face-to-face teaching.

Evaluating the Impact of an Adolescent Sexuality Education Workshop on Medical Student Communication in an Objective Structured Clinical Examination

PURPOSE

Gaps still exist in medical education about the sexual health needs of sexual diverse populations, and little is known about how translatable current learning modules are to patient encounters. Efforts at an academic medical institution have been made to address this need, including a two-hour adolescent sexuality workshop during the Core Clerkship in Pediatrics. This workshop's efficacy was evaluated in an objective structured clinical examination (OSCE) given to rising fourth-year medical students, where the standardized patient case focused on an adolescent cisgender male with dysuria and in a new, same-sex relationship.

METHODS

Performance of students who completed the workshop prior to the OSCE (n = 48) were compared to those of students who did not participate in the workshop prior to the OSCE (n = 17). The encounters were recorded and transcribed, and the deidentified transcripts were scored on a rubric focusing on five domains: sexual identity disclosure, behavioral assessment, psychosocial history, counseling and anticipatory guidance, and relationship building.

RESULTS

Student's t-test comparison of the scores found significantly higher scores for the psychosocial history domain (p = .04), particularly concerning disclosure of a new boyfriend and recent sexual activity (p = .008), for students who had the workshop before the OSCE.

DISCUSSION

Students who took the adolescent sexuality workshop performed better in gathering psychosocial information in an OSCE encounter a sexual minority adolescent. These results affirm prior work that active learning on sexual diverse health in medical school curricula may prepare students for effective engagement with adolescents exploring their sexuality.

[Online training for hearing implant surgery : A new approach to otological training. German version]

OBJECTIVE

Education in microsurgery of the ear includes staged training to allow for mastering of the complex microsurgical procedures, particularly in the context of middle ear reconstruction and cochlear implantation. Traditional surgical training includes temporal bone preparations by cadaver dissection and supervised operating room practice. As these on-site trainings are limited, there is a need to broaden education facilities in an on-line format. Therefore, a first basic on-line training for otosurgery was developed.

MATERIALS AND METHODS

The system consists of an artificial temporal bone model together with a set of basic surgical instruments and implant dummies. As an essential part of the training kit, a high-resolution camera set is included that allows for connection to a video streaming platform and enables remote supervision of the trainees' surgical steps by experienced otological surgeons. In addition, a pre-learning platform covering temporal bone anatomy and instrumentation and pre-recorded lectures and instructional videos has been developed to allow trainees to review and reinforce their understanding before hands-on practice.

RESULTS

Over the three courses held to date, 28 participants with varying levels of prior surgical experience took part in this otological surgical training program. The immediate feedback of the participants was evaluated by means of a questionnaire. On this basis, the high value of the program became apparent and specific areas could by identified where further refinements could lead to an even more robust training experience.

CONCLUSION

The presented program of an otosurgical online training allows for basal education in practical exercises on a remote system. In this way, trainees who have no direct access to on-site instruction facilities in ear surgery now have the chance to start their otosurgical training in an educational setting adapted to modern technologies.

[Multidimensional formats of surgical anatomy in otorhinolaryngology student teaching-a comparison of effectivity]

BACKGROUND

Technological change in healthcare and the digital transformation of teaching require innovations in student teaching in medicine. New technologies are needed to enable the delivery and use of diverse teaching and learning formats by educational institutions independent of time and place. The aim of this study is to analyze the effectiveness of different multidimensional formats in student teaching in surgical ENT medical anatomy.

MATERIALS AND METHODS

During the summer semester 2022 and winter semester 2022/2023, the digital teaching and learning program was expanded by testing different visualization formats (3D glasses, cardboards, or VR glasses) with students in the context of a highly standardized surgical procedure, namely cochlear implantation. A pre- and post-intervention knowledge assessment was carried out in all groups, followed by an evaluation.

RESULTS

Of 183 students, 91 students fully participated in the study. The post-intervention knowledge assessment showed a significant increase in correct answers regardless of visualization format. In a direct comparison, the operating room (OR) group answered correctly significantly more often than the cardboard group (p = 0.0424). The majority of students would like to see 3D teaching as an integral part of the teaching program (87.9%) and more streaming of live surgeries (93.4%). They see the use of the various technologies as a very good addition to conventional surgical teaching (72.5%), as good visualization (89%) increases retention (74.7%) and motivation (81.3%).

CONCLUSION

Application and use of new visualization technologies in everyday clinical practice is a promising approach to expanding student training. Mobile, interactive, and personalized technical formats can be adapted to the learning behavior of students. Last but not least, the use of new media influences learning motivation. An expansion of digital teaching and learning formats can be expressly recommended on the basis of this study.

Exam evaluation in prosthodontics across preclinical and clinical years from students' perspective: A cross-sectional study

INTRODUCTION

The purpose of this study was to explore the students' perceptions and performance in prosthodontics theory exam.

METHODS

A cross-sectional descriptive study was conducted on 560 (80.82%) students of different levels (third, fourth and fifth years) to explore their opinions and performance with regard to a number of issues on a prosthodontics theory exam (exam evaluation, exam preparation, exam material, exam timing). Demographic data were also collected. Descriptive statistics were generated and Chi-square test, independent sample t-test, ANOVA test and Pearson's correlation coefficient were used to examine the associations between different variables. The significance level was set at p < .05.

RESULTS

Students' responses regarding exam evaluation was influenced by their gender, study level, high-school Grade Point Average (GPA) and undergraduate cumulative GPA. Perceived exam difficulty was significantly affected by gender (p = .03) and study level (p < .001), and negatively correlated to both high-school GPA (p < .001) and university GPA (p = .03). The vast majority (88.2%) depended on lecture hand-outs and lecture notes for study. Exam material and preparation were not significantly affected by any of the demographic variables with most respondents (76.8%) thinking that the lectures blended with prosthodontics laboratories/clinics would improve their understanding of the exam material. The suggested best time to conduct the exam was early afternoon (31.6%). Student performance was significantly affected by the study level (p < .001) and cumulative GPA (p < .001) with significant positive correlation between the high-school GPA and the mark in the exam (r = .29, p < .001) and by the amount of time students spent for exam preparation (p < .001). Those students who reported using textbooks to prepare for the exam got significantly higher marks (66.1 ± 8.7) compared to the students who did not (62.8 ± 9.7) (p = .03).

CONCLUSIONS

Course level, GPA and gender were identified as the most influential factors in different aspects of exam evaluation and students' performance. Regular study and use of textbooks were demonstrated to improve academic performance. Additional orientation and guidance relating to the exam (especially for third year students) would be welcomed, as would alternate teaching methods such as small group discussions or study groups.

Ten tips for effective use and quality assurance of multiple-choice questions in knowledge-based assessments

Multiple-choice questions (MCQs) are the most popular type of items used in knowledge-based assessments in undergraduate and postgraduate healthcare education. MCQs allow assessment of candidates' knowledge on a broad range of knowledge-based learning outcomes in a single assessment. Single-best-answer (SBA) MCQs are the most versatile and commonly used format. Although writing MCQs may seem straight-forward, producing decent-quality MCQs is challenging and warrants a range of quality checks before an item is deemed suitable for inclusion in an assessment. Like all assessments, MCQ-based examinations must be aligned with the learning outcomes and learning opportunities provided to the students. This paper provides evidence-based guidance on the effective use of MCQs in student assessments, not only to make decisions regarding student progression but also to build an academic environment that promotes assessment as a driver for learning. Practical tips are provided to the readers to produce authentic MCQ items, along with appropriate pre- and post-assessment reviews, the use of standard setting and psychometric evaluation of assessments based on MCQs. Institutions need to develop an academic culture that fosters transparency, openness, equality and inclusivity. In line with contemporary educational principles, teamwork amongst teaching faculty, administrators and students is essential to establish effective learning and assessment practices.

[Virtual training of practical competences in sonography]

OBJECTIVE

During the COVID-19 pandemic, a particular challenge in the transition to digital teaching was to teach practical skills such as sonography of the head and neck online. The aim of this study was to validate the digital sonography course for medical students established at the Freiburg University Hospital ENT Department.

METHODS

Participants were 178 students of human medicine. The study group simulated the sonography examination at home with a dummy transducer using the Peyton method under the guidance of a tutor via video seminar. In a subsequent learning success check, the results of the students in the online course were compared with those of the control group, who learned sonography in the classroom.

RESULTS

Students of the online course achieved comparable results to the classroom group.

CONCLUSION

This study shows that practical skills which require extensive equipment such as a sonography machine can be taught to a certain extent digitally or at least in a hybrid form.

Effectiveness of a manual test at the beginning of dental school

INTRODUCTION

This study analysed whether the SAZAMED (Self-assessment for dentistry studies) manual test conducted as an exercise or examination in the first year of dental studies, and the official aptitude test for medical studies (EMS), can predict future performance of dental students.

MATERIALS AND METHODS

Students enrolled in the dental curriculum at the University of Basel between 2009/10 and 2019/20 were categorized: Group A conducted the SAZAMED in the third bachelor year (BA) and second master year (MA), group B practiced in the first and third BA and second MA and group C had to pass the test in the first BA with repetition in the third BA and second MA. SAZAMED comprised (i) wire bending, (ii) modelling sphere and cube, (iii) drawing mirror-inverted and (iv) indirect drawing over a mirror. Comparative variables were the EMS per cent rank and test scores in practical-clinical examinations in the third BA and the second MA.

RESULTS

Data were available from 329 students over the 10-year period. Repeaters and dropouts had poorer performance in the first BA SAZAMED than students who did not (p < 0.001). Among the four SAZAMED exercises, mirror-inverted drawing was the strongest predictor of future student performance. The official aptitude test EMS correlated with the third BA certificate (p = 0.012), but failed to predict repeaters and dropouts.

CONCLUSION

Since academic success was associated with significantly better scores on the first BA SAZAMED, it is considered a predictor and recommended in addition to the official EMS to identify students in need of intervention programmes or who should choose a different field of study.

Influence of the incorporation of small-group practical seminars to a traditional lecture for the understanding of root canal morphology classification systems

INTRODUCTION

This study aimed to assess if the incorporation of small-group practical seminars into a traditional lecture format improved the understanding and learning of the root canal classification systems in undergraduate students.

MATERIALS AND METHODS

A small-group practical seminar was designed for 3rd year undergraduate students. The seminar consisted of the analysis of a guided summary of root canal morphology classifications, a participative discussion of several cases and drawing exercises of several internal anatomies based on the different classifications in templates presenting the external anatomy of different dental groups. Students were inquired about their perspectives on the different classifications and the learning strategies with an anonymous questionnaire. The knowledge acquired was assessed in an exam. The scores obtained were compared using the Mann-Whitney U-test with those obtained in the cohort of the previous year in which only the traditional lecture was delivered.

RESULTS

Students' scores (mean = 7.7; SD = 2.5) were significantly higher (p = .006) than those from the previous cohort (mean = 6.9; SD = 2.3). Most students agreed that Ahmed's classification system was the most complete (98.8%) and intuitive (84%), but also the most difficult to understand (67%). All the students believed that the learning activity helped them understand the new classification system.

CONCLUSION

The incorporation of small-group practical seminars into traditional lecture improved the understanding and learning of root canal configurations in undergraduate students. Students rated the classification system suggested by Ahmed et al. as the most complete and intuitive out of the three systems, but also the most difficult to learn.

Flipped Classroom: Improved team performance during resuscitation training through interactive pre-course content - a cluster-randomised controlled study

BACKGROUND

Resuscitation is a team effort, and it is increasingly acknowledged that team cooperation requires training. Staff shortages in many healthcare systems worldwide, as well as recent pandemic restrictions, limit opportunities for collaborative team training. To address this challenge, a learner-centred approach known as flipped learning has been successfully implemented. This model comprises self-directed, asynchronous pre-course learning, followed by knowledge application and skill training during in-class sessions. The existing evidence supports the effectiveness of this approach for the acquisition of cognitive skills, but it is uncertain whether the flipped classroom model is suitable for the acquisition of team skills. The objective of this study was to determine if a flipped classroom approach, with an online workshop prior to an instructor-led course could improve team performance and key resuscitation variables during classroom training.

METHODS

A single-centre, cluster-randomised, rater-blinded study was conducted on 114 final year medical students at a University Hospital in Germany. The study randomly assigned students to either the intervention or control group using a computer script. Each team, regardless of group, performed two advanced life support (ALS) scenarios on a simulator. The two groups differed in the order in which they completed the flipped e-learning curriculum. The intervention group started with the e-learning component, and the control group started with an ALS scenario. Simulators were used for recording and analysing resuscitation performance indicators, while professionals assessed team performance as a primary outcome.

RESULTS

The analysis was conducted on the data of 96 participants in 21 teams, comprising of 11 intervention groups and 10 control groups. The intervention teams achieved higher team performance ratings during the first scenario compared to the control teams (Estimated marginal mean of global rating: 7.5 vs 5.6, p < 0.01; performance score: 4.4 vs 3.8, p < 0.05; global score: 4.4 vs 3.7, p < 0.001). However, these differences were not observed in the second scenario, where both study groups had used the e-learning tool.

CONCLUSION

Flipped classroom approaches using learner-paced e-learning prior to hands-on training can improve team performance.

TRIAL REGISTRATION

German Clinical Trials Register ( https://drks.de/search/de/trial/DRKS00013096 ).

Relationship between learning styles and clinical competency in nursing students

BACKGROUND

The acquisition of clinical competence is considered the ultimate goal of nursing education programs. This study explored the relationship between learning styles and clinical competency in undergraduate nursing students.

METHODS

A descriptive-correlational study was conducted in 2023 with 276 nursing students from the second to sixth semesters at Abhar School of Nursing, Zanjan University of Medical Sciences, Iran. Data were collected using demographic questionnaires, Kolb's learning styles, and Meretoja's clinical competence assessments completed online by participants. Data were analyzed using SPSS version 16, employing descriptive statistics and inferential tests (independent T-test, ANOVA, Pearson correlation) at a significance level 0.05.

RESULTS

The predominant learning styles among nursing students were divergent (31.2%), and the least common was convergent (18.4%). The overall clinical competency score was 77.25 ± 12.65. Also, there was a significant relationship between learning styles and clinical competency, so the clinical competency of students with accommodative and converging learning styles was higher. (P < 0.05).

CONCLUSION

The results of this study showed the association between learning styles and clinical competence in nursing students. It is recommended that educational programs identify talented students and provide workshops tailored to strengthen various learning styles associated with enhanced clinical competence.

Biomedical students' self-efficacy and academic performance by gender in a flipped learning haematology course

INTRODUCTION

This study investigated the impact of flipped learning versus traditional instruction on medical students' academic performance and self-efficacy in a haematology course, and examined gender differences. Flipped learning is an instructional approach where students review pre-recorded lecture content at home, and active learning occurs in the classroom. Self-efficacy refers to students' beliefs in their ability to succeed and accomplish learning goals.

METHODS

A quasi-experimental study was conducted with 86 third-year Saudi medical students (46 males, 40 females) in a 10-week haematology course. Students were assigned to flipped learning group (n = 41) or traditional lecture group (n = 45). Both groups completed pre- and post-intervention academic tests and self-efficacy surveys. Data were analyzed using descriptive statistics and t-tests.

RESULTS

The flipped learning group showed an increase in academic scores (p <.05) and self-efficacy scores (p <.05) compared to the traditional group, but between group differences were not statistically significant. Female students in the flipped learning group showed the greatest increase in academic scores and self-efficacy. Most students perceived flipped learning positively for enhancing learning and preparation for class.

CONCLUSION

Flipped learning promoted self-efficacy compared to traditional lectures in medical students. Gender-specific benefits were observed, highlighting the need to design instruction to meet diverse student needs.

"Seeing inside out": revealing the effectiveness of otoscopy training in virtual reality enhanced practical exams - a randomized controlled trial

BACKGROUND

The study aimed to assess the impact of different training modalities on otoscopy performance during a practical exam using a high-fidelity simulator and to determine if objective evaluation of otoscopy is feasible using a simulator that records insertion depth and tympanic membrane coverage.

METHODS

Participants were assigned to one of four groups: control and three intervention groups with varying training approaches. Participants received otoscopy training and then were assessed through a practical exam on a high-fidelity simulator that uses virtual reality to visualize the ear canal and middle ear. Performance was evaluated using a modified Objective Structured Assessment of Technical Skills checklist and Integrated Procedural Performance Instrument checklist. Insertion depth, tympanic membrane coverage, and correct diagnosis were recorded. Data were tested for normal distribution using the Shapiro-Wilk test. One-way ANOVA and, for non-normally distributed data, Kruskal-Wallis test combined with Dunn's test for multiple comparisons were used. Interrater reliability was assessed using Cohen's κ and Intraclass correlation coefficient.

RESULTS

All groups rated their training sessions positively. Performance on the OSATS checklist was similar among groups. IPPI scores indicated comparable patient handling skills. The feedback group examined larger tympanic membrane areas and had higher rates of correct diagnosis. The correct insertion depth was rarely achieved by all participants. Interrater reliability for OSATS was strong. IPPI reliability showed good correlation.

CONCLUSION

Regardless of training modality, participants perceived learning improvement and skill acquisition. Feedback improved examination performance, indicating simulator-guided training enhances skills. High-fidelity simulator usage in exams provides an objective assessment of performance.

Artificial intelligence and medical education: application in classroom instruction and student assessment using a pharmacology & therapeutics case study

BACKGROUND

Artificial intelligence (AI) tools are designed to create or generate content from their trained parameters using an online conversational interface. AI has opened new avenues in redefining the role boundaries of teachers and learners and has the potential to impact the teaching-learning process.

METHODS

In this descriptive proof-of- concept cross-sectional study we have explored the application of three generative AI tools on drug treatment of hypertension theme to generate: (1) specific learning outcomes (SLOs); (2) test items (MCQs- A type and case cluster; SAQs; OSPE); (3) test standard-setting parameters for medical students.

RESULTS

Analysis of AI-generated output showed profound homology but divergence in quality and responsiveness to refining search queries. The SLOs identified key domains of antihypertensive pharmacology and therapeutics relevant to stages of the medical program, stated with appropriate action verbs as per Bloom's taxonomy. Test items often had clinical vignettes aligned with the key domain stated in search queries. Some test items related to A-type MCQs had construction defects, multiple correct answers, and dubious appropriateness to the learner's stage. ChatGPT generated explanations for test items, this enhancing usefulness to support self-study by learners. Integrated case-cluster items had focused clinical case description vignettes, integration across disciplines, and targeted higher levels of competencies. The response of AI tools on standard-setting varied. Individual questions for each SAQ clinical scenario were mostly open-ended. The AI-generated OSPE test items were appropriate for the learner's stage and identified relevant pharmacotherapeutic issues. The model answers supplied for both SAQs and OSPEs can aid course instructors in planning classroom lessons, identifying suitable instructional methods, establishing rubrics for grading, and for learners as a study guide. Key lessons learnt for improving AI-generated test item quality are outlined.

CONCLUSIONS

AI tools are useful adjuncts to plan instructional methods, identify themes for test blueprinting, generate test items, and guide test standard-setting appropriate to learners' stage in the medical program. However, experts need to review the content validity of AI-generated output. We expect AIs to influence the medical education landscape to empower learners, and to align competencies with curriculum implementation. AI literacy is an essential competency for health professionals.

Enhancing biostatistics education for medical students in Poland: factors influencing perception and educational recommendations

BACKGROUND

A number of recommendations for the teaching of biostatistics have been published to date, however, student opinion on them has not yet been studied. For this reason, the aim of the manuscript was to find out the opinions of medical students at universities in Poland on two forms of teaching biostatistics, namely traditional and practical, as well as to indicate, on the basis of the results obtained, the related educational recommendations.

METHODS

The study involved a group of 527 students studying at seven medical faculties in Poland, who were asked to imagine two different courses. The traditional form of teaching biostatistics was based on the standard teaching scheme of running a test from memory in a statistical package, while the practical one involved reading an article in which a particular test was applied and then applying it based on the instruction provided. Other aspects related to the teaching of the subject were assessed.

RESULTS

According to the students of each course, the practical form of teaching biostatistics reduces the stress level associated with teaching and the student exam (p < 0.001), as well as contributing to an increased level of elevated knowledge (p < 0.001), while the degree of satisfaction after passing the exam is higher (p < 0.001). A greater proportion of students (p < 0.001) believe that credit for the course could be given by doing a statistical review of an article or conducting a survey, followed by the tests learned in class. More than 95% also said that the delivery of the courses should be based on the field of study they were taking, during which time they would also like to have the opportunity to take part in optional activities and hear lectures from experts.

CONCLUSION

It is recommended that more emphasis be placed on practical teaching the subject of biostatistics.

Virtual patients versus standardized patients for improving clinical reasoning skills in ophthalmology residents. A randomized controlled trial

BACKGROUND

History taking and clinical reasoning are important skills that require knowledge, cognition and meta-cognition. It is important that a trainee must experience multiple encounters with different patients to practice these skills. However, patient safety is also important, and trainees are not allowed to handle critically ill patients. To address this issue, a randomized controlled trial was conducted to determine the effectiveness of using Virtual Patients (VP) versus Standardized Patients (SP) in acquiring clinical reasoning skills in ophthalmology postgraduate residents.

METHODS

Postgraduate residents from two hospitals in Lahore, Pakistan, were randomized to either the VP group or the SP group and were exposed to clinical reasoning exercise via the VP or SP for 30 min after the pretest. This was followed by a posttest. One month after this activity, a follow-up posttest was conducted. The data were collected and analysed using IBM-SPSS version 25. Repeated measures ANOVA was used to track the effect of learning skills over time.

RESULTS

The mean age of the residents was 28.5 ± 3 years. The male to female ratio was 1:1.1. For the SP group, the mean scores were 12.6 ± 3.08, 16.39 ± 3.01 and 15.39 ± 2.95, and for the VP group, the mean scores were 12.7 ± 3.84, 16.30 ± 3.19 and 15.65 ± 3.18 for the pretest, posttest and follow-up posttest, respectively (p value < 0.00). However, the difference between the VP and SP groups was not statistically significant (p = 0.896). Moreover, there was no statistically significant difference between the VP and SP groups regarding the retention of clinical reasoning ability. In terms of learning gain, compared with the VP group, the SP group had a score of 51.46% immediately after clinical reasoning exercise as compared to VP group, in which it was 49.1%. After one month, it was 38.01 in SP and 40.12% in VP group.

CONCLUSION

VPs can be used for learning clinical reasoning skills in postgraduate ophthalmology residents in a safe environment. These devices can be used repeatedly without any risk to the real patient. Although similarly useful, SP is limited by its nonavailability for repeated exercises.

An Interactive Mapping and Case Discussion Seminar Introducing Medical Students to Climate Change, Environmental Justice, and Health

Introduction

Integrating climate change and health into a medical school curriculum is critical for future physicians who will manage health crises caused by a rapidly changing climate. Although medical schools have increasingly included climate change in the curriculum, there remains a need to address the link between the climate crisis, environmental justice, and historical policies that shape environmental health disparities in local communities.

Methods

In academic years 2021-2022 (AY22) and 2022-2023 (AY23), second-year medical students participated in a 2.5-hour seminar utilizing didactic teaching and small breakout groups that included interactive mapping activities and case scenarios. Learner knowledge and attitudes were self-assessed using pre- and postcurriculum surveys and a quiz. Qualitative thematic and content analysis was used to evaluate short-answer quiz responses and feedback.

Results

Of 357 students who participated in the seminar, 208 (58%) completed both the precurriculum and postcurriculum surveys. Self-assessed ability increased significantly for all educational objectives across both years. Attitudes on the importance of climate change knowledge for patient health also improved from a mean of 3.5 precurriculum to 4.2 postcurriculum (difference = 0.7, p < .01) in AY22 and from 3.6 pre- to 4.3 postcurriculum (difference = 0.7, p < .01) in AY23 on a 5-point Likert scale.

Discussion

This climate change and health session highlighting the link between environmental policy and climate change health vulnerability in the local context was successful in improving students' self-assessed ability across all stated educational objectives. Students cited the interactive small-group sessions as a major strength.

Development of a validated assessment tool for medical students using simulated patients: an 8-year panel survey

BACKGROUND

The use of simulated patients (SPs) to assess medical students' clinical performance is gaining prominence, underscored by patient safety perspective. However, few reports have investigated the validity of such assessment. Here, we examined the validity and reliability of an assessment tool that serves as a standardized tool for SPs to assess medical students' medical interview.

METHODS

This longitudinal survey was conducted at Keio University School of Medicine in Japan from 2014 to 2021. To establish content validity, the simulated patient assessment tool (SPAT) was developed by several medical education specialists from 2008 to 2013. A cohort of 36 SPs assessed the performance of 831 medical students in clinical practice medical interview sessions from April 2014 to December 2021. The assessment's internal structure was analyzed using descriptive statistics (maximum, minimum, median, mean, and standard deviation) for the SPAT's 13 item total scores. Structural validity was examined with exploratory factor analysis, and internal consistency with Cronbach's alpha coefficients. The mean SPAT total scores across different SPs and scenarios were compared using one way analysis of variance (ANOVA). Convergent validity was determined by correlating SPAT with the post-clinical clerkship obstructive structured clinical examination (post-CC OSCE) total scores using Pearson's correlation coefficient.

RESULTS

Of the 831 assessment sheets, 36 with missing values were excluded, leaving 795 for analysis. Thirty-five SPs, excluding one SP who quit in 2014, completed 795 assessments, for a response rate of 95.6%. Exploratory factor analysis revealed two factors, communication and physician performance. The overall Cronbach's alpha coefficient was 0.929. Significant differences in SPAT total scores were observed across SPs and scenarios via one-way ANOVA. A moderate correlation (r =.212, p <.05) was found between SPAT and post-CC OSCE total scores, indicating convergent validity.

CONCLUSIONS

Evidence for the validity of SPAT was examined. These findings may be useful in the standardization of SP assessment of the scenario-based clinical performance of medical students.

Foundational and Clinical Science Integration in a Team-Based Learning Module Modeling Care of a Patient With Dyslipidemia

Introduction

Foundational and clinical science integration, a long-standing goal of undergraduate medical education, benefits learners by promoting retention of critical knowledge and skills as well as their transfer to the clinical setting. We implemented a team-based learning (TBL) module in which foundational knowledge and skills from the disciplines of biochemistry, nutrition, and genetics were leveraged in a simulated patient encounter for diagnosis and management of a patient with dyslipidemia.

Methods

The TBL was deployed in a first-year medical student cardiovascular system course with 125 students over three academic years. Following individual and team readiness assurance tests (iRAT and tRAT, respectively), teams participated in an initial application exercise requiring consideration of clinical and laboratory data and other risk factors to engage the patient in a shared decision-making process. Using dietary and family history narratives in subsequent application exercises, teams completed recommendations for an individualized diet plan and an assessment of potential disease inheritance patterns to formulate appropriate patient care management strategies.

Results

Student engagement with prelearning materials and session team activities was high as judged by RAT performance and application exercise outcomes: iRAT question performance ranged from 89% to 99% for individual items, and tRAT performance was routinely 100%. Learners reported that the exercises were impactful and believed the learned foundational knowledge and skills were transferable to future patient care.

Discussion

The dyslipidemia TBL module provides an illustration for early clinical learners of how foundational knowledge and skills can be operationalized and transferred for optimal patient care.

Application of PRI-E-a combined learning method in oral and maxillofacial oncology education

The traditional lecture-based learning (LBL) method is facing great challenges due to its low efficiency and single proceeding form. We designed a PRI-E learning mode that combined and modified problem-based, case-based, and evidence-based learning with a step-by-step approach. We evaluated the practical learning outcomes of using the PRI-E mode by comparing it with traditional lecture-based learning in oral and maxillofacial oncology education. "PRI-E" consists of the first letters of the English words Passion, Research, Innovation, and Education, and it means "the best Education". This prospective randomized controlled trial included 40 participants. We evenly divided the participants into the PRI-E (n = 20) and LBL group (n = 20) based on the entrance test scores. The same staff group designed and then taught the learning content with different group measures. The evaluation included the final test scores and questionnaire assessments. Without affecting the examination results, the PRI-E teaching method was more satisfactory and popular with participants in terms of ability development and classroom participation. Enacting the PRI-E teaching method required more time, but this did not affect its popularity among the participants. Compared with the LBL learning mode, the PRI-E learning mode was more organized and efficient in oral and maxillofacial oncology education without affecting academic performance. This model has a high degree of satisfaction, which is conducive to training students' comprehensive ability.

Performance of ChatGPT on Chinese Master's Degree Entrance Examination in Clinical Medicine

BACKGROUND

ChatGPT is a large language model designed to generate responses based on a contextual understanding of user queries and requests. This study utilised the entrance examination for the Master of Clinical Medicine in Traditional Chinese Medicine to assesses the reliability and practicality of ChatGPT within the domain of medical education.

METHODS

We selected 330 single and multiple-choice questions from the 2021 and 2022 Chinese Master of Clinical Medicine comprehensive examinations, which did not include any images or tables. To ensure the test's accuracy and authenticity, we preserved the original format of the query and alternative test texts, without any modifications or explanations.

RESULTS

Both ChatGPT3.5 and GPT-4 attained average scores surpassing the admission threshold. Noteworthy is that ChatGPT achieved the highest score in the Medical Humanities section, boasting a correct rate of 93.75%. However, it is worth noting that ChatGPT3.5 exhibited the lowest accuracy percentage of 37.5% in the Pathology division, while GPT-4 also displayed a relatively lower correctness percentage of 60.23% in the Biochemistry section. An analysis of sub-questions revealed that ChatGPT demonstrates superior performance in handling single-choice questions but performs poorly in multiple-choice questions.

CONCLUSION

ChatGPT exhibits a degree of medical knowledge and the capacity to aid in diagnosing and treating diseases. Nevertheless, enhancements are warranted to address its accuracy and reliability limitations. Imperatively, rigorous evaluation and oversight must accompany its utilization, accompanied by proactive measures to surmount prevailing constraints.

Enhancing application and long-term retention of clinical knowledge using an extracurricular non-credit course

BACKGROUND AND PURPOSE

Curricular overload in doctor of pharmacy (PharmD) programs is necessitating innovative approaches to support student learning. The purpose of this study was to describe the design, delivery, and assessment of a non-credit extracurricular course that reinforced foundational concepts through the application of learning in case-based activities.

EDUCATIONAL ACTIVITY AND SETTING

A 14-week extracurricular course, designed using principles of spaced repetition and interleaving in the context of case-based exercises, was offered to third-year PharmD students. Content focused on Top 300 and over-the-counter medications, brown bag sessions/drug utilization review, and medication therapy management. Short-term course effectiveness was assessed through post-course focus groups. Longitudinal effectiveness was assessed nine months post-course using an online survey. Qualitative data were analyzed using a content analysis process with overarching themes identified. Clinical interventions identified in the post-course survey were analyzed descriptively.

FINDINGS

Twenty-four students completed the course and all assessments. Focus group themes were: (1) making connections to prior learning; (2) moving beyond memorizing facts; and (3) benefit from a low-stakes course. Students identified 162 course-linked clinical interventions during advanced pharmacy practice experiences (APPEs) in 16 different settings.

SUMMARY

Student learning can be enhanced through integration of evidence-based teaching strategies both within and across the curriculum. This can be accomplished not only through introduction of an extracurricular course but through modification of existing courses. Providing additional opportunities for reinforcing core clinical knowledge and applying clinical decision-making in a low-stakes environment was well-received by students and helped them make clinical interventions during APPEs.

Predictors of standardized in-service examination performance and residency outcomes in a graduate periodontics program

PURPOSE/OBJECTIVES

The objectives of this study were to assess the influence of learner- and education-related factors on standardized in-service examination performance and determine whether in-service examination scores predict residency outcomes.

METHODS

American Academy of Periodontology (AAP) In-service Examination (AIE) scores from 10 periodontics residency classes at a single center were recorded and compared against a panel of learner- and education-related variables using multiple linear regression models. Defined residency outcome measures were analyzed against AIE scores using binomial logistic regression.

RESULTS

No evaluated learner- or education-related variable was a statistically significant predictor of AIE score in this study sample. Likewise, AIE score was not a statistically significant predictor of any assessed residency outcome.

CONCLUSIONS

The AAP has performed a tremendous service to periodontics residents and programs by marshaling the leadership and expertise necessary to offer a professionally constructed assessment instrument. However, in the current study, no relationship could be identified between AIE score and any outcome, including first-attempt board certification. The AAP In-service Committee appears well situated to provide additional leadership focusing on exam implementation, which may enhance AIE value in competency decision making.

Experience informed procedural skills training

BACKGROUND

Paediatric critical care (PCC) physicians must perform several emergent procedures independently and competently-requiring transition from novice to competent over a 3-year fellowship. However, skill acquisition is not uniform. Individualised training, adapted to the unique experiences and requirements of each trainee, may enhance competency.

APPROACH

An individualised, longitudinal critical procedure course was initiated at a large academic paediatric medical centre in July 2022 for PCC fellows (n = 5). The course, informed by procedural performance profiles (P3) generated through real-time clinical assessments in the paediatric intensive care unit (PICU), was split into three phases: (1) an Initial Simulation Bootcamp-a 2-day introductory session; (2) Quarterly Structured Booster Sessions (QSBS)-spaced repetition of deliberate practice training individualised to each fellow; and (3) an Annual Refresher Training-a core skills and advanced technique training day.

EVALUATION

Fellows began with minimal experience, which formed their initial P3s. Ninety-two percent (166/180) of bedside procedures received real-time feedback, enabling longitudinal P3 modification, which identified focus areas for the QSBS. The sessions were well attended and received. Eighty-nine percent (QSBS #1 5/5, QSBS #2 3/4) of respondents reflected positively on the course's impact on procedural understanding. The course was perceived as more effective than traditional modalities, except bedside training.

IMPLICATION

Implementation of a spaced repetition, deliberate practice course informed by longitudinally tracked real-life performance data is feasible for educators and preferred by trainees. This educational construct can be applied to other clinical skills, bringing precision medicine approach to training.

Preliminary content and construct validity of a new model to differentiate research skills from evidence-based practice skills: Core, Evidence Application, Research (CEAR) Model

BACKGROUND

Research is the scientific basis for the profession of dietetics, as it must be located and applied in evidence-based practice (EBP). EBP is often presented as a foundational skill for research. CEAR - Core, Evidence Application, Research - is a newly proposed model that separates Research and Evidence Application skills into distinct domains, jointly supported by a set of Core skills, thus acknowledging that education and advancement in one domain neither requires nor precipitates education and advancement in the other. The goal was to investigate the content and construct validity of the new CEAR Model.

METHODS

A cross-sectional online survey of randomly selected dietitians in the United States was used to collect CEAR domain scores, validated measures of research or EBP skills and self-reported characteristics. Exploratory factor analysis, Cronbach's α and Pearson correlation between various tools and CEAR domains were used to assess validity and reliability. Analysis of variance (ANOVA) and multiple linear regression between CEAR domains and participant characteristics were used to assess convergent and divergent validity.

RESULTS

One hundred and fifty-four responses with a valid CEAR score were received and led to a three-factor solution, supporting the theorised differentiation of research from evidence application skills (content validity). Internal reliability for the CEAR Model overall and for each domain was high. The hypothesised correlations between existing research or EBP measurement tools and the relevant CEAR domains were found (construct validity). Known groups analysis demonstrated the expected differences in CEAR domain scores based on participant characteristics.

CONCLUSIONS

The CEAR Model demonstrates preliminary validity and internal reliability. It adds to the current literature by acknowledging the separateness of evidence application skills from research skills.

Trainees' Perspectives on the Next Era of Assessment and Precision Education

ABSTRACT

The next era of assessment in medical education promises new assessment systems, increased focus on ensuring high-quality equitable patient care, and precision education to drive learning and improvement. The potential benefits of using learning analytics and technology to augment medical training abound. To ensure that the ideals of this future for medical education are realized, educators should partner with trainees to build and implement new assessment systems. Coproduction of assessment systems by educators and trainees will help to ensure that new educational interventions are feasible and sustainable. In this paper, the authors provide a trainee perspective on 5 key areas that affect trainees in the next era of assessment: (1) precision education, (2) assessor education, (3) transparency in assessment development and implementation, (4) ongoing evaluation of the consequences of assessment, and (5) patient care data as sources of education outcomes.As precision education is developed, it is critical that trainees understand how their educational data are collected, stored, and ultimately utilized for educational outcomes. Since assessors play a key role in generating assessment data, it is important that they are prepared to give high-quality assessments and are continuously evaluated on their abilities. Transparency in the development and implementation of assessments requires communicating how assessments are created, the evidence behind them, and their intended uses. Furthermore, ongoing evaluation of the intended and unintended consequences that new assessments have on trainees should be conducted and communicated to trainees. Finally, trainees should participate in determining what patient care data are used to inform educational outcomes. The authors believe that trainee coproduction is critical to building stronger assessment systems that utilize evidence-based educational theories for improved learning and ultimately better patient care.

Evolving Licensure Examination: Assessing Student Confidence and Accuracy With Next Generation NCLEX

BACKGROUND

The Next Generation NCLEX (NGN) includes new item types. Little is known about nursing students' confidence and accuracy in answering these questions.

METHOD

A descriptive comparative study examined prelicensure nursing students' confidence and accuracy in answering NGN-style items versus multiple-choice questions (MCQs) of the same content via a 12-item quiz.

RESULTS

Less than one third of students (n = 194; 32.1%) reported feeling confident in answering NGN questions. Students' confidence levels had no relationship on scores with NGN items. When comparing NGN-style items to MCQs, students' (n = 221) scores on NGN-style items were lower with bowtie or a select-all-that-apply questions but higher with highlight table or matrix multiple-choice questions.

CONCLUSION

Students' lack of confidence with certain item types suggests faculty should incorporate these item types into classroom activities or course assignments. NGN test-taking strategies also should be incorporated and frequently reinforced throughout the curriculum. [J Nurs Educ. 2024;63(4):252-255.].

Exploring the curricular and pedagogical decision criteria for research-based learning design in undergraduate studies

Learning design has a multifaceted nature requiring a range of course- and institutional considerations. Analyzing the decision criteria's influence on research-based learning design helps understand the causes of the success/failure of the approach in achieving the teaching goals to improve the study programs. This study explores the interrelationship between the curricular and pedagogical criteria for research-based learning design decisions at the undergraduate level. For this purpose, the DEcision-MAking Trial and Evaluation Laboratory (DEMATEL) method is used to systematically analyze the decisive criteria and their causal relationships. Feedback from education professionals and university professors from Scandinavian universities is used to validate the pedagogy decision framework and provides input into the DEMATEL method. The student's role in the course is identified as the central criterion, featuring the highest level of interactions in the network of curricular and pedagogical decision criteria. Results are supportive of the identified institutional and course-specific criteria as prerequisites for the study outcomes.

Evaluative study of a MOOC on knowledge translation in five French-speaking countries

Over the past few years, the RENARD research team has observed a sharp increase in the need for knowledge translation (KT) training. Given the high demand, it has been nearly impossible for the team to provide training entirely in person, and so a massive open online course (MOOC) was developed. Its aim is to promote the use and appropriation of the KT process by practitioners, decision-makers, and others in the public sphere. The goal of this study was to evaluate the MOOC by collecting users' opinions, reactions, appropriation, and practice changes. A qualitative research design was used. Data were collected through semi-structured individual interviews (n = 16) with professionals from Canada, France, and three West African countries (Burkina Faso, Mali, and Senegal) who had taken the MOOC. All interviews were subjected to thematic content analysis. The MOOC content was generally appreciated and reused by the respondents. The results revealed one main motive for completing the course: the immediate opportunity to apply their learning in their practice environments. However, most respondents deplored the lack of interaction among learners and expressed the need for coaching with an instructor to deepen the topics covered during the course. The results also revealed connection and accessibility issues linked to the Internet network and unstable access to electricity in West African countries. The study highlights the potential of MOOCs for the acquisition of knowledge and competencies by KT professionals. Several recommendations and avenues of exploration were formulated to optimize and improve future designs of MOOCs on KT.

Identifying Medical Students in Difficulty Using Integration of Programmatic Assessment and Real-Time Evaluation Framework

Null.

Review of Objective Structured Clinical Examination Practices Within Pharmacy Programs Throughout the United States

OBJECTIVE

To update the description of current objective structured clinical examination (OSCE) practices within pharmacy schools in the United States and identify barriers to OSCE implementation and expansion.

METHODS

A survey was deployed to all accredited Doctor of Pharmacy programs within the United States. The survey was designed to collect information regarding the curricular mapping of OSCEs, OSCE design, OSCE delivery, assessment of OSCE performance, and barriers to OSCE implementation and expansion.

RESULTS

Of the 135 US-accredited programs identified, 109 (81%) programs completed the survey. In total, 93 (85%) programs reported using OSCEs to assess students; however, implementation throughout the curriculum and current practices varied by institution. Most programs place OSCEs within specific courses (96%), with the applied skills coursework being the most used (80%). The most common number of OSCEs that occur throughout a curriculum is 6 (18%), however, 20 (22%) programs execute 10 or more OSCEs throughout their curriculum. Forty (43%) programs use OSCEs as high-stakes assessments where poor performance could prevent a student from progressing to advanced pharmacy practice experiences. Of the responding programs, over half (56%) use teaching objective structured examinations to enhance learning. Common barriers to OSCE expansion exist and are related to resource utilization.

CONCLUSION

Significant expansion of OSCE development and implementation has occurred over the last decade. There is substantial variability in implementation and utilization among programs. Although standards of best practice for OSCEs exist for other health professions, best practices in pharmacy education have not been widely accepted or adopted.

Association between stereoacuity and simulated clinical performance among dental students: an exploratory investigation

INTRODUCTION

Dentistry is a profession that requires a high level of the hand-eye coordination. Although considerable attention has been put on dental students' manual dexterity, little emphasis was made on students' visual competency as a variable influencing their competency. In this study, we aimed to assess the association between dental students' stereopsis and their pre-clinical performance.

METHODS

Dental students (n = 49) were required to perform the porcelain-fused-to-crown preparation for a lower first molar and an upper central incisor plastic teeth mounted on phantom heads. Performance was assessed by two faculty members based on two rubrics, which collectively included 11 assessment criteria. Afterward, students undertook a stereoacuity test using Randot stereotest. Stereopsis values greater than 40 arc/s were considered abnormal.

RESULTS

A significant negative correlation was seen between stereopsis scores and students' grades for the upper incisor (p < 0.01), but not for the lower molar. An association was also seen between abnormal stereopsis and students' poor performance in preparing the finish line of the incisor tooth (p = 0.024).

CONCLUSION

Abnormal stereopsis can impact dental students' performance, particularly when preparing teeth indirectly using a dental mirror. Dental educators should give attention to students' stereoacuity, in a manner similar to that given to students' manual dexterity.

Comparison of industrial engineering programs in Colombia based on standardized test results

This study aims to evaluate the performance of industrial engineering programs across various Higher Education Institutions (HEIs) in Colombia, focusing on the SaberPRO test outcomes from 2016 to 2018. The evaluation considers three primary dimensions: the type of funding (public vs. private HEIs), geographical region, and program accreditation status. The methodology involved gathering and analyzing Saber Pro test results from students enrolled in industrial engineering programs at 82 Colombian HEIs during the specified period. In Colombia, the SaberPRO Exam, administered by the Colombian Institute for the Evaluation of Education (ICFES), is a critical measure for assessing the quality of future industrial engineering graduates. This research employs an analytical framework that includes Analysis of Variance (ANOVA) and hypothesis testing for mean comparisons to scrutinize the performance of these programs. Findings reveal notable disparities in performance between public and private institutions and between accredited and non-accredited programs. Additionally, through Multiple Factor Analysis (MFA), the study identifies the competencies with the highest discriminative capacity, recognizing 15 exemplary programs out of the 82 evaluated.

Demystifying AI: Current State and Future Role in Medical Education Assessment

ABSTRACT

Medical education assessment faces multifaceted challenges, including data complexity, resource constraints, bias, feedback translation, and educational continuity. Traditional approaches often fail to adequately address these issues, creating stressful and inequitable learning environments. This article introduces the concept of precision education, a data-driven paradigm aimed at personalizing the educational experience for each learner. It explores how artificial intelligence (AI), including its subsets machine learning (ML) and deep learning (DL), can augment this model to tackle the inherent limitations of traditional assessment methods.AI can enable proactive data collection, offering consistent and objective assessments while reducing resource burdens. It has the potential to revolutionize not only competency assessment but also participatory interventions, such as personalized coaching and predictive analytics for at-risk trainees. The article also discusses key challenges and ethical considerations in integrating AI into medical education, such as algorithmic transparency, data privacy, and the potential for bias propagation.AI's capacity to process large datasets and identify patterns allows for a more nuanced, individualized approach to medical education. It offers promising avenues not only to improve the efficiency of educational assessments but also to make them more equitable. However, the ethical and technical challenges must be diligently addressed. The article concludes that embracing AI in medical education assessment is a strategic move toward creating a more personalized, effective, and fair educational landscape. This necessitates collaborative, multidisciplinary research and ethical vigilance to ensure that the technology serves educational goals while upholding social justice and ethical integrity.

A Novel USMLE® Step 1 Based Approach to Introducing Radiation Oncology to Second-Year Preclinical Medical Students

There is a paucity of formalized exposure to Radiation Oncology (RO) for preclinical medical students across the United States as well as barriers to implementation within undergraduate medical education curriculum at many institutions. We present a novel approach to implementing an introductory RO didactic lecture to second-year medical students by interweaving associated oncological and ionizing radiation content represented on the United States Medical Licensing Exam® (USMLE®) Step 1 examination. Students had synchronous and asynchronous opportunities to engage with the 1.0-h didactic lecture administered by an attending Radiation Oncologist faculty member. Students were electronically invited to anonymously rank the effectiveness of the lecture materials on a 5-point Likert scale. Performance on standardized board-style questions regarding radiation biology and radiation side effects was recorded before and after the lecture and compared to the historic performance of previous institutional second-year medical student cohorts. The lecture material effectiveness received a mean score of 4.50 on a 5-point Likert scale. There was a statistically significant improvement in student performance on a board-style radiation side effect question from 39% on a pretest to 76% on a posttest. A USMLE® topic-based approach may be an effective way to implement a formalized introduction to RO to preclinical medical students while simultaneously improving performance on relevant standardized board-style questions. Providing evidence that RO topics appear on the USMLE® Step 1 examination curriculum was a powerful incentive for implementation when negotiating with curriculum offices.

A Look at Demographics and Transition to Virtual Assessments: An Analysis of Bias in the American Board of Surgery General Surgery Certifying Exams

OBJECTIVE

The goals of this study were (1) to assess if examiner ratings in the American Board of Surgery (ABS) General Surgery Cetifying Exam (CE) are biased based on the gender, race, and ethnicity of the candidate or the examiners, and (2) if the format of delivering of the exams, in-person or virtual, affects how examiners rate candidates.

DESIGN

We included every candidate-examiner combination for first time takers of the general surgery oral exam. Total scores and pass/fail outcomes based on the 4 scores given by examiners to candidates were analyzed using multilevel models, with candidates as random effects. Explanatory variables included the gender, race, and ethnicity of candidates and examiners, and the format of the exam (in-person or virtual). Candidates' first attempt scores on the ABS General Surgery Qualifying Exam (QE) were also included in the models to control for the baseline knowledge of the candidate. Three sets of models were evaluated for each demographic variable (gender, race, ethnicity) due to missingness in data. p-values and coefficients of determination R2 were used to quantify the statistical and practical significance of the model coefficients (an existent relationship between the explored variables on CE scores was considered statistically and practically significant if the p-value was lower than 0.01 and R2 higher than 1%).

PARTICIPANTS

All first-time takers of the American Board of Surgery General Surgery Certifying Exam from 2016 to 2022 that had demographic data, and the examiners that participated in those exams.

RESULTS

The number of candidates/examiners for the 3 sets of models was 8665/514 (gender), 5906/465 (race), and 4678/295 (ethnicity). The demographic variables, format of the exam, or their interactions were not found to significantly relate to examiner-candidate ratings or pass/fail outcomes. The only variable that was significantly related to CE scores was candidates' QE scores, which was added to the models as a measure of candidates' initial knowledge; this held for all models for total scores (F[1,8659] = 1069.89, p-value < 0.01, R2 = 5% [gender models], F(1,5696.3) = 589.13, p-value < 0.01, R2 = 5% [race models], F(1,4459.5) = 278.33, p-value < 0.01, R2 = 5% [ethnicity models]), and pass/fail outcomes (CI = 1.61-1.73, p-value < 0.01, R2 = 3% [gender models], CI = 1.67-1.85, p-value < 0.01, R2 = 3% [race models], CI = 2.17-2.90, p-value < 0.01, R2 = 3% [ethnicity models]).

CONCLUSIONS

This study shows that there is not a relationship between candidate and examiner gender, race, or ethnicity, and exam outcomes based on statistical models looking at examiner-candidate ratings and pass/fail outcomes. In addition, the delivery of the certifying exam in a virtual format appears to have no statistical impact on outcomes compared to in-person delivery. This suggests that the ABS is performing well in both demographic bias and virtual space.

Learning functional human anatomy with a new interactive three-dimensional digital tool

Human anatomy requires understanding spatial relationships among anatomical structures and is often perceived as difficult to learn by students. To overcome this concern, several digital tools exist with some strengths and limitations among which the lack of interactivity especially for complex functional anatomy learning. In this way, a new interactive three-dimensional tool called Antepulsio was designed. Antepulsio was assessed by comparing three groups of first year kinesiology students to test whether it is likely to favor functional anatomy learning during three training sessions spread over a week. The experiment was conducted during a real academic course. Laterality judgment, 3D spatial abilities and working memory abilities from all participants were previously collected to create three homogeneous groups: the active group (n = 17, 17.76 ± 0.56 years) interacted with Antepulsio, the passive group (n = 18, 17.89 ± 0.83 years) watched videos of Antepulsio while the control group (n = 15, 18.07 ± 0.80 years) performed a neutral activity unrelated to anatomy. Anatomy knowledge was also assessed during pretest, posttest, and retention test (8 weeks after the posttest). The most significant outcome of this study revealed that in case of better working visual memory, the active group outperformed the passive group between pretest and retention test (p < 0.01). In other words, Antepulsio tool is efficient only for students with high visuospatial working memory. These selective benefits of Antepulsio are discussed in terms of cognitive load, training duration and the necessary period of familiarization with the tool.