Students' performance in the different clinical skills assessed in OSCE: what does it reveal?

6 Months of Competency Based Medical Education for Undergraduate Medical Students- Challenges And Road Ahead

Competency based medical education (CBME) was introduced by National medical Commission (NMC) to improve the medical education to global standards. Lot of infrastructural and curricular reforms have been introduced. This article describes the 6 months experience, challenges and road ahead of CBME curriculum by otolaryngology faculties in a deemed to be university of South India.

Supplementary Information

The online version contains supplementary material available at 10.1007/s12070-022-03358-w.

Formative Objective Structured Clinical Examinations (OSCEs) as an Assessment Tool in UK Undergraduate Medical Education: A Review of Its Utility

The Objective Structured Clinical Examination (OSCE) is a globally established clinical examination; it is often considered the gold standard in evaluating clinical competence within medicine and other healthcare professionals' educations alike. The OSCE consists of a circuit of multiple stations testing a multitude of clinical competencies expected of undergraduate students at certain levels throughout training. Despite its widespread use, the evidence regarding formative renditions of the examination in medical training is highly variable; thus, its suitability as an assessment has been challenged for various reasons. Classically, Van Der Vleuten's formula of utility has been adopted in the appraisal of assessment methods as means of testing, including the OSCE. This review aims to provide a comprehensive overview of the literature surrounding the formative use of OSCEs in undergraduate medical training, whilst specifically focusing on the constituents of the equation and means of mitigating factors that compromise its objectivity.

Investigating the effectiveness of three-dimensionally printed anatomical models compared with plastinated human specimens in learning cardiac and neck anatomy: A randomized crossover study

Three-dimensional printing (3DP) technology has been increasingly applied in health profession education. Yet, 3DP anatomical models compared with the plastinated specimens as learning scaffolds are unclear. A randomized-controlled crossover study was used to evaluate the objective outcomes of 3DP models compared with the plastinated specimens through an introductory lecture and team study for learning relatively simple (cardiac) and complex (neck) anatomies. Given the novel multimaterial and multicolored 3DP models are replicas of the plastinated specimens, it is hypothesized that 3DP models have the same educational benefits to plastinated specimens. This study was conducted in two phases in which participants were randomly assigned to 3DP (n = 31) and plastinated cardiac groups (n = 32) in the first phase, whereas same groups (3DP, n = 15; plastinated, n = 18) used switched materials in the second phase for learning neck anatomy. The pretest, educational activities and posttest were conducted for each phase. Miller's framework was used to assess the cognitive outcomes. There was a significant improvement in students' baseline knowledge by 29.7% and 31.3% for Phase 1; 31.7% and 31.3% for Phase 2 plastinated and 3DP models. Posttest scores for cardiac (plastinated, 3DP mean ± SD: 57.0 ± 13.3 and 60.8 ± 13.6, P = 0.27) and neck (70.3 ± 15.6 and 68.3 ± 9.9, P = 0.68) phases showed no significant difference. In addition, no difference observed when cognitive domains compared for both cases. These results reflect that introductory lecture plus either the plastinated or 3DP modes were effective for learning cardiac and neck anatomy.

Types of clinical reasoning in a summative clerkship oral examination

INTRODUCTION

Dual-process theory characterizes clinical reasoning (CR) as Type 1 (intuitive) and Type 2 (analytical) thinking. This study examined CR on a summative clinical clerkship structured clinical oral examination (SCOE).

METHODS AND SUBJECTS

511 clinical clerks at the University of Toronto underwent SCOEs. Type 1, Type 2, and Global CR performance were compared to other internal medicine clerkship assessments using descriptive statistics and Spearman correlations.

RESULTS

Clinical clerks achieved mean marks >75% on the three clinical reasoning stations, on Type 1 and 2 CR tasks, and the overall SCOE. Performance on the SCOE CR stations correlated with each of the other clerkship assessments: written examination, inpatient, and ambulatory clinic assessments. The correlation of performance between Type 1 and Type 2 clinical reasoning tasks was statistically significant but weak (r_s = 0.28). This suggests that defined measures of Type 1 and Type 2 reasoning were indeed assessing distinct constructs.

CONCLUSION

Clinical clerks used both Type 1 and Type 2 reasoning with success. This study's characterization of Type 1 and Type 2 CR as separate domains, distinct from existing measures on the SCOE as well as the other clerkship assessments, can suggest a further addition to multimodal clerkship assessment.

Comparing Standard Setting Methods for Objective Structured Clinical Examinations in a Caribbean Medical School

BACKGROUND

OSCE are widely used for assessing clinical skills training in medical schools. Use of traditional pass fail cut off yields wide variations in the results of different cohorts of students. This has led to a growing emphasis on the application of standard setting procedures in OSCEs.

PURPOSE/AIM

The purpose of the study was comparing the utility, feasibility and appropriateness of 4 different standard setting methods with OSCEs at XUSOM.

METHODS

A 15-station OSCE was administered to 173 students over 6 months. Five stations were conducted for each organ system (Respiratory, Gastrointestinal and Cardiovascular). Students were assessed for their clinical skills in 15 stations. Four different standard setting methods were applied and compared with a control (Traditional method) to establish cut off scores for pass/fail decisions.

RESULTS

OSCE checklist scores revealed a Cronbach's alpha of 0.711, demonstrating acceptable level of internal consistency. About 13 of 15 OSCE stations performed well with "Alpha if deleted values" lower that 0.711 emphasizing the reliability of OSCE stations. The traditional standard setting method (cut off score of 70) resulted in highest failure rate. The Modified Angoff Method and Relative methods yielded the lowest failure rates, which were typically less than 10% for each system. Failure rates for the Borderline methods ranged from 28% to 57% across systems.

CONCLUSIONS

In our study, Modified Angoff method and Borderline regression method have shown to be consistently reliable and practically suitable to provide acceptable cut-off score across different organ system. Therefore, an average of Modified Angoff Method and Borderline Regression Method appeared to provide an acceptable cutoff score in OSCE. Further studies, in high-stake clinical examinations, utilizing larger number of judges and OSCE stations are recommended to reinforce the validity of combining multiple methods for standard setting.

Fourth-year medical students' experiences of diagnostic consultations in a simulated primary care setting

OBJECTIVES

The aim was to explore the experiences of fourth-year medical students of diagnostic consultations in a simulated primary care setting, in order to gain an insight into the suitability of such simulated consultations for assessing the diagnostic reasoning skills of medical students.

METHODS

This single-centre study employed a qualitative, cross-sectional design. Twelve fourth-year medical students volunteered to be filmed across 21 simulated, primary care consultations. The setting closely resembled OSCE stations, with a clinician present at each station monitoring the students' performance using a station-checklist. Upon completion of each station, participants reflected on their experiences using video-stimulated recall. Interviews were transcribed and analysed using Interpretative Phenomenological Analysis.

RESULTS

The simulated scenarios were often perceived to have limited fidelity with predictable outcomes. At times, preoccupation with the assessment checklist meant that students were more likely to focus on asking questions than interpreting the information they were gaining. Some students felt scrutinized during the consultations, while others struggled to adapt to the time pressure. Overall, the artificial setting seemed to promote a reductionist diagnostic approach and an attitude of 'ticking boxes' rather than engaging in active diagnostic reasoning.

CONCLUSIONS

The present findings call into question the assumption that observation-based assessment of the performance of medical students during simulated consultations can be reliably used to assess their diagnostic skills. Future studies need to explore how current assessment modalities could be better adapted to facilitate active engagement in diagnostic reasoning.

Assessing core competences of medical students with a test for flight school applicants

BACKGROUND

Important competences of physicians regarding patient safety include communication, leadership, stress resistance, adherence to procedures, awareness, and teamwork. Similarly, while selected, prospective flight school applicants are tested for the same set of skills. The aim of our study was to assess these core competences in advanced undergraduate medical students from different medical schools.

METHODS

In 2017, 67 medical students (year 5 and 6) from the universities of Hamburg, Oldenburg, and TU Munich, Germany, participated in the verified Group Assessment Performance (GAP)-Test at the German Aerospace Center (DLR) in Hamburg. All participants were rated by DLR assessment observers with a set of empirically derived behavioural checklists. This lists consisted of 6-point rating scales (1: very low occurrence to 6: very high occurrence) and included the competences leadership, teamwork, stress resistance, communication, awareness, and adherence to procedures. Medical students' scores were compared with the results of 117 admitted flight school applicants.

RESULTS

Medical students showed significantly higher scores than admitted flight school applicants for adherence to procedures (p < .001, d = .63) and communication (p < .01, d = .62). They reached significantly lower ratings for teamwork (p < .001, d = .77), stress resistance (p < 0.001, d = .70), and awareness (p < .001, d = 1.31). Students in semester 10 showed significantly (p < .02, d = .58) higher scores in domain awareness compared to the final year students. On average, flight school entrance level was not reached by either group for this domain.

CONCLUSIONS

Advanced medical students' low results for awareness are alarming as awareness is essential and integrative for clinical reasoning and patient safety. Further studies should elucidate and discuss whether awareness needs to be included in medical student selection or integrated into the curriculum in training units.

Medical knowledge and teamwork predict the quality of case summary statements as an indicator of clinical reasoning in undergraduate medical students

Background: Clinical reasoning refers to a thinking process including medical problem solving and medical decision making skills. Several studies have shown that the clinical reasoning process can be influenced by a number of factors, e.g. context or personality traits, and the results of this thinking process are expressed in case presentation. The aim of this study was to identify factors, which predict the quality of case summary statements as an indicator of clinical reasoning of undergraduate medical students in an assessment simulating the first day of residency. Methods: To investigate factors predicting aspects of clinical reasoning 67 advanced undergraduate medical students participated in the role of a beginning resident in our competence-based assessment, which included a consultation hour, a patient management phase, and a handover. Participants filled out a Post Encounter Form (PEF) to document their case summary statements and other aspects of clinical reasoning. After each phase, they filled out the Strain Perception Questionnaire (STRAIPER) to measure their situation dependent mental strain. To assess medical knowledge the participants completed a 100 questions multiple choice test. To measure stress resistance, adherence to procedures, and teamwork students took part in the Group Assessment of Performance (GAP) test for flight school applicants. These factors were included in a multiple linear regression analysis. Results: Medical knowledge and teamwork predicted the quality of case summary statements as an indicator of clinical reasoning of undergraduate medical students and explained approximately 20.3% of the variance. Neither age, gender, undergraduate curriculum, academic advancement nor high school grade point average of the medical students of our sample had an effect on their clinical reasoning skills. Conclusion: The quality of case summary statements as an indicator of clinical reasoning can be predicted in undergraduate medical students by their medical knowledge and teamwork. Students should be supported in developing abilities to work in a team and to acquire long term knowledge for good case summary statements as an important aspect of clinical reasoning.

Situational awareness within objective structured clinical examination stations in undergraduate medical training - a literature search

BACKGROUND

Medical students may not be able to identify the essential elements of situational awareness (SA) necessary for clinical reasoning. Recent studies suggest that students have little insight into cognitive processing and SA in clinical scenarios. Objective Structured Clinical Examinations (OSCEs) could be used to assess certain elements of situational awareness. The purpose of this paper is to review the literature with a view to identifying whether levels of SA based on Endsley's model can be assessed utilising OSCEs during undergraduate medical training.

METHODS

A systematic search was performed pertaining to SA and OSCEs, to identify studies published between January 1975 (first paper describing an OSCE) and February 2017, in peer reviewed international journals published in English. PUBMED, EMBASE, PsycINFO Ovid and SCOPUS were searched for papers that described the assessment of SA using OSCEs among undergraduate medical students. Key search terms included "objective structured clinical examination", "objective structured clinical assessment" or "OSCE" and "non-technical skills", "sense-making", "clinical reasoning", "perception", "comprehension", "projection", "situation awareness", "situational awareness" and "situation assessment". Boolean operators (AND, OR) were used as conjunctions to narrow the search strategy, resulting in the limitation of papers relevant to the research interest. Areas of interest were elements of SA that can be assessed by these examinations.

RESULTS

The initial search of the literature retrieved 1127 publications. Upon removal of duplicates and papers relating to nursing, paramedical disciplines, pharmacy and veterinary education by title, abstract or full text, 11 articles were eligible for inclusion as related to the assessment of elements of SA in undergraduate medical students.

DISCUSSION

Review of the literature suggests that whole-task OSCEs enable the evaluation of SA associated with clinical reasoning skills. If they address the levels of SA, these OSCEs can provide supportive feedback and strengthen educational measures associated with higher diagnostic accuracy and reasoning abilities.

CONCLUSION

Based on the findings, the early exposure of medical students to SA is recommended, utilising OSCEs to evaluate and facilitate SA in dynamic environments.