Medical students' perception of objective structured clinical examination: a feedback for process improvement

A realist evaluation of how, why and when objective structured clinical exams (OSCEs) are experienced as an authentic assessment of clinical preparedness

INTRODUCTION

Whilst rarely researched, the authenticity with which Objective Structured Clinical Exams (OSCEs) simulate practice is arguably critical to making valid judgements about candidates' preparedness to progress in their training. We studied how and why an OSCE gave rise to different experiences of authenticity for different participants under different circumstances.

METHODS

We used Realist evaluation, collecting data through interviews/focus groups from participants across four UK medical schools who participated in an OSCE which aimed to enhance authenticity.

RESULTS

Several features of OSCE stations (realistic, complex, complete cases, sufficient time, autonomy, props, guidelines, limited examiner interaction etc) combined to enable students to project into their future roles, judge and integrate information, consider their actions and act naturally. When this occurred, their performances felt like an authentic representation of their clinical practice. This didn't work all the time: focusing on unavoidable differences with practice, incongruous features, anxiety and preoccupation with examiners' expectations sometimes disrupted immersion, producing inauthenticity.

CONCLUSIONS

The perception of authenticity in OSCEs appears to originate from an interaction of station design with individual preferences and contextual expectations. Whilst tentatively suggesting ways to promote authenticity, more understanding is needed of candidates' interaction with simulation and scenario immersion in summative assessment.

Medical student perceptions of assessments of clinical reasoning in a general surgery clerkship

BACKGROUND

Components factoring into general surgery clerkship grades vary by institution, and while evaluators attempt to remain unbiased when evaluating medical student performance, subjectivity and implicit bias remain an issue. Our institution recently implemented a case-based structured oral examination to provide the general surgery clerkship director objective insight into students' clinical reasoning skills. We hypothesized that medical students believe this exam, along with graded clinical documentation and the Observed Standardized Clinical Encounter (OSCE), are fair assessments and increase students' awareness of their clinical reasoning skills.

METHODS

A survey was sent to third-year medical students in the classes of 2023 and 2024 at our institution who had completed their general surgery clerkship. Students rated five grading assessments (i.e., preceptor evaluations, the oral examination, clinical documentation, the OSCE, and the shelf exam) on fairness and the ability of the assessment to give them insight into their clinical reasoning on a five-point Likert scale 1-5 (with 1 = Strongly Agree, 5 = Strongly Disagree).

RESULTS

One hundred and ten of 162 (67.9%) students responded to the survey. The shelf examination was the most highly regarded assessment tool followed by the oral examination. Seventy-three percent agreed or strongly agreed that the oral exam was a fair assessment, and 80% agreed or strongly agreed that it gave them insight into their clinical reasoning skills. Alternatively, only 41.8% of students agreed or strongly agreed that preceptor evaluations were fair assessments and 42.7% agreed or strongly agreed that it gave them insight into their clinical reasoning.

CONCLUSIONS

Third-year medical students on a general surgery clerkship favor the shelf examination and a case-based oral examination over other assessment tools regarding fairness and perception of their clinical reasoning. This type of examination can provide general surgery clerkship directors with additional objective data to assess medical students more fairly and improve students' clinical reasoning.

Implementation of peer-to-peer oral care objective structured clinical examination-based learning for nursing students

AIM

Nurses' skills and confidence in providing oral care to patients play important roles in healthcare. Studies have proved that peer-to-peer learning and assessment can help improve nurses' skills and confidence. However, no study has investigated it using an objective assessment checklist. This study investigated an oral hygiene practice training program for nursing students. It focused on their oral healthcare skills, confidence, and perspectives regarding implementing an objective structured clinical examination (OSCE).

METHODS

A pre- and post-survey of 97 nursing students from a nursing school in Japan was conducted as a peer-to-peer oral health education training program that included an assessment and performance checklist.

RESULTS

Confidence in performing and assessing oral care significantly improved after attending the program. Furthermore, the changes were significantly greater in the developed program than in the training program without the checklist. The mean total assessment scores (0-25) of the first, second, third, and fourth participants in the four-student groups were 22.1, 23.4, 23.9, and 24.1, respectively. The mean scores of the second, third, and fourth participant groups were significantly higher than that of the first participant group (p < .01). Fifty-two positive and 58 negative comments were received regarding the implementation of the program with professional assessments and the checklist. The most common positive comment mentioned that the evaluation by professionals improved oral care learning and skills.

CONCLUSIONS

This study developed a program to improve nursing students' confidence and oral care delivery skill building in an OSCE-style delivery.

Evaluation of an OSCE's implementation and a two-step approach for a theoretical and practical training program in Obstetrics and Gynecology

Objective structured clinical examination (OSCE) is a well-known assessment method to evaluate clinical skills and competence in healthcare. Following the recently reformed National Competence-Based Catalog of Learning Objectives in Medicine, the implementation of this assessment method in the training program for medical students is now obligatory in Germany. This major change requires a reorganization not only of the training programs but also of the students themselves and the way they learn. We performed a poll evaluating the students' opinions regarding these major changes and the implementation of the OSCE with a new training program. To implement this assessment method and to evaluate the OSCE, Kern's six-step approach comprising (1) problem identification and general needs assessment, (2) needs assessment of the targeted learners, (3) goals and objectives, (4) educational strategies, (5) implementation, and (6) evaluation and feedback was applied. To evaluate and gather feedback, a poll was used to analyze the student's opinions regarding OSCE in gynecology and obstetrics and OSCE in general, in addition to the regular analysis of the students' results. To reform the educational strategy, a two-step approach was developed: First, the students completed the regular training program and a written examination, and second, they participated in a 1-week clerkship, in small group teaching, and in the OSCE. The OSCE stations were developed primarily based on the National Competence-Based Catalog and the German Catalog of Learning Objectives in Medicine, as well as on the feedback of experts reflecting their expectations for physicians beginning their careers. The students performed well in the OSCE and gave positive feedback regarding this examination method. Furthermore, they welcomed the upcoming changes by considering OSCE a valuable assessment tool, and they showed appreciation for the two-step approach by supporting the combination of an OSCE and a written examination. Thus, this article presents the implementation of an OSCE and a strategy for the adaptation of the curriculum to fulfill the new OSCE requirements and-to our knowledge-reveals students' primary opinions regarding the changes in their medical training program for the first time.

Use of Objective Structured Clinical Examination (OSCE) in a hybrid digital / in-person training for hormonal IUD in Nigeria: findings and applications of the approach

Background: The hormonal intrauterine device, a long-acting reversible contraceptive method, is being introduced to pilot sites in the private and public sector in Nigeria by the Nigerian Federal Ministry of Health since 2019. To inform training of health care providers, a study was conducted on a hybrid digital and in-person training which utilized Objective Structured Clinical Examination (OSCE) to assess competency of provider trainees. This study represents one of few documented experiences using OSCE to assess the effectiveness of a digital training. Methods: From September - October 2021, in Enugu, Kano and Oyo states of Nigeria, 62 health care providers from public and private sector health facilities were trained in hormonal IUD service provision using a hybrid digital / in-person training approach. Providers, who were skilled in provision of copper IUD, underwent a didactic component using digital modules, followed by an in-person practicum, and finally supervised service provision in the provider trainee's workplace. Skills were assessed using OSCE during the one-day practicum. Results: Use of the OSCE to assess skills provided valuable information to study team. The performance of provider trainees was high (average 94% correct completion of steps in the OSCE). Conclusions: OSCE was used as a research methodology as part of this pilot study; to date, OSCE has not been integrated into the training approach to be scaled up by FMOH. Uniformly high performance of provider trainees was seen on the OSCE, unsurprising since provider trainees were experienced in providing copper IUD. If and when training is rolled out to providers inexperienced with copper IUD, OSCE may have a more important role to assess skills before service provision. The role of OSCE in design of hybrid digital / in-person training approaches should be further explored in rollout of hormonal IUD and other contraceptive technologies.

Enhancing Final-Year Medical Students’ Clinical Examinations Performance via a Transdiagnostic ACT-Based Intervention

Introduction: Performance anxiety is a transdiagnostic construct similar in both sports and clinical medical examinations. Given that acceptance and commitment therapy (ACT) and other similar therapies have transdiagnostic evidence in improving the performance of athletes, the same approach can be adopted for medical students in improving their performance during clinical examinations. This pilot study aimed to assess the efficacy of a brief ACT-based intervention in improving transdiagnostic performance anxiety in clinical medical students through assessing changes in psychopathology scores (depression, anxiety, and stress) and psychological process variables (psychological flexibility, cognitive fusion, and mindfulness). Methods: Final-year medical students were randomized into intervention and control groups. A one-day ACT-based intervention was delivered to an intervention group, with the control group crossing over one month post intervention. Both groups filled in sociodemographic questionnaires and questionnaires measuring psychological flexibility, cognitive fusion, mindfulness, depression, anxiety, and stress at three time points: T1 (before intervention), T2 (immediately after intervention), and T3 (one month post intervention). Repeated measures ANOVA was employed to assess the change between the intervention and control groups over time. Results and Conclusions: There was a significant change in anxiety scores from T1 to T3. In addition, there were significant improvements in mindfulness, cognitive fusion, and psychological flexibility scores over time in the intervention group compared with the control group. This pilot study builds on small single-sample evidence bases for the efficacy of an ACT-based intervention in non-sports performance enhancement, suggesting that larger-scale randomized trials of similar interventions in clinical medical students may prove equally efficacious.

Objective structured clinical examinations (OSCEs) for students' training and assessment in the French respiratory medicine departments in 2021: An overview

BACKGROUND

Medical professional performances can be assessed by objective structured clinical examinations (OSCEs) where medical trainees go through a series of simulated clinical situations. OSCEs are now the gold standard for the assessment of medical students' competence during their training. In France, the first national OSCEs will take place in May 2024 and respiratory teachers will be involved in this reform and will use OSCEs for students' training and assessment in respiratory medicine. Students training regarding this final OSCE may vary across medical faculties and may impact students' results. Therefore, we aimed to provide a national overview of OSCE's training performed by respiratory teachers and their interest in developing a common French databank of OSCEs.

METHODS

We conducted a national anonymous online survey among the members of the French college of respiratory teachers (CEP), from 2021 February the 15th to 2021 June the 15th. The survey consisted of 32 questions.

RESULTS

Among 118 French pulmonologists teachers, 52 (45%) responded to the survey. We received a response from at least one of each of the French Medical Universities. Twenty-two (42%) had received specific training on how to conduct an OSCE. Twenty-eight (54%) of respondents used OSCEs for training purposes and 24 (46%) for assessment purposes, for less than 1 year in more than half of the participants. The average satisfaction scores out of 10 about OSCEs was 7.3 ± 1.7 for training and 7.4 ± 1.5 for students' assessment. Respondents were willing (8.9 ± 1.8 out of ten) to develop a common databank to share OSCEs subjects in respiratory medicine in France.

CONCLUSIONS

This survey confirms heterogeneity in the training and the use of OSCEs among French respiratory teachers. However, a common national databank could be a useful tool to reduce these disparities.

Attitude and communication skills of German medical students

Objective

While the development of communication competencies in medical schools plays a pivotal role in the curriculum, studies show that students’ communication skills and patient-centred attitudes may vary based on gender and ethnicity. The goal of this study was to investigate the socio-demographic factors that influence medical students’ communication abilities and, more specifically, to what extent their attitude toward communication skills learning and patient orientation associate with communication abilities. Our population included medical students admitted in 2017. Used tools included a communication score, the patient-provider orientation and communication skills attitudes scales.

Results

Three hundred and sixty-five students participated in the study (56.4% female, 85.2% German native speakers, mean age 24.2 ± 3.5 years). Female and German native speaking students had a better communication skills OSCE performance, were more patient-oriented and had more positive attitudes toward communication skills learning than male and non-native speaking students. There was a significant association between gender, native tongue, attitudes towards communication skills learning and communication skills OSCE performance. In conclusion, to support medical students to improve their communication proficiency and attitudes towards the importance of clear communication and patient-oriented care, medical educators should consider teaching and assessment strategies that address socio-cultural aspects of communication.

Perception of Students and Examiners about Objective Structured Clinical Examination in a Teaching Hospital in Ethiopia

INTRODUCTION

The objective structured clinical examination (OSCE) has become a standard assessment tool in undergraduate medical school training. It is considered an objective assessment of practical skill of students. OSCE is a resource demanding assessment method that can have numerous challenges. Comprehensive assessment of perception regarding OSCE can help identify areas that need improvement. The aim of this study was to assess the perception of students and examiners towards OSCE.

METHODS

A cross-sectional study was conducted on students and examiners undertaking OSCE from May 1 to July 30, 2021, using a structured questionnaire. Comparison of variables was done using Mann-Whitney U-Test and Chi-square test. P-value <0.05 was considered statistically significant.

RESULTS

A total of 141 students and 39 examiners participated in the study. The majority of the students and examiners had a positive response regarding the attributes, structure, organization and validity of OSCE. It was recommended to be used in future exams compared to other assessments by 38.3% of students and 51.3% of examiners. There were certain challenges reported by students and included stressfulness of the exam (51.1%), inadequate time (27.6%), and unsatisfactory orientation (30.5%). One-third of examiners considered it stressful, while 20.5% considered the time provided to be inadequate. Equipment to conduct the exam was considered inadequate by 39.1% and 56.4% of students and examiners, respectively. Around 80.1% of students recommended mock sessions and 23.1% of examiners did not have any prior training on OSCE.

CONCLUSION

An overall positive perception of OSCE by students and examiners was seen. Certain challenges that need improvements were identified. Continuing evaluation and refinement of OSCE by departments is needed. We recommend further wide-scale national evaluation of the OSCE examination system of medical students.

Perception of Medical Undergraduate Students about an Electronic Practical Examination In Orthopaedics During the COVID-19 Pandemic

BACKGROUND/PURPOSE OF STUDY

No study has evaluated the perception of medical undergraduate students to the electronic objective structured practical examination (e-OSPE) in orthopaedics. The aim of the present study is to evaluate the feasibility and perception of the medical undergraduate students to e-OSPE conducted by the department of Orthopaedics to assess problem-solving and clinical decision-making skills of medical undergraduate students.

METHODS

Medical undergraduate students of second and third year at our medical college who completed the orthopaedic clinical posting and appeared for the orthopaedic practical examination during the second wave of the COVID-19 pandemic were included in this prospective observational study. Students appearing for the exams from 20th March 2021 to 26th June 2021 were invited to complete the questionnaire immediately after the e-OSPE. Internal consistency of the survey questions was assessed using Cronbach's alpha.

RESULTS

272 out of 312 eligible students completed the survey and the survey response rate was 87.2%. Nine groups of medical undergraduate students gave the orthopaedic practical exams from 20th March 2021 to 26th June 2021. 91.2% students felt that the e-OSPE represented a valid modality of evaluation of essential orthopaedic practical knowledge during the COVID-19 pandemic. The overall reliability of the 19 questions included in our survey was very high (Internal consistency: Cronbach's alpha = 0.88).

CONCLUSION

The e-OSPE was well received by the medical undergraduate students at our institute and the students had a positive perception about the new examination technique used in orthopaedics during the COVID-19 pandemic.

Examiners' Perceptions in Surgical Education: The Blind Spot in the Assessment of OSCEs

PURPOSE OF THE ARTICLE

Interpreting objective structured clinical examination (OSCE) scores remains a challenging task.1 It remains unclear how examiners make a decision to score an OSCE the way they do. The examiners' thoughts and decision-making processes remain a hidden part of the OSCE assessment. One cannot assume that all examiners have a shared belief about what is important in an OSCE or what is the aim of an OSCE assessment. These differences are problematic, since they lead to students not getting the 'correct' score for their OSCE.

MATERIALS AND METHODS

Using a qualitative case study design,2 this study aims to gain in-depth insight into examiners' perceptions of OSCEs and their role as an examiner. For this purpose, semi-structured interviews with OSCE observers (n = 6) were conducted.

RESULTS AND CONCLUSIONS

The results show that the perception of examiners on the aim of OSCES and their motivation and task as examiner are crucial in understanding the differences among OSCE scores of examiners. Training and discussion among the examiners are needed to decrease the differences among the OSCE scores of examiners.

Impact of integrating objective structured clinical examination into academic student assessment: Large-scale experience in a French medical school

Purpose

Objective structured clinical examinations (OSCE) evaluate clinical reasoning, communication skills, and interpersonal behavior during medical education. In France, clinical training has long relied on bedside clinical practice in academic hospitals. The need for a simulated teaching environment has recently emerged, due to the increasing number of students admitted to medical schools, and the necessity of objectively evaluating practical skills. This study aimed at investigating the relationships between OSCE grades and current evaluation modalities.

Methods

Three-hundred seventy-nine 4^th-year students of University-of-Paris Medical School participated to the first large-scale OSCE at this institution, consisting in three OSCE stations (OSCE#1–3). OSCE#1 and #2 focused on cardiovascular clinical skills and competence, whereas OSCE#3 focused on relational skills while providing explanations before planned cholecystectomy. We investigated correlations of OSCE grades with multiple choice (MCQ)-based written examinations and evaluations of clinical skills and behavior (during hospital traineeships); OSCE grade distribution; and the impact of integrating OSCE grades into the current evaluation in terms of student ranking.

Results

The competence-oriented OSCE#1 and OSCE#2 grades correlated only with MCQ grades (r = 0.19, P<0.001) or traineeship skill grades (r = 0.17, P = 0.001), respectively, and not with traineeship behavior grades (P>0.75). Conversely, the behavior-oriented OSCE#3 grades correlated with traineeship skill and behavior grades (r = 0.19, P<0.001, and r = 0.12, P = 0.032), but not with MCQ grades (P = 0.09). The dispersion of OSCE grades was wider than for MCQ examinations (P<0.001). When OSCE grades were integrated to the final fourth-year grade with an incremental 10%, 20% or 40% coefficient, an increasing proportion of the 379 students had a ranking variation by ±50 ranks (P<0.001). This ranking change mainly affected students among the mid-50% of ranking.

Conclusion

This large-scale French experience showed that OSCE designed to assess a combination of clinical competence and behavioral skills, increases the discriminatory capacity of current evaluations modalities in French medical schools.

Five years after commencing the objective structured clinical examination: are we getting it right? Medical students' assessment as the measuring index

Introduction:

The aim of this study was to assess our use of OSCE from the perception of final year medical students.

Materials and methods:

This is a cross sectional survey of final year medical students undergoing the final examination in Surgery. All 102 medical students in the class were given the self-administered questionnaire to fill. The data were collated into excel spreadsheets and analysed using the SPSS version 21.

Results:

A total of 79 completed questionnaires were retrieved (return rate of 78%).

All the students knew about and had participated previously in OSCE, 94.9% accorded the OSCE fair. 76(93.2%) wanted the examination as the main method of clinical assessment, 38(46.6%) had adequate preparation.

In terms of the OSCE stations not mirroring real clinical scenarios, 38(48.1%) disagreed, and 26(20.5%) strongly disagreed. 34(43%) disagreeing that the logistics was poor. The students rating of the OSCE, on a scale of 1 – 10, gave a mean score of >8 for spread of the OSCE stations, detail of the questions and objectivity of the examination. The nearness to clinical reality was rated as 7.52 with artificiality of the stations rated as 4.12.

Conclusion:

The OSCE has gained acceptance amongst final year medical students.

Turning Objective Structured Clinical Examinations into Reality

Objective structured clinical examinations assess learners "showing how" to perform complex clinical tasks. Devised as summative evaluations, these examinations with immediate feedback are useful formative evaluations to improve learner performance. This review describes how objective structured clinical examinations have been used in rheumatology education. Steps for creating an objective structured clinical examination are discussed. Validity and reproducibility are important considerations, especially for high-stakes summative objective structured clinical examinations. Consideration of the potential benefits in clinical education and their hazards are reviewed. When well-designed, formative objective structured clinical examinations have high educational value for learners and medical educators.

Clinical Examination among Medical Students: Assessment and Comparison of the Strengths and Weaknesses of Objective Structured Clinical Examination and Conventional Examination

Background:

Traditional clinical examination is perceived to be biased and to overcome this, objective structured clinical examination (OSCE) was introduced.

Aims:

The aim was to assess the students’ exposure and perception of OSCE as well as its strengths and weaknesses in comparison to conventional clinical examination.

Subjects and Methods:

A cross-sectional study of students who had surgical OSCE was conducted from July 2018 to October 2018 in Ahmadu Bello University, Zaria. Using a structured questionnaire, exposure and perception of OSCE as well its strength and weaknesses in comparison to the conventional clinical examination were assessed with dichotomous questions and a 5-point Likert scale. Data were analyzed with SPSS Version 20.

Results:

One hundred and thirty-four students responded, mean aged 24.1 ± 4.3 years, with a male-to-female ratio of 2.5:1. Although 64.7% of them strongly agreed that OSCE is the standard mode of examination, only 36.1% strongly agreed that it was easier to pass. More than half of the students perceived that the content of the OSCE was appropriate, wide-scoped, unbiased, and brought out specific areas of weakness. Some students (38.1%), however, felt that there was a need for improvement in the OSCE process, especially in time allocation, but most of them (89.5%) prefer it and agree that it was superior to conventional clinical examination.

Conclusions:

The perception of the process and structure of OSCE among the medical students was good. Compared to conventional clinical examination, students found OSCE was easier and fairer, had a better examination of structure and level of coordination, and induced lesser anxiety and fatigue. However, the entire process of OSCE can be improved.

Between fairness and fear-Dental undergraduates' attitudes towards objective structured clinical examinations

INTRODUCTION

Objective structured clinical examinations (OSCEs) are an established method of assessment for dental undergraduates. There is little published dental education research exploring the views of undergraduates towards OSCEs.

AIM

To explore and describe the views of dental undergraduates towards OSCEs.

METHOD

A sequential mixed methods design was applied. This included a cross-sectional survey of all 204 dental undergraduates in Years 2, 3 and 4 within a UK dental school using an anonymous self-report online questionnaire (response rate 57%), which was followed by two focus group interviews of n = 10 Year-2 students.

RESULTS

Most students gave positive views regarding the OSCE form of assessment. Questionnaire data highlighted that students felt that the OSCE tested their diagnostic, clinical and communication skills, and covered a wide range of skills and disciplines. Students also generally saw them as a useful educational exercise that went beyond testing recall of facts. Most students thought that the OSCEs were "fair" assessments. Negative views focused around student lack of preparation for them, especially in earlier years. Focus group data explored further these concerns but also emphasised the need for more student-centred support pre-OSCE.

CONCLUSION

Objective structured clinical examinations can be an anxiety-inspiring form of assessment for students in the early years of the programme. Dental educators need to be more aware of these concerns to develop strategies to increase student preparedness for OSCEs without increasing stress.

Impact of video feedback system on medical students' perception of their clinical performance assessment

BACKGROUND

Providing feedback on student performance in the clinical performance assessment (CPA) is meaningful in that it helps students understand their strengths and weaknesses. This study compared students' perception of their CPA scores before and after providing personalized video feedback.

METHODS

Two identical online surveys of Year 1 medical students (N = 103) that had undergone CPA were conducted to evaluate students' perceptions about their CPA scores before and after video feedback. Students were given their test scores with assessment analysis reports immediately after completing the CPA. Top-scored students from each station agreed to provide their video-recorded performance to the rest of the students.

RESULTS

After comparing their performance video and top-scored video at each station, medical students were more aware of their CPA total score, clinical performance examination (CPX) total score, score of each CPX station, section score for the CPX station, history taking section score, physical examination section score, and doctor-patient relationship section score. Moreover, students became more convinced of their own weaknesses from their history taking and patient education section after viewing video feedback than before.

CONCLUSION

The use of the video feedback system might help students recognize their CPA results and identify their strengths and weaknesses.

Exploring the Influence of Language on Assessment Given a Mismatch Between Language of Instruction and Language of Practice

STATEMENT

A phenomenon is occurring in international settings where the language of program delivery and assessment does not match the primary language of practice. It is unknown whether determining competence in English disadvantages students for practice in non-English settings. As such, we conducted a pilot study to determine student performance and perceptions after completion of two Objective Structured Clinical Examinations (OSCEs) examinations, one conducted in English and one conducted in Arabic within an Arabic-speaking Middle Eastern setting. Twenty-two students completed both OSCEs. Overall scores were similar but student rankings differed. Students were more confident performing in Arabic, felt that the Arabic examination was more reflective of practice, and believed that use of Arabic OSCEs can promote better patient care. Findings support the notion that student success may be influenced by language of assessment and that we may need to rethink how we determine assessment validity in these emerging international education settings.

Analysis of Psychophysiological Stress Response in Higher Education Students Undergoing Clinical Practice Evaluation

The aim of this study was to analyze the psychophysiological response of Physiotherapy degree students during a complete clinical practice as well as their psychological profile in relation to coping style in stressful situations, and its impact on academic performance. We analysed in 15 students heart rate variability (HRV) to analyse autonomic modulation in three moments during the entire clinical practices of 3 months (4 h per day, 3 days per week), first session, middle session and last session; perceived stress, personality, life engagement, cognitive flexibility before starting the clinical practice the academic performance after the clinical practice. Results based on HRV and related indexes parameters fail to indicate the expected autonomic adaptation during the practice. A complete clinical practice of Physiotherapy degree students did not produce an habituation process since a high sympathetic autonomic nervous system modulation was measured in the beginning, at the middle and at the final of the clinical practice. Below-average scores are presented in a personality factor associated with traditional and non-adaptive coping styles that could explain the non-habituation of the students. Finally, none of the analyzed indexes has been able to relate to academic performance.

OSCE as a Summative Assessment Tool for Undergraduate Students of Surgery-Our Experience

Traditional examination has inherent deficiencies. Objective Structured Clinical Examination (OSCE) is considered as a method of assessment that may overcome many such deficits. OSCE is being increasingly used worldwide in various medical specialities for formative and summative assessment. Although it is being used in various disciplines in our country as well, its use in the stream of general surgery is scarce. We report our experience of assessment of undergraduate students appearing in their pre-professional examination in the subject of general surgery by conducting OSCE. In our experience, OSCE was considered a better assessment tool as compared to the traditional method of examination by both faculty and students and is acceptable to students and faculty alike. Conducting OSCE is feasible for assessment of students of general surgery.

Dental students' concerns regarding OSPE and OSCE: a qualitative feedback for process improvement

OBJECTIVES

Objective structured practical examination (OSPE) and objective structured clinical examination (OSCE) have become established as reliable, valid and objective methods of assessing practical and clinical skills in dental schools. This study explored the perceptions of dental undergraduates' regarding OSPE and OSCE.

DESIGN AND SETTING

Two focus groups were made; the first consisted of students who had recently undergone an OSPE, while the other group was of fresh graduates (FG) who had given an OSCE in the final examination. A trained facilitator conducted the discussion-based interview for each focus group. Both discussions were recorded via audio recorders and transcribed verbatim. The data were thereafter analysed thematically.

RESULTS

Findings from the study suggested that the students and FGs were generally satisfied with the OSPE and OSCE. However, they perceived that the time allocated to the stations was not well balanced, nor were the examiners trained to conduct the examination. More importantly, the FGs opined that practical skills were not adequately tested on the OSCE, and thus the curricular content was not adequately covered.

CONCLUSION

The study highlights issues that may arise while conducting the OSPE and OSCE, thus informing future guidelines for conducting OSPE and OSCE.

Efficacy and Feasibility of Objective Structured Clinical Examination in the Internal Assessment for Surgery Postgraduates

INTRODUCTION

Traditionally assessment in medical training programs has been through subjective faculty evaluations or multiple choice questions. Conventional examinations provide assessment of the global performance rather than individual competencies thus making the final feedback less meaningful. The objective structured clinical examination (OSCE) is a relatively new multidimensional tool for evaluating training. This study was carried out to determine the efficacy and feasibility of OSCE as a tool for the internal assessment of surgery residents.

METHODS

This study was carried out on the marks obtained by surgery residents at different levels of training in a single tertiary center in India over the 4 OSCEs conducted in the years 2015 and 2016. The marks of the OSCE were collected from the departmental records and analyzed. Reliability was calculated using internal consistency using Cronbach's α. Validity was calculated by item total correlation. Content validation was done by obtaining expert reviews from 5 experts using a proforma, to assess the content and checklist of each station of the OSCE.

RESULTS

A total of 49 surgery residents were assessed in small batches during the above mentioned period. Of the 4 OSCEs conducted by us, 3 had a high value of Cronbach's α of greater than 0.9, as opposed to the set standard of 0.7. Out of 23 stations used in the 4 examinations separately, only 3 stations were found to have a low correlation coefficient (item total correlation), and hence, a low validity. The remaining 20 stations were found to have a high validity. Expert review showed unanimous validation of the content of 17 out of the 23 stations, with few suggestions for change in the remaining 6 stations. The material and manpower used was minimal and easy to obtain, thus making the OSCE feasible to conduct.

CONCLUSION

OSCE is a reliable, valid. and feasible method for evaluating surgery residents at various levels of training.

Does Changing Examiner Stations During UK Postgraduate Surgery Objective Structured Clinical Examinations Influence Examination Reliability and Candidates' Scores?

OBJECTIVE

Objective structured clinical examinations (OSCE) are widely used for summative assessment in surgery. Despite standardizing these as much as possible, variation, including examiner scoring, can occur which may affect reliability. In study of a high-stakes UK postgraduate surgical OSCE, we investigated whether examiners changing stations once during a long examining day affected marking, reliability, and overall candidates' scores compared with examiners who examined the same scenario all day.

DESIGN, SETTING, AND PARTICIPANTS

An observational study of 18,262 examiner-candidate interactions from the UK Membership of the Royal College of Surgeons examination was carried at 3 Surgical Colleges across the United Kingdom. Scores between examiners were compared using analysis of variance. Examination reliability was assessed with Cronbach's alpha, and the comparative distribution of total candidates' scores for each day was evaluated using t-tests of unit-weighted z scores.

RESULTS

A significant difference was found in absolute scores differences awarded in the morning and afternoon sessions between examiners who changed stations at lunchtime and those who did not (p < 0.001). No significant differences were found for the main effects of either broad content area (p = 0.290) or station content area (p = 0.450). The reliability of each day was not affected by examiner switching (p = 0.280). Overall, no difference was found in z-score distribution of total candidate scores and categories of examiner switching.

CONCLUSIONS

This large study has found that although the range of marks awarded varied when examiners change OSCE stations, examination reliability and the likely candidate outcome were not affected. These results may have implications for examination design and examiner experience in surgical OSCEs and beyond.

Dental Undergraduate Views of Objective Structured Clinical Examinations (OSCEs): A Literature Review

Objective Structured Clinical Examinations (OSCEs) are widely used in dental undergraduate assessment, often determining progression or graduation. Student evaluation of this assessment process is important, and this includes identifying the views of the student. The aim of this paper is to present a review of the current literature regarding dental student perceptions of OSCEs. A search of the PubMed database covering the period 1975 to 2015 identified 121 possible papers from which only six were suitable for review. The remaining papers were excluded due to them not reporting on dental undergraduate views. Students perceived the OSCE to be a valid assessment in three studies, but not in one. The educational benefit of an OSCE is well supported by these studies. OSCEs can induce high levels of anxiety compared to other forms of assessments, but this did not affect student performance. The majority of students would chose to have a similar format of assessment again, showing support for OSCEs. Further research using larger cohorts of students could be undertaken in order to support these finding which would give added evidence for the continuing use of OSCEs as a valid method of both dental undergraduate education and assessment.

Exploring personality dimensions that influence practice and performance of a simulated laparoscopic task in the objective structured clinical examination

BACKGROUND

Surgical educators have encouraged the investigation of individual differences in aptitude and personality in surgical performance. An individual personality difference that has been shown to influence laparoscopic performance under time pressure is movement specific reinvestment. Movement specific reinvestment has 2 dimensions, movement self-consciousness (MS-C) (i.e., the propensity to consciously monitor movements) and conscious motor processing (CMP) (i.e., the propensity to consciously control movements), which have been shown to differentially influence laparoscopic performance in practice but have yet to be investigated in the context of psychological stress (e.g., the objective structured clinical examination [OSCE]).

OBJECTIVE

This study investigated the role of individual differences in propensity for MS-C and CMP in practice of a fundamental laparoscopic skill and in laparoscopic performance during the OSCE. Furthermore, this study examined whether individual differences during practice of a fundamental laparoscopic skill were predictive of laparoscopic performance during the OSCE.

METHODS

Overall, 77 final-year undergraduate medical students completed the movement specific reinvestment scale, an assessment tool that quantifies the propensity for MS-C and CMP. Participants were trained to proficiency on a fundamental laparoscopic skill. The number of trials to reach proficiency was measured, and completion times were recorded during early practice, later practice, and the OSCE.

RESULTS

There was a trend for CMP to be negatively associated with the number of trials to reach proficiency (p = 0.064). A higher propensity for CMP was associated with fewer trials to reach proficiency (β = -0.70, p = 0.023). CMP and MS-C did not significantly predict completion times in the OSCE (p > 0.05). Completion times in early practice (β = 0.05, p = 0.016) and later practice (β = 0.47, p < 0.001) and number of trials to reach proficiency (β = 0.23, p = 0.003) significantly predicted completion times in the OSCE.

CONCLUSION

It appears that a higher propensity for CMP predicts faster rates of learning of a fundamental laparoscopic skill. Furthermore, laparoscopic performance during practice is indicative of laparoscopic performance in the challenging conditions of the OSCE. The lack of association between the 2 dimensions of movement specific reinvestment and performance during the OSCE is explained using the theory of reinvestment as a framework. Overall, consideration of personality differences and individual differences in ability during practice could help inform the development of individualized surgical training programs.