Situational simulation teaching effectively improves dental students' non-operational clinical competency and objective structured clinical examination performance

BACKGROUND

Appropriate communication with dental patients enhances treatment outcomes and patient satisfaction. Implementing simulated patient interviews courses can improve patient-centered care and reduce conflict during clerkship training. Therefore, this study explored the relationship among student participation in a situational simulation course (SSC), academic performance, clerkship performance, and objective structured clinical examination (OSCE) performance.

METHODS

This study was conducted with a sample of fifth-year dental students undergoing clerkship training. After implementing a situational simulation course to investigate the relationship among participation in SSC, academic performance, clerkship performance, and OSCE performance, a path analysis model was developed and tested.

RESULTS

Eighty-seven fifth-year dental students were eligible for the SSC, and most (n = 70, 80.46%) volunteered to participate. The path analysis model revealed that academic performance had a direct effect on OSCE performance (β = 0.281, P = 0.003) and clerkship performance (β = 0.441, P < 0.001). In addition, SSC teaching had a direct effect on OSCE performance (β = 0.356, P < 0.001).

CONCLUSIONS

SSCs can enhance dental students' non-operational clinical competency and OSCE performance effectively. Simulated patient encounters with feedback, incorporated into the dental curricula, have led to improved communication. Based on our findings, we suggest implementing SSC teaching before the OSCE to improve communication and cognitive skills.

Review of Objective Structured Clinical Examination Practices Within Pharmacy Programs Throughout the United States

OBJECTIVE

To update the description of current objective structured clinical examination (OSCE) practices within pharmacy schools in the United States and identify barriers to OSCE implementation and expansion.

METHODS

A survey was deployed to all accredited Doctor of Pharmacy programs within the United States. The survey was designed to collect information regarding the curricular mapping of OSCEs, OSCE design, OSCE delivery, assessment of OSCE performance, and barriers to OSCE implementation and expansion.

RESULTS

Of the 135 US-accredited programs identified, 109 (81%) programs completed the survey. In total, 93 (85%) programs reported using OSCEs to assess students; however, implementation throughout the curriculum and current practices varied by institution. Most programs place OSCEs within specific courses (96%), with the applied skills coursework being the most used (80%). The most common number of OSCEs that occur throughout a curriculum is 6 (18%), however, 20 (22%) programs execute 10 or more OSCEs throughout their curriculum. Forty (43%) programs use OSCEs as high-stakes assessments where poor performance could prevent a student from progressing to advanced pharmacy practice experiences. Of the responding programs, over half (56%) use teaching objective structured examinations to enhance learning. Common barriers to OSCE expansion exist and are related to resource utilization.

CONCLUSION

Significant expansion of OSCE development and implementation has occurred over the last decade. There is substantial variability in implementation and utilization among programs. Although standards of best practice for OSCEs exist for other health professions, best practices in pharmacy education have not been widely accepted or adopted.

Developing an innovative national ACP-OSCE program in Taiwan: a mixed method study

OBJECTIVES

To evaluate the process and the comprehensiveness of advance care planning (ACP), we designed a national ACP-OSCE (Objective Structured Clinical Examination) program.

METHODS

The program was designed as a 40-minute OSCE test. Participants were categorized as different ACP team members to illustrate realistic scenarios. Preceptors were asked to observe ACP professionals' actions, responses, and communication skills during ACP with standardized patients (SP) through a one-way mirror. Participants' communication skills, medical expertise, legal knowledge, empathetic response and problem-solving skills of ACP were also self-evaluated before and after OSCE. Thematic analysis was used for qualitative analysis.

RESULTS

In Nov 2019, a total of 18 ACP teams with 38 ACP professionals completed the ACP-OSCE program, including 15 physicians, 15 nurses, 5 social workers, and 3 psychologists. After the ACP-OSCE program, the average score of communication skills, medical expertise, legal knowledge, empathetic response, ACP problem-solving all increased. Nurses felt improved in medical expertise, legal knowledge, and problem-solving skills, psychologists and social workers felt improved in legal knowledge, while physicians felt no improved in all domain, statistically. Thematic analysis showed professional skills, doctoral-patient communication, benefit and difficulties of ACP were the topics which participants care about. Meanwhile, most participants agreed that ACP-OSCE program is an appropriate educational tool.

CONCLUSION

This is the first national ACP-OSCE program in Asia. We believe that this ACP-OSCE program could be applied in other countries to improve the ACP process and quality.

Objective structured clinical examination to teach competency in planetary health care and management - a prospective observational study

BACKGROUND

Health professionals are increasingly called upon and willing to engage in planetary health care and management. However, so far, this topic is rarely covered in medical curricula. As the need for professional communication is particularly high in this subject area, this study aimed to evaluate whether the objective structured clinical examination (OSCE) could be used as an accompanying teaching tool.

METHODS

During the winter semester 2022/2023, 20 third- and fifth-year medical students voluntarily participated in a self-directed online course, three workshops, and a formal eight-station OSCE on planetary health care and management. Each examinee was also charged alternatingly as a shadower with the role of providing feedback. Experienced examiners rated students' performance using a scoring system supported by tablet computers. Examiners and shadowers provided timely feedback on candidates` performance in the OSCE. Immediately after the OSCE, students were asked about their experience using a nine-point Likert-scale survey and a videotaped group interview. Quantitative analysis included the presentation of the proportional distribution of student responses to the survey and of box plots showing percentages of maximum scores for the OSCE performance. The student group interview was analyzed qualitatively.

RESULTS

Depending on the sub-theme, 60% -100% of students rated the subject of planetary health as likely to be useful in their professional lives. Similar proportions (57%-100%) were in favour of integrating planetary health into required courses. Students perceived learning success from OSCE experience and feedback as higher compared to that from online courses and workshops. Even shadowers learned from observation and feedback discussions. Examiners assessed students' OSCE performance at a median of 80% (interquartile range: 83%-77%) of the maximum score.

CONCLUSIONS

OSCE can be used as an accompanying teaching tool for advanced students on the topic of planetary health care and management. It supports learning outcomes, particularly in terms of communication skills to sensitise and empower dialogue partners, and to initiate adaptation steps at the level of individual patients and local communities.

The medical licensing assessment will fall short of determining whether a UK medical graduate behaves ethically

UK medical graduates will soon need to pass the medical licensing assessment, which assesses skills and knowledge in ethics using multiple choice questions (eg single best answer questions) and objective structured clinical examination. However, educational leaders have recognised that these methods lack the sophistication needed to accurately assess medical ethics. The reasons are two-fold. First, there may be a knowledge and practice gap in medical schools when it comes to preparing students for the assessment. To this end, this article shares peer advice about how best to use objective structured clinical examinations and single best answer questions for assessing medical ethics to help prepare students for the medical licensing assessment. Second, the design of the assessment is unlikely to adequately measure graduates' ethical values and behaviour in real world scenarios. Further work is needed to design assessments that are sophisticated enough to examine candidates' ethical reasoning and their actual behaviour.

Progress testing of an objective structured clinical examination during undergraduate clinical clerkship: a mixed-methods pilot study

BACKGROUND

Progress testing is an assessment method in which an examination reflecting competencies at graduation is regularly administered to students over multiple years, thereby facilitating self-directed learning. However, the significance of the objective structured clinical examination as a progress test in undergraduate education, needs to be determined. This study provides evidence of the role of the objective structured clinical examination for progress testing and optimal scoring methods for assessing students in different academic years.

METHODS

We conducted a sequential explanatory mixed-methods pilot study. Participants were assessed using the Item Rating Scale, the year-adjusted Global Rating Scale, and the Training Level Rating Scale. The characteristics of each scale were compared quantitatively. In addition, the influence of the objective structured clinical examination as a progress test on learning attitudes was examined. Qualitative data from a post-examination questionnaire were analyzed, using content analysis to explore influences on self-directed learning.

RESULTS

Sixth and fifth year clinical students (n = 235) took the objective structured clinical examination progress test. The total Item Rating Scales were recorded (%) as 59.03 ± 5.27 and 52.64 ± 5.08 (p < 0.01); Training Level Rating Scale was 3.94 ± 0.39 vs 3.22 ± 0.42 (p < 0.01); and the year-adjusted Global Rating Scale was 4.25 ± 0.44 vs 4.32 ± 0.52 (no significant difference), for the sixth and fifth year students, respectively. The correlations across stations and the reliability of each station were satisfactory. Four categories were identified in the qualitative analysis: "motivation to learn during the clinical clerkship was promoted," "dissatisfied with being asked about things they had not experienced," "confusion about being unable to use conventional test-taking strategies," and "insufficient understanding of competencies at graduation." The scores indicated significant differences in performance according to training year.

CONCLUSIONS

This study provides evidence that the objective structured clinical examination can be used as a progress testing tool for undergraduate clinical clerkships. Further enhancement of training opportunities and dissemination of performance competency goals in clerkship curricula are required if we intend to promote self-directed learning through progress testing.

Cognitive load theory in action: e-learning modules improve performance in simulation-based education. A pilot study

PURPOSE

Learners participating in simulation-based education may experience cognitive overload with potential detrimental effects to learning and performance. Multiple strategies have been proposed to mitigate this detrimental response. However, these strategies have not fully considered the potential benefits of using online platforms, such as accessibility, cost-effectiveness, efficiency, and scalability. Addressing this gap in the literature, preparatory online modules were developed by applying concepts from cognitive load theory and simulation-based education. This study assessed whether using preparatory online modules to deliver weekly pre-briefing content could impact cognitive load and performance. The participants were first-year postgraduate medical trainees participating in a simulation-based resuscitation curriculum.

METHODS

Fifty-three trainees were allocated to receive preparatory online modules (online modules group, n = 27) or not (control group, n = 26) during the course component of a simulation-based resuscitation curriculum. Then, these trainees participated in a simulation-based objective structured clinical examination (OSCE). Sources of cognitive load (intrinsic, extraneous, and germane) were measured using a modified cognitive load questionnaire. Performance was assessed using the Ottawa Surgical Competency Operating Room Evaluation. Data were analyzed with descriptive statistics, and principal component analysis.

RESULTS

During the course component, the online modules group was found to have higher intrinsic and germane cognitive load, and lower extraneous cognitive load compared to the control group. During the OSCE, the online modules group scored significantly higher in performance scores (p = 0.0077, d = 0.39, 95% confidence interval = 0.10;0.68) compared to the control group. Principal component analysis supported the results obtained with the modified cognitive load questionnaire.

CONCLUSION

Trainees using preparatory online modules during the course component of a simulation-based resuscitation curriculum experienced cognitive load changes consistent with cognitive optimization. This may have contributed to their superior performance in the subsequent OSCE. Future research should explore the long-term impacts of online preparatory training and consider potential barriers to implementation in diverse healthcare environments.

The development and validation of a standardised eight-station OSCE for registration of undergraduate nursing students: A Delphi study

AIM

To develop and validate a set of OSCEs for use as a nationally applicable final exam of undergraduate nursing students' clinical and professional competencies.

BACKGROUND

Tertiary recommendations for nursing registration require student achievement of a prerequisite level of both nursing knowledge and skill. Assessment of nursing skills prior to provision of nursing registration is an increasingly important focus of university nursing education. Objective Structured Clinical Examinations (OSCEs) are an appropriate tool for assessing the competencies of nursing students, but the mode and content of OSCEs varies widely. Thus, for nationally-based registration, development of a standardized set of OSCEs to determine the clinical competence of nursing students is important.

DESIGN

A modified Delphi study, informed by a systematic scoping review of relevant literature.

METHODS

A two-phase study was conducted. Initially, a scoping review was conducted to establish internationally agreed competences for graduating nursing students. These were included in an online questionnaire send to national experts to develop a validated and accepted nursing competency framework and associated implementation strategies. This was round 1 of a three-round modified Delphi study. A Content Validity Index (CVI) was calculated and thematic analysis was used to analyze the experts' opinions.

RESULTS

Details of a set of OSCEs for final exam of undergraduate nursing students were developed and validated. These OSCEs included nine competencies addressed in eight 10-minute stations.

CONCLUSION

A 'best-practice' OSCE evaluation system, designed from both international literature and the opinions of nation-wide experts was achieved. This well-designed OSCE was shown to be a reliable and valid method for assessing the clinical competence of nursing students. It reflected global procedures as well as the local conditions and Iranian expectations. The results of this study can be used by nursing schools across the country. They can also be used to model equivalent development of locally relevant OSCEs in countries around the world.

Assessing clinical reasoning in the OSCE: pilot-testing a novel oral debrief exercise

INTRODUCTION

Clinical reasoning (CR) is a complex skill enabling transition from clinical novice to expert decision maker. The Objective Structured Clinical Examination (OSCE) is widely used to evaluate clinical competency, though there is limited literature exploring how this assessment is best used to assess CR skills. This proof-of-concept study explored the creation and pilot testing of a post-station CR assessment, named Oral Debrief (OD), in the context of undergraduate medical education.

METHODS

A modified-Delphi technique was used to create a standardised domain-based OD marking rubric encapsulating the key skills of CR that drew upon existing literature and our existing placement-based CR tool. 16 OSCE examiners were recruited to score three simulated OD recordings that were scripted to portray differing levels of competency. Adopting a think-aloud approach, examiners vocalised their thought processes while utilising the rubric to assess each video. Thereafter, semi-structured interviews explored examiners' views on the OD approach. Recordings were transcribed, anonymised and analysed deductively and inductively for recurring themes. Additionally, inter-rater agreement of examiners' scoring was determined using the Fleiss Kappa statistic both within group and in comparison to a reference examiner group.

RESULTS

The rubric achieved fair to good levels of inter-rater reliability metrics across its constituent domains and overall global judgement scales. Think-aloud scoring revealed that participating examiners considered several factors when scoring students' CR abilities. This included the adoption of a confident structured approach, discriminating between relevant and less-relevant information, and the ability to prioritise and justify decision making. Furthermore, students' CR skills were judged in light of potential risks to patient safety and examiners' own illness scripts. Feedback from examiners indicated that whilst additional training in rubric usage would be beneficial, OD offered a positive approach for examining CR ability.

CONCLUSION

This pilot study has demonstrated promising results for the use of a novel post-station OD task to evaluate medical students' CR ability in the OSCE setting. Further work is now planned to evaluate how the OD approach can most effectively be implemented into routine assessment practice.

Evolution of a capstone exam for third-year doctor of pharmacy students

BACKGROUND AND PURPOSE

Pharmacy capstone exams are an example of an assessment strategy to evaluate student competency and practice-readiness. Capstone exam processes have been discussed in the literature; however, description of format, evolution of the process, and implementation in the year prior to rotations is lacking. This manuscript discusses the development, implementation, and utilization of a third professional year pharmacy capstone exam to assess advanced pharmacy practice experience (APPE) readiness.

EDUCATIONAL ACTIVITY AND SETTING

A pharmacy year three (PY3) capstone exam was initially developed prior to enrollment of the inaugural graduating class. Since the first iteration, utility has remained the same, but the overall development and implementation process has evolved due to changes in programmatic and student learning outcomes, curricular mapping, student and faculty feedback, and practice expectations.

FINDINGS

The entities responsible for exam implementation perceive faculty and students have a positive perception of the exam evolution and improvements implemented. Specifically, feedback from students and faculty mentioned clear expectations, comfort with the new capstone exam process, and an appreciation for the focus on interrater reliability. As assessment is an iterative process, quality improvement strategies continue to be implemented to address feedback regarding the PY3 capstone exam.

SUMMARY

Purposeful utilization of capstone exams in the third year is one method to assess APPE readiness. Sharing one college's experience with the evolution of such an exam, quality improvement methods, and experience in implementing reliability and validity measures may provide a model for institutions to implement similar assessment methods efficiently.

Development, Validation, and Reliability of a P1 Objective Structured Clinical Examination Assessing the National EPAs

OBJECTIVE

To document the performance of first-year pharmacy students on a revised objective structured clinical examination (OSCE) based on national entrustable professional activities, identify risk factors for poor performance, and assess its validity and reliability.

METHODS

A working group developed the OSCE to verify students' progress toward readiness for advanced pharmacy practice experiences at the L1 level of entrustment (ready for thoughtful observation) on the national entrustable professional activities, with stations cross-mapped to the Accreditation Council for Pharmacy Education educational outcomes. Baseline characteristics and academic performance were used to investigate risk factors for poor performance and validity, respectively, by comparing students who were successful on the first attempt with those who were not. Reliability was evaluated using re-grading by a blinded, independent grader, and analyzed using Cohen's kappa.

RESULTS

A total of 65 students completed the OSCE. Of these, 33 (50.8%) successfully completed all stations on first attempt, and 32 (49.2%) had to re-attempt at least 1 station. Successful students had higher Health Sciences Reasoning Test scores (mean difference 5, 95% CI 2-9). First professional year grade point average was higher for students who passed all stations on first attempt (mean difference 0.4 on a 4-point scale, 95% CI 0.1-0.7). When evaluated in a multiple logistic regression, no differences were statistically significant between groups. Most kappa values were above 0.4 (range 0.404-0.708), suggesting moderate to substantial reliability.

CONCLUSION

Though predictors of poor performance were not identified when accounting for covariates, the OSCE was found to have good validity and reliability.

The influence of patient gender on medical students' care: Evaluation during an objective structured clinical examination

OBJECTIVES

To assess whether men and women are evaluated and treated differently by medical students.

METHODS

We evaluated patient care provided by 110 fifth-year medical students during an objective structured clinical examination (OSCE), using two clinical cases with standardized patients (SPs): generalized anxiety disorder (GAD) and ascending aortic dissection (AAD). Half of the students encountered male and half female SPs. Except for gender, the cases were identical. We compared diagnosis and treatment of male vs female SPs.

RESULTS

Students diagnosed GAD more often in female SPs than in male SPs (diagnosis completed, partially completed, and not completed in 47%, 16% and 36% respectively vs. 22%, 20%, and 58% for male SPs, p = 0.02). The nature of symptoms was better described for male SPs. For AAD, the emergency was more frequently identified and the examination of femoral pulses better performed in female SPs.

CONCLUSION

Medical students have a gender bias when evaluating patients with GAD and AAD.

PRACTICE IMPLICATION

The observed gender bias in the evaluation of patients, likely leads to differences in treatment between male and female patients (i.e. under-recognition of anxiety in men). Medical schools should implement gender-sensitive medical education initiatives to improve inclusive patient care.

Objective structured clinical examination as a competency assessment tool of students' readiness for advanced pharmacy practice experiences in South Korea: a pilot study

BACKGROUND

The assessment of pharmacy students' readiness to begin the education of an advanced pharmacy practice experience (APPE) in clinical pharmacy settings continues to gain increasing attention. This study aimed to develop an objective structured clinical examination (OSCE) in the core domains acquired through an introductory pharmacy practice experience (IPPE), for evaluating its appropriateness as a tool of assessing clinical pharmacist competency for APPEs in Korean pharmacy students throughout a pilot study.

METHODS

OSCE's core competency domains and case scenarios were developed through a literature review, ideation by researchers, and external experts' consensus by a Delphi method. A prospective single-arm pilot test was conducted to implement the OSCE for Korean pharmacy students who completed a 60-h course of in-class simulation IPPE. Their competencies were assessed by four assessors in each OSCE station with a pass-fail grading system accompanied by a scoring rubric.

RESULTS

OSCE competency areas including patient counseling, provision of drug information, over-the-counter (OTC) counseling, and pharmaceutical care services were developed with four interactive and one non-interactive cases. Twenty pharmacy students participated in the OSCE pilot test, and their competencies were evaluated by 20 assessors. The performance rate was the lowest in the area of patient counseling for a respiratory inhaler (32.1%) and the highest (79.7%) in OTC counseling for constipation. The students had an average performance rate of 60.4% in their communication skills. Most participants agreed on the appropriateness, necessity, and effectiveness of the OSCE in evaluating pharmacy students' clinical performance and communication skills.

CONCLUSIONS

The OSCE model can be used to assess pharmacy students' readiness for off-campus clinical pharmacy practice experience. Our pilot study suggests the necessity of conducting an OSCE domain-based adjustment of difficulty levels, and strengthening simulation-based IPPE education.

Scoring consistency of standard patients and examiners in the developed dental objective structured clinical examination system

OBJECTIVE

To investigate the role of standard patients (SPs) and examiners as assessors for scoring in the dental objective structured clinical examination (OSCE) system and to evaluate the scoring differences between them.

METHODS

We developed the doctor-patient communication and clinical examination station in the OSCE system. The examination time of this station was 10 min, and the examination institution wrote the script and recruited SPs. A total of 146 examinees who received standardized resident training at the Nanjing Stomatological Hospital, Medical School of Nanjing University between 2018 and 2021 were assessed. They were scored by SPs and examiners according to the same scoring rubrics. Subsequently, the SPSS software was used to analyze the examination results of different assessors and evaluate the consistency.

RESULTS

The average score of all examinees provided by SPs and examiners was 90.45 ± 3.52 and 91.53 ± 4.13, respectively. The consistency analysis showed that the intraclass correlation coefficient was 0.718, which was indicative of medium consistency.

CONCLUSION

Our findings showed that SPs could be used directly as assessors, as they could provide a simulated and realistic clinical setting and create favorable conditions for comprehensive competence training and improvement for medical students.

Development and preliminary testing of a virtual reality measurement for assessing intake assessment skills

Objective structured clinical examinations (OSCEs) have been widely used in health care education to simultaneously assess knowledge, skill and attitude. Due to the high cost of running an OSCE, its application in professional psychology is still limited. To solve this problem, virtual standardised patient (VSP) implementations in creating psychology OSCEs can be a cost-effective method for administering a psychology OSCE regularly. This study aimed to develop and examine the psychometric properties of the VSP version of the Intake OSCE (VSP-Intake OSCE) in measuring psychologists' psychological assessment competencies (PACs) from entry to early practice. The initial development of the VSP-Intake OSCE contains a VSP station and a follow-up written station to measure PACs when conducting an intake assessment. To administer the VSP station, we built a new VSP system that allows psychologists to interact with a VSP verbally. A sample of 36 participants, including 27 graduate students and nine psychologists, were recruited to examine the psychometric properties of the VSP-Intake OSCE. As a newly developed instrument, the VSP-Intake OSCE revealed good inter-rater reliability and construct validity. We believe using VSP implementations to develop psychology OSCEs will be essential in promoting OSCE applications in professional psychology.

Feasibility of large-scale eOSCES: the simultaneous evaluation of 500 medical students during a mock examination

The COVID-19 pandemic has led health schools to cancel many on-site training and exams. Teachers were looking for the best option to carry out online OSCEs, and Zoom was the obvious choice since many schools have used it to pursue education purposes.

METHODS

We conducted a feasibility study during the 2020-2021 college year divided into six pilot phases and the large-scale eOSCEs on Zoom on June 30th, 2021. We developed a specific application allowing us to mass create Zoom meetings and built an entire organization, including a technical support system (an SOS room and catching-up rooms) and teachers' training sessions. We assessed satisfaction via an online survey.

RESULTS

On June 30th, 531/794 fifth-year medical students (67%) participated in a large-scale mock exam distributed in 135 Zoom meeting rooms with the mobilization of 298 teachers who either participated in the Zoom meetings as standardized patients (N =135, 45%) or examiners (N =135, 45%) or as supervisors in the catching-up rooms (N =16, 6%) or the SOS room (N =12, 4%). In addition, 32/270 teachers (12%) experienced difficulties connecting to their Zoom meetings and sought the help of an SOS room member. Furthermore, 40/531 students (7%) were either late to their station or had technical difficulties and declared those issues online and were welcomed in one of the catching-up rooms to perform their eOSCE stations. Additionally, 518/531 students (98%) completed the entire circuit of three stations, and 225/531 students (42%) answered the online survey. Among them, 194/225 (86%) found eOSCES helpful for training and expressed their satisfaction with this experience.

CONCLUSION

Organizing large-scale eOSCEs on Zoom is feasible with the appropriate tools. In addition, eOCSEs should be considered complementary to on-site OSCEs and to train medical students in telemedicine.

Supporting pharmacy students' preparation for an entry-to-practice OSCE using video cases

BACKGROUND AND PURPOSE

Objective structured clinical examinations (OSCEs) are widely used, including in licensure examinations. OSCEs assess diverse skills in a structured manner, but can be stressful for students and have a significant organisational burden for faculty. Case-based video OSCE preparation resources were developed for students preparing for a licensure OSCE. The study aimed to examine student engagement, compare scores awarded by students to performances at specific competence standards with faculty scores, and examine usability, usefulness, and acceptability of the video cases.

EDUCATIONAL ACTIVITY AND SETTING

Final year pharmacy students (n = 149) enrolled in an integrated master of pharmacy programme in Ireland were invited to participate. Six sets of recorded OSCE-based video cases were developed, comprising multiple recordings of the same case scenario, with each pitched at a different level of performance. Students watched and scored the video cases. Usability, usefulness, and acceptability were evaluated via questionnaire.

FINDINGS

One or more video cases were accessed by 70.5% of students. Score ratings awarded by the students, when compared to faculty ratings, showed an overall trend towards inter-rater agreement between students and faculty. Students felt positively in terms of the usability, usefulness, and acceptability of the videos.

SUMMARY

Video cases designed to support OSCE preparation for a high-stakes national pharmacy licensure examination were widely used by students and were perceived to be usable, useful, and acceptable. Such video cases may be a feasible alternative to additional mock OSCEs to support student preparation.

Is online objective structured clinical examination teaching an acceptable replacement in post-COVID-19 medical education in the United Kingdom?: a descriptive study

PURPOSE

Coronavirus disease 2019 (COVID-19) restrictions resulted in an increased emphasis on virtual communication in medical education. This study assessed the acceptability of virtual teaching in an online objective structured clinical examination (OSCE) series and its role in future education.

METHODS

Six surgical OSCE stations were designed, covering common surgical topics, with specific tasks testing data interpretation, clinical knowledge, and communication skills. These were delivered via Zoom to students who participated in student/patient/examiner role-play. Feedback was collected by asking students to compare online teaching with previous experiences of in-person teaching. Descriptive statistics were used for Likert response data, and thematic analysis for free-text items.

RESULTS

Sixty-two students provided feedback, with 81% of respondents finding online instructions preferable to paper equivalents. Furthermore, 65% and 68% found online teaching more efficient and accessible, respectively, than in-person teaching. Only 34% found communication with each other easier online; Forty percent preferred online OSCE teaching to in-person teaching. Students also expressed feedback in positive and negative free-text comments.

CONCLUSION

The data suggested that generally students were unwilling for online teaching to completely replace in-person teaching. The success of online teaching was dependent on the clinical skill being addressed; some were less amenable to a virtual setting. However, online OSCE teaching could play a role alongside in-person teaching.

Effectiveness of interprofessional shared decision-making training: A mixed-method study

OBJECTIVE

This study evaluated the learning effects and examined the participants' perceptions of an interprofessional shared decision-making (IP-SDM) training program.

METHODS

This mixed-method study used a quasi-experimental pretest-posttest design in the quantitative phase and semi-structured interviews in the qualitative phase. The 6-week curriculum design, based on Kolb's experiential learning cycle, consisted of two simulated objective structured clinical examinations with standardized patients and blended teaching methods through various course modules.

RESULTS

A total of 39 multidisciplinary healthcare personnel completed the 6-week training program, and 32 of them participated in qualitative interviews. The IP-SDM training program effectively improved the SDM process competency of the participants from the perspectives of the participants, standardized patients, and clinical teachers. The interviews illustrated how the curriculum design enhanced learning; the effectiveness results indicated improvements in learners' attitude, knowledge, skills, and teamwork.

CONCLUSION

This IP-SDM training program improved multidisciplinary healthcare personnel's competency, self-efficacy, and intention to engage in IP-SDM.

PRACTICE IMPLICATIONS

Applying Kolb's experiential learning cycle and blended teaching methods to develop and implement the IP-SDM training program can improve multidisciplinary healthcare personnel's knowledge, attitude, skills, and teamwork in IP-SDM.

Development of a telehealth obesity OSCE and reliable checklist for assessment of resident physicians: a pilot study

BACKGROUND

Obesity is a major public health problem, yet residents undergo little formal training and assessment in obesity-related care. Given the recent growth of telehealth, physicians must further learn to apply these skills using a virtual platform. Therefore, we aimed to develop an objective structured clinical examination (OSCE) with reliable checklists to assess resident ability to take a patient-centered obesity-focused history that was feasible over telehealth based on published obesity competencies for medical education.

METHODS

We developed a 15-minute telehealth OSCE to simulate an obesity-related encounter for residents modified from a script used to assess medical student obesity competencies. We designed three checklists to assess resident skills in history taking, communication and professionalism during the obesity-related encounter. Resident performance was assessed as the percentage of obesity-related history taking questions asked during the encounter and as the mean communication and professionalism scores on a scale of 1 through 5 with 1 representing unacceptable/offensive behavior and 5 representing excellent skills. Encounters and assessments were completed by two commissioned actors (standardized patients) and 26 internal medicine residents over a secure online platform. We assessed the reliability of each checklist by calculating the percent agreement between standardized patients and the kappa (κ) statistic on each checklist overall and by each checklist item.

RESULTS

Overall agreement between standardized patients on the history taking, communication and professionalism checklists were 83.2% (κ = 0.63), 99.5% (κ = 0.72) and 97.8% (κ =0.44), respectively. On average, residents asked 64.8% of questions on the history taking checklist and scored 3.8 and 3.9 out of 5 on the communication and professionalism checklists, respectively.

CONCLUSIONS

Results from this pilot study suggest that our telehealth obesity OSCE and checklists are moderately reliable for assessing key obesity competencies among residents on a virtual platform. Integrating obesity OSCEs and other educational interventions into residency curricula are needed to improve resident ability to take an obesity-focused history.

Novel use of an OSCE to assess medical students' responses to a request for a low value diagnostic imaging test: A mixed methods analysis

OBJECTIVE

Evaluate medical students' communication skills with a standardized patient (SP) requesting a low value test and describe challenges students identify in addressing the request.

METHODS

In this mixed-methods study, third-year students from two medical schools obtained a history, performed a physical examination, and counseled an SP presenting with uncomplicated low back pain who requests an MRI which is not indicated. SP raters evaluated student communication skills using a 14-item checklist. Post-encounter, students reported whether they ordered an MRI and challenges faced.

RESULTS

Students who discussed practice guidelines and risks of unnecessary testing with the SP were less likely to order an MRI. Students cited several challenges in responding to the SP request including patient characteristics and circumstances, lack of knowledge about MRI indications and alternatives, and lack of communication skills to address the patient request.

CONCLUSIONS

Most students did not order an MRI for uncomplicated LBP, but only a small number of students educated the patient about the evidence to avoid unnecessary imaging or the harm of unnecessary testing.

PRACTICE IMPLICATIONS

Knowledge about unnecessary imaging in uncomplicated LBP may be insufficient to adhere to best practices and longitudinal training in challenging conversations is needed.

Validation of a Surgical Objective Structured Clinical Examination (S-OSCE) Using Convergent, Divergent, And Trainee-Based Assessments of Fidelity

OBJECTIVE

Describe the validation of a surgical objective structured clinical examination (S-OSCE) for the purpose of competency assessment based on the Royal College of Canada's Can-MEDS framework.

DESIGN

A surgical OSCE was developed to evaluate the management of common orthopedic surgical problems. The scores derived from this S-OSCE were compared to Ottawa Surgical Competency Operating Room Evaluation (O-SCORE), a validated entrustability assessment, to establish convergent validity. The S-OSCE scores were compared to Orthopedic In-Training Examination (OITE) scores to evaluate divergent validity. Resident evaluations of the clinical encounter with a standardized patient and the operative procedure were scored on a 10-point Likert scale for fidelity.

SETTING

A tertiary level academic teaching hospital.

PARTICIPANTS

21 postgraduate year 2 to 5 trainees of a 5-year Canadian orthopedic residency program creating 160 operative case performances for review.

RESULTS

There were 5 S-OSCE days, over a 4-year period (2016-2019) encompassing a variety of surgical procedures. Performance on the S-OSCE correlated strongly with the O-SCORE (Pearson correlation coefficient 0.88), and a linear regression analysis correlated moderately with year of training (R² = 0.5345). The Pearson correlation coefficient between the S-OSCE and OITE scores was 0.57. There was a significant increase in the average OITE score after the introduction of the surgical OSCE. Resident fidelity ratings were available from 16 residents encompassing 8 different surgical cases. The average score for the overall simulation (8.0±1.6) was significantly higher than the cadaveric surgical simulation (6.5 ± 0.8) (p < 0.001) CONCLUSIONS: The S-OSCE scores correlate strongly with an established form of assessment demonstrating convergent validity. The correlation between the S-OSCE and OITE scores was less, demonstrating divergent validity. Although residents rank the overall simulation highly, the fidelity of the cadaveric simulation may need improvement. Administration of a surgical OSCE can be used to evaluate preoperative and intraoperative decision making and complement other forms of assessment.

Midwifery students' experiences of objective structured clinical examinations: A qualitative evidence synthesis

OBJECTIVES

To identify and synthesise the evidence of midwifery students' views and experiences of Objective Structured Clinical Examinations (OSCEs).

DESIGN

A qualitative evidence synthesis was undertaken using Thomas and Harden's synthesis methods and reported according to the ENTEREQ guidelines.

DATA SOURCES

PubMED, CINAHL Plus, Embase, ERIC, PsycINFO, Social Science Citation Index (via Web of Science) and Scopus were searched from inception to 19th February 2020 and an updated search was performed on 12th May 2021 to identify qualitative and mixed method studies, reporting qualitative data of undergraduate midwifery students' views and experiences of OSCEs.

REVIEW METHODS

Studies were independently screened by title and abstract according to a predetermined inclusion and exclusion criteria. Full text review was then conducted independently by two authors. Studies providing qualitative data on undergraduate midwifery students' views and experiences of OSCEs during their midwifery registration education programmes were included. Quality appraisal was performed using the Critical Appraisal Skills Programme (CASP) tool. Data synthesis was performed using Thomas and Harden's thematic synthesis method.

RESULTS

Eleven studies were identified and included. Thematic synthesis identified 13 descriptive themes which translated into four analytical themes that represented midwifery students' views and experiences of OSCEs: the preparation phase, the assessment phase, the feedback phase and the translation phase.

CONCLUSIONS

This synthesis offers insight into undergraduate midwifery students' views and experiences of OSCEs and how they are experienced across four phases. This is determined to be a useful structure to guide midwifery educators and researchers in their consideration of the student experience of OSCEs in the future. Midwifery students in this synthesis reported high levels of nervousness, anxiety, and stress during the OSCE process. The findings reveal that there are a number of controllable factors impacting on midwifery students' feelings of safety in the assessment environment, and that these should be addressed to maximise the advantages that the OSCE has to offer in terms of learning and preparation for clinical practice.

Comparison of student performance in therapeutics and communications courses to outcomes of objective structured clinical examinations: A retrospective analysis

INTRODUCTION

The objective was to examine the association between course grades and objective structured clinical examination (OSCE) performances in a pharmacy student cohort.

METHODS

A retrospective analysis of demographics, final grades in the Therapeutics I through IV and Communications courses, and multiple OSCE scores (therapeutic knowledge and general communication skills [GCS]) was performed for two cohorts.

RESULTS

Female students scored higher than male students on the warfarin OSCE therapeutic knowledge (P = .008) and GCS scores (P = .02). Age was inversely correlated with warfarin OSCE therapeutic knowledge score (P = .001). The warfarin OSCE therapeutic knowledge score was positively related to Therapeutics II final grades (P < .001). The Communications course final grade was positively correlated to the warfarin OSCE faculty-rated GCS score (P = .005). Therapeutics final grades were not significantly related to station scores of a multi-station OSCE (P > .05). The final regression model included gender and Therapeutics II final grade and explained 6% of the variance in warfarin OSCE therapeutic knowledge scores.

CONCLUSIONS

A number of significant associations were found between demographics, final course grades, and specific OSCE scores. A regression model was significant, but only explained a low percentage of the warfarin OSCE therapeutic knowledge score variance, suggesting other factors not evaluated had a greater effect on scores. This research suggests that OSCEs play an important role in demonstrating student competency in educational domains other than knowledge and brings forth new data suggesting that age and/or gender may influence OSCE performance.

Use of an objective structured clinical examination (OSCE) to assess intern performance in an advanced pharmacy practice experiences (APPE) Ambulatory Care rotation

Purpose

Intern assessment during advanced pharmacy practice experiences (APPEs) are generally based upon each individual preceptor's perceptions without an objective measurement of intern understanding and performance. Therefore, we sought to determine whether a pre- and post-OSCE could be used to confirm that interns achieved the goals and objectives of the Ambulatory Care rotation. The aims of this study were to determine whether a pre-rotation OSCE can help pharmacy interns self-assess their clinical strengths and weaknesses and assess whether their knowledge and skills improved after completing a post-rotation OSCE.

Methods

Pharmacy interns undergoing APPE Ambulatory Care rotations from September 2018 to March 2020 participated in a pre- and post-rotation OSCE to assess their knowledge of various chronic disease states. Interns completed pre- and post-OSCE surveys to assess their perceptions about their knowledge and the OSCE experience.

Results

Pharmacy intern knowledge about diabetes, hypertension, dyslipidemia, and atrial fibrillation significantly improved post-OSCE compared to their pre-OSCE scores (p < 0.001). The mean post-OSCE scores for diabetes (p < 0.001), dyslipidemia (P = 0.046), anticoagulation (P = 0.006), and the overall mean post-OSCE scores (P = 0.005) were significantly higher compared to interns' pre-OSCE scores. Students believed that the post-OSCE significantly highlighted their strengths and weaknesses in skills and knowledge compared to the pre-OSCE (P = 0.008).

Conclusion

Pre- and post-APPE OSCE assessments are important tools that can provide interns and preceptors with objective evaluations of student performance. OSCEs can either be used as an alternative to perception-based assessments or integrated into existing preceptor evaluations. Furthermore, OSCEs can help preceptors identify areas that require more emphasis in their rotations.

Benefits of semiology taught using near-peer tutoring are sustainable

BACKGROUND

Near-peer tutoring appears to be an efficient approach for teaching clinical skills. However, the clinical experience gained in the form of student medical internships may offset any interest in such tutoring programme. We then investigated the long-term benefits of this programme.

METHODS

This study was conducted in a medical school that experimented in near-peer tutoring for semiology intended for undergraduate medical students. Objective Structured Clinical Examinations and a written semiology test were used to assess students' clinical skills immediately on its conclusion and repeated one and 2 years after the tutoring was completed.

RESULTS

116 students were evaluated initially (80 tutored and 36 untutored), 38 at 1 year (16 tutored and 22 untutored), 42 at 2 years (21 tutored and 21 untutored). In the global score for Objective Structured Clinical Examinations: at 1 year, the tutored group scored 14.0 ± 1.05 and the untutored group scored 11.3 ± 2.3 (p < 0.001), at 2 years, the tutored group scored 15.1 ± 1.5 and the untutored group scored 12.4 ± 2.2 (p < 0.001). We found a similar but smaller difference for the written semiology test. The difference for Objective Structured Clinical Examinations between tutored and untutored students vanished over time for cross-cutting skills.

CONCLUSIONS

Near-peer tutoring in semiology for undergraduate medical students led to better results that remained with the passing of time. Though internships do allow an improvement in the clinical skills of untutored students, they did not reach the level of tutored students.

Tutor-Student Partnership in Practice OSCE to Enhance Medical Education

BACKGROUND

Training of examiners is essential to ensure the quality of objective structured clinical examination (OSCE). We aimed to study a perceived effectiveness of tutor-student partnership in a practice OSCE module by novice OSCE tutors and medical students.

METHOD

We implemented a practice OSCE at a medical faculty in France with novice tutors and third year medical students as partners. Each tutor (n = 44) served as a partner for the group of 5 students in the conception of the scenario and as an evaluator of the tutored station. Students (n = 303) were involved in the conception of a case and the roles of a physician, evaluator and a simulated patient. Data were obtained through self-assessment questionnaires. Descriptive statistics were used to analyze items of the questionnaires. Free-form answers were coded and analyzed thematically.

RESULTS

A total of 36 tutors (82%) and 185 students (61%) responded to the questionnaires. The intervention was well perceived. Thirty-two percent of the tutors reported some difficulties in the assessment of student performance and were disposed to receive further training. Fifty-five percent of the students considered the participation in the OSCE case development appropriate to their level of knowledge, and 70% perceived it as beneficial allowing them to set their learning goals.

CONCLUSION

This initiative provides a relevant method beneficial to OSCE tutors, medical students, and the faculty. Tutors learn how to assess student performance according to expected achievement levels. It allows students to be engaged as partners in co-creation of learning and teaching.

SUPPLEMENTARY INFORMATION

The online version contains supplementary material available at 10.1007/s40670-021-01421-9.

Comparing the cut score for the borderline group method and borderline regression method with norm-referenced standard setting in an objective structured clinical examination in medical school in Korea

PURPOSE

Setting standards is critical in health professions. However, appropriate standard setting methods do not always apply to the set cut score in performance assessment. The aim of this study was to compare the cut score when the standard setting is changed from the norm-referenced method to the borderline group method (BGM) and borderline regression method (BRM) in an objective structured clinical examination (OSCE) in medical school.

METHODS

This was an explorative study to model of the BGM and BRM. A total of 107 fourth-year medical students attended the OSCE at seven stations with encountering standardized patients (SPs) and one station with performing skills on a manikin on 15 July 2021. Thirty-two physician examiners evaluated the performance by completing a checklist and global rating scales.

RESULTS

The cut score of the norm-referenced method was lower than that of the BGM (p<0.01) and BRM (p<0.02). There was no significant difference in the cut score between the BGM and BRM (p=0.40). The station with the highest standard deviation and the highest proportion of the borderline group showed the largest cut score difference in standard setting methods.

CONCLUSION

Prefixed cut scores by the norm-referenced method without considering station contents or examinee performance can vary due to station difficulty and content, affecting the appropriateness of standard setting decisions. If there is an adequate consensus on the criteria for the borderline group, standard setting with the BRM could be applied as a practical and defensible method to determine the cut score for OSCE.

'Why didn't they see my scars?' Critical thematic analysis of simulated participants' perceived tensions surrounding objective structured clinical examinations

Background

Simulated participants (SPs) play an important role in simulated assessments of clinical encounters between medical students and patients, most notably in objective structured clinical examinations (OSCEs). SP contributions to OSCEs are invaluable, taking the role of a patient or carer. While SPs in some settings/contexts may rate students, their role has been problematized in the literature for their lack of agency within a standardised format of OSCEs that promotes reliability, objectivity and accountability. In this study, we explored SP experiences for tensions that result from simulated assessments and their potential implications for education.

Methods

Semi-structured interviews were conducted with seven SPs who were also tasked with providing a global mark for students. They were purposively selected to include women and men of different ages, occupation, education and experience as an SP. The interviews were analysed using a critical thematic analysis using a phenomenological approach.

Results

SP experiences directly addressed tensions and contradictions around OSCEs. SP participants described their experiences under four themes: industrialising, reducing, performativity and patient safety. OSCEs were compared to an industrial process that promoted efficiency but which bore no resemblance to real-life doctor-patient encounters. They were perceived to have a power and agency that reduced SPs to verbalising scripts to ensure that students were exposed to a standardised simulated experience that also underlined the performative role of SPs as props. These performative and reductionist experiences extended to students, for whom the mark sheet acted as a checklist, promoting standardised responses that lacked genuineness. All of this created a tension for SPs in promoting patient safety by ensuring that those medical students who passed were clinically competent.

Conclusions

OSCEs often form part of high-stakes exams. As such, they are governed by processes of industrialisation, accountability and standardisation. OSCEs possess a power and agency that can have unintended negative consequences. These include ‘conditioning’ students to adopt behaviours that are not suited to real-life clinical encounters and are not person-centred.

Evaluating the effect of interventions for strengthening non-physician anesthetists' education in Ethiopia: a pre- and post-evaluation study

BACKGROUND

Access to safe surgery has been recognized as an indispensable component of universal health coverage. A competent anesthesia workforce is a prerequisite for safe surgical care. In Ethiopia, non-physician anesthetists are the main anesthesia service providers. The Government of Ethiopia implemented a program intervention to improve the quality of non-physician anesthetists' education, which included faculty development, curricula strengthening, student support, educational resources, improved infrastructure and upgraded regulations. This study aimed to assess changes following the implementation of this program.

METHODS

A pre-and post-evaluation design was employed to evaluate improvement in the quality of non-physician anesthetists' education. A 10-station objective structured clinical examination (OSCE) was administered to graduating class anesthetists of 2016 (n = 104) to assess changes in competence from a baseline study performed in 2013 (n = 122). Moreover, a self-administered questionnaire was used to collect data on students' perceptions of the learning environment.

RESULTS

The overall competence score of 2016 graduates was significantly higher than the 2013 class (65.7% vs. 61.5%, mean score difference = 4.2, 95% CI = 1.24-7.22, p < 0.05). Although we found increases in competence scores for 6 out of 10 stations, the improvement was statistically significant for three tasks only (pre-operative assessment, postoperative complication, and anesthesia machine check). Moreover, the competence score in neonatal resuscitation declined significantly from baseline (from 74.4 to 68.9%, mean score difference = - 5.5, 95% CI = -10.5 to - 0.5, p < 0.05). Initial gender-based performance differences disappeared (66.3% vs. 65.3%, mean score difference = - 1.0, 95% CI = - 6.11-3.9, p > 0.05 in favor of females), and female students scored better in some stations. Student perceptions of the learning environment improved significantly for almost all items, with the largest percentage point increase in the availability of instructors from 38.5 to 70.2% (OR = 3.76, 95% CI = 2.15-6.55, p < 0.05).

CONCLUSION

The results suggest that the quality of non-physician anesthetists' education has improved. Stagnation in competence scores of some stations and student perceptions of the simulated learning environment require specific attention.

Challenges of objective structured clinical examination in undergraduate nursing curriculum: Experiences of faculties and students

BACKGROUND

Assessment of clinical competence of nursing students is necessary to ensure safe practice and the realization of professional clinical standards. An Objective Structured Clinical Examination (OSCE) is commonly used internationally in nursing education programs to assess clinical competence, but is a new process in Iranian nursing curricula.

OBJECTIVE

The aim of this study was to explore and describe challenges associated with OSCE implementation based on the experiences of faculty members and nursing students, with the objective of further improving the assessment of clinical competence in nursing education.

DESIGN

This study used a qualitative approach that included thematic analysis of the transcribed interviews.

SETTING

Faculty of Nursing and Midwifery, Tabriz University of Medical Sciences (TBZMED), Tabriz, Iran.

PARTICIPANTS

Purposive sampling was used to select eighteen faculty members and fifteen students from those interested. These participants represent a reasonable cross-section of those who had participated in the OSCE.

METHODS

A pre-developed interview guide was used to inform semi-structured in-depth interviews. These were recorded and analyzed using the 6-phase thematic analysis method as described by Braun and Clarke (2013). Themes were independently verified.

RESULTS

Challenges identified by participants around OSCEs as part of undergraduate nursing education were extracted from the interview data. Two main themes were evident from the data: 1) shortcomings in executive and technical infrastructure, 2) shortcomings in educational infrastructure. These themes reflected both student and faculty experiences. These also aligned with themes that commonly emerge in related literature.

CONCLUSION

The study findings illustrate several key challenges associated with organizing and implementing OSCEs and so provides unique insights into the development of strategies to implementing and promoting OSCEs in nursing education. We recommend that managers and authorities in nursing education focus on these challenges and explore processes to successfully introduce this exam for assessment of nursing student capacity.

Changes in the Objective Structured Clinical Examination (OSCE) of University Schools of Medicine during COVID-19. Experience with a computer-based case simulation OSCE (CCS-OSCE)

Background and objectives

The COVID-19 pandemic has forced universities to move the completion of university studies online. Spain’s National Conference of Medical School Deans coordinates an objective, structured clinical competency assessment called the Objective Structured Clinical Examination (OSCE), which consists of 20 face-to-face test sections for students in their sixth year of study. As a result of the pandemic, a computer-based case simulation OSCE (CCS-OSCE) has been designed. The objective of this article is to describe the creation, administration, and development of the test.

Materials and methods

This work is a descriptive study of the CCS-OSCE from its planning stages in April 2020 to its administration in June 2020.

Results

The CCS-OSCE evaluated the competences of anamnesis, exploration, clinical judgment, ethical aspects, interprofessional relations, prevention, and health promotion. No technical or communication skills were evaluated. The CCS-OSCE consisted of ten test sections, each of which had a 12-min time limit and ranged from six to 21 questions (mean: 1.1 min/question). The CCS-OSCE used the virtual campus platform of each of the 16 participating medical schools, which had a total of 2829 students in their sixth year of study. It was jointly held on two dates in June 2020.

Conclusions

The CCS-OSCE made it possible to bring together the various medical schools and carry out interdisciplinary work. The CCS-OSCE conducted may be similar to Step 3 of the United States Medical Licensing Examination.

Are different medical school admission tests associated with the outcomes of a simulation-based OSCE?

BACKGROUND

Medical school admission procedures have the common goal to select applicants with the greatest potential of becoming successful physicians. Hamburg Medical Faculty selects medical students by grade point average (GPA) and employs a two-step selection process of a natural sciences test (HAM-Nat), in some cases followed by multiple mini-interviews (HAM-Int). Multiple mini-interviews can predict non-cognitive outcomes, while GPA has predictive validity for cognitive outcomes. The aim of our study was to explore communication skills and clinical knowledge of advanced medical students according to their respective admission procedure.

METHODS

In July 2019, 146 students grouped according to their admission procedure into GPA-only (19.2 %), HAM-Nat (33.6 %), HAM-Int (30.8 %), and Waiting List (16.4 %) participated in four OSCE stations which equally assessed students' communication skills (OSCE part 1) and clinical knowledge (OSCE part 2) in simulated patient encounters, rated by physicians with checklists. Additionally, psychosocial assessors ranked communication skills with a global rating scale (GR). The students also participated in a multiple choice (MC) exam testing clinical knowledge. Kruskal-Wallis analyses of variance of test performance and Spearman correlation of instruments were calculated.

RESULTS

Students from the Waiting List group performed significantly worse on the MC exam compared to GPA-only and HAM-Int (adjusted p = .029 and 0.018, respectively). No significant differences were found between the admission groups with respect to communication skills. Global Rating and OSCE part 1 (communication) correlated significantly (ρ = 0.228, p = .006) as did OSCE part 2 (clinical knowledge) and MC exam (ρ = 0.242, p = .003), indicating criterion validity. Constructs did not overlap, indicating divergent validity.

CONCLUSIONS

Advanced medical students selected for undergraduate studies by multiple mini-interviews assessing psychosocial skills showed similar communication skills compared to students admitted to medical school by other entryways. It is unclear whether these similarities are due to an effective undergraduate longitudinal communication curriculum. Assessing baseline communication skills of all medical students at entry-level may aid with this question.

Improved detection of patient centeredness in objective structured clinical examinations through authentic scenario design

OBJECTIVE

In current objective structured clinical examinations (OSCEs), simulated encounters lacking realism reduce authenticity of assessment as students can take the OSCEs with a search-and-scan approach and trained empathy. Accordingly, patient-centeredness, the fundamental goal of OSCE, is not well assessed. This study evaluated the effect of a change in the OSCE scenario and checklist with respect to detecting students' patient-centeredness.

METHODS

A scenario script for valid representation of a real clinical encounter was developed and defined as authenticated scenario. The OSCE scores and the measure of patient-centered communication (MPCC) scores of 79 medical students were compared between the two OSCE stations, one using the traditional scenario and another using the authenticated scenario.

RESULTS

The MPCC total score was higher in the OSCE station using the authenticated scenario than that of the traditional scenario (p < 0.001). For the OSCE scores, the patient satisfaction score and the patient-physician interaction score were higher in the station using the authenticated scenario than the traditional one (p < 0.001).

CONCLUSION

The OSCE station using the authenticated scenario better detected medical student level of patient-centeredness.

PRACTICE IMPLICATIONS

Strengthening the authenticity of the OSCE scenario is critical for detecting the medical students' levels of patient-centeredness.

Twelve tips for introducing simulation based assessment in the objective structured clinical examination

The use of simulation as a tool for assessment in medical examinations is expanding. We describe 12 practical tips for the implementation of simulation based critical evaluation for the Objective Structured Clinical Examination (OSCE). The material in this article is a combination of personal experience and the available literature. Three themes are discussed encompassing simulation theory for the OSCE, practical features of simulation for the clinical examination and contingency planning. As clinical assessment evolves, the utility for simulation-based practice will advance alongside. An improved understanding of the processes for incorporating simulation into the OSCE will be of benefit to both the medical educator and the student.

Comparison of self and simulated patient assessments of first-year medical students' Interpersonal and Communication Skills (ICS) during Objective Structured Clinical Examinations (OSCE)

BACKGROUND

Interpersonal and communication skills (ICS) are important core competencies in medical education and certification. In this study, we identified self- and simulated patient (SP)-reported ratings of US first-year medical students' ICS and the influence of age and gender on performance appraisal during the Objective-Structured Clinical Examination (OSCE).

METHODS

OSCE participants, including 172 first-year medical students and 15 SPs were asked to evaluate the students' ICS using the American Board of Internal Medicine-Patient-Satisfaction Questionnaire (ABIM-PSQ), electronically and via paper, respectively. Self- and SP-reported ratings of students' ICS were presented as the median on a 5-point Likert-scale and as three categories defined as "good," "very good," and "inadequate."

RESULTS

SPs assessed all 172 students in the OSCE, while 43.6% of students assessed their own performance. The majority of students and SPs evaluated the students' ICS as very good. 23.3% of SPs and 5.3% of students rated the medical students' ability to encourage patient question-asking and answer questions as inadequate (P <  0.002). Neither age nor gender influenced the medical students' self-assessment of ICS. Female SPs assigned lower scores to students in regard to respecting patients and encouraging patient question-asking and answering. Older SPs was more likely to assign lower scores on all survey questions.

CONCLUSIONS

In the present study, self- and SP-reported ratings of first-year medical students' ICS were mainly "very good" with no influence of students' age or gender. Older age and female gender among the SPs were associated with a reduction in SP-reported ratings of students' ICS.

[Changes in the Objective Structured Clinical Examination (OSCE) of University Schools of Medicine during COVID-19. Experience with a computer-based case simulation OSCE (CCS-OSCE)◊]

BACKGROUND AND OBJECTIVES

The COVID-19 pandemic has forced universities to move the completion of university studies online. Spain's National Conference of Medical School Deans coordinates an objective, structured clinical competency assessment called the Objective Structured Clinical Examination (OSCE), which consists of 20 face-to-face test sections for students in their sixth year of study. As a result of the pandemic, a computer-based case simulation OSCE (CCS-OSCE) has been designed. The objective of this article is to describe the creation, administration, and development of the test.

MATERIALS AND METHODS

This work is a descriptive study of the CCS-OSCE from its planning stages in April 2020 to its administration in June 2020.

RESULTS

The CCS-OSCE evaluated the competences of anamnesis, exploration, clinical judgment, ethical aspects, interprofessional relations, prevention, and health promotion. No technical or communication skills were evaluated. The CCS-OSCE consisted of ten test sections, each of which had a 12-minutes time limit and ranged from six to 21 questions (mean: 1.1 minutes/question). The CCS-OSCE used the virtual campus platform of each of the 16 participating medical schools, which had a total of 2,829 students in their sixth year of study. It was jointly held on two dates in June 2020.

CONCLUSIONS

The CCS-OSCE made it possible to bring together the various medical schools and carry out interdisciplinary work. The CCS-OSCE conducted may be similar to Step 3 of the United States Medical Licensing Examination.

Completing the picture on student performances in OSCEs: A mixed-methods study on integration of a standardized patient rating

OBJECTIVE

Good physician communication skills increase patient satisfaction and improve healing processes. Although physicians and patients appear to value communicative competencies differently, students are often evaluated solely by physicians. This study examines whether additional assessment of students by 'standardized patients' (SPs) is useful.

METHODS

During their Objective Structured Clinical Examination (OSCE) 238 medical students were additionally rated by SPs at 9 stations according to two items that defined the 'physician-patient relationship' and 'communication'. SPs were informed that their assessment was for research purposes only, with no impact on the assessment of the students. SPs also had the opportunity to comment on their rating of the students.

RESULTS

The SPs rated the communicative competencies of students differently than physicians. The two parts of the SP rating are closely related. Inclusion of SP rating in the OSCE would provide higher measurement precision, with more students failing. SPs considered five factors relevant in their rating: 'human connection', 'information flow', 'professionalism', 'competence', and 'exam situation'.

CONCLUSION

Our study suggests inclusion of SP rating as additional assessment of student communication skills.

PRACTICE IMPLICATIONS

Addition of SP rating in assessments is worthwhile, as it appears to complete the picture of the student performance in their OSCEs.

Development and evaluation of a simulation-based transition to clerkship course

BACKGROUND

Transition to clerkship courses bridge the curricular gap between preclinical and clinical medical education. However, despite the use of simulation-based teaching techniques in other aspects of medical training, these techniques have not been adequately described in transition courses. We describe the development, structure and evaluation of a simulation-based transition to clerkship course.

APPROACH

Beginning in 2012, our institution embarked upon an extensive curricular transformation geared toward competency-based education. As part of this effort, a group of 12 educators designed, developed and implemented a simulation-based transition course. The course curriculum involved seven goals, centered around the 13 Association of American Medical Colleges Core Entrustable Professional Activities for entering residency. Instructional techniques included high-fidelity simulation, and small and large group didactics. Student competency was determined through a simulation-based inpatient-outpatient objective structured clinical examination, with real-time feedback and remediation. The effectiveness of the course was assessed through a mixed methods approach involving pre- and post-course surveys and a focus group.

EVALUATION

Of 166 students, 152 (91.6%) completed both pre- and post-course surveys, and nine students participated in the focus group. Students reported significant improvements in 21 out of 22 course objectives. Qualitative analysis revealed three key themes: learning environment, faculty engagement and collegiality. The main challenge to executing the course was procuring adequate faculty, material and facility resources.

REFLECTION

This simulation-based, resource-heavy transition course achieved its educational objectives and provided a safe, supportive learning environment for practicing and refining clinical skills.

Systematic review of student anxiety and performance during objective structured clinical examinations

INTRODUCTION

Test anxiety is well studied in higher education, but studies primarily concern traditional assessments, such as written examinations. As use of objective structured clinical examination (OSCE) in pharmacy education increases, a closer examination of non-cognitive factors such as test anxiety is warranted. The purpose of this review was to determine the association between OSCE-associated test anxiety with OSCE performance in health professional students.

METHODS

A literature search was conducted to identify peer-reviewed literature concerning test anxiety in health professional students associated with OSCE. Investigators searched for a combination of OSCE-related terms with anxiety-related terms using PubMed. Articles were included if they assessed OSCE-related anxiety by quantitative or qualitative methods. Data extracted from eligible articles included demographic data, type of the anxiety survey, associations between OSCE-related anxiety and performance, and other student-factors associated with OSCE-related anxiety.

RESULTS

The literature search yielded 339 articles. Nine articles met eligibility criteria and were included in the review. Results included students from medical, pharmacy, dental, and nursing professional programs. Anxiety was assessed via multiple scales. Six out of the eight studies assessing the relationship OSCE-related anxiety and OSCE performance found no association between the two measures. Contrary to literature concerning test anxiety in higher education, female gender was not associated with OSCE-related anxiety.

CONCLUSION

OSCE-related anxiety appears to have minimal to no influence on student performance. Future studies should utilize standardized anxiety assessments and should seek to understand anxiety's effects on student wellbeing and burnout.

Comparing learning outcomes among postgraduate year trainee groups

Background

The objective of postgraduate year (PGY) training programs is to inculcate in medical graduates the expected levels of skills in patient care. This study compared the core clinical competencies of trainees who received PGY training at Chang Gung Memorial Hospital by attending the pilot training program in different groups.

Methods

We used six 10-min test stations for clinical performance evaluation, which comprised four and two test stations designed for objective structured clinical examination and procedural skill, respectively, to evaluate the learning outcomes of the trainees. The trainees were divided into three groups according to the training programs that they had attended.

Results

The aspects of clinical performance included history taking, physical examination, medical communication, logical thinking, and problem-solving abilities. The trainees who selected the surgery-based training program exhibited a higher performance at the station for aseptic surgical preparation than the other two groups (p = 0.0261). The trainees who selected the internal medical training program (p = 0.0020) exhibited a higher performance at the station for abdominal pain in children.

Conclusions

A well-designed postgraduate training program should develop trainees' competencies, particularly clinical operational skills. The results of this study may provide useful insight into methods for improving the design of training programs. Additional investigation is necessary for understanding the effects of different programs on the clinical performance of trainees.

Reflective metacognition and objective structured clinical examination performance in introductory pharmacy practice experiences

BACKGROUND AND PURPOSE

Objective structured clinical examinations (OSCEs) allow for holistic evaluation of clinical competence, but integration of reflective practices on OSCE performance has not been well-studied.

EDUCATIONAL ACTIVITY AND SETTING

To develop metacognitive skills, second-year pharmacy students completed a self-reflection assignment on a mock medication therapy management OSCE in an introductory pharmacy practice experience course. Students who completed the course during the prior year (no self-reflection assignment) served as the control. The study assessed the correlation of students' reflections of their OCSE abilities to OSCE performance and evaluated if student demographics influenced the correlation.

FINDINGS

There were 107 students in the control group and 95 students in the intervention group. A higher proportion of students passed the overall OSCE in the control group (67.4%) compared to the intervention group (58.9%), but this was not statistically significant. Neither gender, age, nor Pharmacy College Admissions Test score had an effect on OSCE pass rates. Grade point average did have a significant effect on OSCE pass rates (P = .019). Overall, no statistically significant association was seen in summative OSCE performance between students who used structured, guided questions to reflect on their formative OSCE performance and those who did not (P > .05).

SUMMARY

Despite this study's results, faculty continue to embrace the concept of reflective practice. Further research is needed to assess the value and role of reflective practice in pharmacy education.

Assessment of student performance on a mock new prescription counseling session and an objective structured clinical examination across five years

INTRODUCTION

New prescription counseling (NPC) provides vital information to patients regarding newly prescribed medications to improve treatment outcomes. This evaluation's goals were to describe the implementation of teaching and assessment methods incorporated into an NPC module across two courses at Auburn University and evaluate student performance on assessments of NPC skills over five years.

METHODS

Assessments included self and peer evaluations of a recorded mock NPC session, NPC objective structured clinical examination (OSCE), and overall course grade. Scores were analyzed over a five-year period (2012 to 2016) to evaluate mean student performance in each of four domains (gathering information, communication, management strategies, and monitoring and follow-up) and overall for each assessment. Pearson's correlation coefficients between these scores were calculated by combining all five years of data.

RESULTS

Seven-hundred thirty-three students were included in the analysis. No trends in mean domain and total assessment scores were noted across years. Self and peer evaluation domain and total scores were significantly, though moderately, correlated across assessments (r = 0.43-0.51), except in the communication domain (r = 0.12). Overall, NPC OSCE total scores were not correlated with NPC recording self or peer evaluation total scores (r = 0.06 and r = 0.11, respectively). Assessment total scores were all moderately correlated with overall course grades.

CONCLUSIONS

Utilization of a scaffolding approach with class discussion, demonstration, role-play, self and peer evaluation, and OSCEs, is an effective means of building student competence in patient counseling for the NPC context.

Comparison of students' performance of objective structured clinical examination during clinical practice

PURPOSE

Clinical practice for medical students aims to improve their clinical competency. Since students gain experience in clinical practice, it is assumed that their capacity for care is strengthened. This study aimed to verify whether there is a change in the competency of medical students after clinical practice.

METHODS

We assessed the clinical performance of 120 medical students who were enrolled in the fifth year over the academic years 2016-2018. Objective structured clinical examination (OSCE) was used to measure the clinical competency.

RESULTS

The OSCE scores for November fifth grade improved significantly compared to May fifth grade. The scores in September of sixth grade were slightly lower than those of November of fifth grade, but there was no statistically significant difference.

CONCLUSION

It was confirmed that the clinical practice of fifth-year students improved their clinical competency, but it did not show any significant shift in the clinical competency of sixth-year students. The results of this study suggest that it is very important to organize the curriculum so that students can continue to experience patient-facing care throughout the entire clinical practice course to produce highly qualified physicians.

[Competencies of Primary Care residents in the management of prevalent problems of the ear, nose and throat]

Ear, nose and throat problems are very common in Primary Care settings. Numerous studies have reported that Primary Care physicians feel dissatisfied with their performance and insufficient training to diagnose and manage these problems. The objective of this study was to assess the competency in diagnosis and management of prevalent ear, nose and throat problems in medical residents of Family Medicine, Internal Medicine and Pediatrics at a University Hospital in Buenos Aires, Argentina, using an Objective Structured Clinical Examination. The participants were 25 residents of Pediatrics (7), Internal Medicine (10) and Family Medicine (8). Residents of Family Medicine and Pediatrics showed better performance than residents of Internal Medicine. Residents of Primary Care demonstrated dissimilar competencies in the management of prevalent ear, nose and throat problems. Areas of common improvement opportunities were detected for the three groups, which provided direction and motivation in future learnings in both knowledge and skills.

Impressions on Reliability and Students' Perceptions of Learning in a Peer-Based OSCE

BACKGROUND

Peer assessment of performance in the objective structured clinical examination (OSCE) is emerging as a learning instrument. While peers can provide reliable scores, there may be a trade-off with students' learning. The purpose of this study is to evaluate a peer-based OSCE as a viable assessment instrument and its potential to promote learning and explore the interplay between these two roles.

METHODS

A total of 334 medical students completed an 11-station OSCE from 2015 to 2016. Each station had 1-2 peer examiners (PE) and one faculty examiner (FE). Examinees were rated on a 7-point scale across 5 dimensions: Look, Feel, Move, Special Tests and Global Impression. Students participated in voluntary focus groups in 2016 to provide qualitative feedback on the OSCE. Authors analysed assessment data and transcripts of focus group discussions.

RESULTS

Overall, PE awarded higher ratings compared with FE, sources of variance were similar across 2 years with unique variance consistently being the largest source, and reliability (r _φ ) was generally low. Focus group analysis revealed four themes: Conferring with Faculty Examiners, Difficulty Rating Peers, Insider Knowledge, and Observing and Scoring.

CONCLUSIONS

While peer assessment was not reliable for evaluating OSCE performance, PE's perceived that it was beneficial for their learning. Insight gained into exam technique and self-appraisal of skills allows students to understand expectations in clinical situations and plan approaches to self-assessment of competence.

Team-based learning for teaching musculoskeletal ultrasound skills: a prospective randomised trial

OBJECTIVE

The aim of this prospective randomised trial was to assess the impact of the team-based learning approach on basic musculoskeletal ultrasound skills in comparison to both peer-assisted and conventional teaching and to examine the influence of gender and learning style on learning outcomes.

METHODS

In this prospective randomised trial, we randomly assigned 88 students to 3 groups: team-based learning (n = 19), peer-assisted learning (n = 36) and conventional teaching (n = 33). Pre-existing knowledge was assessed using a multiple-choice (MC) exam. Student performance after completing the course was measured using an Objective Structured Clinical Examination (OSCE) and a second MC exam. Students were asked to complete Kolb's Learning Style Inventory and to evaluate the course.

RESULTS

There was a significant gain in theoretical knowledge for all students (p < 0.001). The team-based learning groups' performance proved to be significantly superior on the OSCE (p = 0.001). As gender had no significant effect on practical or theoretical performance, learning style was linked to differences in the practical outcome. An evaluation showed overall satisfaction with the course and with the respective teaching methods.

CONCLUSION

Team-based learning proved to be superior to peer-assisted and conventional teaching of musculoskeletal ultrasound skills.

The development of a six-station OSCE for evaluating the clinical competency of the student nurses before graduation: A validity and reliability analysis

BACKGROUND

Assessing clinical competency in undergraduate nursing students is essential before they enter clinical practice. The objective structured clinical examination (OSCE) is widely used and is part of clinical competency assessment. Aim The aim of this study was to develop and validate a six-station OSCE for evaluating the clinical competency of the student nurses before graduation.

METHODS

One-group longitudinal observational study. An OSCE of 6 stations of 10 min each was adopted. Blueprint setting, checklist development, and standard patient and examiner training were conducted. The mean, standard deviation, item scale (item-total correlation) and comparisons of extreme groups for item analysis. The inter-coder of examiners and inter-case of students were used to test reliability. Content Validity and concurrent validity were used to evaluate validity of OSCE.

RESULTS

One hundred nursing students participated in this study. The mean OSCE score of students who passed the registered nurses' examination was significantly higher than that of students who failed. Significant differences in OSCE score between students who remain versus left clinical job were observed at 3, 6 and 9 months after graduation.

CONCLUSION

The OSCE is predictable method for assessing undergraduate students' first year clinical retention.