Speaking up: using OSTEs to understand how medical students address professionalism lapses

Medical Students' Speak Up Barriers: A Randomized Controlled Trial With Written Vignettes

OBJECTIVES

Little is known about medical students' speak-up barriers upon recognizing or becoming aware of risky or deficient actions of others. Improving our knowledge on these helps in preparing student to function in actual health care organizations. The aim was to examine medical students' perceived reasons for silence in respect to different speak-up situations (i.e., vignette content) and to test if vignette difficulty had an effect on reasons indicated.

METHODS

This study was a randomized, controlled, single-blind trial, with text-based vignettes to investigate speak-up barriers. Vignette contents described speak-up situations that varied systematically with respect to speak up barrier (i.e., environmental norm, uncertainty, hierarchy) and difficulty (i.e., easy, difficult). For each vignette, participants indicated which speak-up barriers they regarded as important.Descriptive analysis was performed for the study population, the numbers of barriers perceived and rating of vignette difficulty. Logistic regression analysis was used to examine the association between barriers perceived and vignette contents, designed vignette difficulty and subjectively rated vignette difficulty.

RESULTS

A total of 265 students were included. The response rate was 100%. Different barriers were relevant for the different vignettes and varied in a consistent way with the theme of the vignette. Significantly more speak-up barriers were indicated for participants with the difficult version for vignette 1 (not an environmental norm) and vignette 3 (hierarchy) with odds ratio (OR) = 1.52 and 95% confidence interval (95% CI: 1.33-1.73) and OR = 1.25 (95% CI: 1.09-1.44). For (OR) estimates, confidence intervals were rather large.

CONCLUSIONS

Perceived barriers for speak-up vary consistently with the characteristics of the situation and more barriers preventing speak up were related to the difficult versions of the vignettes.

Natural Language Processing of Learners' Evaluations of Attendings to Identify Professionalism Lapses

Unprofessional faculty behaviors negatively impact the well-being of trainees yet are infrequently reported through established reporting systems. Manual review of narrative faculty evaluations provides an additional avenue for identifying unprofessional behavior but is time- and resource-intensive, and therefore of limited value for identifying and remediating faculty with professionalism concerns. Natural language processing (NLP) techniques may provide a mechanism for streamlining manual review processes to identify faculty professionalism lapses. In this retrospective cohort study of 15,432 narrative evaluations of medical faculty by medical trainees, we identified professionalism lapses using automated analysis of the text of faculty evaluations. We used multiple NLP approaches to develop and validate several classification models, which were evaluated primarily based on the positive predictive value (PPV) and secondarily by their calibration. A NLP-model using sentiment analysis (quantifying subjectivity of the text) in combination with key words (using the ensemble technique) had the best performance overall with a PPV of 49% (CI 38%-59%). These findings highlight how NLP can be used to screen narrative evaluations of faculty to identify unprofessional faculty behaviors. Incorporation of NLP into faculty review workflows enables a more focused manual review of comments, providing a supplemental mechanism to identify faculty professionalism lapses.

The Objective Structured Teaching Encounter (OSTE) in health professions education: A systematic review

PURPOSE

New emphasis on the assessment of health professions educators' teaching competence has led to greater use of the Objective Structured Teaching Encounter (OSTE). The purpose of this study is to review and further describe the current uses and learning outcomes of the OSTE in health professions education.

MATERIALS AND METHODS

PubMed, MEDLINE, and CINAHL (March 2010 to February 2022) were searched for English-language studies describing the use of an OSTE for any educational purpose within health professions education.

RESULTS

Of the 29 articles that met inclusion criteria, over half of the studies (17 of 29, 58.6%) were published during or after 2017. Seven studies described OSTE use outside of the traditional medical education context. These new contexts included basic sciences, dental, pharmacy, and Health Professions Education program graduates. Eleven articles described novel OSTE content, which included leadership skills, emotional intelligence, medical ethics, inter-professional conduct, and a procedural OSTE. There is increasing evidence supporting the use of OSTEs for the assessment of clinical educators' teaching skills.

CONCLUSIONS

The OSTE is a valuable tool for the improvement and assessment of teaching within a variety of health professions education contexts. Further study is required to determine the impact of OSTEs on teaching behaviors in real-life contexts.

Why do few medical students report their experiences of mistreatment to administration?

INTRODUCTION

Over 50% of medical students worldwide report experiencing mistreatment and abuse during their clinical education, yet only a small proportion of students report these concerns to administration. It is unknown how medical students make sense of their experiences of mistreatment and come to decide whether to formally report these experiences. Improved understanding of this phenomenon will facilitate changes at the administrative and institutional levels to better support students.

METHODS

Using Constructivist Grounded Theory, we interviewed 19 current and former medical students from one institution about their experiences with mistreatment and reporting. Data were analysed in an iterative fashion, using focused and theoretical forms of coding.

RESULTS

The decision of whether to report mistreatment is only one phase in the process that students report experiencing when encountering mistreatment. This process can be understood as a journey consisting of five phases: Situating, Experiencing and Appraising, Reacting, Deciding and Moving Forward. Students move through these phases as they come to understand their position as medical learners and their ability to trust and be safe within this institution. Each experience of mistreatment causes students to react to what has happened to them, decide if they will share their experiences and reach out for support. They choose if they are going to report the mistreatment, at what cost and for what outcomes. Students continue through their training while incorporating their experiences into their understanding of the culture in which they are learning and continually resituating themselves within the institution.

DISCUSSION

Student perceptions of trust or mistrust in their educational institution are highly influential when it comes to reporting mistreatment. Interventions designed to support students and decrease exposure to mistreatment may be best focused on increasing organisational trust between students and the medical school.

Receiving Real-Time Clinical Feedback: A Workshop and OSTE Assessment for Medical Students

BACKGROUND

Many programs designed to improve feedback to students focus on faculty's ability to provide a safe learning environment, and specific, actionable suggestions for improvement. Little attention has been paid to improving students' attitudes and skills in accepting and responding to feedback effectively. Effective "real-time" feedback in the clinical setting is dependent on both the skill of the teacher and the learner's ability to receive the feedback. Medical students entering their clinical clerkships are not formally trained in receiving feedback, despite the significant amount of feedback received during this time.

METHODS

We developed and implemented a one-hour workshop to teach medical students strategies for effectively receiving and responding to "real-time" (formative) feedback in the clinical environment. Subjective confidence and skill in receiving real-time feedback were assessed in pre- and post-workshop surveys. Objective performance of receiving feedback was evaluated before and after the workshop using a simulated feedback encounter designed to re-create common clinical and cognitive pitfalls for medical students, called an objective structured teaching exercise (OSTE).

RESULTS

After a single workshop, students self-reported increased confidence (mean 6.0 to 7.4 out of 10, P<0.01) and skill (mean 6.0 to 7.0 out of 10, P=0.10). Compared to pre-workshop OSTE scores, post-workshop OSTE scores objectively measuring skill in receiving feedback were also significantly higher (mean 28.8 to 34.5 out of 40, P=0.0131).

CONCLUSION

A one-hour workshop dedicated to strategies in receiving real-time feedback may improve effective feedback reception as well as self-perceived skill and confidence in receiving feedback. Providing strategies to trainees to improve their ability to effectively receive feedback may be a high-yield approach to both strengthen the power of feedback in the clinical environment and enrich the clinical experience of the medical student.

Investigating US medical students' motivation to respond to lapses in professionalism

CONTEXT

As unprofessional behaviour in physicians can compromise patient safety, all physicians should be willing and able to respond to lapses in professionalism. Although students endorse an obligation to respond to lapses, they experience difficulties in doing so. If medical educators knew how students respond and why they choose certain responses, they could support students in responding appropriately.

OBJECTIVES

The aim of this study was to describe medical students' responses to professionalism lapses in peers and faculty staff, and to understand students' motivation for responding or not responding.

METHODS

We conducted an explorative, qualitative study using template analysis, in which three researchers independently coded transcripts of semi-structured, face-to-face interviews. We purposefully sampled 18 student representatives convening at a medical education conference. Preliminary open coding of a data subset yielded an initial template, which was applied to further data and modified as necessary. All transcripts were coded using the final template. Finally, three sensitising concepts from the Expectancy-Value-Cost model were used to map participants' responses.

RESULTS

Students mentioned having observed lapses in professionalism in both faculty staff and peers. Students' responses to these lapses were avoiding, addressing, reporting or initiating policy change. Generally, students were not motivated to respond if they did not know how to respond, if they believed responding was futile and if they feared retaliation. Students were motivated to respond if they were personally affected, if they perceived the individual as approachable and if they thought that the whole group of students could benefit from their actions. Expectancy of success, value and costs each appeared to be influenced by (inter)personal and system factors.

CONCLUSIONS

The Expectancy-Value-Cost model effectively explains students' motivation for responding to lapses. The (inter)personal and system factors influencing students' motivation to respond are modifiable and can be used by medical educators to enhance students' motivation to respond to lapses in professionalism observed in medical school.

Top Medical Education Studies of 2016: A Narrative Review

Education, like clinical medicine, should be based on the most current evidence in the field. Unfortunately, medical educators can be overwhelmed by the sheer volume and range of resources for this literature. This article provides an overview of 15 articles from 2016 that the authors consider the top articles in the field of pediatric medical education. The 7 authors, all medical educators with combined leadership and expertise across the continuum of pediatric medical education, used an iterative 3-stage process to review more than 6339 abstracts published in 2016. This process was designed to identify a small subset of articles that were most relevant to educational practices and most applicable to pediatric medical education. In the first 2 stages, pairs of authors independently reviewed and scored abstracts in 13 medical education-related journals and reached consensus to identify the articles that best met these criteria. In the final stage, all articles were discussed using a group consensus model to select the final articles included in this review. This article presents summaries of the 15 articles that were selected. The results revealed a cluster of studies related to observed standardized clinical encounters, self-assessment, professionalism, clinical teaching, competencies/milestones, and graduate medical education management strategies. We provide suggestions on how medical educators can apply the findings to their own practice and educational settings. This narrative review offers a useful tool for educators interested in keeping informed about the most relevant and valuable information in the field.

Difficult Interpersonal Encounters with Medical Students and Residents: Two Objective Standardized Teaching Encounters

INTRODUCTION

Objective standardized teaching exercises (OSTEs) are widely used to develop professional competencies, especially in the health care professions. An OSTE involves exposing different providers to the same, time-limited scenario that is concurrently observed and/or recorded for either formative or summative evaluation. As there are limited resources available for creating a resident-specific OSTE, especially those applicable to family and community medicine residents, we created and evaluated a resident OSTE (R-OSTE) for second- and third-year family and community medicine residents.

METHODS

This R-OSTE involved two cases. The first featured Taylor, a third-year medical student resistant to feedback. The second featured Kris, a first-year resident nervous about approaching the attending on duty. Our R-OSTE had residents teaching interpersonal skills to trained actors in a standardized learner role.

RESULTS

Residents in the teaching role were formatively evaluated by peer observers (fellow residents) and standardized learners on interpersonal domains such as communication and professionalism. Learners gave residents an average performance rating of 4.9 on a 1 to 6 scale with 1 = Very Poor and 6 = Excellent. Residents also evaluated the OSTE itself, rating their experience on multiple teaching-related statements. Eighty-six percent of residents agreed this exercise was an appropriate development activity for family medicine residents. Overall, our R-OSTE was rated highly for relevance to teaching by the residents.

DISCUSSION

The residents were rated highly by both peer observers and standardized learners. However, there was little variability in peer observer scores, indicating the need for an alternative method of measurement.

Teachers as Learners: Developing Professionalism Feedback Skills via Observed Structured Teaching Encounters

This Conversations Starter article presents a selected research abstract from the 2017 Association of American Medical Colleges Southern Region Group on Educational Affairs annual spring meeting. The abstract is paired with the integrative commentary of 4 experts who shared their thoughts stimulated by the study. These thoughts explore the value of the Observed Structured Teaching Encounter in providing structured opportunities for medical students to engage with the complexities of providing peer feedback on professionalism.

Using Objective Structured Teaching Encounters (OSTEs) to prepare chief residents to be emotionally intelligent leaders

BACKGROUND

Chief Residents must lead, manage and mentor a diverse and often large group of residents, however there is a lack of formal leadership training throughout graduate medical education.

OBJECTIVE

Development of a 3-part Chief Resident (CR) Program focused on leading, managing and mentoring.

DESIGN

Each participant completes an Emotional Intelligence (EI) Inventory prior to the day-long event. Participants receive their EI scores at the beginning of the program, which features interactive sessions on leadership, management, and feedback skills. The program then reinforces the application of their new knowledge about EI through a four station OSTE (Observed Structured Teaching Encounter). CRs practice feedback and coaching skills in a simulated environment where they need to provide the context of formative feedback to a standardized resident.

RESULTS

The aggregated mean pre-session EI score for all participants was 76.9 (an ideal score is >85). An independent-samples t-test compared the CRs' leadership and feedback performance on their first and second OSTE performance within a single afternoon session. There was a significant difference between the first OSTE performance (M = 47.92, SD = 7.8) and the second OSTE performance (M = 51.22, SD = 6.9); t (68) = 1.99, p = 0.006. These results suggest that participating in multiple OSTEs positively reinforces the core interpersonal and communication skills discussed in the didactic and practiced in the interactive portions of the program.

CONCLUSION

The low mean pre-session EI score achieved by our participants supports the idea that CRs enter their new roles with a level of EI that can be enhanced. CRs had an overall positive reaction to EI and its application to the core skills addressed in the program, highlighting the fact that similar programs could be used to train early career physicians to be more skilled and comfortable with leading, managing and mentoring.

ABBREVIATIONS

CR: Chief resident; EI: Emotional intelligence; GME: Graduate medical education; OSTE: Objective structured teaching encounter.