Learner deficits and academic outcomes of medical students, residents, fellows, and attending physicians referred to a remediation program, 2006-2012

For the General Internist: A Summary of Key Innovations in Medical Education

We conducted a review of published medical education articles to identify high-quality research and innovation relevant to educators in general medicine. Our review team consisted of six general internists with expertise in medical education and a professional medical librarian. We manually searched 15 journals in pairs (a total of 3062 citations) for original research articles in medical education published in 2014. Each pair of reviewers independently rated the relevance, importance, and generalizability of articles on medical education in their assigned journals using a 27-point scale (maximum of 9 points for each characteristic). From this list, each team member independently reviewed the 22 articles that received a score of 20 or higher from both initial reviewers, and for each selected article rated the quality and global relevance for the generalist educator. We included the seven top-rated articles for presentation in this review, and categorized the studies into four general themes: continuity clinic scheduling, remediation, interprofessional education, and quality improvement and patient safety. We summarized key findings and identified significant limitations of each study. Further studies assessing patient outcomes are needed to strengthen the literature in medical education. This summary of relevant medical education articles can inform future research, teaching, and practice.

Twelve tips for developing and maintaining a remediation program in medical education

Remediation in medical education, the process of facilitating corrections for physician trainees who are not on course to competence, predictably consumes significant institutional resources. Although remediation is a logical consequence of mandating, measuring, and reporting clinical competence, many program leaders continue to take an unstructured approach toward organizing effective, efficient plans for struggling trainees, almost all of who will become practicing physicians. The following 12 tips derive from a decade of remediation experience at each of the authors' three institutions. It is informed by the input of a group of 34 interdisciplinary North American experts assembled to contribute two books on the subject. We intend this summary to guide program leaders to build better remediation systems and emphasize that developing such systems is an important step toward enabling the transition from time-based to competency-based medical education.

Association of Characteristics, Deficits, and Outcomes of Residents Placed on Probation at One Institution, 2002-2012

PURPOSE

To describe the population of residents placed on probation, identify learner characteristics associated with being placed on probation, and describe immediate and long-term career outcomes for those placed on probation as compared with matched controls.

METHOD

The authors collected data for residents at the University of Colorado School of Medicine placed on probation from July 2002 to June 2012, including postgraduate year placed on probation, deficits identified, mandated evaluation for physical and mental health, duration of probation, disability accommodations requested, and number of additional training months required. They were retrospectively compared with 102 controls matched for specialty, matriculation, and postgraduate year. Variables assessed included demographics, academic performance, license status, specialty, state board certification, and board citations.

RESULTS

Of 3,091 residents, 3.3% were placed on probation (88 residents; 14 fellows). Compared with controls, those on probation were more likely to be international medical graduates, married, not Caucasian, older (all P < .001), male (P = .01), to have transferred from another graduate medical education training program, and to have taken time off between medical school and residency (all P < .001). Among those currently in practice, 53 (63.9%) were board certified compared with 93 (100%) of the controls. Placement on probation was associated with failure to graduate and lack of board certification. All 7 graduates cited by state medical boards were in the probation group.

CONCLUSIONS

Further research is needed to understand these associations and to determine whether changes in curricula or remediation programs may alter these outcomes.

Time to Loosen the Apron Strings: Cohort-based Evaluation of a Learner-driven Remediation Model at One Medical School

BACKGROUND

Remediation in the era of competency-based assessment demands a model that empowers students to improve performance.

AIM

To examine a remediation model where students, rather than faculty, develop remedial plans to improve performance.

SETTING/PARTICIPANTS

Private medical school, 177 medical students.

PROGRAM DESCRIPTION

A promotion committee uses student-generated portfolios and faculty referrals to identify struggling students, and has them develop formal remediation plans with personal reflections, improvement strategies, and performance evidence. Students submit reports to document progress until formally released from remediation by the promotion committee.

PROGRAM EVALUATION

Participants included 177 students from six classes (2009-2014). Twenty-six were placed in remediation, with more referrals occurring during Years 1 or 2 (n = 20, 76 %). Unprofessional behavior represented the most common reason for referral in Years 3-5. Remedial students did not differ from classmates (n = 151) on baseline characteristics (Age, Gender, US citizenship, MCAT) or willingness to recommend their medical school to future students (p < 0.05). Two remedial students did not graduate and three did not pass USLME licensure exams on first attempt. Most remedial students (92 %) generated appropriate plans to address performance deficits.

DISCUSSION

Students can successfully design remedial interventions. This learner-driven remediation model promotes greater autonomy and reinforces self-regulated learning.

A contemporary approach to validity arguments: a practical guide to Kane's framework

CONTEXT

Assessment is central to medical education and the validation of assessments is vital to their use. Earlier validity frameworks suffer from a multiplicity of types of validity or failure to prioritise among sources of validity evidence. Kane's framework addresses both concerns by emphasising key inferences as the assessment progresses from a single observation to a final decision. Evidence evaluating these inferences is planned and presented as a validity argument.

OBJECTIVES

We aim to offer a practical introduction to the key concepts of Kane's framework that educators will find accessible and applicable to a wide range of assessment tools and activities.

RESULTS

All assessments are ultimately intended to facilitate a defensible decision about the person being assessed. Validation is the process of collecting and interpreting evidence to support that decision. Rigorous validation involves articulating the claims and assumptions associated with the proposed decision (the interpretation/use argument), empirically testing these assumptions, and organising evidence into a coherent validity argument. Kane identifies four inferences in the validity argument: Scoring (translating an observation into one or more scores); Generalisation (using the score[s] as a reflection of performance in a test setting); Extrapolation (using the score[s] as a reflection of real-world performance), and Implications (applying the score[s] to inform a decision or action). Evidence should be collected to support each of these inferences and should focus on the most questionable assumptions in the chain of inference. Key assumptions (and needed evidence) vary depending on the assessment's intended use or associated decision. Kane's framework applies to quantitative and qualitative assessments, and to individual tests and programmes of assessment.

CONCLUSIONS

Validation focuses on evaluating the key claims, assumptions and inferences that link assessment scores with their intended interpretations and uses. The Implications and associated decisions are the most important inferences in the validity argument.

Twelve tips for addressing medical student and resident physician lapses in professionalism

Medical educators have gained significant ground in the practical and scholarly approach to professionalism. When a lapse occurs, thoughtful remediation to address the underlying issue can have a positive impact on medical students and resident physicians, while failure to address lapses, or to do so ineffectively, can have long-term consequences for learners and potentially patients. Despite these high stakes, educators are often hesitant to address lapses in professionalism, possibly due to a lack of time and familiarity with the process. Attention must be paid to generalizable, hands-on recommendations for daily use so that clinicians and administrators feel well equipped to tackle this often difficult yet valuable task. This article reviews the literature related to addressing unprofessional behavior among trainees in medicine and connects it to the shared experience of medical educators at one institution. The framework presented aims to provide practical guidance and empowerment for educators responsible for addressing medical student and resident physician lapses in professionalism.

Methods and outcomes for the remediation of clinical reasoning

INTRODUCTION

There is no widely accepted structured, evidence based strategy for the remediation of clinical reasoning skills.

AIM

To assess the effectiveness of a standardized clinical reasoning remediation plan for medical learners at various stages of training.

SETTING

Learners enrolled in the University of Colorado School of Medicine Remediation Program.

PROGRAM DESCRIPTION

From 2006 to 2012, the learner remediation program received 151 referrals. Referrals were made by medical student clerkship directors, residency and fellowship program directors, and through self-referrals. Each learner's deficiencies were identified using a standardized assessment process; 53 were noted to have clinical reasoning deficits. The authors developed and implemented a ten-step clinical reasoning remediation plan for each of these individuals, whose subsequent performance was independently assessed by unbiased faculty and senior trainees. Participant demographics, faculty time invested, and learner outcomes were tracked.

PROGRAM EVALUATION

Prevalence of clinical reasoning deficits did not differ by level of training of the remediating individual (p = 0.49). Overall, the mean amount of faculty time required for remediation was 29.6 h (SD = 29.3), with a median of 18 h (IQR 5-39) and a range of 2-100 h. Fifty-one of the 53 (96%) passed the post remediation reassessment. Thirty-eight (72%) learners either graduated from their original program or continue to practice in good standing. Four (8%) additional residents who were placed on probation and five (9%) who transferred to another program have since graduated.

DISCUSSION

The ten-step remediation plan proved to be successful for the majority of learners struggling with clinical reasoning based on reassessment and limited subsequent educational outcomes. Next steps include implementing the program at other institutions to assess generalizability and tracking long-term outcomes on clinical care.