The ACGME competencies in the operating room

Competency-Based Education: Will This be the New Training Paradigm in Plastic Surgery?

The Accreditation Council for Graduate Medical Education created the "Next Accreditation System" in 2013 requiring residents to meet educational milestones based on core competencies over the course of their training. The 6 core competencies include patient care and technical skills, medical knowledge, practice-based learning and improvement, interpersonal and communication skills, professionalism, and systems-based practice. Since the traditional time-based model requires a predetermined length of training irrespective of learning style, pace, or activity, a competency-based model is appealing because it refocuses education on deliberate and relevant skills acquisition and retention. Plastic surgery has been slowly transitioning to competency-based education (CBE), thereby permitting residents to learn at their own pace to master each competency. We performed a nonsystematic literature review of the efficacy of CBE and implementation efforts, particularly within plastic surgery. The literature revealed perceived barriers to implementation, as well as the nuts and bolts of implementation. We highlighted possible solutions and training tools with practical applications in plastic surgery. Success of CBE in plastic surgery requires instituting a transparent process that involves continuously piloting multiple assessment tools and a discussion of related costs. CBE may be particularly appealing for trainees focused on further training in craniofacial or pediatric plastic surgery after completion of an integrated or independent training program in plastic surgery to allow them to focus on their career interests once competence is achieved in the core skills required of a plastic surgeon.

Core Professionalism Education in Surgery: A Systematic Review

BACKGROUND

Professionalism education is one of the major elements of surgical residency education.

AIMS

To evaluate the studies on core professionalism education programs in surgical professionalism education.

STUDY DESIGN

Systematic review.

METHODS

This systematic literature review was performed to analyze core professionalism programs for surgical residency education published in English with at least three of the following features: program developmental model/instructional design method, aims and competencies, methods of teaching, methods of assessment, and program evaluation model or method. A total of 27083 articles were retrieved using EBSCOHOST, PubMed, Science Direct, Web of Science, and manual search.

RESULTS

Eight articles met the selection criteria. The instructional design method was presented in only one article, which described the Analysis, Design, Development, Implementation, and Evaluation model. Six articles were based on the Accreditation Council for Graduate Medical Education criterion, although there was significant variability in content. The most common teaching method was role modeling with scenario- and case-based learning. A wide range of assessment methods for evaluating professionalism education were reported. The Kirkpatrick model was reported in one article as a method for program evaluation.

CONCLUSION

It is suggested that for a core surgical professionalism education program, developmental/instructional design model, aims and competencies, content, teaching methods, assessment methods, and program evaluation methods/models should be well defined, and the content should be comparable.

Perioperative feedback in surgical training: A systematic review

BACKGROUND

Changes in surgical training have raised concerns about residents' operative exposure and preparedness for independent practice. One way of addressing this concern is by optimizing teaching and feedback in the operating room (OR). The objective of this study was to perform a systematic review on perioperative teaching and feedback.

METHODS

A systematic literature search identified articles from 1994 to 2014 that addressed teaching, feedback, guidance, or debriefing in the perioperative period. Data was extracted according to ENTREQ guidelines, and a qualitative analysis was performed.

RESULTS

Thematic analysis of the 26 included studies identified four major topics. Observation of teaching behaviors in the OR described current teaching practices. Identification of effective teaching strategies analyzed teaching behaviors, differentiating positive and negative teaching strategies. Perceptions of teaching behaviors described resident and attending satisfaction with teaching in the OR. Finally models for delivering structured feedback cited examples of feedback strategies and measured their effectiveness.

CONCLUSIONS

This study provides an overview of perioperative teaching and feedback for surgical trainees and identifies a need for improved quality and quantity of structured feedback.

Resident training in a teaching hospital: How do attendings teach in the real operative environment?

BACKGROUND

The study aim was to explore the nature of intraoperative education and its interaction with the environment where surgical education occurs.

METHODS

Video and audio recording captured teaching interactions between colorectal surgeons and general surgery residents during laparoscopic segmental colectomies. Cases and collected data were analyzed for teaching behaviors and workflow disruptions. Flow disruptions (FDs) are considered deviations from natural case progression.

RESULTS

Across 10 cases (20.4 operative hours), attendings spent 11.2 hours (54.7%) teaching, using directing (M = 250.1), and confirming (M = 236.1) most. FDs occurred 410 times, accounting for 4.4 hours of case time (21.57%). Teaching occurred with FD events for 2.4 hours (22.2%), whereas 77.8% of teaching happened outside FD occurrence. Teaching methods shifted from active to passive during FD events to compensate for patient safety.

CONCLUSIONS

Understanding how FDs impact operative learning will inform faculty development in managing interruptions and improve its integration into resident education.

Exploring the Content of Intraoperative Teaching

INTRODUCTION

Much teaching to surgical residents takes place in the operating room (OR). The explicit content of what is taught in the OR, however, has not previously been described. This study investigated the content of what is taught in the OR, specifically during laparoscopic cholecystectomies (LCs), for which a cognitive task analysis (CTA), explicitly delineating individual steps, was available in the literature.

METHODS

A checklist of necessary technical and decision-making steps to be executed during performance of LCs, anchored in the previously published CTA, was developed. A convenience sample of LCs was identified over a 12-month period from February 2011 to February 2012. Using the checklist, a trained observer recorded explicit teaching that occurred regarding these steps during each observed case. All observations were tallied and analyzed.

RESULTS

In all, 51 LCs were observed; 14 surgery attendings and 33 residents participated in the observed cases. Of 1042 observable teaching points, only 560 (53.7%) were observed during the study period. As a proportion of all observable steps, technical steps were observed more frequently, 377 (67.3%), than decision-making steps, 183 (32.7%). Also when focusing on technical and decision-making steps alone, technical steps were taught more frequently (60.9% vs 43.3%).

CONCLUSIONS

Only approximately half of all possible observable teaching steps were explicitly taught during LCs in this study. Technical steps were more frequently taught than decision-making steps. These findings may have important implications: a better understanding of the content of intraoperative teaching would allow educators to steer residents' preoperative preparation, modulate intraoperative instruction by members of the surgical faculty, and guide residents to the most appropriate teaching venues.

An Apprenticeship Rotation Teaches Chief Residents Nontechnical Skills and ACGME Core Competencies

BACKGROUND

Traditionally, surgical training has used an apprenticeship model but has more recently moved to a service-based model, with groups of residents working with groups of attending surgeons. We developed an apprenticeship rotation to enhance one-on-one interaction between chief residents and selected faculty. We hypothesized that the apprenticeship rotation would be effective for teaching nontechnical skills (NTS) and core competencies.

MATERIALS AND METHODS

An apprenticeship rotation was created at a university-based surgery residency in which each chief resident selected a single attending surgeon with whom to work exclusively with for a 4-week period. Emphasis was placed on teaching intraoperative NTS as well as the 4 difficult-to-teach Accreditation Council for Graduate Medical Education core competencies (DCC): Interpersonal and Communication Skills, Practice-Based Learning and Improvement, Professionalism, and Systems-Based Practice. Participants were surveyed afterwards about their rotation using a 5-point Likert scale. A Wilcoxon signed rank test was used to compare differences depending on data distribution.

RESULTS

All (13/13) the chief residents and 67% (8/12) faculty completed the survey. Overall, 85% of residents and 87.5% of faculty would recommend the rotation to other residents/faculty members. Both residents and faculty reported improvement in trainees' technical skills and NTS. Residents reported improvement in all 4 DCC, particularly, Practice-Based Learning and Improvement, Professionalism, and Interpersonal and Communication Skills.

CONCLUSION

The apprenticeship rotation is an effective means of teaching residents both NTS and DCC essential for independent practice. Consideration should be given to introducing this program into surgical curricula nationally.

Simulation Improves Nontechnical Skills Performance of Residents During the Perioperative and Intraoperative Phases of Surgery

OBJECTIVE

Failures in nontechnical skills (NTS) rather than technical expertise are frequently at the root of medical errors in the operating room (OR). NTS are the cognitive (decision making and situation awareness) and interpersonal (communication and teamwork) skills that are recognized but are not formally addressed in surgical training. The purpose of the study was to examine the effect of simulation-based training (SBT) on NTS performance of surgical residents during simulated laparoscopic cholecystectomy (LC).

SETTING

The study was performed in a simulated OR at the Center for Medical Education and Innovation at Riverside Methodist Hospital, Columbus, OH. The simulated OR was arranged with standard equipment for LC, a high-fidelity patient simulator, and a real OR team.

DESIGN

General surgical residents completed 2 identical SBT sessions. For each session, residents were briefed on the LC case, completed the case in the simulated OR, and debriefed their videotaped simulation performance with a content expert. The video recordings were reviewed and the residents' NTS were scored using a perioperative time-out checklist and an intraoperative checklist for LC by 4 raters who were blinded to both the residents' postgraduate year level and the order of the videotaped simulation sessions.

RESULTS

Residents showed a significant improvement in completeness of the perioperative time-out checklist from session 1 (mean score = 1.27 ± 1.00) to session 2 (mean score = 5.00 ± 1.28), p < 0.001. Residents' scores on the intraoperative checklist also improved from session 1 to session 2, p < 0.05. Overall, residents felt that the simulation was a valuable teaching and training tool and recommend that it be incorporated into residency training.

CONCLUSION

SBT appears to be an effective technique for improving NTS of surgical residents during the perioperative and intraoperative phases of surgery. As surgical proficiency is 75% nontechnical and 25% technical, it could be reasonably argued that improved NTS of surgeons could improve surgical outcomes.

On orthopedic surgical skill prediction--the limited value of traditional testing

OBJECTIVES

Primary: to assess the utility of our distal radius fracture repair model as a tool for examining residents' surgical skills. Secondary: to compare the residents' ability to achieve specific biomechanically measured fracture stability with traditional test scores.

DESIGN

Our laboratory pioneered a model that measures biomechanical qualities of a repaired distal radius fracture. Before participation, all residents to be tested completed specified knowledge examinations. During the laboratory exercise, proctors observed each resident and completed Objective Structured Assessment of Technical Skills forms. At the completion of the laboratory, each specimen was tested biomechanically. Written examinations were completed in a proctored setting and computer examinations at home following the honor system. The laboratory exercise had adequate space and materials and allowed 60 minutes to complete the procedure. Residents had equal access to x-ray imaging.

SETTING

The examination environment of the study resembled an operating room.

PARTICIPANTS

Postgraduate years 3 and 4 orthopedic residents in our program were asked to participate. The institutional review board reviewed and approved the study as exempt.

RESULTS

Fracture repair constructs capable of resisting loads expected during rehabilitation were created by approximately half the residents tested. However, traditional written and computer-based testing methods failed to predict which resident's fracture construct would pass the biomechanical testing. Prior in vivo similar case experience was not predictive.

CONCLUSIONS

The idea that "book smart does not equal street smart" applies to the tested model. To measure surgical skill acquisition and increase public safety related to surgery, it will be necessary to employ new and specific examination methods that identify the skill to be acquired and test the acquisition of this skill as precisely as possible.

What can we learn from narratives in medical education?

Medical literature has demonstrated the effectiveness of narrative writing in enhancing self-reflection and empathy, which opens the door for deeper understanding of patients' experiences of illness. Similarly, it promotes practitioner well-being. Therefore, it is no surprise that narrative writing finds a new home in medical education. The Accreditation Council of Graduate Medical Education (ACGME), through its Outcome Project, established six core competencies that every residency program must teach. However, no specific pedagogies were suggested. We explored the role that narrative writing can play in reconciling the ACGME core competencies with daily encounters in medical education. Our study suggests a hidden wealth in reflective writing through narratives with a promising potential for application in medical education. Reflective writing may turn out to be an innovative tool for teaching and evaluating ACGME core competencies.

Guided outcomes in learned efficiency model in clinical medical education: a randomized controlled trial of self-regulated learning

OBJECTIVE

The guided outcomes in learned efficiency (GOLE) model emphasizes the use of evidence-based resources to understand the diagnosis, treatment, follow-up, and prevention of disease. We seek to determine whether presentations created using the GOLE model are superior to an unstructured approach in achieving Accreditation Council for Graduate Medical Education (ACGME) Core Competencies.

STUDY DESIGN

Consenting medical students were randomized to GOLE or control groups to individually research a self-selected clinical topic. A validated survey instrument was used prepresentation and postpresentation to assess perceived improvement in knowledge. Subjects completed self-evaluations at enrollment and after presentation of their chosen clinical topic. Other students, residents, and a faculty member also completed evaluations after each student presentation. Standard statistical methods (analysis of variance, 2-tailed t test) were used to determine if a statistically significant difference existed between intervention and control groups.

RESULTS

Self-assessments were similar in the GOLE and control groups. Externally perceived presentation scores were greater in the GOLE group (ACGME global P < .0001, presentation global P = .07), which demonstrated a significant improvement in 5 core competencies. Time spent preparing the presentation and resources utilized did not differ between groups.

CONCLUSION

The presentations prepared using the GOLE model were rated higher by observers than those prepared using traditional techniques.

War stories: a qualitative analysis of narrative teaching strategies in the operating room

BACKGROUND

"War stories" are commonplace in surgical education, yet little is known about their purpose, construct, or use in the education of trainees.

METHODS

Ten complex operations were videotaped and audiotaped. Narrative stories were analyzed using grounded theory to identify emergent themes in both the types of stories being told and the teaching objectives they illustrated.

RESULTS

Twenty-four stories were identified in 9 of the 10 cases (mean, 2.4/case). They were brief (mean, 58 seconds), illustrative of multiple teaching points (mean, 1.5/story), and appeared throughout the operations. Anchored in personal experience, these stories taught both clinical (eg, operative technique, decision making, error identification) and programmatic (eg, resource management, professionalism) topics.

CONCLUSIONS

Narrative stories are used frequently and intuitively by physicians to emphasize a variety of intraoperative teaching points. They socialize trainees in the culture of surgery and may represent an underrecognized approach to teaching the core competencies. More understanding is needed to maximize their potential.

Observational tools for assessment of procedural skills: a systematic review

BACKGROUND

Assessment by direct observation of procedural skills is an important source of constructive feedback. The aim of this study was to identify observational tools for technical skill assessment, to assess characteristics of these tools, and to assess their usefulness for assessment.

METHODS

Included studies reported tools for observational assessment of technical skills. A total of 106 articles were included.

RESULTS

Three main categories included global assessment scales evaluating generic skills (n = 29), task-specific methods assessing procedure-specific skills (n = 30), and combinations of tools evaluating both generic and task-specific skills (n = 47). In most studies, content validity was not evaluated using an accepted scientific method. All tools were assessed for inter-rater reliability and construct validity. Data on feasibility, acceptability, and educational impact were sparse.

CONCLUSIONS

There is evidence of validity and reliability for observational assessment tools at the trainee level. In most studies a comprehensive analysis of the tools was not achieved. Evaluation of technical skill using current observational assessment tools is not reliable and valid at the specialist level. Future research needs to focus on further systematic tool development and analysis, especially at the specialist level.

Measurement of the general competencies of the accreditation council for graduate medical education: a systematic review

PURPOSE

To evaluate published evidence that the Accreditation Council for Graduate Medical Education's six general competencies can each be measured in a valid and reliable way.

METHOD

In March 2008, the authors conducted searches of Medline and ERIC using combinations of search terms "ACGME," "Accreditation Council for Graduate Medical Education," "core competencies," "general competencies," and the specific competencies "systems-based practice" (SBP) and "practice based learning and improvement (PBLI)." Included were all publications presenting new qualitative or quantitative data about specific assessment modalities related to the general competencies since 1999; opinion pieces, review articles, and reports of consensus conferences were excluded. The search yielded 127 articles, of which 56 met inclusion criteria. Articles were subdivided into four categories: (1) quantitative/psychometric evaluations, (2) preliminary studies, (3) studies of SBP and PBLI, and (4) surveys.

RESULTS

Quantitative/psychometric studies of evaluation tools failed to develop measures reflecting the six competencies in a reliable or valid way. Few preliminary studies led to published quantitative data regarding reliability or validity. Only two published surveys met quality criteria. Studies of SBP and PBLI generally operationalized these competencies as properties of systems, not of individual trainees.

CONCLUSIONS

The peer-reviewed literature provides no evidence that current measurement tools can assess the competencies independently of one another. Because further efforts are unlikely to be successful, the authors recommend using the competencies to guide and coordinate specific evaluation efforts, rather than attempting to develop instruments to measure the competencies directly.