Reflections on competency-based education and training for surgical residents

Practical Guide to Assessment Tool Development for Surgical Education Research

This Guide to Statistics and Methods describes the process of validation and gathering validity evidence for assessment tool development for surgical education research.

Lessons learned spanning 17 years of experience with three consecutive nationwide competency based medical education training plans

Introduction

Curricula for postgraduate medical education have transformed since the introduction of competency based medical education (CBME). Postgraduate training plans offer broader training with different competencies and an outcome-based approach, in addition to the medical technical aspects of training. However, CBME also has its challenges. Over the past years, critical views have been shared on the potential drawbacks of CBME, such as assessment burden and conflicts with practicality in the workplace. Recent studies identified a need for a better understanding of how the evolving concept of CBME has been translated to curriculum design and implemented in the practice of postgraduate training. The aim of this study was to describe the development of CBME translations to curriculum design, based on three consecutive postgraduate training programs spanning 17 years.

Method

We performed a document analysis of three consecutive Dutch gynecology and obstetrics training plans that were implemented in 2005, 2013, and 2021. We used template analysis to identify changes over time.

Results

Over time, CBME-based curriculum design changed in several domains. Assessment changed from a model with a focus on summative decision to one with an emphasis on formative, low-stakes assessments aimed at supporting learning. The training plans evolved in parallel to evolving educational insights, e.g., by placing increasing emphasis on personal development. The curricula focused on a competency-based concept by introducing training modules and personalized authorization based on feedback rather than on a set duration of internships. There was increasing freedom in personalized training trajectories in the training plans, together with increasing trust towards the resident.

Conclusion

The way CBME was translated into training plans has evolved in the course of 17 years of experience with CMBE-based education. The main areas of change were the structure of the training plans, which became increasingly open, the degree to which learning outcomes were mandatory or not, and the way these outcomes were assessed.

Validation of a 3D-Printed Percutaneous Injection Laryngoplasty Simulator: A Randomized Controlled Trial

OBJECTIVE

Simulation may be a valuable tool in training laryngology office procedures on unsedated patients. However, no studies have examined whether existing awake procedure simulators improve trainee performance in laryngology. Our objective was to evaluate the transfer validity of a previously published 3D-printed laryngeal simulator in improving percutaneous injection laryngoplasty (PIL) competency compared with conventional educational materials with a single-blinded randomized controlled trial.

METHODS

Otolaryngology residents with fewer than 10 PIL procedures in their case logs were recruited. A pretraining survey was administered to participants to evaluate baseline procedure-specific knowledge and confidence. The participants underwent block randomization by postgraduate year to receive conventional educational materials either with or without additional training with a 3D-printed laryngeal simulator. Participants performed PIL on an anatomically distinct laryngeal model via trans-thyrohyoid and trans-cricothyroid approaches. Endoscopic and external performance recordings were de-identified and evaluated by two blinded laryngologists using an objective structured assessment of technical skill scale and PIL-specific checklist.

RESULTS

Twenty residents completed testing. Baseline characteristics demonstrate no significant differences in confidence level or PIL experience between groups. Senior residents receiving simulator training had significantly better respect for tissue during the trans-thyrohyoid approach compared with control (p < 0.0005). There were no significant differences in performance for junior residents.

CONCLUSIONS

In this first transfer validity study of a simulator for office awake procedure in laryngology, we found that a previously described low-cost, high-fidelity 3D-printed PIL simulator improved performance of PIL amongst senior otolaryngology residents, suggesting this accessible model may be a valuable educational adjunct for advanced trainees to practice PIL.

LEVEL OF EVIDENCE

NA Laryngoscope, 134:318-323, 2024.

Standard setting for orthopaedic trauma competencies in postgraduate specialty training- catching those falling behind the curve

BACKGROUND

The Curriculum for Trauma and Orthopaedics focuses on producing competent Day-One Consultants. However, the expected development trajectory is not clear. It is important, yet difficult to objectively identify trainees who are "falling behind". This project proposes practical, consensus-based thresholds of Operative Trauma Competence at each Waypoint stage of training.

METHODS

32 trainers and 73 trainees in one Deanery were identified. The trainers and trainees were asked their PBA level expectation of a trainee at ST4, ST6 and ST8 for nine trauma competencies. Lower quartile values were calculated providing thresholds.

RESULTS

53 (72%) trainees and 22 (69%) trainers responded. At ST8, the lower quartile threshold was level 4 for all procedures. At ST6, three operation groups became apparent: Group 1 (hip hemiarthroplasty, Dynamic hip screw; k-wire distal radius fracture and Weber C Ankle open reduction, internal fixation (ORIF)) Group 2 (Tibial Nail; Olecranon Tension band wire, ORIF radial shaft; distal radius plate fixation)- Group 3 (supracondylar fracture fixation)Threshold levels for procedures were: Group 1- 4a; Group 2-3b and Group 3- 3a.At ST4, there was more variation and spread in responses, however, expectations could still be similarly grouped: Group 1- 3a; Group 2- 2b and Group 3- 2a.

CONCLUSION

In an increasingly competency-based training environment we provide tangible thresholds for expectations of orthopaedic trainees' progression and development. We identified two groups: basic trauma (Group 1 where level 4 competencies should be attained by ST6) and intermediate trauma (Groups 2 and 3 where level 4 competencies should be attained by ST8.).

Predicting Resident Competence for Otolaryngology Key Indicator Procedures

OBJECTIVE

Competency-based surgical education requires practical assessments and meaningful benchmarks. In otolaryngology, key indicator procedure (KIP) minima are indicators of surgical exposure during training, yet it remains unknown how many times trainees must be evaluated on KIPs to ensure operative competence. Herein, we used Bayesian mixed effects models to compute predicted performance expectations for KIPs.

METHODS

From November 2017 to September 2021, a smartphone application (SIMPL OR) was used by attendings at five otolaryngology training programs to rate resident operative performance after each case on a five-level scale. Bayesian mixed effects models were used to estimate the probability that postgraduate year (PGY) 3, 4, or 5 trainees would earn a "practice-ready" (PR) rating on a subsequent evaluation based on their previously earned PR ratings for each KIP. Probabilities of earning a subsequent PR rating were examined for interpretability, and cross-validation was used to assess predictive validity.

RESULTS

A total of 842 assessments of KIPs were submitted by 72 attendings for 92 residents PGY 2-5. The predictive model had an average Area Under the Receiver Operating Curve of 0.77. The number of prior PR ratings that senior residents needed to attain a 95% probability of earning a PR rating on a subsequent evaluation was estimated for each KIP. For example, for mastoidectomies, PGY4 residents needed to earn 10 PR ratings whereas PGY5 residents needed 4 PR ratings on average to have a 95% probability of attaining a PR rating on a subsequent evaluation.

CONCLUSION

Predictive modeling can inform assessment benchmarks for competency-based surgical education.

LEVEL OF EVIDENCE

NA Laryngoscope, 133:3341-3345, 2023.

A Novel Low-Cost, Open-Source, Three-Dimensionally Printed Thyroplasty Simulator

OBJECTIVE

Training of surgical procedures on awake patients, such as medialization thyroplasty, poses challenges to educators and trainees. Three-dimensionally (3D)-printed simulators provide opportunity to practice in low-stakes settings. We present the first 3D-printed thyroplasty simulator incorporating a cartilaginous framework, endolaryngeal soft tissue housed in a 3D-printed manikin with endoscopic visualization.

METHODS

Male and female laryngeal cartilages and endolarynx molds were 3D printed from an existing open-source design. Cartilage models were made of heat-treated polylactic acid (HTPLA), a material chosen for its thermal stability, allowing drilling. They were combined with molded silicone endolarynges modeling glottic insufficiency. Larynges were set in a 3D-printed head-and-neck manikin with an attached borescope for internal visualization similar to distal chip laryngoscopy. Eight laryngologists evaluated the simulator by drilling a thyroplasty window, inserting an implant for medialization, and rating the model using a modified Michigan Standard Simulation Experience Scale (1 = strongly disagree, 5 = strongly agree).

RESULTS

The model was well rated in educational value (mean 4.7, standard deviation [SD] 0.3), fidelity (mean 3.8, SD 0.2), and overall value (mean 4.8, SD 0.5). Qualitative assessment concluded the model was anatomically realistic and that HTPLA was a good approximation of the density and texture of thyroid cartilage. The materials for one larynx cost $4.09.

CONCLUSION

This high-fidelity 3D-printed simulator demonstrates educational value for thyroplasty training. The low-cost, open-source design has broad implications for universal access to this simulator platform.

The Beginner Laparoscopists Trends in the Learning Process of Laparoscopy for Adnexal Gynecological Pathologies-The Experience of Our Center

BACKGROUND

Laparoscopy for benign ovarian pathology is the appropriate surgical approach and it has many well-known advantages. Minimal invasive gynecological surgery increases the quality of life of the patient. The learning process of laparoscopy is difficult and requires many interventions to acquire manual skills. The objectives of the study were to assess the learning process of laparoscopy for adnexal pathology surgery performed by beginner laparoscopists.

MATERIALS AND METHODS

This study included three gynecological surgeons who were beginners in laparoscopy and who were named A, B, and C. We collected information about patients, diagnosis, surgical technique, and complications.

RESULTS

We have analyzed the data from 159 patients. The most frequent primary diagnosis was functional ovarian cyst, and the laparoscopic cystectomy was performed in 49.1% of interventions. The need to convert a laparoscopy into laparotomy was necessary in 1.3% of patients. There were no cases of reintervention, blood transfusion, or ureteral lesions. The duration of the surgical intervention varied statistically significantly according to patient's BMI and to the surgeon. After 20 laparoscopic interventions, a significant improvement was found in the time needed to perform ovarian cystectomy (operators A and B) and salpingectomy (operator C).

CONCLUSIONS

The process of learning laparoscopy is laborious and difficult. We found a significant decrease in operating time after a twenty laparoscopic interventions.

The Changing Environment in Postgraduate Education in Orthopedic Surgery and Neurosurgery and Its Impact on Technology-Driven Targeted Interventional and Surgical Pain Management: Perspectives from Europe, Latin America, Asia, and The United States

Personalized care models are dominating modern medicine. These models are rooted in teaching future physicians the skill set to keep up with innovation. In orthopedic surgery and neurosurgery, education is increasingly influenced by augmented reality, simulation, navigation, robotics, and in some cases, artificial intelligence. The postpandemic learning environment has also changed, emphasizing online learning and skill- and competency-based teaching models incorporating clinical and bench-top research. Attempts to improve work-life balance and minimize physician burnout have led to work-hour restrictions in postgraduate training programs. These restrictions have made it particularly challenging for orthopedic and neurosurgery residents to acquire the knowledge and skill set to meet the requirements for certification. The fast-paced flow of information and the rapid implementation of innovation require higher efficiencies in the modern postgraduate training environment. However, what is taught typically lags several years behind. Examples include minimally invasive tissue-sparing techniques through tubular small-bladed retractor systems, robotic and navigation, endoscopic, patient-specific implants made possible by advances in imaging technology and 3D printing, and regenerative strategies. Currently, the traditional roles of mentee and mentor are being redefined. The future orthopedic surgeons and neurosurgeons involved in personalized surgical pain management will need to be versed in several disciplines ranging from bioengineering, basic research, computer, social and health sciences, clinical study, trial design, public health policy development, and economic accountability. Solutions to the fast-paced innovation cycle in orthopedic surgery and neurosurgery include adaptive learning skills to seize opportunities for innovation with execution and implementation by facilitating translational research and clinical program development across traditional boundaries between clinical and nonclinical specialties. Preparing the future generation of surgeons to have the aptitude to keep up with the rapid technological advances is challenging for postgraduate residency programs and accreditation agencies. However, implementing clinical protocol change when the entrepreneur-investigator surgeon substantiates it with high-grade clinical evidence is at the heart of personalized surgical pain management.

Cognitive Load Management: An Invaluable Tool for Safe and Effective Surgical Training

This article highlights the importance of considering Cognitive Load (CL) and Cognitive Load Theory (CLT) during surgical training, focusing on the acquisition of intra-operative skills. It describes the basis of CLT with the overarching aim of describing CLT-based techniques to enhance current training strategies and surgical performance, many of which are instinctively already employed in surgical practice. Currently, methods of feedback and assessment are imperfect - typically subjective, unsystematic, opportunistic, or retrospective, and at risk of human bias. Surgical Sabermetrics, the advanced analytics of surgical and audio-visual data, aims to enhance this feedback by providing objective, real-time, digital-based feedback. This article introduces the benefit of real-time measurement of CL to enhance feedback and its applications to surgical performance that follow the ethos of Surgical Sabermetrics.¹ The 2022 theme for ICOSET was "Making it Better." Cognitive Load and Surgical Sabermetrics principles provide tools to make Surgical training better, with the goal of higher quality care for patients.

Video-based assessment (VBA) of an open, simulated orthopedic surgical procedure: a pilot study using a single-angle camera to assess surgical skill and decision making

BACKGROUND

Videos have been used in many settings including medical simulation. Limited information currently exists on video-based assessment in surgical training. Effective assessment tools have substantial impact on the future of training. The objectives of this study were as follows: to evaluate the inter-rater reliability of video-based assessment of orthopedic surgery residents performing open cadaveric simulation procedures and to explore the benefits and limitations of video-based assessment.

METHODS

A multi-method technique was used. In the quantitative portion, four residents participated in a Surgical Objective Structured Clinical Examination in 2017 at a quaternary care training center. A single camera bird's-eye view was used to videotape the procedures. Five orthopedic surgeons evaluated the surgical videos using the Ottawa Surgical Competency Operating Room Evaluation. Interclass correlation coefficient was used to calculate inter-rater reliability. In the qualitative section, semi-structured interviews were used to explore the perceived strengths and limitations of video-based assessment.

RESULTS AND DISCUSSION

The scores using video-based assessment demonstrated good inter-rater reliability (ICC = 0.832, p = 0.014) in assessing open orthopedic procedures on cadavers. Qualitatively, the strengths of video-based assessment in this study are its ability to assess global performance and/or specific skills, ability to reassess missed points during live assessment, and potential use for less common procedures. It also allows for detailed constructive feedback, flexible assessment time, anonymous assessment, multiple assessors and serves as a good coaching tool. The main limitations of video-based assessment are poor audio-video quality, and questionable feasibility for assessing readiness for practice.

CONCLUSION

Video-based assessment is a potential adjunct to live assessment in orthopedic open procedures with good inter-rater reliability. Improving audio-video quality will enhance the quality of the assessment and improve the effectiveness of using this tool in surgical training.

Surgical residents' approach to training: are elements of deliberate practice observed?

Background: Deliberate practice research has consistently shown that intense, concentrated, goal-oriented practice in a focused domain, such as medicine, can improve skill development and performance. To date, little is known about how surgical residents approach their surgical training, how they evaluate their current weaknesses, and how they plan to transition from one milestone to another. Without knowledge of residents' role in their development, educators miss the opportunity to optimize progression of these lifelong learning skills. Therefore, the purpose of this study was to gain a better understanding of how surgical residents approach their surgical training from the perspective of the surgical residents themselves and to explore if elements of deliberate practice are observed. Methods: Eight surgical trainees participated in one of two focus groups depending on their training level (five junior residents; three senior residents). With the exploratory nature of this research, a focus group methodology was utilized. Results: By employing both deductive and inductive thematic analysis techniques, three themes were extracted from the data: learning resources and strategies, role of a junior/senior, and approaching weaknesses. Conclusions: Although elements of deliberate practice were discussed, higher functioning is necessary to achieve performance excellence, leading to improved patient outcomes.

American College of Surgeons Objective Assessment of Skills in Surgery (ACS OASIS): A Formative Assessment of Junior Residents' Technical Skills

OBJECTIVE

The objective assessment of technical skills of junior residents is essential in implementing competency-based training and providing specific feedback regarding areas for improvement. An innovative assessment that can be easily implemented by training programs nationwide has been developed by expert surgeon educators under the aegis of the American College of Surgeons (ACS) Division of Education. This assessment, ACS Objective Assessment of Skills in Surgery (ACS OASIS) uses eight stations to address technical skills important for junior residents within the domains of laparoscopic appendectomy, excision of lipoma, central line placement, laparoscopic cholecystectomy, trocar placement, exploratory laparotomy, repair of enterotomy, and tube thoracostomy. The purpose of this study was to implement ACS OASIS at a number of sites to study its psychometric rigor.

DESIGN

The ACS OASIS was pre-piloted at two programs to establish feasibility and to gather information regarding implementation. Each skills station was 12 minutes long, and the faculty completed a checklist with 5 to 15 items, and a global assessment scale. The study was then repeated at three pilot sites and included 29 junior residents who were assessed by a total of 44 faculty. Psychometric data for the stations and checklists were collected and analyzed.

SETTING

The pre-pilot sites were Geisinger and University of Tennessee Knoxville.Data were gathered from pilot sites that included Wellspan Health, Duke University, and University of California Los Angeles.

RESULTS

The mean checklist score for all learners was 76% (IQR of 66%-85%). The average global rating was 3.36 on a 5-point scale with a standard deviation of 0.56. The overall cut score derived using the borderline group method was at 68% with 34% of performances requiring remediation. Using this criterion, the average number of stations that were completed by each learner without need for remediation was five.The station discrimination index ranged from 0.27 to 0.65 (all above the threshold of 0.25), demonstrating solid psychometric characteristics at the station level. The internal-consistency reliability was 0.76 with SEM of 5.8%. The inter-rater reliability (intraclass correlation) was high at 0.73 with general agreement of 79% between the two raters. The station discrimination was at 0.45 (range of 0.27 to 0.65) indicating a high level of differentiation between high and low performers. Using the generalizability theory, the G-coefficient reliability was at 0.72 with the reliability projection flattening after 8 stations. Overall, 75% to 82% the faculty and learners rated ACS OASIS as realistic and beneficial.

CONCLUSIONS

ACS OASIS is a psychometrically sound technical skills assessment tool that can provide useful information for feedback to junior residents and support efforts to remediate gaps in performance.

Surgical residents’ approach to training: are elements of deliberate practice observed?

Background: Research in the area of deliberate practice has consistently shown that intense, concentrated, goal-oriented practice in a focused domain, such as medicine, can improve both skill development and performance to attain a progressively higher standard of excellence. In theory, utilizing deliberate practice in a medical context could result in improved surgical training and in turn better patient outcomes. Therefore, the purpose of this study was to gain a better understanding of how surgical residents approach their training from the perspective of the surgical residents themselves and to explore if elements of deliberate practice are observed. Methods: Eight surgical trainees participated in one of two focus groups depending on their training level (five junior residents; three senior residents). With the exploratory nature of this research, a focus group methodology was utilized. Results: By employing both deductive and inductive thematic analysis techniques, three themes were extracted from the data: learning resources and strategies, role of a junior/senior, and approaching weaknesses. Conclusions: Although elements of deliberate practice were discussed, higher functioning is necessary to achieve performance excellence, leading to improved patient outcomes.

Reviewing ACGME Plastic Surgery Fellowship Case Logs: Is Surgical Experience Increasing?

INTRODUCTION

Over the last decade, there has been a 32% decrease in independent plastic surgery fellowships. The growing prevalence of 6-year integrated plastic surgery residencies, duty hour restrictions, and new subspecialty training fellowships for general surgeons have changed the training experience of plastic surgery fellows.

METHODS

A retrospective review of the Accreditation Council for Graduate Medical Education (ACGME) case logs for graduating fellows of independent plastic surgery fellowships in the United States was conducted from 2011 to 2019. A linear regression analysis was conducted for each case log code and category, and a 95% level of confidence was assumed (α = 0.05).

RESULTS

In 2011, 141 residents from 69 programs graduated with an average of 1469.7 cases. In 2019, 84 residents from 47 programs graduated with an average of 1952 cases. Index procedures significantly increased overall during the 9 y (P < 0.001). Categorical cases increased in esthetics (P < 0.001), including facelift, browlift, blepharoplasty, and more. Categorical cases increased in reconstructive surgery (P < 0.001), including treatment of deformities of the skin, lower extremities, and trunk, nerve decompression, and hand reconstruction. In breast procedures, an increase was seen in the reduction of mammoplasty, reconstruction, and treatment of other breast deformities. In head and neck procedures, an increase was seen in resection of head and neck neoplasms and secondary cleft lip repair. Decreases in procedural numbers were seen in primary cleft lip repair and hand reconstruction by primary closure.

CONCLUSIONS

Despite a 32% decline in the number of independent plastic surgery fellowships over the last 9 y, plastic surgery fellows are obtaining significantly more surgical experience, both in esthetic and reconstructive surgery.

Multi-Modal Deep Learning for Assessing Surgeon Technical Skill

This paper introduces a new dataset of a surgical knot-tying task, and a multi-modal deep learning model that achieves comparable performance to expert human raters on this skill assessment task. Seventy-two surgical trainees and faculty were recruited for the knot-tying task, and were recorded using video, kinematic, and image data. Three expert human raters conducted the skills assessment using the Objective Structured Assessment of Technical Skill (OSATS) Global Rating Scale (GRS). We also designed and developed three deep learning models: a ResNet-based image model, a ResNet-LSTM kinematic model, and a multi-modal model leveraging the image and time-series kinematic data. All three models demonstrate performance comparable to the expert human raters on most GRS domains. The multi-modal model demonstrates the best overall performance, as measured using the mean squared error (MSE) and intraclass correlation coefficient (ICC). This work is significant since it demonstrates that multi-modal deep learning has the potential to replicate human raters on a challenging human-performed knot-tying task. The study demonstrates an algorithm with state-of-the-art performance in surgical skill assessment. As objective assessment of technical skill continues to be a growing, but resource-heavy, element of surgical education, this study is an important step towards automated surgical skill assessment, ultimately leading to reduced burden on training faculty and institutes.

Artificial Intelligence Methods and Artificial Intelligence-Enabled Metrics for Surgical Education: A Multidisciplinary Consensus

BACKGROUND

Artificial intelligence (AI) methods and AI-enabled metrics hold tremendous potential to advance surgical education. Our objective was to generate consensus guidance on specific needs for AI methods and AI-enabled metrics for surgical education.

STUDY DESIGN

The study included a systematic literature search, a virtual conference, and a 3-round Delphi survey of 40 representative multidisciplinary stakeholders with domain expertise selected through purposeful sampling. The accelerated Delphi process was completed within 10 days. The survey covered overall utility, anticipated future (10-year time horizon), and applications for surgical training, assessment, and feedback. Consensus was agreement among 80% or more respondents. We coded survey questions into 11 themes and descriptively analyzed the responses.

RESULTS

The respondents included surgeons (40%), engineers (15%), affiliates of industry (27.5%), professional societies (7.5%), regulatory agencies (7.5%), and a lawyer (2.5%). The survey included 155 questions; consensus was achieved on 136 (87.7%). The panel listed 6 deliverables each for AI-enhanced learning curve analytics and surgical skill assessment. For feedback, the panel identified 10 priority deliverables spanning 2-year (n = 2), 5-year (n = 4), and 10-year (n = 4) timeframes. Within 2 years, the panel expects development of methods to recognize anatomy in images of the surgical field and to provide surgeons with performance feedback immediately after an operation. The panel also identified 5 essential that should be included in operative performance reports for surgeons.

CONCLUSIONS

The Delphi panel consensus provides a specific, bold, and forward-looking roadmap for AI methods and AI-enabled metrics for surgical education.

Competency-Based Education and Practice in Physical Therapy: It's Time to Act!

Competency-based education (CBE) is a concept, a philosophy, and an approach to educational design where learner progression occurs when competency is demonstrated. It assumes a set of standard defined performance outcomes for any level of professional practice-students, residents, or practicing physical therapists. Those outcomes are based on the health needs of society and guide the curricular design, implementation, and evaluation of health professions education programs. Lack of a CBE framework-with no required demonstration of competence throughout one's career-has the potential to lead to variation in physical therapists' skills and to unwarranted variation in practice, potentially hindering delivery of the highest quality of patient care. CBE requires a framework that includes a commonly understood language; standardized, defined performance outcomes at various stages of learner development; and a process to assess whether competence has been demonstrated. The purpose of this perspective article is to (1) highlight the need for a shared language, (2) provide an overview of CBE and the impetus for the change, (3) propose a shift toward CBE in physical therapy, and (4) discuss the need for the profession to adopt a mindset requiring purposeful practice across one's career to safely and most efficiently practice in a given area. Utilizing a CBE philosophy throughout one's career should ensure high-quality and safe patient care to all-patient care that can adapt to the changing scope of physical therapist practice as well as the health care needs of society. The physical therapy profession is at a point at which we must step up the transition to a competency-based system of physical therapist education.

The Effect of Advanced Practice Providers on ACGME Colon and Rectal Surgery Resident Diagnostic Index Case Volumes

OBJECTIVE

Prior to 2015 residents in our Accreditation Council for Graduation Medical Education (ACGME) colon and rectal surgery training program were in charge of managing, with faculty oversight, the outpatient anorectal clinic at our institution. Starting in 2015 advanced practice providers (APPs) working in the division assumed management of the clinic. The effect of APPs on ACGME resident index diagnostic case volumes has not been explored. Herein we examine ACGME case log graduate statistics to determine if the inclusion of APPs into our anorectal clinic practice has negatively affected resident index diagnostic anorectal case volumes.

DESIGN

ACGME year-end program reports were obtained for the years 2011 to 2019. Program anorectal diagnostic index volumes were recorded and compared to division volumes. Analysis of variance (ANOVA) and analysis of covariance (ANCOVA) tests were conducted to assess whether the number of cases per year (for each respective case type) prior to the introduction of APPs into the anorectal clinic (2011-2014) differed from the number of cases per year with the APP clinic in place (2015-2018). A p-value <0.05 was considered statistically significant.

SETTING

Mayo Clinic, Rochester, Minnesota (quaternary referral center).

PARTICIPANTS

Colon and rectal surgery resident year-end ACGME reports (2011-2019).

RESULTS

ANOVAs revealed a marginally significant (p = 0.007) downtrend for hemorrhoid diagnostic codes, and a significant uptrend (p = 0.000) for fistula cases. Controlling for overall division volume, ANCOVA only reveled significance for fistula cases (p = 0.004) with the involvement of APPs.

CONCLUSIONS

At our institution we found the inclusion of APPs into our anorectal clinic practice did not negatively affect colon and rectal surgery resident ACGME index diagnostic anorectal case volumes. Inclusion of APPs into a multidisciplinary practice can promote resident education by allowing trainees to pursue other educational opportunities without hindering ACGME index case volumes.

Trigger videos: a novel application of a tool for surgical faculty development

BACKGROUND

Trigger videos have occasionally been used in medical education; however, their application to surgical faculty development is novel. We assessed participants' attitudes towards workshops on intraoperative teaching (IOT) that were anchored by trigger videos, and studied whether they could generate discussion-for-learning among surgeons in this workshop setting.

METHODS

Surgeons from multiple specialties attended one of six faculty development workshops where IOT trigger videos were shown and discussed during break-out sessions. Participants completed questionnaires to (1) evaluate videos via survey and feedback, and (2) identify adoptable and discardable IOT techniques. Teaching techniques were collated to identify planned IOT changes and survey data and feedback were analyzed.

RESULTS

A total of 135 surgeons identified 292 adoptable and 202 discardable IOT techniques based on trigger videos and discussions, and 94% of participants reported that the trigger videos were useful and encouraged them to discuss and consider new IOT techniques in their own practice.

CONCLUSIONS

Participants reported that the trigger videos were useful and motivating. Surgeons critically reflected on IOT during the sessions, identifying numerous adoptable and discardable techniques relevant to their own teaching styles. Trigger videos can be a valuable tool for surgical faculty development and can be tailored to other medical specialties.

Remediation of Underperformance in Surgical Trainees - A Scoping Review

OBJECTIVE

Surgical trainees with significant underperformance require formal support to return to an expected standard, termed remediation. The aim of this scoping review was to define remediation interventions, approaches, and contexts.

DESIGN

Following scoping review protocols, we set out to identify the evidence-base for remediation of surgical trainees, outline key concepts and uncover areas to stimulate further research.

RESULTS

From a screen of 80 articles, 24 reported on remediation of surgical trainees. Most were from medical journals (n = 21, 88%) and published in the United States (n = 20, 83%). Ten articles (41%) reported outcomes of remediation of a trainee cohort and 7 (19%) were survey reports from surgical directors. The remainder were a mix of commentaries, editorials or reviews. Thirteen articles (54%) described trainees with deficiencies in multiple competencies, 8 articles (33%) had a singular focus on academic performance, and 1 article (3%) on technical skills. All articles used targeted individualized remediation strategies, a range of intervention methods (some multimodal) and recommended a 6- to 12-month period of remediation (n = 7, 29%). The program director was often the only supervisor (n = 12, 50%). One article reported trainees' perspective of the process and one used educational theory to inform remediation.

CONCLUSIONS

Data with clearly reported outcomes were limited, but we found that targeted, individualized, multimodal and long-term remediation covering a range of competencies have been reported in the literature for surgical trainees. There is a need for development of explicit frameworks, strengthen the support for supervisors and trainees and further apply educational theory to develop better interventions that remediate deficiencies for all competencies.

Are Clerks Proficient in the Basic Sciences? Assessment of Third-Year Medical Students' Basic Science Knowledge Prior to and at the Completion of Core Clerkship Rotations

Basic sciences are a cornerstone of undergraduate medical education (UME), yet research indicates that students' basic science knowledge is not well retained. Many UME curricula are increasing the integration between the basic and clinical sciences with the goal of enhancing students' knowledge levels; however, the impact of clerkship training on students' basic science knowledge remains inconclusive. Thus, using clerkship directors' expectations as framework, we aimed to assess third-year medical students' basic science knowledge during clerkship training and evaluate the influence of clerkship training on their basic science knowledge. Using concepts deemed necessary by clerkship directors, we created a basic science assessment for each clerkship rotation. Assessments were distributed to third-year medical students as a pre- and post-test to assess their basic science knowledge prior to and at the completion of each rotation. On average, students retained ≥ 60% of relevant basic science knowledge from pre-clerkship, and neither clerkship rotation order, nor the basic science discipline being assessed, impacted students' basic science knowledge levels. Post-test data revealed that students, on average, reinforced fundamental concepts during clerkship. Interestingly, even though lower-performing students demonstrated the greatest post-test improvement, they still left each rotation with knowledge deficits compared with their highest-performing peers, suggesting that the clinical experience of clerkship appears to be particularly beneficial for lower-performing students, in regard to enhancing their basic science knowledge. Overall, results indicate that earlier exposure to clinical learning in UME, along with integration of basic science education into clerkship, could promote students' basic science knowledge acquisition and retention.

A Framework for Understanding the Association Between Training Paradigm and Trainee Preparedness for Independent Surgical Practice

Importance

The sociopolitical and cultural context of graduate surgical education has changed considerably over the past 2 decades. Although new structures of graduate surgical training programs have been developed in response and the comparative value of formats are continually debated, it remains unclear how different time-based structural paradigms are preparing trainees for independent practice after program completion.

Objective

To investigate the factors associated with trainees' and program directors' perception of trainee preparedness for independent surgical practice.

Design, Setting, and Participants

This qualitative study used an instrumental case study approach and obtained information through semistructured interviews, which were analyzed using open-and-focused coding. Participants were recent graduates and program directors of vascular surgery training programs in the United States. The 2 training paradigms analyzed were the integrated vascular surgery residency program (0 + 5, with 0 indicating that the general surgery training experiences are fully integrated into the 5 years of overall training and 5 indicating the total number of years of training) and the traditional vascular surgery fellowship program (5 + 2, with 5 indicating the number of years of general surgery training and 2 indicating the number of years of vascular surgery training). All graduates completed their training in 2018. All interviews were conducted between July 1, 2018, and September 30, 2018.

Main Outcomes and Measures

A conceptual framework to inform current and ongoing efforts to optimize graduate surgical training programs across specialties.

Results

A total of 22 semistructured interviews were completed, involving 7 graduates of 5 + 2 programs, 9 graduates of 0 + 5 programs, and 6 vascular surgery program directors. Of the 22 participants, 15 were men (68%). Participants described 4 interconnected domains that were associated with trainees' perceived preparedness for practice: structural, individual, relational, and organizational. Structural factors included the overall and vascular surgery-specific time spent in training, whereas individual factors included innate technical skills, confidence, maturity, and motivation. Faculty-trainee relationships (or relational factors) were deemed important for building trust and granting of autonomy. Organizational factors included features of the local organization, including patient population, case volume, and case mix.

Conclusions and Relevance

Findings suggest that restructuring training paradigms alone is insufficient to address the issue of trainees' perceived preparedness for practice. A framework was created from the results for evaluating and improving residency and fellowship programs as well as for developing graduate surgical training paradigms that incorporate all 4 domains associated with preparedness.

Strengthening breast surgery workforce capacity: implementation of competency-based training programme

Background and rationale

Quality education is a prerequisite for building a sustainable health system. To address this requirement, it is necessary to strengthen capacity and expand the training opportunities to ensure equitable and efficient development of core professional competencies for specific contexts and educational needs.

Methods and results

A competency-based training programme for Breast Surgeons was built and was applied based on the Consolidated Framework for Implementation Research (CFIR). This framework provides a pragmatic structure for approaching complex interactions, multi-level and transient constructs in the real world. CFIR guided the implementation process and verified what works, where and why across each step. CFIR guided implementation was through an adaptable approach of the domains and creating relevant constructs that set up an ideal roadmap to analyse and improve learning needs, the curriculum design and the learning environment.

Conclusion

The outcomes described in this manuscript demonstrate that evidence-based principles can be implemented in health professionals’ training and clinical practice even in resource-constrained settings. Building strong and sustainable healthcare workforce capacity is an urgent need for improved health service delivery and addresses real-life workplace needs in low-middle income countries. This programme integrates training with service to solve problems and develop initiatives to address existing local health priorities. While the article focuses on a training programme development, findings are shared to promote dissemination into other settings.

Assumptions About Competency-Based Medical Education and the State of the Underlying Evidence: A Critical Narrative Review

PURPOSE

As educators have implemented competency-based medical education (CBME) as a framework for training and assessment, they have made decisions based on available evidence and on the medical education community's assumptions about CBME. This critical narrative review aimed to collect, synthesize, and judge the existing evidence underpinning assumptions the community has made about CBME.

METHOD

The authors searched Ovid MEDLINE to identify empirical studies published January 2000 to February 2019 reporting on competence, competency, and CBME. The knowledge synthesis focused on "core" assumptions about CBME, selected via a survey of stakeholders who judged 31 previously identified assumptions. The authors judged, independently and in pairs, whether evidence from included studies supported, did not support, or was mixed related to each of the core assumptions. Assumptions were also analyzed to categorize their shared or contrasting purposes and foci.

RESULTS

From 8,086 unique articles, the authors reviewed 709 full-text articles and included 189 studies reporting evidence related to 15 core assumptions. Most studies (80%; n = 152) used a quantitative design. Many focused on procedural skills (48%; n = 90) and assessed behavior in clinical settings (37%; n = 69). On aggregate, the studies produced a mixed evidence base, reporting 362 data points related to the core assumptions (169 supportive, 138 not supportive, and 55 mixed). The 31 assumptions were organized into 3 categories: aspirations, conceptualizations, and assessment practices.

CONCLUSIONS

The reviewed evidence base is significant but mixed, with limited diversity in research designs and the types of competencies studied. This review pinpoints tensions to resolve (where evidence is mixed) and research questions to ask (where evidence is absent). The findings will help the community make explicit its assumptions about CBME, consider the value of those assumptions, and generate timely research questions to produce evidence about how and why CBME functions (or not).

Does staged surgical training for minimally invasive esophagectomy have an impact on short-term outcomes?

BACKGROUND

sophageal cancer has a low incidence, and the anatomy is difficult to understand during esophagectomy. This necessitates a precise and lengthy operation. Therefore, the establishment of a training system in esophageal surgery is of critical importance. In this study, we compared the short-term outcomes of minimally invasive esophagectomy (MIE) performed by consultants versus trainees and explored the factors that impacted the thoracic operation time for each group.

METHODS

We have introduced standardized MIE surgical techniques to our trainees in 2016. Our procedure consists of a laparoscopic phase and a thoracoscopic phase and is systematically designed to be learned in a step-by-step manner in each phase. We retrospectively identified 308 patients who underwent MIE from April 2016 to April 2018. The patients were divided into those who underwent MIE by consultants and those who underwent MIE by trainees. The preoperative background factors, operation-related factors, and postoperative complications were compared between the two groups. We also assessed the association between a prolonged thoracic operation time and tumor-and patient-related factors in each of the consults and trainees.

RESULTS

Significantly more patients had stage ≥ III cancer in the consultant than trainee group. However, the postoperative complications were comparable, specifically pneumonia (11% vs. 18%), anastomotic leakage (11% vs. 13%), and mortality (0.6% vs. 1.3%). There was no significant difference in the lymph node yield (20 vs. 17) or R0 resection rate (94% vs. 91%) between the two groups. However, the trainees had a significantly longer thoracic operation time (143 ± 34 vs. 190 ± 28 min) and significantly greater blood loss (93 vs. 183 ml). Oncological factors were correlated with a prolonged thoracic operation time in the consultants, but not in the trainees.

CONCLUSIONS

Under standardized surgical management using a stepwise educational program, performance of MIE by trainees has no impact on short-term outcomes.

Surgical Competencies Required in Newly Commencing Colorectal Surgeons: an Educational and Training Spectrum

PURPOSE

Surgical training models have changed from master-apprentice to competency-based training. We aimed to determine the relative importance and peak periods of acquiring these competencies in newly commencing colorectal surgeons.

METHODS

A mailed questionnaire to all current Colorectal Surgical Society of Australia and New Zealand (CSSANZ) members was conducted between October and December 2016 assessing the relative importance of each competency and the period or activity of learning or training contributing most to achieving that competency.

RESULTS

The response rate was 43% (90/208) with 87% (n = 75) agreed or strongly agreed to the relevance and applicability of the nine RACS competencies. Competencies varied in perceived importance (strongly agreed: judgment-clinical decision-making (JU) 63%, collaboration/teamwork (CT) 53%, technical expertise (TE) 47%, communication (CO) 44%, medical expertise (ME) 34%, scholarship/teaching (ST) 33%, professionalism (PR) 33%/ethics (ET) 24%, health advocacy (HA) 18%, management (MX) 13%/leadership (LE) 17%), and the peak period for acquiring them (registrar: CO 39%, ST 30%; fellow: TE 62%, CT 44%, ME 40%, JU 38%; consultant: MX/LE 52%, HA 48%, PR/ET 33%).

CONCLUSION

Surgical competencies for colorectal surgeons are accumulated and acquired at varying degrees and periods across a spectrum of continuing registrar, fellow, and consultant education and training. These findings serve as a baseline for further refinement of current and continuing educational and training programs.

An Open-Source Three-Dimensionally Printed Laryngeal Model for Injection Laryngoplasty Training

OBJECTIVES/HYPOTHESIS

A limited number of three-dimensionally (3D)-printed laryngeal simulators have been described in the literature, only one of which is specifically designed for percutaneous injection laryngoplasty (PIL) training and is currently of limited availability. This study describes the development and evaluation of a high-fidelity, open-source, low-cost 3D-printed simulator for PIL training, improving on existing models.

STUDY DESIGN

Simulator design and survey evaluation.

METHODS

Computed tomography scans of the upper airways were processed with 3D Slicer to generate a computer model of the endolarynx. Blender and Fusion 360 were used to refine the mucosal model and develop casts for silicone injection molding. The casted endolaryngeal structures were inserted into a modified version of a publicly available laryngeal cartilage model. The final models were evaluated by 10 expert laryngologists using a customized version of the Michigan Standard Simulation Experience Scale. Internal consistency and interrater reliability of the survey were evaluated using Cronbach's α and intraclass correlation, respectively.

RESULTS

Expert laryngologists highly rated the model for measures of fidelity, educational value, and overall quality (mean = 4.8, standard deviation = 0.5; 1 = strongly disagree, 5 = strongly agree). All reviewers rated the model as ready for use as is or with slight modifications. The filament needed for one cartilage model costs $0.96, whereas the silicone needed for one soft-tissue model costs $1.89.

CONCLUSIONS

Using 3D-printing technology, we successfully created the first open-source, low-cost, and anatomically accurate laryngeal model for injection laryngoplasty training. Our simulator is made freely available for download on Wikifactory with step-by-step tutorials for 3D printing, silicone molding, assembly, and use.

LEVEL OF EVIDENCE

NA Laryngoscope, 131:E890-E895, 2021.

Heterogeneity in urology teaching curricula among Canadian urology residency programs

INTRODUCTION

Postgraduate education is transitioning to a competency-based curriculum in an effort to standardize the quality of graduating trainees. The learning experiences and opportunities in each institution are likely variable, as no standard exists regarding the teaching curriculum offered through residency. The objective of this study is to examine the various teaching curricula among different Canadian urology residency programs and to identify which teaching modalities are prioritized by program directors.

METHODS

A 10-question anonymous survey was sent electronically to program directors at all 12 urology residency programs across Canada. Questions were designed to quantify the time allotted for teaching and to assess the various teaching session types prioritized by programs to ensure the successful training of their graduates. We assessed each program's perceived value of written exams, oral exams, didactic teaching session, and simulation sessions. Responses were assessed using a Likert-scale and a ranking format. Descriptive statistics were performed.

RESULTS

Overall survey response rate from residency program directors was 75% (9/12). Sixty-seven percent of programs designated one day of teaching per week, whereas 33% split resident teaching over two days. Review of chapters directly from Campbell-Walsh Urology textbook were deemed the most valuable teaching session. Practice oral exams were also prioritized, whereas most programs felt that simulation labs contributed the least to residency education. All programs included review of the core urology textbook in their weekly teaching, while only 67% of programs included faculty-led didactic sessions and case presentations. Forty-four percent of programs included resident-led didactic sessions. Practice oral exams and simulation labs were the least commonly included teaching modalities.

CONCLUSIONS

Although most program directors prioritize the review of chapters in the core urology textbook, we found significant heterogeneity in the teaching sessions prioritized and offered in current urology residency curricula. As we move to standardize the quality of graduating trainees, understanding the impact of variable educational opportunities on residency training may become increasingly important.

Three-dimensional printing in orthopaedic surgery: a scoping review

Three-dimensional printing (3DP) has become more frequently used in surgical specialties in recent years. These uses include pre-operative planning, patient-specific instrumentation (PSI), and patient-specific implant production.The purpose of this review was to understand the current uses of 3DP in orthopaedic surgery, the geographical and temporal trends of its use, and its impact on peri-operative outcomesOne-hundred and eight studies (N = 2328) were included, published between 2012 and 2018, with over half based in China.The most commonly used material was titanium.Three-dimensional printing was most commonly reported in trauma (N = 41) and oncology (N = 22). Pre-operative planning was the most common use of 3DP (N = 63), followed by final implants (N = 32) and PSI (N = 22).Take-home message: Overall, 3DP is becoming more common in orthopaedic surgery, with wide range of uses, particularly in complex cases. 3DP may also confer some important peri-operative benefits. Cite this article: EFORT Open Rev 2020;5:430-441. DOI: 10.1302/2058-5241.5.190024.

Changing Autonomy in Operative Experience Through UK General Surgery Training: A National Cohort Study

Supplemental Digital Content is available in the text

Objectives:

To determine the operative experience of UK general surgery trainees and assess the changing procedural supervision and acquisition of competency assessments through the course of training.

Background:

Competency assessment is changing with concepts of trainee autonomy decisions (termed entrustment decisions) being introduced to surgical training.

Methods:

Data from the Intercollegiate Surgical Curriculum Programme and the eLogbook databases for all UK General Surgery trainees registered from August 1, 2007 who had completed training were used. Total and index procedures (IP) were counted and variation by year of training assessed. Recorded supervision codes and competency assessment outcomes for IPs were assessed by year of training.

Results:

We identified 311 trainees with complete data. Appendicectomy was the most frequently undertaken IP during first year of training [mean procedures (mp) = 26] and emergency laparotomy during final year of training (mp = 27). The proportion of all IPs recorded as unsupervised increased through training (P < 0.05) and varied between IPs with 91.2% of appendicectomies (mp = 20), 40.6% of emergency laparotomies (mp = 27), and 17.4% of segmental colectomies (mp = 15) recorded as unsupervised during the final year of training. Acquisition of competency assessments increased through training and varied by IP.

Conclusions:

The changing autonomy of trainees through the course of an entire training scheme, alongside formal competency assessments, may provide evidence of changing entrustment decisions made by trainers for different key procedures. Other countries utilizing electronic logbooks could adopt similar techniques to further understanding of competency attainment amongst their surgical trainees.

Trainee performance in radical gastrectomy and its effect on outcomes

Background

This study aimed to determine whether trainee involvement in D2 gastrectomies was associated with adverse outcomes.

Methods

Data from a prospectively created database of consecutive patients undergoing open D2 total (TG) or subtotal (STG) gastrectomy with curative intent between January 2009 and January 2014 were reviewed. Short‐ and long‐term clinical outcomes were compared in patients operated on by consultants and those treated by trainees under consultant supervision.

Results

A total of 272 D2 open gastrectomies were performed, 123 (45·2 per cent) by trainees. There was no significant difference between consultants and trainees in median duration of surgery (TG: 240 (range 102–505) versus 240 (170–375) min respectively, P = 0·452; STG: 225 (150–580) versus 212 (125–380) min, P = 0·192), number of resected nodes (TG: 30 (13–101) versus 30 (11–102), P = 0·681; STG: 26 (5–103) versus 25 (1–63), P = 0·171), length of hospital stay (TG: 15 (7–78) versus 15 (8–65) days, P = 0·981; STG: 10 (6–197) versus 14 (7–85) days, P = 0·242), overall morbidity (TG: 44 versus 49 per cent, P = 0·314; STG: 34 versus 25 per cent, P = 0·113) or mortality (TG: 4 versus 2 per cent; P = 0·293). No difference in predicted 5‐year overall survival was noted between the two cohorts (TG: 68 per cent for consultants versus 77 per cent for trainees, P = 0·254; STG: 70 versus 75 per cent respectively, P = 0·512). The trainee cohort had lower median blood loss for both TG (360 (range 90–1200) ml versus 600 (70–2350) ml for consultants; P = 0·042) and STG (235 (50–1000) versus 360 (50–3000) ml respectively; P = 0·053).

Conclusion

Clinical outcomes were not compromised by supervised trainee involvement in D2 open gastrectomy.

Who owns responsibility? An administrator's take on implementing time-variable medical training in teaching hospitals

Introduction: Developments in outcome-based medical education led to the introduction of time-variable medical training (TVMT). Although this idea of training may be a consequence of competency-based training that calls for individualized learning, its implementation has posed significant challenges. As a new paradigm it is likely to have repercussions on the organization of teaching hospitals. The purpose of this study is therefore to explore how hospital administrators cope with this implementation process. Methods: We conducted an exploratory qualitative study for which we interviewed administrators of hospitals who were actively implementing TVMT in their postgraduate programs. Results: Several problems of implementation were identified: existing governance structures proved unfit to cope with the financial and organizational implications of TVMT. Administrators responded to these problems by delegating responsibilities to departments, reallocating tasks, learning from other hospitals and scaling up their teaching facilities. Conclusions: Hospital administrators perceived the implementation of TVMT as challenging. TVMT affects the existing equilibrium between education and clinical service. Administrators' initial attempts to regain control, using steering strategies that were based on known concepts and general outcomes, including cutting departmental budgets did not work, nor did their subsequent wait-and-see approach of leaving the implementation to the individual departments.

Perceptions of competency-based medical education from medical student discussion forums

BACKGROUND

Competency-based medical education (CBME) is becoming widely implemented in medical education. Trainees' perceptions of CBME are important factors in the implementation and acceptance of CBME. Online discussion groups allow unique insight into trainees' perceptions of CBME during residency training.

METHODS

We analysed 867 posts from 20 discussion threads in Premed 101 (Canadian) and 2756 posts from 50 threads in Student Doctor Network (SDN) (American) using NVivo 11. Inductive content analysis was used to develop a data-driven coding scheme that evolved throughout the analysis. Measures were taken to ensure the trustworthiness of findings, including co-coding of a subsample of 600 posts, peer debriefing, consensus-based analytical decision making and the maintenance of an audit trial.

RESULTS

Medical residents and students participating in the discussion forums emphasised select themes regarding the implementation of CBME in residency training. Concerns about CBME in Canada primarily involved its implications for the length of residency and post-residency opportunities. Posts on the American forum had a prominent focus on differing areas, such as the subjectivity in the assessment of core competencies and the role of CBME in termination of a resident's position.

CONCLUSIONS

Online discussion groups have the potential to provide unique insight into perceptions of CBME. The presented concerns may have implications for refining the model of CBME and illustrate the importance of providing clarification for trainees regarding length of training and evaluation structures from those involved in designing of CBME programmes.

Power of Judgment: The Significance of Kant's Philosophy for the Medical System Today

The ways of thinking in the manufacturing sciences are increasingly determining the rationality within medicine as a practical or action-based science. This "technological paradigm" infiltrates the field of medicine with the promise of increasing efficiency while simultaneously improving quality at various points in the system. Simple linear causal relationships generally need to be taken into account when manufacturing products. Even complex manufacturing processes can be broken down into the smallest units and, therefore, also be automated. The situation in complex systems such as the human body, however, is completely different. In order for doctors to be able to carry out their actions within this complex system, medicine as a science provides the physician with rules on the means that should be used to decide which remedy should be used, when and how. This judgment of which remedy should be used, when and how, what is known as the indication, is a central medical moment. This requires a power of judgment sharpened by experience. The indication, in turn, essentially determines the course of a disease and thus the quality of the treatment or the quality of result so often referred to these days.

Time Out of General Surgery Specialty Training in the UK: A National Database Study

OBJECTIVE

General surgery specialty training in the United Kingdom takes 6 years and allows trainees to take time out of training. Studies from the United States have highlighted an increasing trend for taking time out of surgical training for research. This study aimed to evaluate trends in time out of training and the impact on the duration of UK general surgical specialty training.

DESIGN, SETTING, AND PARTICIPANTS

A cohort study using routinely collected surgical training data from the Intercollegiate Surgical Curriculum Program database for General surgery trainees registered from August 1, 2007. Trainees were classified as Completed Training or In-Training. Out of training periods were identified and time in training calculated (both unadjusted and adjusted for out of training periods) with a predicted time in training for those In-Training.

RESULTS

Of the trainees still In-Training (n = 994), a greater proportion had taken time out of training compared with those who had completed training (n = 360; 54.5% vs 45.9%, p < 0.01). A greater proportion of the In-Training group had undertaken a formal research period compared with the Completed Training group (35.1% vs 6.1%, p < 0.01). Total unadjusted training time in the Completed Training group was a median 6.0 (interquartile range 6.0-7.0) years compared with a predicted unadjusted training time in the In-Training group, with an out of training period recorded, of a median 8.0 (interquartile range 7.0-9.0) years.

CONCLUSIONS

Trainees are increasingly taking time out of surgical training, particularly for research, with a subsequent increase in total time of training. This should be considered when redesigning surgical training programs and planning the future surgical workforce.

Perceptions on Competence by Design in urology

INTRODUCTION

The Royal College of Physicians and Surgeons of Canada has begun implementing Competence by Design (CBD). However, it is unclear how much urology trainees and faculty know about CBD, their attitudes towards this change, and their willingness to embrace and participate in this new model of training.

METHODS

This cross-sectional study was conducted through an online survey, which was administered to all trainees and faculty at Canadian urology programs prior to the implementation of CBD. The final survey consisted of eight demographic questions, 17 five-point Likert items, one visual analog scale question, 11 multiple selection questions, and two open-ended questions.

RESULTS

A total of 74 participants (38 faculty and 36 trainees) across 12 universities responded, with a completion rate of 82.4%. This corresponded to an overall response rate of 20.5%. Overall, there was a lack of resounding enthusiasm towards this shift to CBD in urology. Although both trainees and faculty had overall positive perceptions of CBD on assessment, teaching, and readiness, most agreed that this transition will be costly and associated with increased requirements for time, funding, and administrative support. Furthermore, there were significant concerns regarding the lack of valid assessment tools and evidence for the validity of entrustable professional activities.

CONCLUSIONS

While this survey has demonstrated an appreciation for the benefits of CBD, challenges are equally anticipated. CBD in urology will be a fertile research area; this study has identified several important educational questions regarding the model's effectiveness and consequences, thus, providing collaborative opportunities among all Canadian programs.

What They May Not Tell You and You May Not Know to Ask: What is Expected of Surgeons in Their First Year of Independent Practice

OBJECTIVE

The objective of this study was to explore the views and expectations that practicing general surgeons have of their junior colleagues who have recently finished training.

DESIGN

This is a qualitative study performed using focus group data consisting of open-ended questions concentrating on essential qualities and attributes of surgeons, behaviors observed in newly-graduated surgeons, and appropriate oversight of junior partners. Qualitative analysis was performed using grounded theory methodology with transcripts coded by 3 independent reviewers.

SETTING

Focus groups were conducted with surgeons practicing in rural and urban community settings.

PARTICIPANTS

Focus groups consisted of practicing general surgeons throughout the state of Oregon.

RESULTS

Focus groups were comprised of 31 practicing surgeons (10 female, 21 male) with varying ages and levels of experience practicing in both rural and urban environments. Qualitative analysis revealed the need for surgeons with strong interpersonal skills, teamwork, judgment, and broad technical skills who possess the appropriate amount of confidence and know when to ask for help. Frequently noted themes identified, included not knowing when to ask for help, overconfidence or underconfidence, as well as lack of judgment and lack of either quality or breadth of technical skill. Current oversight included direct observation, subjective evaluations from staff and colleagues, analysis of outcomes/quality, and either formal or informal mentorship arrangements.

CONCLUSIONS

This study highlights the need for graduating surgeons to be competent in multiple domains. The importance of knowing when to ask for help was stressed by practicing surgeons in both the rural and urban community setting, but is underemphasized in residency training, possibly due to less indirect resident supervision. Surgeons also emphasized the importance of mentorship, as professional growth continues long after completion of training.

Changes in Perceived Supervision Quality After Introduction of Competency-Based Orthopedic Residency Training: A National 6-Year Follow-Up Study

OBJECTIVE

To evaluate the perceived quality of the learning environment, before and after introduction of competency-based postgraduate orthopedic education.

DESIGN

From 2009 to 2014, we conducted annual surveys among Dutch orthopedic residents. The validated Dutch Residency Educational Climate Test (D-RECT, 50 items on 11 subscales) was used to assess the quality of the learning environment. Scores range from 1 (poor) to 5 (excellent).

SETTING

Dynamic cohort follow-up study.

PARTICIPANTS

All Dutch orthopedic residents were surveyed during annual compulsory courses.

RESULTS

Over the 6-year period, 641 responses were obtained (response rate 92%). Scores for "supervision" (95% CI for difference 0.06-0.28, p = 0.002) and "coaching and assessment" (95% CI 0.11-0.35, p < 0.001) improved significantly after introduction of competency-based training. There was no significant change in score on the other subscales of the D-RECT.

CONCLUSIONS

After the introduction of some of the core components of competency-based postgraduate orthopedic education the perceived quality of "supervision" and "coaching and assessment" improved significantly.

Assessment of Technical Skills Competence in the Operating Room: A Systematic and Scoping Review

PURPOSE

While academic accreditation bodies continue to promote competency-based medical education (CBME), the feasibility of conducting regular CBME assessments remains challenging. The purpose of this study was to identify evidence pertaining to the practical application of assessments that aim to measure technical competence for surgical trainees in a nonsimulated, operative setting.

METHOD

In August 2016, the authors systematically searched Medline, Embase, and the Cochrane Database of Systematic Reviews for English-language, peer-reviewed articles published in or after 1996. The title, abstract, and full text of identified articles were screened. Data regarding study characteristics, psychometric and measurement properties, implementation of assessment, competency definitions, and faculty training were extracted. The findings from the systematic review were supplemented by a scoping review to identify key strategies related to faculty uptake and implementation of CBME assessments.

RESULTS

A total of 32 studies were included. The majority of studies reported reasonable scores of interrater reliability and internal consistency. Seven articles identified minimum scores required to establish competence. Twenty-five articles mentioned faculty training. Many of the faculty training interventions focused on timely completion of assessments or scale calibration.

CONCLUSIONS

There are a number of diverse tools used to assess competence for intraoperative technical skills and a lack of consensus regarding the definition of technical competence within and across surgical specialties. Further work is required to identify when and how often trainees should be assessed and to identify strategies to train faculty to ensure timely and accurate assessment.

Developing the Blueprint for a General Surgery Technical Skills Certification Examination: A Validation Study

INTRODUCTION

There is a recognized need to develop high-stakes technical skills assessments for decisions of certification and resident promotion. High-stakes examinations requires a rigorous approach in accruing validity evidence throughout the developmental process. One of the first steps in development is the creation of a blueprint which outlines the potential content of examination. The purpose of this validation study was to develop an examination blueprint for a Canadian General Surgery assessment of technical skill certifying examination.

METHODS

A Delphi methodology was used to gain consensus amongst Canadian General Surgery program directors as to the content (tasks or procedures) that could be included in a certifying Canadian General Surgery examination. Consensus was defined a priori as a Cronbach's α ≥ 0.70. All procedures or tasks reaching a positive consensus (defined as ≥80% of program directors rated items as ≥4 on the 5-point Likert scale) were then included in the final examination blueprint.

RESULTS

Two Delphi rounds were needed to reach consensus. Of the 17 General Surgery Program directors across the country, 14 (82.4%) and 10 (58.8%) program directors responded to the first and second round, respectively. A total of 59 items and procedures reached positive consensus and were included in the final examination blueprint.

CONCLUSIONS

The present study has outlined the development of an examination blueprint for a General Surgery certifying examination using a consensus-based methodology. This validation study will serve as the foundational work from which simulated model will be developed, pilot tested and evaluated.

Force-based learning curve tracking in fundamental laparoscopic skills training

BACKGROUND

Within minimally invasive surgery (MIS), structural implementation of courses and structured assessment of skills are challenged by availability of trainers, time, and money. We aimed to establish and validate an objective measurement tool for preclinical skills acquisition in a basic laparoscopic at-home training program.

METHODS

A mobile laparoscopic simulator was equipped with a state-of-the-art force, motion, and time tracking system (ForceSense, MediShield B.V., Delft, the Netherlands). These performance parameters respectively representing tissue manipulation and instrument handling were continuously tracked during every trial. Proficiency levels were set by clinical experts for six different training tasks. Resident's acquisition and development of fundamental skills were evaluated by comparing pre- and post-course assessment measurements and OSATS forms. A questionnaire was distributed to determine face and content validity.

RESULTS

Out of 1842 captured attempts by novices, 1594 successful trials were evaluated. A decrease in maximum exerted absolute force was shown in comparison of four training tasks (p ≤ 0.023). Three of the six comparisons also showed lower mean forces during tissue manipulation (p ≤ 0.024). Lower instrument handling outcomes (i.e., time and motion parameters) were observed in five tasks (resp. (p ≤ 0.019) and (p ≤ 0.025)). Simultaneously, all OSATS scores increased (p ≤ 0.028). Proficiency levels for all tasks can be reached in 2 weeks of at home training.

CONCLUSIONS

Monitoring force, motion, and time parameters during training showed to be effective in determining acquisition and development of basic laparoscopic tissue manipulation and instrument handling skills. Therefore, we were able to gain insight into the amount of training needed to reach certain levels of competence. Skills improved after sufficient amount of training at home. Questionnaire outcomes indicated that skills and self-confidence improved and that this training should therefore be part of the regular residency training program.