Vascular Anastomosis Model: Relation Between Competency in a Laboratory-based Model and Surgical Competency

A Systematic Review of Simulation-Based Training in Vascular Surgery

Introduction

Recent advancements in surgical technology, reduced working hours, and training opportunities exacerbated by the COVID-19 pandemic have led to an increase in simulation-based training. Furthermore, a rise in endovascular procedures has led to a requirement for high-fidelity simulators that offer comprehensive feedback. This review aims to identify vascular surgery simulation models and assess their validity and levels of effectiveness (LoE) for each model in order to successfully implement them into current training curricula.

Methods

PubMed and EMBASE were searched on January 1, 2021, for full-text English studies on vascular surgery simulators. Eligible articles were given validity ratings based on Messick’s modern concept of validity alongside an LoE score according to McGaghie’s translational outcomes.

Results

Overall 76 eligible articles validated 34 vascular surgery simulators and training courses for open and endovascular procedures. High validity ratings were achieved across studies for: content (35), response processes (12), the internal structure (5), relations to other variables (57), and consequences (2). Only seven studies achieved an LoE greater than 3/5. Overall, ANGIO Mentor was the most highly validated and effective simulator and was the only simulator to achieve an LoE of 5/5.

Conclusions

Simulation-based training in vascular surgery is a continuously developing field with exciting future prospects, demonstrated by the vast number of models and training courses. To effectively integrate simulation models into current vascular surgery curricula and assessments, there is a need for studies to look at trainee skill retention over a longer period of time. A more detailed discussion on cost-effectiveness is also needed.

Utilization of Flexible-Wearable Sensors to Describe the Kinematics of Surgical Proficiency

BACKGROUND

Traditional assessment (e.g., checklists, videotaping) for surgical proficiency may lead to subjectivity and does not predict performance in the clinical setting. Hand motion analysis is evolving as an objective tool for grading technical dexterity; however, most devices accompany with technical limitations or discomfort. We purpose the use of flexible wearable sensors to evaluate the kinematics of surgical proficiency.

METHODS

Surgeons were recruited and performed a vascular anastomosis task in a single institution. A modified objective structured assessment of technical skills (mOSATS) was used for technical qualification. Flexible wearable sensors (BioStamp RCTM, mc10 Inc., Lexington, MA) were placed on the dorsum of the dominant hand (DH) and nondominant hand (nDH) to measure kinematic parameters: path length (T_path), mean (V_mean) and peak (V_peak) velocity, number of hand movements (N_move), ratio of DH to nDH movements (rMov), and time of task (tTask) and further compared with the mOSATS score.

RESULTS

Participants were categorized as experts (n = 12) and novices (n = 8) based on a cutoff mean mOSATS score. Significant differences for tTask (P = 0.02), rMov (P = 0.07), DH T_path (P = 0.04), V_mean (P = 0.07), V_peak (P = 0.04), and nDH N_move (P = 0.02) were in favor of the experts. Overall, mOSATS had significant correlation with tTask (r = -0.69, P = 0.001), N_move of DH (r = -0.44, P = 0.047) and nDH (r = -0.66, P = 0.001), and rMov (r = 0.52, P = 0.017).

CONCLUSIONS

Hand motion analysis evaluated by flexible wearable sensors is feasible and informative. Experts utilize coordinated two-handed motion, whereas novices perform one-handed tasks in a hastily jerky manner. These tendencies create opportunity for improvement in surgical proficiency among trainees.

Simulation Based Training and Assessment in Open Vascular Surgery: A Systematic Review

OBJECTIVE

The aim of this study was to systematically review the literature and give evidence based recommendations for future initiatives for simulation based training (SBT) and assessment in open vascular surgery.

DATA SOURCES

PubMed, Embase, and the Cochrane Library.

REVIEW METHODS

A systematic review of PubMed, Embase, and the Cochrane Library was performed, with the last search on 31 March 2020, to identify studies describing SBT and assessment in open vascular surgery. Kirkpatrick's levels for efficacy of training were evaluated. Validity evidence for assessment tools was evaluated according to the recommended contemporary framework by Messick.

RESULTS

Of 2 844 studies, 51 were included for data extraction. A high degree of heterogeneity in reporting standards and varying types of simulation was found. Vascular anastomosis was the most frequently simulated technical skill (43%). Assessment was mostly carried out using the Objective Structured Assessment of Technical Skills (55%). Validity evidence for assessment tools was found using outdated frameworks, and only one study used Messick's framework. Self directed training is valuable, the low trainer to trainee ratio is important to maximise efficiency, and experienced vascular surgeons are the most effective trainers.

CONCLUSION

Carefully designed and structured SBT is effective and can improve technical skills, especially in less experienced trainees. However, the supporting evidence lacks homogeneity in the reporting standards and types of simulations. Pass/fail standards that support proficiency based learning and studies investigating skills transfer should be the focus in future studies. Validity evidence of assessment tools needs to be addressed using contemporary frameworks.

The "Crater" Arteriotomy: A Technique Aiding Precise Intimal Apposition in End-to-side Microvascular Anastomosis

End-to-side arterial anastomoses require a high level of technical competency. The main challenge to a successfully patent anastomosis is intimal interposition during the standardized microvascular suturing. Technical errors during arteriotomy pose a significant challenge for the microsurgical technique, making the end-to-side anastomosis prone to failure. We describe a basic yet fundamental method of performing an arteriotomy, the “crater” technique, which facilitates good visualization of all vessel layers before placement of microsurgical sutures. Using curved microsurgical scissors, the adventitia layer is dissected off the outer surface of the side vessel, a V-shaped cut is then made obliquely at a 30–45 degrees angle to the longitudinal axis of the vessel, and a full thickness oblique cut is made along an elliptical circumference, as the curved scissors enable the creation of a slope-like crater. This concept ensures the intimal layer is adequately exposed through the complete circumference of the arteriotomy rim, while enabling a variable increase in the arterial wall hypotenuse-width circumference. When performed in a standardized manner, the crater arteriotomy can minimize the risk of endothelial misalignment and further technical errors during suturing, thus minimizing the risk of anastomotic failure.

Arteriovenous anastomosis learning curve using low cost simulator

Background

In order to reduce difficulties with learning surgical techniques, supplementary tools for training were developed. This paper describes the learning curve followed by student volunteer research subjects who used an alternative model for practicing vascular anastomosis.

Objectives

To evaluate the vascular anastomosis technique learning curve and development of manual skills using a low-cost experimental model.

Methods

Experimental and prospective study using end-to-side vascular anastomosis in latex balloons over five successive phases, initiated after theoretical and practical guidance given by experienced vascular surgeon. The study subjects were six undergraduate medical students from Universidade Federal da Paraíba (UFPB), João Pessoa, PB, Brazil, in their third to fifth years of the course. Cluster analysis was used to interpret the data collected on the quality of anastomoses and the time taken.

Results

The time taken to perform anastomosis reduced for all students, with statistical differences from phase 1 compared to phases 4 and 5. There was also a trend to increasing scores on the quality index as the phases progressed. However, no statistical differences were detected using the Friedman test, which is appropriate for data measured with ordinal levels (quality was assessed on a scale of 1 to 5).

Conclusions

It was found that the training model used was effective for increasing learning of this technique. It is believed that future studies with larger samples or a higher number of phases could demonstrate both reduced time and improved quality of the anastomoses performed with statistical significance.

Simulation versus real-world performance: a direct comparison of emergency medicine resident resuscitation entrustment scoring

Background

Simulation is increasingly being used in postgraduate medical education as an opportunity for competency assessment. However, there is limited direct evidence that supports performance in the simulation lab as a surrogate of workplace-based clinical performance for non-procedural tasks such as resuscitation in the emergency department (ED). We sought to directly compare entrustment scoring of resident performance in the simulation environment to clinical performance in the ED.

Methods

The resuscitation assessment tool (RAT) was derived from the previously implemented and studied Queen's simulation assessment tool (QSAT) via a modified expert review process. The RAT uses an anchored global assessment scale to generate an entrustment score and narrative comments. Emergency medicine (EM) residents were assessed using the RAT on cases in simulation-based examinations and in the ED during resuscitation cases from July 2016 to June 2017. Resident mean entrustment scores were compared using Pearson's correlation coefficient to determine the relationship between entrustment in simulation cases and in the ED. Inductive thematic analysis of written commentary was conducted to compare workplace-based with simulation-based feedback.

Results

There was a moderate, positive correlation found between mean entrustment scores in the simulated and workplace-based settings, which was statistically significant (r = 0.630, n = 17, p < 0.01). Further, qualitative analysis demonstrated overall management and leadership themes were more common narratives in the workplace, while more specific task-based feedback predominated in the simulation-based assessment. Both workplace-based and simulation-based narratives frequently commented on communication skills.

Conclusions

In this single-center study with a limited sample size, assessment of residents using entrustment scoring in simulation settings was demonstrated to have a moderate positive correlation with assessment of resuscitation competence in the workplace. This study suggests that resuscitation performance in simulation settings may be an indicator of competence in the clinical setting. However, multiple factors contribute to this complicated and imperfect relationship. It is imperative to consider narrative comments in supporting the rationale for numerical entrustment scores in both settings and to include both simulation and workplace-based assessment in high-stakes decisions of progression.

Original Homebuilt Off-the-Job Training System for Vascular Surgeons: System Analysis and Assessment

Objective: We assessed the effectiveness and appropriateness of our original off-the-job training (Off JT) system using data acquired from recruited medical students and doctors.

Materials and Methods: We presented our original homebuilt Off JT system, which is simple and inexpensive. In our unique system, we performed anastomosis at the bottom of a plastic pot, which mimics the actual open surgical procedure at a deep site. There were four evaluation points: (A) operating time, (B) performance of anastomosis by semi-automatically analyzing the image with the coefficient of variation (standard deviation/length) of the “bite” and the “pitch,” (C) scoring of the total surgical skill evaluated by the trainers according to the Operative Performance Rating System (OPRS), and (D) the relationship of these three factors (A, B, and C).

Results: The procedural time and coefficient of variation of the bite and pitch decreased and the OPRS score increased after training. There was a strong correlation between procedural time, anastomotic performance, and OPRS score.

Conclusion: The effectiveness of our original homebuilt system was shown by reduced procedural time, improved anastomotic quality, and increased OPRS score.

Latex balloons: an alternative, low-cost model for vascular anastomosis training in medical education

Simulators are increasingly being used in medical education, but accessibility is restricted by their elevated cost. A accessible and low-cost model was developed for teaching and learning vascular sutures and anastomoses at a Basic Surgical Techniques Laboratory. Latex balloons of varying colors, polypropylene 6.0 sutures, and other materials specifically for suturing (needle holder and forceps) were used. The balloons were fixed to screws inserted into wooden boards in order to facilitate repairs. E end-to-end, end-to-side, and side-to-side anastomoses and patching were performed. Anastomosis patency was tested by injecting water into one extremity of the balloon and observing the liquid exit via the opposite extremity. The advantages observed with this training model for anastomoses were malleability, resistance to passage of the suture, and the fact that it is inorganic. Latex balloons are an inexpensive option that are non‑perishable and offer prolonged use for teaching and practice of arterial sutures and anastomoses.

Case Reporting, Competence, and Confidence: A Discrepancy in the Numbers

PURPOSE

The Accreditation Council for Graduate Medical Education (ACGME) continues to play an integral role in accreditation of surgical programs. The institution of case logs to demonstrate competency of graduating residents is a key component of evaluation. This study compared the number of vascular cases a surgical resident has completed according to the ACGME operative log to their operative proficiency, quality of anastomosis, operative experience, and confidence in both a simulation and operative setting.

MATERIALS AND METHODS

General surgery residents ranging from PGY 1 to 5 participated in a simulation laboratory in which they completed an end-to-side vascular anastomosis. Each participant was given a weighted score based on technical proficiency and anastomosis quality using a previously validated Global Rating Scale (Duran et al, 2014). These scores were correlated to the General Surgery Milestones. Participants completed preoperative and postoperative surveys assessing resident operative experience using the 4-level Zwisch scale (DaRosa et al., 2013), confidence with vascular procedures and confidence performing simulated anastomoses. Confidence was assessed on a scale from 1 to 9 (not confident to extremely confident). Case logs were recorded for each participant. An IRB approved questionnaire was distributed to assess preoperative and postoperative roles of both the resident physician and faculty, with a defined goal. Univariate and multivariate analysis was performed.

RESULTS

Twenty-one general surgery residents were evaluated in the simulation laboratory and 8 residents were assessed intraoperatively. The residents were evenly distributed throughout clinical years. Groups of residents were divided into quartiles based upon the number of vascular cases recorded in the ACGME database. No correlation was found between number of cases, Milestones score and the weighted score (p = 0.94). No statistical significance was found between confidence and quality of anastomosis (p = 0.1). Resident operative experience per the Zwisch scale was categorized most commonly as "Smart Help" by both the trainee and attending surgeon, despite mean resident confidence ratings of 6.67 (± 1.61) with vascular procedures.

CONCLUSIONS

ACGME case logs, which are utilized to assess readiness for completion of general surgery residency, may not be indicative of a resident's operative competency and technical proficiency. Confidence is not correlated with technical ability. Faculty and resident insight as to their role in a procedure differ, as faculty feel that they are providing less help than the resident perceives. Careful examination of resident operative technique is the best measure of competency.

Treinamento de anastomoses vasculares de baixo custo: o cirurgião vai à feira

Anastomoses vasculares são procedimentos comuns realizados por grande parte dos cirurgiões e cujo treinamento ocorre principalmente em seres humanos, contrariando os princípios éticos vigentes. Esse fato se deve, sobretudo, à carência e ao alto custo relacionados aos atuais modelos de treinamento. Assim, este estudo visa avaliar a viabilidade de três vegetais para a realização de anastomoses vasculares. Foram utilizadas cinco unidades de cebolinha, vagem e feijão-verde. Em cada uma tentou-se realizar uma anastomose término-terminal. Conseguiu-se a realização da anastomose apenas na vagem e no feijão-verde. Contudo, por apresentar uma menor espessura, o feijão-verde assemelhou-se mais aos vasos humanos.

Variation in the learning curves of general surgery residents performing arteriovenous fistulas

BACKGROUND

An arteriovenous fistula (AVF), performed for hemodialysis access, provides one of the few remaining opportunities for general surgery residents to perform an open vascular anastomosis (VA). Limited data exist regarding the learning curve of residents performing this procedure. The objective of this study was to determine how residents improve in performance of VA by implementing real-time tracking of anastomosis time as well as technical errors.

STUDY DESIGN

From April 2012 to January 2014, we conducted a prospective intraoperative assessment of 9 postgraduate year 3 general surgery residents during the performance of AVFs using a checklist of common errors in VA. Time for AVF anastomosis completion and number and types of technical errors during anastomosis were recorded. Primary end points were the change in anastomosis time and change in technical errors over time.

RESULTS

A total of 86 AVFs were performed and assessed intraoperatively. Each resident performed a median of 10 AVFs (interquartile range [IQR]: 7-11). The mean anastomosis time was 18.1 minutes. The mean number of technical errors was 13.8 per case. Overall, for every additional AVF performed, mean anastomosis time decreased by 0.63 minutes (95% CI: 0.45-0.81, p < 0.0001) and the mean number of technical errors decreased by 1.0 (95% CI: 0.7-1.3, p < 0.0001). The greatest improvement in overall errors (mean difference = 7.9, p = 0.03) and time (mean difference = 4.7min, p = 0.03) occurred after the performance of 3 AVFs. However, when analyzed by individual resident, the R(2) value for anastomotic time by number of AVFs performed ranged from 0.01 to 0.69. Similarly, for technical errors, the R(2) value by number of AVFs performed ranged from 0.04 to 0.62.

CONCLUSIONS

In novice surgical residents performing AVFs, improvement in VA skill can readily be tracked via anastomosis time and technical errors. Collectively, there is a strong association between number of cases performed and reduction in time and errors. However, individually, the number of cases completed did not correlate well with time and errors. These findings suggest that for VA skills, determining progression from novice to competence cannot rely on case volume but rather needs to be individualized.

Technical skills assessment toolbox: a review using the unitary framework of validity

OBJECTIVE

The purpose of this study was to create a technical skills assessment toolbox for 35 basic and advanced skills/procedures that comprise the American College of Surgeons (ACS)/Association of Program Directors in Surgery (APDS) surgical skills curriculum and to provide a critical appraisal of the included tools, using contemporary framework of validity.

BACKGROUND

Competency-based training has become the predominant model in surgical education and assessment of performance is an essential component. Assessment methods must produce valid results to accurately determine the level of competency.

METHODS

A search was performed, using PubMed and Google Scholar, to identify tools that have been developed for assessment of the targeted technical skills.

RESULTS

A total of 23 assessment tools for the 35 ACS/APDS skills modules were identified. Some tools, such as Operative Performance Rating System (OSATS) and Objective Structured Assessment of Technical Skill (OPRS), have been tested for more than 1 procedure. Therefore, 30 modules had at least 1 assessment tool, with some common surgical procedures being addressed by several tools. Five modules had none. Only 3 studies used Messick's framework to design their validity studies. The remaining studies used an outdated framework on the basis of "types of validity." When analyzed using the contemporary framework, few of these studies demonstrated validity for content, internal structure, and relationship to other variables.

CONCLUSIONS

This study provides an assessment toolbox for common surgical skills/procedures. Our review shows that few authors have used the contemporary unitary concept of validity for development of their assessment tools. As we progress toward competency-based training, future studies should provide evidence for various sources of validity using the contemporary framework.

Linking simulation-based educational assessments and patient-related outcomes: a systematic review and meta-analysis

PURPOSE

To examine the evidence supporting the use of simulation-based assessments as surrogates for patient-related outcomes assessed in the workplace.

METHOD

The authors systematically searched MEDLINE, EMBASE, Scopus, and key journals through February 26, 2013. They included original studies that assessed health professionals and trainees using simulation and then linked those scores with patient-related outcomes assessed in the workplace. Two reviewers independently extracted information on participants, tasks, validity evidence, study quality, patient-related and simulation-based outcomes, and magnitude of correlation. All correlations were pooled using random-effects meta-analysis.

RESULTS

Of 11,628 potentially relevant articles, the 33 included studies enrolled 1,203 participants, including postgraduate physicians (n = 24 studies), practicing physicians (n = 8), medical students (n = 6), dentists (n = 2), and nurses (n = 1). The pooled correlation for provider behaviors was 0.51 (95% confidence interval [CI], 0.38 to 0.62; n = 27 studies); for time behaviors, 0.44 (95% CI, 0.15 to 0.66; n = 7); and for patient outcomes, 0.24 (95% CI, -0.02 to 0.47; n = 5). Most reported validity evidence was favorable, though studies often included only correlational evidence. Validity evidence of internal structure (n = 13 studies), content (n = 12), response process (n = 2), and consequences (n = 1) were reported less often. Three tools showed large pooled correlations and favorable (albeit incomplete) validity evidence.

CONCLUSIONS

Simulation-based assessments often correlate positively with patient-related outcomes. Although these surrogates are imperfect, tools with established validity evidence may replace workplace-based assessments for evaluating select procedural skills.

A model of cardiopulmonary bypass staged training integrating technical and non-technical skills dedicated to cardiac trainees

OBJECTIVES

To develop a standardized simulation-based curriculum to teach medical knowledge and technical, communication and critical thinking skills necessary to initiate and wean from cardiopulmonary bypass (CPB) to junior cardiac trainees (CTs) in France. Performance on post-curricular tests was compared between CTs who participated in the new curriculum to those who did not.

METHODS

The simulation-based curriculum was developed by content and education experts. Simulations sequentially taught the skills necessary for initiating and weaning from CPB as well as managing crises by adding fidelity and complexity to scenarios. Nine CTs were randomly assigned to the new curriculum (n=5) or the traditional curriculum (n=4). Skills were assessed using tests of medical knowledge and technical, communication (GRS) and critical thinking (SCT) skills. A two-sample Wilcoxon rank-sum test compared average scores between the two groups. Alpha of 0.05 was set to indicate statistically significant differences.

RESULTS

The resutls revealed that CTs in the new curriculum significantly outperformed CTs in the traditional curriculum on technical (18.2 vs 14.8, p=0.05) and communication (3.5 vs 2.2, p=0.013) skills. There was no significant difference between CTs in the new curriculum in the Script Concordance Test (16.5 vs 14.8, p=0.141) and knowledge tests (26.9 vs 24.6, p=0.14) compared to CTs in the traditional curriculum.

CONCLUSION

Our new curriculum teaches communication and technical skills necessary for CPB. The results of this pilot study are encouraging and relevant. They give grounds for future research with a larger panel of trainees. Based on the current distribution of scores, a sample size of 12 CTs per group should yield significant results for all tests.

A systematic review of assessment of skill acquisition and operative competency in vascular surgical training

OBJECTIVE

The aim of this systematic review is to describe the literature and assessment tools evaluating vascular surgical operative performance that could potentially be used for the assessment of educational outcomes applicable to the Milestone Project and the Next Accreditation System.

METHODS

A systematic review of PubMed/MEDLINE, EMBASE, PsycINFO, and key journals from 1985 to 2013 was performed to identify English-language articles describing assessment of vascular surgical skills and competence. Qualifying studies were abstracted for data concerning study aims, study and assessment setting, skills measured, and metrics used to determine competency. Strengths, weaknesses, and psychometric robustness of the assessment tools were determined.

RESULTS

The literature search identified 617 citations. After title and abstract review, 65 articles were retrieved for full-text assessment and 48 articles were included in the final review. Twenty-nine articles assessed open vascular skills; 19, endovascular skills; six, nontechnical skills; and one, teamwork skills. The majority (84%) of studies were performed in a simulated environment, four (8%) were performed in the operating room, and the remaining three were performed in both a simulated environment and an operating room. Strengths and weaknesses of assessment tools were study and assessor dependent, with none applicable to all study scenarios or procedures.

CONCLUSIONS

The literature describing assessment tools pertinent to vascular surgery is diverse. Existing assessment tools may be relevant to individual technical skill acquisition assessment; however, an operative assessment tool relevant to vascular/endovascular surgery and generalizable to the wide spectrum of technical and nontechnical skills pertinent to vascular surgery needs to be developed, validated, and implemented to allow the practical assessment of resident readiness to operate in an unsupervised setting.

Perspective: Reconsidering the focus on "outcomes research" in medical education: a cautionary note

Researchers in medical education have been placing increased emphasis on "outcomes research," or the observable impact of educational interventions on patient care. However, although patient outcomes are obviously important, they should not be the sole focus of attention in medical education research. The purpose of this perspective is both to highlight the limitations of outcomes research in medical education and to offer suggestions to facilitate a proper balance between learner-centered and patient-centered assessments. The authors cite five challenges to research using patient outcomes in medical education, namely (1) dilution (the progressively attenuated impact of education as filtered through other health care providers and systems), (2) inadequate sample size, (3) failure to establish a causal link, (4) potentially biased outcome selection, and (5) teaching to the test. Additionally, nonpatient outcomes continue to hold value, particularly in theory-building research and in the evaluation of program implementation. To educators selecting outcomes and instruments in medical education research, the authors offer suggestions including to clarify the study objective and conceptual framework before selecting outcomes, and to consider the development and use of behavioral and other intermediary outcomes. Deliberately weighing the available options will facilitate informed choices during the design of research that, in turn, informs the art and science of medical education.

Surgical trainee participation during infrainguinal bypass grafting procedures is associated with increased early postoperative graft failure

OBJECTIVE

This study was conducted to determine the potential effect of surgical trainee participation during infrainguinal bypass procedures on postoperative graft patency rates.

METHODS

Data from the National Surgical Quality Improvement Program (NSQIP) Participant User Files from 2005 through 2009 were retrospectively reviewed, using propensity score matching, to identify all patients undergoing infrainguinal bypass grafting procedures, excluding those who had prior operation ≤30 days of the index procedure. A separate analysis was performed on a subset of procedures from the entire NSQIP sample that was matched on propensity for intraoperative surgical trainee participation. The primary predictor variable was intraoperative surgical trainee participation. The main outcome measure was the 30-day postoperative graft failure rate.

RESULTS

For the entire sample of 14,723 NSQIP patients undergoing infrainguinal bypass grafting, 30-day graft failure rates were significantly higher when a surgical trainee participated (5.8%) vs without participation (3.9%; P < .0001). For the cohort of 9234 patients matched on their propensity for intraoperative trainee participation, this difference in graft failure rate remained significant (5.0% with participation vs 4.0% without participation; P = .02).

CONCLUSIONS

Surgical trainee participation is an independent risk factor for technical failure after infrainguinal bypass grafting. Prospective evaluation is needed to determine the cause of this increase in graft failure rates for procedures that involve surgical trainees.

Observational tools for assessment of procedural skills: a systematic review

BACKGROUND

Assessment by direct observation of procedural skills is an important source of constructive feedback. The aim of this study was to identify observational tools for technical skill assessment, to assess characteristics of these tools, and to assess their usefulness for assessment.

METHODS

Included studies reported tools for observational assessment of technical skills. A total of 106 articles were included.

RESULTS

Three main categories included global assessment scales evaluating generic skills (n = 29), task-specific methods assessing procedure-specific skills (n = 30), and combinations of tools evaluating both generic and task-specific skills (n = 47). In most studies, content validity was not evaluated using an accepted scientific method. All tools were assessed for inter-rater reliability and construct validity. Data on feasibility, acceptability, and educational impact were sparse.

CONCLUSIONS

There is evidence of validity and reliability for observational assessment tools at the trainee level. In most studies a comprehensive analysis of the tools was not achieved. Evaluation of technical skill using current observational assessment tools is not reliable and valid at the specialist level. Future research needs to focus on further systematic tool development and analysis, especially at the specialist level.

Description and evaluation of a bench porcine model for teaching surgical residents vascular anastomosis skills

BACKGROUND

Numerous models, of variable quality, exist to impart the complex skills required to perform vascular anastomosis. These models differ with regard to the kinds of materials used, as well as their sizes, the time needed for their preparation, their availability, and the associated costs. The present study describes a bench model that uses formalin-fixed porcine aorta, and its evaluation by young surgical residents during a recent skills course.

FINDINGS

The aortic segments used were a by-product of slaughtering. They were fixed and stored after harvesting for eventual use. Ten young surgical residents participated, and each performed one end-to-side vascular anastomosis. The evaluation was a questionnaire maintaining anonymity of the participant containing questions addressing particular aspects of the model and the experiences of the trainee, along with their ratings concerning the need for a training course to learn vascular anastomosis techniques. The scoring on the survey was done using a global 6-point rating scale (Likert Scale). In addition, we ranked the present model by reviewing the current literature for models that address vascular anastomosis skills.The trainees who participated were within their first two years of training (1.25 +/- 0.46). A strong agreement in terms of the necessity of training for vascular anastomosis techniques was evident among the participating trainees (5.90 +/- 0.32), who had only few prior manual experiences (total number 1.50 +/- 0.53). The query revealed a strong agreement that porcine aorta is a suitable model that fits the needs for training vascular anastomosis skills (5.70 +/- 0.48). Only a few bench models designed to teach surgical residents vascular anastomosis techniques were available in the literature.

CONCLUSIONS

The preparatory and financial resources needed to perform anastomosis skills training using porcine aorta are few. The presented bench model appears to be appropriate for learning vascular anastomosis skills, as rated by the surgical trainees themselves.

Objective assessment of technical surgical skills

Background

Surgeons are increasingly being scrutinized for their performance and there is growing interest in objective assessment of technical skills. The purpose of this study was to review all evidence for these methods, in order to provide a guideline for use in clinical practice.

Methods

A systematic search was performed using PubMed and Web of Science for studies addressing the validity and reliability of methods for objective skills assessment within surgery and gynaecology only. The studies were assessed according to the Oxford Centre for Evidence-based Medicine levels of evidence.

Results

In total 104 studies were included, of which 20 (19·2 per cent) had a level of evidence 1b or 2b. In 28 studies (26·9 per cent), the assessment method was used in the operating room. Virtual reality simulators and Objective Structured Assessment of Technical Skills (OSATS) have been studied most. Although OSATS is seen as the standard for skills assessment, only seven studies, with a low level of evidence, addressed its use in the operating room.

Conclusion

Based on currently available evidence, most methods of skills assessment are valid for feedback or measuring progress of training, but few can be used for examination or credentialing. The purpose of the assessment determines the choice of method.