Assessing Endovascular Skills using the Simulator for Testing and Rating Endovascular Skills (STRESS) Machine

An Image Information-Based Objective Assessment Method of Technical Manipulation Skills for Intravascular Interventions

The clinical success of vascular interventional surgery relies heavily on a surgeon's catheter/guidewire manipulation skills and strategies. An objective and accurate assessment method plays a critical role in evaluating the surgeon's technical manipulation skill level. Most of the existing evaluation methods incorporate the use of information technology to find more objective assessment models based on various metrics. However, in these models, sensors are often attached to the surgeon's hands or to interventional devices for data collection, which constrains the surgeon's operational movements or exerts an influence on the motion trajectory of interventional devices. In this paper, an image information-based assessment method is proposed for the evaluation of the surgeon's manipulation skills without the requirement of attaching sensors to the surgeon or catheters/guidewires. Surgeons are allowed to use their natural bedside manipulation skills during the data collection process. Their manipulation features during different catheterization tasks are derived from the motion analysis of the catheter/guidewire in video sequences. Notably, data relating to the number of speed peaks, slope variations, and the number of collisions are included in the assessment. Furthermore, the contact forces, resulting from interactions between the catheter/guidewire and the vascular model, are sensed by a 6-DoF F/T sensor. A support vector machine (SVM) classification framework is developed to discriminate the surgeon's catheterization skill levels. The experimental results demonstrate that the proposed SVM-based assessment method can obtain an accuracy of 97.02% to distinguish between the expert and novice manipulations, which is higher than that of other existing research achievements. The proposed method has great potential to facilitate skill assessment and training of novice surgeons in vascular interventional surgery.

Evaluation of tools to assess operative competence in endovascular procedures: a systematic review

BACKGROUND

With an increase in the use of endovascular interventions as an alternative to open surgery and the unique technical skills required, current methods for assessing the competence of vascular surgery trainees may not be optimal, suggesting a need for a shift in assessment modalities. We conducted this systematic review to explore current assessment methods used in vascular surgery training to assess competence specific to endovascular procedures.

METHODS

A comprehensive literature search was performed with a structured search strategy using terms focusing on endovascular procedures and assessment. Inclusion and exclusion criteria were used in order to screen for suitable articles.

RESULTS

We identified 54 articles that satisfied the inclusion criteria. These included a single randomized controlled trial, a single systematic review, a single narrative review and a single literature review, with the vast majority having level 2 evidence. Global rating scales, proficiency assessments and written/oral examinations were described as standard current assessment tools. These modalities lack reproducibility and objectivity, neglecting the needs of assessment of endovascular procedures requiring specialized decision making and finger dexterity. Novel methods such as high fidelity simulation and virtual reality promote reproducible and objective assessment methods in the context of endovascular surgery, and have a promising future.

CONCLUSION

While current assessment methods in vascular surgery are widely supported the changing skills required of a vascular surgery trainee warrants a shift in assessment modalities to better align to these requirements. High fidelity simulations show promise, although they require more extensive research to understand their relative merits.

Qualitative and Quantitative Assessment of Technical Skills in Percutaneous Coronary Intervention: In Vivo Porcine Studies

OBJECTIVE

Technical skill assessment plays an important role in the professional development of an interventionalist in percutaneous coronary intervention (PCI). However, most of the traditional assessment methods are time consuming and subjective. This paper aims to develop objective assessment techniques.

METHODS

In this study, a natural-behavior-based assessment framework is proposed to qualitatively and quantitatively assess technical skills in PCI. In vivo porcine studies were conducted to deliver a medical guidewire to two target coronaries of left circumflex arteries by six novice and four expert interventionalists. Simultaneously, four types of natural behaviors (i.e., hand motion, proximal force, muscle activity, and finger motion) were acquired from the subjects' dominant hand and arm. The features extracted from the behaviors of different skill-level groups were compared using the Mann-Whitney U-test for effective behavior selection. The effective ones were further applied in the Gaussian-mixture-model-based qualitative assessment and Mahalanobis-distance-based quantitative assessment.

RESULTS

The qualitative assessment achieves an accuracy of 92% to distinguish the novice and expert attempts, which is significantly higher than that of using single guidewire motions. Furthermore, the quantitative assessment can assign objective and effective scores for all attempts, indicating high correlation ( R = 0.9225) to those obtained by traditional methods.

CONCLUSION

The objective, effective, and comprehensive assessment of technical skills can be provided by qualitatively and quantitatively analyzing interventionalists' natural behaviors in PCI.

SIGNIFICANCE

This paper suggests a novel approach for the technical skill assessment and the promising results demonstrate the great importance and effectiveness of the proposed method for promoting the development of objective assessment techniques.

Artificial vascular models for endovascular training (3D printing)

The endovascular technique has led to a revolution in the care of patients with vascular disease; however, acquiring and maintaining proficiency over a broad spectrum of procedures is challenging. Three-dimensional (3D) printing technology allows the production of models that can be used for endovascular training. This article aims to explain the process and technologies available to produce vascular models for endovascular training, using 3D printing technology. The data are based on the group experience and a review of the literature. Different 3D printing methods are compared, describing their advantages, disadvantages and potential roles in surgical training. The process of 3D printing a vascular model based on an imaging examination consists of the following steps: image acquisition, image post-processing, 3D printing and printed model post-processing. The entire process can take a week. Prospective studies have shown that 3D printing can improve surgical planning, especially in complex endovascular procedures, and allows the production of efficient simulators for endovascular training, improving residents’ surgical performance and self-confidence.

Validation of an assessment tool for pre-operative EVAR planning

INTRODUCTION

Current methods of teaching endovascular aortic aneurysm repair (EVAR) planning involve non-criteria-based observations that lack validity. The primary aim of this study was to validate an EVAR Planning Objective Structured Assessment of Skill (EpOSAS) tool for the assessment of pre-operative EVAR planning skills.

METHODS

Content analysis was performed in order to inform the formulation of EpOSAS domains. Thirty-five participants planned two cases of infra-renal abdominal aortic aneurysm for EVAR, using the OsiriX 7 platform. EVAR planning measurements, with accompanying screenshots, were uploaded onto an electronic data collection sheet. Performance was assessed by three blinded assessors using the EpOSAS tool. Construct and concurrent validity were evaluated.

RESULTS

Inter-observer reliability for the three assessors for total EpOSAS scores was high (Cronbach's α 0.89). There were statistically significant differences in total EpOSAS scores between the different experience groups, demonstrating construct validity (Novice (5.3, IQR 5-5.3), Intermediate (15.3, IQR 14.8-16.8) and Experts (17.5, IQR 17-17.7), p<0.001). A statistically significant correlation was found between total EpOSAS scores and percentage error in measurements, demonstrating concurrent validity (Spearman's rank correlation coefficient: R=-0.250, p<0.001). Receiver-operator characteristics analysis established a cut-off point of 16 out of 18 for determining competence.

CONCLUSION

We have developed and validated a tool that can be used for the assessment of pre-operative EVAR planning skills.

Constructing a validity argument for the Objective Structured Assessment of Technical Skills (OSATS): a systematic review of validity evidence

In order to construct and evaluate the validity argument for the Objective Structured Assessment of Technical Skills (OSATS), based on Kane's framework, we conducted a systematic review. We searched MEDLINE, EMBASE, CINAHL, PsycINFO, ERIC, Web of Science, Scopus, and selected reference lists through February 2013. Working in duplicate, we selected original research articles in any language evaluating the OSATS as an assessment tool for any health professional. We iteratively and collaboratively extracted validity evidence from included articles to construct and evaluate the validity argument for varied uses of the OSATS. Twenty-nine articles met the inclusion criteria, all focussed on surgical technical skills assessment. We identified three intended uses for the OSATS, namely formative feedback, high-stakes assessment and program evaluation. Following Kane's framework, four inferences in the validity argument were examined (scoring, generalization, extrapolation, decision). For formative feedback and high-stakes assessment, there was reasonable evidence for scoring and extrapolation. However, for high-stakes assessment there was a dearth of evidence for generalization aside from inter-rater reliability data and an absence of evidence linking multi-station OSATS scores to performance in real clinical settings. For program evaluation, the OSATS validity argument was supported by reasonable generalization and extrapolation evidence. There was a complete lack of evidence regarding implications and decisions based on OSATS scores. In general, validity evidence supported the use of the OSATS for formative feedback. Research to provide support for decisions based on OSATS scores is required if the OSATS is to be used for higher-stakes decisions and program evaluation.

A systematic review of assessment of skill acquisition and operative competency in vascular surgical training

OBJECTIVE

The aim of this systematic review is to describe the literature and assessment tools evaluating vascular surgical operative performance that could potentially be used for the assessment of educational outcomes applicable to the Milestone Project and the Next Accreditation System.

METHODS

A systematic review of PubMed/MEDLINE, EMBASE, PsycINFO, and key journals from 1985 to 2013 was performed to identify English-language articles describing assessment of vascular surgical skills and competence. Qualifying studies were abstracted for data concerning study aims, study and assessment setting, skills measured, and metrics used to determine competency. Strengths, weaknesses, and psychometric robustness of the assessment tools were determined.

RESULTS

The literature search identified 617 citations. After title and abstract review, 65 articles were retrieved for full-text assessment and 48 articles were included in the final review. Twenty-nine articles assessed open vascular skills; 19, endovascular skills; six, nontechnical skills; and one, teamwork skills. The majority (84%) of studies were performed in a simulated environment, four (8%) were performed in the operating room, and the remaining three were performed in both a simulated environment and an operating room. Strengths and weaknesses of assessment tools were study and assessor dependent, with none applicable to all study scenarios or procedures.

CONCLUSIONS

The literature describing assessment tools pertinent to vascular surgery is diverse. Existing assessment tools may be relevant to individual technical skill acquisition assessment; however, an operative assessment tool relevant to vascular/endovascular surgery and generalizable to the wide spectrum of technical and nontechnical skills pertinent to vascular surgery needs to be developed, validated, and implemented to allow the practical assessment of resident readiness to operate in an unsupervised setting.

[Improve your skills!: evaluation of a 2.5-day basic course in vascular surgery for surgical trainees]

BACKGROUND

The traditional surgical training in the operating room (OR) is often complemented by participation in workshops and on simulators. The foundation Vascular International offers basic courses for vascular surgery techniques with training on pulsatile circulation, lifelike anatomical models. The aim of this study was to assess the efficacy of a 2.5-day intensive course on basic skills in vascular surgery.

MATERIAL AND METHODS

A total of 24 participants (67% male with an average age of 35 years) performed a vein patch-plasty before and after the basic vascular surgery instruction course. Endpoints of the study were the time needed for suturing and the technical quality, which were evaluated by two course trainers on a scale of 0-10. Furthermore, the participants were asked to evaluate their own technical competence. The statistical analysis was carried out using MS Excel (t-test and analysis of correlation).

RESULTS

A significantly shortened time for the suturing (19.5 min versus 14.1 min, p < 0.001) and improved quality of the vein patch were found after the workshop (p < 0.05) with a high correlation between the two observers (r = 0.885). The participants also evaluated their own surgical competence better at the end of the training but there was no correlation between the self-assessment and the quality of the patch (r = 0.146 before and r = 0.109 after the workshop).

CONCLUSIONS

A significant improvement in the time needed for suturing and the quality of the vein patch-plasty was shown in this study. Further studies are necessary to demonstrate the long-term success and possible shortening of the learning curve in hospitals with professional training. With regard to the current curriculum of surgical trainees in Germany basic vascular surgery courses should be considered as a potential valuable part of the surgical common trunk.

TheCardiovascular Fellows Bootcamp

General surgery residents graduate with an inconsistent knowledge of cardiovascular disease and an inadequate skill set for the practice of vascular surgery. Vascular educators have proposed to overcome these challenges by incorporating standardized milestones and simulation curricula into vascular education, but at present, nonclinical vascular education remains nonstandardized. The first step in this direction is to lay a foundation of knowledge and skill for trainees nationwide, and regionalized courses have begun to be offered to address this need. Through the Methodist DeBakey Institute for Cardiovascular Education, we have developed our own course, The Cardiovascular Fellows Bootcamp. The principles behind The Cardiovascular Fellows Bootcamp are teach early, teach the basics, teach broadly, and teach multiple cardiovascular disciplines, and over a 3-day weekend of didactic lectures and skill training, we aim to lay a foundation for cardiovascular training that prepares the trainees for their fellowship. In this article, we describe the way in which our course is run and the thought process behind our approach. We also address some of the practical concerns that make hosting a course of this magnitude feasible and our plans for improving and expanding in the future.