Capturing the Essence of Developing Endovascular Expertise for the Construction of a Global Assessment Instrument

Learning Skill Characteristics From Manipulations

Percutaneous coronary intervention (PCI) has increasingly become the main treatment for coronary artery disease. The procedure requires high experienced skills and dexterous manipulations. However, there are few techniques to model PCI skill so far. In this study, a learning framework with local and ensemble learning is proposed to learn skill characteristics of different skill-level subjects from their PCI manipulations. Ten interventional cardiologists (four experts and six novices) were recruited to deliver a medical guidewire to two target arteries on a porcine model for in vivo studies. Simultaneously, translation and twist manipulations of thumb, forefinger, and wrist are acquired with electromagnetic (EM) and fiber-optic bend (FOB) sensors, respectively. These behavior data are then processed with wavelet packet decomposition (WPD) under 1-10 levels for feature extraction. The feature vectors are further fed into three candidate individual classifiers in the local learning layer. Furthermore, the local learning results from different manipulation behaviors are fused in the ensemble learning layer with three rule-based ensemble learning algorithms. In subject-dependent skill characteristics learning, the ensemble learning can achieve 100% accuracy, significantly outperforming the best local result (90%). Furthermore, ensemble learning can also maintain 73% accuracy in subject-independent schemes. These promising results demonstrate the great potential of the proposed method to facilitate skill learning in surgical robotics and skill assessment in clinical practice.

Radiation Safety Performance is More than Simply Measuring Doses! Development of a Radiation Safety Rating Scale

PURPOSE

Radiation safety performance is often evaluated using dose parameters measured by personal dosimeters and/or the C-arm, which provide limited information about teams' actual radiation safety behaviors. This study aimed to develop a rating scale to evaluate team radiation safety behaviors more accurately and investigate its reliability.

MATERIALS AND METHODS

A modified Delphi consensus was organized involving European vascular surgeons (VS), interventional radiologists, and interventional cardiologists. Initial items and anchors were drafted a priori and rated using five-point Likert scales. Participants could suggest additional items or adjustments. Consensus was defined as ≥ 80% agreement (rating ≥ 4) with Cronbach's alpha ≥ .80. Two VS with expertise in radiation safety evaluated 15 video-recorded endovascular repairs of infrarenal aortic aneurysms (EVAR) to assess usability, inter and intrarater reliability.

RESULTS

Thirty-one of 46 invited specialists completed three rating rounds to generate the final rating scale. Five items underwent major adjustments. In the final round, consensus was achieved for all items (alpha = .804; agreement > 87%): 'Pre-procedural planning', 'Preparation in angiosuite/operating room', 'Shielding equipment', 'Personal protective equipment', 'Position of operator/team', 'Radiation usage awareness', 'C-arm handling', 'Adjusting image quality', 'Additional dose reducing functions', 'Communication/leadership', and 'Overall radiation performance and ALARA principle'. All EVARs were rated, yielding excellent Cronbach's alpha (.877) with acceptable interrater and excellent intrarater reliability (ICC = .782; ICC = .963, respectively).

CONCLUSION

A reliable framework was developed to assess radiation safety behaviors in endovascular practice and provide teams with formative feedback. The final scale is provided in this publication.

Machine learning-based operation skills assessment with vascular difficulty index for vascular intervention surgery

An accurate assessment of surgical operation skills is essential for improving the vascular intervention surgical outcome and the performance of endovascular surgery robots. In existing studies, subjective and objective assessments of surgical operation skills use a variety of indicators, such as the operation speed and operation smoothness. However, the vascular conditions of particular patients have not been considered in the assessment, leading to deviations in the evaluation. Therefore, in this paper, an operation skills assessment method including the vascular difficulty level index for catheter insertion at the aortic arch in endovascular surgery is proposed. First, the model describing the difficulty of the vascular anatomical structure is established with characteristics of different aortic arch branches based on machine learning. Afterwards, the vascular difficulty level is set as an objective index combined with operating characteristics extracted from the operations performed by surgeons to evaluate the surgical operation skills at the aortic arch using machine learning. The accuracy of the assessment improves from 86.67 to 96.67% after inclusion of the vascular difficulty as an evaluation indicator to more objectively and accurately evaluate skills. The method described in this paper can be adopted to train novice surgeons in endovascular surgery, and for studies of vascular interventional surgery robots. Graphical abstract Operation skill assessment with vascular difficulty for vascular interventional surgery.

Qualitative and Quantitative Assessment of Technical Skills in Percutaneous Coronary Intervention: In Vivo Porcine Studies

OBJECTIVE

Technical skill assessment plays an important role in the professional development of an interventionalist in percutaneous coronary intervention (PCI). However, most of the traditional assessment methods are time consuming and subjective. This paper aims to develop objective assessment techniques.

METHODS

In this study, a natural-behavior-based assessment framework is proposed to qualitatively and quantitatively assess technical skills in PCI. In vivo porcine studies were conducted to deliver a medical guidewire to two target coronaries of left circumflex arteries by six novice and four expert interventionalists. Simultaneously, four types of natural behaviors (i.e., hand motion, proximal force, muscle activity, and finger motion) were acquired from the subjects' dominant hand and arm. The features extracted from the behaviors of different skill-level groups were compared using the Mann-Whitney U-test for effective behavior selection. The effective ones were further applied in the Gaussian-mixture-model-based qualitative assessment and Mahalanobis-distance-based quantitative assessment.

RESULTS

The qualitative assessment achieves an accuracy of 92% to distinguish the novice and expert attempts, which is significantly higher than that of using single guidewire motions. Furthermore, the quantitative assessment can assign objective and effective scores for all attempts, indicating high correlation ( R = 0.9225) to those obtained by traditional methods.

CONCLUSION

The objective, effective, and comprehensive assessment of technical skills can be provided by qualitatively and quantitatively analyzing interventionalists' natural behaviors in PCI.

SIGNIFICANCE

This paper suggests a novel approach for the technical skill assessment and the promising results demonstrate the great importance and effectiveness of the proposed method for promoting the development of objective assessment techniques.

A Multicentre Trial of Patient specific Rehearsal Prior to EVAR: Impact on Procedural Planning and Team Performance

OBJECTIVE

Patient specific rehearsal (PsR) prior to endovascular aneurysm repair (EVAR) enables the endovascular team to practice and evaluate the procedure prior to treating the real patient. This multicentre trial aimed to evaluate the utility of PsR prior to EVAR as a pre-operative planning and briefing tool.

MATERIAL AND METHODS

Patients with an aneurysm suitable for EVAR were randomised to pre-operative or post-operative PsR. Before and after the PsR, the lead implanter completed a questionnaire to identify any deviation from the initial treatment plan. All team members completed a questionnaire evaluating realism, technical issues, and human factor aspects pertinent to PsR. Technical and human factor skills, and technical and clinical success rates were compared between the randomised groups.

RESULTS

100 patients were enrolled between September 2012 and June 2014. The plan to visualise proximal and distal landing zones was adapted in 27/50 (54%) and 38/50 (76%) cases, respectively. The choice of the main body, contralateral limb, or iliac extensions was adjusted in 8/50 (16%), 17/50 (34%), and 14/50 (28%) cases, respectively. At least one of the abovementioned parameters was changed in 44/50 (88%) cases. For 100 EVAR cases, 199 subjective questionnaires post-PsR were completed. PsR was considered to be useful for selecting the optimal C-arm angulation (median 4, IQR 4-5) and was recognised as a helpful tool for team preparation (median 4, IQR 4-4), to improve communication (median 4, IQR 3-4), and encourage confidence (median 4, IQR 3-4). Technical and human factor skills and technical and initial clinical success rates were similar between the randomisation groups.

CONCLUSION

PsR prior to EVAR has a significant impact on the treatment plan and may be useful as a pre-operative planning and briefing tool. Subjective ratings indicate that this technology may facilitate planning of optimal C-arm angulation and improve non-technical skills.

TRIAL REGISTRATION

URL://www.clinicaltrials.gov. Unique identifier: NCT01632631.

Fine-motor skills testing and prediction of endovascular performance

BACKGROUND

Performing endovascular procedures requires good control of fine-motor digital movements and hand-eye coordination. Objective assessment of such skills is difficult. Trainees acquire control of catheter/wire movements at various paces. However, little is known to what extent talent plays for novice candidates at entry to practice.

PURPOSE

To study the association between performance in a novel aptitude test of fine-motor skills and performance in simulated procedures.

MATERIAL AND METHODS

The test was based on manual course-tracking using a proprietary hand-operated roller-bar device coupled to a personal computer with monitor view rotation. A total of 40 test repetitions were conducted separately with each hand. Test scores were correlated with simulator performance. Group A (n = 14), clinicians with various levels of endovascular experience, performed a simulated procedure of contralateral iliac artery stenting. Group B (n = 19), medical students, performed 10 repetitions of crossing a challenging aortic bifurcation in a simulator.

RESULTS

The test score differed markedly between the individuals in both groups, in particular with the non-dominant hand. Group A: the test score with the non-dominant hand correlated significantly with simulator performance assessed with the global rating scale SAVE (R = -0.69, P = 0.007). There was no association observed from performances with the dominant hand. Group B: there was no significant association between the test score and endovascular skills acquisition neither with the dominant nor with the non-dominant hand.

CONCLUSION

Clinicians with increasing levels of endovascular technical experience had developed good fine-motor control of the non-dominant hand, in particular, that was associated with good procedural performance in the simulator. The aptitude test did not predict endovascular skills acquisition among medical students, thus, cannot be suggested for selection of novice candidates. Procedural experience and practice probably supplant the influence of innate abilities (talent) over time.

Assessment of an introductory cervicocerebral catheter angiography learning program: a pilot study

BACKGROUND AND PURPOSE

There is no standardized curriculum currently available at most institutions for establishing procedural competency in trainees performing cervicocerebral angiography. The purpose of this study was to evaluate a simple learning program to supplement the teaching of basic cervicocerebral angiography.

MATERIALS AND METHODS

An 11-session interactive curriculum was implemented covering anatomic, clinical, and radiographic topics for the novice cervicocerebral angiographer. The target learner was the neuroradiology fellow. Data were gathered regarding fellow comfort level on topics relating to cervicocerebral angiography by using a 5-point Likert scale. Improvement in scores on knowledge-based questions after completion of the curriculum was calculated (McNemar test). Trainee-perceived utility of the program was also recorded by using a 5-point Likert scale. Focus sessions were held at the completion of the curriculum to gather feedback regarding the strengths and weaknesses of the program from participants.

RESULTS

Ten subjects were enrolled in this pilot study for 3 years. Topics where participants reported a poor initial comfort level (4 or higher) included selection of injection rates and volumes and reformation of reverse-curve catheters. Trainees demonstrated a statistically significant change in the distribution of scores of 29.3% (49.4%-78.7% correct response rate, P < .0001). The average perceived utility was 1.5 (1 = most useful, 5 = least useful).

CONCLUSIONS

This simple learning program was a useful adjunct to the training of fellows in diagnostic cervicocerebral angiography, resulting in quantitative improvements in knowledge.

Use of performance curves in estimating number of procedures required to achieve proficiency in coronary angiography

BACKGROUND

Current guidelines in cardiology training programs recommend 100-300 coronary angiography procedures for certification. We aimed to assess the number of procedures needed to reach sufficient proficiency.

METHODS

Procedure time, fluoroscopy time, dose area product (DAP), and contrast media volume were used as indicators of quality of performance. We analyzed data from 4,200 coronary angiographies. Performance curves of seven trainees were compared with recommended reference levels and to those of seven interventional cardiologists.

RESULTS

On average, the number of procedures needed for trainees to reach recommended reference levels was estimated as 226 and 353, for DAP and use of contrast media, respectively. After 300 procedures, trainees' procedure time, fluoroscopy time, DAP, and contrast media volume were significantly higher compared with experts' performance, P < 0.001 for all parameters. To approach the experts' level of DAP and contrast media use, trainees need 394 and 588 procedures, respectively. Performance curves showed large individual differences in the development of competence.

CONCLUSION

On average, trainees needed 300 procedures to reach sufficient level of proficiency, and this is in accordance with current guidelines. However, because of large individual differences, performance curves might be useful in monitoring individual trainees' progress and ensure documentation of sufficient competence when dealing with patients at risk.