Qualitative and Quantitative Assessment of Technical Skills in Percutaneous Coronary Intervention: In Vivo Porcine Studies

Learning Skill Characteristics From Manipulations

Percutaneous coronary intervention (PCI) has increasingly become the main treatment for coronary artery disease. The procedure requires high experienced skills and dexterous manipulations. However, there are few techniques to model PCI skill so far. In this study, a learning framework with local and ensemble learning is proposed to learn skill characteristics of different skill-level subjects from their PCI manipulations. Ten interventional cardiologists (four experts and six novices) were recruited to deliver a medical guidewire to two target arteries on a porcine model for in vivo studies. Simultaneously, translation and twist manipulations of thumb, forefinger, and wrist are acquired with electromagnetic (EM) and fiber-optic bend (FOB) sensors, respectively. These behavior data are then processed with wavelet packet decomposition (WPD) under 1-10 levels for feature extraction. The feature vectors are further fed into three candidate individual classifiers in the local learning layer. Furthermore, the local learning results from different manipulation behaviors are fused in the ensemble learning layer with three rule-based ensemble learning algorithms. In subject-dependent skill characteristics learning, the ensemble learning can achieve 100% accuracy, significantly outperforming the best local result (90%). Furthermore, ensemble learning can also maintain 73% accuracy in subject-independent schemes. These promising results demonstrate the great potential of the proposed method to facilitate skill learning in surgical robotics and skill assessment in clinical practice.

Learning-Based Multimodal Information Fusion and Behavior Recognition of Vascular Interventionists' Operating Skills

The operating skills of vascular interventionists have an important impact on the effect of surgery. However, current research on behavior recognition and skills learning of interventionists' operating skills is limited. In this study, an innovative deep learning-based multimodal information fusion architecture is proposed for recognizing and analyzing eight common operating behaviors of interventionists. An experimental platform integrating four modal sensors is used to collect multimodal data from interventionists. The ANOVA and Manner-Whitney tests is used for relevance analysis of the data. The analysis results demonstrate that there is almost no significant difference ( p <0.001) between the actions related to the unimodal data, which cannot be used for accurate behavior recognition. Therefore, a study of the fusion architecture based on the existing machine learning classifier and the proposed deep learning fusion architecture is carried out. The research findings indicate that the proposed deep learning-based fusion architecture achieves an impressive overall accuracy of 98.5%, surpassing both the machine learning classifier (93.51%) and the unimodal data (90.05%). The deep learning-based multimodal information fusion architecture proves the feasibility of behavior recognition and skills learning of interventionist's operating skills. Furthermore, the application of deep learning-based multimodal fusion technology of surgeon's operating skills will help to improve the autonomy and intelligence of surgical robotic systems.

Relating process and outcome metrics for meaningful and interpretable cannulation skill assessment: A machine learning paradigm

BACKGROUND AND OBJECTIVES

The quality of healthcare delivery depends directly on the skills of clinicians. For patients on hemodialysis, medical errors or injuries caused during cannulation can lead to adverse outcomes, including potential death. To promote objective skill assessment and effective training, we present a machine learning approach, which utilizes a highly-sensorized cannulation simulator and a set of objective process and outcome metrics.

METHODS

In this study, 52 clinicians were recruited to perform a set of pre-defined cannulation tasks on the simulator. Based on data collected by sensors during their task performance, the feature space was then constructed based on force, motion, and infrared sensor data. Following this, three machine learning models- support vector machine (SVM), support vector regression (SVR), and elastic net (EN)- were constructed to relate the feature space to objective outcome metrics. Our models utilize classification based on the conventional skill classification labels as well as a new method that represents skill on a continuum.

RESULTS

With less than 5% of trials misplaced by two classes, the SVM model was effective in predicting skill based on the feature space. In addition, the SVR model effectively places both skill and outcome on a fine-grained continuum (versus discrete divisions) that is representative of reality. As importantly, the elastic net model enabled the identification of a set of process metrics that highly impact outcomes of the cannulation task, including smoothness of motion, needle angles, and pinch forces.

CONCLUSIONS

The proposed cannulation simulator, paired with machine learning assessment, demonstrates definite advantages over current cannulation training practices. The methods presented here can be adopted to drastically increase the effectiveness of skill assessment and training, thereby potentially improving clinical outcomes of hemodialysis treatment.

An Image Information-Based Objective Assessment Method of Technical Manipulation Skills for Intravascular Interventions

The clinical success of vascular interventional surgery relies heavily on a surgeon's catheter/guidewire manipulation skills and strategies. An objective and accurate assessment method plays a critical role in evaluating the surgeon's technical manipulation skill level. Most of the existing evaluation methods incorporate the use of information technology to find more objective assessment models based on various metrics. However, in these models, sensors are often attached to the surgeon's hands or to interventional devices for data collection, which constrains the surgeon's operational movements or exerts an influence on the motion trajectory of interventional devices. In this paper, an image information-based assessment method is proposed for the evaluation of the surgeon's manipulation skills without the requirement of attaching sensors to the surgeon or catheters/guidewires. Surgeons are allowed to use their natural bedside manipulation skills during the data collection process. Their manipulation features during different catheterization tasks are derived from the motion analysis of the catheter/guidewire in video sequences. Notably, data relating to the number of speed peaks, slope variations, and the number of collisions are included in the assessment. Furthermore, the contact forces, resulting from interactions between the catheter/guidewire and the vascular model, are sensed by a 6-DoF F/T sensor. A support vector machine (SVM) classification framework is developed to discriminate the surgeon's catheterization skill levels. The experimental results demonstrate that the proposed SVM-based assessment method can obtain an accuracy of 97.02% to distinguish between the expert and novice manipulations, which is higher than that of other existing research achievements. The proposed method has great potential to facilitate skill assessment and training of novice surgeons in vascular interventional surgery.

A Multilayer and Multimodal-Fusion Architecture for Simultaneous Recognition of Endovascular Manipulations and Assessment of Technical Skills

The clinical success of the percutaneous coronary intervention (PCI) is highly dependent on endovascular manipulation skills and dexterous manipulation strategies of interventionalists. However, the analysis of endovascular manipulations and related discussion for technical skill assessment are limited. In this study, a multilayer and multimodal-fusion architecture is proposed to recognize six typical endovascular manipulations. The synchronously acquired multimodal motion signals from ten subjects are used as the inputs of the architecture independently. Six classification-based and two rule-based fusion algorithms are evaluated for performance comparisons. The recognition metrics under the determined architecture are further used to assess technical skills. The experimental results indicate that the proposed architecture can achieve the overall accuracy of 96.41%, much higher than that of a single-layer recognition architecture (92.85%). In addition, the multimodal fusion brings significant performance improvement in comparison with single-modal schemes. Furthermore, the K -means-based skill assessment can obtain an accuracy of 95% to cluster the attempts made by different skill-level groups. These hopeful results indicate the great possibility of the architecture to facilitate clinical skill assessment and skill learning.

A Unified Framework for Multi-Guidewire Endpoint Localization in Fluoroscopy Images

OBJECTIVE

In this paper, Keypoint Localization Region-based CNN (KL R-CNN) is proposed, which can simultaneously accomplish the guidewire detection and endpoint localization in a unified model.

METHODS

KL R-CNN modifies Mask R-CNN by replacing the mask branch with a novel keypoint localization branch. Besides, some settings of Mask R-CNN are also modified to generate the keypoint localization results at a higher detail level. At the same time, based on the existing metrics of Average Precision (AP) and Percentage of Correct Keypoints (PCK), a new metric named AP PCK is proposed to evaluate the overall performance on the multi-guidewire endpoint localization task. Compared with existing metrics, AP PCK is easy to use and its results are more intuitive.

RESULTS

Compared with existing methods, KL R-CNN has better performance when the threshold is loose, reaching a mean AP PCK of 90.65% when the threshold is 9 pixels.

CONCLUSION

KL R-CNN achieves the state-of-the-art performance on the multi-guidewire endpoint localization task and has application potentials.

SIGNIFICANCE

KL R-CNN can achieve the localization of guidewire endpoints in fluoroscopy images, which is a prerequisite for computer-assisted percutaneous coronary intervention. KL R-CNN can also be extended to other multi-instrument localization tasks.

Research Progress on the Early Monitoring of Pine Wilt Disease Using Hyperspectral Techniques

Pine wilt disease (PWD) caused by pine wood nematode (PWN, Bursaphelenchus xylophilus) originated in North America and has since spread to Asia and Europe. PWN is currently a quarantine object in 52 countries. In recent years, pine wilt disease has caused considerable economic losses to the pine forest production industry in China, as it is difficult to control. Thus, one of the key strategies for controlling pine wilt disease is to identify epidemic points as early as possible. The use of hyperspectral cameras mounted on drones is expected to enable PWD monitoring over large areas of forest, and hyperspectral images can reflect different stages of PWD. The trend of applying hyperspectral techniques to the monitoring of pine wilt disease is analyzed, and the corresponding strategies to address the existing technical problems are proposed, such as data collection of early warning stages, needs of using unmanned aerial vehicles (UAVs), and establishment of models after preprocessing.

Machine learning-based operation skills assessment with vascular difficulty index for vascular intervention surgery

An accurate assessment of surgical operation skills is essential for improving the vascular intervention surgical outcome and the performance of endovascular surgery robots. In existing studies, subjective and objective assessments of surgical operation skills use a variety of indicators, such as the operation speed and operation smoothness. However, the vascular conditions of particular patients have not been considered in the assessment, leading to deviations in the evaluation. Therefore, in this paper, an operation skills assessment method including the vascular difficulty level index for catheter insertion at the aortic arch in endovascular surgery is proposed. First, the model describing the difficulty of the vascular anatomical structure is established with characteristics of different aortic arch branches based on machine learning. Afterwards, the vascular difficulty level is set as an objective index combined with operating characteristics extracted from the operations performed by surgeons to evaluate the surgical operation skills at the aortic arch using machine learning. The accuracy of the assessment improves from 86.67 to 96.67% after inclusion of the vascular difficulty as an evaluation indicator to more objectively and accurately evaluate skills. The method described in this paper can be adopted to train novice surgeons in endovascular surgery, and for studies of vascular interventional surgery robots. Graphical abstract Operation skill assessment with vascular difficulty for vascular interventional surgery.