Assessment of laparoscopic skills based on force and motion parameters

Energy-Based Metrics for Arthroscopic Skills Assessment

Minimally invasive skills assessment methods are essential in developing efficient surgical simulators and implementing consistent skills evaluation. Although numerous methods have been investigated in the literature, there is still a need to further improve the accuracy of surgical skills assessment. Energy expenditure can be an indication of motor skills proficiency. The goals of this study are to develop objective metrics based on energy expenditure, normalize these metrics, and investigate classifying trainees using these metrics. To this end, different forms of energy consisting of mechanical energy and work were considered and their values were divided by the related value of an ideal performance to develop normalized metrics. These metrics were used as inputs for various machine learning algorithms including support vector machines (SVM) and neural networks (NNs) for classification. The accuracy of the combination of the normalized energy-based metrics with these classifiers was evaluated through a leave-one-subject-out cross-validation. The proposed method was validated using 26 subjects at two experience levels (novices and experts) in three arthroscopic tasks. The results showed that there are statistically significant differences between novices and experts for almost all of the normalized energy-based metrics. The accuracy of classification using SVM and NN methods was between 70% and 95% for the various tasks. The results show that the normalized energy-based metrics and their combination with SVM and NN classifiers are capable of providing accurate classification of trainees. The assessment method proposed in this study can enhance surgical training by providing appropriate feedback to trainees about their level of expertise and can be used in the evaluation of proficiency.

Analysis of Energy-Based Metrics for Laparoscopic Skills Assessment

OBJECTIVE

The complexity of minimally invasive surgery (MIS) requires that trainees practice MIS skills in numerous training sessions. The goal of these training sessions is to learn how to move the instruments smoothly without damaging the surrounding tissue and achieving operative tasks with accuracy. In order to enhance the efficiency of these training sessions, the proficiency of the trainees should be assessed using an objective assessment method. Several performance metrics have been proposed and analyzed for MIS tasks. The differentiation of various levels of expertise is limited without the presence of an external evaluator.

METHODS

In this study, novel objective performance metrics are proposed based on mechanical energy expenditure and work. The three components of these metrics are potential energy, kinetic energy, and work. These components are optimally combined through both one-step and two-step methods. Evaluation of these metrics is accomplished for suturing and knot-tying tasks based on the performance of 30 subjects across four levels of experience.

RESULTS

The results of this study show that the one-step combined metric provides 47 and 60 accuracy in determining the level of expertise of subjects for the suturing and knot-tying tasks, respectively. The two-step combined metric provided 67 accuracy for both of the tasks studied.

CONCLUSION

The results indicate that energy expenditure is a useful metric for developing objective and efficient assessment methods.

SIGNIFICANCE

These metrics can be used to evaluate and determine the proficiency levels of trainees, provide feedback and, consequently, enhance surgical simulators.

Objective Assessment of Surgical Technical Skill and Competency in the Operating Room

Training skillful and competent surgeons is critical to ensure high quality of care and to minimize disparities in access to effective care. Traditional models to train surgeons are being challenged by rapid advances in technology, an intensified patient-safety culture, and a need for value-driven health systems. Simultaneously, technological developments are enabling capture and analysis of large amounts of complex surgical data. These developments are motivating a "surgical data science" approach to objective computer-aided technical skill evaluation (OCASE-T) for scalable, accurate assessment; individualized feedback; and automated coaching. We define the problem space for OCASE-T and summarize 45 publications representing recent research in this domain. We find that most studies on OCASE-T are simulation based; very few are in the operating room. The algorithms and validation methodologies used for OCASE-T are highly varied; there is no uniform consensus. Future research should emphasize competency assessment in the operating room, validation against patient outcomes, and effectiveness for surgical training.

Navigation forces during wrist arthroscopy: assessment of expert levels

PURPOSE

To facilitate effective and efficient training in skills laboratory, objective metrics can be used. Forces exerted on the tissues can be a measure of safe tissue manipulation. To provide feedback during training, expert threshold levels need to be determined. The purpose of this study was to define the magnitude and the direction of navigation forces used during arthroscopic inspection of the wrist.

METHODS

We developed a set-up to mount a cadaver wrist to a 3D force platform that allowed measurement of the forces exerted on the wrist. Six experts in wrist arthroscopy performed two tasks: (1) Introduction of the camera and visualization of the hook. (2) Navigation through the wrist with visualization of five anatomic structures. The magnitude (Fabs) and direction of force were recorded, with the direction defined as α being the angle in the vertical plane and β being the angle in the horizontal plane. The 10th-90th percentile of the data were used to set threshold levels for training.

RESULTS

The results show distinct force patterns for each of the anatomic landmarks. Median Fabs of the navigation task is 3.8 N (1.8-7.3), α is 3.60 (-54-44) and β is 260 (0-72).

CONCLUSION

Unique expert data on navigation forces during wrist arthroscopy were determined. The defined maximum allowable navigation force of 7.3 N (90th percentile) can be used in providing feedback on performance during skills training. The clinical value is that this study contributes to objective assessment of skills levels.

Prospective Evaluation of Innovative Force Assessing Firmware in Simulation to Improve the Technical Competence of Surgical Trainees

INTRODUCTION

Suturing is an integral part of all surgeries. In minimal access surgery, the force exerted is based only on visual perception (tautness of the thread and degree of tissue deformation). An unbalanced suture force can cause tissue rupture or cut-through resulting in avoidable morbidity and mortality. There is a need to find ways of improving surgical dexterity and finesse without adversely affecting patient outcomes.

AIM

We aimed to calculate the knot-tying force in minimal access pancreatic surgery (MAPS) performed by experienced surgeons (ES) and use this information to develop a surgical suturing model to train the surgical trainees. We have developed a firmware for force sensor calibration and post-data analysis, using which we aimed to compare the differences in forces applied by a trainee as compared to ES.

RESULTS

Our technology showed that, as compared to the ES, the trainee's (TS) knot was unbalanced with significant differences in force applied per knot for each of the knots (P < 0.01). The shape of the Force curve for each suture was also different for the TS as compared to the ES. After using the training tool, the forces applied by the TS and the Force curve for the whole suture were similar to those of the ES.

CONCLUSION

Our firmware promises to be an excellent training tool for organ anastomosis. Considering the complexity and likely complications of MAPS, it is a sine qua non that the surgeon be highly experienced and skilled. Surgical simulation is attractive because it avoids the use of patients for skills practice and provides relevant technical training for trainees before they can safely operate on humans.

Objective Assessment of Laparoscopic Force and Psychomotor Skills in a Novel Virtual Reality-Based Haptic Simulator

BACKGROUND

Most of the commercially available virtual reality-based laparoscopic simulators do not effectively evaluate combined psychomotor and force-based laparoscopic skills. Consequently, the lack of training on these critical skills leads to intraoperative errors.

OBJECTIVES

To assess the effectiveness of the novel virtual reality-based simulator, this study analyzed the combined psychomotor (i.e., motion or movement) and force skills of residents and expert surgeons. The study also examined the effectiveness of real-time visual force feedback and tool motion during training.

DESIGN

Bimanual fundamental (i.e., probing, pulling, sweeping, grasping, and twisting) and complex tasks (i.e., tissue dissection) were evaluated. In both tasks, visual feedback on applied force and tool motion were provided. The skills of the participants while performing the early tasks were assessed with and without visual feedback. Participants performed 5 repetitions of fundamental and complex tasks. Reaction force and instrument acceleration were used as metrics.

SETTING

Surgical Gastroenterology, Government Stanley Medical College and Hospital; Institute of Surgical Gastroenterology, Madras Medical College and Rajiv Gandhi Government General Hospital.

PARTICIPANTS

Residents (N = 25; postgraduates and surgeons with <2 years of laparoscopic surgery) and expert surgeons (N = 25; surgeons with >4 and ≤10 years of laparoscopic surgery).

RESULTS

Residents applied large forces compared with expert surgeons and performed abrupt tool movements (p < 0.001). However, visual + haptic feedback improved the performance of residents (p < 0.001). In complex tasks, visual + haptic feedback did not influence the applied force of expert surgeons, but influenced their tool motion (p < 0.001). Furthermore, in complex tissue sweeping task, expert surgeons applied more force, but were within the tissue damage limits. In both groups, exertion of large forces and abrupt tool motion were observed during grasping, probing or pulling, and tissue sweeping maneuvers (p < 0.001).

CONCLUSIONS

Modern day curriculum-based training should evaluate the skills of residents with robust force and psychomotor-based exercises for proficient laparoscopy. Visual feedback on force and motion during training has the potential to enhance the learning curve of residents.

Energy-based metrics for laparoscopic skills assessment

The growing popularity of minimally invasive surgery (MIS) can be attributed to its advantages, which include reduced post-operative pain, a shorter hospital stay, and faster recovery. However, MIS requires extensive training for surgeons to become experts in their field of practice. Different assessment methods have been proposed for evaluating the performance of surgeons and residents on surgical simulators. Nonetheless, optimal objective performance measures are still lacking. In this study, three metrics for minimally invasive skills assessment are proposed based on energy expenditure: work, potential energy and kinetic energy. In order to evaluate these metrics, two laparoscopic tasks consisting of suturing and knot-tying are investigated, involving expert and novice subjects. This study shows that measures based on energy expenditure can be used for skills assessment: all three metrics can discriminate between experts and novices for the two tasks investigated here. These measures can also reflect the efficiency of subjects when performing MIS tasks. Further modification and investigation of these metrics can extend their use to different tasks and for discriminating between various levels of experience.

Improving training of laparoscopic tissue manipulation skills using various visual force feedback types

Background

Visual force feedback allows trainees to learn laparoscopic tissue manipulation skills. The aim of this experimental study was to find the most efficient visual force feedback method to acquire these skills. Retention and transfer validity to an untrained task were assessed.

Methods

Medical students without prior experience in laparoscopy were randomized in three groups: Constant Force Feedback (CFF) (N = 17), Bandwidth Force Feedback (BFF) (N = 16) and Fade-in Force Feedback (N = 18). All participants performed a pretest, training, post-test and follow-up test. The study involved two dissimilar tissue manipulation tasks, one for training and one to assess transferability. Participants performed six trials of the training task. A force platform was used to record several force parameters.

Results

A paired-sample t test showed overall lower force parameter outcomes in the post-test compared to the pretest (p < .001). A week later, the force parameter outcomes were still significantly lower than found in the pretest (p < .005). Participants also performed the transfer task in the post-test (p < .02) and follow-up (p < .05) test with lower force parameter outcomes compared to the pretest. A one-way MANOVA indicated that in the post-test the CFF group applied 50 % less Mean Absolute Nonzero Force (p = .005) than the BFF group.

Conclusion

All visual force feedback methods showed to be effective in decreasing tissue manipulation force as no major differences were found between groups in the post and follow-up trials. The BFF method is preferred for it respects individual progress and minimizes distraction.