How can surgical skills in laparoscopic colon surgery be objectively assessed?-a scoping review

Vascular pedicle dissection time in laparoscopic colectomies as a novel marker of surgical skill: a prospective cohort study

BACKGROUND

Outcomes after colorectal resections depend on patient, pathology and operative factors. Existing validated surgical skills scores (such as the competency assessment tool (CAT)) are directly correlated with outcomes but are time-consuming to administer, limiting their clinical utility. The vascular pedicle dissection time (VPDT) is a novel, simple surgical skill assessment measure with the potential for computer vision automation. This study aimed to assess the VPDT and benchmark it against the CAT score.

METHODS

A prospective multicentre study was performed in New Zealand, recording videos of laparoscopic colorectal resections. Patient, operation and histology characteristics were also collected. The VPDT was calculated from retraction of the vascular pedicle to completion of medial dissection, including vascular division. Each laparoscopic video was scored by two independent colorectal surgeons, and the median CAT score was grouped into tertiles.

RESULTS

In total, 154 patients were included between December 2020 and November 2023 (74 (48.1%) right-sided and 80 (51.9%) left-sided resections). Median VPDT was significantly different between the CAT score groups for the right-sided resections (lower, 15 min; middle, 13 min; higher, 10 min; p = 0.036) and the left-sided resections (lower, 46 min; middle, 40 min; higher, 26 min; p =  < 0.001). There was no significant difference in R1 resection, anastomotic leak rate, the occurrence of Clavien-Dindo > 3 complications or re-admission between the CAT groups.

CONCLUSIONS

This study showed that the VPDT was inversely correlated with the CAT score, indicating that it quantifies operative technical skill. A current study is evaluating the suitability of VPDT for real-time measurement using computer vision algorithms. This could allow for automated assessment of surgeons' learning curve and skills.

Objective assessment tools in laparoscopic or robotic-assisted gynecological surgery: A systematic review

INTRODUCTION

There is a growing emphasis on proficiency-based progression within surgical training. To enable this, clearly defined metrics for those newly acquired surgical skills are needed. These can be formulated in objective assessment tools. The aim of the present study was to systematically review the literature reporting on available tools for objective assessment of minimally invasive gynecological surgery (simulated) performance and evaluate their reliability and validity.

MATERIAL AND METHODS

A systematic search (1989-2022) was conducted in MEDLINE, Embase, PubMed, Web of Science in accordance with PRISMA. The trial was registered with the Prospective Register of Systematic Reviews (PROSPERO) ID: CRD42022376552. Randomized controlled trials, prospective comparative studies, prospective single-group (with pre- and post-training assessment) or consensus studies that reported on the development, validation or usage of assessment tools of surgical performance in minimally invasive gynecological surgery, were included. Three independent assessors assessed study setting and validity evidence according to a contemporary framework of validity, which was adapted from Messick's validity framework. Methodological quality of included studies was assessed using the modified medical education research study quality instrument (MERSQI) checklist. Heterogeneity in data reporting on types of tools, data collection, study design, definition of expertise (novice vs. experts) and statistical values prevented a meaningful meta-analysis.

RESULTS

A total of 19 746 titles and abstracts were screened of which 72 articles met the inclusion criteria. A total of 37 different assessment tools were identified of which 13 represented manual global assessment tools, 13 manual procedure-specific assessment tools and 11 automated performance metrices. Only two tools showed substantive evidence of validity. Reliability and validity per tool were provided. No assessment tools showed direct correlation between tool scores and patient related outcomes.

CONCLUSIONS

Existing objective assessment tools lack evidence on predicting patient outcomes and suffer from limitations in transferability outside of the research environment, particularly for automated performance metrics. Future research should prioritize filling these gaps while integrating advanced technologies like kinematic data and AI for robust, objective surgical skill assessment within gynecological advanced surgical training programs.

Training and assessment for colorectal surgery and appendicectomy- a systematic review

AIM

There is currently an increased focus on competency-based training, in which training and assessment play a crucial role. The aim of this systematic review is to create an overview of hands-on training methods and assessment tools for appendicectomy and colon and rectal surgery procedures using either an open, laparoscopic or robot-assisted approach.

METHOD

A systematic review of Medline, Embase, Cochrane and Scopus databases was conducted following the PRISMA guidelines. We conducted the last search on 9 March 2023. All published papers describing hands-on training, evaluation of performance data and development of assessment tools were eligible. The quality of studies and the validity evidence of assessment tools are reported.

RESULTS

Fifty-one studies were identified. Laparoscopic assessment tools are abundant, but the literature still lacks good-quality assessment tools for open appendicectomy, robotic colectomy and open rectal surgery. Overall, there is a lack of discussion regarding the establishment of pass/fail standards and the consequences of assessment. Virtual reality simulation is used more for appendicectomy than colorectal procedures. Only a few of the studies investigating training were of acceptable quality. There is a need for high-quality studies in open and robotic-assisted colon surgery and all approaches to rectal surgery.

CONCLUSION

This review provides an overview of current training methods and assessment tools and identifies where more research is needed based on the quality of the studies and the current validity evidence.

Machine learning-based Automatic Evaluation of Tissue Handling Skills in Laparoscopic Colorectal Surgery: A Retrospective Experimental Study

OBJECTIVE

To develop a machine learning model that automatically quantifies the spread of blood in the surgical field using intraoperative videos of laparoscopic colorectal surgery and evaluate whether the index measured with the developed model can be used to assess tissue handling skill.

BACKGROUND

Although skill evaluation is crucial in laparoscopic surgery, existing evaluation systems suffer from evaluator subjectivity and are labor-intensive. Therefore, automatic evaluation using machine learning is potentially useful.

MATERIALS AND METHODS

In this retrospective experimental study, we used training data with annotated labels of blood or non-blood pixels on intraoperative images to develop a machine learning model to classify pixel RGB values into blood and non-blood. The blood pixel count per frame (the total number of blood pixels throughout a surgery divided by the number of frames) was compared among groups of surgeons with different tissue handling skills.

RESULTS

The overall accuracy of the machine learning model for the blood classification task was 85.7%. The high tissue handling skill group had the lowest blood pixel count per frame, and the novice surgeon group had the highest count (mean [SD]: high tissue handling skill group 20972.23 [19287.05] vs. low tissue handling skill group 34473.42 [28144.29] vs. novice surgeon group 50630.04 [42427.76], P <0.01). The difference between any 2 groups was significant.

CONCLUSIONS

We developed a machine learning model to measure blood pixels in laparoscopic colorectal surgery images using RGB information. The blood pixel count per frame measured with this model significantly correlated with surgeons' tissue handling skills.

Video-based tools for surgical quality assessment of technical skills in laparoscopic procedures: a systematic review

BACKGROUND

Quality of surgery has substantial impact on both short- and long-term clinical outcomes. This stresses the need for objective surgical quality assessment (SQA) for education, clinical practice and research purposes. The aim of this systematic review was to provide a comprehensive overview of all video-based objective SQA tools in laparoscopic procedures and their validity to objectively assess surgical performance.

METHODS

PubMed, Embase.com and Web of Science were systematically searched by two reviewers to identify all studies focusing on video-based SQA tools of technical skills in laparoscopic surgery performed in a clinical setting. Evidence on validity was evaluated using a modified validation scoring system.

RESULTS

Fifty-five studies with a total of 41 video-based SQA tools were identified. These tools were used in 9 different fields of laparoscopic surgery and were divided into 4 categories: the global assessment scale (GAS), the error-based assessment scale (EBAS), the procedure-specific assessment tool (PSAT) and artificial intelligence (AI). The number of studies focusing on these four categories were 21, 6, 31 and 3, respectively. Twelve studies validated the SQA tool with clinical outcomes. In 11 of those studies, a positive association between surgical quality and clinical outcomes was found.

CONCLUSION

This systematic review included a total of 41 unique video-based SQA tools to assess surgical technical skills in various domains of laparoscopic surgery. This study suggests that validated SQA tools enable objective assessment of surgical performance with relevance for clinical outcomes, which can be used for training, research and quality improvement programs.