Assessment methods in laparoscopic colorectal surgery: a systematic review of available instruments

BACKGROUND

Laparoscopic surgery has become the golden standard for many procedures, requiring new skills and training methods. The aim of this review is to appraise literature on assessment methods for laparoscopic colorectal procedures and quantify these methods for implementation in surgical training.

MATERIALS AND METHODS

PubMed, Embase and Cochrane Central Register of Controlled Trials databases were searched in October 2022 for studies reporting learning and assessment methods for laparoscopic colorectal surgery. Quality was scored using the Downs and Black checklist. Included articles were categorized in procedure-based assessment methods and non-procedure-based assessment methods. A second distinction was made between capability for formative and/or summative assessment.

RESULTS

In this systematic review, nineteen studies were included. These studies showed large heterogeneity despite categorization. Median quality score was 15 (range 0-26). Fourteen studies were categorized as procedure-based assessment methods (PBA), and five as non-procedure-based assessment methods. Three studies were applicable for summative assessment.

CONCLUSIONS

The results show a considerable diversity in assessment methods with varying quality and suitability. To prevent a sprawl of assessment methods, we argue for selection and development of available high-quality assessment methods. A procedure-based structure combined with an objective assessment scale and possibility for summative assessment should be cornerstones.

Validity, reliability and support for implementation of independence-scaled procedural assessment in laparoscopic surgery

Background

There is no widely used method to evaluate procedure-specific laparoscopic skills. The first aim of this study was to develop a procedure-based assessment method. The second aim was to compare its validity, reliability and feasibility with currently available global rating scales (GRSs).

Methods

An independence-scaled procedural assessment was created by linking the procedural key steps of the laparoscopic cholecystectomy to an independence scale. Subtitled and blinded videos of a novice, an intermediate and an almost competent trainee, were evaluated with GRSs (OSATS and GOALS) and the independence-scaled procedural assessment by seven surgeons, three senior trainees and six scrub nurses. Participants received a short introduction to the GRSs and independence-scaled procedural assessment before assessment. The validity was estimated with the Friedman and Wilcoxon test and the reliability with the intra-class correlation coefficient (ICC). A questionnaire was used to evaluate user opinion.

Results

Independence-scaled procedural assessment and GRS scores improved significantly with surgical experience (OSATS p = 0.001, GOALS p < 0.001, independence-scaled procedural assessment p < 0.001). The ICCs of the OSATS, GOALS and independence-scaled procedural assessment were 0.78, 0.74 and 0.84, respectively, among surgeons. The ICCs increased when the ratings of scrub nurses were added to those of the surgeons. The independence-scaled procedural assessment was not considered more of an administrative burden than the GRSs (p = 0.692).

Discussion/conclusion

A procedural assessment created by combining procedural key steps to an independence scale is a valid, reliable and acceptable assessment instrument in surgery. In contrast to the GRSs, the reliability of the independence-scaled procedural assessment exceeded the threshold of 0.8, indicating that it can also be used for summative assessment. It furthermore seems that scrub nurses can assess the operative competence of surgical trainees.