Objective assessment of technique in laparoscopic colorectal surgery: what are the existing tools?

Procedure-based assessment implementation in colorectal surgery: a scoping review

BACKGROUND

Competency-based training (CBT) programs use procedure-based assessments (PBAs) to evaluate trainees' abilities to perform specific procedures in clinical settings, similar to Entrustable Professional Activities (EPAs). PBAs help determine trainees' readiness for advanced training levels. However, there is limited evidence on implementing colorectal-specific PBAs in surgical training schemes. This review aims to identify observed and perceived challenges to implementing PBAs in workplace settings.

METHODS

A scoping review following the Joanna Briggs Institute Protocol for Scoping Reviews (JBI-ScR) was conducted. Eligible studies provided evidence on the implementation, feasibility, and challenges of PBAs in colorectal surgery, including various study designs from retrospective to prospective.

RESULTS

Of the 80 screened studies, 75 were excluded based on exclusion criteria. Most of the included studies were conducted in national training institutions in the United Kingdom, assessing 778 colorectal procedures with specific PBAs. The main facilitators of implementing PBAs were structured assessments, focused assessors' training, and electronic forms usage.

CONCLUSION

This review offers insight into the practicality and feasibility of implementing PBAs in colorectal surgery. Identified challenges include the need for adequate assessor training and the time-consuming nature of the assessment. These findings could improve PBA implementation in colorectal surgery and enhance surgical education quality. However, the limited number of studies and existing literature heterogeneity call for more research to identify other gaps.

Assessing Laparoscopic Surgical Skills Using Similarity Network Models: A Pilot Study

Background: Medical devices are becoming more complex, and doctors need to learn quickly how to use new medical tools. However, it is challenging to objectively assess the fundamental laparoscopic surgical skill level and determine skill readiness for advancement. There is a lack of objective models to compare performance between medical trainees and experienced doctors. Methods: This article discusses the use of similarity network models for individual tasks and a combination of tasks to show the level of similarity between residents and medical students while performing each task and their overall laparoscopic surgical skill level using a medical device (eg laparoscopic instruments). When a medical student is connected to most residents, that student is competent to the next training level. Performance of sixteen participants (5 residents and 11 students) while performing 3 tasks in 3 different training schedules is used in this study. Results: The promising result shows the general positive progression of students over 4 training sessions. Our results also indicate that students with different training schedules have different performance levels. Students' progress in performing a task is quicker if the training sessions are held more closely compared to when the training sessions are far apart in time. Conclusions: This study provides a graph-based framework for evaluating new learners' performance on medical devices and their readiness for advancement. This similarity network method could be used to classify students' performance using similarity thresholds, facilitating decision-making related to training and progression through curricula.

Supporting laparoscopic general surgery training with digital technology: The United Kingdom and Ireland paradigm

Surgical training in the UK and Ireland has faced challenges following the implementation of the European Working Time Directive and postgraduate training reform. The health services are undergoing a digital transformation; digital technology is remodelling the delivery of surgical care and surgical training. This review aims to critically evaluate key issues in laparoscopic general surgical training and the digital technology such as virtual and augmented reality, telementoring and automated workflow analysis and surgical skills assessment. We include pre-clinical, proof of concept research and commercial systems that are being developed to provide solutions. Digital surgical technology is evolving through interdisciplinary collaboration to provide widespread access to high-quality laparoscopic general surgery training and assessment. In the future this could lead to integrated, context-aware systems that support surgical teams in providing safer surgical care.

A National study on the adoption of laparoscopic colorectal surgery in the elderly population: current state and value proposition

BACKGROUND

The economic and clinical benefits of laparoscopic colorectal surgery are proven, yet may be underutilized in appropriate cases, especially in the elderly. Since the elderly constitute the greatest colorectal surgical volume, our goal was to identify trends in utilization and impact of laparoscopy in this cohort.

METHODS

A national review of elective inpatient colorectal resections from the Premier Inpatient Database between 2010 and 2015 was performed. Patients were included if elderly (≥ 65 years), then grouped into open or laparoscopic procedures. The main outcome measures were trends in utilization by approach and total costs for the episode of care, length of stay (LOS), readmission, and complications by approach in the elderly. Multivariable regression models controlled for differences across platforms, adjusting for patient demographic, comorbidities and hospital characteristics.

RESULTS

In 70,655 elderly patients evaluated, laparoscopic adoption remained lower than open throughout the study period. Rates increased until 2013, then declined, with increasing rates of open surgery. Laparoscopy was associated with significantly lower mean total costs ($4012 less/case), complications and readmissions (36% and 33% less, respectively), and shorter LOS (2.6 less days) than open cases (all p < 0.0001). When complications occurred, they were less severe and the readmission episodes were less costly with laparoscopy than open colorectal surgery.

CONCLUSION

The adoption of laparoscopy in the elderly has lagged behind open surgery and even declined in recent years despite being associated with improved clinical outcomes and reduced cost. With this tremendous value proposition to increase use of laparoscopic surgery in the elderly, further work needs to evaluate root causes of the disparity.

Measuring Competence in Surgical Training through Assessment of Surgical Entrustable Professional Activities

OBJECTIVE

To assess competency in surgical residents with bundled assessments using a surgical entrustable professional activity (SEPA) framework.

DESIGN

A pilot study was conducted using a combination of validated assessment tools (multiple choice exam (MCE) questions from the Surgical Council on Resident Education [SCORE], the Clinical Assessment and Management Examination - Outpatient (CAMEO) form, the Virtual Surgical Patient (VSP) website, and a procedure-specific Operative Performance Rating System [OPRS]) to determine competency in surgical residents in the treatment of breast cancer and gallbladder disease, respectively.

SETTING

A large academic institution with a surgical training program in the Mid West of the United States.

PARTICIPANTS

A total of 10 categorical surgical residents were invited to participate. Five completed the breast surgical EPA (SEPA) and 5 different residents completed the gallbladder SEPA.

RESULTS

In terms of performance on the assessments, for the breast SEPA, scores did not appear to be related to PGY level, and residents' performance in general was the least strong on the MCE and the VSP case. The gallbladder SEPA showed a more expected pattern, distinguishing between junior and senior residents. As expected, all junior residents were required to remediate the OPRS assessment, while the senior residents passed. For the OPRS, senior level residents consistently were rated as "excellent" in terms of operative flow (5/5), while junior residents were all given a score of "good" (3/5).

CONCLUSIONS

Assessing competence among surgical residents has been a discussion for several years. Varying methods of assessing competence have been proposed, but surgical competence is presently defined in a very general way through both the ACGME and American Board of Surgery (ABS). Using a SEPA format, as proposed, we could ensure specific understanding of each graduating resident's ability. These results show that the SEPA may be a valid tool for defining and capturing multiple areas of competence that are associated with different disease processes.

EAES classification of intraoperative adverse events in laparoscopic surgery

BACKGROUND

Surgical outcomes are traditionally evaluated by post-operative data such as histopathology and morbidity. Although these outcomes are reported using accepted systems, their ability to influence operative performance is limited by their retrospective application. Interest in direct measurement of intraoperative events is growing but no available systems applicable to routine practice exist. We aimed to develop a structured, practical method to report intraoperative adverse events enacted during minimal access surgical procedures.

METHODS

A structured mixed methodology approach was adopted. Current intraoperative adverse event reporting practices and desirable system characteristics were sought through a survey of the EAES executive. The observational clinical human reliability analysis method was applied to a series of laparoscopic total mesorectal excision (TME) case videos to identify intraoperative adverse events. In keeping with survey results, observed events were further categorised into non-consequential and consequential, which were further subdivided into four levels based upon the principle of therapy required to correct the event. A second survey phase explored usability, acceptability, face and content validity of the novel classification.

RESULTS

217 h of TME surgery were analysed to develop and continually refine the five-point hierarchical structure. 34 EAES expert surgeons (69%) responded. The lack of an accepted system was the main barrier to routine reporting. Simplicity, reproducibility and clinical utility were identified as essential requirements. The observed distribution of intraoperative adverse events was 60.1% grade I (non-consequential), 37.1% grade II (minor corrective action), 2.4% grade III (major correction or change in post-operative care) and 0.1% grade IV (life threatening). 84% agreed with the proposed classification (Likert scale 4.04) and 92% felt it was applicable to their practice and incorporated all desirable characteristics.

CONCLUSION

A clinically applicable intraoperative adverse event classification, which is acceptable to expert surgeons, is reported and complements the objective assessment of minimal access surgical performance.

Development and implementation of a virtual reality laparoscopic colorectal training curriculum

BACKGROUND

Contemporary surgical training can be compromised by fewer practical opportunities. Simulation can fill this gap to optimize skills' development and progress monitoring. A structured virtual reality (VR) laparoscopic sigmoid colectomy curriculum is constructed and its validity and outcomes assessed.

METHODS

Parameters and thresholds were defined by analysing the performance of six expert surgeons completing the relevant module on the LAP Mentor simulator. Fourteen surgical trainees followed the curriculum, performance being recorded and analysed. Evidence of validity was assessed.

RESULTS

Time to complete procedure, number of movements of right and left instrument, and total path length of right and left instrument movements demonstrated evidence of validity and clear learning curves, with a median of 14 attempts needed to complete the curriculum.

CONCLUSIONS

A structured curriculum is proposed for training in laparoscopic sigmoid colectomy in a VR environment based on objective metrics in addition to expert consensus. Validity has been demonstrated for some key metrics.

Application of objective clinical human reliability analysis (OCHRA) in assessment of technical performance in laparoscopic rectal cancer surgery

BACKGROUND

Laparoscopic rectal resection is technically challenging, with outcomes dependent upon technical performance. No robust objective assessment tool exists for laparoscopic rectal resection surgery. This study aimed to investigate the application of the objective clinical human reliability analysis (OCHRA) technique for assessing technical performance of laparoscopic rectal surgery and explore the validity and reliability of this technique.

METHODS

Laparoscopic rectal cancer resection operations were described in the format of a hierarchical task analysis. Potential technical errors were defined. The OCHRA technique was used to identify technical errors enacted in videos of twenty consecutive laparoscopic rectal cancer resection operations from a single site. The procedural task, spatial location, and circumstances of all identified errors were logged. Clinical validity was assessed through correlation with clinical outcomes; reliability was assessed by test-retest.

RESULTS

A total of 335 execution errors identified, with a median 15 per operation. More errors were observed during pelvic tasks compared with abdominal tasks (p < 0.001). Within the pelvis, more errors were observed during dissection on the right side than the left (p = 0.03). Test-retest confirmed reliability (r = 0.97, p < 0.001). A significant correlation was observed between error frequency and mesorectal specimen quality (r s = 0.52, p = 0.02) and with blood loss (r s = 0.609, p = 0.004).

CONCLUSIONS

OCHRA offers a valid and reliable method for evaluating technical performance of laparoscopic rectal surgery.