Peer Assessment of Operative Videos with Sleeve Gastrectomy to Determine Optimal Operative Technique

Nationwide standardization of minimally invasive right hemicolectomy for colon cancer and development and validation of a video-based competency assessment tool (the Right study)

BACKGROUND

Substantial variation exists when performing a minimally invasive right hemicolectomy (MIRH) due to disparities in training, expertise and differences in implementation of innovations. This study aimed to achieve national consensus on an optimal and standardized MIRH technique for colon cancer and to develop and validate a video-based competency assessment tool (CAT) for MIRH.

METHOD

Statements covering all elements of MIRH were formulated. Subsequently, the Delphi technique was used to reach consensus on a standardized MIRH among 76 colorectal surgeons from 43 different centres. A CAT was developed based on the Delphi results. Nine surgeons assessed the same 12 unedited full-length videos using the CAT, allowing evaluation of the intraclass correlation coefficient (ICC).

RESULTS

After three Delphi rounds, consensus (≥80% agreement) was achieved on 23 of the 24 statements. Consensus statements included the use of low intra-abdominal pressure, detailed anatomical outline how to perform complete mesocolic excision with central vascular ligation, the creation of an intracorporeal anastomosis, and specimen extraction through a Pfannenstiel incision using a wound protector. The CAT included seven consecutive steps to measure competency of the MIRH and showed high consistency among surgeons with an overall ICC of 0.923.

CONCLUSION

Nationwide consensus on a standardized and optimized technique of MIRH was reached. The CAT developed showed excellent interrater reliability. These achievements are crucial steps to an ongoing nationwide quality improvement project (the Right study).

Video Grading of Pancreatic Anastomoses During Robotic Pancreatoduodenectomy to Assess Both Learning Curve and the Risk of Pancreatic Fistula: A Post Hoc Analysis of the LAELAPS-3 Training Program

OBJECTIVE

To assess the learning curve of pancreaticojejunostomy during robotic pancreatoduodenectomy (RPD) and to predict the risk of postoperative pancreatic fistula (POPF) by using the objective structured assessment of technical skills (OSATS), taking the fistula risk into account.

BACKGROUND

RPD is a challenging procedure that requires extensive training and confirmation of adequate surgical performance. Video grading, modified for RPD, of the pancreatic anastomosis could assess the learning curve of RPD and predict the risk of POPF.

METHODS

Post hoc assessment of patients prospectively included in 4 Dutch centers in a nationwide LAELAPS-3 training program for RPD. Video grading of the pancreaticojejunostomy was performed by 2 graders using OSATS (attainable score: 12-60). The main outcomes were the combined OSATS of the 2 graders and POPF (grade B/C). Cumulative sum analyzed a turning point in the learning curve for surgical skill. Logistic regression determined the cutoff for OSATS. Patients were categorized for POPF risk (ie, low, intermediate, and high) based on the updated alternative fistula risk scores.

RESULTS

Videos from 153 pancreatic anastomoses were included. Median OSATS score was 48 (interquartile range: 41-52) points and with a turning point at 33 procedures. POPF occurred in 39 patients (25.5%). An OSATS score below 49, present in 77 patients (50.3%), was associated with an increased risk of POPF (odds ratio: 4.01, P =0.004). The POPF rate was 43.6% with OSATS < 49 versus 15.8% with OSATS ≥49. The updated alternative fistula risk scores category "soft pancreatic texture" was the second strongest prognostic factor of POPF (odds ratio: 3.37, P =0.040). Median cumulative surgical experience was 17 years (interquartile range: 8-21).

CONCLUSIONS

Video grading of the pancreatic anastomosis in RPD using OSATS identified a learning curve and a reduced risk of POPF in case of better surgical performance. Video grading may provide a valid method to surgical training, quality control, and improvement.

National implementation of an optimal standardised technique for right-sided colon cancer: protocol of an interventional sequential cohort study (Right study)

PURPOSE

Minimally invasive right hemicolectomy (MIRH) is the cornerstone of treatment for patients with right-sided colon cancer. This operation has evolved during recent decades, with many innovations and improvements but this has also resulted in high variability of uptake with subsequent substantial variableness. The aim of this ongoing study is to identify current surgical variations, determine the most optimal and standardised MIRH and nationally train and implement that technique to improve short-term clinical and long-term oncological outcomes.

METHODS

The Right study is a national multicentre prospective interventional sequential cohort study. Firstly, current local practice was evaluated. Subsequently, a standardised surgical technique for right-sided colon cancer was determined using the Delphi consensus method, and this procedure was trained during hands-on courses. The standardised MIRH will be implemented with proctoring (implementation cohort), after which the performance will be monitored (consolidation cohort). Patients who will receive a minimally invasive (extended) right hemicolectomy for cT1-3N0-2M0 colon cancer will be included. The primary outcome is patient safety reflected in the 90-day overall complication rate according to the Clavien-Dindo classification. Secondary outcomes will include intraoperative complications, 90-day mortality rate, number of resected tumour-positive lymph nodes, completeness of mesocolic excision, surgical quality score, locoregional and distant recurrence and 5-year overall survival. A total number of 1095 patients (365 per cohort) will be included.

DISCUSSION

The Right study is designed to safely implement the best surgical practice concerning patients with right-sided colon cancer aiming to standardise and improve the surgical quality of MIRH at a national level.

TRIAL REGISTRATION

ClinicalTrials.gov: NCT04889456, May 2021.

Video-based tools for surgical quality assessment of technical skills in laparoscopic procedures: a systematic review

BACKGROUND

Quality of surgery has substantial impact on both short- and long-term clinical outcomes. This stresses the need for objective surgical quality assessment (SQA) for education, clinical practice and research purposes. The aim of this systematic review was to provide a comprehensive overview of all video-based objective SQA tools in laparoscopic procedures and their validity to objectively assess surgical performance.

METHODS

PubMed, Embase.com and Web of Science were systematically searched by two reviewers to identify all studies focusing on video-based SQA tools of technical skills in laparoscopic surgery performed in a clinical setting. Evidence on validity was evaluated using a modified validation scoring system.

RESULTS

Fifty-five studies with a total of 41 video-based SQA tools were identified. These tools were used in 9 different fields of laparoscopic surgery and were divided into 4 categories: the global assessment scale (GAS), the error-based assessment scale (EBAS), the procedure-specific assessment tool (PSAT) and artificial intelligence (AI). The number of studies focusing on these four categories were 21, 6, 31 and 3, respectively. Twelve studies validated the SQA tool with clinical outcomes. In 11 of those studies, a positive association between surgical quality and clinical outcomes was found.

CONCLUSION

This systematic review included a total of 41 unique video-based SQA tools to assess surgical technical skills in various domains of laparoscopic surgery. This study suggests that validated SQA tools enable objective assessment of surgical performance with relevance for clinical outcomes, which can be used for training, research and quality improvement programs.

Skill-qualified surgeons positively affect short-term outcomes after laparoscopic gastrectomy for gastric cancer: a survey of the National Clinical Database of Japan

BACKGROUND

Proficiency of the operating surgeon is one of the most critical factors potentially associated with reductions in complications and surgery-related mortality. With video-rating systems having shown potential for assessing laparoscopic surgeons' proficiency, the Endoscopic Surgical Skill Qualification System (ESSQS) was developed by the Japan Society for Endoscopic Surgery to subjectively assess the proficiency of laparoscopic surgeons by rating applicants' non-edited case videos. We conducted a study to evaluate how ESSQS skill-qualified (SQ) surgeon involvement influences short-term outcomes of laparoscopic gastrectomy performed for gastric cancer.

METHODS

Data from the National Clinical Database regarding laparoscopic distal and total gastrectomy performed for gastric cancer between January 2016 and December 2018 were analyzed. Operative mortality, defined as 30-day mortality or 90-day in-hospital mortality, and anastomotic leakage rates were compared per involvement vs. non-involvement of an SQ surgeon. Outcomes were also compared per involvement of a gastrectomy-, colectomy-, or cholecystectomy-qualified surgeon. The association between the area of qualification and operative mortality/anastomotic leakage was also analyzed with a generalized estimating equation logistic regression model used to account for patient-level risk factors and institutional differences.

RESULTS

Of 104,093 laparoscopic distal gastrectomies, 52,143 were suitable for inclusion in the study; 30,366 (58.2%) were performed by an SQ surgeon. Of 43,978 laparoscopic total gastrectomies, 10,326 were suitable for inclusion; 6501 (63.0%) were performed by an SQ surgeon. Gastrectomy-qualified surgeons outperformed non-SQ surgeons in terms of both operative mortality and anastomotic leakage. They also outperformed cholecystectomy- and colectomy-qualified surgeons in terms of operative mortality or anastomotic leakage in distal and total gastrectomy, respectively.

CONCLUSION

The ESSQS appears to discriminate laparoscopic surgeons who can be expected to achieve significantly improved gastrectomy outcomes.

Identification of Main Influencers of Surgical Efficiency and Variability Using Task-Level Objective Metrics: A Five-Year Robotic Sleeve Gastrectomy Case Series

Objective

Surgical efficiency and variability are critical contributors to optimal outcomes, patient experience, care team experience, and total cost to treat per disease episode. Opportunities remain to develop scalable, objective methods to quantify surgical behaviors that maximize efficiency and reduce variability. Such objective measures can then be used to provide surgeons with timely and user-specific feedbacks to monitor performances and facilitate training and learning. In this study, we used objective task-level analysis to identify dominant contributors toward surgical efficiency and variability across the procedural steps of robotic-assisted sleeve gastrectomy (RSG) over a five-year period for a single surgeon. These results enable actionable insights that can both complement those from population level analyses and be tailored to an individual surgeon's practice and experience.

Methods

Intraoperative video recordings of 77 RSG procedures performed by a single surgeon from 2015 to 2019 were reviewed and segmented into surgical tasks. Surgeon-initiated events when controlling the robotic-assisted surgical system were used to compute objective metrics. A series of multi-staged regression analysis were used to determine: if any specific tasks or patient body mass index (BMI) statistically impacted procedure duration; which objective metrics impacted critical task efficiency; and which task(s) statistically contributed to procedure variability.

Results

Stomach dissection was found to be the most significant contributor to procedure duration (β = 0.344, p< 0.001; R = 0.81, p< 0.001) followed by surgical inactivity and stomach stapling. Patient BMI was not found to be statistically significantly correlated with procedure duration (R = −0.01, p = 0.90). Energy activation rate, a robotic system event-based metric, was identified as a dominant feature in predicting stomach dissection duration and differentiating earlier and later case groups. Reduction of procedure variability was observed between earlier (2015-2016) and later (2017-2019) groups (IQR = 14.20 min vs. 6.79 min). Stomach dissection was found to contribute most to procedure variability (β = 0.74, p < 0.001).

Conclusions

A surgical task-based objective analysis was used to identify major contributors to surgical efficiency and variability. We believe this data-driven method will enable clinical teams to quantify surgeon-specific performance and identify actionable opportunities focused on the dominant surgical tasks impacting overall procedure efficiency and consistency.

Development of a laparoscopic surgical skills simulation curriculum: Enhancing resident training through directed coaching and closed-loop feedback

BACKGROUND

Performance feedback through peer coaching and rigorous self-assessment is a critical part of technical skills improvement. However, formal collaborative programs using operative video-based skills assessments to generate peer coaching feedback have only been validated among attending surgeons. In this study, we developed a unique longitudinal, simulation video-based laparoscopic skills resident curriculum using video-based peer coaching and evaluated its association with skills acquisition among surgical trainees.

METHODS

The laparoscopic simulation curriculum consists of a pre-practice laparoscopic skill video recording, followed by receipt of directed coaching and feedback on performance from a faculty coach, a peer coach, and self-coaching. Residents then completed 6 weeks of feedback-directed practice and submitted a second post-practice laparoscopic skill video recording of the same skill, which was evaluated by a minimally invasive surgery expert grader. All general surgery residents in a single institution were enrolled, with 107 residents completing the curriculum in its initial 2 years.

RESULTS

Overall, more than two-thirds of residents achieved skills proficiency on their expert assessments, with similar rates of residents achieving skills proficiency at all postgraduate year levels. Significant improvements between the pre-practice assessments and post-practice assessments were most frequently seen in the instrument handling, precision, and motion & flow categories (P < .05 each). Faculty provided the highest number and proportion of closed-loop comments; residents' self-coaching feedback had the lowest number of closed-loop comments, with 83% of self-assessments containing none.

CONCLUSION

In this study, we describe the successful implementation of a longitudinal laparoscopic skills video-based coaching curriculum designed to improve residents' laparoscopic technical abilities through iterative directed practice supplemented by formative closed-loop feedback. This feasible, reproducible, and low-cost simulation curriculum can be adapted to other training programs and skills acquisition endeavors. This program also prepares trainees for ongoing performance feedback after completion of residency through rigorous self-assessment and peer-to-peer coaching.

Objective evaluation of technical dexterity in robotic hepaticojejunostomy: Assessment of hepatopancreatobiliary fellows using cumulative sum analytics

BACKGROUND

The development of technical dexterity is a critical for surgeons in training. This study describes and assesses the feasibility of an objective method for the evaluation of procedure-specific technical dexterity in hepatopancreatobiliary (HPB) surgery using cumulative sum (CUSUM) analysis.

METHODS

Dry-lab HPB procedures were divided into procedural steps with binary outcomes (success or failure). Two HPB fellows completed 20 dry lab hepaticojejunostomy (HJ) procedures. Participant progress was tracked over time with CUSUM analytics to establish a learning curve for procedural proficiency.

RESULTS

The CUSUM charts for 20 consecutive dry-lab HJ procedures were analysed. A learning curve was created and used to identify areas of weakness to facilitate improvement in technical proficiency.

CONCLUSIONS

CUSUM is effective tool for objective evaluation of technical dexterity offering both simplicity and adaptability. We demonstrate its use and feasibility for surgical education and plan to expand its' application to assess residents performing general surgery procedures.