Structured learning for robotic surgery utilizing a proficiency score: a pilot study

Essential components and validation of multi-specialty robotic surgical training curricula: a systematic review

INTRODUCTION

The rapid adoption of robotic surgical systems has overtook the development of standardized training and competency assessment for surgeons, resulting in an unmet educational need in this field. This systematic review aims to identify the essential components and evaluate the validity of current robotic training curricula across all surgical specialties.

METHODS

A systematic search of MEDLINE, EMBASE, Emcare, and CINAHL databases was conducted to identify the studies reporting on multi-specialty or specialty-specific surgical robotic training curricula, between January 2000 and January 2024. We extracted the data according to Kirkpatrick's curriculum evaluation model and Messick's concept of validity. The quality of studies was assessed using the Medical Education Research Study Quality Instrument (MERSQI).

RESULTS

From the 3687 studies retrieved, 66 articles were included. The majority of studies were single-center ( n = 52, 78.8%) and observational ( n = 58, 87.9%) in nature. The most commonly reported curriculum components include didactic teaching ( n = 48, 72.7%), dry laboratory skills ( n = 46, 69.7%), and virtual reality (VR) simulation ( n = 44, 66.7%). Curriculum assessment methods varied, including direct observation ( n = 44, 66.7%), video assessment ( n = 26, 39.4%), and self-assessment (6.1%). Objective outcome measures were used in 44 studies (66.7%). None of the studies were fully evaluated according to Kirkpatrick's model, and five studies (7.6%) were fully evaluated according to Messick's framework. The studies were generally found to have moderate methodological quality with a median MERSQI of 11.

CONCLUSIONS

Essential components in robotic training curricula identified were didactic teaching, dry laboratory skills, and VR simulation. However, variability in assessment methods used and notable gaps in curricula validation remain evident. This highlights the need for standardized evidence-based development, evaluation, and reporting of robotic curricula to ensure the effective and safe adoption of robotic surgical systems.

Objective surgical skill assessment based on automatic recognition of dissection and exposure times in robot-assisted radical prostatectomy

PURPOSE

Assessing surgical skills is vital for training surgeons, but creating objective, automated evaluation systems is challenging, especially in robotic surgery. Surgical procedures generally involve dissection and exposure (D/E), and their duration and proportion can be used for skill assessment. This study aimed to develop an AI model to acquire D/E parameters in robot-assisted radical prostatectomy (RARP) and verify if these parameters could distinguish between novice and expert surgeons.

METHODS

This retrospective study used 209 RARP videos from 18 Japanese institutions. Dissection time was defined as the duration of forceps energy activation, and exposure time as the combined duration of manipulating the third arm and camera. To measure these times, an AI-based interface recognition model was developed to automatically extract instrument status from the da Vinci Surgical System® UI. We compared novices and experts by measuring dissection and exposure times from the model's output.

RESULTS

The overall accuracies of the UI recognition model for recognizing the forceps type, energy activation status, and camera usage status were 0.991, 0.998, and 0.991, respectively. Dissection time was 45.2 vs. 35.1 s (novice vs. expert, p = 0.374), exposure time was 195.7 vs. 89.7 s (novice vs. expert, p < 0.001), and the D/E ratio was 0.174 vs. 0.315 (novice vs. expert, p = 0.003).

CONCLUSIONS

We successfully developed a model to automatically acquire dissection and exposure parameters for RARP. Exposure time may serve as an objective parameter to distinguish between novices and experts in RARP, and automated technical evaluation in RARP is feasible.

TRIAL REGISTRATION NUMBER AND DATE

This study was approved by the Institutional Review Board of the National Cancer Center Hospital East (No.2020 - 329) on January 28, 2021.

Current Standards for Training in Robot-assisted Surgery and Endourology: A Systematic Review

BACKGROUND AND OBJECTIVE

Different training programs have been developed to improve trainee outcomes in urology. However, evidence on the optimal training methodology is sparse. Our aim was to provide a comprehensive description of the training programs available for urological robotic surgery and endourology, assess their validity, and highlight the fundamental elements of future training pathways.

METHODS

We systematically reviewed the literature using PubMed/Medline, Embase, and Web of Science databases. The validity of each training model was assessed. The methodological quality of studies on metrics and curricula was graded using the MERSQI scale. The level of evidence (LoE) and level of recommendation for surgical curricula were awarded using the educational Oxford Centre for Evidence-Based Medicine classification.

KEY FINDINGS AND LIMITATIONS

A total of 75 studies were identified. Many simulators have been developed to aid trainees in mastering skills required for both robotic and endourology procedures, but only four demonstrated predictive validity. For assessment of trainee proficiency, we identified 18 in robotics training and six in endourology training; however, the majority are Likert-type scales. Although proficiency-based progression (PBP) curricula demonstrated superior outcomes to traditional training in preclinical settings, only four of six (67%) in robotics and three of nine (33%) in endourology are PBP-based. Among these, the Fundamentals of Robotic Surgery and the SIMULATE curricula have the highest LoE (level 1b). The lack of a quantitative synthesis is the main limitation of our study.

CONCLUSIONS AND CLINICAL IMPLICATIONS

Training curricula that integrate simulators and PBP methodology have been introduced to standardize trainee outcomes in robotics and endourology. However, evidence regarding their educational impact remains restricted to preclinical studies. Efforts should be made to expand these training programs to different surgical procedures and assess their clinical impact.

Implementation and Validation of an Automated, Longitudinal Robotic Surgical Evaluation and Feedback Program at a High-volume Center and Impact on Training

Background

Surgical education lacks a standardized, proficiency-based approach to evaluation and feedback.

Objective

To assess the implementation and reception (ie, feasibility) of an automated, standardized, longitudinal surgical skill assessment and feedback system, and identify baseline trainee (resident and fellow) characteristics associated with achieving proficiency in robotic surgery while learning robotic-assisted laparoscopic prostatectomy.

Design setting and participants

A quality improvement study assessing a pilot of a surgical experience tracking program was conducted over 1 yr. Participants were six fellows, eight residents, and nine attending surgeons at a tertiary cancer center.

Intervention

Trainees underwent baseline self-assessment. After each surgery, an evaluation was completed independently by the trainee and attending surgeons. Performance was rated on a five-point anchored Likert scale (trainees were considered "proficient" when attending surgeons' rating was ≥4). Technical skills were assessed using the Global Evaluative Assessment of Robotic Skills (GEARS) and Prostatectomy Assessment and Competency Evaluation (PACE).

Outcome measurements and statistical analysis

Program success and utility were assessed by evaluating completion rates, evaluation completion times, and concordance rates between attending and trainee surgeons, and exit surveys. Baseline characteristics were assessed to determine associations with achieving proficiency.

Results and limitations

Completion rates for trainees and attending surgeons were 72% and 77%, respectively. Fellows performed more steps/cases than residents (median [interquartile range]: 5 [3-7] and 3 [2-4], respectively; p < 0.01). Prior completion of robotics or laparoscopic skill courses and surgical experience measures were associated with achieving proficiency in multiple surgical steps and GEARS domains. Interclass correlation coefficients on individual components were 0.27-0.47 on GEARS domains.

Conclusions

An automated surgical experience tracker with structured, longitudinal evaluation and feedback can be implemented with good participation and minimal participant time commitment, and can guide curricular development in a proficiency-based education program by identifying modifiable factors associated with proficiency, individualizing education, and identifying improvement areas within the education program.

Patient summary

An automated, standardized, longitudinal surgical skill assessment and feedback system can be implemented successfully in surgical education settings and used to inform education plans and predict trainee proficiency.

Measures of performance and proficiency in robotic assisted surgery: a systematic review

Robotic assisted surgery (RAS) has seen a global rise in adoption. Despite this, there is not a standardised training curricula nor a standardised measure of performance. We performed a systematic review across the surgical specialties in RAS and evaluated tools used to assess surgeons' technical performance. Using the PRISMA 2020 guidelines, Pubmed, Embase and the Cochrane Library were searched systematically for full texts published on or after January 2020-January 2022. Observational studies and RCTs were included; review articles and systematic reviews were excluded. The papers' quality and bias score were assessed using the Newcastle Ottawa Score for the observational studies and Cochrane Risk Tool for the RCTs. The initial search yielded 1189 papers of which 72 fit the eligibility criteria. 27 unique performance metrics were identified. Global assessments were the most common tool of assessment (n = 13); the most used was GEARS (Global Evaluative Assessment of Robotic Skills). 11 metrics (42%) were objective tools of performance. Automated performance metrics (APMs) were the most widely used objective metrics whilst the remaining (n = 15, 58%) were subjective. The results demonstrate variation in tools used to assess technical performance in RAS. A large proportion of the metrics are subjective measures which increases the risk of bias amongst users. A standardised objective metric which measures all domains of technical performance from global to cognitive is required. The metric should be applicable to all RAS procedures and easily implementable. Automated performance metrics (APMs) have demonstrated promise in their wide use of accurate measures.

Evaluation of objective tools and artificial intelligence in robotic surgery technical skills assessment: a systematic review

BACKGROUND

There is a need to standardize training in robotic surgery, including objective assessment for accreditation. This systematic review aimed to identify objective tools for technical skills assessment, providing evaluation statuses to guide research and inform implementation into training curricula.

METHODS

A systematic literature search was conducted in accordance with the PRISMA guidelines. Ovid Embase/Medline, PubMed and Web of Science were searched. Inclusion criterion: robotic surgery technical skills tools. Exclusion criteria: non-technical, laparoscopy or open skills only. Manual tools and automated performance metrics (APMs) were analysed using Messick's concept of validity and the Oxford Centre of Evidence-Based Medicine (OCEBM) Levels of Evidence and Recommendation (LoR). A bespoke tool analysed artificial intelligence (AI) studies. The Modified Downs-Black checklist was used to assess risk of bias.

RESULTS

Two hundred and forty-seven studies were analysed, identifying: 8 global rating scales, 26 procedure-/task-specific tools, 3 main error-based methods, 10 simulators, 28 studies analysing APMs and 53 AI studies. Global Evaluative Assessment of Robotic Skills and the da Vinci Skills Simulator were the most evaluated tools at LoR 1 (OCEBM). Three procedure-specific tools, 3 error-based methods and 1 non-simulator APMs reached LoR 2. AI models estimated outcomes (skill or clinical), demonstrating superior accuracy rates in the laboratory with 60 per cent of methods reporting accuracies over 90 per cent, compared to real surgery ranging from 67 to 100 per cent.

CONCLUSIONS

Manual and automated assessment tools for robotic surgery are not well validated and require further evaluation before use in accreditation processes.PROSPERO: registration ID CRD42022304901.

Development of a Classification System for Live Surgical Feedback

Importance

Live feedback in the operating room is essential in surgical training. Despite the role this feedback plays in developing surgical skills, an accepted methodology to characterize the salient features of feedback has not been defined.

Objective

To quantify the intraoperative feedback provided to trainees during live surgical cases and propose a standardized deconstruction for feedback.

Design, Setting, and Participants

In this qualitative study using a mixed methods analysis, surgeons at a single academic tertiary care hospital were audio and video recorded in the operating room from April to October 2022. Urological residents, fellows, and faculty attending surgeons involved in robotic teaching cases during which trainees had active control of the robotic console for at least some portion of a surgery were eligible to voluntarily participate. Feedback was time stamped and transcribed verbatim. An iterative coding process was performed using recordings and transcript data until recurring themes emerged.

Exposure

Feedback in audiovisual recorded surgery.

Main Outcomes and Measures

The primary outcomes were the reliability and generalizability of a feedback classification system in characterizing surgical feedback. Secondary outcomes included assessing the utility of our system.

Results

In 29 surgical procedures that were recorded and analyzed, 4 attending surgeons, 6 minimally invasive surgery fellows, and 5 residents (postgraduate years, 3-5) were involved. For the reliability of the system, 3 trained raters achieved moderate to substantial interrater reliability in coding cases using 5 types of triggers, 6 types of feedback, and 9 types of responses (prevalence-adjusted and bias-adjusted κ range: a 0.56 [95% CI, 0.45-0.68] minimum for triggers to a 0.99 [95% CI, 0.97-1.00] maximum for feedback and responses). For the generalizability of the system, 6 types of surgical procedures and 3711 instances of feedback were analyzed and coded with types of triggers, feedback, and responses. Significant differences in triggers, feedback, and responses reflected surgeon experience level and surgical task being performed. For example, as a response, attending surgeons took over for safety concerns more often for fellows than residents (prevalence rate ratio [RR], 3.97 [95% CI, 3.12-4.82]; P = .002), and suturing involved more errors that triggered feedback than dissection (RR, 1.65 [95% CI, 1.03-3.33]; P = .007). For the utility of the system, different combinations of trainer feedback had associations with rates of different trainee responses. For example, technical feedback with a visual component was associated with an increased rate of trainee behavioral change or verbal acknowledgment responses (RR, 1.11 [95% CI, 1.03-1.20]; P = .02).

Conclusions and Relevance

These findings suggest that identifying different types of triggers, feedback, and responses may be a feasible and reliable method for classifying surgical feedback across several robotic procedures. Outcomes suggest that a system that can be generalized across surgical specialties and for trainees of different experience levels may help galvanize novel surgical education strategies.

Survival Analysis Using Surgeon Skill Metrics and Patient Factors to Predict Urinary Continence Recovery After Robot-assisted Radical Prostatectomy

BACKGROUND

It has been shown that metrics recorded for instrument kinematics during robotic surgery can predict urinary continence outcomes.

OBJECTIVE

To evaluate the contributions of patient and treatment factors, surgeon efficiency metrics, and surgeon technical skill scores, especially for vesicourethral anastomosis (VUA), to models predicting urinary continence recovery following robot-assisted radical prostatectomy (RARP).

DESIGN, SETTING, AND PARTICIPANTS

Automated performance metrics (APMs; instrument kinematics and system events) and patient data were collected for RARPs performed from July 2016 to December 2017. Robotic Anastomosis Competency Evaluation (RACE) scores during VUA were manually evaluated. Training datasets included: (1) patient factors; (2) summarized APMs (reported over RARP steps); (3) detailed APMs (reported over suturing phases of VUA); and (4) technical skills (RACE). Feature selection was used to compress the dimensionality of the inputs.

OUTCOME MEASUREMENTS AND STATISTICAL ANALYSIS

The study outcome was urinary continence recovery, defined as use of 0 or 1 safety pads per day. Two predictive models (Cox proportional hazards [CoxPH] and deep learning survival analysis [DeepSurv]) were used.

RESULTS AND LIMITATIONS

Of 115 patients undergoing RARP, 89 (77.4%) recovered their urinary continence and the median recovery time was 166 d (interquartile range [IQR] 82-337). VUAs were performed by 23 surgeons. The median RACE score was 28/30 (IQR 27-29). Among the individual datasets, technical skills (RACE) produced the best models (C index: CoxPH 0.695, DeepSurv: 0.708). Among summary APMs, posterior/anterior VUA yielded superior model performance over other RARP steps (C index 0.543-0.592). Among detailed APMs, metrics for needle driving yielded top-performing models (C index 0.614-0.655) over other suturing phases. DeepSurv models consistently outperformed CoxPH; both approaches performed best when provided with all the datasets. Limitations include feature selection, which may have excluded relevant information but prevented overfitting.

CONCLUSIONS

Technical skills and "needle driving" APMs during VUA were most contributory. The best-performing model used synergistic data from all datasets.

PATIENT SUMMARY

One of the steps in robot-assisted surgical removal of the prostate involves joining the bladder to the urethra. Detailed information on surgeon performance for this step improved the accuracy of predicting recovery of urinary continence among men undergoing this operation for prostate cancer.

A Probabilistic Approach To Surgical Tasks and Skill Metrics

Identifying and quantifying the activities that compose surgery is essential for effective interventions, computer-aided analyses and the advancement of surgical data science. For example, recent studies have shown that objective metrics (referred to as objective performance indicators, OPIs) computed during key surgical tasks correlate with surgeon skill and clinical outcomes. Unambiguous identification of these surgical tasks can be particularly challenging for both human annotators and algorithms. Each surgical procedure has multiple approaches, each surgeon has their own level of skill, and the initiation and termination of surgical tasks can be subject to interpretation. As such, human annotators and machine learning models face the same basic problem, accurately identifying the boundaries of surgical tasks despite variable and unstructured information. For use in surgeon feedback, OPIs should also be robust to the variability and diversity in this data. To mitigate this difficulty, we propose a probabilistic approach to surgical task identification and calculation of OPIs. Rather than relying on tasks that are identified by hard temporal boundaries, we demonstrate an approach that relies on distributions of start and stop times, for a probabilistic interpretation of when the task was performed. We first use hypothetical data to outline how this approach is superior to other conventional approaches. Then we present similar analyses on surgical data. We find that when surgical tasks are identified by their individual probabilities, the resulting OPIs are less sensitive to noise in the identification of the start and stop times. These results suggest that this probabilistic approach holds promise for the future of surgical data science.

Learning From Those who Learned: A Survey of Fellowship Trained HoLEP Surgeons and Their Current Practice Patterns

OBJECTIVE

To gain insight from the experience of learning Holmium laser enucleation of the prostate (HoLEP), teaching HoLEP, and the current HoLEP practice patterns of fellowship-trained endourologists.

METHODS

Surveys were electronically distributed to United States (U.S.) practicing urologists who completed American Endourology fellowships (that included HoLEP) within the past 6 years. Questions focused on HoLEP training and current practice patterns.

RESULTS

As of September 2019, 12% (6/49) of U.S. endourology fellowships reported including HoLEP as a component of training. With a 73% response rate (16 of 22), 81% participated in over 20 cases during training, while 50% participated in over 50. A total of 25% independently completed over 50 cases from start to finish. At training completion, most (80%) felt comfortable/somewhat comfortable completing an entire HoLEP independently and managing post-op complications. Seventy-five percent practice HoLEP currently, and 25% teach to trainees. When asked "What is most challenging about HoLEP in current practice?" common responses were: efficiency/profitability concerns, poor reimbursement, educating OR/hospital staff, establishing case volume, minimizing sphincter trauma, and large glands (>200gm).

CONCLUSION

With diverse exposure in fellowship, most incorporate HoLEP into their practice after training. Aspects of the procedure remain challenging after several years of experience. Profitability/reimbursement concerns should be further explored to increase HoLEP adoption.

A deep-learning model using automated performance metrics and clinical features to predict urinary continence recovery after robot-assisted radical prostatectomy

OBJECTIVES

To predict urinary continence recovery after robot-assisted radical prostatectomy (RARP) using a deep learning (DL) model, which was then used to evaluate surgeon's historical patient outcomes.

SUBJECTS AND METHODS

Robotic surgical automated performance metrics (APMs) during RARP, and patient clinicopathological and continence data were captured prospectively from 100 contemporary RARPs. We used a DL model (DeepSurv) to predict postoperative urinary continence. Model features were ranked based on their importance in prediction. We stratified eight surgeons based on the five top-ranked features. The top four surgeons were categorized in 'Group 1/APMs', while the remaining four were categorized in 'Group 2/APMs'. A separate historical cohort of RARPs (January 2015 to August 2016) performed by these two surgeon groups was then used for comparison. Concordance index (C-index) and mean absolute error (MAE) were used to measure the model's prediction performance. Outcomes of historical cases were compared using the Kruskal-Wallis, chi-squared and Fisher's exact tests.

RESULTS

Continence was attained in 79 patients (79%) after a median of 126 days. The DL model achieved a C-index of 0.6 and an MAE of 85.9 in predicting continence. APMs were ranked higher by the model than clinicopathological features. In the historical cohort, patients in Group 1/APMs had superior rates of urinary continence at 3 and 6 months postoperatively (47.5 vs 36.7%, P = 0.034, and 68.3 vs 59.2%, P = 0.047, respectively).

CONCLUSION

Using APMs and clinicopathological data, the DeepSurv DL model was able to predict continence after RARP. In this feasibility study, surgeons with more efficient APMs achieved higher continence rates at 3 and 6 months after RARP.

Effect of surgeon experience and bony pelvic dimensions on surgical performance and patient outcomes in robot-assisted radical prostatectomy

OBJECTIVES

To evaluate the effects of surgeon experience, body habitus, and bony pelvic dimensions on surgeon performance and patient outcomes after robot-assisted radical prostatectomy (RARP).

PATIENTS, SUBJECTS AND METHODS

The pelvic dimensions of 78 RARP patients were measured on preoperative magnetic resonance imaging and computed tomography by three radiologists. Surgeon automated performance metrics (APMs [instrument motion tracking and system events data, i.e., camera movement, third-arm swap, energy use]) were obtained by a systems data recorder (Intuitive Surgical, Sunnyvale, CA, USA) during RARP. Two analyses were performed: Analysis 1, examined effects of patient characteristics, pelvic dimensions and prior surgeon RARP caseload on APMs using linear regression; Analysis 2, the effects of patient body habitus, bony pelvic measurement, and surgeon experience on short- and long-term outcomes were analysed by multivariable regression.

RESULTS

Analysis 1 showed that while surgeon experience affected the greatest number of APMs (P < 0.044), the patient's body mass index, bony pelvic dimensions, and prostate size also affected APMs during each surgical step (P < 0.043, P < 0.046, P < 0.034, respectively). Analysis 2 showed that RARP duration was significantly affected by pelvic depth (β = 13.7, P = 0.039) and prostate volume (β = 0.5, P = 0.024). A wider and shallower pelvis was less likely to result in a positive margin (odds ratio 0.25, 95% confidence interval [CI] 0.09-0.72). On multivariate analysis, urinary continence recovery was associated with surgeon's prior RARP experience (hazard ratio [HR] 2.38, 95% CI 1.18-4.81; P = 0.015), but not on pelvic dimensions (HR 1.44, 95% CI 0.95-2.17).

CONCLUSION

Limited surgical workspace, due to a narrower and deeper pelvis, does affect surgeon performance and patient outcomes, most notably in longer surgery time and an increased positive margin rate.

Developing a five-step training model for transperineal prostate biopsies in a naïve residents' group: a prospective observational randomised study of two different techniques

PURPOSE

To evaluate a five-step training model for transperineal prostate biopsies (TPPB) and the differences in terms of the detection rate (DR) and the ease of execution when using either the "fan technique" (FT) or the use of a Free Hand technique (FH).

METHODS

A prospective observational randomised study was conducted from September 2015 to November 2017. Six naïve residents, who underwent the same five-steps training model, were randomly subdivided into two different groups of three residents based on the selected TPPB technique: A (FT) and B (FH). Patient characteristics (age, PSA, prostatic volume, DRE, MRI), intraoperative (operative time, number of samples) and postoperative parameters (histologic, pain) were evaluated in the 2 groups. The overall and stratified DR for PSA ranges and prostate volume (PV), operative time and complications were compared.

RESULTS

The overall detection rate was very high in both groups (FT 58.2% vs FH 59.6%) and not statistically different between the two techniques. There were no differences in terms of complication rates and pain. The FH showed a better detection rate in prostates smaller than 40 cc (p = 0.023) and a faster operative time (p = 0.025) compared to FT.

CONCLUSIONS

Within the TPPB, FH is associated with a higher detection rate in patients with prostate < 40 cc compared to an FT when performed by inexperienced trainees. Standardised training organised in consecutive steps seems to contribute to the achievement of overall high detection rates with both methods.

Experts vs super-experts: differences in automated performance metrics and clinical outcomes for robot-assisted radical prostatectomy

OBJECTIVES

To evaluate automated performance metrics (APMs) and clinical data of experts and super-experts for four cardinal steps of robot-assisted radical prostatectomy (RARP): bladder neck dissection; pedicle dissection; prostate apex dissection; and vesico-urethral anastomosis.

SUBJECTS AND METHODS

We captured APMs (motion tracking and system events data) and synchronized surgical video during RARP. APMs were compared between two experience levels: experts (100-750 cases) and super-experts (2100-3500 cases). Clinical outcomes (peri-operative, oncological and functional) were then compared between the two groups. APMs and outcomes were analysed for 125 RARPs using multi-level mixed-effect modelling.

RESULTS

For the four cardinal steps selected, super-experts showed differences in select APMs compared with experts (P < 0.05). Despite similar PSA and Gleason scores, super-experts outperformed experts clinically with regard to peri-operative outcomes, with a greater lymph node yield of 22.6 vs 14.9 nodes, respectively (P < 0.01), less blood loss (125 vs 130 mL, respectively; P < 0.01), and fewer readmissions at 30 days (1% vs 13%, respectively; P = 0.02). A similar but nonsignificant trend was seen for oncological and functional outcomes, with super-experts having a lower rate of biochemical recurrence compared with experts (5% vs 15%, respectively; P = 0.13) and a higher continence rate at 3 months (36% vs 18%, respectively; P = 0.14).

CONCLUSION

We found that experts and super-experts differed significantly in select APMs for the four cardinal steps of RARP, indicating that surgeons do continue to improve in performance even after achieving expertise. We hope ultimately to identify associations between APMs and clinical outcomes to tailor interventions to surgeons and optimize patient outcomes.

Robotic Thoracic Surgery Training for Residency Programs

Objective

Robotic-assisted surgery is increasingly being used in thoracic surgery. Currently, the Integrated Thoracic Surgery Residency Program lacks a standardized curriculum or requirement for training residents in robotic-assisted thoracic surgery. In most circumstances, because of the lack of formal residency training in robotic surgery, hospitals are requiring additional training, mentorship, and formal proctoring of cases before granting credentials to perform robotic-assisted surgery. Therefore, there is necessity for residents in Integrated Thoracic Surgery Residency Program to have early exposure and formal training on the robotic platform. We propose a curriculum that can be incorporated into such programs that would satisfy both training needs and hospital credential requirements.

Methods

We surveyed all 26 Integrated Thoracic Surgery Residency Program Directors in the United States. We also performed a PubMed literature search using the key word “robotic surgery training curriculum.” We reviewed various robotic surgery training curricula and evaluation tools used by urology, obstetrics gynecology, and general surgery training programs. We then designed a proposed curriculum geared toward thoracic Integrated Thoracic Surgery Residency Program adopted from our credentialing experience, literature review, and survey consensus.

Results

Of the 26 programs surveyed, we received 17 responses. Most Integrated Thoracic Surgery Residency Program directors believe that it is important to introduce robotic surgery training during residency. Our proposed curriculum is integrated during postgraduate years 2 to 6. In the preclinical stage postgraduate years 2 to 3, residents are required to complete introductory online modules, virtual reality simulator training, and in-house workshops. During clinical stage (postgraduate years 4–6), the resident will serve as a supervised bedside assistant and progress to a console surgeon. Each case will have defined steps that the resident must demonstrate competency. Evaluation will be based on standardized guidelines.

Conclusions

Expansion and utilization of robotic assistance in thoracic surgery have increased. Our proposed curriculum aims to enable Integrated Thoracic Surgery Residency Program residents to achieve competency in robotic-assisted thoracic surgery and to facilitate the acquirement of hospital privileges when they enter practice.

Development of a technical checklist for the assessment of suturing in robotic surgery

BACKGROUND

With the increased use of simulation for surgical training, there is a need for objective forms of assessment to evaluate trainees. The Global Evaluative Assessment of Robotic Skills (GEARS) is widely used for assessing skills in robotic surgery, but there are no recognised checklist scoring systems. This study aimed to develop a checklist for suturing in robotic surgery.

METHODS

A suturing checklist for needle driving and knot tying was constructed following evaluation of participants performing urethrovesical anastomoses. Key procedural steps were identified from expert videos, while assessing novice videos allowed identification of common technical errors. 22 novice and 13 expert videos were marked on needle driving, while 18 novices and 10 experts were assessed on knot tying. Validation of the finalised checklist was performed with the assessment of 39 separate novices by an expert surgeon and compared to GEARS scoring.

RESULTS

The internal consistency of the preliminary checklist was high (Cronbach's alpha = 0.870 for needle driving items; 0.736 for knot tying items), and after removal of poorly correlating items, the final checklist contained 23 steps. Both the needle driving and knot tying categories discriminated between novices and experts, p < 0.005. While the GEARS score demonstrated construct validity for needle driving, it could not significantly differentiate between novices and experts for knot tying, p = 0.286. The needle driving category significantly correlated with the corresponding GEARS scores (r_s = 0.613, p < 0.005), but the correlation for knot tying was insignificant (r_s = 0.296, p = 0.127). The pilot data indicates the checklist significantly correlated with the GEARS score (p < 0.005).

CONCLUSION

This study reports the development of a valid assessment tool for suturing in robotic surgery. Given that checklists are simple to use, there is significant scope for this checklist to be used in surgical training.

Use of Automated Performance Metrics to Measure Surgeon Performance during Robotic Vesicourethral Anastomosis and Methodical Development of a Training Tutorial

PURPOSE

We sought to develop and validate automated performance metrics to measure surgeon performance of vesicourethral anastomosis during robotic assisted radical prostatectomy. Furthermore, we sought to methodically develop a standardized training tutorial for robotic vesicourethral anastomosis.

MATERIALS AND METHODS

We captured automated performance metrics for motion tracking and system events data, and synchronized surgical video during robotic assisted radical prostatectomy. Nonautomated performance metrics were manually annotated by video review. Automated and nonautomated performance metrics were compared between experts with 100 or more console cases and novices with fewer than 100 cases. Needle driving gestures were classified and compared. We then applied task deconstruction, cognitive task analysis and Delphi methodology to develop a standardized robotic vesicourethral anastomosis tutorial.

RESULTS

We analyzed 70 vesicourethral anastomoses with a total of 1,745 stitches. For automated performance metrics experts outperformed novices in completion time (p <0.01), EndoWrist® articulation (p <0.03), instrument movement efficiency (p <0.02) and camera manipulation (p <0.01). For nonautomated performance metrics experts had more optimal needle to needle driver positioning, fewer needle driving attempts, a more optimal needle entry angle and less tissue trauma (each p <0.01). We identified 14 common robotic needle driving gestures. Random gestures were associated with lower efficiency (p <0.01), more attempts (p <0.04) and more trauma (p <0.01). The finalized tutorial contained 66 statements and figures. Consensus among 8 expert surgeons was achieved after 2 rounds, including among 58 (88%) after round 1 and 8 (12%) after round 2.

CONCLUSIONS

Automated performance metrics can distinguish surgeon expertise during vesicourethral anastomosis. The expert vesicourethral anastomosis technique was associated with more efficient movement and less tissue trauma. Standardizing robotic vesicourethral anastomosis and using a methodically developed tutorial may help improve robotic surgical training.

Implementing assessments of robot-assisted technical skill in urological education: a systematic review and synthesis of the validity evidence

OBJECTIVES

To systematically review and synthesise the validity evidence supporting intraoperative and simulation-based assessments of technical skill in urological robot-assisted surgery (RAS), and make evidence-based recommendations for the implementation of these assessments in urological training.

MATERIALS AND METHODS

A literature search of the Medline, PsycINFO and Embase databases was performed. Articles using technical skill and simulation-based assessments in RAS were abstracted. Only studies involving urology trainees or faculty were included in the final analysis.

RESULTS

Multiple tools for the assessment of technical robotic skill have been published, with mixed sources of validity evidence to support their use. These evaluations have been used in both the ex vivo and in vivo settings. Performance evaluations range from global rating scales to psychometrics, and assessments are carried out through automation, expert analysts, and crowdsourcing.

CONCLUSION

There have been rapid expansions in approaches to RAS technical skills assessment, both in simulated and clinical settings. Alternative approaches to assessment in RAS, such as crowdsourcing and psychometrics, remain under investigation. Evidence to support the use of these metrics in high-stakes decisions is likely insufficient at present.

Development and Validation of Objective Performance Metrics for Robot-Assisted Radical Prostatectomy: A Pilot Study

PURPOSE

We explore and validate objective surgeon performance metrics using a novel recorder ("dVLogger") to directly capture surgeon manipulations on the da Vinci® Surgical System. We present the initial construct and concurrent validation study of objective metrics during preselected steps of robot-assisted radical prostatectomy.

MATERIALS AND METHODS

Kinematic and events data were recorded for expert (100 or more cases) and novice (less than 100 cases) surgeons performing bladder mobilization, seminal vesicle dissection, anterior vesicourethral anastomosis and right pelvic lymphadenectomy. Expert/novice metrics were compared using mixed effect statistical modeling (construct validation). Expert reviewers blindly rated seminal vesicle dissection and anterior vesicourethral anastomosis using GEARS (Global Evaluative Assessment of Robotic Skills). Intraclass correlation measured inter-rater variability. Objective metrics were correlated to corresponding GEARS metrics using Spearman's test (concurrent validation).

RESULTS

The performance of 10 experts (mean 810 cases, range 100 to 2,000) and 10 novices (mean 35 cases, range 5 to 80) was evaluated in 100 robot-assisted radical prostatectomy cases. For construct validation the experts completed operative steps faster (p <0.001) with less instrument travel distance (p <0.01), less aggregate instrument idle time (p <0.001), shorter camera path length (p <0.001) and more frequent camera movements (p <0.03). Experts had a greater ratio of dominant-to-nondominant instrument path distance for all steps (p <0.04) except anterior vesicourethral anastomosis. For concurrent validation the median experience of 3 expert reviewers was 300 cases (range 200 to 500). Intraclass correlation among reviewers was 0.6-0.7. For anterior vesicourethral anastomosis and seminal vesicle dissection, kinematic metrics had low associations with GEARS metrics.

CONCLUSIONS

Objective metrics revealed experts to be more efficient and directed during preselected steps of robot-assisted radical prostatectomy. Objective metrics had limited associations to GEARS. These findings lay the foundation for developing standardized metrics for surgeon training and assessment.

Proctorship and mentoring: Its backbone and application in robotic surgery

In pursuit of continuing medical education in robotic surgery, several forms of training have been implemented. This variable application of curriculum has brought acquisition of skills in a heterogeneous and unstandardized fashion from different parts of the world. Recently, efforts have been made to provide cost effective and well-structured curricula with the aim of bridging the gap between formal fellowship training and short courses. Proctorship training has been implicated on some curriculum to provide excellent progression during the learning curve while ensuring patient safety.