Is competency assessment at the specialist level achievable? A study for the national training programme in laparoscopic colorectal surgery in England. Ann Surg. 2013;257:476-482. [PMID: 23386240 DOI: 10.1097/sla.0b013e318275b72a]

Validation of Motion Tracking Software for Evaluation of Surgical Performance in Laparoscopic Cholecystectomy

Motion tracking software for assessing laparoscopic surgical proficiency has been proven to be effective in differentiating between expert and novice performances. However, with several indices that can be generated from the software, there is no set threshold that can be used to benchmark performances. The aim of this study was to identify the best possible algorithm that can be used to benchmark expert, intermediate and novice performances for objective evaluation of psychomotor skills. 12 video recordings of various surgeons were collected in a blinded fashion. Data from our previous study of 6 experts and 23 novices was also included in the analysis to determine thresholds for performance. Video recording were analyzed both by the Kinovea 0.8.15 software and a blinded expert observer using the CAT form. Multiple algorithms were tested to accurately identify expert and novice performances. ½ L + [Formula: see text] A + [Formula: see text] J scoring of path length, average movement and jerk index respectively resulted in identifying 23/24 performances. Comparing the algorithm to CAT assessment yielded in a linear regression coefficient R2 of 0.844. The value of motion tracking software in providing objective clinical evaluation and retrospective analysis is evident. Given the prospective use of this tool the algorithm developed in this study proves to be effective in benchmarking performances for psychomotor skills evaluation.

Factors Predicting Operative Difficulty of Laparoscopic Total Mesorectal Excision

BACKGROUND

Laparoscopic total mesorectal excision is a challenging procedure requiring high-quality surgery for optimal outcomes. Patient, tumor, and pelvic factors are believed to determine difficulty, but previous studies were limited to postoperative data.

OBJECTIVE

This study aimed to report factors predicting laparoscopic total mesorectal excision performance by using objective intraoperative assessment.

DESIGN

Data from a multicenter laparoscopic total mesorectal excision randomized trial (ISRCTN59485808) were reviewed.

SETTING

This study was conducted at 4 centers in the United Kingdom.

PATIENTS AND INTERVENTION

Seventy-one patients underwent elective laparoscopic total mesorectal excision for rectal adenocarcinoma with curative intent: 53% were men, mean age was 69 years, body mass index was 27.7, tumor height was 8.5 cm, 24% underwent neoadjuvant therapy, and 25% had previous surgery.

MAIN OUTCOME MEASURES

Surgical performance was assessed through the identification of intraoperative adverse events by using observational clinical human reliability analysis. Univariate analysis and multivariate binomial regression were performed to establish factors predicting the number of intraoperative errors, surgeon-reported case difficulty, and short-term clinical and histopathological outcomes.

RESULTS

A total of 1331 intraoperative errors were identified from 365 hours of surgery (median, 18 per case; interquartile range, 16-22; and range, 9-49). No patient, tumor, or bony pelvimetry measurement correlated with total or pelvic error count, surgeon-reported case difficulty, cognitive load, operative data, specimen quality, number or severity of 30-day morbidity events and length of stay (all r not exceeding ±0.26, p > 0.05). Mesorectal area was associated with major intraoperative adverse events (OR, 1.09; 95%CI, 1.01-1.16; p = 0.015) and postoperative morbidity (OR, 1.1; 95% CI, 1.01-1.2; p = 0.033). Obese men were subjectively reported as harder cases (24 vs 36 mm, p = 0.042), but no detrimental effects on performance or outcomes were seen.

LIMITATIONS

Our sample size is modest, risking type II errors and overfitting of the statistical models.

CONCLUSION

Patient, tumor, and bony pelvic anatomical characteristics are not seen to influence laparoscopic total mesorectal excision operative difficulty. Mesorectal area is identified as a risk factor for intraoperative and postoperative morbidity. See Video Abstract at http://links.lww.com/DCR/B35. FACTORES QUE PREDICEN LA DIFICULTAD OPERATIVA DE LA ESCISIÓN MESORRECTAL TOTAL LAPAROSCÓPICA: La escisión mesorrectal total laparoscópica es un procedimiento desafiante. Para obtener resultados óptimos, se requiere cirugía de alta calidad. Se cree que, factores como el paciente, el tumor y la pelvis, determinan la dificultad, pero estudios previos solamente se han limitado a datos postoperatorios.Informar de los factores que predicen el resultado de la escisión mesorrectal total laparoscópica, mediante una evaluación intraoperatoria objetiva.Datos de un ensayo multicéntrico y randomizado de escisión mesorrectal total laparoscópica (ISRCTN59485808).Cuatro centros del Reino Unido.Un total de 71 pacientes fueron sometidos a escisión mesorrectal total laparoscópica electiva, para adenocarcinoma rectal con intención curativa. 53% hombres, edad media, índice de masa corporal y altura del tumor 69, 27.7 y 8.5 cm respectivamente, 24% terapia neoadyuvante y 25% cirugía previa.Rendimiento quirúrgico evaluado mediante la identificación de eventos intraoperatorios adversos, mediante el análisis clínico observacional de confiabilidad humana. Se realizaron análisis univariado y la regresión binomial multivariada para establecer factores que predicen el número de errores intraoperatorios, reportes del cirujano sobre la dificultad del caso y los resultados clínicos e histopatológicos a corto plazo.Se identificaron un total de 1,331 errores intraoperatorios en 365 horas de cirugía (media de 18 por caso, IQR 16-22, rango 9-49). Ningún paciente, tumor o medición de pelvimetría pélvica, se correlacionó con la cuenta de errores pélvicos o totales, reporte del cirujano sobre dificultad del caso, carga cognitiva, datos operativos, calidad de la muestra, número o gravedad de eventos de morbilidad de 30 días y duración de la estadía (todos r <± 0.26, p > 0.05). El área mesorrectal se asoció con eventos adversos intraoperatorios importantes (OR, 1.09; IC 95%, 1.01-1.16; p = 0.015) y morbilidad postoperatoria (OR, 1.1; IC 95%, 1.01-1.2; p = 0.033). Como información subjetiva, hombres obesos fueron casos más difíciles (24 mm frente a 36 mm, p = 0.042) pero no se observaron efectos perjudiciales sobre el rendimiento o los resultados.Nuestro tamaño de muestra es un modesto riesgo de errores de tipo II y el sobreajuste de los modelos estadísticos.No se observa que las características anatómicas del paciente, tumor y pelvis ósea influyan en la dificultad operatoria de la escisión mesorrectal laparoscópica total. El área mesorrectal se identifica como un factor de riesgo para la morbilidad intraoperatoria y postoperatoria. Vea el resumen del video en http://links.lww.com/DCR/B35.

Video-based coaching in surgical education: a systematic review and meta-analysis

BACKGROUND

In the era of competency-based surgical education, VBC has gained increased attention and may enhance the efficacy of surgical education. The objective of this systematic review was to summarize the existing evidence of video-based coaching (VBC) and compare VBC to traditional master-apprentice-based surgical education.

METHODS

We performed a systematic review and meta-analysis of randomized controlled trials (RCT) assessing VBC according to the PRISMA and Cochrane guidelines. The MEDLINE, EMBASE, and COCHRANE and Researchgate databases were searched for eligible manuscripts. Standard mean difference (SMD) of performance scoring scales was used to assess the effect of VBC versus traditional training without VBC (control).

RESULTS

Of 627 studies identified, 24 RCTs were eligible and evaluated. The studies included 778 surgical trainees (n = 386 VBC vs. n = 392 control). 13 performance scoring scales were used to assess technical competence; OSATS-GRS was the most common (n = 15). VBC was provided preoperative (n = 11), intraoperative (n = 1), postoperative (n = 10), and perioperative (n = 2). The majority of studies were unstructured, where identified coaching frameworks were PRACTICE (n = 1), GROW (n = 2) and Wisconsin Coaching Framework (n = 1). There was an effect on performance scoring scales in favor of VBC coaching (SMD 0.87, p < 0.001). In subgroup analyses, the residents had a larger relative effect (SMD 1.13; 0.61-1.65, p < 0.001) of VBC compared to medical students (SMD 0.43, 0.06-0.81, p < 0.001). The greatest source of potential bias was absence of blinding of the participants and personnel (n = 20).

CONCLUSION

Video-based coaching increases technical performance of medical students and surgical residents. There exist significant study and intervention heterogeneity that warrants further exploration, showing the need to structure and standardize video-based coaching tools.

In Search of Characterizing Surgical Skill

OBJECTIVE

This paper provides a literature review and detailed discussion of surgical skill terminology. Culminating in a novel model that proposes a set of unique definitions, this review is designed to facilitate shared understanding to study and develop metrics quantifying surgical skill.

DESIGN

Objective surgical skill analysis depends on consistent definitions and shared understanding of terms like performance, expertise, experience, aptitude, ability, competency, and proficiency.

STRUCTURE

Each term is discussed in turn, drawing from existing literature and colloquial uses.

IMPLICATIONS

A new model of definitions is proposed to cement a common and consistent lexicon for future skills analysis, and to quantitatively describe a surgeon's performance throughout their career.

Stepwise Training in Laparoscopic Surgery for Complex Ileocolonic Crohn's Disease: Analysis of 127 Training Episodes

INTRODUCTION

The inflammation encountered in Crohn's disease makes a minimally invasive approach challenging due to a thickened mesentery, fistulas, abscesses, and large phlegmons with high reported rates of conversion and septic complications. Aim of this study was to evaluate the feasibility of a stepwise approach to training in laparoscopic surgery for complex Crohn's disease.

METHODS

Every surgical procedure was divided in 4 different training tasks: access and exposure, bowel mobilization, division of the mesentery, anastomosis. Extensive adhesiolysis and division and repair of fistulae were considered as additional tasks when present. The laparoscopic competence assessment tool was used to evaluate the safety and proficiency of the surgical performance. The primary outcome was the rate of training tasks successfully completed by surgical trainees.

RESULTS

One hundred and twenty seven training episodes were included and 86 were performed by trainees (67.7%). Fistula division was the less commonly performed training task (25%), while mobilisation and anastomosis were performed by the supervised trainee in 90% and 85% of the cases. Safety and proficiency scores were significantly higher for senior trainees compared to junior trainees.

CONCLUSIONS

Laparoscopic surgery for complex Crohn's disease can be safely performed in a supervised setting with acceptable operating time, postoperative length of hospital stay, and 30 day morbidity.

Competency assessment tool for laparoscopic suturing: development and reliability evaluation

BACKGROUND

Laparoscopic suturing can be technically challenging and requires extensive training to achieve competency. To date no specific and objective assessment method for laparoscopic suturing and knot tying is available that can guide training and monitor performance in these complex surgical skills. In this study we aimed to develop a laparoscopic suturing competency assessment tool (LS-CAT) and assess its inter-observer reliability.

METHODS

We developed a bespoke CAT tool for laparoscopic suturing through a structured, mixed methodology approach, overseen by a steering committee with experience in developing surgical assessment tools. A wide Delphi consultation with over twelve experts in laparoscopic surgery guided the development stages of the tool. Following, subjects with different levels of laparoscopic expertise were included to evaluate this tool, using a simulated laparoscopic suturing task which involved placing of two surgical knots. A research assistant video recorded and anonymised each performance. Two blinded expert surgeons assessed the anonymised videos using the developed LS-CAT. The LS-CAT scores of the two experts were compared to assess the inter-observer reliability. Lastly, we compared the subjects' LS-CAT performance scores at the beginning and end of their learning curve.

RESULTS

This study evaluated a novel LS-CAT performance tool, comprising of four tasks. Thirty-six complete videos were analysed and evaluated with the LS-CAT, of which the scores demonstrated excellent inter-observer reliability. Cohen's Kappa analysis revealed good to excellent levels of agreement for almost all tasks of both instrument handling and tissue handling (0.87; 0.77; 0.75; 0.86; 0.85, all with p < 0.001). Subjects performed significantly better at the end of their learning curve compared to their first attempt for all LS-CAT items (all with p < 0.001).

CONCLUSIONS

We developed the LS-CAT, which is a laparoscopic suturing grading matrix, with excellent inter-rater reliability and to discriminate between experience levels. This LS-CAT has a potential for wider use to objectively assess laparoscopic suturing skills.

Validation and educational impact study of the NANEP high-fidelity simulation model for open preperitoneal mesh repair of umbilical hernia

OBJECTIVE

The aim of the study was to develop, validate and analyze the educational impact of a high-fidelity simulation model for open preperitoneal mesh repair of an umbilical hernia. The number of surgical simulators available for training residents is limited. Primary for ethical reasons and secondary for the emerging pay-per-quality policies, practicing-on simulators rather than patients is considered gold standard. Validated full-procedural surgical models will become more and more important in training residents. Such models may assure that evidence-based standards regarding technical aspects of the procedures become integral part of the curriculum. Furthermore, they can be employed as a quality control of residents' skills (Fonseca et al. in J Surg Educ 70:129-137, 2013).

METHODS

In a repeated measures design, medical students, residents in their last year of training and attending surgeons performed an open preperitoneal mesh repair on the NANEP model [NANEP stands for the German acronym Nabelhernien-Netzimplatation-Präperitonal (English: Umbilical hernia mesh implantation preperitoneal)]. Subjects were categorized as "Beginners" (internship students) or "Experts" (residents and surgeons). Content validity was analyzed by criteria of subject-matter-experts. Blinded raters assessed surgical skills by means of the Competency Assessment Tool (CAT) using the online platform "CATLIVE". Differential validity was measured by group differences. Proficiency gain was analyzed by monitoring the learning curve (Gallagher et al. in Ann Surg 241:364-372, 2005). Post-operative examination of the simulators shed light on criterion validity.

RESULTS

The NANEP model-proofed content and construct-valid significant Bonferroni-corrected differences were found between beginners and experts (p < 0.05). Beginners showed a significant learning increase from the first to the second surgery (p < 0.05). Post-operative examination data confirmed criterion validity.

CONCLUSION

The NANEP model is an inexpensive, simple and efficient simulation model. It has highly realistic features, it has been shown to be of high-fidelity, full-procedural and benchtop-model. The NANEP model meets the main needs of surgical educational courses at the beginning of residency.

Identifying the Essential Portions of the Skill Acquisition Process Using Item Response Theory

OBJECTIVE

Item response theory (IRT) was originally developed to make performance assessments more accurate. However, IRT analysis of the intraoperative performance of surgical trainees could help identify the elements that the trainees find difficult during the skill acquisition process. The aim of this study was to identify the essential portions of the skill acquisition process of a surgical procedure using the IRT.

DESIGN

The 24-item assessment checklist was used to evaluate a recorded intra-operative performance of a laparoscopic inguinal hernia repair. The scores were analyzed using IRT to calculate the difficulty and discrimination level of each item.

SETTING

Fifteen institutes.

PARTICIPANTS

Thirty surgical trainees.

RESULTS

A total of 123 assessments were analyzed. The item analysis showed the procedure specific item "traction of peritoneum (difficulty: -0.45; discrimination: 19.37)" and generic items "instrument handling (difficulty: -0.59; discrimination: 3.82)" and "flow of procedure (difficulty: 0.09; discrimination: 3.27)" to be key elements in the skill acquisition process of the procedure.

CONCLUSIONS

Key elements in the skill acquisition process of the procedure were quantitatively identified by applying the IRT analysis. This could lead to the use of IRT in designing and developing a more effective training curriculum.

How to train practising gynaecologists in total laparoscopic hysterectomy: protocol for the stepped-wedge IMAGINE trial

INTRODUCTION

Hysterectomy is the most common major gynaecological procedure in women and minimally invasive approaches should be used wherever possible; total laparoscopic hysterectomy (TLH) is one such surgical approach which allows removal of the uterus entirely laparoscopically. However, lack of surgical training opportunities is impeding its increased adoption. This study will formally test a surgical outreach training model to equip surgeons with the skills to provide TLH as an alternative to total abdominal hysterectomy (TAH).

METHODS AND ANALYSIS

Stepped wedge implementation trial of a surgical training programme for practising obstetrician gynaecologist specialists in four hospitals.

PRIMARY OUTCOMES

Change in the proportion of hysterectomies performed by TAH, measured between preintervention and postintervention; we aim to reduce TAH by at least 30% in 75% of the trainees.

SECONDARY OUTCOMES

(1) Number of hospitals screened, eligible, agree to training and complete the training; (2) number of surgeons screened for eligibility, eligible, agree to training, who complete training and achieve proficiency; (3) proportion of trainees achieving proficiency in correct theatre setup, vascular exposure, mobilisation and surgery closure; change in proportion proficient over time; (4) adverse events (conversion from TLH to TAH, anaesthetic incident, intraoperative visceral injury, red cell transfusions, hospital stay >7 days, incidental finding of malignancy, unplanned readmission, admission to intensive care, return to theatre, postoperative pulmonary embolism or deep vein thrombosis, development of a fistula, vault haematoma, vaginal vault dehiscence or pelvic infection); (5) hospital length-of-stay; (6) cost-effectiveness and (7) trainee surgeon proficiency with TLH.

ETHICS AND DISSEMINATION

The study has been approved by the Royal Brisbane and Women's Hospital Human Research Ethics Committee and has received site-specific approval from all participating hospitals. Results will be submitted for publication in a peer-reviewed journal.

TRIAL REGISTRATION NUMBER

NCT03617354; Pre-results.

A novel assessment tool for evaluating competence in video-assisted thoracoscopic surgery lobectomy

BACKGROUND

Specific assessment tools can accelerate trainees' learning through structured feedback and ensure that trainees attain the knowledge and skills required to practice as competent, independent surgeons (competency-based surgical education). The objective was to develop an assessment tool for video-assisted thoracoscopic surgery (VATS) lobectomy by achieving consensus within an international group of VATS experts.

METHOD

The Delphi method was used as a structured process for collecting and distilling knowledge from a group of internationally recognized VATS experts. Opinions were obtained in an iterative process involving answering repeated rounds of questionnaires. Responses to one round were summarized and integrated into the next round of questionnaires until consensus was reached.

RESULTS

Thirty-one VATS experts were included and four Delphi rounds were conducted. The response rate for each round were 68.9% (31/45), 100% (31/31), 96.8% (30/31), and 93.3% (28/30) for the final round where consensus was reached. The first Delphi round contained 44 items and the final VATS lobectomy Assessment Tool (VATSAT) comprised eight items with rating anchors: (1) localization of tumor and other pathological tissue, (2) dissection of the hilum and veins, (3) dissection of the arteries, (4) dissection of the bronchus, (5) dissection of lymph nodes, (6) retrieval of lobe in bag, (7) respect for tissue and structures, and (8) technical skills in general.

CONCLUSION

A novel and dedicated assessment tool for VATS lobectomy was developed based on VATS experts' consensus. The VATSAT can support the learning of VATS lobectomy by providing structured feedback and help supervisors make the important decision of when trainees have acquired VATS lobectomy competencies for independent performance.

"Reflection-Before-Practice" Improves Self-Assessment and End-Performance in Laparoscopic Surgical Skills Training

OBJECTIVE

To establish whether a systematized approach to self-assessment in a laparoscopic surgical skills course improves accordance between expert- and self-assessment.

DESIGN

A systematic training course in self-assessment using Competency Assessment Tool was introduced into the normal course of evaluation within a Laparoscopic Surgical Skills training course for the test group (n = 30). Differences between these and a control group (n = 30) who did not receive the additional training were assessed.

SETTING

Catharina Hospital, Eindhoven, The Netherlands (n = 27), and GSL Medical College, Rajahmundry, India (n = 33).

PARTICIPANTS

Sixty postgraduate year 2 and 3 surgical residents who attended the 2-day Laparoscopic Surgical Skills grade 1 level 1 curriculum were invited to participate.

RESULTS

The test group (n = 30) showed better accordance between expert- and self-assessment (difference of 1.5, standard deviation [SD] = 0.2 versus 3.83, SD = 0.6, p = 0.009) as well as half the number (7 versus 14) of cases of overreporting. Furthermore, the test group also showed higher overall mean performance (mean = 38.1, SD = 0.7 versus mean = 31.8, SD = 1.0, p < 0.001) than the control group (n = 30). The systematic approach to self-assessment can be viewed as responsible for this and can be seen as "reflection-before-practice" within the framework of reflective practice as defined by Donald Schon.

CONCLUSION

Our results suggest that "reflection-before-practice" in implementing self-assessment is an important step in the development of surgical skills, yielding both better understanding of one's strengths and weaknesses and also improving overall performance.

Skills in minimally invasive and open surgery show limited transferability to robotic surgery: results from a prospective study

BACKGROUND

There is limited evidence on the transferability of conventional laparoscopic and open surgical skills to robotic-assisted surgery. The primary aim of this study was to evaluate the transferability of expertise in conventional laparoscopy and open surgery to robotic-assisted surgery using the da Vinci Skills Simulator (dVSS). Secondary aims included evaluating the influence of individual participants' characteristics.

METHODS

Participants performed four tasks on the dVSS: Peg Board 1 (PB), Pick and Place (PP), Thread the Rings (TR), and Suture Sponge 1 (SS). Participants were classified into three groups (Novice, Intermediate, Experts) according to experience in laparoscopic and open surgery. All tasks were performed twice except for SS. Performance was assessed using the built-in scoring system.

RESULTS

37 medical students and 25 surgeons participated. Experts did not perform significantly better than less experienced participants on the dVSS. Specifically, with regard to laparoscopic experience, total simulator scores were: Novices 68.2 ± 28.8; Intermediates 65.1 ± 31.2; Experts 65.1 ± 30.0; p = 0.611. Regarding open surgical experience, scores were: Novices 68.6 ± 28.7; Intermediates 68.2 ± 30.8; Experts 63.2 ± 30.3; p = 0.305. Although there were some significant differences among groups for single parameters in specific tasks, there was no constant superiority of one group. Laparoscopic and open surgical Novices improved significantly in overall score and time for all three tasks (p < 0.05). Laparoscopic intermediates improved only in PP time (4.64 ± 3.42; p = 0.006), open Intermediates in PB score (11.98 ± 13.01; p = 0.025), and open Experts in PP score (6.69 ± 11.48; p = 0.048). Laparoscopic experts showed no improvement. Participants with gaming experience had better overall scores than non-gamers when comparing all second attempts (Gamer 83.62 ± 7.57; Non-Gamer 76.31 ± 12.78; p = 0.008) as well as first and second attempts together (Gamer 72.08 ± 8.86; Non-Gamer 65.45 ± 11.68; p = 0.039). Musical and sports experience showed no correlation with robotic performance.

CONCLUSIONS

Robotic-assisted surgery requires skills distinct from conventional laparoscopy or open surgery. Basic robotic skills training prior to patient contact should be required.

Comparing open and minimally invasive surgical procedures for oesophagectomy in the treatment of cancer: the ROMIO (Randomised Oesophagectomy: Minimally Invasive or Open) feasibility study and pilot trial

BACKGROUND

Localised oesophageal cancer can be curatively treated with surgery (oesophagectomy) but the procedure is complex with a risk of complications, negative effects on quality of life and a recovery period of 6-9 months. Minimal-access surgery may accelerate recovery.

OBJECTIVES

The ROMIO (Randomised Oesophagectomy: Minimally Invasive or Open) study aimed to establish the feasibility of, and methodology for, a definitive trial comparing minimally invasive and open surgery for oesophagectomy. Objectives were to quantify the number of eligible patients in a pilot trial; develop surgical manuals as the basis for quality assurance; standardise pathological processing; establish a method to blind patients to their allocation in the first week post surgery; identify measures of postsurgical outcome of importance to patients and clinicians; and establish the main cost differences between the surgical approaches.

DESIGN

Pilot parallel three-arm randomised controlled trial nested within feasibility work.

SETTING

Two UK NHS departments of upper gastrointestinal surgery.

PARTICIPANTS

Patients aged ≥ 18 years with histopathological evidence of oesophageal or oesophagogastric junctional adenocarcinoma, squamous cell cancer or high-grade dysplasia, referred for oesophagectomy or oesophagectomy following neoadjuvant chemo(radio)therapy.

INTERVENTIONS

Oesophagectomy, with patients randomised to open surgery, a hybrid open chest and minimally invasive abdomen or totally minimally invasive access.

MAIN OUTCOME MEASURE

The primary outcome measure for the pilot trial was the number of patients recruited per month, with the main trial considered feasible if at least 2.5 patients per month were recruited.

RESULTS

During 21 months of recruitment, 263 patients were assessed for eligibility; of these, 135 (51%) were found to be eligible and 104 (77%) agreed to participate, an average of five patients per month. In total, 41 patients were allocated to open surgery, 43 to the hybrid procedure and 20 to totally minimally invasive surgery. Recruitment is continuing, allowing a seamless transition into the definitive trial. Consequently, the database is unlocked at the time of writing and data presented here are for patients recruited by 31 August 2014. Random allocation achieved a good balance between the arms of the study, which, as a high proportion of patients underwent their allocated surgery (69/79, 87%), ensured a fair comparison between the interventions. Dressing patients with large bandages, covering all possible incisions, was successful in keeping patients blind while pain was assessed during the first week post surgery. Postsurgical length of stay and risk of adverse events were within the typical range for this group of patients, with one death occurring within 30 days among 76 patients. There were good completion rates for the assessment of pain at 6 days post surgery (88%) and of the patient-reported outcomes at 6 weeks post randomisation (74%).

CONCLUSIONS

Rapid recruitment to the pilot trial and the successful refinement of methodology indicated the feasibility of a definitive trial comparing different approaches to oesophagectomy. Although we have shown a full trial of open compared with minimally invasive oesophagectomy to be feasible, this is necessarily based on our findings from the two clinical centres that we could include in this small preliminary study.

TRIAL REGISTRATION

Current Controlled Trials ISRCTN59036820.

FUNDING

This project was funded by the NIHR Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 20, No. 48. See the NIHR Journals Library website for further project information.

EAES classification of intraoperative adverse events in laparoscopic surgery

BACKGROUND

Surgical outcomes are traditionally evaluated by post-operative data such as histopathology and morbidity. Although these outcomes are reported using accepted systems, their ability to influence operative performance is limited by their retrospective application. Interest in direct measurement of intraoperative events is growing but no available systems applicable to routine practice exist. We aimed to develop a structured, practical method to report intraoperative adverse events enacted during minimal access surgical procedures.

METHODS

A structured mixed methodology approach was adopted. Current intraoperative adverse event reporting practices and desirable system characteristics were sought through a survey of the EAES executive. The observational clinical human reliability analysis method was applied to a series of laparoscopic total mesorectal excision (TME) case videos to identify intraoperative adverse events. In keeping with survey results, observed events were further categorised into non-consequential and consequential, which were further subdivided into four levels based upon the principle of therapy required to correct the event. A second survey phase explored usability, acceptability, face and content validity of the novel classification.

RESULTS

217 h of TME surgery were analysed to develop and continually refine the five-point hierarchical structure. 34 EAES expert surgeons (69%) responded. The lack of an accepted system was the main barrier to routine reporting. Simplicity, reproducibility and clinical utility were identified as essential requirements. The observed distribution of intraoperative adverse events was 60.1% grade I (non-consequential), 37.1% grade II (minor corrective action), 2.4% grade III (major correction or change in post-operative care) and 0.1% grade IV (life threatening). 84% agreed with the proposed classification (Likert scale 4.04) and 92% felt it was applicable to their practice and incorporated all desirable characteristics.

CONCLUSION

A clinically applicable intraoperative adverse event classification, which is acceptable to expert surgeons, is reported and complements the objective assessment of minimal access surgical performance.

Mentored Trainees have Similar Short-Term Outcomes to a Consultant Trainer Following Laparoscopic Colorectal Resection

BACKGROUND

Laparoscopic colorectal surgery has a long learning curve. Using a modular-based training programme may shorten this. Concerns with laparoscopic surgery have been oncological compromise and poor surgical outcomes when training more junior surgeons. This study aimed to compare operative and oncological outcomes between trainees undergoing a mentored training programme and a consultant trainer.

METHODS

A prospective study of all elective laparoscopic colorectal resections was undertaken in a single institution. Operative and oncological outcomes were recorded. All trainees were mentored by a National Laparoscopic Trainer (Lapco), and results between trainer and trainees compared.

RESULTS

Three hundred cases were included, with 198 (66%) performed for cancer. The trainer undertook 199 (66%) of operations, whilst trainees performed 101 (34%). Anterior resection was the commonest operation (n = 124, 41%). There were no differences between trainer and trainees for the majority of surgical outcomes, including blood loss (p = 0.598), conversion to open (p = 0.113), anastomotic leak (p = 0.263), readmission (p = 1.000) and death rates (p = 0.549). Only length of stay (p = 0.034), stoma formation (p < 0.01) and operative duration (p = 0.007) were higher in the trainer cohort, reflecting the more complex cases undertaken. Overall, there were no significant differences in both short- and longer-term oncology outcomes according to the grade of operating surgeon, including lymph nodes in specimen, circumferential resection margin and 1- and 2-year radiological recurrence.

CONCLUSION

When a modular-based training system was combined with case selection, both clinical and histopathological outcomes following resectional laparoscopic colorectal surgery were similar between trainees and trainer. This should encourage the use of more training opportunities in laparoscopic colorectal surgery.

Self-assessment in laparoscopic surgical skills training: Is it reliable?

BACKGROUND

The concept of self-assessment has been widely acclaimed for its role in the professional development cycle and self-regulation. In the field of medical education, self-assessment has been most used to evaluate the cognitive knowledge of students. The complexity of training and evaluation in laparoscopic surgery has previously acted as a barrier in determining the benefits self-assessment has to offer in comparison with other fields of medical education.

METHODS

Thirty-five surgical residents who attended the 2-day Laparoscopic Surgical Skills Grade 1 Level 1 curriculum were invited to participate from The Netherlands, India and Romania. The competency assessment tool (CAT) for laparoscopic cholecystectomy was used for self- and expert-assessment and the resulting distributions assessed.

RESULTS

A comparison between the expert- and self-assessed aggregates of scores from the CAT agreed with previous studies. Uniquely to this study, the aggregates of individual sub-categories-'use of instruments'; 'tissue handling'; and errors 'within the component tasks' and the 'end product' from both self- and expert-assessments-were investigated. There was strong positive correlation (r s > 0.5; p < 0.001) between the expert- and self-assessment in all categories with only the 'tissue handling' having a weaker correlation (r s = 0.3; p = 0.04). The distribution of the mean of the differences between self-assessment and expert-assessment suggested no significant difference between the scores of experts and the residents in all categories except the 'end product' evaluation where the difference was significant (W = 119, p = 0.03).

CONCLUSION

Self-assessment using the CAT form gives results that are consistently not different from expert-assessment when assessing one's proficiency in surgical skills. Areas where there was less agreement could be explained by variations in the level of training and understanding of the assessment criteria.

Objective assessment of minimally invasive total mesorectal excision performance: a systematic review

INTRODUCTION

Laparoscopy is widely used in colorectal practice, but recent trial results have questioned its use in rectal cancer resections. Patient outcomes are directly linked to the quality of total mesorectal excision (TME) specimen. Objective assessment of intraoperative performance could help ensure competence and delivery of optimal outcomes. Objective tools may also contribute to TME intervention trials, but their nature, structure and utilisation is unknown.

AIM

To systemically review the available literature to report on the available tools for the objective assessment of minimally invasive TME operative performance and their use within multicentre laparoscopic TME randomised controlled trials.

METHODS

A systematic search of the PubMed and Cochrane databases was performed to identify tools used in the objective intraoperative assessment of minimally invasive TME performance in accordance with the PRISMA guidelines, independently by two authors. The identified tools were then evaluated within reported TME RCTs.

RESULTS

A total of 8642 abstracts were screened of which 12 papers met the inclusion criteria; ten prospective observational studies, one randomised trial and one educational consensus. Eight assessment methods were described, which include formative and summative tools. The tools were mostly adaptations of colonic surgery tools based on either operative video review or post-operative trainer rating. All studies reported objective assessment of intraoperative performance was feasible, but only 126 (7%) of the 1762 included laparoscopic cases were TME. No multicentre laparoscopic TME trial reported using any objective surgical performance assessment tool.

CONCLUSION

Objective intraoperative laparoscopic TME performance assessment is feasible, but most of the current tools are adaptation of colonic surgery. There is a need to develop dedicated assessment tools for minimal access rectal surgery. No multicentre minimally invasive TME RCT reported using any objective assessment tool.

Training value of laparoscopic colorectal videos on the World Wide Web: a pilot study on the educational quality of laparoscopic right hemicolectomy videos

INTRODUCTION

Instructive laparoscopy videos with appropriate exposition could be ideal for initial training in laparoscopic surgery, but unfortunately there are no guidelines for annotating these videos or agreed methods to measure the educational content and the safety of the procedure presented. Aim of this study is to systematically search the World Wide Web to determine the availability of laparoscopic colorectal surgery videos and to objectively establish their potential training value.

METHODS

A search for laparoscopic right hemicolectomy videos was performed on the three most used English language web search engines Google.com, Bing.com, and Yahoo.com; moreover, a survey among 25 local trainees was performed to identify additional websites for inclusion. All laparoscopic right hemicolectomy videos with an English language title were included. Videos of open surgery, single incision laparoscopic surgery, robotic, and hand-assisted surgery were excluded. The safety of the demonstrated procedure was assessed with a validated competency assessment tool specifically designed for laparoscopic colorectal surgery and data on the educational content of the video were extracted.

RESULTS

Thirty-one websites were identified and 182 surgical videos were included. One hundred and seventy-three videos (95%) detailed the year of publication; this demonstrated a significant increase in the number of videos published per year from 2009. Characteristics of the patient were rarely presented, only 10 videos (5.4%) reported operating time and only 6 videos (3.2%) reported 30-day morbidity; 34 videos (18.6%) underwent a peer-review process prior to publication. Formal case presentation, the presence of audio narration, the use of diagrams, and snapshots and a step-by-step approach are all characteristics of peer-reviewed videos but no significant difference was found in the safety of the procedure.

CONCLUSIONS

Laparoscopic videos can be a useful adjunct to operative training. There is a large and increasing amount of material available for free on the internet, but this is currently unregulated.

Assessment of Surgical Competence: Development and Validation of Rating Scales Used for Laparoscopic Supracervical Hysterectomy

OBJECTIVE

To develop a procedure-specific rating scale for laparoscopic supracervical hysterectomy (LSH), and to compare the construct validity and reliability with a general rating scale in laparoscopic surgery, global operative assessment of laparoscopic skills (GOALS).

DESIGN

Prospective interobserver study. In collaboration with an expert group, we developed the procedure-specific rating scale, competence assessment tool for laparoscopic supracervical hysterectomy (CAT-LSH). LSH was performed by gynecologists with different levels of surgical competence levels (13 procedures were performed by inexperienced trainees, 13 by intermediate experienced, and 15 by laparoscopic experts). All procedures were video-recorded. Surgical performance was evaluated in all procedures using both CAT-LSH and GOALS by the surgical assistant, as well as by 2 blinded observers evaluating the video recordings.

SETTING

University teaching hospital.

PARTICIPANTS

Laparoscopic experts, consultants and gynecological registrars from the Department of Gynecology.

RESULTS

There were significant differences between the 3 proficiency groups in both the rating scales. Mean GOALS score evaluated by the operating assistant and the 2 observers were for inexperienced surgeons 16.4 vs. 13.6 (p < 0.01), for surgeons with intermediate experienced 22.6 vs. 19.5 (p < 0.05) and for expert surgeons 26.1 vs. 22.4 (p < 0.01), respectively. Corresponding results for the CAT-LSH scores were 41.0/34.6 (p < 0.01), 49.2/43.1 (p < 0.01), and 58.7/51.1 (p < 0.01), respectively. The interrater reliability measured by the interclass correlation coefficient between the surgical assistant and the 2 blinded observers for GOALS and CAT-LSH were 0.71 and 0.75, respectively.

CONCLUSIONS

The GOALS and CAT-LSH appear to have construct validity and high interrater reliability. Assessment of surgical competence during LSH is feasible in daily practice with objective rating scales like CAT-LSH and GOALS.

Education and Training to Address Specific Needs During the Career Progression of Surgeons

Surgeons have specific education and training needs as they enter practice, progress through the core period of active practice, and then as they wind down their clinical work before retirement. These transitions and the career progression process, combined with the dynamic health care environment, present specific opportunities for innovative education and training based on practice-based learning and improvement, and continuous professional development methods. Cutting-edge technologies, blended models, simulation, mentoring, preceptoring, and integrated approaches can play critical roles in supporting surgeons as they provide the best surgical care throughout various phases of their careers.

Laparoscopic and robot-assisted laparoscopic digestive surgery: Present and future directions

Laparoscopic surgery is applied today worldwide to most digestive procedures. In some of them, such as cholecystectomy, Nissen's fundoplication or obesity surgery, laparoscopy has become the standard in practice. In others, such as colon or gastric resection, the laparoscopic approach is frequently used and its usefulness is unquestionable. More complex procedures, such as esophageal, liver or pancreatic resections are, however, more infrequently performed, due to the high grade of skill necessary. As a result, there is less clinical evidence to support its implementation. In the recent years, robot-assisted laparoscopic surgery has been increasingly applied, again with little evidence for comparison with the conventional laparoscopic approach. This review will focus on the complex digestive procedures as well as those whose use in standard practice could be more controversial. Also novel robot-assisted procedures will be updated.

A competency framework for colonoscopy training derived from cognitive task analysis techniques and expert review

BACKGROUND

Colonoscopy is a difficult cognitive-perceptual-motor task. Designing an appropriate instructional program for such a task requires an understanding of the knowledge, skills and attitudes underpinning the competency required to perform the task. Cognitive task analysis techniques provide an empirical means of deriving this information.

METHODS

Video recording and a think-aloud protocol were conducted while 20 experienced endoscopists performed colonoscopy procedures. "Cued-recall" interviews were also carried out post-procedure with nine of the endoscopists. Analysis of the resulting transcripts employed the constant comparative coding method within a grounded theory framework. The resulting draft competency framework was modified after review during semi-structured interviews conducted with six expert endoscopists.

RESULTS

The proposed colonoscopy competency framework consists of twenty-seven skill, knowledge and attitude components, grouped into six categories (clinical knowledge; colonoscope handling; situation awareness; heuristics and strategies; clinical reasoning; and intra- and inter-personal).

CONCLUSIONS

The colonoscopy competency framework provides a principled basis for the design of a training program, and for the design of formative assessment to gauge progress towards attaining the knowledge, skills and attitudes underpinning the achievement of colonoscopy competence.

What errors make a laparoscopic cancer surgery unsafe? An ad hoc analysis of competency assessment in the National Training Programme for laparoscopic colorectal surgery in England

BACKGROUND

The National Training Programme for laparoscopic colorectal surgery in England was implemented to ensure training was supervised, structured, safe and effective. Delegates were required to pass a competency assessment (sign-off) before undertaking independent practice. This study described the types of errors identified and associated these errors with competency to progress to independent laparoscopic colorectal practice.

METHODS

All sign-off submissions from the start of the process in January 2008 until July 2013 were included. Content analysis was used to categorise errors. Bayes factor (BF) was used to measure the impact of individual error on assessment outcome. A smaller BF indicates that an error has stronger associations with unsuccessful assessments. Bayesian network was employed to graphically represent the reasoning process whereby the chance of successful assessment diminished with the identification of each error. Quality of the error feedback was measured by the area under the ROC curve which linked the predictions from the Bayesian model to the expert verdict.

RESULTS

Among 370 assessments analysed, 240 passed and 130 failed. On average, 2.5 different types of error were identified in each assessment. Cases that were more likely to fail had three or more different types of error (χ(2) = 72, p < 0.0001) and demonstrated poorer technical skills (CAT score <2.7, χ(2) = 164, p < 0.0001). Case complexity or right- versus left-sided resection did not have a significant impact. Errors associated with dissection (BF = 0.18), anastomosis (BF = 0.23) and oncological quality (BF = 0.19) were critical determinants of surgical competence, each reducing the odds of pass by at least fourfold. The area under the ROC curve was 0.84.

CONCLUSIONS

Errors associated with dissection, anastomosis and oncological quality were critical determinants of surgical competency. The detailed error analysis reported in this study can guide the design of future surgical education and clinical training programmes.

Clinical validity of consultant technical skills assessment in the English National Training Programme for Laparoscopic Colorectal Surgery

Background

The English National Training Programme for Laparoscopic Colorectal Surgery introduced a validated objective competency assessment tool to accredit surgeons before independent practice. The aim of this study was to determine whether this technical skills assessment predicted clinical outcomes.

Methods

Established consultants, training in laparoscopic colorectal surgery, were asked to submit two operative videos for evaluation by two blinded assessors using the competency assessment tool. A mark of 2·7 or above was considered a pass. Clinical and oncological outcomes were compared above and below this mark, including regression analysis.

Results

Eighty-five consultant surgeons submitted 171 videos. Of these, 44 (25·7 per cent) were in the fail group (score less than 2·7). This low scoring group had more postoperative morbidity (25 versus 8·7 per cent; P = 0·005), including surgical complications (18 versus 6·3 per cent; P = 0·020) and fewer lymph nodes harvested (median 13 versus 18; P = 0·004). A score of less than 2·7 was an independent predictor of surgical complication, lymph node yield and distal resection margin clearance. Consultants with higher scores had performed similar numbers of laparoscopic colorectal operations (median 37 versus 40; P = 0·373) but more structured training operations (18 versus 9; P &lt; 0·001).

Conclusion

An objective technical skills assessment provided a discriminatory tool with which to accredit laparoscopic colorectal surgeons.

Reliable and valid assessment of point-of-care ultrasonography

OBJECTIVE

To explore the reliability and validity of the Objective Structured Assessment of Ultrasound Skills (OSAUS) scale for point-of-care ultrasonography (POC US) performance.

BACKGROUND

POC US is increasingly used by clinicians and is an essential part of the management of acute surgical conditions. However, the quality of performance is highly operator-dependent. Therefore, reliable and valid assessment of trainees' ultrasonography competence is needed to ensure patient safety.

METHODS

Twenty-four physicians, representing novices, intermediates, and experts in POC US, scanned 4 different surgical patient cases in a controlled set-up. All ultrasound examinations were video-recorded and assessed by 2 blinded radiologists using OSAUS. Reliability was examined using generalizability theory. Construct validity was examined by comparing performance scores between the groups and by correlating physicians' OSAUS scores with diagnostic accuracy.

RESULTS

The generalizability coefficient was high (0.81) and a D-study demonstrated that 1 assessor and 5 cases would result in similar reliability. The construct validity of the OSAUS scale was supported by a significant difference in the mean scores between the novice group (17.0; SD 8.4) and the intermediate group (30.0; SD 10.1), P = 0.007, as well as between the intermediate group and the expert group (72.9; SD 4.4), P = 0.04, and by a high correlation between OSAUS scores and diagnostic accuracy (Spearman ρ correlation coefficient = 0.76; P < 0.001).

CONCLUSIONS

This study demonstrates high reliability as well as evidence of construct validity of the OSAUS scale for assessment of POC US competence. Hence, the OSAUS scale may be suitable for both in-training as well as end-of-training assessment.

Methods of quality assurance in multicenter trials in laparoscopic colorectal surgery: a systematic review

OBJECTIVES

To assess the risk of bias in multicenter randomized controlled trials (RCTs) investigating laparoscopic colorectal cancer surgery and review the use of quality assurance mechanisms to reduce performance bias.

BACKGROUND

RCTs represent the criterion standard comparison for health care interventions. For trials investigating interventional techniques, performance bias can arise through variation in delivery of the intervention.

METHODS

A comprehensive systematic review was undertaken using MEDLINE and EMBASE databases to identify all large RCTs investigating laparoscopic colorectal cancer surgery. Risk of performance bias was evaluated through assessment of publications and protocols to identify methods used for quality assurance of surgical technique. In addition, the Cochrane Collaboration's "risk of bias" tool was used to evaluate other potential sources of bias.

RESULTS

The literature search identified 48 publications, reporting upon 8 individual RCTs. All studies used mechanisms for quality assurance of laparoscopic colorectal surgery. Methods employed included credentialing of surgeons or units through assessment of experience and expertise, standardization of surgical technique, and monitoring. None report the use of structure objective assessment tools for accrediting expertise. All 8 were assessed as low risk of bias using the Cochrane tool. A framework is proposed for use as a model for quality assurance in future surgical trials.

CONCLUSIONS

Consideration of risk of performance bias is important when appraising trials investigating an interventional technique. Laparoscopic colorectal surgery RCTs have all employed quality assurance mechanisms to reduce risk of performance bias. Further research is indicated to investigate adopting objective assessment tools for quality assurance within multicenter RCTs.

Objective assessment of technique in laparoscopic colorectal surgery: what are the existing tools?

Assessment can improve the effectiveness of surgical training and enable valid judgments of competence. Laparoscopic colon resection surgery is now taught within surgical residency programs, and assessment tools are increasingly used to stimulate formative feedback and enhance learning. Formal assessment of technical performance in laparoscopic colon resection has been successfully applied at the specialist level in the English "LAPCO" National Training Program. Objective assessment tools need to be developed for training and assessment in laparoscopic rectal cancer resection surgery. Simulation may have a future role in assessment and accreditation in laparoscopic colorectal surgery; however, existing virtual reality models are not ready to be used for assessment of this advanced surgery.

Laparoscopy for colorectal cancer

The laparoscopic approach for colorectal cancer resection has been evolved from an experimental procedure with oncological concerns to routine daily practice within a period of two decades. Numerous randomized controlled trials and meta-analyses have shown that laparoscopic resection results in faster recovery with similar oncological outcome compared to an open approach, both for colon and rectal cancer. Besides improved cosmesis, other long-term advantages seem to be less adhesion related small bowel obstruction and reduced incisional hernia rate. Adequate patient selection and surgical experience are of crucial importance. Experience can be gradually expanded step by step, by increasing the complexity of the procedure. A decision to convert should be made early in the procedure, because the outcome after a reactive conversion is worse than initial open resection or strategic conversion. The additive value of new techniques such as robotic surgery has to be proven in randomized studies including a cost-effectiveness assessment.

Experts reviews of the multidisciplinary consensus conference colon and rectal cancer 2012: science, opinions and experiences from the experts of surgery

The first multidisciplinary consensus conference on colon and rectal cancer was held in December 2012, achieving a majority of consensus for diagnostic and treatment decisions using the Delphi Method. This article will give a critical appraisal of the topics discussed during the meeting and in the consensus document by well-known leaders in surgery that were involved in this multidisciplinary consensus process. Scientific evidence, experience and opinions are collected to support multidisciplinary teams (MDT) with arguments for medical decision-making in diagnosis, staging and treatment strategies for patients with colon or rectal cancer. Surgery is the cornerstone of curative treatment for colon and rectal cancer. Standardizing treatment is an effective instrument to improve outcome of multidisciplinary cancer care for patients with colon and rectal cancer. In this article, a review of the following focuses; Perioperative care, age and colorectal surgery, obstructive colorectal cancer, stenting, surgical anatomical considerations, total mesorectal excision (TME) surgery and training, surgical considerations for locally advanced rectal cancer (LARC) and local recurrent rectal cancer (LRRC), surgery in stage IV colorectal cancer, definitions of quality of surgery, transanal endoscopic microsurgery (TEM), laparoscopic colon and rectal surgery, preoperative radiotherapy and chemoradiotherapy, and how about functional outcome after surgery?

Randomized controlled trial on the effect of coaching in simulated laparoscopic training

BACKGROUND

The effect of coaching on surgical quality and understanding in simulated training remains unknown. The aim of this study was compare the effects of structured coaching and autodidactic training in simulated laparoscopic surgery.

METHODS

Seventeen surgically naive medical students were randomized into two groups: eight were placed into an intervention group and received structured coaching, and nine were placed into a control group and received no training. They each performed 10 laparoscopic cholecystectomies on a virtual reality simulator. The surgical quality of the first, fifth, and 10th operations was evaluated by 2 independent blinded assessors using the Competency Assessment Tool (CAT) for cholecystectomy. Understanding of operative strategy was tested before the first, fifth, and 10th operation. Performance metrics, path length, total number of movements, operating time, and error frequency were evaluated. The groups were compared by the Mann-Whitney U test. Proficiency gain curves were plotted using curve fit and CUSUM models; change point analysis was performed by multiple Wilcoxon signed rank analyses.

RESULTS

The intervention group scored significantly higher on the CAT assessment of procedures 1, 5, and 10, with increasing disparity. They also performed better in the knowledge test at procedures 5 and 10, again with an increasing difference. The learning curve for error frequency of the intervention group reached competency after operation 7, whereas the control group did not plateau by procedure 10. The learning curves of both groups for path length and number movements were almost identical; the mean operation time was shorter for the control group.

CONCLUSIONS

Clinically relevant markers of proficiency including error reduction, understanding of surgical strategy, and surgical quality are significantly improved with structured coaching. Path length and number of movements representing merely manual skills are developed with task repetition rather than influenced by coaching. Structured coaching may represent a key component in the acquisition of procedural skills.