Postoperative video debriefing reduces technical errors in laparoscopic surgery

Structured feedback and operative video debriefing with critical view of safety annotation in training of laparoscopic cholecystectomy: a randomized controlled study

BACKGROUND

The learning curve in minimally invasive surgery (MIS) is lengthened compared to open surgery. It has been reported that structured feedback and training in teams of two trainees improves MIS training and MIS performance. Annotation of surgical images and videos may prove beneficial for surgical training. This study investigated whether structured feedback and video debriefing, including annotation of critical view of safety (CVS), have beneficial learning effects in a predefined, multi-modal MIS training curriculum in teams of two trainees.

METHODS

This randomized-controlled single-center study included medical students without MIS experience (n = 80). The participants first completed a standardized and structured multi-modal MIS training curriculum. They were then randomly divided into two groups (n = 40 each), and four laparoscopic cholecystectomies (LCs) were performed on ex-vivo porcine livers each. Students in the intervention group received structured feedback after each LC, consisting of LC performance evaluations through tutor-trainee joint video debriefing and CVS video annotation. Performance was evaluated using global and LC-specific Objective Structured Assessments of Technical Skills (OSATS) and Global Operative Assessment of Laparoscopic Skills (GOALS) scores.

RESULTS

The participants in the intervention group had higher global and LC-specific OSATS as well as global and LC-specific GOALS scores than the participants in the control group (25.5 ± 7.3 vs. 23.4 ± 5.1, p = 0.003; 47.6 ± 12.9 vs. 36 ± 12.8, p < 0.001; 17.5 ± 4.4 vs. 16 ± 3.8, p < 0.001; 6.6 ± 2.3 vs. 5.9 ± 2.1, p = 0.005). The intervention group achieved CVS more often than the control group (1. LC: 20 vs. 10 participants, p = 0.037, 2. LC: 24 vs. 8, p = 0.001, 3. LC: 31 vs. 8, p < 0.001, 4. LC: 31 vs. 10, p < 0.001).

CONCLUSIONS

Structured feedback and video debriefing with CVS annotation improves CVS achievement and ex-vivo porcine LC training performance based on OSATS and GOALS scores.

Effect of video-based self-reflection on intraoperative skills: A pilot randomized controlled trial

BACKGROUND

The value of video-based self-assessment in enhancing surgical skills is uncertain. This study investigates the feasibility and estimates sample size for a full-scale randomized controlled trial to evaluate the effectiveness of video-based self-assessment to improve surgical performance of laparoscopic cholecystectomy in trainees.

METHODS

This parallel pilot randomized controlled trial included general surgery trainees performing supervised laparoscopic cholecystectomy randomized 1:1 to control (traditional intraoperative teaching) or intervention group (traditional teaching plus video-based self-assessment). Operative performance was measured by the attending surgeon blinded to group assignment at the time of surgery using standardized assessment tools (Global Operative Assessment of Laparoscopic Skills and Operative Performance Rating System). The intervention group had access to their video recordings on a web-based platform for review and self-assessment using the same instruments. The primary outcome for the estimation of sample size was the difference in faculty-assessed final operative performance (third submitted case). Feasibility criteria included >85% participation, >85% adherence to case submission and >85% completion of self-assessment.

RESULTS

Of 37 eligible trainees approached, 32 consented and were randomized (86%). There were 16 in the intervention group, 15 in the control group (55% male, 55% junior trainees), and 1 was excluded for protocol violation. Twenty-four (75%) of participants submitted 3 cases. Thirteen trainees (81%) accessed the platform and completed 26 (63.2%) case self-assessments. Fifty-five trainees per arm will be needed to power a full-scale laparoscopic cholecystectomy with Global Operative Assessment of Laparoscopic Skills and 130 trainees per arm with Operative Performance Rating System as the assessment tool.

CONCLUSION

This pilot study contributes important data to inform the design of an adequately powered randomized controlled trial of video-based self-assessment to improve trainee performance of laparoscopic cholecystectomy. Although a priori trial feasibility criteria were not achieved, automated video capture and storage could significantly improve adherence in future trials.

Validity of video-based general and procedure-specific self-assessment tools for surgical trainees in laparoscopic cholecystectomy

INTRODUCTION

Self-review of recorded surgical procedures offers new opportunities for trainees to extend technical learning outside the operating-room. Valid tools for self-assessment are required prior to evaluating the effectiveness of video-review in enhancing technical learning. Therefore, we aimed to contribute evidence regarding the validity of intraoperative performance assessment tools for video-based self-assessment by general surgery trainees when performing laparoscopic cholecystectomies.

METHODS AND PROCEDURES

Using a web-based platform, general surgery trainees in a university-based residency program submitted recorded laparoscopic cholecystectomy procedures where they acted as the supervised primary surgeon. Attending surgeons measured operative performance at the time of surgery using general and procedure-specific assessment tools [Global Operative Assessment of Laparoscopic Skills (GOALS) and Operative Performance Rating System (OPRS), respectively] and entrustability level (O-SCORE). Trainees self-evaluated their performance from video-review using the same instruments. The validity of GOALS and OPRS for trainee self-assessment was investigated by testing the hypotheses that self-assessment scores correlate with (H1) expert assessment scores, (H2) O-SCORE, and (H3) procedure time and that (H4) self-assessment based on these instruments differentiates junior [postgraduate year (PGY) 1-3] and senior trainees (PGY 4-5), as well as (H5)simple [Visual Analogue Scale (VAS) ≤ 4] versus complex cases (VAS > 4). All hypotheses were based on previous literature, defined a priori, and were tested according to the COSMIN consensus on measurement properties.

RESULTS

A total of 35 videos were submitted (45% female and 45% senior trainees) and self-assessed. Our data supported 2 out of 5 hypotheses (H1 and H4) for GOALS and 3 out of 5 hypotheses (H1, H4 and H5) for OPRS, for trainee self-assessment.

CONCLUSIONS

OPRS, a procedure-specific assessment tool, was better able to differentiate between groups expected to have different levels of intraoperative performance, compared to GOALS, a general assessment tool. Given the interest in video-based learning, there is a need to further develop valid procedure-specific tools to support video-based self-assessment by trainees in a range of procedures.

Learning Curve of a Novel Three-Port Laparoscopic Radical Cystectomy with Urinary Diversion: A Single-Center Retrospective Analysis

Objective: Three-port laparoscopic radical cystectomy (LRC) is a novel method of radical cystectomy, which is being spread by our team in primary hospitals in our country. The purpose of this study was to evaluate the learning curve of urologists using this technique for bladder cancer patients. Methods: We retrospectively evaluated clinical data from patients with bladder cancer who received three-port LRC with urinary diversion at our medical center between January 2018 and December 2021. Consecutive cases were grouped according to different surgical years, and perioperative parameters among groups were assessed as variables for the learning curve, including operative time, estimated blood loss (EBL), lymph nodes (LN) yield, and postoperative hospital stay. Results: We assessed 154 patients who were divided into three groups, all of which were comparable in terms of preoperative characteristics. With the increase in surgical experience, the operation time of urologists is obviously reduced (P < .05), especially after 100 surgeries, whereas no statistically significant difference was observed in terms of EBL, LN yield, and postoperative hospital stay in the different surgical experience groups (P > .05). Conclusions: Our early learning curve experience indicates that the three-port LRC with urinary diversion is a safe and feasible technique that can be mastered by urologists after learning from a large sample. Given its advantages in cost and significantly improved learning curve, we recommend this technique to surgeons with extensive laparoscopic experience.

Operative Coaching for General Surgery Residents: Review of Implementation Requirements

Operative coaching offers a unique opportunity to strengthen surgery residents' skill sets and practice readiness. However, institutional organizational capacity may influence the ability to successfully implement and sustain a coaching program. This review concentrates on the implementation requirements as they relate to institutional organizational capacity to help evaluate and determine if adopting such a coaching model is feasible. We searched English-language, peer-reviewed articles concerning operative coaching of general surgery residents between 2000 and 2020 with the MEDLINE database. The abstracts of 267 identified articles were further screened based on the presence of 2 inclusion criteria: general surgery residents and operative coaching. Then we summarized the reported implementation requirements. Findings revealed the implementation requirements (ie people, processes, technology/support resources, physical resources, and organizational systems) of 3 major types of resident operative coaching models were different. Video-assisted coaching faces the most barriers to implementation followed by video-based coaching; in-person coaching encounters the least barriers. Six questions are generated helping residency education leaders assess their readiness for an operative coaching program. Evaluation of the implementation requirements of a desired coaching program using the 5 organizational capacity elements is recommended to ensure the residency's ability to achieve a successful and sustainable program.

Performance measures for the SACRED team-centered approach to advanced gastrointestinal endoscopy: European Society of Gastrointestinal Endoscopy (ESGE) Quality Improvement Initiative

The European Society of Gastrointestinal Endoscopy and United European Gastroenterology have defined performance measures for upper and lower gastrointestinal, pancreaticobiliary, and small-bowel endoscopy. Quality indicators to guide endoscopists in the growing field of advanced endoscopy are also underway. We propose that equal attention is given to developing the entire advanced endoscopy team and not the individual endoscopist alone.We suggest that the practice of teams intending to deliver high quality advanced endoscopy is underpinned by six crucial principles concerning: selection, acceptance, complications, reconnaissance, envelopment, and documentation (SACRED).

Video Coaching: A National Survey of Surgical Residency Program Directors

OBJECTIVE

Video coaching has been demonstrated to improve resident and attending skills and is overwhelmingly well received by the participants. However, misperceptions about its utility among those who do not use video coaching may be a barrier to widespread implementation.

DESIGN

Cross-sectional web-based survey SETTING: National survey PARTICIPANTS: Surgical program director members of the Association of Program Directors of Surgery STUDY DESIGN: The survey was developed via a deductive approach after a literature review and was piloted with surgical attendings and residents. All Likert scale were averaged and comparisons between groups was performed via independent t-tests.

RESULTS

There were 52 responses from PDs. 27/52(51.9%) PDs reported their program supported video coaching of residents. PDs from residences with video coaching programs were more likely to believe that video coaching was useful in identifying their own strengths and weakness (p = 0.005), was a useful adjunct for resident feedback (p = 0.024), and a personal library of video recordings would be helpful (p = 0.015) when compared to PDs from residencies without video coaching. Programs without video coaching were more likely to believe barriers to implementation included it being ineffective (p = 0.024) and that the technology was unavailable (p = 0.006). Over 50% of respondents from both groups believed expense, difficulty with set up, time required, and patient privacy were "Very" or "Extremely" likely to be barriers to implementation.

CONCLUSIONS

This is the first national survey of PDs regarding the use of video coaching. Residency programs without video coaching may underestimate the utility of video coaching in training surgical residents.

Video Coaching in Surgical Education: Utility, Opportunities, and Barriers to Implementation

OBJECTIVE

This review discusses the literature on Video-Based Coaching (VBC) and explores the barriers to widespread implementation.

DESIGN

A search was performed on Scopus and PubMed for the terms "operation," "operating room," "surgery," "resident," "house staff," "graduate medical education," "teaching," "coaching," "assessment," "reflection," "camera," and "video" on July 27, 2021, in English. This yielded 828 results. A single author reviewed the titles and abstracts and eliminated any results that did not pertain to operative VBC or assessment. All bibliographies were reviewed, and appropriate manuscripts were included in this study. This resulted in a total of 52 manuscripts included in this review.

SETTING/PARTICIPANTS

Original, peer-reviewed studies focused on VBC or assessment.

RESULTS

VBC has been both subjectively and objectively found to be a valuable educational tool. Nearly every study of video recording in the operating room found that subjects, including surgical residents and seasoned surgeons alike, overwhelmingly considered it a useful, non-redundant adjunct to their training. Most studies that evaluated skill acquisition via standardized assessment tools found that surgical residents who underwent a VBC program had significant improvements compared to their counterparts who did not undergo video review. Despite this evidence of effectiveness, fewer than 5% of residency programs employ video recording in the operating room. Barriers to implementation include significant time commitments for proposed coaching curricula and difficulty with integration of video cameras into the operating room.

CONCLUSIONS

VBC has significant educational benefits, but a scalable curriculum has not been developed. An optimal solution would ensure technical ease and expediency, simple, high-quality cameras, immediate review, and overcoming entrenched surgical norms and culture.

Exploring the nature and focus of feedback when using video playback in gynecology laparoscopy training

BACKGROUND

Feedback about intraoperative performance remains a cornerstone of surgical training. Video playback offers one potential method for more effective feedback to surgical residents. More research is needed to better understand this method. This study explores the nature of instructional interactions and feedback in the operating room (OR) and when using video playback during post-operative review in obstetrics and gynecology (OBGYN) training.

METHOD

This case study occurred between September 2016 and February 2017. Three OBGYN residents and five OBGYN supervising surgeons were involved in six laparoscopic cases. Intraoperative and video playback dialogues were recorded and analysed, the former deductively using codes identified from published literature, and the latter both deductively, using the same codes, and inductively, with codes that emerged from the data during analysis.

RESULTS

1090 intraoperative interactions were identified within 376 minutes of dialogue. Most interactions were didactic, instructing the resident how to use an instrument to perform a task. Deductive analysis of postoperative video playback review identified 146 interactions within 155 minutes. While the most common interaction type remained didactic, a teaching component was included more often. It became apparent that deductive analysis using the intraoperative codes did not adequately capture the nature and focus of feedback during video playback. Hermeneutic phenomenological analysis identified more dialogic video playback sessions with more resident-initiated questions and reflection.

CONCLUSIONS

This study demonstrates that the nature of feedback during video playback is fundamentally different from that in the OR, offering a greater potential for collaborative and improved learning.

A Comparative Analysis of Use of Google Glass versus. GoPro Intraoperative Video Recording of Scleral Buckle Surgery

Importance The scleral buckle technique is difficult to teach, given a decrease in the frequency of its use and the challenges of its video recording.

Background This study compares two different modalities for recording intraoperative videos of scleral buckle surgery.

Design Present study is a cross-sectional survey conducted in an academic hospital.

Participants A total of 36 medical students, ophthalmologic residents, and vitreoretinal surgery fellows participated in this survey.

Methods Five scleral buckle surgeries were filmed simultaneously using Google Glass and GoPro. Single-masked survey of trainees comparing video and audio quality of 38 10-second clips from each device, and listing the steps of scleral buckle surgery. Institutional review board approval and informed consent were obtained.

Main Outcome Measures Trainees were asked for grading the video device with respect to audio and video quality as favorable or unfavorable.

Results GoPro clips had more favorable ratings of video (73.3%) and audio (75.9%), compared with Google Glass (36.7 and 58.5%, respectively; p < 0.005). A total of 2.8% respondents listed the correct order of surgical steps in pretest, compared with 55.6% in posttest (p < 0.005).

Conclusion and Relevance Wearable recording devices can allow for video recording of scleral buckle surgery which has high utility in teaching trainees. GoPro and Google Glass each have distinct advantages.

System Factors Affecting Patient Safety in the OR: An Analysis of Safety Threats and Resiliency

OBJECTIVE

The objective of this study is to determine the characteristics and frequency of intraoperative safety threats and resilience supports using a human factors measurement tool.

BACKGROUND

Human factors analysis can provide insight into how system elements contribute to intraoperative adverse events. Empiric evidence on safety threats and resilience in surgical practice is lacking.

METHODS

A cross-sectional study of 24 patients undergoing elective laparoscopic general surgery at a single center in the Netherlands from May to November, 2017 was conducted. Video, audio, and patient physiologic data from all included procedures were obtained through a multichannel synchronized recording device. Trained analysts reviewed the recordings and coded safety threats and resilience supports. The codes were categorized into 1 of 6 categories (person, task, tools and technology, physical environment, organization, and external environment).

RESULTS

A median of 14 safety threats [interquartile range (IQR) 11-16] and 12 resilience supports (IQR 11-16) were identified per case. Most safety threat codes (median 9, IQR 7-12) and resilience support codes (median 10, IQR 7-12) were classified in the person category. The organization category contained a median of 2 (IQR 1-2) safety threat codes and 2 (IQR 2-3) resilience support codes per case. The tools and technology category contributed a small number of safety threats (median 1 per case, IQR 0-1), but rarely provided resilience support.

CONCLUSIONS

Through a detailed human factors analysis of elective laparoscopic general surgery cases, this study provided a quantitative analysis of the existing safety threats and resilience supports in a modern endoscopic operating room.

Application of technology to educational needs in surgery

Innovations in surgical education follow advancing clinical technology. New surgical methods have prompted demand for systematic methods to leverage computing power and internet tools to achieve proficiency-based training goals. Virtual reality, high-fidelity patient simulation, web-based resources to facilitate performance assessment, and telementoring have become mainstream practices, although patient outcomes benefits are not well studied. Remote virtual meeting and mentoring have had transformative effects on resident experiences, the full effects of which remain to be seen.

Learning in the Operating Theatre: A Thematic Analysis of Opportunities Lost and Found

OBJECTIVE

The operating theatre (OT) is an important learning environment. Trainees face barriers to learning in the OT that may reduce meaningful educational interactions. The impact of these barriers on the intraoperative learning experience of trainees and the strategies that they employ to overcome them are not known. This qualitative study aimed to describe the intraoperative learning experiences of senior general surgery trainees in Australia and their strategies to optimize learning in the OT.

DESIGN, SETTING, PARTICIPANTS

The authors developed a semi-structured interview guide based on published literature. Purposive sampling was used to identify a representative group of general surgery trainees in Australia, who were interviewed in a private setting with audio recordings deidentified for verbatim transcription and analysis. Thematic analysis was conducted using an interpretivist approach to produce a coding framework.

RESULTS

Ten trainees participated in the study. Themes were divided into external and internal barriers to learning, promoters of effective learning and actions to facilitate learning. External barriers included cultural neglect of an important issue, with inadequate prioritization of teaching and a lack of structure for intraoperative learning. From this, we identified the theme of missed opportunities. Internal barriers included difficulties in developing assertiveness required to address these issues and a failure to adequately plan for learning, with reliance on the mentor to initiate. Actions to facilitate learning were rarely employed by trainees, as most were unaware of strategies to maximize intraoperative learning.

CONCLUSIONS

Trainees find the barriers to learning in the OT difficult to address and are not well acquainted with strategies that may allow them to maximize their learning.

Maximizing Learning in the Operating Room: Residents' Perspectives

BACKGROUND

Few studies examine how residents can optimize their educational experience in the OR on their terms. This study aimed to examine residents' perceptions of how learners can maximize their education in the OR.

METHOD

Using constructivist grounded theory methodology, the authors conducted focus groups with general surgery residents, PGY1-5, followed by semi-structured interviews with attending surgeons from a single, academic medical center. Constant comparison was used to identify themes and explore their relationships. Theoretical sampling was used until saturation was achieved.

RESULTS

Residents and attendings participated. Two phases of OR learning were identified, intra-operative and inter-operative. Characters that made optimized learning included control, struggling, and reflection. Residents who practiced self-reflection with their experiences, and were able to articulate this awareness to attendings, felt the OR was an ideal learning environment. Attendings echoed similar findings.

CONCLUSIONS

Providing residents with a method of maximizing OR learning is critical to postgraduate clinical education. Currently, observation passively morphs into active learning and eventually independent operating in the OR. However, residents who practice self-regulated learning, and are able to discuss their educational goals with attendings, seem to find the OR a better learning environment and progress to independence more quickly. This was echoed by practicing attendings. Providing residents with a generalizable, self-regulated learning framework specific to operative educational experiences could maximize learning potential and expedite resident progression in the OR.

Association Between Hospital Debriefing Practices With Adherence to Resuscitation Process Measures and Outcomes for In-Hospital Cardiac Arrest

Background Identifying actionable resuscitation practices that vary across hospitals could improve adherence to process measures or outcomes after in-hospital cardiac arrest (IHCA). We sought to examine whether hospital debriefing frequency after IHCA varies across hospitals and whether hospitals which routinely perform debriefing have higher rates of process-of-care compliance or survival. Methods We conducted a nationwide survey of hospital resuscitation practices in April of 2018, which were then linked to data from the Get With The Guidelines-Resuscitation national registry for IHCA. Hospitals were categorized according to their reported frequency of debriefing immediately after IHCA; rarely (0%-20% of all IHCA cases), occasionally (21%-80%), and frequently (81%-100%). Hospital-level rates of timely defibrillation (≤2 minutes), epinephrine administration (≤5 minutes), survival to discharge, return of spontaneous circulation, and neurologically intact survival were comparted for patients with IHCA from 2015 to 2017. Results Overall, there were 193 hospitals comprising 44 477 IHCA events. Mean patient age was 65±16, 41% were females, and 68% were of White race. Across hospitals, 84 (43.5%) rarely performed debriefings immediately after an IHCA, 82 (42.5%) performed debriefing sessions occasionally, and 27 (14.0%) performed debriefing frequently. There was no association between higher reported debriefing frequency and hospital rates of timely defibrillation and epinephrine administration. Mean hospital rates of risk-standardized survival to discharge were similar across debriefing frequency groups (rarely 25.6%; occasionally 26.0%; frequently 25.2%, P=0.72), as were hospital rates of risk-adjusted return of spontaneous circulation (rarely 72.2%; occasionally 73.0%; frequently 70.0%, P=0.06) and neurologically intact survival (rarely 21.9%, occasionally 22.2%, frequently 21.1%, P=0.75). Conclusions In a large contemporary nationwide quality improvement registry, hospitals varied widely in how often they conducted debriefings immediately after IHCA. However, hospital debriefing frequency was not associated with better adherence to timely delivery of epinephrine or defibrillation or higher rates of IHCA survival.

Impact of targeted trainer feedback via video review on trainee performance of laparoscopic cholecystectomy

INTRODUCTION

Tools for improving operative performance for surgical trainees are increasingly desirable, particularly in the context of EWTD and 'run-through' training programmes. In addition, positive direct trainer feedback to trainees can improve skill acquisition and motivation, whilst negative feedback may have the opposite effect.¹ We aimed to examine the impact of targeted trainer feedback based on video analysis on trainee confidence and objective operative performance in laparoscopic cholecystectomy.

METHODS

Selected procedures designated as training cases were recorded. These were assessed by the trainers using the Independence-Scaled Procedural Assessment Score for laparoscopic cholecystectomy. Targeted feedback based on video review of selected procedures was then delivered by the trainers to the trainees. Trainees completed a self-reported questionnaire based on their response to this feedback. Subsequent to the feedback intervention, further training procedures were recorded and assessed.

RESULTS

A total of 6 trainees and 4 trainers participated in the study. For the pre-intervention assessment 15 cases were recorded, with a further 13 for the post-intervention assessment (total n = 28). The overall scores for the procedures performed post video feedback were improved, with a trend towards statistical significance (p = 0.08). However, there was a statistically significant improvement in the scores for performance of the triangle of Calot dissection after the feedback intervention (p = 0.009). The response rate to the questionnaire was 100%, with all trainees agreeing that they felt more confident and competent after the feedback intervention.

CONCLUSION

Targeted feedback to trainees based on post-procedure video review improves trainee confidence and may also improve performance. ACGME Core Competencies; Patient Care and Procedural Skills; Practice Based Learning and Improvement.

Explicating practice norms and tensions between values in resident training in family medicine

Background

Residency programs have the intricate and complex role of training health care providers. But little is known about what residents and attendings consider norms of practice or the tensions among different values residents are expected to uphold. Thus, dialogical and reflective frameworks are being explored for resident learning.

Methods

This study examined the use of facilitated conversations with groups of residents and attending physicians while reviewing video-recorded resident–patient interactions. The conversations were recorded, transcribed, and qualitatively analyzed.

Results

A total of 24 residents and 10 attendings participated in conversations while separately and in parallel groups reviewing 15 resident sessions. Residents explicated the norms of practice and evaluated their performance, which often agreed with those of attending physicians in calling out important learning opportunities. When disagreement occurred, residents’ explications of their reasoning were often relevant and, via reflection and dialogue, helped clarify intentions that were not apparent in the videos. Residents and attendings often judged actions on more than one domain of value. For instance, if a resident addressed problems, built relationships in a timely manner, and acted autonomously without jeopardizing the quality of care, she satisfactorily performed her duty.

Conclusions

Practice norms and value struggles were addressed by participants during reviews, which provided a promising framework for learning and assessment. Also, the non-hierarchical structure opened space to acknowledge a diversity of positions and for tensions among values to be explicated.

Implementing structured team debriefing using a Black Box in the operating room: surveying team satisfaction

Background

Surgical safety may be improved using a medical data recorder (MDR) for the purpose of postoperative team debriefing. It provides the team in the operating room (OR) with the opportunity to look back upon their joint performance objectively to discuss and learn from suboptimal situations or possible adverse events. The aim of this study was to investigate the satisfaction of the OR team using an MDR, the OR Black Box®, in the OR as a tool providing output for structured team debriefing.

Methods

In this longitudinal survey study, 35 gastro-intestinal laparoscopic operations were recorded using the OR Black Box® and the output was subsequently debriefed with the operating team. Prior to study, a privacy impact assessment was conducted to ensure alignment with applicable legal and regulatory requirements. A structured debrief model and an OR Back Box® performance report was developed. A standardized survey was used to measure participant’s satisfaction with the team debriefing, the debrief model used and the performance report. Factor analysis was performed to assess the questionnaire’s quality and identified contributing satisfaction factors. Multivariable analysis was performed to identify variables associated with participants’ opinions.

Results

In total, 81 team members of various disciplines in the OR participated, comprising 35 laparoscopic procedures. Mean satisfaction with the OR Black Box® performance report and team debriefing was high for all 3 identified independent satisfaction factors. Of all participants, 98% recommend using the OR Black Box® and the outcome report in team debriefing.

Conclusion

The use of an MDR in the OR for the purpose of team debriefing is considered to be both beneficial and important. Team debriefing using the OR Black Box® outcome report is highly recommended by 98% of team members participating.

Electronic supplementary material

The online version of this article (10.1007/s00464-020-07526-3) contains supplementary material, which is available to authorized users.

How to report educational videos in robotic surgery: an international multidisciplinary consensus statement

The swift endorsement of the robotic surgical platform indicates that it might prevail as the preferred technique for many complex abdominal and pelvic operations. Nonetheless, use of the surgical robotic system introduces further layers of complexity into the operating theatre necessitating new training models. Instructive videos with relevant exposition could be optimal for early training in robotic surgery and the aim of this study was to develop consensus guidelines on how to report a robotic surgery video for educational purposes to achieve high quality educational video outputs that could enhance surgical training. A steering group prepared a Delphi survey of 46 statements, which was distributed and voted on utilising an electronic survey tool. The selection of committee members was designed to include representative surgical trainers worldwide across different specialties, including lower and upper gastrointestinal surgery, general surgery, gynaecology and urology. 36 consensus statements were approved and classified in seven categories: author’s information and video introduction, case presentation, demonstration of the surgical procedure, outcomes of the procedure, associated educational content, review of surgical videos quality and use of surgical videos in educational curricula. Consensus guidelines on how to report robotic surgery videos for educational purposes have been elaborated utilising Delphi methodology. We recommend that adherence to the guidelines presented could support advancing the educational quality of video outputs when designed for training.

Training in Bariatric Surgery: a National Survey of German Bariatric Surgeons

INTRODUCTION

Skill in bariatric surgery has been associated with postoperative outcome. Appropriate surgical training is of paramount importance. In order to continuously improve training strategies, it is necessary to assess current practices.

AIM

To determine how German bariatric surgeons have been trained and to assess current training strategies.

METHODS

Between February 2017 and March 2017, an online census of surgeons registered as members of the German Society for Bariatric and Metabolic Surgery was conducted. A total of three reminders were sent out. Data were analyzed using descriptive statistics. Data was reported as median (interquartile range); percentages were adjusted for completed answers only.

RESULTS

A response rate of 51% (n = 214) was achieved. Surgeons reported a median of 14.5 (8-20) years of surgical experience after initial training, with a specific bariatric experience of 7 (4-13) years. The total cumulative bariatric case volume was 240 (80-500) cases, with an annual case volume of 50 (25-80). The most commonly applied approaches to bariatric skills acquisition were "learning by doing" (71%), "course participation" (70%) and "observerships" (70%). Fellowships and the use of operating videos were less frequently applied strategies (19%/ 47%). Interestingly, observerships (94%) and course participation (89%) were rated as very important/important, whereas "learning by doing" (62%), watching operation videos (59%), and fellowships (48%) were less frequently perceived as important/very important training strategies.

CONCLUSIONS

The majority of surgeons performing bariatric cases were senior surgeons with more than 10 years of post-training experience; nevertheless, the survey revealed a lack of structured approaches to bariatric specialization training.

The Utilization of Video Technology in Surgical Education: A Systematic Review

BACKGROUND

The use of surgical video has great potential to enhance surgical education, but there exists limited information about how to effectively use surgical videos. We performed a systematic review of video technology in surgical training and provided evidence-based recommendations for its effective use.

MATERIALS AND METHODS

A systematic review of literature on surgical video in residency education was conducted. All articles meeting inclusion criteria were evaluated for technical characteristics pertaining to video usage. Included studies were critically appraised using a quality-scoring system. Recommendations were provided for the effective implementation of video in surgical education based on associations with improved training outcomes.

RESULTS

Twenty articles met inclusion criteria. In these studies, the source of video acquisition was primarily laparoscopy (40.0% of papers), and the main perspective of video was endoscopy (45.0%). Features of videos included supplementation with other educational tools (55.0%), schematic diagrams or images (50.0%), audio (40.0%), and narration (25.0%). Videos were primarily viewed preoperatively (60.0%) or postoperatively (50.0%). The intended viewer for videos was usually residents (70.0%) but also included attendings/faculty (30.0%). When compared with a nonvideo training group, video training was associated with improved resident knowledge (100%), improved operative performance (81.3%), and greater participant satisfaction (100%).

CONCLUSIONS

Based on this review, we recommend that surgical training programs incorporate schematics and imaging into video, supplement video with other education tools, and utilize audio in video. For video review, we recommend that residents review video preoperatively and postoperatively for learning and that attendings review video postoperatively for assessment.

How can positive and negative trainer feedback in the operating theatre impact a surgical trainee's confidence and well-being: a qualitative study in the north of England

OBJECTIVE

To identify the perception of positive feedback (PF) and negative feedback (NF) provided by trainers in the operating theatre on surgical trainees' confidence and well-being.

DESIGN

Narrative interview study.

SETTING

Twelve hospitals that form part of one deanery within the UK.

PARTICIPANTS

Maximum variation sampling of 15 higher general surgical trainees provided insight into how PF and NF from trainers in the operating theatre affect confidence and well-being.

METHODS

Narrative telephone interviews were conducted with general surgical trainees between April and June 2016. All interviews were recorded, transcribed and anonymised. Transcriptions were analysed using the five-step framework analysis by two independent researchers.

RESULTS

Fifteen trainees (age 28-38 years) were interviewed (median interview time: 29 min). Thematic framework analysis identified nine themes within the data. PF, which included corrective feedback, helped the trainees to relax and seemed to enhance their operative performance. All trainees reported significant and unjustified NF, some of which would be defined as undermining and bullying. Many believed this to have a negative impact on their training with minimal educational benefit. Many trainees felt NF adversely affected their performance in the operating theatre with some expressing a wish to leave the profession as a consequence.

CONCLUSION

Both PF and NF exist in the operating theatre. Both have an important influence on the trainee, their performance and career. PF, if specific, helped aid progression of learning, increased motivation and performance of surgical trainees. In contrast, NF was perceived to have detrimental effects on trainees' performance and their well-being and, in some, introduced a desire to pursue an alternative career.

Video Coaching as an Efficient Teaching Method for Surgical Residents-A Randomized Controlled Trial

BACKGROUND

As surgical training is evolving and operative exposure is decreasing, new, effective, and experiential learning methods are needed to ensure surgical competency and patient safety. Video coaching is an emerging concept in surgery that needs further investigation.

DESIGN

In this randomized controlled trial conducted at a single teaching hospital, participating residents were filmed performing a side-to-side intestinal anastomosis on cadaveric dog bowel for baseline assessment. The Surgical Video Coaching (SVC) group then participated in a one-on-one video playback coaching and debriefing session with a surgeon, during which constructive feedback was given. The control group went on with their normal clinical duties without coaching or debriefing. All participants were filmed making a second intestinal anastomosis. This was compared to their first anastomosis using a 7-category-validated technical skill global rating scale, the Objective Structured Assessment of Technical Skills. A single independent surgeon who did not participate in coaching or debriefing to the SVC group reviewed all videos. A satisfaction survey was then sent to the residents in the coaching group.

SETTING

Department of Surgery, HôpitalMaisonneuve-Rosemont, tertiary teaching hospital affiliated to the University of Montreal, Canada.

PARTICIPANTS

General surgery residents from University of Montreal were recruited to take part in this trial. A total of 28 residents were randomized and completed the study.

RESULTS

After intervention, the SVC group (n = 14) significantly increased their Objective Structured Assessment of Technical Skills score (mean of differences 3.36, [1.09-5.63], p = 0.007) when compared to the control group (n = 14) (mean of differences 0.29, p = 0.759). All residents agreed or strongly agreed that video coaching was a time-efficient teaching method.

CONCLUSIONS

Video coaching is an effective and efficient teaching intervention to improve surgical residents' technical skills.

Learning curve patterns generated by a training method for laparoscopic small bowel anastomosis

Background

The identification of developmental curve patterns generated by a simulation-based educational method and the variables that can accelerate the learning process will result in cost-effective training. This study describes the learning curves of a simulation-based instructional design (ID) that uses ex vivo animal models to teach laparoscopic latero-lateral small bowel anastomosis.

Methods

Twenty general surgery residents were evaluated on their performance of laparoscopic latero-lateral jejuno-jejunal anastomoses (JJA) and gastro-jejunal anastomoses (GJA), using swine small bowel and stomach on an endotrainer. The ID included the following steps: (1) provision of references and videos demonstrating the surgical technique, (2) creation of an engaging context for learning, (3) critical review of the literature and video on the procedures, (4) demonstration of the critical steps, (5) hands-on practice, (6) in-action instructor’s feedback, (7) quality assessment, (8) debriefing at the end of the session, and (9) deliberate and repetitive practice. Time was recorded from the beginning to the completion of the procedure, along with the presence or absence of anastomotic leaks.

Results

The participants needed to perform 23.8 ± 6.96 GJA (12–35) and 24.2 ± 6.96 JJA (9–43) to attain proficiency. The starting point of the learning curve was higher for the GJA than for the JJA, although the slope and plateau were parallel. Further, four types of learning curves were identified: (1) exponential, (2) rapid, (3) slow, and (4) no tendency. The type of pattern could be predicted after procedure number 8.

Conclusions

These findings may help to identify the learning curve of a trainee early in the developmental process, estimate the number of sessions required to reach a performance goal, determine a trainee’s readiness to practice the procedure on patients, and identify the subjects who lack the innate technical abilities. It may help motivated individuals to become reflective and self-regulated learners. Moreover, the standardization of the ID may help to measure the effectiveness of learning strategies and make comparisons with other educational strategies.

Do You See What I See? How We Use Video as an Adjunct to General Surgery Resident Education

OBJECTIVE

Preparation of learners for surgical operations varies by institution, surgeon staff, and the trainees themselves. Often the operative environment is overwhelming for surgical trainees and the educational experience is substandard due to inadequate preparation. We sought to develop a simple, quick, and interactive tool that might assess each individual trainee's knowledge baseline before participating in minimally invasive surgery (MIS).

DESIGN

A 4-minute video with 5 separate muted clips from laparoscopic procedures (splenectomy, gastric band removal, cholecystectomy, adrenalectomy, and inguinal hernia repair) was created and shown to medical students (MS), general surgery residents, and staff surgeons. Participants were asked to watch the video and commentate (provide facts) on the operation, body region, instruments, anatomy, pathology, and surgical technique. Comments were scored using a 100-point grading scale (100 facts agreed upon by 8 surgical staff and trainees) with points deducted for incorrect answers. All participants were video recorded. Performance was scored by 2 separate raters.

SETTING

An academic medical center.

PARTICIPANTS

MS = 10, interns (n = 8), postgraduate year 2 residents (PGY)2s (n = 11), PGY3s (n = 10), PGY4s (n = 9), PGY5s (n = 7), and general surgery staff surgeons (n = 5).

RESULTS

Scores ranged from -5 to 76 total facts offered during the 4-minute video examination. MS scored the lowest (mean, range; 5, -5 to 8); interns were better (17, 4-29), followed by PGY2s (31, 21-34), PGY3s (33, 10-44), PGY4s (44, 19-47), PGY5s (48, 28-49), and staff (48, 17-76), p < 0.001. Rater concordance was 0.98-measured using a concordance correlation coefficient (95% CI: 0.96-0.99). Only 2 of 8 interns acknowledged the critical view during the laparoscopic cholecystectomy video clip vs 10 of 11 PGY2 residents (p < 0.003). Of 8 interns, 7 misperceived the spleen as the liver in the splenectomy clip vs 2 of 7 chief residents (p = 0.02).

CONCLUSIONS

Not surprisingly, more experienced surgeons were able to relay a larger number of laparoscopic facts during a 4-minute video clip of 5 MIS operations than inexperienced trainees. However, even tenured staff surgeons relayed very few facts on procedures they were unfamiliar with. The potential differentiating capabilities of such a quick and inexpensive effort has pushed us to generate better online learning tools (operative modules) and hands-on simulation resources for our learners. We aim to repeat this and other studies to see if our learners are better prepared for video assessment and ultimately, MIS operations.

The effect of video review of resident laparoscopic surgical skills measured by self- and external assessment

BACKGROUND

Video review of surgical skills is an educational modality that allows trainees to reflect on self-performance. The purpose of this study was to determine whether resident and attending assessments of a resident's laparoscopic performance differ and whether video review changes assessments.

METHODS

Third-year surgery residents were invited to participate. Elective laparoscopic procedures were video recorded. The Global Operative Assessment of Laparoscopic Skills evaluation was completed immediately after the procedure and again 7 to 10 days later by both resident and attending. Scores were compared using t tests.

RESULTS

Nine residents participated and 76 video reviews were completed. Residents scored themselves significantly lower than the faculty scores both before and after video review. Resident scores did not change significantly after video review.

CONCLUSIONS

Attending and resident self-assessment of laparoscopic skills differs and subsequent video review does not significantly affect Global Operative Assessment of Laparoscopic Skills scores. Further studies should evaluate the impact of video review combined with verbal feedback on skill acquisition and assessment.

A cross-sectional pilot study to investigate patient attitudes and perception regarding the use of real time digital recording of urological procedures for research and teaching purposes

Little conclusive evidence exists regarding the best way to educate and evaluate skill acquisition of advanced surgical trainees, despite it being recognised as one of the most important aspects of training. Many laparoscopic trainers have been produced with complex engineering at great cost, but, there seems to be a reluctance to use the most precious entity available to us; the patient. We thus propose the use of real time digital recording of urological procedures for research and teaching purposes. This study was prompted by the lack of literature regarding such issues. A 19 question questionnaire was circulated at a single urology out-patient department (Essex, England) over a 6 month period to evaluate attitudes and perceptions of urological patients on potentially having their procedure digitally recorded for educational and research purposes. 11 patients declined, 187 questionnaires were included in the final analysis. Male patients are more willing to consent than female patients. Older patients resulted to have a higher propensity in being recorded for medical teaching. Greater than 50% believe being recorded is intrusive but the majority do not think privacy is an issue. Lastly, the vast majority require a formal debrief post operatively. Our results show that a percentage of the public are potentially willing to be digitally recorded but many financial and social barriers exist. We have also highlighted areas of possible future research, namely the reluctance behind young urology patients to consent and questions regarding how best to educate possible study participants to ensure proper informed consent is gained.

How surgical mentors teach: a classification of in vivo teaching behaviors part 1: verbal teaching guidance

OBJECTIVES

To illuminate surgical teaching at a fine level of detail by filming intraoperative communication between surgical attending physicians and trainees and provide a naturalistic categorization and analysis of verbal teaching behaviors.

DESIGN

Live, intraoperative verbal exchanges between surgical attending physicians and their trainees (residents and fellows) were filmed, and key verbal teaching moments were transcribed. In follow-up interviews, attending physicians and trainees watched video clips of their teaching case and answered open-ended questions about their surgical teaching methods. Using a grounded theory approach, we examined the videos and interviews for what might be construed as a teaching behavior and refined verbal teaching categories through constant comparison.

SETTING

We filmed 5 cases in the operating suite of a university teaching hospital that provides gynecologic surgical care.

PARTICIPANTS

We included 5 attending gynecologic surgeons, 3 fellows, and 5 residents for this study.

RESULTS

More than 6 hours of film, 3 hours of interviews, and more than 400 verbal teaching utterances from our participating attending surgeons were transcribed. We found that attending surgeons used unique types of verbal guidance to describe relevant anatomy, explain the rationale behind a specific surgical action, command the trainee to perform the next step, reference a specific aspect of the surgery, and provide an indirect verbal construct. Attending physicians prefixed speech with polite terms and used terse language, colorful verbal analogies, and sometimes humor. Our participants denied a significant Hawthorne effect. Interrater reliability was high using Cohen κ with 0.77 for the verbal categories.

CONCLUSIONS

Our categorization of live intraoperative verbal teaching can provide a measurable, replicable basis for studying how spoken guidance can lead to the best intraoperative learning. Because surgical teaching occurs on a microscopic level, film review is important when analyzing intraoperative teaching behaviors.

Comprehensive feedback on trainee surgeons' non-technical skills

OBJECTIVE

This study aimed to explore the content of conversations, feedback style, and perceived usefulness of feedback to trainee surgeons when conversations were stimulated by a tool for assessing surgeons' non-technical skills.

METHODS

Trainee surgeons and their supervisors used the Non-Technical Skills for Surgeons in Denmark tool to stimulate feedback conversations. Audio recordings of post-operation feedback conversations were collected. Trainees and supervisors provided questionnaire responses on the usefulness and comprehensiveness of the feedback. The feedback conversations were qualitatively analyzed for content and feedback style. Usefulness was investigated using a scale from 1 to 5 and written comments were qualitatively analyzed.

RESULTS

Six trainees and six supervisors participated in eight feedback conversations. Eighty questionnaires (response rate 83 percent) were collected from 13 trainees and 12 supervisors. Conversations lasted median eight (2-15) minutes. Supervisors used the elements and categories in the tool to structure the content of the conversations. Supervisors tended to talk about the trainees' actions and their own frames rather than attempting to understand the trainees' perceptions. Supervisors and trainees welcomed the feedback opportunity and agreed that the conversations were useful and comprehensive.

CONCLUSIONS

The content of the feedback conversations reflected the contents of the tool and the feedback was considered useful and comprehensive. However, supervisors talked primarily about their own frames, so in order for the feedback to reach its full potential, supervisors may benefit from training techniques to stimulate a deeper reflection among trainees.

A Comprehensive Methodology for Examining the Impact of Surgical Team Briefings and Debriefings on Teamwork

The adoptions of briefing and debriefing protocols have evolved from the Joint Commission’s initiative to improve communication and safety in the operating room. Briefing normally occurs prior to incision and is used to discuss and confirm critical information, while debriefing occurs during or after surgery. Debriefing provides a unique opportunity for individuals and teams to immediately reflect on their performance, allowing them to more easily identify errors and develop plans to improve their next performance. Studies have shown that using briefings and debriefings improve communication and teamwork. However, there is still much to learn about the value of both for surgical teams. This paper presents a robust methodology for examining and measuring the impacts of surgical team briefings and debriefings on teamwork. The methodology includes (1) audio/video recording the surgical care process, (2) prospective observations using a validated electronic data collection tool, (3) pre- and post-surgery surveys, and (4) individual surgical team member interviews. The current paper describes the methodology to obtain a robust and comprehensive data set for analyzing the impacts of briefing and debriefing on teamwork; the results of the surgeries recorded using this methodology will be presented in subsequent papers.

Operation debrief: a SHARP improvement in performance feedback in the operating room

OBJECTIVES

To explore the current status of performance feedback (debriefing) in the operating room and to develop and evaluate an evidence-based, user-informed intervention termed "SHARP" to improve debriefing in surgery.

BACKGROUND

Effective debriefing is a key educational technique for optimizing learning in surgical settings. However, there is a lack of a debriefing culture within surgery. Few studies have prospectively evaluated educational interventions to improve the quality and quantity of performance feedback in surgery.

METHODS

This was a prospective pre- and post-study of 100 cases involving 22 trainers (attendings) and 30 surgical residents (postgraduate years 3-8). A trained researcher assessed the quality of debriefings provided to the trainee using the validated Objective Structured Assessment of Debriefing (OSAD) tool alongside ethnographic observation. Following the first 50 cases, an educational intervention termed "SHARP" was introduced and measures repeated for a further 50 cases. User satisfaction with SHARP was assessed via questionnaire. Twenty percent of the cases were observed independently by a second researcher to test interrater reliability.

RESULTS

Interrater reliability for OSAD was excellent (ICC = 0.994). Objective scores of debriefing (OSAD) improved significantly after the SHARP intervention: median pre = 19 (range, 8-31); median post = 33 (range, 26-40), P < 0.001. Strong correlations between observer (OSAD) and trainee rating of debriefing were obtained (median ρ = 0.566, P < 0.01). Ethnographic observations also supported a significant improvement in both quality and style of debriefings. Users reported high levels of satisfaction in terms of usefulness, feasibility, and comprehensiveness of the SHARP tool.

CONCLUSIONS

SHARP is an effective and efficient means of improving performance feedback in the operating room. Its routine use should be promoted to optimize workplace-based learning and foster a positive culture of debriefing and performance improvement within surgery.

Defining technical errors in laparoscopic surgery: a systematic review

BACKGROUND

Technical errors, a distinct subcomponent of surgical proficiency, have a significant impact on patient safety and clinical outcomes. To date, only a few studies have been designed to describe and evaluate these errors. This review was performed to assess technical errors described in laparoscopic surgery.

METHODS

A literature search of Medline, Cochrane, EMBASE, and OVID databases (1946-2012, week 14) using the terms "technical/medical error," "technical skill," and "adverse event" in combination with the terms "laparoscopy/laparoscopic surgery" was conducted. English language peer review articles with a description of technical errors were included. Opinion papers, reviews, and articles not addressing laparoscopic surgery were excluded.

RESULTS

The search returned 2,282 articles. Application of the inclusion criteria reduced the number of articles to 21. Of these 21 articles, 14 (67 %) were observational studies, 3 (14 %) were randomized trials, 2 (10 %) were prospective interventional studies, and 2 (10 %) were retrospective analyses. Eight articles (38 %) applied error analysis as an approach to determine error rates within routine procedures. The remaining 13 articles (62 %) used the assessment of errors to describe and quantify surgical skill in an educational setting.

CONCLUSIONS

A number of approaches for the assessment of surgical technical errors exist. The error definitions vary greatly, making a comparison of error rates between groups impossible. Complexity of scale design and subjectivity in ratings have resulted in limited use of these scores outside the experimental setting. To facilitate error analysis as a self-assessment method of continuous learning and quality control, further research and better tools are required.

Do team and individual debriefs enhance performance? A meta-analysis

OBJECTIVE

Debriefs (or "after-action reviews") are increasingly used in training and work environments as a means of learning from experience. We sought to unify a fragmented literature and assess the efficacy of debriefs with a quantitative review.

BACKGROUND

Used by the U.S. Army to improve performance for decades, and increasingly in medical, aviation, and other communities, debriefs systematize reflection, discussion, and goal setting to promote experiential learning. Unfortunately, research and theory on debriefing has been spread across diverse disciplines, so it has been difficult to definitively ascertain debriefing effectiveness and how to enhance its effectiveness.

METHOD

We conducted an extensive quantitative meta-analysis across a diverse body of published and unpublished research on team- and individual-level debriefs.

RESULTS

Findings from 46 samples (N = 2,136) indicate that on average, debriefs improve effectiveness over a control group by approximately 25% (d = .67). Average effect sizes were similar for teams and individuals, across simulated and real settings, for within- or between-group control designs, and for medical and nonmedical samples. Meta-analytic methods revealed a bolstering effect of alignment and the potential impact of facilitation and structure.

CONCLUSION

Organizations can improve individual and team performance by approximately 20% to 25% by using properly conducted debriefs.

APPLICATION

Debriefs are a relatively inexpensive and quick intervention for enhancing performance. Our results lend support for continued and expanded use of debriefing in training and in situ. To gain maximum results, it is important to ensure alignment between participants, focus and intent, and level of measurement.

Objective structured assessment of debriefing: bringing science to the art of debriefing in surgery

OBJECTIVE

: To identify the features of effective debriefing and to use this to develop and validate a tool for assessing such debriefings.

INTRODUCTION

: Simulation-based training has become an accepted means of surgical skill acquisition. A key component of this is debriefing-yet there is a paucity of research to guide best practice.

METHODS

: Phase 1-Identification of best practice and tool development. A search of the Medline, Embase, PsycINFO, and ERIC databases identified current evidence on debriefing. End-user input was obtained through 33 semistructured interviews conducted with surgeons (n = 18) and other operating room personnel (n = 15) from 3 continents (UK, USA, Australia) using standardized qualitative methodology. An expert panel (n = 7) combined the data to create the Objective Structured Assessment of Debriefing (OSAD) tool. Phase 2-Psychometric testing. OSAD was tested for feasibility, reliability, and validity by 2 independent assessors who rated 20 debriefings following high-fidelity simulations.

RESULTS

: Phase 1: 28 reports on debriefing were retrieved from the literature. Key components of an effective debriefing identified from these reports and the 33 interviews included: approach to debriefing, learning environment, learner engagement, reaction, reflection, analysis, diagnosis of strengths and areas for improvement, and application to clinical practice. Phase 2: OSAD was feasible, reliable [inter-rater ICC (intraclass correlation coefficient) = 0.88, test-retest ICC = 0.90], and face and content valid (content validity index = 0.94).

CONCLUSIONS

: OSAD provides an evidence-based, end-user informed approach to debriefing in surgery. By quantifying the quality of a debriefing, OSAD has the potential to identify areas for improving practice and to optimize learning during simulation-based training.