Protocol for a scoping review on 'surgical sabermetrics:' technology-enhanced measurement of operative non-technical skills

Surgical Sabermetrics: A Scoping Review of Technology-enhanced Assessment of Nontechnical Skills in the Operating Room

OBJECTIVE

To evaluate the current evidence for surgical sabermetrics: digital methods of assessing surgical nontechnical skills and investigate the implications for enhancing surgical performance.

BACKGROUND

Surgeons need high-quality, objective, and timely feedback to optimize performance and patient safety. Digital tools to assess nontechnical skills have the potential to reduce human bias and aid scalability. However, we do not fully understand which of the myriad of digital metrics of performance assessment have efficacy for surgeons.

METHODS

A systematic review was conducted by searching PubMed, EMBASE, CINAHL, and PSYCINFO databases following PRISMA-ScR guidelines. MeSH terms and keywords included "Assessment," "Surgeons," and "Technology". Eligible studies included a digital assessment of nontechnical skills for surgeons, residents, and/or medical students within an operative context.

RESULTS

From 19,229 articles screened, 81 articles met the inclusion criteria. The studies varied in surgical specialties, settings, and outcome measurements. A total of 122 distinct objective, digital metrics were utilized. Studies digitally measured at least 1 category of surgical nontechnical skill using a single (n=54) or multiple objective measures (n=27). The majority of studies utilized simulation (n=48) over live operative settings (n=32). Surgical Sabermetrics has been demonstrated to be beneficial in measuring cognitive load (n=57), situation awareness (n=24), communication (n=3), teamwork (n=13), and leadership (n=2). No studies measured intraoperative decision-making.

CONCLUSIONS

The literature detailing the intersection between surgical data science and operative nontechnical skills is diverse and growing rapidly. Surgical Sabermetrics may provide a promising modifiable technique to achieve desirable outcomes for both the surgeon and the patient. This study identifies a diverse array of measurements possible with sensor devices and highlights research gaps, including the need for objective assessment of decision-making. Future studies may advance the integration of physiological sensors to provide a holistic assessment of surgical performance.

Instructional Video of a Standardized Interprofessional Postsimulation Facilitator-guided Debriefing of a Fatality in Plastic Surgery

Background

Postsimulation facilitator-guided debriefing (PSFGD) is the process of intentional discussion of thoughts, actions, and events that took place during simulation amongst the facilitator(s) and trainees. Despite the significance of delivering high-quality debriefings, there is a lack of evidence-based guidelines. Our study aimed to provide an instructional video demonstration of a PSFGD of a fatality.

Methods

Fifty surgical interns participated in a burn simulation scenario in two groups. Group 1 (control, or "no exposure," n = 25) consisted of residents who received oral postsimulation debriefing from an independent faculty member who had no exposure to our instructional video on how to debrief effectively. Group 2 (intervention, or "exposure," n = 25) consisted of interns who were debriefed by the second faculty member who did watch our instructional video before the simulation and learned about "advocacy and inquiry" techniques. The outcome measures were the Debriefing Assessment for Simulation in Healthcare score and the postdebrief multiple-choice question (MCQ) quiz scores to assess debriefers' performance and interns' knowledge consolidation, respectively.

Results

The "exposure" group presented statistically significantly higher values for the Debriefing Assessment for Simulation in Healthcare score (P < 0.001) and MCQ score (P < 0.001) compared with the "no exposure" group.

Conclusions

Debriefers who followed the methodology as demonstrated in our instructional video were considered more competent, and the residents achieved higher MCQ scores. The quality of the debriefing ensures improved critical thinking and problem-solving skills. Safer practice and better patient outcomes are achieved by developing debriefing programs for educators.