Evaluating competency in video-assisted thoracoscopic surgery (VATS) lobectomy performance using a novel assessment tool and virtual reality simulation

Thoracic surgery - An underestimated dream job?

Background

Thoracic surgery is an expanding surgical specialty, but a shortage of qualified and motivated thoracic surgery residents is expected. It is estimated that around 23 % of all surgical specialist jobs will be vacant by 2030. We therefore need to assess the current clinical situation and urgently improve the recruitment and training of our next generation of surgeons.

Methods

Using the online survey (Lime Survey), a questionnaire (28 questions) was created to analyze the current nationwide situation of residency in thoracic surgery. The survey was sent to all postgraduate trainees in thoracic surgery departments in Germany. The current status on residency-programs, scientific interest, the attractiveness of thoracic surgery, and the demographics of the participants were evaluated. The survey also evaluated specific ways to increase attractiveness.The survey was conducted in collaboration with "Frauen in der Thoraxchirurgie" by the "Junges Forum der Deutschen Gesellschaft für Thoraxchirurgie". The survey was sent to all residents and young specialists in Germany. A total of 187 participants responded.

Results

Out of 187 participants, 123 questionnaires (65.8 %) were completed. Mean age was 36.3 ± 6.9 years. 62.6 % of the participants were male. About 70 % work in an independent thoracic surgery department, while the rest is affiliated with another specialty department. 50 % have completed a doctorate in medicine.

Conclusion

The development of an objective and structured training plan could define the roles and responsibilities of the senior surgeon and the trainee, leading to improved training and, at the very least, ensuring good recruitment of junior thoracic surgeons.

Assessing Chest Tube Insertion Skills Using a Porcine Rib Model-A Validity Study

INTRODUCTION

Assessments require sufficient validity evidence before their use. The Assessment for Competence in Chest Tube Insertion (ACTION) tool evaluates proficiency in chest tube insertion (CTI), combining a rating scale and an error checklist. The aim of this study was to collect validity evidence for the ACTION tool on a porcine rib model according to the Messick framework.

METHODS

A rib model, consisting of a porcine hemithorax that was placed in a wooden frame, was used as simulator. Participants were recruited from the departments of surgery, pulmonology, and emergency medicine. After familiarization with the rib model and the equipment, standardized instructions and clinical context were provided. They performed 2 CTIs while being scored with the ACTION tool. All performances were assessed live by 1 rater and by 3 blinded raters using video recordings. Generalizability-analysis was performed and mean scores and errors of both groups on the first performance were compared. A pass/fail score was established using the contrasting groups' method.

RESULTS

Nine novice and 8 experienced participants completed the study. Generalizability coefficients where high for the rating scale (0.92) and the error checklist (0.87). In the first CTI, novices scored lower than the experienced group (38.1/68 vs. 47.1/68, P = 0.042), but no difference was observed on the error checklist. A pass/fail score of 44/68 was established.

CONCLUSION

A solid validity argument for the ACTION tool's rating scale on a porcine rib model is presented, allowing formative and summative assessment of procedural skills during training before patient contact.

The bibliometric analysis of extended reality in surgical training: Global and Chinese perspective

Objectives

The prospect of extended reality (XR) being integrated with surgical training curriculum has attracted scholars. However, there is a lack of bibliometric analysis to help them better understand this field. Our aim is to analyze relevant literature focusing on development trajectory and research directions since the 21st century to provide valuable insights.

Methods

Papers were retrieved from the Web of Science Core Collection. Microsoft Excel, VOSviewer, and CiteSpace were used for bibliometric analysis.

Results

Of the 3337 papers published worldwide, China contributed 204, ranking fifth. The world's enthusiasm for this field has been growing since 2000, whereas China has been gradually entering since 2001. Although China had a late start, its growth has accelerated since around 2016 due to the reform of the medical postgraduate education system and the rapid development of Chinese information technology, despite no research explosive period has been yet noted. International institutions, notably the University of Toronto, worked closely with others, while Chinese institutions lacked of international and domestic cooperation. Sixteen stable cooperation clusters of international scholars were formed, while the collaboration between Chinese scholars was not yet stable. XR has been primarily applied in orthopedic surgery, cataract surgery, laparoscopic training and intraoperative use in neurosurgery worldwide.

Conclusions

There is strong enthusiasm and cooperation in the international research on the XR-based surgical training. Chinese scholars are making steady progress and have great potential in this area. There has not been noted an explosive research phase yet in the Chinese pace. The research on several surgical specialties has been summarized at the very first time. AR will gradually to be more involved and take important role of the research.

Training simulator efficacy in developing thoracic and general surgical skills in a residency programme: a pilot study

OBJECTIVES

Virtual training simulators have been introduced in several surgical disciplines to improve residents' abilities. Through the use of the LapSim® virtual training simulator (Surgical Science, Göteborg, Sweden), this study aims to plan an effective learning path in minimally invasive thoracic and general surgery.

METHODS

All thoracic and general surgery trainees in their 1st and 2nd year of residency at the University of Insubria were enrolled and randomized into 2 groups: residents undergoing an intensive twice-a-week virtual training programme (group A: n = 8) and those undergoing a once-weekly non-intensive virtual training programme (group B: n = 9). The virtual training programme was divided into 4 modules, each of 12 weeks. In the 1st module, trainees repeated grasping, cutting, clip application, lifting and grasping, and fine dissection exercises during each training session. Seal-and-cut exercise was performed as the initial and final test. Data on surgical manoeuvres (time and on mistakes) were collected; intra- and inter-group comparisons were planned.

RESULTS

No significant differences were observed between groups A and B at the 1st session, confirming that the 2 groups had similar skills at the beginning. After 12 weeks, both groups showed improvements, but comparing data between initial and final test, only Group A registered a significant reduction in total time (P-value = 0.0015), left (P-value = 0.0017) and right (P-value = 0.0186) instrument path lengths, and in left (P-value = 0.0010) and right (P-value = 0.0073) instrument angular path lengths, demonstrating that group A acquired greater precision in surgical manoeuvres.

CONCLUSIONS

Virtual simulator training programme performed at least twice a week was effective for implementing basic surgical skills required for the trainee's professional growth. Additional virtual training modules focused on more complex exercises are planned to confirm these preliminary results.

Video-assisted thoracoscopic surgery simulation and training: a comprehensive literature review

BACKGROUND

Video-assisted thoracic surgery (VATS) has become the standard for lung cancer diagnosis and treatment. However, this surgical technique requires specific and dedicated training. In the past 20 years, several simulator systems have been developed to promote VATS training. Advances in virtual reality may facilitate its integration into the VATS training curriculum. The present review aims to first provide a comprehensive overview of the simulators for thoracoscopic surgery, focused especially on simulators for lung lobectomy; second, it explores the role and highlights the possible efficacy of these simulators in the surgical trainee curriculum.

METHODS

A literature search was conducted in the PubMed, EMBASE, Science Direct, Scopus and Web of Science databases using the following keywords combined with Boolean operators "AND" and "OR": virtual reality, VR, augmented reality, virtual simulation, mixed reality, extended reality, thoracic surgery, thoracoscopy, VATS, video-assisted thoracoscopic surgery, simulation, simulator, simulators, training, and education. Reference lists of the identified articles were hand-searched for additional relevant articles to be included in this review.

RESULTS

Different types of simulators have been used for VATS training: synthetic lung models (dry simulators); live animals or animal tissues (wet simulators); and simulators based on virtual or augmented reality. Their role in surgical training has been generally defined as useful. However, not enough data are available to ascertain which type is the most appropriate.

CONCLUSIONS

Simulator application in the field of medical education could revolutionize the regular surgical training curriculum. Further studies are required to better define their impact on surgeons' training programs and, finally, on patients' quality of care.

Training and assessment using the LapSim laparoscopic simulator: a scoping review of validity evidence

INTRODUCTION

The LapSim (Surgical Science, Sweden) laparoscopic simulator is a high-fidelity virtual reality simulator for use in endoscopic surgical training. This review critiques the current validity evidence for the LapSim laparoscopic simulator, specifically with respect to its potential use as a tool and method of training and assessment in surgery.

METHODS

A scoping review of the MEDLINE (PubMed), EMBASE, Cochrane and Web of Science databases was conducted in accordance with PRISMA guidelines (2020)-scoping review extension. Articles were included if they presented validity evidence for the use of the LapSim in operative skill training or assessment, in accordance with Messick's validity framework. European Association of Endoscopic Surgeons (EAES) guidelines (2005) were used to provide recommendations for the use of the LapSim in operative performance training and assessments.

RESULTS

Forty-nine articles were included. An EAES level 2 recommendation was provided with regard to the internal consistency reliability of automated performance metrics in assessing performance. An EAES recommendation of 2 was awarded with respect to the ability of the LapSim to discriminate based on case volume and overall laparoscopic experience (relationships with other variables). Performance assessment metrics on the LapSim correlate with improved performance in the operating room (EAES level of recommendation 1-consequential validity).

CONCLUSION

The LapSim has accumulated substantial evidence supporting the validity of its use in surgical training and assessment. Future studies should explore the relationship between the achievement of performance benchmarks on the LapSim and subsequent patient outcomes, and interrogate the benefits of implementing virtual reality simulation training and assessment curricula in post-graduate surgical training programmes.

Assessing VATS competence based on simulated lobectomies of all five lung lobes

OBJECTIVES

To determine the number of procedures and expert raters necessary to provide a reliable assessment of competence in Video-Assisted Thoracoscopic Surgery (VATS) lobectomy.

METHODS

Three randomly selected VATS lobectomies were performed on a virtual reality simulator by participants with varying experience in VATS. Video recordings of the procedures were independently rated by three blinded VATS experts using a modified VATS lobectomy assessment tool (VATSAT). The unitary framework of validity was used to describe validity evidence, and generalizability theory was used to explore the reliability of different assessment options.

RESULTS

Forty-one participants (22 novices, 10 intermediates, and 9 experienced) performed a total of 123 lobectomies. Internal consistency reliability, inter-rater reliability, and test-retest reliability were 0.94, 0.85, and 0.90, respectively. Generalizability theory found that a minimum of two procedures and four raters or three procedures and three raters were needed to ensure the overall reliability of 0.8. ANOVA showed significant differences in test scores between the three groups (P < 0.001). A pass/fail level of 19 out of 25 points was established using the contrasting groups' standard setting method, leaving one false positive (one novice passed) and zero false negatives (all experienced passed).

CONCLUSION

We demonstrated validity evidence for a VR simulator test with different lung lobes, and a credible pass/fail level was identified. Our results can be used to implement a standardized mastery learning training program for trainees in VATS lobectomies that ensures that everyone reaches basic competency before performing supervised operations on patients.

The utilization of educational resources published by the Thoracic Surgery Residents Association

OBJECTIVE

The Thoracic Surgery Residents Association (TSRA) is a trainee-led cardiothoracic surgery organization in North America that has published a multitude of educational resources. However, the utilization of these resources remains unknown.

METHODS

Surveys were constructed, pilot-tested, and emailed to 527 current cardiothoracic trainees (12 questions) and 780 former trainees who graduated between 2012 and 2019 (16 questions). The surveys assessed the utilization of TSRA educational resources in preparing for clinical practice as well as in-training and American Board of Thoracic Surgery (ABTS) certification examinations.

RESULTS

A total of 143 (27%) current trainees and 180 (23%) recent graduates responded. A higher proportion of recent graduates compared with current trainees identified as male (84% vs 66%; P = .001) and graduated from 2- or 3-year traditional training programs (81% vs 41%; P < .001), compared with integrated 6-year (8% vs 49%; P < .001) or 4 + 3 (11% vs 10%; P = .82) pathways. Current trainees most commonly used TSRA resources to prepare for the in-training exam (75%) and operations (73%). Recent graduates most commonly used them to prepare for Oral and/or Written Board Exams (92%) and the in-training exam (89%). Among recent graduates who passed the ABTS Oral Board Exam on the first attempt, 82% (97/118) used TSRA resources to prepare, versus only 48% (25/52) of recent graduates who passed after multiple attempts, failed, have not taken the exam, or preferred not to answer (P < .001).

CONCLUSIONS

Current cardiothoracic trainees and recent graduates have utilized TSRA educational resources extensively, including to prepare for in-training and ABTS Board examinations.

The use of objective assessments in the evaluation of technical skills in cardiothoracic surgery: a systematic review

OBJECTIVES

With reductions in training time and intraoperative exposure, there is a need for objective assessments to measure trainee progression. This systematic review focuses on the evaluation of trainee technical skill performance using objective assessments in cardiothoracic surgery and its incorporation into training curricula.

METHODS

Databases (EBSCOHOST, Scopus and Web of Science) and reference lists of relevant articles for studies that incorporated objective assessment of technical skills of trainees/residents in cardiothoracic surgery were included. Data extraction included task performed; assessment setting and tool used; number/level of assessors; study outcome and whether the assessments were incorporated into training curricula. The methodological rigour of the studies was scored using the Medical Education Research Study Quality Instrument (MERSQI).

RESULTS

Fifty-four studies were included for quantitative synthesis. Six were randomized-controlled trials. Cardiac surgery was the most common speciality utilizing objective assessment methods with coronary anastomosis the most frequently tested task. Likert-based assessment tools were most commonly used (61%). Eighty-five per cent of studies were simulation-based with the rest being intraoperative. Expert surgeons were primarily used for objective assessments (78%) with 46% using blinding. Thirty (56%) studies explored objective changes in technical performance with 97% demonstrating improvement. The other studies were primarily validating assessment tools. Thirty-nine per cent of studies had established these assessment tools into training curricula. The mean ± standard deviation MERSQI score for all studies was 13.6 ± 1.5 demonstrating high validity.

CONCLUSIONS

Despite validated technical skill assessment tools being available and demonstrating trainee improvement, their regular adoption into training curricula is lacking. There is a need to incorporate these assessments to increase the efficiency and transparency of training programmes for cardiothoracic surgeons.

Validity Evidence for Procedure-Specific Competency Assessment Tools in Cardiovascular and Thoracic Surgery: A Scoping Review

OBJECTIVE

Surgical education has shifted from a time-based approach to the achievement and demonstration of procedural competency. High quality, objective assessment instruments are required to support this new approach. This study comprehensively reviewed the literature to identify and evaluate available procedure-specific assessment instruments in cardiothoracic and vascular surgery.

DESIGN

A systematic search of 8 databases identified studies containing procedure-specific operative assessment instruments in cardiothoracic and vascular surgery. Generic global rating scales were excluded, unless modified to be procedure-specific. Two reviewers independently evaluated the validity evidence, methodological rigour and educational utility of each instrument using objective scoring criteria. Validity evidence was evaluated with a scoring tool aligned with the contemporary framework of validity. Methodological rigour was evaluated using the Medical Education Research Study Quality Instrument. Educational utility was evaluated according to the Accreditation Council for Graduate Medical Education (ACGME) framework.

RESULTS

There were 2130 unique studies describing procedure-specific assessment in surgery. Of these, 9 studies evaluating 8 procedure-specific assessment instruments met inclusion criteria for cardiothoracic and vascular surgery. Four instruments were identified in thoracic surgery, 2 in cardiac surgery, and 2 in vascular surgery. Only 1 instrument was designed to evaluate surgeon performance, with the remainder designed to evaluate residents. No single instrument scored the maximum score of 15 for validity evidence. The highest score was 11, with 62.5% (n = 5) of instruments scoring greater than 10. All tools attained high scores in content validity, with minimal evidence generally presented regarding the consequences of assessment using a particular instrument. All but 1 instrument scored greater than 11 out of a maximum 16.5 points for methodological rigour. Very few studies reported on the ACGME domains of educational utility.

CONCLUSIONS

In an era where surgical education is shifting towards the demonstration of procedural competency, objective procedure-specific assessment is critical. This review identified that few procedure-specific assessment instruments in cardiothoracic and vascular surgery exist, emphasizing the need for such instruments to ensure the success of competency-based education models.

Artificial intelligence assisted display in thoracic surgery: development and possibilities

In this golden age of rapid development of artificial intelligence (AI), researchers and surgeons realized that AI could contribute to healthcare in all aspects, especially in surgery. The popularity of low-dose computed tomography (LDCT) and the improvement of the video-assisted thoracoscopic surgery (VATS) not only bring opportunities for thoracic surgery but also bring challenges on the way forward. Preoperatively localizing lung nodules precisely, intraoperatively identifying anatomical structures accurately, and avoiding complications requires a visual display of individuals’ specific anatomy for surgical simulation and assistance. With the advance of AI-assisted display technologies, including 3D reconstruction/3D printing, virtual reality (VR), augmented reality (AR), and mixed reality (MR), computer tomography (CT) imaging in thoracic surgery has been fully utilized for transforming 2D images to 3D model, which facilitates surgical teaching, planning, and simulation. AI-assisted display based on surgical videos is a new surgical application, which is still in its infancy. Notably, it has potential applications in thoracic surgery education, surgical quality evaluation, intraoperative assistance, and postoperative analysis. In this review, we illustrated the current AI-assisted display applications based on CT in thoracic surgery; focused on the emerging AI applications in thoracic surgery based on surgical videos by reviewing its relevant researches in other surgical fields and anticipate its potential development in thoracic surgery.

Use of Generalizability Theory for Exploring Reliability of and Sources of Variance in Assessment of Technical Skills: A Systematic Review and Meta-Analysis

PURPOSE

Competency-based education relies on the validity and reliability of assessment scores. Generalizability (G) theory is well suited to explore the reliability of assessment tools in medical education but has only been applied to a limited extent. This study aimed to systematically review the literature using G-theory to explore the reliability of structured assessment of medical and surgical technical skills and to assess the relative contributions of different factors to variance.

METHOD

In June 2020, 11 databases, including PubMed, were searched from inception through May 31, 2020. Eligible studies included the use of G-theory to explore reliability in the context of assessment of medical and surgical technical skills. Descriptive information on study, assessment context, assessment protocol, participants being assessed, and G-analyses was extracted. Data were used to map G-theory and explore variance components analyses. A meta-analysis was conducted to synthesize the extracted data on the sources of variance and reliability.

RESULTS

Forty-four studies were included; of these, 39 had sufficient data for meta-analysis. The total pool included 35,284 unique assessments of 31,496 unique performances of 4,154 participants. Person variance had a pooled effect of 44.2% (95% confidence interval [CI], 36.8%-51.5%). Only assessment tool type (Objective Structured Assessment of Technical Skills-type vs task-based checklist-type) had a significant effect on person variance. The pooled reliability (G-coefficient) was 0.65 (95% CI, .59-.70). Most studies included decision studies (39, 88.6%) and generally seemed to have higher ratios of performances to assessors to achieve a sufficiently reliable assessment.

CONCLUSIONS

G-theory is increasingly being used to examine reliability of technical skills assessment in medical education, but more rigor in reporting is warranted. Contextual factors can potentially affect variance components and thereby reliability estimates and should be considered, especially in high-stakes assessment. Reliability analysis should be a best practice when developing assessment of technical skills.

Extended, virtual and augmented reality in thoracic surgery: a systematic review

OBJECTIVES

Extended reality (XR), encompassing both virtual reality (VR) and augmented reality, allows the user to interact with a computer-generated environment based on reality. In essence, the immersive nature of VR and augmented reality technology has been warmly welcomed in all aspects of medicine, gradually becoming increasingly feasible to incorporate into everyday practice. In recent years, XR has become increasingly adopted in thoracic surgery, although the extent of its applications is unclear. Here, we aim to review the current applications of XR in thoracic surgery.

METHODS

A systematic database search was conducted of original articles that explored the use of VR and/or augmented reality in thoracic surgery in EMBASE, MEDLINE, Cochrane database and Google Scholar, from inception to December 2020.

RESULTS

Our search yielded 1494 citations, of which 21 studies published from 2007 to 2019 were included in this review. Three main areas were identified: (i) the application of XR in thoracic surgery training; (ii) preoperative planning of thoracic procedures; and (iii) intraoperative assistance. Overall, XR could produce progression along the learning curve, enabling trainees to reach acceptable standards before performing in the operating theatre. Preoperatively, through the generation of 3D-renderings of the thoracic cavity and lung anatomy, VR increases procedural accuracy and surgical confidence through familiarization of the patient's anatomy. XR-assisted surgery may have therapeutic use particularly for complex cases, where conventional methods would yield inadequate outcomes due to inferior accuracy.

CONCLUSION

XR represents a salient step towards improving thoracic surgical training, as well as enhancing preoperative planning and intraoperative guidance.

ROBOT-ASSISTED VITREORETINAL SURGERY IMPROVES SURGICAL ACCURACY COMPARED WITH MANUAL SURGERY: A Randomized Trial in a Simulated Setting

Robot-assisted vitreoretinal surgery increases precision and limits tissue damage compared with manual surgery especially for the novice surgeon. The Eyesi Simulator is a feasible platform for investigating robot-assisted vitreoretinal surgery.

Purpose:

To compare manual and robot-assisted vitreoretinal surgery using a virtual-reality surgical simulator.

Methods:

Randomized controlled crossover study. Ten experienced vitreoretinal surgeons and 10 novice ophthalmic surgeons were included. The participants were randomized to start with either manual or robot-assisted surgery. Participants completed a test session consisting of three vitreoretinal modules on the Eyesi virtual-reality simulator. The automated metrics of performance supplied by the Eyesi simulator were used as outcome measures. Primary outcome measures were time with instruments inserted (seconds), instrument movement (mm), and tissue treatment (mm²).

Results:

Robot-assisted surgery was slower than manual surgery for both novices and vitreoretinal surgeons, 0.24 SD units (P = 0.024) and 0.73 SD units (P < 0.001), respectively. Robot-assisted surgery allowed for greater precision in novices and vitreoretinal surgeons, −0.96 SD units (P < 0.001) and −0.47 SD units (P < 0.001), respectively. Finally, novices using robot-assisted surgery inflicted less tissue damage when compared with that using manual surgery, −0.59 SD units (P = 0.009).

Conclusion:

At the cost of time, robot-assisted vitreoretinal surgery seems to improve precision and limit tissue damage compared with that of manual surgery. In particular, the performance of novice surgeons is enhanced with robot-assisted vitreoretinal surgery.

Auditorium of the future: e learning platform

Principles of surgical training have not changed, but methods of training are evolving very fast. Online tools are being adopted in both knowledge and skills training for surgical residents. As a result, to evaluate the outcome of these tools, online assessment is also developing. Knowledge resources are very diverse ranging from lectures, webinars, surgical videos to three-dimensional planning and printing. Skills resources include virtual reality simulators, remote skills training and interdisciplinary teamwork. Assessment of E-learning tools can be performed using online questions, task-based simulations, branching scenarios and online interviews/discussions. In thoracic surgery, video assisted thoracic surgery (VATS) lobectomy simulator has been developed and it appears to be an important tool for minimally invasive thoracic surgery education. Training programs incorporate e-Learning in their curriculum and online training and assessment will become an important part of thoracic surgical training as well.

Structured and evidence-based training of technical skills in respiratory medicine and thoracic surgery

Within the last decade, the number of technical procedures in respiratory medicine and thoracic surgery has grown and created a higher demand for effective and evidence-based education. Today, trainees are often allowed to perform procedures unsupervised on patients after obtaining a course certificate from a theoretic course and having performed a fixed number of supervised procedures. However, these methods do not ensure adequate competence. Well-structured and effective educational programmes including validated tests are needed to reduce economic expenses, optimize time spend, and ensure patient safety. The aim of this article is to summarize current state of educational strategies for technical procedures within respiratory medicine and thoracic surgery. Thus, to discuss future recommendations for curriculum development and assessment of competences based on Kern’s framework. The approach by Kern consists of six topics, which needs to be considered and evaluated and in order to educate physicians and surgeons most effective and evidence-based. We present a practical guide contributing to future educators’ considerations on (I) problem identification and general needs assessment, (II) targeted needs assessment, (III) goals and objectives, (IV) educational strategies, (V) implementation, and finally (VI) evaluation and feedback.

Training Thoracic Ultrasound Skills: A Randomized Controlled Trial of Simulation-Based Training versus Training on Healthy Volunteers

INTRODUCTION

As ultrasound becomes more accessible, the use of point-of-care ultrasound examinations performed by clinicians has increased. Sufficient theoretical and practical skills are prerequisites to integrate thoracic ultrasound into a clinical setting and to use it as supplement in the clinical decision-making. Recommendations on how to educate and train clinicians for these ultrasound examinations are debated, and simulation-based training may improve clinical performance.

OBJECTIVES

The aim of this study was to explore the effect of simulation-based training in thoracic ultrasound compared to training on healthy volunteers.

METHOD

A total of 66 physicians with no previous experience in thoracic ultrasound completed a training program and assessment of competences from November 2018 to May 2019. After a theoretical session in ultrasound physics, sonoanatomy, and thoracic ultrasound, the physicians were randomized into one of three groups for practical training: (1) simulation-based training, (2) training on a healthy volunteer, or (3) no training (control group). Primary outcome was difference in the clinical performance score after the training period.

RESULTS

Using a multiple comparison, ANOVA with Bonferroni correction for multiplicity, there was no statistical significant difference between the two trained groups' performance score: 45.1 points versus 41.9 points (minimum 17 points, maximum 68 points; p = 0.38). The simulation-based training group scored significantly higher than the control group without hands-on training, 36.7 points (p = 0.009).

CONCLUSIONS

The use of simulation-based training in thoracic ultrasound does not improve the clinical performance score compared to conventional training on healthy volunteers. As focused, thoracic ultrasound is a relatively uncomplicated practical procedure when taught; focus should mainly be on the theoretical part and the supervised clinical training in a curriculum. However, simulation can be used instead or as an add-on to training on simulated patients.

Reliable Assessment of Surgical Technical Skills Is Dependent on Context: An Exploration of Different Variables Using Generalizability Theory

PURPOSE

Reliable assessment of surgical skills is vital for competency-based medical training. Several factors influence not only the reliability of judgments but also the number of observations needed for making judgments of competency that are both consistent and reproducible. The aim of this study was to explore the role of various conditions-through the analysis of data from large-scale, simulation-based assessments of surgical technical skills-by examining the effects of those conditions on reliability using generalizability theory.

METHOD

Assessment data from large-scale, simulation-based temporal bone surgical training research studies in 2012-2018 were pooled, yielding collectively 3,574 assessments of 1,723 performances. The authors conducted generalizability analyses using an unbalanced random-effects design, and they performed decision studies to explore the effect of the different variables on projections of reliability.

RESULTS

Overall, 5 observations were needed to achieve a generalizability coefficient > 0.8. Several variables modified the projections of reliability: increased learner experience necessitated more observations (5 for medical students, 7 for residents, and 8 for experienced surgeons), the more complex cadaveric dissection required fewer observations than virtual reality simulation (2 vs 5 observations), and increased fidelity simulation graphics reduced the number of observations needed from 7 to 4. The training structure (either massed or distributed practice) and simulator-integrated tutoring had little effect on reliability. Finally, more observations were needed during initial training when the learning curve was steepest (6 observations) compared with the plateau phase (4 observations).

CONCLUSIONS

Reliability in surgical skills assessment seems less stable than it is often reported to be. Training context and conditions influence reliability. The findings from this study highlight that medical educators should exercise caution when using a specific simulation-based assessment in other contexts.

Neurosurgical Operative Videos: An Analysis of an Increasingly Popular Educational Resource

OBJECTIVE

Surgical education has increasingly relied on electronic learning. In particular, online operative videos have become a core resource within neurosurgery. We analyze the forums for neurosurgical operative videos.

METHODS

Operative videos from 5 sources were reviewed: 1) the NEUROSURGERY Journal YouTube channel; 2) the American Association of Neurological Surgeons Neurosurgery YouTube channel; 3) The Neurosurgical Atlas Operative Video Cases; 4) Operative Neurosurgery; and 5) Neurosurgical Focus: Video. Title, year of publication, senior author, institution, country, and subspecialty were documented for each video.

RESULTS

A total of 1233 videos showing 1247 surgeries were identified. Ten videos included >1 surgery; of those, there was a median of 2 surgeries (interquartile range, 2.0-2.5) per video. The most frequently represented subspecialties included vascular (48.3%), tumor (35.2%), and skull base surgery (27.5%), with almost 40% of videos showing >1 category. Videos were submitted by investigators from 28 countries, but 82.1% of the videos originated in the United States.

CONCLUSIONS

Neurosurgical operative videos have become increasingly common through a variety of online platforms. Future efforts may benefit from collecting videos from underrepresented regions and subspecialties, providing long-term follow-up data and showing techniques for managing complications.

Naples Prognostic Score as a novel prognostic prediction tool in video-assisted thoracoscopic surgery for early-stage lung cancer: a propensity score matching study

BACKGROUNDS

The Naples Prognostic Score (NPS) emerges as a novel prognostic scoring system in surgical oncology. We aim to assess the prognostic significance of preoperative NPS in patients undergoing completely video-assisted thoracoscopic surgery (VATS) lobectomy for early-stage non-small cell lung cancer (NSCLC) by propensity score matching (PSM) analysis.

METHODS

The present study was conducted on our single-center prospectively maintained database between January 2014 and December 2015. A Kaplan-Meier survival analysis using the log-rank test was used to distinguish differences in both overall survival (OS) and disease-free survival (DFS) between the patients stratified by preoperative NPS. Finally, multivariable Cox-proportional hazards regression analysis and PSM analysis were carried out to determine the independent prognostic factors for both OS and DFS.

RESULTS

There were 457 patients with operable primary stage I-II NSCLC included. Per 1-point increase in NPS was found to be significantly associated with unfavorable OS and DFS of NSCLC. Both OS and DFS were significantly shortened along with each number increase in the NPS group, showing a step-wise fashion. Such strong correlations between preoperative NPS and survival outcomes still remained validated after PSM analysis. In addition, NPS held the best discriminatory power for predicting both OS and DFS when compared to the other peripheral biomarkers. Multivariable analyses on the entire cohort and the PSM cohort demonstrated that preoperative NPS could be an independent prognostic indicator for both OS and DFS.

CONCLUSIONS

The NPS scoring system can serve as a novel risk stratification tool to refine prognostic prediction after VATS lobectomy for surgically resected NSCLC.

Simulation-based surgical education in cardiothoracic training

Simulation has emerged as a feasible adjunct to surgical education and training for most specialties. It provides trainees with an immersive, realistic way to learn a variety of skills in a safe environment with the end goal of improving patient safety. There are three broad types of simulators: full mannequin simulators, part-task trainers or bench models and virtual reality systems. This review aims to describe the current use of simulation in cardiothoracic surgical education and training. We identified multiple procedures that can be simulated in cardiothoracic surgery using a combination of the above simulators, three-dimensional printing and computer-based simulation. All studies that assessed the efficacy of simulators showed that simulation enhances learning and trainee performance allowing for repetitive training until the acquisition of competence but further research into how it translates into the operating theatre is required. In Australia, cardiac surgery simulation is not yet part of the training curricula, but simulators are available for certain tasks and procedures.