Machine Learning Identification of Surgical and Operative Factors Associated With Surgical Expertise in Virtual Reality Simulation

Expertise in surgical neuro-oncology. Results of a survey by the EANS neuro-oncology section

Introduction

Technical advances and the increasing role of interdisciplinary decision-making may warrant formal definitions of expertise in surgical neuro-oncology.

Research question

The EANS Neuro-oncology Section felt that a survey detailing the European neurosurgical perspective on the concept of expertise in surgical neuro-oncology might be helpful.

Material and methods

The EANS Neuro-oncology Section panel developed an online survey asking questions regarding criteria for expertise in neuro-oncological surgery and sent it to all individual EANS members.

Results

Our questionnaire was completed by 251 respondents (consultants: 80.1%) from 42 countries. 67.7% would accept a lifetime caseload of >200 cases and 86.7% an annual caseload of >50 as evidence of neuro-oncological surgical expertise. A majority felt that surgeons who do not treat children (56.2%), do not have experience with spinal fusion (78.1%) or peripheral nerve tumors (71.7%) may still be considered experts. Majorities believed that expertise requires the use of skull-base approaches (85.8%), intraoperative monitoring (83.4%), awake craniotomies (77.3%), and neuro-endoscopy (75.5%) as well as continuing education of at least 1/year (100.0%), a research background (80.0%) and teaching activities (78.7%), and formal interdisciplinary collaborations (e.g., tumor board: 93.0%). Academic vs. non-academic affiliation, career position, years of neurosurgical experience, country of practice, and primary clinical interest had a minor influence on the respondents' opinions.

Discussion and conclusion

Opinions among neurosurgeons regarding the characteristics and features of expertise in neuro-oncology vary surprisingly little. Large majorities favoring certain thresholds and qualitative criteria suggest a consensus definition might be possible.

The Development And Applications Of Augmented And Virtual Reality Technology In Spine Surgery Training: A Systematic Review

BACKGROUND

The COVID-19 pandemic has accelerated the growing global interest in the role of augmented and virtual reality in surgical training. While this technology grows at a rapid rate, its efficacy remains unclear. To that end, we offer a systematic review of the literature summarizing the role of virtual and augmented reality on spine surgery training.

METHODS

A systematic review of the literature was conducted on May 13th, 2022. PubMed, Web of Science, Medline, and Embase were reviewed for relevant studies. Studies from both orthopedic and neurosurgical spine programs were considered. There were no restrictions placed on the type of study, virtual/augmented reality modality, nor type of procedure. Qualitative data analysis was performed, and all studies were assigned a Medical Education Research Study Quality Instrument (MERSQI) score.

RESULTS

The initial review identified 6752 studies, of which 16 were deemed relevant and included in the final review, examining a total of nine unique augmented/virtual reality systems. These studies had a moderate methodological quality with a MERSQI score of 12.1 + 1.8; most studies were conducted at single-center institutions, and unclear response rates. Statistical pooling of the data was limited by the heterogeneity of the study designs.

CONCLUSION

This review examined the applications of augmented and virtual reality systems for training residents in various spine procedures. As this technology continues to advance, higher-quality, multi-center, and long-term studies are required to further the adaptation of VR/AR technologies in spine surgery training programs.

Effect of Feedback Modality on Simulated Surgical Skills Learning Using Automated Educational Systems- A Four-Arm Randomized Control Trial

OBJECTIVE

To explore optimal feedback methodologies to enhance trainee skill acquisition in simulated surgical bimanual skills learning during brain tumor resections.

HYPOTHESES

(1) Providing feedback results in better learning outcomes in teaching surgical technical skill when compared to practice alone with no tailored performance feedback. (2) Providing more visual and visuospatial feedback results in better learning outcomes when compared to providing numerical feedback.

DESIGN

A prospective 4-parallel-arm randomized controlled trial.

SETTING

Neurosurgical Simulation and Artificial Intelligence Learning Centre, McGill University, Canada.

PARTICIPANTS

Medical students (n = 120) from 4 Quebec medical schools.

RESULTS

Participants completed a virtually simulated tumor resection task 5 times while receiving 1 of 4 feedback based on their group allocation: (1) practice-alone without feedback, (2) numerical feedback, (3) visual feedback, and (4) visuospatial feedback. Outcome measures were participants' scores on 14-performance metrics and the number of expert benchmarks achieved during each task. There were no significant differences in the first task which determined baseline performance. A statistically significant interaction between feedback allocation and task repetition was found on the number of benchmarks achieved, F (10.558, 408.257)=3.220, p < 0.001. Participants in all feedback groups significantly improved their performance compared to baseline. The visual feedback group achieved significantly higher number of benchmarks than the practice-alone group by the third repetition of the task, p = 0.005, 95%CI [0.42 3.25]. Visual feedback and visuospatial feedback improved performance significantly by the second repetition of the task, p = 0.016, 95%CI [0.19 2.71] and p = 0.003, 95%CI [0.4 2.57], respectively.

CONCLUSION

Simulations with autonomous visual computer assistance may be effective pedagogical tools in teaching bimanual operative skills via visual and visuospatial feedback information delivery.

Surgical Artificial Intelligence in Urology: Educational Applications

Surgical education has seen immense change recently. Increased demand for iterative evaluation of trainees from medical school to independent practice has led to the generation of an overwhelming amount of data related to an individual's competency. Artificial intelligence has been proposed as a solution to automate and standardize the ability of stakeholders to assess the technical and nontechnical abilities of a surgical trainee. In both the simulation and clinical environments, evidence supports the use of machine learning algorithms to both evaluate trainee skill and provide real-time and automated feedback, enabling a shortened learning curve for many key procedural skills and ensuring patient safety.

Continuous Instrument Tracking in a Cerebral Corticectomy Ex Vivo Calf Brain Simulation Model: Face and Content Validation

BACKGROUND AND OBJECTIVES

Subpial corticectomy involving complete lesion resection while preserving pial membranes and avoiding injury to adjacent normal tissues is an essential bimanual task necessary for neurosurgical trainees to master. We sought to develop an ex vivo calf brain corticectomy simulation model with continuous assessment of surgical instrument movement during the simulation. A case series study of skilled participants was performed to assess face and content validity to gain insights into the utility of this training platform, along with determining if skilled and less skilled participants had statistical differences in validity assessment.

METHODS

An ex vivo calf brain simulation model was developed in which trainees performed a subpial corticectomy of three defined areas. A case series study assessed face and content validity of the model using 7-point Likert scale questionnaires.

RESULTS

Twelve skilled and 11 less skilled participants were included in this investigation. Overall median scores of 6.0 (range 4.0-6.0) for face validity and 6.0 (range 3.5-7.0) for content validity were determined on the 7-point Likert scale, with no statistical differences between skilled and less skilled groups identified.

CONCLUSION

A novel ex vivo calf brain simulator was developed to replicate the subpial resection procedure and demonstrated face and content validity.

Validity evidence supporting clinical skills assessment by artificial intelligence compared with trained clinician raters

BACKGROUND

Artificial intelligence (AI) is becoming increasingly used in medical education, but our understanding of the validity of AI-based assessments (AIBA) as compared with traditional clinical expert-based assessments (EBA) is limited. In this study, the authors aimed to compare and contrast the validity evidence for the assessment of a complex clinical skill based on scores generated from an AI and trained clinical experts, respectively.

METHODS

The study was conducted between September 2020 to October 2022. The authors used Kane's validity framework to prioritise and organise their evidence according to the four inferences: scoring, generalisation, extrapolation and implications. The context of the study was chorionic villus sampling performed within the simulated setting. AIBA and EBA were used to evaluate performances of experts, intermediates and novice based on video recordings. The clinical experts used a scoring instrument developed in a previous international consensus study. The AI used convolutional neural networks for capturing features on video recordings, motion tracking and eye movements to arrive at a final composite score.

RESULTS

A total of 45 individuals participated in the study (22 novices, 12 intermediates and 11 experts). The authors demonstrated validity evidence for scoring, generalisation, extrapolation and implications for both EBA and AIBA. The plausibility of assumptions related to scoring, evidence of reproducibility and relation to different training levels was examined. Issues relating to construct underrepresentation, lack of explainability, and threats to robustness were identified as potential weak links in the AIBA validity argument compared with the EBA validity argument.

CONCLUSION

There were weak links in the use of AIBA compared with EBA, mainly in their representation of the underlying construct but also regarding their explainability and ability to transfer to other datasets. However, combining AI and clinical expert-based assessments may offer complementary benefits, which is a promising subject for future research.

Neurosurgical Management of Cerebrospinal Tumors in the Era of Artificial Intelligence : A Scoping Review

Central nervous system tumors are identified as tumors of the brain and spinal cord. The associated morbidity and mortality of cerebrospinal tumors are disproportionately high compared to other malignancies. While minimally invasive techniques have initiated a revolution in neurosurgery, artificial intelligence (AI) is expediting it. Our study aims to analyze AI's role in the neurosurgical management of cerebrospinal tumors. We conducted a scoping review using the Arksey and O'Malley framework. Upon screening, data extraction and analysis were focused on exploring all potential implications of AI, classification of these implications in the management of cerebrospinal tumors. AI has enhanced the precision of diagnosis of these tumors, enables surgeons to excise the tumor margins completely, thereby reducing the risk of recurrence, and helps to make a more accurate prediction of the patient's prognosis than the conventional methods. AI also offers real-time training to neurosurgeons using virtual and 3D simulation, thereby increasing their confidence and skills during procedures. In addition, robotics is integrated into neurosurgery and identified to increase patient outcomes by making surgery less invasive. AI, including machine learning, is rigorously considered for its applications in the neurosurgical management of cerebrospinal tumors. This field requires further research focused on areas clinically essential in improving the outcome that is also economically feasible for clinical use. The authors suggest that data analysts and neurosurgeons collaborate to explore the full potential of AI.

Current clinical applications of artificial intelligence in shoulder surgery: what the busy shoulder surgeon needs to know and what's coming next

Background

Artificial intelligence (AI) is a continuously expanding field with the potential to transform a variety of industries-including health care-by providing automation, efficiency, precision, accuracy, and decision-making support for simple and complex tasks. Basic knowledge of the key features as well as limitations of AI is paramount to understand current developments in this field and to successfully apply them to shoulder surgery. The purpose of the present review is to provide an overview of AI within orthopedics and shoulder surgery exploring current and forthcoming AI applications.

Methods

PubMed and Scopus databases were searched to provide a narrative review of the most relevant literature on AI applications in shoulder surgery.

Results

Despite the enormous clinical and research potential of AI, orthopedic surgery has been a relatively late adopter of AI technologies. Image evaluation, surgical planning, aiding decision-making, and facilitating patient evaluations over time are some of the current areas of development with enormous opportunities to improve surgical practice, research, and education. Furthermore, the advancement of AI-driven strategies has the potential to create a more efficient medical system that may reduce the overall cost of delivering and implementing quality health care for patients with shoulder pathology.

Conclusion

AI is an expanding field with the potential for broad clinical and research applications in orthopedic surgery. Many challenges still need to be addressed to fully leverage the potential of AI to clinical practice and research such as privacy issues, data ownership, and external validation of the proposed models.

Weight-adjusted caffeine and β-blocker use in novice versus senior retina surgeons: a self-controlled study of simulated performance

BACKGROUND/OBJECTIVES

Tremor and expertise are potentially influenced variables in vitreoretinal surgery. We investigated whether surgeon experience impacts the association of microsurgical performance with caffeine and β-blockers weight-adjusted intake.

SUBJECTS/METHODS

Novice and senior surgeons (<2 and >10 practice years, respectively) were recruited in this self-controlled, cross-sectional study. A simulator's task sequence was repeated over 2 days, 30 min after the following exposures: day 1, placebo, 2.5 mg/kg caffeine, 5.0 mg/kg caffeine, and 0.6 mg/kg propranolol; and day 2, placebo, 0.2 mg/kg propranolol, 0.6 mg/kg propranolol, and 5.0 mg/kg caffeine. Outcomes were total score (0-700, worst-best), simulation time (minutes), intraocular trajectory (centimeters), and tremor-specific score (0-100, worst-best).

RESULTS

We recruited 15 novices (9 men [60%], 1.33 ± 0.49 practice years) and 11 seniors (8 men [72.7%], 16.00 ± 4.24 practice years). Novices performed worse after 2.5 mg/kg caffeine and improved following 0.2 mg/kg propranolol in total score (557 vs. 617, p = 0.009), trajectory (229.86 vs. 208.07, p = 0.048), time (14.9 vs. 12.7, p = 0.048), and tremor-score (55 vs. 75, p = 0.009). Surgical performance improved with propranolol post-caffeine but remained worse than 0.2 mg/kg propranolol in total score (570 vs. 617, p = 0.014), trajectory (226.59 vs. 208.07, p = 0.033), and tremor-score (50 vs. 75, p = 0.029). Seniors' tremor-score was lower after 2.5 mg/kg caffeine compared to 0.2 mg/kg propranolol (8 vs. 37, p = 0.015). Tremor-score following propranolol post-caffeine remained inferior to 0.6 mg/kg propranolol alone (17 vs. 38, p = 0.012).

CONCLUSION

While caffeine and propranolol were associated with performance changes among novices, only tremor was affected in seniors, without dexterity changes. The pharmacologic exposure impact on surgical dexterity seems to be offset by increased experience.

Performance in a Simulated Virtual Reality Anterior Cervical Discectomy and Fusion Task: Disc Residual, Rate of Removal, and Efficiency Analyses

BACKGROUND AND OBJECTIVES

Anterior cervical discectomy and fusion (ACDF) is among the most common spine procedures. The Sim-Ortho virtual reality simulator platform contains a validated ACDF simulated task for performance assessment. This study aims to develop a methodology to extract three-dimensional data and reconstruct and quantitate specific simulated disc tissues to generate novel metrics to analyze performance metrics of skilled and less skilled participants.

METHODS

We used open-source platforms to develop a methodology to extract three-dimensional information from ACDF simulation data. Metrics generated included, efficiency index, disc volumes removed from defined regions, and rate of tissue removal from superficial, central, and deep disc regions. A pilot study was performed to assess the utility of this methodology to assess expertise during the ACDF simulated procedure.

RESULTS

The system outlined, extracts data allowing the development of a methodology which accurately reconstructs and quantitates 3-dimensional disc volumes. In the pilot study, data sets from 27 participants, divided into postresident, resident, and medical student groups, allowed assessment of multiple novel metrics, including efficiency index (surgical time spent in actively removing disc), where the postresident group spent 61.8% of their time compared with 53% and 30.2% for the resident and medical student groups, respectively ( P = .01). During the annulotomy component, the postresident group removed 47.4% more disc than the resident groups and 102% more than the medical student groups ( P = .03).

CONCLUSION

The methodology developed in this study generates novel surgical procedural metrics from 3-dimensional data generated by virtual reality simulators and can be used to assess surgical performance.

AI in Surgical Curriculum Design and Unintended Outcomes for Technical Competencies in Simulation Training

Importance

To better elucidate the role of artificial intelligence (AI) in surgical skills training requires investigations in the potential existence of a hidden curriculum.

Objective

To assess the pedagogical value of AI-selected technical competencies and their extended effects in surgical simulation training.

Design, Setting, and Participants

This cohort study was a follow-up of a randomized clinical trial conducted at the Neurosurgical Simulation and Artificial Intelligence Learning Centre at the Montreal Neurological Institute, McGill University, Montreal, Canada. Surgical performance metrics of medical students exposed to an AI-enhanced training curriculum were compared with a control group of participants who received no feedback and with expert benchmarks. Cross-sectional data were collected from January to April 2021 from medical students and from March 2015 to May 2016 from experts. This follow-up secondary analysis was conducted from June to September 2022. Participants included medical students (undergraduate year 0-2) in the intervention cohorts and neurosurgeons to establish expertise benchmarks.

Exposure

Performance assessment and personalized feedback by an intelligent tutor on 4 AI-selected learning objectives during simulation training.

Main Outcomes and Measures

Outcomes of interest were unintended performance outcomes, measured by significant within-participant difference from baseline in 270 performance metrics in the intervention cohort that was not observed in the control cohort.

Results

A total of 46 medical students (median [range] age, 22 [18-27] years; 27 [59%] women) and 14 surgeons (median [range] age, 45 [35-59] years; 14 [100%] men) were included in this study, and no participant was lost to follow-up. Feedback on 4 AI-selected technical competencies was associated with additional performance change in 32 metrics over the entire procedure and 20 metrics during tumor removal that was not observed in the control group. Participants exposed to the AI-enhanced curriculum demonstrated significant improvement in safety metrics, such as reducing the rate of healthy tissue removal (mean difference, -7.05 × 10-5 [95% CI, -1.09 × 10-4 to -3.14 × 10-5] mm3 per 20 ms; P < .001) and maintaining a focused bimanual control of the operative field (mean difference in maximum instrument divergence, -4.99 [95% CI, -8.48 to -1.49] mm, P = .006) compared with the control group. However, negative unintended effects were also observed. These included a significantly lower velocity and acceleration in the dominant hand (velocity: mean difference, -0.13 [95% CI, -0.17 to -0.09] mm per 20 ms; P < .001; acceleration: mean difference, -2.25 × 10-2 [95% CI, -3.20 × 10-2 to -1.31 × 10-2] mm per 20 ms2; P < .001) and a significant reduction in the rate of tumor removal (mean difference, -4.85 × 10-5 [95% CI, -7.22 × 10-5 to -2.48 × 10-5] mm3 per 20 ms; P < .001) compared with control. These unintended outcomes diverged students' movement and efficiency performance metrics away from the expertise benchmarks.

Conclusions and Relevance

In this cohort study of medical students, an AI-enhanced curriculum for bimanual surgical skills resulted in unintended changes that improved performance in safety but negatively affected some efficiency metrics. Incorporating AI in course design requires ongoing assessment to maintain transparency and foster evidence-based learning objectives.

The Application of Adaptive Minimum Match k-Nearest Neighbors to Identify At-Risk Students in Health Professions Education

INTRODUCTION

When learners fail to reach milestones, educators often wonder if any warning signs could have allowed them to intervene sooner. Machine learning can predict which students are at risk for failing a high-stakes certification examination. If predictions can be made well before the examination, educators can meaningfully intervene before students take the examination to reduce their chances of failing.

METHODS

The authors used already-collected, first-year student assessment data from 5 cohorts in a single Master of Physician Assistant Studies program to implement an "adaptive minimum match" version of the k-nearest neighbors algorithm using changing numbers of neighbors to predict each student's future examination scores on the Physician Assistant National Certifying Exam (PANCE). Validation occurred in 2 ways by using leave-one-out cross-validation (LOOCV) and by evaluating predictions in a new cohort.

RESULTS

"Adaptive minimum match" version of the k-nearest neighbors algorithm achieved an accuracy of 93% in LOOCV. "Adaptive minimum match" version of the k-nearest neighbors algorithm generates a predicted PANCE score for each student one year before they take the examination. Students are classified into extra support, optional extra support, or no extra support categories. Then, one year remains to provide appropriate support to each category of student.

DISCUSSION

Predictive analytics can identify at-risk students who might need additional support or remediation before high-stakes certification examinations. Educators can use the included methods and code to generate predicted test outcomes for students. The authors recommend that educators use predictive modeling responsibly and transparently, as one of many tools used to support students. More research is needed to test alternative machine learning methods across a variety of educational programs.

Investigating and acquiring motor expertise using virtual reality

After just months of simulated training, on January 19, 2019 a 23-year-old E-sports pro-gamer, Enzo Bonito, took to the racetrack and beat Lucas di Grassi, a Formula E and ex-Formula 1 driver with decades of real-world racing experience. This event raised the possibility that practicing in virtual reality can be surprisingly effective for acquiring motor expertise in real-world tasks. Here, we evaluate the potential of virtual reality to serve as a space for training to expert levels in highly complex real-world tasks in time windows much shorter than those required in the real world and at much lower financial cost without the hazards of the real world. We also discuss how VR can also serve as an experimental platform for exploring the science of expertise more generally.

Neurosurgical skills analysis by machine learning models: systematic review

Machine learning (ML) models are being actively used in modern medicine, including neurosurgery. This study aimed to summarize the current applications of ML in the analysis and assessment of neurosurgical skills. We conducted this systematic review in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. We searched the PubMed and Google Scholar databases for eligible studies published until November 15, 2022, and used the Medical Education Research Study Quality Instrument (MERSQI) to assess the quality of the included articles. Of the 261 studies identified, we included 17 in the final analysis. Studies were most commonly related to oncological, spinal, and vascular neurosurgery using microsurgical and endoscopic techniques. Machine learning-evaluated tasks included subpial brain tumor resection, anterior cervical discectomy and fusion, hemostasis of the lacerated internal carotid artery, brain vessel dissection and suturing, glove microsuturing, lumbar hemilaminectomy, and bone drilling. The data sources included files extracted from VR simulators and microscopic and endoscopic videos. The ML application was aimed at classifying participants into several expertise levels, analysis of differences between experts and novices, surgical instrument recognition, division of operation into phases, and prediction of blood loss. In two articles, ML models were compared with those of human experts. The machines outperformed humans in all tasks. The most popular algorithms used to classify surgeons by skill level were the support vector machine and k-nearest neighbors, and their accuracy exceeded 90%. The "you only look once" detector and RetinaNet usually solved the problem of detecting surgical instruments - their accuracy was approximately 70%. The experts differed by more confident contact with tissues, higher bimanuality, smaller distance between the instrument tips, and relaxed and focused state of the mind. The average MERSQI score was 13.9 (from 18). There is growing interest in the use of ML in neurosurgical training. Most studies have focused on the evaluation of microsurgical skills in oncological neurosurgery and on the use of virtual simulators; however, other subspecialties, skills, and simulators are being investigated. Machine learning models effectively solve different neurosurgical tasks related to skill classification, object detection, and outcome prediction. Properly trained ML models outperform human efficacy. Further research on ML application in neurosurgery is needed.

Anatomic Depth Estimation and 3-Dimensional Reconstruction of Microsurgical Anatomy Using Monoscopic High-Definition Photogrammetry and Machine Learning

BACKGROUND

Immersive anatomic environments offer an alternative when anatomic laboratory access is limited, but current three-dimensional (3D) renderings are not able to simulate the anatomic detail and surgical perspectives needed for microsurgical education.

OBJECTIVE

To perform a proof-of-concept study of a novel photogrammetry 3D reconstruction technique, converting high-definition (monoscopic) microsurgical images into a navigable, interactive, immersive anatomy simulation.

METHODS

Images were acquired from cadaveric dissections and from an open-access comprehensive online microsurgical anatomic image database. A pretrained neural network capable of depth estimation from a single image was used to create depth maps (pixelated images containing distance information that could be used for spatial reprojection and 3D rendering). Virtual reality (VR) experience was assessed using a VR headset, and augmented reality was assessed using a quick response code-based application and a tablet camera.

RESULTS

Significant correlation was found between processed image depth estimations and neuronavigation-defined coordinates at different levels of magnification. Immersive anatomic models were created from dissection images captured in the authors' laboratory and from images retrieved from the Rhoton Collection. Interactive visualization and magnification allowed multiple perspectives for an enhanced experience in VR. The quick response code offered a convenient method for importing anatomic models into the real world for rehearsal and for comparing other anatomic preparations side by side.

CONCLUSION

This proof-of-concept study validated the use of machine learning to render 3D reconstructions from 2-dimensional microsurgical images through depth estimation. This spatial information can be used to develop convenient, realistic, and immersive anatomy image models.

Artificial Intelligence Technologies in the Microsurgical Operating Room (Review)

Surgery performed by a novice neurosurgeon under constant supervision of a senior surgeon with the experience of thousands of operations, able to handle any intraoperative complications and predict them in advance, and never getting tired, is currently an elusive dream, but can become a reality with the development of artificial intelligence methods. This paper has presented a review of the literature on the use of artificial intelligence technologies in the microsurgical operating room. Searching for sources was carried out in the PubMed text database of medical and biological publications. The key words used were "surgical procedures", "dexterity", "microsurgery" AND "artificial intelligence" OR "machine learning" OR "neural networks". Articles in English and Russian were considered with no limitation to publication date. The main directions of research on the use of artificial intelligence technologies in the microsurgical operating room have been highlighted. Despite the fact that in recent years machine learning has been increasingly introduced into the medical field, a small number of studies related to the problem of interest have been published, and their results have not proved to be of practical use yet. However, the social significance of this direction is an important argument for its development.

Virtual neurosurgery anatomy laboratory: A collaborative and remote education experience in the metaverse

Background:

Advances in computer sciences, including novel 3-dimensional rendering techniques, have enabled the creation of cloud-based virtual reality (VR) interfaces, making real-time peer-to-peer interaction possible even from remote locations. This study addresses the potential use of this technology for microsurgery anatomy education.

Methods:

Digital specimens were created using multiple photogrammetry techniques and imported into a virtual simulated neuroanatomy dissection laboratory. A VR educational program using a multiuser virtual anatomy laboratory experience was developed. Internal validation was performed by five multinational neurosurgery visiting scholars testing and assessing the digital VR models. For external validation, 20 neurosurgery residents tested and assessed the same models and virtual space.

Results:

Each participant responded to 14 statements assessing the virtual models, categorized under realism (n = 3), usefulness (n = 2), practicality (n = 3), enjoyment (n = 3), and recommendation (n = 3). Most responses expressed agreement or strong agreement with the assessment statements (internal validation, 94% [66/70] total responses; external validation, 91.4% [256/280] total responses). Notably, most participants strongly agreed that this system should be part of neurosurgery residency training and that virtual cadaver courses through this platform could be effective for education.

Conclusion:

Cloud-based VR interfaces are a novel resource for neurosurgery education. Interactive and remote collaboration between instructors and trainees is possible in virtual environments using volumetric models created with photogrammetry. We believe that this technology could be part of a hybrid anatomy curriculum for neurosurgery education. More studies are needed to assess the educational value of this type of innovative educational resource.

Real-time simulation of viscoelastic tissue behavior with physics-guided deep learning

Finite element methods (FEM) are popular approaches for simulation of soft tissues with elastic or viscoelastic behavior. However, their usage in real-time applications, such as in virtual reality surgical training, is limited by computational cost. In this application scenario, which typically involves transportable simulators, the computing hardware severely constrains the size or the level of details of the simulated scene. To address this limitation, data-driven approaches have been suggested to simulate mechanical deformations by learning the mapping rules from FEM generated datasets. Prior data-driven approaches have ignored the physical laws of the underlying engineering problem and have consequently been restricted to simulation cases of simple hyperelastic materials where the temporal variations were effectively ignored. However, most surgical training scenarios require more complex hyperelastic models to deal with the viscoelastic properties of tissues. This type of material exhibits both viscous and elastic behaviors when subjected to external force, requiring the implementation of time-dependant state variables. Herein, we propose a deep learning method for predicting displacement fields of soft tissues with viscoelastic properties. The main contribution of this work is the use of a physics-guided loss function for the optimization of the deep learning model parameters. The proposed deep learning model is based on convolutional (CNN) and recurrent layers (LSTM) to predict spatiotemporal variations. It is augmented with a mass conservation law in the lost function to prevent the generation of physically inconsistent results. The deep learning model is trained on a set of FEM datasets that are generated from a commercially available state-of-the-art numerical neurosurgery simulator. The use of the physics-guided loss function in a deep learning model has led to a better generalization in the prediction of deformations in unseen simulation cases. Moreover, the proposed method achieves a better accuracy over the conventional CNN models, where improvements were observed in unseen tissue from 8% to 30% depending on the magnitude of external forces. It is hoped that the present investigation will help in filling the gap in applying deep learning in virtual reality simulators, hence improving their computational performance (compared to FEM simulations) and ultimately their usefulness.

Artificial Intelligence in Neurosurgery: A Bibliometric Analysis

BACKGROUND

Artificial intelligence (AI) has the potential to augment clinicians' diagnostic and decision-making capabilities. It is well suited to identify patterns and correlations within data sets and may be applied to identify elements of importance in complex and data-laden areas such as patient selection, diagnostics, treatment, and outcome prediction. The development of modern neurosurgery has been dependent on major technological advances. In line with this, a growing interest is seen in the use of AI to assist in neurosurgical research and enhance neurosurgical practices.

METHODS

A bibliometric analysis of the 50 most-cited articles alluding to the use of AI in neurosurgery, from inception until July of 2022, was undertaken using the Web of Science database. Statistical analyses were performed on R.

RESULTS

The citation count ranged from 29 to 159 (mean: 51.9, standard deviation: 24.8), and the top-cited article was a 2018 systematic review published in World Neurosurgery. Most articles were published after 2015 (85%). The United States was the largest contributing country on the list with 22 articles. Four first and last authors, each, had 2 or more publications. Female first and last authorship was attributed to 18% and 0% of the articles, respectively.

CONCLUSIONS

This review highlights the most-impactful articles pertaining to AI in the field of neurosurgery. Although female authors were significantly underrepresented on the list, their work was at least as impactful as their male peers. Finally, the striking dominance of articles originating from the developed world raises concerns as to the future of AI in attending to the global health crisis.

Artificial Intelligence in Surgical Learning

(1) Background: Artificial Intelligence (AI) is transforming healthcare on all levels. While AI shows immense potential, the clinical implementation is lagging. We present a concise review of AI in surgical learning; (2) Methods: A non-systematic review of AI in surgical learning of the literature in English is provided; (3) Results: AI shows utility for all components of surgical competence within surgical learning. AI presents with great potential within robotic surgery specifically (4) Conclusions: Technology will evolve in ways currently unimaginable, presenting us with novel applications of AI and derivatives thereof. Surgeons must be open to new modes of learning to be able to implement all evidence-based applications of AI in the future. Systematic analyses of AI in surgical learning are needed.

Deconstructing forearm casting task by videos with step-by-step simulation teaching improved performance of medical students: is making working student's memory work better similar to a process of artificial intelligence or just an improvement of the prefrontal cortex homunculus?

PURPOSE

To compare two teaching methods of a forearm cast in medical students through simulation, the traditional method (Trad) based on a continuous demonstration of the procedure and the task deconstruction method (Decon) with the procedure fragmenting into its constituent parts using videos.

METHODS

During simulation training of the below elbow casting technique, 64 medical students were randomized in two groups. Trad group demonstrated the entire procedure without pausing. Decon group received step-wise teaching with educational videos emphasizing key components of the procedure. Direct and video evaluations were performed immediately after training (day 0) and at six months. Performance in casting was assessed using a 25-item checklist, a seven item global rating scale (GRS Performance), and a one item GRS (GRS Final Product).

RESULTS

Fifty-two students (Trad n = 24; Decon n = 28) underwent both day zero and six month assessments. At day zero, the Decon group showed higher performance via video evaluation for OSATS (p = 0.035); GRS performance (p < 0.001); GRS final product (p < 0.001), and for GRS performance (p < 0.001) and GRS final product (p = 0.011) via direct evaluation. After six months, performance was decreased in both groups with ultimately no difference in performance between groups via both direct and video evaluation. Having done a rotation in orthopaedic surgery was the only independent factor associated to higher performance.

CONCLUSIONS

The modified video-based version simulation led to a higher performance than the traditional method immediately after the course and could be the preferred method for teaching complex skills.

Modeling of Learning Processes Using Continuous-Time Markov Chain for Virtual-Reality-Based Surgical Training in Laparoscopic Surgery

Recent usage of Virtual Reality (VR) technology in surgical training has emerged because of its cost-effectiveness, time savings, and cognition-based feedback generation. However, the quantitative evaluation of its effectiveness in training is still not studied thoroughly. This paper demonstrates the effectiveness of a VR-based surgical training simulator in laparoscopic surgery and investigates how stochastic modeling represented as Continuous-time Markov-chain (CTMC) can be used to explicit the training status of the surgeon. By comparing the training in real environments and in VR-based training simulators, the authors also explore the validity of the VR simulator in laparoscopic surgery. The study further aids in establishing learning models of surgeons, supporting continuous evaluation of training processes for the derivation of real-time feedback by CTMC-based modeling.

Utilizing artificial intelligence and electroencephalography to assess expertise on a simulated neurosurgical task

Virtual reality surgical simulators have facilitated surgical education by providing a safe training environment. Electroencephalography (EEG) has been employed to assess neuroelectric activity during surgical performance. Machine learning (ML) has been applied to analyze EEG data split into frequency bands. Although EEG is widely used in fields requiring expert performance, it has yet been used to classify surgical expertise. Thus, the goals of this study were to (a) develop an ML model to accurately differentiate skilled and less-skilled performance using EEG data recorded during a simulated surgery, (b) explore the relative importance of each EEG bandwidth to expertise, and (c) analyze differences in EEG band powers between skilled and less-skilled individuals. We hypothesized that EEG recordings during a virtual reality surgery task would accurately predict the expertise level of the participant. Twenty-one participants performed three simulated brain tumor resection procedures on the NeuroVR™ platform (CAE Healthcare, Montreal, Canada) while EEG data was recorded. Participants were divided into 2 groups. The skilled group was composed of five neurosurgeons and five senior neurosurgical residents (PGY4-6), and the less-skilled group was composed of six junior residents (PGY1-3) and five medical students. A total of 13 metrics from EEG frequency bands and ratios (e.g., alpha, theta/beta ratio) were generated. Seven ML model types were trained using EEG activity to differentiate between skilled and less-skilled groups. The artificial neural network achieved the highest testing accuracy of 100% (AUROC = 1.0). Model interpretation via Shapley analysis identified low alpha (8-10 Hz) as the most important metric for classifying expertise. Skilled surgeons displayed higher (p = 0.044) low-alpha than the less-skilled group. Furthermore, skilled surgeons displayed significantly lower TBR (p = 0.048) and significantly higher beta (13-30 Hz, p = 0.049), beta 1 (15-18 Hz, p = 0.014), and beta 2 (19-22 Hz, p = 0.015), thus establishing these metrics as important markers of expertise. ACGME CORE COMPETENCIES: Practice-Based Learning and Improvement.

Advancing Surgical Education: The Use of Artificial Intelligence in Surgical Training

The technology of artificial intelligence (AI) has made significant in-roads into the field of medicine over the last decade. With surgery being a discipline where repetition is the key to mastery, the scope of AI presents enormous potential for resident education through the analysis of technique and delivery of structured feedback for performance improvement. In an era marred by a raging pandemic that has decreased exposure and opportunity, AI offers an attractive solution towards improving operating room efficiency, safe patient care in the hands of supervised residents and can ultimately culminate in reduced health care costs. Through this article, we elucidate the current adoption of the artificial intelligence technology and its prospects for advancing surgical education.

The role of artificial intelligence in surgical simulation

Artificial Intelligence (AI) plays an integral role in enhancing the quality of surgical simulation, which is increasingly becoming a popular tool for enriching the training experience of a surgeon. This spans the spectrum from facilitating preoperative planning, to intraoperative visualisation and guidance, ultimately with the aim of improving patient safety. Although arguably still in its early stages of widespread clinical application, AI technology enables personal evaluation and provides personalised feedback in surgical training simulations. Several forms of surgical visualisation technologies currently in use for anatomical education and presurgical assessment rely on different AI algorithms. However, while it is promising to see clinical examples and technological reports attesting to the efficacy of AI-supported surgical simulators, barriers to wide-spread commercialisation of such devices and software remain complex and multifactorial. High implementation and production costs, scarcity of reports evidencing the superiority of such technology, and intrinsic technological limitations remain at the forefront. As AI technology is key to driving the future of surgical simulation, this paper will review the literature delineating its current state, challenges, and prospects. In addition, a consolidated list of FDA/CE approved AI-powered medical devices for surgical simulation is presented, in order to shed light on the existing gap between academic achievements and the universal commercialisation of AI-enabled simulators. We call for further clinical assessment of AI-supported surgical simulators to support novel regulatory body approved devices and usher surgery into a new era of surgical education.

Video Analysis of Otologic Instrument Movement During Resident Mastoidectomies

OBJECTIVE

To measure surgical instrument movement during resident mastoidectomies and identify metrics that correlate with experience.

STUDY DESIGN

Retrospective case series.

SETTING

Tertiary care center.

SUBJECTS

Ten postgraduate year (PGY) 2, 6 PGY3, 7 PGY4, and 19 PGY5 recordings of mastoidectomy performed by otolaryngology residents.

INTERVENTIONS

One-minute intraoperative recordings of mastoidectomies performed during cochlear implantation were collected. Drill and suction-irrigator motion were analyzed with sports motion tracking software.

MAIN OUTCOME MEASURES

Mean instrument speed, angle, and angular velocity were calculated. Mann-Whitney U tests compared mean instrument metrics between PGY levels. Change in drill speed for seven residents between their PGY2 to PGY5 years was individually analyzed.

RESULTS

Mean drill speed was significantly greater for PGY5 residents compared with PGY2s (2.9 versus 1.8 cm/s, p = 0.001). Compared with PGY2 residents, suction speed was greater as a PGY5 (1.2 versus 0.9 cm/s; p = 0.201) and significantly greater as a PGY4 (1.5 versus 0.9 cm/s, p = 0.039). Of the seven residents individually analyzed, group mean drill speed increased by 0.4 cm/s, yearly.

CONCLUSIONS

Drill and suction-irrigator movement during the second minute of drilling of a cortical mastoidectomy seems to increase with resident level. Objective video analysis is a potential adjunct for differentiating novices from more experienced surgeons and monitoring surgical skills progress.

Assessment of learning curves on a simulated neurosurgical task using metrics selected by artificial intelligence

OBJECTIVE

Understanding the variation of learning curves of experts and trainees for a given surgical procedure is important in implementing formative learning paradigms to accelerate mastery. The study objectives were to use artificial intelligence (AI)-derived metrics to determine the learning curves of participants in 4 groups with different expertise levels who performed a series of identical virtual reality (VR) subpial resection tasks and to identify learning curve differences among the 4 groups.

METHODS

A total of 50 individuals participated, 14 neurosurgeons, 4 neurosurgical fellows and 10 senior residents (seniors), 10 junior residents (juniors), and 12 medical students. All participants performed 5 repetitions of a subpial tumor resection on the NeuroVR (CAE Healthcare) platform, and 6 a priori-derived metrics selected using the K-nearest neighbors machine learning algorithm were used to assess participant learning curves. Group learning curves were plotted over the 5 trials for each metric. A mixed, repeated-measures ANOVA was performed between the first and fifth trial. For significant interactions (p < 0.05), post hoc Tukey's HSD analysis was conducted to determine the location of the significance.

RESULTS

Overall, 5 of the 6 metrics assessed had a significant interaction (p < 0.05). The 4 groups, neurosurgeons, seniors, juniors, and medical students, showed an improvement between the first and fifth trial on at least one of the 6 metrics evaluated.

CONCLUSIONS

Learning curves generated using AI-derived metrics provided novel insights into technical skill acquisition, based on expertise level, during repeated VR-simulated subpial tumor resections, which will allow educators to develop more focused formative educational paradigms for neurosurgical trainees.

Medical Data Classification Assisted by Machine Learning Strategy

With the development of science and technology, data plays an increasingly important role in our daily life. Therefore, much attention has been paid to the field of data mining. Data classification is the premise of data mining, and how well the data is classified directly affects the performance of subsequent models. In particular, in the medical field, data classification can help accurately determine the location of patients' lesions and reduce the workload of doctors in the treatment process. However, medical data has the characteristics of high noise, strong correlation, and high data dimension, which brings great challenges to the traditional classification model. Therefore, it is very important to design an advanced model to improve the effect of medical data classification. In this context, this paper first introduces the structure and characteristics of the convolutional neural network (CNN) model and then demonstrates its unique advantages in medical data processing, especially in data classification. Secondly, we design a new kind of medical data classification model based on the CNN model. Finally, the simulation results show that the proposed method achieves higher classification accuracy with faster model convergence speed and the lower training error when compared with conventional machine leaning methods, which has demonstrated the effectiveness of the new method in respect to medical data classification.

Extended Reality in Neurosurgical Education: A Systematic Review

Surgical simulation practices have witnessed a rapid expansion as an invaluable approach to resident training in recent years. One emerging way of implementing simulation is the adoption of extended reality (XR) technologies, which enable trainees to hone their skills by allowing interaction with virtual 3D objects placed in either real-world imagery or virtual environments. The goal of the present systematic review is to survey and broach the topic of XR in neurosurgery, with a focus on education. Five databases were investigated, leading to the inclusion of 31 studies after a thorough reviewing process. Focusing on user performance (UP) and user experience (UX), the body of evidence provided by these 31 studies showed that this technology has, in fact, the potential of enhancing neurosurgical education through the use of a wide array of both objective and subjective metrics. Recent research on the topic has so far produced solid results, particularly showing improvements in young residents, compared to other groups and over time. In conclusion, this review not only aids to a better understanding of the use of XR in neurosurgical education, but also highlights the areas where further research is entailed while also providing valuable insight into future applications.

Artificial Neural Network Approach to Competency-Based Training Using a Virtual Reality Neurosurgical Simulation

BACKGROUND

The methodology of assessment and training of surgical skills is evolving to deal with the emergence of competency-based training. Artificial neural networks (ANNs), a branch of artificial intelligence, can use newly generated metrics not only for assessment performance but also to quantitate individual metric importance and provide new insights into surgical expertise.

OBJECTIVE

To outline the educational utility of using an ANN in the assessment and quantitation of surgical expertise. A virtual reality vertebral osteophyte removal during a simulated surgical spine procedure is used as a model to outline this methodology.

METHODS

Twenty-one participants performed a simulated anterior cervical diskectomy and fusion on the Sim-Ortho virtual reality simulator. Participants were divided into 3 groups, including 9 postresidents, 5 senior residents, and 7 junior residents. Data were retrieved from the osteophyte removal component of the scenario, which involved using a simulated burr. The data were manipulated to initially generate 83 performance metrics spanning 3 categories (safety, efficiency, and motion) of which only the most relevant metrics were used to train and test the ANN.

RESULTS

The ANN model was trained on 6 safety metrics to a testing accuracy of 83.3%. The contributions of these performance metrics to expertise were revealed through connection weight products and outlined 2 identifiable learning patterns of technical skills.

CONCLUSION

This study outlines the potential utility of ANNs which allows a deeper understanding of the composites of surgical expertise and may contribute to the paradigm shift toward competency-based surgical training.

Nondominant Hand Skills Spatial and Psychomotor Analysis During a Complex Virtual Reality Neurosurgical Task-A Case Series Study

BACKGROUND

Virtual reality surgical simulators provide detailed psychomotor performance data, allowing qualitative and quantitative assessment of hand function. The nondominant hand plays an essential role in neurosurgery in exposing the operative area, assisting the dominant hand to optimize task execution, and hemostasis. Outlining expert-level nondominant hand skills may be critical to understand surgical expertise and aid learner training.

OBJECTIVE

To (1) provide validity for the simulated bimanual subpial tumor resection task and (2) to use this simulation in qualitative and quantitative evaluation of nondominant hand skills for bipolar forceps utilization.

METHODS

In this case series study, 45 right-handed participants performed a simulated subpial tumor resection using simulated bipolar forceps in the nondominant hand for assisting the surgery and hemostasis. A 10-item questionnaire was used to assess task validity. The nondominant hand skills across 4 expertise levels (neurosurgeons, senior trainees, junior trainees, and medical students) were analyzed by 2 visual models and performance metrics.

RESULTS

Neurosurgeon median (range) overall satisfaction with the simulated scenario was 4.0/5.0 (2.0-5.0). The visual models demonstrated a decrease in high force application areas on pial surface with increased expertise level. Bipolar-pia mater interactions were more focused around the tumoral region for neurosurgeons and senior trainees. These groups spent more time using the bipolar while interacting with pia. All groups spent significantly higher time in the left upper pial quadrant than other quadrants.

CONCLUSION

This work introduces new approaches for the evaluation of nondominant hand skills which may help surgical trainees by providing both qualitative and quantitative feedback.

Continuous monitoring of surgical bimanual expertise using deep neural networks in virtual reality simulation

In procedural-based medicine, the technical ability can be a critical determinant of patient outcomes. Psychomotor performance occurs in real-time, hence a continuous assessment is necessary to provide action-oriented feedback and error avoidance guidance. We outline a deep learning application, the Intelligent Continuous Expertise Monitoring System (ICEMS), to assess surgical bimanual performance at 0.2-s intervals. A long-short term memory network was built using neurosurgeon and student performance in 156 virtually simulated tumor resection tasks. Algorithm predictive ability was tested separately on 144 procedures by scoring the performance of neurosurgical trainees who are at different training stages. The ICEMS successfully differentiated between neurosurgeons, senior trainees, junior trainees, and students. Trainee average performance score correlated with the year of training in neurosurgery. Furthermore, coaching and risk assessment for critical metrics were demonstrated. This work presents a comprehensive technical skill monitoring system with predictive validation throughout surgical residency training, with the ability to detect errors.

Design Synthesis of a Robotic Uniaxial Torque Device for Orthopedic Haptic Simulation

Robotic devices are commonly used in surgical simulators to provide tactile, or haptic, feedback. They can provide customized feedback that can be rapidly modified with minimal hardware changes in comparison to non-robotic systems. This work describes the design, development, and evaluation of one such tool: a novel uniaxial torque haptic device for a surgical training simulator. The objective of the work was to design a single connection haptic device that could augment an existing six degree of freedom haptic device to mimic a Concorde Clear vacuum curette. Design and evaluations focused on the tool's ability to deliver adequate torque, imitate a surgical tool, and be integrated into the haptic device. Twenty-nine surgeons tested the tool in the simulator and evaluated it via a questionnaire. The device was found to deliver the 800 N·mm of torque necessary to mimic an orthopedic procedure. Surgeons found it accurately imitated surgical tool physical appearance and maneuverability, scoring them 3.9±1.0 and 3.3±1.2, respectively, on a 1-5 Likert scale. By virtue of the functionality necessary for testing and evaluation, the device could be connected to the haptic device for mechanical and electrical engagement. This device is a step forward in the field of augmentable haptic devices for surgical simulation. By changing the number of robotically-controlled degrees of freedom of a haptic device, existing devices can be tuned to meet the demands of a particular simulator, which has the potential to improve surgeon training standards.

An AI-Assisted and Self-Powered Smart Robotic Gripper Based on Eco-EGaIn Nanocomposite for Pick-and-Place Operation

High compliance and muscle-alike soft robotic grippers have shown promising performance in addressing the challenges in traditional rigid grippers. Nevertheless, a lack of control feedback (gasping speed and contact force) in a grasping operation can result in undetectable slipping and false positioning. In this study, a pneumatically driven and self-powered soft robotic gripper that can recognize the grabbed object is reported. We integrated pressure (P-TENG) and bend (B-TENG) triboelectric sensors into a soft robotic gripper to transduce the features of gripped objects in a pick-and-place operation. Both the P-TENG and B-TENG sensors are fabricated using a porous structure made of soft Ecoflex and Euthethic Gallium-Indium nanocomposite (Eco-EGaIn). The output voltage of this porous setup has been improved by 63%, as compared to the non-porous structure. The developed soft gripper successfully recognizes three different objects, cylinder, cuboid, and pyramid prism, with a good accuracy of 91.67% and has shown its potential to be beneficial in the assembly lines, sorting, VR/AR application, and education training.

Machine learning for technical skill assessment in surgery: a systematic review

Accurate and objective performance assessment is essential for both trainees and certified surgeons. However, existing methods can be time consuming, labor intensive, and subject to bias. Machine learning (ML) has the potential to provide rapid, automated, and reproducible feedback without the need for expert reviewers. We aimed to systematically review the literature and determine the ML techniques used for technical surgical skill assessment and identify challenges and barriers in the field. A systematic literature search, in accordance with the PRISMA statement, was performed to identify studies detailing the use of ML for technical skill assessment in surgery. Of the 1896 studies that were retrieved, 66 studies were included. The most common ML methods used were Hidden Markov Models (HMM, 14/66), Support Vector Machines (SVM, 17/66), and Artificial Neural Networks (ANN, 17/66). 40/66 studies used kinematic data, 19/66 used video or image data, and 7/66 used both. Studies assessed the performance of benchtop tasks (48/66), simulator tasks (10/66), and real-life surgery (8/66). Accuracy rates of over 80% were achieved, although tasks and participants varied between studies. Barriers to progress in the field included a focus on basic tasks, lack of standardization between studies, and lack of datasets. ML has the potential to produce accurate and objective surgical skill assessment through the use of methods including HMM, SVM, and ANN. Future ML-based assessment tools should move beyond the assessment of basic tasks and towards real-life surgery and provide interpretable feedback with clinical value for the surgeon.

PROSPERO: CRD42020226071

Artificial Intelligence and Surgical Education: A Systematic Scoping Review of Interventions

OBJECTIVE

To synthesize peer-reviewed evidence related to the use of artificial intelligence (AI) in surgical education DESIGN: We conducted and reported a scoping review according to the standards outlined in the Preferred Reporting Items for Systematic Reviews and Meta-Analysis with extension for Scoping Reviews guideline and the fourth edition of the Joanna Briggs Institute Reviewer's Manual. We systematically searched eight interdisciplinary databases including MEDLINE-Ovid, ERIC, EMBASE, CINAHL, Web of Science: Core Collection, Compendex, Scopus, and IEEE Xplore. Databases were searched from inception until the date of search on April 13, 2021.

SETTING/PARTICIPANTS

We only examined original, peer-reviewed interventional studies that self-described as AI interventions, focused on medical education, and were relevant to surgical trainees (defined as medical or dental students, postgraduate residents, or surgical fellows) within the title and abstract (see Table 2). Animal, cadaveric, and in vivo studies were not eligible for inclusion.

RESULTS

After systematically searching eight databases and 4255 citations, our scoping review identified 49 studies relevant to artificial intelligence in surgical education. We found diverse interventions related to the evaluation of surgical competency, personalization of surgical education, and improvement of surgical education materials across surgical specialties. Many studies used existing surgical education materials, such as the Objective Structured Assessment of Technical Skills framework or the JHU-ISI Gesture and Skill Assessment Working Set database. Though most studies did not provide outcomes related to the implementation in medical schools (such as cost-effective analyses or trainee feedback), there are numerous promising interventions. In particular, many studies noted high accuracy in the objective characterization of surgical skill sets. These interventions could be further used to identify at-risk surgical trainees or evaluate teaching methods.

CONCLUSIONS

There are promising applications for AI in surgical education, particularly for the assessment of surgical competencies, though further evidence is needed regarding implementation and applicability.

Virtual Reality in the Neurosciences: Current Practice and Future Directions

Virtual reality has made numerous advancements in recent years and is used with increasing frequency for education, diversion, and distraction. Beginning several years ago as a device that produced an image with only a few pixels, virtual reality is now able to generate detailed, three-dimensional, and interactive images. Furthermore, these images can be used to provide quantitative data when acting as a simulator or a rehabilitation device. In this article, we aim to draw attention to these areas, as well as highlight the current settings in which virtual reality (VR) is being actively studied and implemented within the field of neurosurgery and the neurosciences. Additionally, we discuss the current limitations of the applications of virtual reality within various settings. This article includes areas in which virtual reality has been used in applications both inside and outside of the operating room, such as pain control, patient education and counseling, and rehabilitation. Virtual reality's utility in neurosurgery and the neurosciences is widely growing, and its use is quickly becoming an integral part of patient care, surgical training, operative planning, navigation, and rehabilitation.

Effect of Artificial Intelligence Tutoring vs Expert Instruction on Learning Simulated Surgical Skills Among Medical Students: A Randomized Clinical Trial

IMPORTANCE

To better understand the emerging role of artificial intelligence (AI) in surgical training, efficacy of AI tutoring systems, such as the Virtual Operative Assistant (VOA), must be tested and compared with conventional approaches.

OBJECTIVE

To determine how VOA and remote expert instruction compare in learners' skill acquisition, affective, and cognitive outcomes during surgical simulation training.

DESIGN, SETTING, AND PARTICIPANTS

This instructor-blinded randomized clinical trial included medical students (undergraduate years 0-2) from 4 institutions in Canada during a single simulation training at McGill Neurosurgical Simulation and Artificial Intelligence Learning Centre, Montreal, Canada. Cross-sectional data were collected from January to April 2021. Analysis was conducted based on intention-to-treat. Data were analyzed from April to June 2021.

INTERVENTIONS

The interventions included 5 feedback sessions, 5 minutes each, during a single 75-minute training, including 5 practice sessions followed by 1 realistic virtual reality brain tumor resection. The 3 intervention arms included 2 treatment groups, AI audiovisual metric-based feedback (VOA group) and synchronous verbal scripted debriefing and instruction from a remote expert (instructor group), and a control group that received no feedback.

MAIN OUTCOMES AND MEASURES

The coprimary outcomes were change in procedural performance, quantified as Expertise Score by a validated assessment algorithm (Intelligent Continuous Expertise Monitoring System [ICEMS]; range, -1.00 to 1.00) for each practice resection, and learning and retention, measured from performance in realistic resections by ICEMS and blinded Objective Structured Assessment of Technical Skills (OSATS; range 1-7). Secondary outcomes included strength of emotions before, during, and after the intervention and cognitive load after intervention, measured in self-reports.

RESULTS

A total of 70 medical students (41 [59%] women and 29 [41%] men; mean [SD] age, 21.8 [2.3] years) from 4 institutions were randomized, including 23 students in the VOA group, 24 students in the instructor group, and 23 students in the control group. All participants were included in the final analysis. ICEMS assessed 350 practice resections, and ICEMS and OSATS evaluated 70 realistic resections. VOA significantly improved practice Expertise Scores by 0.66 (95% CI, 0.55 to 0.77) points compared with the instructor group and by 0.65 (95% CI, 0.54 to 0.77) points compared with the control group (P < .001). Realistic Expertise Scores were significantly higher for the VOA group compared with instructor (mean difference, 0.53 [95% CI, 0.40 to 0.67] points; P < .001) and control (mean difference. 0.49 [95% CI, 0.34 to 0.61] points; P < .001) groups. Mean global OSATS ratings were not statistically significant among the VOA (4.63 [95% CI, 4.06 to 5.20] points), instructor (4.40 [95% CI, 3.88-4.91] points), and control (3.86 [95% CI, 3.44 to 4.27] points) groups. However, on the OSATS subscores, VOA significantly enhanced the mean OSATS overall subscore compared with the control group (mean difference, 1.04 [95% CI, 0.13 to 1.96] points; P = .02), whereas expert instruction significantly improved OSATS subscores for instrument handling vs control (mean difference, 1.18 [95% CI, 0.22 to 2.14]; P = .01). No significant differences in cognitive load, positive activating, and negative emotions were found.

CONCLUSIONS AND RELEVANCE

In this randomized clinical trial, VOA feedback demonstrated superior performance outcome and skill transfer, with equivalent OSATS ratings and cognitive and emotional responses compared with remote expert instruction, indicating advantages for its use in simulation training.

TRIAL REGISTRATION

ClinicalTrials.gov Identifier: NCT04700384.

Exploring the roles of artificial intelligence in surgical education: A scoping review

BACKGROUND

Technology-enhanced teaching and learning, including Artificial Intelligence (AI) applications, has started to evolve in surgical education. Hence, the purpose of this scoping review is to explore the current and future roles of AI in surgical education.

METHODS

Nine bibliographic databases were searched from January 2010 to January 2021. Full-text articles were included if they focused on AI in surgical education.

RESULTS

Out of 14,008 unique sources of evidence, 93 were included. Out of 93, 84 were conducted in the simulation setting, and 89 targeted technical skills. Fifty-six studies focused on skills assessment/classification, and 36 used multiple AI techniques. Also, increasing sample size, having balanced data, and using AI to provide feedback were major future directions mentioned by authors.

CONCLUSIONS

AI can help optimize the education of trainees and our results can help educators and researchers identify areas that need further investigation.

Artificial Neural Networks Predict 30-Day Mortality After Hip Fracture: Insights From Machine Learning

OBJECTIVES

Accurately stratifying patients in the preoperative period according to mortality risk informs treatment considerations and guides adjustments to bundled reimbursements. We developed and compared three machine learning models to determine which best predicts 30-day mortality after hip fracture.

METHODS

The 2016 to 2017 National Surgical Quality Improvement Program for hip fracture (AO/OTA 31-A-B-C) procedure-targeted data were analyzed. Three models-artificial neural network, naive Bayes, and logistic regression-were trained and tested using independent variables selected via backward variable selection. The data were split into 80% training and 20% test sets. Predictive accuracy between models was evaluated using area under the curve receiver operating characteristics. Odds ratios were determined using multivariate logistic regression with P < 0.05 for significance.

RESULTS

The study cohort included 19,835 patients (69.3% women). The 30-day mortality rate was 5.3%. In total, 47 independent patient variables were identified to train the testing models. Area under the curve receiver operating characteristics for 30-day mortality was highest for artificial neural network (0.92), followed by the logistic regression (0.87) and naive Bayes models (0.83).

DISCUSSION

Machine learning is an emerging approach to develop accurate risk calculators that account for the weighted interactions between variables. In this study, we developed and tested a neural network model that was highly accurate for predicting 30-day mortality after hip fracture. This was superior to the naive Bayes and logistic regression models. The role of machine learning models to predict orthopaedic outcomes merits further development and prospective validation but shows strong promise for positively impacting patient care.

Machine learning in gastrointestinal surgery

Machine learning (ML) is a collection of algorithms allowing computers to learn directly from data without predetermined equations. It is used widely to analyze "big data". In gastrointestinal surgery, surgeons deal with various data such as clinical parameters, surgical videos, and pathological images, to stratify surgical risk, perform safe surgery and predict patient prognosis. In the current "big data" era, the accelerating accumulation of a large amount of data drives studies using ML algorithms. Three subfields of ML are supervised learning, unsupervised learning, and reinforcement learning. In this review, we summarize applications of ML to surgical practice in the preoperative, intraoperative, and postoperative phases of care. Prediction and stratification using ML is promising; however, the current overarching concern is the availability of ML models. Information systems that can manage "big data" and integrate ML models into electronic health records are essential to incorporate ML into daily practice. ML is fundamental technology to meaningfully process data that exceeds the capacity of the human mind to comprehend. The accelerating accumulation of a large amount of data is changing the nature of surgical practice fundamentally. Artificial intelligence (AI), represented by ML, is being incorporated into daily surgical practice.

Utilizing a multilayer perceptron artificial neural network to assess a virtual reality surgical procedure

BACKGROUND

Virtual reality surgical simulators are a safe and efficient technology for the assessment and training of surgical skills. Simulators allow trainees to improve specific surgical techniques in risk-free environments. Recently, machine learning has been coupled to simulators to classify performance. However, most studies fail to extract meaningful observations behind the classifications and the impact of specific surgical metrics on the performance. One benefit from integrating machine learning algorithms, such as Artificial Neural Networks, to simulators is the ability to extract novel insights into the composites of the surgical performance that differentiate levels of expertise.

OBJECTIVE

This study aims to demonstrate the benefits of artificial neural network algorithms in assessing and analyzing virtual surgical performances. This study applies the algorithm on a virtual reality simulated annulus incision task during an anterior cervical discectomy and fusion scenario.

DESIGN

An artificial neural network algorithm was developed and integrated. Participants performed the simulated surgical procedure on the Sim-Ortho simulator. Data extracted from the annulus incision task were extracted to generate 157 surgical performance metrics that spanned three categories (motion, safety, and efficiency).

SETTING

Musculoskeletal Biomechanics Research Lab; Neurosurgical Simulation and Artificial Intelligence Learning Center, McGill University, Montreal, Canada.

PARTICIPANTS

Twenty-three participants were recruited and divided into 3 groups: 11 post-residents, 5 senior and 7 junior residents.

RESULTS

An artificial neural network model was trained on nine selected surgical metrics, spanning all three categories and achieved 80% testing accuracy.

CONCLUSIONS

This study outlines the benefits of integrating artificial neural networks to virtual reality surgical simulators in understanding composites of expertise performance.

Best Practices Using Ex Vivo Animal Brain Models in Neurosurgical Education to Assess Surgical Expertise

BACKGROUND

Ex vivo animal brain simulation models are being increasingly used for neurosurgical training because these models can replicate human brain conditions. The goal of the present report is to provide the neurosurgical community interested in using ex vivo animal brain simulation models with guidelines for comprehensively and rigorously conducting, documenting, and assessing this type of research.

METHODS

In consultation with an interdisciplinary group of physicians and researchers involved in ex vivo models and a review of the literature on the best practices guidelines for simulation research, we developed the "ex vivo brain model to assess surgical expertise" (EVBMASE) checklist. The EVBMASE checklist provides a comprehensive quantitative framework for analyzing and reporting studies involving these models. We applied The EVBMASE checklist to the studies reported of ex vivo animal brain models to document how current ex vivo brain simulation models are used to train surgical expertise.

RESULTS

The EVBMASE checklist includes defined subsections and a total score of 20, which can help investigators improve studies and provide readers with techniques to better assess the quality and any deficiencies of the research. We classified 18 published ex vivo brain models into modified (group 1) and nonmodified (group 2) models. The mean total EVBMASE score was 11 (55%) for group 1 and 4.8 (24.2%) for group 2, a statistically significant difference (P = 0.006) mainly attributed to differences in the simulation study design section (P = 0.003).

CONCLUSIONS

The present findings should help contribute to more rigorous application, documentation, and assessment of ex vivo brain simulation research.

Artificial Intelligence Surgery: How Do We Get to Autonomous Actions in Surgery?

Most surgeons are skeptical as to the feasibility of autonomous actions in surgery. Interestingly, many examples of autonomous actions already exist and have been around for years. Since the beginning of this millennium, the field of artificial intelligence (AI) has grown exponentially with the development of machine learning (ML), deep learning (DL), computer vision (CV) and natural language processing (NLP). All of these facets of AI will be fundamental to the development of more autonomous actions in surgery, unfortunately, only a limited number of surgeons have or seek expertise in this rapidly evolving field. As opposed to AI in medicine, AI surgery (AIS) involves autonomous movements. Fortuitously, as the field of robotics in surgery has improved, more surgeons are becoming interested in technology and the potential of autonomous actions in procedures such as interventional radiology, endoscopy and surgery. The lack of haptics, or the sensation of touch, has hindered the wider adoption of robotics by many surgeons; however, now that the true potential of robotics can be comprehended, the embracing of AI by the surgical community is more important than ever before. Although current complete surgical systems are mainly only examples of tele-manipulation, for surgeons to get to more autonomously functioning robots, haptics is perhaps not the most important aspect. If the goal is for robots to ultimately become more and more independent, perhaps research should not focus on the concept of haptics as it is perceived by humans, and the focus should be on haptics as it is perceived by robots/computers. This article will discuss aspects of ML, DL, CV and NLP as they pertain to the modern practice of surgery, with a focus on current AI issues and advances that will enable us to get to more autonomous actions in surgery. Ultimately, there may be a paradigm shift that needs to occur in the surgical community as more surgeons with expertise in AI may be needed to fully unlock the potential of AIS in a safe, efficacious and timely manner.

A systematic review of virtual reality for the assessment of technical skills in neurosurgery

OBJECTIVE

Virtual reality (VR) and augmented reality (AR) systems are increasingly available to neurosurgeons. These systems may provide opportunities for technical rehearsal and assessments of surgeon performance. The assessment of neurosurgeon skill in VR and AR environments and the validity of VR and AR feedback has not been systematically reviewed.

METHODS

A systematic review following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines was conducted through MEDLINE and PubMed. Studies published in English between January 1990 and February 2021 describing the use of VR or AR to quantify surgical technical performance of neurosurgeons without the use of human raters were included. The types and categories of automated performance metrics (APMs) from each of these studies were recorded.

RESULTS

Thirty-three VR studies were included in the review; no AR studies met inclusion criteria. VR APMs were categorized as either distance to target, force, kinematics, time, blood loss, or volume of resection. Distance and time were the most well-studied APM domains, although all domains were effective at differentiating surgeon experience levels. Distance was successfully used to track improvements with practice. Examining volume of resection demonstrated that attending surgeons removed less simulated tumor but preserved more normal tissue than trainees. More recently, APMs have been used in machine learning algorithms to predict level of training with a high degree of accuracy. Key limitations to enhanced-reality systems include limited AR usage for automated surgical assessment and lack of external and longitudinal validation of VR systems.

CONCLUSIONS

VR has been used to assess surgeon performance across a wide spectrum of domains. The VR environment can be used to quantify surgeon performance, assess surgeon proficiency, and track training progression. AR systems have not yet been used to provide metrics for surgeon performance assessment despite potential for intraoperative integration. VR-based APMs may be especially useful for metrics that are difficult to assess intraoperatively, including blood loss and extent of resection.