1
|
Yilmaz R, Fazlollahi AM, Winkler-Schwartz A, Wang A, Makhani HH, Alsayegh A, Bakhaidar M, Tran DH, Santaguida C, Del Maestro RF. Effect of Feedback Modality on Simulated Surgical Skills Learning Using Automated Educational Systems- A Four-Arm Randomized Control Trial. J Surg Educ 2024; 81:275-287. [PMID: 38160107 DOI: 10.1016/j.jsurg.2023.11.001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/03/2023] [Revised: 09/05/2023] [Accepted: 11/01/2023] [Indexed: 01/03/2024]
Abstract
OBJECTIVE To explore optimal feedback methodologies to enhance trainee skill acquisition in simulated surgical bimanual skills learning during brain tumor resections. HYPOTHESES (1) Providing feedback results in better learning outcomes in teaching surgical technical skill when compared to practice alone with no tailored performance feedback. (2) Providing more visual and visuospatial feedback results in better learning outcomes when compared to providing numerical feedback. DESIGN A prospective 4-parallel-arm randomized controlled trial. SETTING Neurosurgical Simulation and Artificial Intelligence Learning Centre, McGill University, Canada. PARTICIPANTS Medical students (n = 120) from 4 Quebec medical schools. RESULTS Participants completed a virtually simulated tumor resection task 5 times while receiving 1 of 4 feedback based on their group allocation: (1) practice-alone without feedback, (2) numerical feedback, (3) visual feedback, and (4) visuospatial feedback. Outcome measures were participants' scores on 14-performance metrics and the number of expert benchmarks achieved during each task. There were no significant differences in the first task which determined baseline performance. A statistically significant interaction between feedback allocation and task repetition was found on the number of benchmarks achieved, F (10.558, 408.257)=3.220, p < 0.001. Participants in all feedback groups significantly improved their performance compared to baseline. The visual feedback group achieved significantly higher number of benchmarks than the practice-alone group by the third repetition of the task, p = 0.005, 95%CI [0.42 3.25]. Visual feedback and visuospatial feedback improved performance significantly by the second repetition of the task, p = 0.016, 95%CI [0.19 2.71] and p = 0.003, 95%CI [0.4 2.57], respectively. CONCLUSION Simulations with autonomous visual computer assistance may be effective pedagogical tools in teaching bimanual operative skills via visual and visuospatial feedback information delivery.
Collapse
Affiliation(s)
- Recai Yilmaz
- Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology & Neurosurgery, Montreal Neurological Institute, McGill University, Montreal, Quebec, Canada; Faculty of Medicine and Health Sciences, McGill University, Montreal, Canada.
| | - Ali M Fazlollahi
- Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology & Neurosurgery, Montreal Neurological Institute, McGill University, Montreal, Quebec, Canada; Faculty of Medicine and Health Sciences, McGill University, Montreal, Canada
| | - Alexander Winkler-Schwartz
- Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology & Neurosurgery, Montreal Neurological Institute, McGill University, Montreal, Quebec, Canada; Department of Neurology and Neurosurgery, Montreal Neurological Institute and Hospital, McGill University, Montreal, Quebec, Canada
| | - Anna Wang
- Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology & Neurosurgery, Montreal Neurological Institute, McGill University, Montreal, Quebec, Canada
| | - Hafila Hassan Makhani
- Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology & Neurosurgery, Montreal Neurological Institute, McGill University, Montreal, Quebec, Canada
| | - Ahmad Alsayegh
- Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology & Neurosurgery, Montreal Neurological Institute, McGill University, Montreal, Quebec, Canada; Department of Neurology and Neurosurgery, Montreal Neurological Institute and Hospital, McGill University, Montreal, Quebec, Canada; Division of Neurosurgery, Department of Surgery, Faculty of Medicine, King Abdulaziz University, Jeddah, Saudi Arabia
| | - Mohamad Bakhaidar
- Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology & Neurosurgery, Montreal Neurological Institute, McGill University, Montreal, Quebec, Canada; Department of Neurology and Neurosurgery, Montreal Neurological Institute and Hospital, McGill University, Montreal, Quebec, Canada; Division of Neurosurgery, Department of Surgery, Faculty of Medicine, King Abdulaziz University, Jeddah, Saudi Arabia
| | - Dan Huy Tran
- Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology & Neurosurgery, Montreal Neurological Institute, McGill University, Montreal, Quebec, Canada
| | - Carlo Santaguida
- Department of Neurology and Neurosurgery, Montreal Neurological Institute and Hospital, McGill University, Montreal, Quebec, Canada
| | - Rolando F Del Maestro
- Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology & Neurosurgery, Montreal Neurological Institute, McGill University, Montreal, Quebec, Canada; Faculty of Medicine and Health Sciences, McGill University, Montreal, Canada; Department of Neurology and Neurosurgery, Montreal Neurological Institute and Hospital, McGill University, Montreal, Quebec, Canada
| |
Collapse
|
2
|
Natheir S, Christie S, Yilmaz R, Winkler-Schwartz A, Bajunaid K, Sabbagh AJ, Werthner P, Fares J, Azarnoush H, Del Maestro R. Utilizing artificial intelligence and electroencephalography to assess expertise on a simulated neurosurgical task. Comput Biol Med 2023; 152:106286. [PMID: 36502696 DOI: 10.1016/j.compbiomed.2022.106286] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/10/2021] [Revised: 10/18/2022] [Accepted: 10/25/2022] [Indexed: 11/17/2022]
Abstract
Virtual reality surgical simulators have facilitated surgical education by providing a safe training environment. Electroencephalography (EEG) has been employed to assess neuroelectric activity during surgical performance. Machine learning (ML) has been applied to analyze EEG data split into frequency bands. Although EEG is widely used in fields requiring expert performance, it has yet been used to classify surgical expertise. Thus, the goals of this study were to (a) develop an ML model to accurately differentiate skilled and less-skilled performance using EEG data recorded during a simulated surgery, (b) explore the relative importance of each EEG bandwidth to expertise, and (c) analyze differences in EEG band powers between skilled and less-skilled individuals. We hypothesized that EEG recordings during a virtual reality surgery task would accurately predict the expertise level of the participant. Twenty-one participants performed three simulated brain tumor resection procedures on the NeuroVR™ platform (CAE Healthcare, Montreal, Canada) while EEG data was recorded. Participants were divided into 2 groups. The skilled group was composed of five neurosurgeons and five senior neurosurgical residents (PGY4-6), and the less-skilled group was composed of six junior residents (PGY1-3) and five medical students. A total of 13 metrics from EEG frequency bands and ratios (e.g., alpha, theta/beta ratio) were generated. Seven ML model types were trained using EEG activity to differentiate between skilled and less-skilled groups. The artificial neural network achieved the highest testing accuracy of 100% (AUROC = 1.0). Model interpretation via Shapley analysis identified low alpha (8-10 Hz) as the most important metric for classifying expertise. Skilled surgeons displayed higher (p = 0.044) low-alpha than the less-skilled group. Furthermore, skilled surgeons displayed significantly lower TBR (p = 0.048) and significantly higher beta (13-30 Hz, p = 0.049), beta 1 (15-18 Hz, p = 0.014), and beta 2 (19-22 Hz, p = 0.015), thus establishing these metrics as important markers of expertise. ACGME CORE COMPETENCIES: Practice-Based Learning and Improvement.
Collapse
Affiliation(s)
- Sharif Natheir
- Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology & Neurosurgery, Montreal Neurological Institute and Hospital, McGill University, Montreal, Quebec, Canada.
| | - Sommer Christie
- Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology & Neurosurgery, Montreal Neurological Institute and Hospital, McGill University, Montreal, Quebec, Canada
| | - Recai Yilmaz
- Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology & Neurosurgery, Montreal Neurological Institute and Hospital, McGill University, Montreal, Quebec, Canada
| | - Alexander Winkler-Schwartz
- Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology & Neurosurgery, Montreal Neurological Institute and Hospital, McGill University, Montreal, Quebec, Canada
| | - Khalid Bajunaid
- Department of Surgery, College of Medicine, University of Jeddah, Jeddah, Saudi Arabia
| | - Abdulrahman J Sabbagh
- Division of Neurosurgery, Department of Surgery, College of Medicine, King Abdulaziz University, Jeddah, Saudi Arabia; Clinical Skills and Simulation Center, King Abdulaziz University, Jeddah, Saudi Arabia
| | - Penny Werthner
- University of Calgary, Faculty of Kinesiology, Calgary, Alberta, Canada
| | - Jawad Fares
- Department of Neurological Surgery Feinberg School of Medicine, Northwestern University Chicago, Illinois, USA
| | - Hamed Azarnoush
- Department of Biomedical Engineering, Amirkabir University of Technology (Tehran Polytechnic), Tehran, Iran
| | - Rolando Del Maestro
- Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology & Neurosurgery, Montreal Neurological Institute and Hospital, McGill University, Montreal, Quebec, Canada
| |
Collapse
|
3
|
Ng PY, Bing EG, Cuevas A, Aggarwal A, Chi B, Sundar S, Mwanahamuntu M, Mutebi M, Sullivan R, Parham GP. Virtual reality and surgical oncology. Ecancermedicalscience 2023; 17:1525. [PMID: 37113716 PMCID: PMC10129400 DOI: 10.3332/ecancer.2023.1525] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/30/2022] [Indexed: 04/29/2023] Open
Abstract
More than 80% of people diagnosed with cancer will require surgery. However, less than 5% have access to safe, affordable and timely surgery in low- and middle-income countries (LMICs) settings mostly due to the lack of trained workforce. Since its creation, virtual reality (VR) has been heralded as a viable adjunct to surgical training, but its adoption in surgical oncology to date is poorly understood. We undertook a systematic review to determine the application of VR across different surgical specialties, modalities and cancer pathway globally between January 2011 and 2021. We reviewed their characteristics and respective methods of validation of 24 articles. The results revealed gaps in application and accessibility of VR with a proclivity for high-income countries and high-risk, complex oncological surgeries. There is a lack of standardisation of clinical evaluation of VR, both in terms of clinical trials and implementation science. While all VR illustrated face and content validity, only around two-third exhibited construct validity and predictive validity was lacking overall. In conclusion, the asynchrony between VR development and actual global cancer surgery demand means the technology is not effectively, efficiently and equitably utilised to realise its surgical capacity-building potential. Future research should prioritise cost-effective VR technologies with predictive validity for high demand, open cancer surgeries required in LMICs.
Collapse
Affiliation(s)
- Peng Yun Ng
- King’s College London, London WC2R 2LS, UK
- Guy’s and St Thomas’ Trust, London SE1 9R, UK
| | - Eric G Bing
- Institute for Leadership Impact, Southern Methodist University, Dallas, TX 75205, USA
| | - Anthony Cuevas
- Department of Teaching and Learning, Technology-Enhanced Immersive Learning Cluster, Annette Simmons School of Education and Human Development, Southern Methodist University, Dallas, TX 75205, USA
| | - Ajay Aggarwal
- King’s College London, London WC2R 2LS, UK
- Guy’s and St Thomas’ Trust, London SE1 9R, UK
- London School of Hygiene and Tropical Medicine, London WC1E 7HT, UK
| | - Benjamin Chi
- Icahn School of Medicine, New York, NY 10029-6574, USA
| | - Sudha Sundar
- Institute of Cancer and Genomic Sciences, University of Birmingham, Birmingham B152TT, UK
- Pan Birmingham Gynaecological Cancer Centre, City Hospital, Birmingham, B187QH, UK
| | | | - Miriam Mutebi
- Department of Surgery, Aga Khan University Hospital, Nairobi 30270-00100, Kenya
| | - Richard Sullivan
- Conflict & Health Research Group, King’s College London, London WC2R 2LS, UK
| | - Groesbeck P Parham
- Department of Surgery, Aga Khan University Hospital, Nairobi 30270-00100, Kenya
| |
Collapse
|
4
|
Iop A, El-Hajj VG, Gharios M, de Giorgio A, Monetti FM, Edström E, Elmi-Terander A, Romero M. Extended Reality in Neurosurgical Education: A Systematic Review. Sensors (Basel) 2022; 22:6067. [PMID: 36015828 PMCID: PMC9414210 DOI: 10.3390/s22166067] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 07/06/2022] [Revised: 08/06/2022] [Accepted: 08/12/2022] [Indexed: 06/15/2023]
Abstract
Surgical simulation practices have witnessed a rapid expansion as an invaluable approach to resident training in recent years. One emerging way of implementing simulation is the adoption of extended reality (XR) technologies, which enable trainees to hone their skills by allowing interaction with virtual 3D objects placed in either real-world imagery or virtual environments. The goal of the present systematic review is to survey and broach the topic of XR in neurosurgery, with a focus on education. Five databases were investigated, leading to the inclusion of 31 studies after a thorough reviewing process. Focusing on user performance (UP) and user experience (UX), the body of evidence provided by these 31 studies showed that this technology has, in fact, the potential of enhancing neurosurgical education through the use of a wide array of both objective and subjective metrics. Recent research on the topic has so far produced solid results, particularly showing improvements in young residents, compared to other groups and over time. In conclusion, this review not only aids to a better understanding of the use of XR in neurosurgical education, but also highlights the areas where further research is entailed while also providing valuable insight into future applications.
Collapse
Affiliation(s)
- Alessandro Iop
- Department of Neurosurgery, Karolinska University Hospital, 141 86 Stockholm, Sweden
- Department of Clinical Neuroscience, Karolinska Institutet, 171 77 Stockholm, Sweden
- KTH Royal Institute of Technology, 114 28 Stockholm, Sweden
| | - Victor Gabriel El-Hajj
- Department of Neurosurgery, Karolinska University Hospital, 141 86 Stockholm, Sweden
- Department of Clinical Neuroscience, Karolinska Institutet, 171 77 Stockholm, Sweden
| | - Maria Gharios
- Department of Neurosurgery, Karolinska University Hospital, 141 86 Stockholm, Sweden
- Department of Clinical Neuroscience, Karolinska Institutet, 171 77 Stockholm, Sweden
| | - Andrea de Giorgio
- SnT—Interdisciplinary Center for Security, Reliability and Trust, University of Luxembourg, 4365 Esch-sur-Alzette, Luxembourg
| | | | - Erik Edström
- Department of Neurosurgery, Karolinska University Hospital, 141 86 Stockholm, Sweden
- Department of Clinical Neuroscience, Karolinska Institutet, 171 77 Stockholm, Sweden
| | - Adrian Elmi-Terander
- Department of Neurosurgery, Karolinska University Hospital, 141 86 Stockholm, Sweden
- Department of Clinical Neuroscience, Karolinska Institutet, 171 77 Stockholm, Sweden
| | - Mario Romero
- KTH Royal Institute of Technology, 114 28 Stockholm, Sweden
| |
Collapse
|
5
|
Yilmaz R, Ledwos N, Sawaya R, Winkler-Schwartz A, Mirchi N, Bissonnette V, Fazlollahi AM, Bakhaidar M, Alsayegh A, Sabbagh AJ, Bajunaid K, Del Maestro R. Nondominant Hand Skills Spatial and Psychomotor Analysis During a Complex Virtual Reality Neurosurgical Task-A Case Series Study. Oper Neurosurg (Hagerstown) 2022; 23:22-30. [PMID: 35726926 DOI: 10.1227/ons.0000000000000232] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/21/2021] [Accepted: 02/09/2022] [Indexed: 11/19/2022] Open
Abstract
BACKGROUND Virtual reality surgical simulators provide detailed psychomotor performance data, allowing qualitative and quantitative assessment of hand function. The nondominant hand plays an essential role in neurosurgery in exposing the operative area, assisting the dominant hand to optimize task execution, and hemostasis. Outlining expert-level nondominant hand skills may be critical to understand surgical expertise and aid learner training. OBJECTIVE To (1) provide validity for the simulated bimanual subpial tumor resection task and (2) to use this simulation in qualitative and quantitative evaluation of nondominant hand skills for bipolar forceps utilization. METHODS In this case series study, 45 right-handed participants performed a simulated subpial tumor resection using simulated bipolar forceps in the nondominant hand for assisting the surgery and hemostasis. A 10-item questionnaire was used to assess task validity. The nondominant hand skills across 4 expertise levels (neurosurgeons, senior trainees, junior trainees, and medical students) were analyzed by 2 visual models and performance metrics. RESULTS Neurosurgeon median (range) overall satisfaction with the simulated scenario was 4.0/5.0 (2.0-5.0). The visual models demonstrated a decrease in high force application areas on pial surface with increased expertise level. Bipolar-pia mater interactions were more focused around the tumoral region for neurosurgeons and senior trainees. These groups spent more time using the bipolar while interacting with pia. All groups spent significantly higher time in the left upper pial quadrant than other quadrants. CONCLUSION This work introduces new approaches for the evaluation of nondominant hand skills which may help surgical trainees by providing both qualitative and quantitative feedback.
Collapse
Affiliation(s)
- Recai Yilmaz
- Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology and Neurosurgery, Montreal Neurological Institute, McGill University, Montreal, Quebec, Canada
| | - Nicole Ledwos
- Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology and Neurosurgery, Montreal Neurological Institute, McGill University, Montreal, Quebec, Canada
| | - Robin Sawaya
- Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology and Neurosurgery, Montreal Neurological Institute, McGill University, Montreal, Quebec, Canada
| | - Alexander Winkler-Schwartz
- Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology and Neurosurgery, Montreal Neurological Institute, McGill University, Montreal, Quebec, Canada
- Department of Neurology and Neurosurgery, Montreal Neurological Institute and Hospital, McGill University, Montreal, Quebec, Canada
| | - Nykan Mirchi
- Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology and Neurosurgery, Montreal Neurological Institute, McGill University, Montreal, Quebec, Canada
| | - Vincent Bissonnette
- Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology and Neurosurgery, Montreal Neurological Institute, McGill University, Montreal, Quebec, Canada
- Division of Orthopaedic Surgery, Montreal General Hospital, McGill University, Montreal, Quebec, Canada
| | - Ali M Fazlollahi
- Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology and Neurosurgery, Montreal Neurological Institute, McGill University, Montreal, Quebec, Canada
| | - Mohamad Bakhaidar
- Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology and Neurosurgery, Montreal Neurological Institute, McGill University, Montreal, Quebec, Canada
- Department of Neurology and Neurosurgery, Montreal Neurological Institute and Hospital, McGill University, Montreal, Quebec, Canada
- Division of Neurosurgery, Department of Surgery, Faculty of Medicine, King Abdulaziz University, Jeddah, Saudi Arabia
| | - Ahmad Alsayegh
- Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology and Neurosurgery, Montreal Neurological Institute, McGill University, Montreal, Quebec, Canada
- Division of Neurosurgery, Department of Surgery, Faculty of Medicine, King Abdulaziz University, Jeddah, Saudi Arabia
| | - Abdulrahman J Sabbagh
- Division of Neurosurgery, Department of Surgery, Faculty of Medicine, King Abdulaziz University, Jeddah, Saudi Arabia
- Clinical Skills and Simulation Center, King Abdulaziz University, Jeddah, Saudi Arabia
| | - Khalid Bajunaid
- Department of Surgery, Faculty of Medicine, University of Jeddah, Jeddah, Saudi Arabia
| | - Rolando Del Maestro
- Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology and Neurosurgery, Montreal Neurological Institute, McGill University, Montreal, Quebec, Canada
- Department of Neurology and Neurosurgery, Montreal Neurological Institute and Hospital, McGill University, Montreal, Quebec, Canada
| |
Collapse
|
6
|
Yilmaz R, Winkler-Schwartz A, Mirchi N, Reich A, Christie S, Tran DH, Ledwos N, Fazlollahi AM, Santaguida C, Sabbagh AJ, Bajunaid K, Del Maestro R. Continuous monitoring of surgical bimanual expertise using deep neural networks in virtual reality simulation. NPJ Digit Med 2022; 5:54. [PMID: 35473961 DOI: 10.1038/s41746-022-00596-8] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/23/2021] [Accepted: 03/29/2022] [Indexed: 11/22/2022] Open
Abstract
In procedural-based medicine, the technical ability can be a critical determinant of patient outcomes. Psychomotor performance occurs in real-time, hence a continuous assessment is necessary to provide action-oriented feedback and error avoidance guidance. We outline a deep learning application, the Intelligent Continuous Expertise Monitoring System (ICEMS), to assess surgical bimanual performance at 0.2-s intervals. A long-short term memory network was built using neurosurgeon and student performance in 156 virtually simulated tumor resection tasks. Algorithm predictive ability was tested separately on 144 procedures by scoring the performance of neurosurgical trainees who are at different training stages. The ICEMS successfully differentiated between neurosurgeons, senior trainees, junior trainees, and students. Trainee average performance score correlated with the year of training in neurosurgery. Furthermore, coaching and risk assessment for critical metrics were demonstrated. This work presents a comprehensive technical skill monitoring system with predictive validation throughout surgical residency training, with the ability to detect errors.
Collapse
|
7
|
Ledwos N, Mirchi N, Yilmaz R, Winkler-Schwartz A, Sawni A, Fazlollahi AM, Bissonnette V, Bajunaid K, Sabbagh AJ, Del Maestro RF. Assessment of learning curves on a simulated neurosurgical task using metrics selected by artificial intelligence. J Neurosurg 2022; 137:1-12. [PMID: 35120309 DOI: 10.3171/2021.12.jns211563] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/28/2021] [Accepted: 12/09/2021] [Indexed: 01/10/2023]
Abstract
OBJECTIVE Understanding the variation of learning curves of experts and trainees for a given surgical procedure is important in implementing formative learning paradigms to accelerate mastery. The study objectives were to use artificial intelligence (AI)-derived metrics to determine the learning curves of participants in 4 groups with different expertise levels who performed a series of identical virtual reality (VR) subpial resection tasks and to identify learning curve differences among the 4 groups. METHODS A total of 50 individuals participated, 14 neurosurgeons, 4 neurosurgical fellows and 10 senior residents (seniors), 10 junior residents (juniors), and 12 medical students. All participants performed 5 repetitions of a subpial tumor resection on the NeuroVR (CAE Healthcare) platform, and 6 a priori-derived metrics selected using the K-nearest neighbors machine learning algorithm were used to assess participant learning curves. Group learning curves were plotted over the 5 trials for each metric. A mixed, repeated-measures ANOVA was performed between the first and fifth trial. For significant interactions (p < 0.05), post hoc Tukey's HSD analysis was conducted to determine the location of the significance. RESULTS Overall, 5 of the 6 metrics assessed had a significant interaction (p < 0.05). The 4 groups, neurosurgeons, seniors, juniors, and medical students, showed an improvement between the first and fifth trial on at least one of the 6 metrics evaluated. CONCLUSIONS Learning curves generated using AI-derived metrics provided novel insights into technical skill acquisition, based on expertise level, during repeated VR-simulated subpial tumor resections, which will allow educators to develop more focused formative educational paradigms for neurosurgical trainees.
Collapse
Affiliation(s)
- Nicole Ledwos
- 1Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology & Neurosurgery, Montreal Neurological Institute and Hospital, McGill University
| | - Nykan Mirchi
- 1Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology & Neurosurgery, Montreal Neurological Institute and Hospital, McGill University
| | - Recai Yilmaz
- 1Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology & Neurosurgery, Montreal Neurological Institute and Hospital, McGill University
| | - Alexander Winkler-Schwartz
- 1Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology & Neurosurgery, Montreal Neurological Institute and Hospital, McGill University
- 3Department of Neurology and Neurosurgery, Montreal Neurological Institute and Hospital, McGill University, Montreal, Quebec, Canada
| | - Anika Sawni
- 1Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology & Neurosurgery, Montreal Neurological Institute and Hospital, McGill University
| | - Ali M Fazlollahi
- 1Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology & Neurosurgery, Montreal Neurological Institute and Hospital, McGill University
| | - Vincent Bissonnette
- 1Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology & Neurosurgery, Montreal Neurological Institute and Hospital, McGill University
- 2Division of Orthopaedic Surgery, Montreal General Hospital, McGill University
| | - Khalid Bajunaid
- 6Department of Surgery, College of Medicine, University of Jeddah, Jeddah, Saudi Arabia
| | - Abdulrahman J Sabbagh
- 4Division of Neurosurgery, Department of Surgery, College of Medicine, King Abdulaziz University
- 5Clinical Skills and Simulation Center, King Abdulaziz University; and
| | - Rolando F Del Maestro
- 1Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology & Neurosurgery, Montreal Neurological Institute and Hospital, McGill University
- 3Department of Neurology and Neurosurgery, Montreal Neurological Institute and Hospital, McGill University, Montreal, Quebec, Canada
| |
Collapse
|
8
|
Patel S, Alkadri S, Driscoll M. Development and Validation of a Mixed Reality Configuration of a Simulator for a Minimally Invasive Spine Surgery Using the Workspace of a Haptic Device and Simulator Users. Biomed Res Int 2021; 2021:2435126. [PMID: 35005014 PMCID: PMC8741356 DOI: 10.1155/2021/2435126] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 04/28/2021] [Accepted: 12/07/2021] [Indexed: 11/18/2022]
Abstract
Most surgical simulators leverage virtual or bench models to simulate reality. This study proposes and validates a method for workspace configuration of a surgical simulator which utilizes a haptic device for interaction with a virtual model and a bench model to provide additional tactile feedback based on planned surgical manoeuvers. Numerical analyses were completed to determine the workspace and position of a haptic device, relative to the bench model, used in the surgical simulator, and the determined configuration was validated using device limitations and user data from surgical and nonsurgical users. For the validation, surgeons performed an identical surgery on a cadaver prior to using the simulator, and their trajectories were then compared to the determined workspace for the haptic device. The configuration of the simulator was determined appropriate through workspace analysis and the collected user trajectories. Statistical analyses suggest differences in trajectories between the participating surgeons which were not affected by the imposed haptic workspace. This study, therefore, demonstrates a method to optimally position a haptic device with respect to a bench model while meeting the manoeuverability needs of a surgical procedure. The validation method identified workspace position and user trajectory towards ideal configuration of a mixed reality simulator.
Collapse
Affiliation(s)
- Sneha Patel
- Department of Mechanical Engineering, McGill University, MacDonald Engineering Building, 817 Rue Sherbrooke Ouest #270, Montréal, Québec, Canada H3A 0C3
| | - Sami Alkadri
- Department of Mechanical Engineering, McGill University, MacDonald Engineering Building, 817 Rue Sherbrooke Ouest #270, Montréal, Québec, Canada H3A 0C3
| | - Mark Driscoll
- Department of Mechanical Engineering, McGill University, MacDonald Engineering Building, 817 Rue Sherbrooke Ouest #270, Montréal, Québec, Canada H3A 0C3
| |
Collapse
|
9
|
Alkadri S, Ledwos N, Mirchi N, Reich A, Yilmaz R, Driscoll M, Del Maestro RF. Utilizing a multilayer perceptron artificial neural network to assess a virtual reality surgical procedure. Comput Biol Med 2021; 136:104770. [PMID: 34426170 DOI: 10.1016/j.compbiomed.2021.104770] [Citation(s) in RCA: 16] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/29/2021] [Revised: 08/12/2021] [Accepted: 08/13/2021] [Indexed: 11/18/2022]
Abstract
BACKGROUND Virtual reality surgical simulators are a safe and efficient technology for the assessment and training of surgical skills. Simulators allow trainees to improve specific surgical techniques in risk-free environments. Recently, machine learning has been coupled to simulators to classify performance. However, most studies fail to extract meaningful observations behind the classifications and the impact of specific surgical metrics on the performance. One benefit from integrating machine learning algorithms, such as Artificial Neural Networks, to simulators is the ability to extract novel insights into the composites of the surgical performance that differentiate levels of expertise. OBJECTIVE This study aims to demonstrate the benefits of artificial neural network algorithms in assessing and analyzing virtual surgical performances. This study applies the algorithm on a virtual reality simulated annulus incision task during an anterior cervical discectomy and fusion scenario. DESIGN An artificial neural network algorithm was developed and integrated. Participants performed the simulated surgical procedure on the Sim-Ortho simulator. Data extracted from the annulus incision task were extracted to generate 157 surgical performance metrics that spanned three categories (motion, safety, and efficiency). SETTING Musculoskeletal Biomechanics Research Lab; Neurosurgical Simulation and Artificial Intelligence Learning Center, McGill University, Montreal, Canada. PARTICIPANTS Twenty-three participants were recruited and divided into 3 groups: 11 post-residents, 5 senior and 7 junior residents. RESULTS An artificial neural network model was trained on nine selected surgical metrics, spanning all three categories and achieved 80% testing accuracy. CONCLUSIONS This study outlines the benefits of integrating artificial neural networks to virtual reality surgical simulators in understanding composites of expertise performance.
Collapse
Affiliation(s)
- Sami Alkadri
- Musculoskeletal Biomechanics Research Lab, Department of Mechanical Engineering, McGill University, Macdonald Engineering Building, 815 Sherbrooke St W, Montreal, H3A 2K7, QC, Canada
| | - Nicole Ledwos
- Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology & Neurosurgery, Montreal Neurological Institute, McGill University, 3801 University Street, Room E2.89, H3A 2B4, Montreal, Quebec, Canada
| | - Nykan Mirchi
- Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology & Neurosurgery, Montreal Neurological Institute, McGill University, 3801 University Street, Room E2.89, H3A 2B4, Montreal, Quebec, Canada
| | - Aiden Reich
- Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology & Neurosurgery, Montreal Neurological Institute, McGill University, 3801 University Street, Room E2.89, H3A 2B4, Montreal, Quebec, Canada
| | - Recai Yilmaz
- Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology & Neurosurgery, Montreal Neurological Institute, McGill University, 3801 University Street, Room E2.89, H3A 2B4, Montreal, Quebec, Canada
| | - Mark Driscoll
- Musculoskeletal Biomechanics Research Lab, Department of Mechanical Engineering, McGill University, Macdonald Engineering Building, 815 Sherbrooke St W, Montreal, H3A 2K7, QC, Canada.
| | - Rolando F Del Maestro
- Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology & Neurosurgery, Montreal Neurological Institute, McGill University, 3801 University Street, Room E2.89, H3A 2B4, Montreal, Quebec, Canada
| |
Collapse
|
10
|
Davids J, Manivannan S, Darzi A, Giannarou S, Ashrafian H, Marcus HJ. Simulation for skills training in neurosurgery: a systematic review, meta-analysis, and analysis of progressive scholarly acceptance. Neurosurg Rev 2021; 44:1853-1867. [PMID: 32944808 PMCID: PMC8338820 DOI: 10.1007/s10143-020-01378-0] [Citation(s) in RCA: 28] [Impact Index Per Article: 9.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/17/2020] [Revised: 07/17/2020] [Accepted: 08/21/2020] [Indexed: 02/07/2023]
Abstract
At a time of significant global unrest and uncertainty surrounding how the delivery of clinical training will unfold over the coming years, we offer a systematic review, meta-analysis, and bibliometric analysis of global studies showing the crucial role simulation will play in training. Our aim was to determine the types of simulators in use, their effectiveness in improving clinical skills, and whether we have reached a point of global acceptance. A PRISMA-guided global systematic review of the neurosurgical simulators available, a meta-analysis of their effectiveness, and an extended analysis of their progressive scholarly acceptance on studies meeting our inclusion criteria of simulation in neurosurgical education were performed. Improvement in procedural knowledge and technical skills was evaluated. Of the identified 7405 studies, 56 studies met the inclusion criteria, collectively reporting 50 simulator types ranging from cadaveric, low-fidelity, and part-task to virtual reality (VR) simulators. In all, 32 studies were included in the meta-analysis, including 7 randomised controlled trials. A random effects, ratio of means effects measure quantified statistically significant improvement in procedural knowledge by 50.2% (ES 0.502; CI 0.355; 0.649, p < 0.001), technical skill including accuracy by 32.5% (ES 0.325; CI - 0.482; - 0.167, p < 0.001), and speed by 25% (ES - 0.25, CI - 0.399; - 0.107, p < 0.001). The initial number of VR studies (n = 91) was approximately double the number of refining studies (n = 45) indicating it is yet to reach progressive scholarly acceptance. There is strong evidence for a beneficial impact of adopting simulation in the improvement of procedural knowledge and technical skill. We show a growing trend towards the adoption of neurosurgical simulators, although we have not fully gained progressive scholarly acceptance for VR-based simulation technologies in neurosurgical education.
Collapse
Affiliation(s)
- Joseph Davids
- Department of Neurosurgery, National Hospital for Neurology and Neurosurgery, Queen Square, Holborn, London, WC1N 3BG, UK.
- Imperial College Healthcare NHS Trust, St Mary's Praed St, Paddington, London, W2 1NY, UK.
| | - Susruta Manivannan
- Department of Neurosurgery, Southampton University NHS Trust, Tremona Road, Southampton, SO16 6YD, UK
| | - Ara Darzi
- Imperial College Healthcare NHS Trust, St Mary's Praed St, Paddington, London, W2 1NY, UK
| | - Stamatia Giannarou
- Imperial College Healthcare NHS Trust, St Mary's Praed St, Paddington, London, W2 1NY, UK
| | - Hutan Ashrafian
- Imperial College Healthcare NHS Trust, St Mary's Praed St, Paddington, London, W2 1NY, UK
| | - Hani J Marcus
- Department of Neurosurgery, National Hospital for Neurology and Neurosurgery, Queen Square, Holborn, London, WC1N 3BG, UK
- Imperial College Healthcare NHS Trust, St Mary's Praed St, Paddington, London, W2 1NY, UK
| |
Collapse
|
11
|
Tran DH, Winkler-Schwartz A, Tuznik M, Gueziri HE, Rudko DA, Reich A, Yilmaz R, Karlik B, Collins DL, Del Maestro A, Del Maestro R. Quantitation of Tissue Resection Using a Brain Tumor Model and 7-T Magnetic Resonance Imaging Technology. World Neurosurg 2021; 148:e326-39. [PMID: 33418122 DOI: 10.1016/j.wneu.2020.12.141] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/18/2020] [Revised: 12/27/2020] [Accepted: 12/27/2020] [Indexed: 01/17/2023]
Abstract
BACKGROUND Animal brain tumor models can be useful educational tools for the training of neurosurgical residents in risk-free environments. Magnetic resonance imaging (MRI) technologies have not used these models to quantitate tumor, normal gray and white matter, and total tissue removal during complex neurosurgical procedures. This pilot study was carried out as a proof of concept to show the feasibility of using brain tumor models combined with 7-T MRI technology to quantitatively assess tissue removal during subpial tumor resection. METHODS Seven ex vivo calf brain hemispheres were used to develop the 7-T MRI segmentation methodology. Three brains were used to quantitate brain tissue removal using 7-T MRI segmentation methodology. Alginate artificial brain tumor was created in 4 calf brains to assess the ability of 7-T MRI segmentation methodology to quantitate tumor and gray and white matter along with total tissue volumes removal during a subpial tumor resection procedure. RESULTS Quantitative studies showed a correlation between removed brain tissue weights and volumes determined from segmented 7-T MRIs. Analysis of baseline and postresection alginate brain tumor segmented 7-T MRIs allowed quantification of tumor and gray and white matter along with total tissue volumes removed and detection of alterations in surrounding gray and white matter. CONCLUSIONS This pilot study showed that the use of animal tumor models in combination with 7-T MRI technology provides an opportunity to increase the granularity of data obtained from operative procedures and to improve the assessment and training of learners.
Collapse
|
12
|
Scullen T, Mathkour M, Dumont A. Commentary: Virtual Reality Anterior Cervical Discectomy and Fusion Simulation on the Novel Sim-Ortho Platform: Validation Studies. Oper Neurosurg (Hagerstown) 2020; 20:E17-E19. [PMID: 32970133 DOI: 10.1093/ons/opaa285] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/03/2020] [Accepted: 07/11/2020] [Indexed: 11/13/2022] Open
|
13
|
Winkler-Schwartz A, Yilmaz R, Tran DH, Gueziri HE, Ying B, Tuznik M, Fonov V, Collins L, Rudko DA, Li J, Debergue P, Pazos V, Del Maestro R. Creating a Comprehensive Research Platform for Surgical Technique and Operative Outcome in Primary Brain Tumor Neurosurgery. World Neurosurg 2020; 144:e62-e71. [DOI: 10.1016/j.wneu.2020.07.209] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/26/2020] [Revised: 07/26/2020] [Accepted: 07/28/2020] [Indexed: 02/05/2023]
|
14
|
Ledwos N, Mirchi N, Bissonnette V, Winkler-Schwartz A, Yilmaz R, Del Maestro RF. Virtual Reality Anterior Cervical Discectomy and Fusion Simulation on the Novel Sim-Ortho Platform: Validation Studies. Oper Neurosurg (Hagerstown) 2020; 20:74-82. [DOI: 10.1093/ons/opaa269] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/29/2020] [Accepted: 06/28/2020] [Indexed: 11/13/2022] Open
Abstract
ABSTRACT
BACKGROUND
Virtual reality spine simulators are emerging as potential educational tools to assess and train surgical procedures in safe environments. Analysis of validity is important in determining the educational utility of these systems.
OBJECTIVE
To assess face, content, and construct validity of a C4-C5 anterior cervical discectomy and fusion simulation on the Sim-Ortho virtual reality platform, developed by OSSimTechTM (Montreal, Canada) and the AO Foundation (Davos, Switzerland).
METHODS
Spine surgeons, spine fellows, along with neurosurgical and orthopedic residents, performed a simulated C4-C5 anterior cervical discectomy and fusion on the Sim-Ortho system. Participants were separated into 3 categories: post-residents (spine surgeons and spine fellows), senior residents, and junior residents. A Likert scale was used to assess face and content validity. Construct validity was evaluated by investigating differences between the 3 groups on metrics derived from simulator data. The Kruskal-Wallis test was employed to compare groups and a post-hoc Dunn's test with a Bonferroni correction was utilized to investigate differences between groups on significant metrics.
RESULTS
A total of 21 individuals were included: 9 post-residents, 5 senior residents, and 7 junior residents. The post-resident group rated face and content validity, median ≥4, for the overall procedure and at least 1 tool in each of the 4 steps. Significant differences (P < .05) were found between the post-resident group and senior and/or junior residents on at least 1 metric for each component of the simulation.
CONCLUSION
The C4-C5 anterior cervical discectomy and fusion simulation on the Sim-Ortho platform demonstrated face, content, and construct validity suggesting its utility as a formative educational tool.
Collapse
Affiliation(s)
- Nicole Ledwos
- Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology & Neurosurgery, Montreal Neurological Institute and Hospital, McGill University, Montreal, Canada
| | - Nykan Mirchi
- Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology & Neurosurgery, Montreal Neurological Institute and Hospital, McGill University, Montreal, Canada
| | - Vincent Bissonnette
- Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology & Neurosurgery, Montreal Neurological Institute and Hospital, McGill University, Montreal, Canada
- Division of Orthopaedic Surgery, Montreal General Hospital, McGill University, Montreal, Canada
| | - Alexander Winkler-Schwartz
- Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology & Neurosurgery, Montreal Neurological Institute and Hospital, McGill University, Montreal, Canada
| | - Recai Yilmaz
- Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology & Neurosurgery, Montreal Neurological Institute and Hospital, McGill University, Montreal, Canada
| | - Rolando F Del Maestro
- Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology & Neurosurgery, Montreal Neurological Institute and Hospital, McGill University, Montreal, Canada
| |
Collapse
|
15
|
Yan EG, Rennert RC, Levy DM, Levy ML. Three-Dimensional Modeling of Complex Pediatric Intracranial Aneurysmal Malformations With a Virtual Reality System. Simul Healthc 2021; 16:295-300. [PMID: 32890320 DOI: 10.1097/SIH.0000000000000498] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/25/2022]
Abstract
INTRODUCTION Surgical simulation is valuable in neurovascular surgery given the progressive rarity of these cases and their technical complexity, but its use has not been well described for pediatric vascular pathologies. We herein review the use of surgical simulation at our institution for complex pediatric aneurysmal malformations. METHODS A retrospective review of patients treated for middle cerebral artery aneurysmal malformations with surgical simulation assistance (SuRgical Planner [SRP]; Surgical Theater, Mayfield Village, OH) during a 2-year period at Rady Children's Hospital of San Diego was performed. RESULTS In 5 pediatric patients with complex MCA aneurysmal malformations (mean age = 33.2 ± 49.9 months), preoperative 3-dimensional (3D) interactive modeling informed treatment planning and enhanced surgeon understanding of the vascular pathology. Availability of intraoperative simulation also aided real-time anatomical understanding during surgery. Specific benefits of simulation for these cases included characterization of involved perforating vessels, as well as an enhanced understanding of flow patterns within associated complex arteriovenous fistulas and feeding vessel/daughter branch anatomy. Despite the complexity of the lesions treated, use of simulation qualitatively enhanced surgeon confidence. There were no perioperative complications for patients treated with open surgery. CONCLUSIONS Surgical simulation may aid in the treatment of complex pediatric aneurysmal malformations.
Collapse
|
16
|
Siyar S, Azarnoush H, Rashidi S, Del Maestro RF. Tremor Assessment during Virtual Reality Brain Tumor Resection. J Surg Educ 2020; 77:643-651. [PMID: 31822389 DOI: 10.1016/j.jsurg.2019.11.011] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/06/2019] [Revised: 11/25/2019] [Accepted: 11/26/2019] [Indexed: 06/10/2023]
Abstract
OBJECTIVE Assessment of physiological tremor during neurosurgical procedures may provide further insights into the composites of surgical expertise. Virtual reality platforms may provide a mechanism for the quantitative assessment of physiological tremor. In this study, a virtual reality simulator providing haptic feedback was used to study physiological tremor in a simulated tumor resection task with participants from a "skilled" group and a "novice" group. DESIGN The task involved using a virtual ultrasonic aspirator to remove a series of virtual brain tumors with different visual and tactile characteristics without causing injury to surrounding tissue. Power spectral density analysis was employed to quantitate hand tremor during tumor resection. Statistical t test was used to determine tremor differences between the skilled and novice groups obtained from the instrument tip x, y, z coordinates, the instrument roll, pitch, yaw angles, and the instrument haptic force applied during tumor resection. SETTING The study was conducted at the Neurosurgical Simulation and Artificial Intelligence Learning Centre, Montreal Neurological Institute and Hospital, McGill University, Montreal, Canada. PARTICIPANTS The skilled group comprised 23 neurosurgeons and senior residents and the novice group comprised 92 junior residents and medical students. RESULTS The spectral analysis allowed quantitation of physiological tremor during virtual reality tumor resection. The skilled group displayed smaller physiological tremor than the novice group in all cases. In 3 out of 7 cases the difference was statistically significant. CONCLUSIONS The first investigation of the application of a virtual reality platform is presented for the quantitation of physiological tremor during a virtual reality tumor resection task. The goal of introducing such methodology to assess tremor is to highlight its potential educational application in neurosurgical resident training and in helping to further define the psychomotor skill set of surgeons.
Collapse
Affiliation(s)
- Samaneh Siyar
- Department of Biomedical Engineering, Amirkabir University of Technology (Tehran Polytechnic), Tehran, Iran; Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurosurgery, Montreal Neurological Institute and Hospital, McGill University, Montreal, Canada
| | - Hamed Azarnoush
- Department of Biomedical Engineering, Amirkabir University of Technology (Tehran Polytechnic), Tehran, Iran; Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurosurgery, Montreal Neurological Institute and Hospital, McGill University, Montreal, Canada.
| | - Saeid Rashidi
- Science and Research Branch, Islamic Azad University, Tehran, Iran
| | - Rolando F Del Maestro
- Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurosurgery, Montreal Neurological Institute and Hospital, McGill University, Montreal, Canada
| |
Collapse
|
17
|
Sabbagh AJ, Bajunaid KM, Alarifi N, Winkler-Schwartz A, Alsideiri G, Al-Zhrani G, Alotaibi FE, Bugdadi A, Laroche D, Del Maestro RF. Roadmap for Developing Complex Virtual Reality Simulation Scenarios: Subpial Neurosurgical Tumor Resection Model. World Neurosurg 2020; 139:e220-9. [PMID: 32289510 DOI: 10.1016/j.wneu.2020.03.187] [Citation(s) in RCA: 20] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/28/2020] [Revised: 03/25/2020] [Accepted: 03/26/2020] [Indexed: 01/03/2023]
Abstract
BACKGROUND Advancement and evolution of current virtual reality (VR) surgical simulation technologies are integral to improve the available armamentarium of surgical skill education. This is especially important in high-risk surgical specialties. Such fields including neurosurgery are beginning to explore the utilization of virtual reality simulation in the assessment and training of psychomotor skills. An important issue facing the available VR simulation technologies is the lack of complexity of scenarios that fail to replicate the visual and haptic realities of complex neurosurgical procedures. Therefore there is a need to create more realistic and complex scenarios with the appropriate visual and haptic realities to maximize the potential of virtual reality technology. METHODS We outline a roadmap for creating complex virtual reality neurosurgical simulation scenarios using a step-wise description of our team's subpial tumor resection project as a model. RESULTS The creation of complex neurosurgical simulations involves integrating multiple modules into a scenario-building roadmap. The components of each module are described outlining the important stages in the process of complex VR simulation creation. CONCLUSIONS Our roadmap of a stepwise approach for the creation of complex VR-simulated neurosurgical procedures may also serve as a guide to aid the development of other VR scenarios in a variety of surgical fields. The generation of new VR complex simulated neurosurgical procedures, by surgeons for surgeons, with the help of computer scientists and engineers may improve the assessment and training of residents and ultimately improve patient care.
Collapse
|
18
|
Siyar S, Azarnoush H, Rashidi S, Winkler-Schwartz A, Bissonnette V, Ponnudurai N, Del Maestro RF. Machine learning distinguishes neurosurgical skill levels in a virtual reality tumor resection task. Med Biol Eng Comput 2020; 58:1357-1367. [DOI: 10.1007/s11517-020-02155-3] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [What about the content of this article? (0)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/05/2019] [Accepted: 03/12/2020] [Indexed: 10/24/2022]
|
19
|
Moiraghi A, Perin A, Sicky N, Godjevac J, Carone G, Ayadi R, Galbiati T, Gambatesa E, Rocca A, Fanizzi C, Schaller K, DiMeco F, Meling TR. EANS Basic Brain Course (ABC): combining simulation to cadaver lab for a new concept of neurosurgical training. Acta Neurochir (Wien) 2020; 162:453-460. [PMID: 31965316 DOI: 10.1007/s00701-020-04216-w] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/25/2019] [Accepted: 01/06/2020] [Indexed: 11/25/2022]
Abstract
BACKGROUND Neurosurgical training has traditionally been based on an apprenticeship model that requires considerable time and exposure to surgeries. Unfortunately, nowadays these requirements are hampered by several limitations (e.g., decreased caseload, worktime restrictions). Furthermore, teaching methods vary among residency programs due to cultural differences, monetary restrictions, and infrastructure conditions, with the possible consequence of jeopardizing residents' training. METHODS The EANS Basic Brain Course originated from a collaboration between the Besta NeuroSim Center in Milano and the Swiss Foundation for Innovation and Training in Surgery in Geneva. It was held for 5 neurosurgical residents (PGY1-3) who participated to this first pilot experience in January 2019. The main goal was to cover the very basic aspects of cranial surgery, including both technical and non-technical skills. The course was developed in modules, starting from the diagnostic paths and communication with patients (played by professional actors), then moving to practical simulation sessions, rapid theoretical lessons, and discussions based on real cases and critical ethical aspects. At the end, the candidates had cadaver lab sessions in which they practiced basic emergency procedures and craniotomies. The interaction between the participants and the faculties was created and maintained using role plays that smoothly improved the cooperation during debriefs and discussions, thus making the sessions exceedingly involving. RESULTS At the end of the course, every trainee was able to complete the course curriculum and all the participants expressed their appreciation for this innovative format, with a particular emphasis on the time spent learning non-technical skills, confirming that they feel this to be a fundamental aspect of a comprehensive training in neurosurgery. CONCLUSIONS It is possible that this combined concept of training on technical and non-technical skills, using emerging technologies along with pedagogic techniques and cadaver dissection, may become the state-of-the-art for European Neurosurgical training programs in the next future.
Collapse
Affiliation(s)
- Alessandro Moiraghi
- Division of Neurosurgery, Geneva University Hospitals and University of Geneva Faculty of Medicine, Geneva, Switzerland.
- Department of Neurosurgery, Sainte-Anne Hospital, Paris, France.
- Swiss Foundation for Innovation and Training in Surgery (SFITS), Geneva, Switzerland.
| | - Alessandro Perin
- Department of Neurosurgery, Fondazione IRCCS Istituto Neurologico "C. Besta", Milan, Italy
- Besta NeuroSim Center, Fondazione IRCCS Istituto Neurologico Nazionale "C. Besta", Milan, Italy
- Department of Life Sciences, University of Trieste, Trieste, Italy
| | - Nicolas Sicky
- Swiss Foundation for Innovation and Training in Surgery (SFITS), Geneva, Switzerland
| | - Jelena Godjevac
- Swiss Foundation for Innovation and Training in Surgery (SFITS), Geneva, Switzerland
| | - Giovanni Carone
- Besta NeuroSim Center, Fondazione IRCCS Istituto Neurologico Nazionale "C. Besta", Milan, Italy
- University of Brescia, Brescia, Italy
| | - Roberta Ayadi
- Department of Neurosurgery, Fondazione IRCCS Istituto Neurologico "C. Besta", Milan, Italy
- Besta NeuroSim Center, Fondazione IRCCS Istituto Neurologico Nazionale "C. Besta", Milan, Italy
| | - Tommaso Galbiati
- Besta NeuroSim Center, Fondazione IRCCS Istituto Neurologico Nazionale "C. Besta", Milan, Italy
- University of Milan, Milan, Italy
| | - Enrico Gambatesa
- Department of Neurosurgery, Fondazione IRCCS Istituto Neurologico "C. Besta", Milan, Italy
- University of Milan, Milan, Italy
| | - Alessandra Rocca
- Besta NeuroSim Center, Fondazione IRCCS Istituto Neurologico Nazionale "C. Besta", Milan, Italy
- University of Milan, Milan, Italy
| | - Claudia Fanizzi
- Department of Neurosurgery, Fondazione IRCCS Istituto Neurologico "C. Besta", Milan, Italy
- Besta NeuroSim Center, Fondazione IRCCS Istituto Neurologico Nazionale "C. Besta", Milan, Italy
| | - Karl Schaller
- Division of Neurosurgery, Geneva University Hospitals and University of Geneva Faculty of Medicine, Geneva, Switzerland
- Swiss Foundation for Innovation and Training in Surgery (SFITS), Geneva, Switzerland
| | - Francesco DiMeco
- Department of Neurosurgery, Fondazione IRCCS Istituto Neurologico "C. Besta", Milan, Italy
- Besta NeuroSim Center, Fondazione IRCCS Istituto Neurologico Nazionale "C. Besta", Milan, Italy
- EANS Training Committee, , Cirencester, UK
- Department of Pathophysiology and Transplantation, University of Milan, Milan, Italy
- Department of Neurological Surgery, Johns Hopkins Medical School, Baltimore, MD, USA
| | - Torstein R Meling
- Division of Neurosurgery, Geneva University Hospitals and University of Geneva Faculty of Medicine, Geneva, Switzerland
- Swiss Foundation for Innovation and Training in Surgery (SFITS), Geneva, Switzerland
- EANS Training Committee, , Cirencester, UK
| |
Collapse
|
20
|
Mirchi N, Bissonnette V, Yilmaz R, Ledwos N, Winkler-Schwartz A, Del Maestro RF. The Virtual Operative Assistant: An explainable artificial intelligence tool for simulation-based training in surgery and medicine. PLoS One 2020; 15:e0229596. [PMID: 32106247 DOI: 10.1371/journal.pone.0229596] [Citation(s) in RCA: 74] [Impact Index Per Article: 18.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/04/2019] [Accepted: 02/10/2020] [Indexed: 01/16/2023] Open
Abstract
Simulation-based training is increasingly being used for assessment and training of psychomotor skills involved in medicine. The application of artificial intelligence and machine learning technologies has provided new methodologies to utilize large amounts of data for educational purposes. A significant criticism of the use of artificial intelligence in education has been a lack of transparency in the algorithms' decision-making processes. This study aims to 1) introduce a new framework using explainable artificial intelligence for simulation-based training in surgery, and 2) validate the framework by creating the Virtual Operative Assistant, an automated educational feedback platform. Twenty-eight skilled participants (14 staff neurosurgeons, 4 fellows, 10 PGY 4-6 residents) and 22 novice participants (10 PGY 1-3 residents, 12 medical students) took part in this study. Participants performed a virtual reality subpial brain tumor resection task on the NeuroVR simulator using a simulated ultrasonic aspirator and bipolar. Metrics of performance were developed, and leave-one-out cross validation was employed to train and validate a support vector machine in Matlab. The classifier was combined with a unique educational system to build the Virtual Operative Assistant which provides users with automated feedback on their metric performance with regards to expert proficiency performance benchmarks. The Virtual Operative Assistant successfully classified skilled and novice participants using 4 metrics with an accuracy, specificity and sensitivity of 92, 82 and 100%, respectively. A 2-step feedback system was developed to provide participants with an immediate visual representation of their standing related to expert proficiency performance benchmarks. The educational system outlined establishes a basis for the potential role of integrating artificial intelligence and virtual reality simulation into surgical educational teaching. The potential of linking expertise classification, objective feedback based on proficiency benchmarks, and instructor input creates a novel educational tool by integrating these three components into a formative educational paradigm.
Collapse
|
21
|
Sawaya R, Alsideiri G, Bugdadi A, Winkler-Schwartz A, Azarnoush H, Bajunaid K, Sabbagh AJ, Del Maestro R. Development of a performance model for virtual reality tumor resections. J Neurosurg 2019; 131:192-200. [DOI: 10.3171/2018.2.jns172327] [Citation(s) in RCA: 12] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/16/2017] [Accepted: 02/16/2018] [Indexed: 11/06/2022]
Abstract
OBJECTIVEPrevious work from the authors has shown that hand ergonomics plays an important role in surgical psychomotor performance during virtual reality brain tumor resections. In the current study they propose a hypothetical model that integrates the human and task factors at play during simulated brain tumor resections to better understand the hand ergonomics needed for optimal safety and efficiency. They hypothesize that 1) experts (neurosurgeons), compared to novices (residents and medical students), spend a greater proportion of their time in direct contact with critical tumor areas; 2) hand ergonomic conditions (most favorable to unfavorable) prompt participants to adapt in order to optimize tumor resection; and 3) hand ergonomic adaptation is acquired with increasing expertise.METHODSIn an earlier study, experts (neurosurgeons) and novices (residents and medical students) were instructed to resect simulated brain tumors on the NeuroVR (formerly NeuroTouch) virtual reality neurosurgical simulation platform. For the present study, the simulated tumors were divided into four quadrants (Q1 to Q4) to assess hand ergonomics at various levels of difficulty. The spatial distribution of time expended, force applied, and tumor volume removed was analyzed for each participant group (total of 22 participants).RESULTSNeurosurgeons spent a significantly greater percentage of their time in direct contact with critical tumor areas. Under the favorable hand ergonomic conditions of Q1 and Q3, neurosurgeons and senior residents spent significantly more time in Q1 than in Q3. Although forces applied in these quadrants were similar, neurosurgeons, having spent more time in Q1, removed significantly more tumor in Q1 than in Q3. In a comparison of the most favorable (Q2) to unfavorable (Q4) hand ergonomic conditions, neurosurgeons adapted the forces applied in each quadrant to resect similar tumor volumes. Differences between Q2 and Q4 were emphasized in measures of force applied per second, tumor volume removed per second, and tumor volume removed per unit of force applied. In contrast, the hand ergonomics of medical students did not vary across quadrants, indicating the existence of an “adaptive capacity” in neurosurgeons.CONCLUSIONSThe study results confirm the experts’ (neurosurgeons) greater capacity to adapt their hand ergonomics during simulated neurosurgical tasks. The proposed hypothetical model integrates the study findings with various human and task factors that highlight the importance of learning in the acquisition of hand ergonomic adaptation.
Collapse
Affiliation(s)
- Robin Sawaya
- 1Neurosurgical Simulation Research and Training Centre, Department of Neurology & Neurosurgery, McGill University, Montreal, Quebec, Canada
| | - Ghusn Alsideiri
- 1Neurosurgical Simulation Research and Training Centre, Department of Neurology & Neurosurgery, McGill University, Montreal, Quebec, Canada
- 2Department of Surgery, College of Medicine, Sultan Qaboos University, Muscat, Oman
| | - Abdulgadir Bugdadi
- 1Neurosurgical Simulation Research and Training Centre, Department of Neurology & Neurosurgery, McGill University, Montreal, Quebec, Canada
- 3Department of Surgery, Faculty of Medicine, Umm Al-Qura University, Mecca, Saudi Arabia
| | - Alexander Winkler-Schwartz
- 1Neurosurgical Simulation Research and Training Centre, Department of Neurology & Neurosurgery, McGill University, Montreal, Quebec, Canada
| | - Hamed Azarnoush
- 4Department of Biomedical Engineering, Amirkabir University of Technology (Tehran Polytechnic), Tehran, Iran
| | - Khalid Bajunaid
- 1Neurosurgical Simulation Research and Training Centre, Department of Neurology & Neurosurgery, McGill University, Montreal, Quebec, Canada
- 5Divison of Neurosurgery, Faculty of Medicine, University of Jeddah, Saudi Arabia
| | - Abdulrahman J. Sabbagh
- 1Neurosurgical Simulation Research and Training Centre, Department of Neurology & Neurosurgery, McGill University, Montreal, Quebec, Canada
- 6Department of Neurosurgery, National Neuroscience Institute, King Fahad Medical City, Riyadh, Saudi Arabia; and
- 7Division of Neurosurgery, Department of Surgery, Faculty of Medicine and Clinical Skill and Simulation Center, King Abdulaziz University, Jeddah, Saudi Arabia
| | - Rolando Del Maestro
- 1Neurosurgical Simulation Research and Training Centre, Department of Neurology & Neurosurgery, McGill University, Montreal, Quebec, Canada
| |
Collapse
|
22
|
Winkler-Schwartz A, Marwa I, Bajunaid K, Mullah M, Alotaibi FE, Bugdadi A, Sawaya R, Sabbagh AJ, Del Maestro R. A Comparison of Visual Rating Scales and Simulated Virtual Reality Metrics in Neurosurgical Training: A Generalizability Theory Study. World Neurosurg 2019; 127:e230-e235. [PMID: 30880209 DOI: 10.1016/j.wneu.2019.03.059] [Citation(s) in RCA: 12] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/10/2018] [Revised: 03/05/2019] [Accepted: 03/06/2019] [Indexed: 11/27/2022]
Abstract
BACKGROUND Adequate assessment and feedback remains a cornerstone of psychomotor skills acquisition, particularly within neurosurgery where the consequence of adverse operative events is significant. However, a critical appraisal of the reliability of visual rating scales in neurosurgery is lacking. Therefore, we sought to design a study to compare visual rating scales with simulated metrics in a neurosurgical virtual reality task. METHODS Neurosurgical faculty rated anonymized participant video recordings of the removal of simulated brain tumors using a visual rating scale made up of seven composite elements. Scale reliability was evaluated using generalizability theory, and scale subcomponents were compared with simulated metrics using Pearson correlation analysis. RESULTS Four staff neurosurgeons evaluated 16 medical student neurosurgery applicants. Overall scale reliability and internal consistency were 0.73 and 0.90, respectively. Reliability of 0.71 was achieved with two raters. Individual participants, raters, and scale items accounted for 27%, 11%, and 0.6% of the data variability. The hemostasis scale component related to the greatest number of simulated metrics, whereas respect for no-go zones and tissue was correlated with none. Metrics relating to instrument force and patient safety (brain volume removed and blood loss) were captured by the fewest number of rating scale components. CONCLUSIONS To our knowledge, this is the first study comparing participant's ratings with simulated performance. Given rating scales capture less well instrument force, quantity of brain volume removed, and blood loss, we suggest adopting a hybrid educational approach using visual rating scales in an operative environment, supplemented by simulated sessions to uncover potentially problematic surgical technique.
Collapse
Affiliation(s)
- Alexander Winkler-Schwartz
- Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology & Neurosurgery, McGill University, Montreal, Quebec, Canada.
| | - Ibrahim Marwa
- Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology & Neurosurgery, McGill University, Montreal, Quebec, Canada
| | - Khalid Bajunaid
- Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology & Neurosurgery, McGill University, Montreal, Quebec, Canada; Division of Neurosurgery, Department of Surgery, Faculty of Medicine, University of Jeddah, Jeddah, Saudi Arabia
| | - Muhammad Mullah
- Department of Epidemiology, Biostatistics & Occupational Health, McGill University, Montreal, Quebec, Canada
| | - Fahad E Alotaibi
- Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology & Neurosurgery, McGill University, Montreal, Quebec, Canada; Neurosurgical Department, National Neuroscience Institute, King Fahad Medical City, Riyadh, Saudi Arabia
| | - Abdulgadir Bugdadi
- Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology & Neurosurgery, McGill University, Montreal, Quebec, Canada; Department of Surgery, Faculty of Medicine, Umm Al Qura University, Makkah, Saudi Arabia
| | - Robin Sawaya
- Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology & Neurosurgery, McGill University, Montreal, Quebec, Canada
| | - Abdulrahman J Sabbagh
- Division of Neurosurgery, Department of Surgery, Faculty of Medicine, University of Jeddah, Jeddah, Saudi Arabia; Clinical Skills and Simulation Center, King Abdulaziz University, Jeddah, Saudi Arabia
| | - Rolando Del Maestro
- Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology & Neurosurgery, McGill University, Montreal, Quebec, Canada
| |
Collapse
|
23
|
Bugdadi A, Sawaya R, Bajunaid K, Olwi D, Winkler-Schwartz A, Ledwos N, Marwa I, Alsideiri G, Sabbagh AJ, Alotaibi FE, Al-Zhrani G, Maestro RD. Is Virtual Reality Surgical Performance Influenced by Force Feedback Device Utilized? J Surg Educ 2019; 76:262-273. [PMID: 30072262 DOI: 10.1016/j.jsurg.2018.06.012] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/02/2018] [Revised: 05/19/2018] [Accepted: 06/19/2018] [Indexed: 06/08/2023]
Abstract
OBJECTIVE The study objectives were to assess if surgical performance and subjective assessment of a virtual reality simulator platform was influenced by changing force feedback devices. DESIGN Participants used the NeuroVR (formerly NeuroTouch) simulator to perform 5 practice scenarios and a realistic scenario involving subpial resection of a virtual reality brain tumor with simulated bleeding. The influence of force feedback was assessed by utilizing the Omni and Entact haptic systems. Tier 1, tier 2, and tier 2 advanced metrics were used to compare results. Operator subjective assessment of the haptic systems tested utilized seven Likert criteria (score 1 to 5). SETTING The study is carried out at the McGill Neurosurgical Simulation Research and Training Centre, Montreal Neurological Institute and Hospital, Montreal, Canada. PARTICIPANTS Six expert operators in the utilization of the NeuroVR simulator platform. RESULTS No significant differences in surgical performance were found between the two haptic devices. Participants significantly preferred the Entact system on all 7 Likert criteria of subjective assessment. CONCLUSIONS Our results show no statistical differences in virtual reality surgical performance utilizing the two bimanual haptic devices tested. Subjective assessments demonstrated that participants preferred the Entact system. Our results suggest that to maximize realism of the training experience educators employing virtual reality simulators may find it useful to assess expert opinion before choosing a force feedback device.
Collapse
Affiliation(s)
- Abdulgadir Bugdadi
- Neurosurgical Simulation Research and Training Centre, Department of Neurosurgery and Neurology, McGill University, Montreal, Quebec, Canada; Department of Surgery, Faculty of Medicine, Umm Al-Qura University, Makkah Almukarramah, Saudi Arabia.
| | - Robin Sawaya
- Neurosurgical Simulation Research and Training Centre, Department of Neurosurgery and Neurology, McGill University, Montreal, Quebec, Canada
| | - Khalid Bajunaid
- Neurosurgical Simulation Research and Training Centre, Department of Neurosurgery and Neurology, McGill University, Montreal, Quebec, Canada; Division of Neurosurgery, Faculty of Medicine, University of Jeddah, Jeddah, Saudi Arabia
| | - Duaa Olwi
- Neurosurgical Simulation Research and Training Centre, Department of Neurosurgery and Neurology, McGill University, Montreal, Quebec, Canada; King Abdullah International Medical Research Center, Ministry of National Guard Health Affairs, Jeddah, Saudi Arabia
| | - Alexander Winkler-Schwartz
- Neurosurgical Simulation Research and Training Centre, Department of Neurosurgery and Neurology, McGill University, Montreal, Quebec, Canada
| | - Nicole Ledwos
- Neurosurgical Simulation Research and Training Centre, Department of Neurosurgery and Neurology, McGill University, Montreal, Quebec, Canada
| | - Ibrahim Marwa
- Neurosurgical Simulation Research and Training Centre, Department of Neurosurgery and Neurology, McGill University, Montreal, Quebec, Canada
| | - Ghusn Alsideiri
- Neurosurgical Simulation Research and Training Centre, Department of Neurosurgery and Neurology, McGill University, Montreal, Quebec, Canada; Department of Surgery, College of Medicine, Sultan Qaboos University, Muscat, Oman
| | - Abdulrahman Jafar Sabbagh
- Neurosurgical Simulation Research and Training Centre, Department of Neurosurgery and Neurology, McGill University, Montreal, Quebec, Canada; Division of Neurosurgery, Department of Surgery, Faculty of Medicine, King Abdulaziz University, Jeddah, Saudi Arabia; Clinical Skill and Simulation Center, King Abdulaziz University, Jeddah, Saudi Arabia
| | - Fahad E Alotaibi
- Neurosurgical Simulation Research and Training Centre, Department of Neurosurgery and Neurology, McGill University, Montreal, Quebec, Canada; National Neuroscience Institute, Department of Neurosurgery, King Fahad Medical City, Riyadh, Saudi Arabia
| | - Gmaan Al-Zhrani
- Neurosurgical Simulation Research and Training Centre, Department of Neurosurgery and Neurology, McGill University, Montreal, Quebec, Canada; National Neuroscience Institute, Department of Neurosurgery, King Fahad Medical City, Riyadh, Saudi Arabia
| | - Rolando Del Maestro
- Neurosurgical Simulation Research and Training Centre, Department of Neurosurgery and Neurology, McGill University, Montreal, Quebec, Canada
| |
Collapse
|