Validation of the updated ArthroS simulator: face and construct validity of a passive haptic virtual reality simulator with novel performance metrics

Neuromechanisms of simulation-based arthroscopic skills assessment: a fNIRS study

BACKGROUND

The neural mechanisms underlying differences in the performance of simulated arthroscopic skills across various skill levels remain unclear. Our primary objective is to investigate the learning mechanisms of simulated arthroscopic skills using functional near-infrared spectroscopy (fNIRS).

METHODS

We recruited 27 participants, divided into three groups: novices (n = 9), intermediates (n = 9), and experts (n = 9). Participants completed seven arthroscopic tasks on a simulator, including diagnostic navigation, triangulation, grasping stars, diagnostic exploration, meniscectomy, synovial membrane cleaning, and loose body removal. All tasks were videotaped and assessed via the simulator system and the Arthroscopic Surgical Skill Evaluation Tool (ASSET), while cortical activation data were collected using fNIRS. Simulator scores and ASSET scores were analyzed to identify different level of performance of all participants. Brain region activation and functional connectivity (FC) of different types of participants were analyzed from fNIRS data.

RESULTS

Both the expert and intermediate groups scored significantly higher than the novice group (p < 0.001). There were significant differences in ASSET scores between experts and intermediates, experts and novices, and intermediates and novices (p = 0.0047, p < 0.0001, p < 0.0001), with the trend being experts > intermediates > novices. The intermediate group exhibited significantly greater activation in the left primary motor cortex (LPMC) and left prefrontal cortex (LPFC) compared to the novice group (p = 0.0152, p = 0.0021). Compared to experts, the intermediate group demonstrated significantly increased FC between the presupplementary motor area (preSMA) and the right prefrontal cortex (RPFC; p < 0.001). Additionally, the intermediate group showed significantly increased FC between the preSMA and LPFC, RPFC and LPFC, and LPMC and LPFC compared to novices (p = 0.0077, p = 0.0285, p = 0.0446).

CONCLUSION

Cortical activation and functional connectivity reveal varying levels of activation intensity in the PFC, PMC, and preSMA among novices, intermediates, and experts. The intermediate group exhibited the highest activation intensity.

Virtual and Augmented Reality Simulators Show Intraoperative, Surgical Training, and Athletic Training Applications: A Scoping Review

PURPOSE

To review published literature to identify and evaluate the effect of virtual reality (complete immersion) and augmented reality (overlay of digital information onto the physical world) simulators on intraoperative use for orthopaedic surgeons, orthopaedic surgical education, and athletic training.

METHODS

A systematic review was performed using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines to identify studies, published since 2014, that evaluated the role of augmented/virtual reality on intraoperative use for orthopaedic surgeons, orthopaedic surgical education, and athletic training.

RESULTS

Virtual reality (VR) simulators provide 3-dimensional graphical simulation of the physical world, and augmented reality (AR) simulators overlay digital information onto the physical world. Simulators can include interactive features (i.e., replication of intraoperative bleeding), haptic feedback, and unrestricted task repetition, and they can record, compare, and analyze performance while being easily accessible and eliminating the need for the presence of a mentor or coach. Four studies reported on VR for intraoperative use, 47 studies on surgical education, and 10 studies on athletic training. Two studies revealed the advantages of using VR simulation during intraoperative procedures, specifically showcasing its benefits for elbow arthroscopy, while 2 studies demonstrated similar positive outcomes for hip arthroscopy. Seventeen studies demonstrated that a VR simulator could be a beneficial tool to assist in surgical education for the knee, while 12 studies found that VR simulation is a valuable tool for aiding in surgical education of shoulder arthroscopy. Ten studies demonstrated that VR simulation improves skills in the operating room. Three studies revealed that individuals with more experience exhibit superior performance on these simulators compared to those with less experience. In the realm of athletic training, 10 studies showcased the potential of VR simulation to play a significant role in athletic performance and injury rehabilitation.

CONCLUSIONS

VR simulation shows benefits in the operating room, is a valuable tool for surgical education resulting in improved skills, and can be used to enhance athletic performance and injury rehabilitation.

CLINICAL RELEVANCE

Understanding that VR simulators can improve surgical outcomes, surgical skill training, and athletic training and rehabilitation could facilitate development and adoption of this advanced technology.

Virtual reality simulation training improve diagnostic knee arthroscopy and meniscectomy skills: a prospective transfer validity study

PURPOSE

Limited data exist on the actual transfer of skills learned using a virtual reality (VR) simulator for arthroscopy training because studies mainly focused on VR performance improvement and not on transfer to real word (transfer validity). The purpose of this single-blinded, controlled trial was to objectively investigate transfer validity in the context of initial knee arthroscopy training.

METHODS

For this study, 36 junior resident orthopaedic surgeons (postgraduate year one and year two) without prior experience in arthroscopic surgery were enrolled to receive standard knee arthroscopy surgery training (NON-VR group) or standard training plus training on a hybrid virtual reality knee arthroscopy simulator (1 h/month) (VR group). At inclusion, all participants completed a questionnaire on their current arthroscopic technical skills. After 6 months of training, both groups performed three exercises that were evaluated independently by two blinded trainers: i) arthroscopic partial meniscectomy on a bench-top knee simulator; ii) supervised diagnostic knee arthroscopy on a cadaveric knee; and iii) supervised knee partial meniscectomy on a cadaveric knee. Training level was determined with the Arthroscopic Surgical Skill Evaluation Tool (ASSET) score.

RESULTS

Overall, performance (ASSET scores) was better in the VR group than NON-VR group (difference in the global scores: p < 0.001, in bench-top meniscectomy scores: p = 0.03, in diagnostic knee arthroscopy on a cadaveric knee scores: p = 0.04, and in partial meniscectomy on a cadaveric knee scores: p = 0.02). Subgroup analysis by postgraduate year showed that the year-one NON-VR subgroup performed worse than the other subgroups, regardless of the exercise.

CONCLUSION

This study showed the transferability of the technical skills acquired by novice residents on a hybrid virtual reality simulator to the bench-top and cadaveric models. Surgical skill acquired with a VR arthroscopy surgical simulator might safely improve arthroscopy competences in the operating room, also helping to standardise resident training and follow their progress.

LEVEL OF EVIDENCE: 2

TACTICS VR Stroke Telehealth Virtual Reality Training for Health Care Professionals Involved in Stroke Management at Telestroke Spoke Hospitals: Module Design and Implementation Study

BACKGROUND

Stroke management in rural areas is more variable and there is less access to reperfusion therapies, when compared with metropolitan areas. Delays in treatment contribute to worse patient outcomes. To improve stroke management in rural areas, health districts are implementing telestroke networks. The New South Wales Telestroke Service provides neurologist-led telehealth to 23 rural spoke hospitals aiming to improve treatment delivery and patient outcomes. The training of clinical staff was identified as a critical aspect for the successful implementation of this service. Virtual reality (VR) training has not previously been used in this context.

OBJECTIVE

We sought to develop an evidence-based VR training module specifically tailored for stroke telehealth. During implementation, we aimed to assess the feasibility of workplace deployment and collected feedback from spoke hospital staff involved in stroke management on training acceptability and usability as well as perceived training impact.

METHODS

The TACTICS VR Stroke Telehealth application was developed with subject matter experts. During implementation, both quantitative and qualitative data were documented, including VR use and survey feedback. VR hardware was deployed to 23 rural hospitals, and use data were captured via automated Wi-Fi transfer. At 7 hospitals in a single local health district, staff using TACTICS VR were invited to complete surveys before and after training.

RESULTS

TACTICS VR Stroke Telehealth was deployed to rural New South Wales hospitals starting on April 14, 2021. Through August 20, 2023, a total of 177 VR sessions were completed. Survey respondents (n=20) indicated a high level of acceptability, usability, and perceived training impact (eg, accuracy and knowledge transfer; mean scores 3.8-4.4; 5=strongly agree). Furthermore, respondents agreed that TACTICS VR increased confidence (13/18, 72%), improved understanding (16/18, 89%), and improved awareness (17/18, 94%) regarding stroke telehealth. A comparison of matched pre- and posttraining responses revealed that training improved the understanding of telehealth workflow practices (after training: mean 4.2, SD 0.6; before training: mean 3.2, SD 0.9; P<.001), knowledge on accessing stroke telehealth (mean 4.1, SD 0.6 vs mean 3.1, SD 1.0; P=.001), the awareness of stroke telehealth (mean 4.1, SD 0.6 vs mean 3.4, SD 0.9; P=.03), ability to optimally communicate with colleagues (mean 4.2, SD 0.6 vs mean 3.7, SD 0.9; P=.02), and ability to make improvements (mean 4.0, SD 0.6 vs mean 3.5, SD 0.9; P=.03). Remote training and deployment were feasible, and limited issues were identified, although uptake varied widely (0-66 sessions/site).

CONCLUSIONS

TACTICS VR Stroke Telehealth is a new VR application specifically tailored for stroke telehealth workflow training at spoke hospitals. Training was considered acceptable, usable, and useful and had positive perceived training impacts in a real-world clinical implementation context. Additional work is required to optimize training uptake and integrate training into existing education pathways.

Technological innovations in shoulder replacement: current concepts and the future of robotics in total shoulder arthroplasty

BACKGROUND

Total shoulder arthroplasty (TSA) has been rapidly evolving over the last several decades, with innovative technological strategies being investigated and developed in order to achieve optimal component precision and joint alignment and stability, preserve implant longevity, and improve patient outcomes. Future advancements such as robotic-assisted surgeries, augmented reality, artificial intelligence, patient-specific instrumentation (PSI) and other peri- and preoperative planning tools will continue to revolutionize TSA. Robotic-assisted arthroplasty is a novel and increasingly popular alternative to the conventional arthroplasty procedure in the hip and knee but has not yet been investigated in the shoulder. Therefore, the purpose of this study was to conduct a narrative review of the literature on the evolution and projected trends of technological advances and robotic assistance in total shoulder arthroplasty.

METHODS

A narrative synthesis method was employed for this review, rather than a meta-analysis or systematic review of the literature. This decision was based on 2 primary factors: (1) the lack of eligible, peer-reviewed studies with high-quality level of evidence available for review on robotic-assisted shoulder arthroplasty, and (2) a narrative review allows for a broader scope of content analysis, including a comprehensive review of all technological advances-including robotics-within the field of TSA. A general literature search was performed using PubMed, Embase, and Cochrane Library databases. These databases were queried by 2 independent reviewers from database inception through November 11, 2022, for all articles investigating the role of robotics and technology assistance in total shoulder arthroplasty. Inclusion criteria included studies describing "shoulder arthroplasty" and "robotics."

RESULTS

After exclusion criteria were applied, 4 studies on robotic-assisted TSA were described in the review. Given the novelty of this technology and limited data on robotics in TSA, these studies consisted of a literature review, nonvalidated experimental biomechanical studies in sawbones models, and preclinical proof-of-concept cadaveric studies using prototype robotic technology primarily in conjunction with PSI. The remaining studies described the technological advancements in TSA, including PSI, computer-assisted navigation, artificial intelligence, machine learning, and virtual, augmented, and mixed reality. Although not yet commercially available, robotic-assisted TSA confers the theoretical advantages of precise humeral head cuts for restoration of proximal humerus anatomy, more accurate glenoid preparation, and improved soft-tissue assessment in limited early studies.

CONCLUSION

The evidence for the use of robotics in total hip arthroplasty and total knee arthroplasty demonstrates improved component accuracy, more precise radiographic measurements, and improved early/mid-term patient-reported and functional outcomes. Although no such data currently exist for shoulder arthroplasty given that the technology has not yet been commercialized, the lessons learned from robotic hip and knee surgery in conjunction with its rapid adoption suggests robotic-assisted TSA is on the horizon of innovation. By achieving a better understanding of the past, present, and future innovations in TSA through this narrative review, orthopedic surgeons can be better prepared for future applications.

Hand Dominance Is Not of Significance in Performing Fundamental Arthroscopic Skills Simulation Training Tasks

Purpose

To compare the performance of the dominant and nondominant hand during fundamental arthroscopic simulator training.

Methods

Surgical trainees who participated in a 2-day simulator training course between 2021 and 2023 were classified, according to their arthroscopic experience in beginners and competents. Only right-handed individuals with complete data sets were included in the study. Ambidexterity was trained using a box trainer (Fundamentals of Arthroscopic Surgery Training, Virtamed AG, Schlieren, Switzerland).Two tasks, periscoping for learning camera guidance and triangulation for additional instrument handling, were performed 4 times with the camera in the dominant hand and then in the nondominant hand. For each task, exercise time, camera path length, and instrument path length were recorded and analyzed.

Results

Out of 94 participants 74 right-handed individuals (22 females, 52 males) were classified to novices (n = 43, less than 10 independently performed arthroscopies) and competents (n = 31, more than 10 independently performed arthroscopies). Competents performed significantly better than novices. No significant difference was found after changing the guiding hand for the camera from the dominant to the nondominant hand regarding the camera path length and the instrument path length. Notably, tasks were performed even faster when using the camera in the nondominant hand.

Conclusions

Our data demonstrate that the learned manual skills during basic arthroscopic training are quickly transferred to the contralateral side. In consequence, additional fundamental skills training for camera guidance and instrument handling of the nondominant hand are not necessary.

Clinical Relevance

For skillful arthroscopy, camera guidance and instrument handing must be equally mastered with both hands. It is important to understand how hand dominance may affect learning during arthroscopic simulator training.

Perceived realism of haptic rendering methods for bimanual high force tasks: original and replication study

Realistic haptic feedback is a key for virtual reality applications in order to transition from solely procedural training to motor-skill training. Currently, haptic feedback is mostly used in low-force medical procedures in dentistry, laparoscopy, arthroscopy and alike. However, joint replacement procedures at hip, knee or shoulder, require the simulation of high-forces in order to enable motor-skill training. In this work a prototype of a haptic device capable of delivering double the force (35 N to 70 N) of state-of-the-art devices is used to examine the four most common haptic rendering methods (penalty-, impulse-, constraint-, rigid body-based haptic rendering) in three bimanual tasks (contact, rotation, uniaxial transition with increasing forces from 30 to 60 N) regarding their capabilities to provide a realistic haptic feedback. In order to provide baseline data, a worst-case scenario of a steel/steel interaction was chosen. The participants needed to compare a real steel/steel interaction with a simulated one. In order to substantiate our results, we replicated the study using the same study protocol and experimental setup at another laboratory. The results of the original study and the replication study deliver almost identical results. We found that certain investigated haptic rendering method are likely able to deliver a realistic sensation for bone-cartilage/steel contact but not for steel/steel contact. Whilst no clear best haptic rendering method emerged, penalty-based haptic rendering performed worst. For simulating high force bimanual tasks, we recommend a mixed implementation approach of using impulse-based haptic rendering for simulating contacts and combine it with constraint or rigid body-based haptic rendering for rotational and translational movements.

Simulation training to retool practicing orthopedic surgeons is rare

Purpose

Modern surgical practice is continuously changing as technology develops. New techniques are often implemented after a surgeon has made the transition to independent clinical practice. There is therefore a need to 'retool' technical skills. Additionally, practicing surgeons must maintain and develop skills such as leadership, communication, critical thinking, teaching, and mentoring. Our aim was to perform a scoping review to assess the current status of simulation education for practicing Orthopedic Surgeons (OS).

Methods

A 10 year search of PubMed, ERIC, and Web of Science was performed with a medical librarian. Controlled vocabulary Medical Subject Headings terms and natural language were developed with subject matter experts describing simulation, training and OS. Two trained reviewers evaluated all abstracts for inclusion. Exclusion criteria were articles that did not assess simulation education involving practicing OS. Data were extracted from the included full text articles by two reviewers: details of study design, type of participants, type of simulation and role of OS in the educational event.

Results

Initial search identified 1824 articles of which 443 were duplicates, and 1381 articles were further screened. Of these, 1155 were excluded, 226 full text articles were assessed for eligibility and 80 included in analysis. Most were published in the last 6 years and from the United States. The majority (99%) described technical skill simulations (arthroscopy 56%, screw placement 23%, ligament reconstruction 19%). OS were rarely the only learners with 91% studies also having residents participate. OS were the targeted learner in 6% studies. OS provided content validity for 15 (19%) and construct validity in 59 (74%) studies.

Conclusions

Simulation training to educate practicing OS is rare. OS are often used to validate work rather than being the center of an educational endeavor. A refocusing is needed to provide adequate training for practicing surgeons to retool skills as new techniques become available.

Current status of virtual reality simulation education for orthopedic residents: the need for a change in focus

Introduction

Advances in technology are changing surgical education. Simulation provides an important adjunct to operative experience. This pedagogy has arguably become more important in light of the COVID-19 pandemic, with resultant reduction in operative exposure for trainees. Virtual reality (VR) simulators may provide significant contribution to experiential learning; however, much of the investigative focus to date has, correctly, been on establishing validity evidence for these constructs. The aim of this work was to perform a scoping review to assess the current status of VR simulation education to determine curricular development efforts for orthopedic residents.

Methods

With a trained medical librarian, searches of PubMed, EMBASE, and Web of Science were conducted for all articles in the last 10 years (September 2011-September 2021). Controlled vocabulary Medical Subject Headings (MeSH) terms and natural language developed with subject matter experts describing virtual reality or VR simulation and orthopedic training were used. Two trained reviewers evaluated all abstracts for inclusion. Exclusion criteria were all articles that did not assess VR simulation education involving orthopedic residents. Data were extracted from the included full-text articles including: study design, type of participants, type of VR simulation, simulated orthopedic skill, type of educational event, learner assessment including Kirkpatrick's level, assessment of quality using the Medical Education Research Study Quality Instrument (MERSQI), and level of effectiveness (LoE).

Results

Initial search identified 1,394 articles, of which 61 were included in the final qualitative synthesis. The majority (54%) were published in 2019- 2021, 49% in Europe. The commonest VR simulator was ArthroS (23%) and the commonest simulated skill was knee arthroscopy (33%). The majority of studies (70%) focused on simulator validation. Twenty-three studies described an educational module or curriculum, and of the 21 (34%) educational modules, 43% were one-off events. Most modules (18/21, 86%) assessed learners at Kirkpatrick level 2. With regard to methodological quality, 44% of studies had MERSQI 11.5-15 and 89% of studies had LoE of 2. Two studies had LoE of 3.

Conclusion

Current literature pertaining to VR training for orthopedic residents is focused on establishing validity and rarely forms part of a curriculum. Where the focus is education, the majority are discrete educational modules and do not teach a comprehensive amalgam of orthopedic skills. This suggests focus is needed to embed VR simulation training within formal curricula efforts guided by the work of Kern, and assess the efficacy of these against patient outcomes.

The Validation of Surgical Simulators: A Technical Report on Current Validation Terminology as a Reference for Future Research

In recent years, surgical trainees have been exposed to a lower volume of operative procedures. In part, this is due to the reduction in working hours and further disruption by the coronavirus disease 2019 pandemic. Much has been done to develop the techniques of surgical skill training outside of the operating theatre. Simulation-based interventions must undergo a process of validation to assess their appropriateness and effectiveness for use in training. The terminology of validation within current literature, however, has not evolved in line with the education community, resulting in varying definitions for the same phrase across domains. This can result in confusion and misinterpretation among researchers and surgeons working within this domain. This technical report describes the "types of validity" definitions used in the traditional framework of surgical simulation literature and the contemporary, unitary framework of validity adopted by educationalist theorists. There is a clear overlap between the traditional "types of validity" and the contemporary, unitary framework. The divergence in the use of those definitions seems, at least partly, influenced by the context of the investigations being conducted. By utilising the contemporary definitions, authors may have struggled to provide the evidence required to justify the use of the multitude of surgical skill simulators developed in the recent past. This report has provided an overview of the current terminology within the validation frameworks and can be used as a reference for future surgical simulation research.

Comparing approaches for selection, development, and deployment of extended reality (XR) teaching applications: A case study at The University of Newcastle Australia

The use of extended reality (XR) technology in education offers many advantages for transferring knowledge and practical skills training at the higher education level. As a result, many Universities over the past 5 + years have undertaken pilot programs to both develop XR content and assess how to best implement it within existing teaching and learning systems. Unfortunately, very few of these efforts have included structured evaluation or documentation. As such, limited published evidence exists to inform processes and approaches that may assist or hinder broad scale implementation. This leads many Universities to unnecessarily commit significant time and resources to testing identical or similar approaches, resulting in repeated identification of the same or similar challenges. In response to this situation, The University of Newcastle, Australia decided to systematically document the approach for selection, development and implementation of four new virtual-reality (VR) teaching applications. The current paper contains a detailed intrinsic case study, outlining the process and critical elements that shaped the selection of suitable teaching content, software development, hardware solutions and implementation. Details are provided on how decisions were made, what components were considered helpful, challenges identified, and important lessons outlined. These findings will be useful to organisations and individuals as they look to develop pathways and processes to integrate XR technology, particularly within their existing training and educational frameworks.

Supplementary Information

The online version contains supplementary material available at 10.1007/s10639-022-11364-2.

Digital body preservation: Technique and applications

High-fidelity anatomical models can be produced with three-dimensional (3D) scanning techniques and as such be digitally preserved, archived, and subsequently rendered through various media. Here, a novel methodology-digital body preservation-is presented for combining and matching scan geometry with radiographic imaging. The technique encompasses joining layers of 3D surface scans in an anatomical correct spatial relationship. To do so, a computed tomography (CT) volume is used as template to join and merge different surface scan geometries by means of nonrigid registration into a single environment. In addition, the use and applicability of the generated 3D models in digital learning modalities is presented. Finally, as computational expense is usually the main bottleneck in extended 3D applications, the influence of mesh simplification in combination with texture mapping on the quality of 3D models was investigated. The physical fidelity of the simplified meshes was evaluated in relation to their resolution and with respect to key anatomical features. Large- and medium-scale features were well preserved despite extensive 3D mesh simplification. Subtle fine-scale features, particular in curved areas demonstrated the major limitation to extensive mesh size reduction. Depending on the local topography, workable mesh sizes ranging from 10% to 3% of the original size could be obtained, making them usable in various learning applications and environments.

Ten hours of simulator training in arthroscopy are insufficient to reach the target level based on the Diagnostic Arthroscopic Skill Score

PURPOSE

Simulator arthroscopy training has gained popularity in recent years. However, it remains unclear what level of competency surgeons may achieve in what time frame using virtual training. It was hypothesized that 10 h of training would be sufficient to reach the target level defined by experts based on the Diagnostic Arthroscopic Skill Score (DASS).

METHODS

The training concept was developed by ten instructors affiliated with the German-speaking Society of Arthroscopy and Joint Surgery (AGA). The programme teaches the basics of performing arthroscopy; the main focus is on learning and practicing manual skills using a simulator. The training was based on a structured programme of exercises designed to help users reach defined learning goals. Initially, camera posture, horizon adjustment and control of the direction of view were taught in a virtual room. Based on these skills, further training was performed with a knee model. The learning progress was assessed by quantifying the exercise time, camera path length and instrument path length for selected tasks. At the end of the course, the learners' performance in diagnostic arthroscopy was evaluated using DASS. Participants were classified as novice or competent based on the number of arthroscopies performed prior to the assessment.

RESULTS

Except for one surgeon, 131 orthopaedic residents and surgeons (29 women, 102 men) who participated in the seven courses agreed to anonymous data analysis. Fifty-eight of them were competents with more than ten independently performed arthroscopies, and 73 were novices, with fewer than ten independently performed arthroscopies. There were significant reductions in exercise time, camera path length and instrument path length for all participants after the training, indicating a rapid increase in performance. No difference in camera handling between the dominant and non-dominant sides was found in either group. The competents performed better than the novices in various tasks and achieved significantly better DASS values on the final performance test.

CONCLUSIONS

Our data have demonstrated that arthroscopic skills can be taught effectively on a simulator, but a 10-h course is not sufficient to reach the target level set by experienced arthroscopists. However, learning progress can be monitored more objectively during simulator training than in the operating room, and simulation may partially replace the current practice of arthroscopic training.

LEVEL OF EVIDENCE

III.

Is the Virtual Reality Fundamentals of Arthroscopic Surgery Training Program a Valid Platform for Resident Arthroscopy Training?

BACKGROUND

Training in arthroscopy is associated with a steep learning curve for trainees and bears risks for patients. Virtual reality (VR) arthroscopy simulation platforms seek to overcome this and to provide a safe environment for surgical learners. The Fundamentals of Arthroscopic Surgery Training (FAST) program is one such platform. It is currently not known whether the VR FAST program can be employed as a useful teaching or examination tool to assess the basic arthroscopic skills of surgical trainees.

QUESTIONS/PURPOSES

(1) Does the VR FAST program differentiate among novice, intermediate, and expert arthroscopists? (2) Does ambidextrous performance in the VR FAST program correlate with arthroscopic experience?

METHODS

We prospectively recruited orthopaedic interns (novices), residents (intermediates), and fellows and attendings (experts) to complete the VR FAST program over a 1-year period from four major orthopaedic training programs on a voluntary basis. Sixty-six of 156 invited orthopaedic surgeons participated: 26 of 50 novices (16 men and 10 women), 27 of 65 intermediates (20 men and seven women), and 13 of 41 experts (10 men and three women). Surgeons of any arthroscopic experience were included, with only those with prior experience on the VR FAST program being excluded. The program consists of eight modules: three basic camera modules (Image Centering, Horizon Control, and Telescoping), three advanced camera modules (Periscoping, Trace the Line, and Trace the Curve), and two instrumented bimanual-dexterity modules (Probe Triangulation and Gather the Stars). Time taken to complete each task and measures of economy of movement (camera and instrument path length, camera alignment) were used as measures of arthroscopic experience. Every participant completed the modules using their dominant and nondominant hands. Equality in proficiency in completing the tasks using the dominant and nondominant hands were determined to be measures of arthroscopic experience. Due to the large number of outcome variables, only p values < 0.01 were considered to be statistically significant.

RESULTS

Six of eight VR FAST modules did not discriminate among novice, intermediate, and expert arthroscopy participants. However, two did, and the ones that were most effective at distinguishing participants by level of experience were the Periscoping and Gather the Stars modules. For the Periscoping module using the dominant hand, novices required longer to complete the task with a median time of 231 seconds (IQR 149 to 358) and longer camera path length median of 191 cm (IQR 128 to 273) compared with intermediates who needed 127 seconds (IQR 106 to 233) and 125 cm (IQR 92 to 159) and experts who needed 121 seconds (IQR 93 to 157) and 119 cm (IQR 90 to 134) (p = 0.001 and p = 0.003, respectively). When using the nondominant hand, novices took longer to complete the task with a median time of 231 seconds (IQR 170 to 350) and longer camera path length 204 cm (IQR 169 to 273) compared with intermediates who required 132 seconds (IQR 97 to 162) and 111 cm (IQR 88 to 143) and experts who needed 119 seconds (IQR 104 to 183) and 120 cm (IQR 108 to 166) (p < 0.001 and p < 0.001, respectively). For the Gather the Stars module using the nondominant hand, only the novices needed longer to complete the task at a median of 131 seconds (IQR 112 to 157) and needed a longer grasper path length of 290 cm (IQR 254 to 332) compared with intermediates who needed 84 seconds (IQR 72 to 119) and 232 cm (IQR 195 to 254) and experts who needed 98 seconds (IQR 87 to 107) and 244 cm (IQR 215 to 287) (p < 0.001 and p = 0.001, respectively).

CONCLUSION

Six of eight VR FAST modules did not demonstrate construct validity, and we found no correlation between arthroscopic experience and ambidextrous performance. Two modules demonstrated construct validity; however, refinement and expansion of the modules is needed with further validation in large prospective trials so that pass-fail thresholds can be set for use in high-stakes examinations.

CLINICAL RELEVANCE

Most VR FAST modules were not discriminatory; however, they can form essential conceptual and procedural building blocks in an arthroscopic curriculum that are beneficial for novices when developing key psychomotor skills. In their present format, however, they are unsuitable for assessing arthroscopic proficiency.

A Narrative Review of the Current State of Extended Reality Technology and How it can be Utilised in Sport

Extended reality is an umbrella term used to describe three computer-generated technologies including virtual reality, augmented reality and mixed reality. Extended reality is an emerging technology that has been utilised in many high-performance domains including psychology, medicine and the military, with the aim of enhancing perceptual-cognitive skills and motor skills. However, the use of extended reality in sport, particularly at the elite level, has only recently started to receive attention. While the growth of extended reality technology continues to accelerate at a rapid rate, empirical evidence aimed at understanding how these devices can best be applied in high-performance sport has not followed suit. Therefore, the purpose of this review is to provide clarity for high-performance sport organisations, researchers, sport scientists, coaches and athletes about the current state of extended reality technology and how it has been utilised in sport. In doing so, we first define and give examples of the types of extended reality technology including virtual reality, augmented reality and mixed reality that are available at the present time. Second, we detail how skill acquisition principles underpinned by the theoretical framework of ecological dynamics can be used to help inform the design and assessment of extended reality training tools. Third, we describe how extended reality has been utilised in sport, including how extended reality tools have been assessed for their level of representativeness, and the effectiveness of extended reality training interventions for improving perceptual-cognitive skills and motor skills. Finally, we discuss the future utilisation of extended reality in sport, including the key learnings that can be drawn from other domains, future research directions, practical applications and areas for consideration related to the use of extended reality for training skills in sport.

The Diagnostic Arthroscopy Skill Score (DASS): a reliable and suitable assessment tool for arthroscopic skill training

PURPOSE

To develop and validate a novel score to more objectively assess the performance of diagnostic knee arthroscopy using a simulator.

METHODS

A Diagnostic Arthroscopy Skill Score (DASS) was developed by ten AGA (AGA-Society for Arthroscopy and Joint-Surgery) instructors for the assessment of arthroscopic skills. DASS consists of two parts: the evaluation of standardized diagnostic knee arthroscopy (DASS_part1) and the evaluation of manual dexterity, including ambidexterity and triangulation, using objective measurement parameters (DASS_part2). Content validity was determined by the Delphi method. One hundred and eleven videos of diagnostic knee arthroscopies were recorded during simulator training courses and evaluated by six specially trained instructors using DASS. Construct validity, measurement error calculated by the minimum detectable change (MDC), internal consistency using Cronbach's alpha and interrater and intrarater reliability were assessed. The Bland-Altman method was used to calculate the intrarater agreement.

RESULTS

Six skill domains were identified and evaluated for each knee compartment. DASS, DASS_part1, and DASS_part2 showed construct validity, with experts achieving significantly higher scores than competents and novices. MDC was 4.5 ± 1.7 points for DASS_part1. There was high internal consistency for all domains in each compartment from 0.78 to 0.86. The interrater reliability showed high agreement between the six raters (ICC = 0.94). The evaluation of intrarater reliability demonstrated good and excellent agreement for five raters (ICC > 0.80) and moderate agreement for one rater (ICC = 0.68). The Bland-Altman comparison showed no difference between the first and second evaluations in five out of six raters. Precision, estimated by the regression analysis and comparison with the method of Bland and Altman, was excellent for four raters and moderate for two raters.

CONCLUSIONS

The results of this study indicate good validity and reliability of DASS for the assessment of the surgical performance of diagnostic knee arthroscopy during simulator training. Standardized training is recommended before arthroscopy surgery is considered in patients.

LEVEL OF EVIDENCE

II.

Development and Pilot Implementation of TACTICS VR: A Virtual Reality-Based Stroke Management Workflow Training Application and Training Framework

Delays in acute stroke treatment contribute to severe and negative impacts for patients and significant healthcare costs. Variability in clinical care is a contributor to delayed treatment, particularly in rural, regional and remote (RRR) areas. Targeted approaches to improve stroke workflow processes improve outcomes, but numerous challenges exist particularly in RRR settings. Virtual reality (VR) applications can provide immersive and engaging training and overcome some existing training barriers. We recently initiated the TACTICS trial, which is assessing a "package intervention" to support advanced CT imaging and streamlined stroke workflow training. As part of the educational component of the intervention we developed TACTICS VR, a novel VR-based training application to upskill healthcare professionals in optimal stroke workflow processes. In the current manuscript, we describe development of the TACTICS VR platform which includes the VR-based training application, a user-facing website and an automated back-end data analytics portal. TACTICS VR was developed via an extensive and structured scoping and consultation process, to ensure content was evidence-based, represented best-practice and is tailored for the target audience. Further, we report on pilot implementation in 7 Australian hospitals to assess the feasibility of workplace-based VR training. A total of 104 healthcare professionals completed TACTICS VR training. Users indicated a high level of usability, acceptability and utility of TACTICS VR, including aspects of hardware, software design, educational content, training feedback and implementation strategy. Further, users self-reported increased confidence in their ability to make improvements in stroke management after TACTICS VR training (post-training mean ± SD = 4.1 ± 0.6; pre-training = 3.6 ± 0.9; 1 = strongly disagree, 5 = strongly agree). Very few technical issues were identified, supporting the feasibility of this training approach. Thus, we propose that TACTICS VR is a fit-for-purpose, evidence-based training application for stroke workflow optimisation that can be readily deployed on-site in a clinical setting.

Validation of virtual reality arthroscopy simulator relevance in characterising experienced surgeons

BACKGROUND

Virtual reality (VR) simulation is particularly suitable for learning arthroscopy skills. Despite significant research, one drawback often outlined is the difficulty in distinguishing performance levels (Construct Validity) in experienced surgeons. Therefore, it seems adequate to search new methods of performance measurements using probe trajectories instead of commonly used metrics.

HYPOTHESIS

It was hypothesized that a larger experience in surgical shoulder arthroscopy would be correlated with better performance on a VR shoulder arthroscopy simulator and that experienced operators would share similar probe trajectories.

MATERIALS & METHODS

After answering to standardized questionnaires, 104 trajectories from 52 surgeons divided into 2 cohorts (26 intermediates and 26 experts) were recorded on a shoulder arthroscopy simulator. The procedure analysed was the "loose body removal" in a right shoulder joint. 10 metrics were computed on the trajectories including procedure duration, overall path length, economy of motion and smoothness. Additionally, Dynamic Time Warping (DTW) was computed on the trajectories for unsupervised hierarchical clustering of the surgeons.

RESULTS

Experts were significantly faster (Median 70.9s Interquartile range [56.4-86.3] vs. 116.1s [82.8-154.2], p<0.01), more fluid (4.6.10⁵mm.s^-3 [3.1.10⁵-7.2.10⁵] vs. 1.5.10⁶mm.s^-3 [2.6.10⁶-3.5.10⁶], p=0.05), and economical in their motion (19.3mm² [9.1-25.9] vs. 33.8mm² [14.8-50.5], p<0.01), but there was no significant difference in performance for path length (671.4mm [503.8-846.1] vs 694.6mm [467.0-1090.1], p=0.62). The DTW clustering differentiates two expertise related groups of trajectories with performance similarities, respectively including 48 expert trajectories for the first group and 52 intermediates and 4 expert trajectories for the second group (Sensitivity of 92%, Specificity of 100%). Hierarchical clustering with DTW significantly identified expert operators from intermediate operators and found trajectory similarities among 24/26 experts.

CONCLUSION

This study demonstrated the Construct Validity of the VR shoulder arthroscopy simulator within groups of experienced surgeons. With new types of metrics simply based on the simulator's raw trajectories, it was possible to significantly distinguish levels of expertise. We demonstrated that clustering analysis with Dynamic Time Warping was able to reliably discriminate between expert operators and intermediate operators.

CLINICAL RELEVANCE

The results have implications for the future of arthroscopic surgical training or post-graduate accreditation programs using virtual reality simulation.

LEVEL OF EVIDENCE

III; prospective comparative study.

The FAST Workstation Shows Construct Validity and Participant Endorsement

Purpose

To determine in what way the proposed simulation-based intervention (SBI) is an effective intervention for use in basic arthroscopic skills training.

Methods

Twenty candidates were recruited and grouped according to experience. Performance metrics included the time to activity completion, errors made, and Global Rating Scale score. Qualitative data were collected using a structured questionnaire.

Results

Performance on the SBI differed depending on previous arthroscopic training received. Performance on the simulator differed between groups to a statistically significant level regarding time to completion. A difference was also present between participants with no previous training and those with previous training when assessed using the Global Rating Scale. The SBI was deemed acceptable, user-friendly, and realistic. Participants practicing at the expert level believe that such an SBI would be beneficial in developing basic arthroscopic skills.

Conclusions

The results of this study provide evidence that the use of an SBI consisting of a benchtop workstation, laptop viewing platform, 30° arthroscope, and defined performance metrics can detect differences in the level of arthroscopic experience. This format of SBI has been deemed acceptable and useful to the intended user, increasing the feasibility of introducing it into surgical training.

Clinical Relevance

This study adds to the existing body of evidence supporting the potential benefits of benchtop SBIs in arthroscopic skills training. Improved performance on such an SBI may be beneficial for the purpose of basic arthroscopic skills training, and we would support the inclusion of this system in surgical training programs such as those developed by the Arthroscopy Association of North America and American Board of Orthopaedic Surgery.

Analysis of Tools Used in Assessing Technical Skills and Operative Competence in Trauma and Orthopaedic Surgical Training: A Systematic Review

BACKGROUND

Robust assessment of skills acquisition and surgical performance during training is vital to ensuring operative competence among orthopaedic surgeons. A move to competency-based surgical training requires the use of tools that can assess surgical skills objectively and systematically. The aim of this systematic review was to describe the evidence for the utility of assessment tools used in evaluating operative performance in trauma and orthopaedic surgical training.

METHODS

We performed a comprehensive literature search of MEDLINE, Embase, and Google Scholar databases to June 2019. From eligible studies we abstracted data on study aim, assessment format (live theater or simulated setting), skills assessed, and tools or metrics used to assess surgical performance. The strengths, limitations, and psychometric properties of the assessment tools are reported on the basis of previously defined utility criteria.

RESULTS

One hundred and five studies published between 1990 and 2019 were included. Forty-two studies involved open orthopaedic surgical procedures, and 63 involved arthroscopy. The majority (85%) were used in the simulated environment. There was wide variation in the type of assessment tools in used, the strengths and weaknesses of which are assessor and setting-dependent.

CONCLUSIONS

Current technical skills-assessment tools in trauma and orthopaedic surgery are largely procedure-specific and limited to research use in the simulated environment. An objective technical skills-assessment tool that is suitable for use in the live operative theater requires development and validation, to ensure proper competency-based assessment of surgical performance and readiness for unsupervised clinical practice.

CLINICAL RELEVANCE

Trainers and trainees can gain further insight into the technical skills assessment tools that they use in practice through the utility evidence provided.

Training Surgeons to Perform Arthroscopic All-Inside Meniscal Repair: A Randomized Controlled Trial Evaluating the Effectiveness of a Novel Cognitive Task Analysis Teaching Tool, Imperial College London/University College London Meniscus Repair Cognitive Task Analysis (IUMeRCTA)

BACKGROUND

All-inside meniscal repair is an increasingly common technique for the surgical treatment of meniscal tears. There are currently no standardized techniques for training residents in this procedure. Cognitive task analysis (CTA) is a method of analyzing and standardizing key steps in a procedure that allows training to be conducted in a validated and reproducible manner.

PURPOSE

(1) To design a digital CTA teaching tool for a standardized all-inside meniscal repair. (2) To evaluate whether CTA-trained residents would perform better in a meniscal repair task compared with a control group who underwent traditional apprenticeship methods of training.

STUDY DESIGN

Controlled laboratory study.

METHODS

Three expert knee surgeons were interviewed using a modified Delphi method to generate a consensus among the ideal technical steps, cognitive decision points, and common errors and solutions for an all-inside meniscal repair. This written information was then combined with visual and audio components and integrated onto a digital platform to create the Imperial College London/University College London Meniscus Repair Cognitive Task Analysis (IUMeRCTA) tool. Eighteen novice residents were randomized into an intervention group (digital CTA tool) and control group (equipment instruction manual). Both groups performed an all-inside meniscal repair on high-fidelity, phantom knee models and were assessed by expert surgeons, blinded to the interventions, using a validated global rating scale (GRS). After a power calculation, median GRS scores were compared between groups using the Mann-Whitney U test; significance was set at P < .05.

RESULTS

For the IUMeRCTA tool design, the procedure was divided into 55 steps across 9 phases: (1) preoperative planning, (2) theater and patient setup, (3) portal placement, (4) meniscal examination, (5) tear reduction, (6) suture planning, (7) suture insertion, (8) repair completion, and (9) postoperative care and rehabilitation. For the trial, the intervention group (mean ± SD GRS, 32 ± 2.9) performed significantly better than did the control group (GRS, 24 ± 3.3; P < .001).

CONCLUSION

This is the first CTA tool to demonstrate objective benefits in training novices to perform an arthroscopic all-inside meniscal repair.

CLINICAL RELEVANCE

The IUMeRCTA tool is an easily accessible and effective adjunct to traditional teaching that enhances learning the all-inside meniscal repair for novice surgeons.

Module-Based Arthroscopic Knee Simulator Training Improves Technical Skills in Naive Learners: A Randomized Trial

Purpose

To compare the effectiveness, in comparison to a control group (C), of module-based training (MBT) and traditional learning (TL) as a means of acquiring arthroscopic skills on an arthroscopic surgery simulator.

Methods

Thirty health sciences students with no previous arthroscopy experience were recruited and randomized into 1 of 3 groups: MBT, TL, or C (1:1:1 ratio). Participants in MBT were required to independently practice on a VirtaMed ArthroS simulator (VirtaMed AG, Zurich, Switzerland) for a minimum of 2 hours per week, whereas TL received one-on-one coaching by a senior orthopaedic resident for 15 minutes per week. The control group received no training. All groups were assessed at baseline and at 4 weeks based on objective measures generated by the surgical simulator (procedure time, camera path length, meniscus cutting score, detailed visualization, safety score and total score), and subjective ratings scales (Objective Assessment of Arthroscopic Skill [OAAS] global assessment form, and Competency-Based Assessment form).

Results

Participants in the MBT group trained on average 113 min/week whereas the TL group trained on average 24 min/week. Three-way repeated-measures analysis of variance showed significant group by time interactions for procedure time (P = .006), camera path length (P = .008), safety score (P = .013), total score (P = .003), OAAS form (P < .001), and Competency-Based Assessment form (P < .001). MBT group was superior to C group for procedure time (P = .02), camera path length (P = .003), total score (P = .004), and OAAS form (P = .021), but there were no significant post-hoc differences between MBT and TL groups, or TL and C groups after Bonferroni correction. Total practice time explained 37.5% of the final simulator total score variance.

Conclusions

Knee arthroscopy simulation training with self-learning modules can improve skills in areas such as procedure time, camera path length, and total score in untrained participants compared with a control group.

Clinical Relevance

Module-based simulation training provides additional training time and improves technical skills in naive health science students. It is hoped that this effect can be preserved and applied to junior resident developing in a busy residency program.

Arthroscopic Simulation: The Future of Surgical Training: A Systematic Review

BACKGROUND

Arthroscopic simulation has rapidly evolved recently with the introduction of higher-fidelity simulation models, such as virtual reality simulators, which provide trainees an environment to practice skills without causing undue harm to patients. Simulation training also offers a uniform approach to learn surgical skills with immediate feedback. The aim of this article is to review the recent research investigating the use of arthroscopy simulators in training and the teaching of surgical skills.

METHODS

A systematic review of the Embase, MEDLINE, and Cochrane Library databases for English-language articles published before December 2019 was conducted. The search terms included arthroscopy or arthroscopic in combination with simulation or simulator.

RESULTS

We identified a total of 44 relevant studies involving benchtop or virtually simulated ankle, knee, shoulder, and hip arthroscopy environments. The majority of these studies demonstrated construct and transfer validity; considerably fewer studies demonstrated content and face validity.

CONCLUSIONS

Our review indicates that there is a considerable evidence base regarding the use of arthroscopy simulators for training purposes. Further work should focus on the development of a more uniform simulator training course that can be compared with current intraoperative training in large-scale trials with long-term follow-up at tertiary centers.

Active vs passive haptic feedback technology in virtual reality arthroscopy simulation: Which is most realistic?

BACKGROUND

Virtual Reality (VR) simulators are playing an increasingly prominent role in orthopaedic training and education. Face-validity - the degree to which reality is accurately represented - underpins the value of a VR simulator as a learning tool for trainees. Despite the importance of tactile feedback in arthroscopy, there is a paucity for evidence regarding the role of haptics in VR arthroscopy simulator realism.

PURPOSE

To assess the difference in face validity between two high fidelity VR simulators employing passive and active haptic feedback technology respectively.

METHOD

38 participants were recruited and divided into intermediate and expert groups based on orthopaedic training grade. Each participant completed a 12-point diagnostic knee arthroscopy VR module using the active haptic Simbionix ARTHRO Mentor and passive haptic VirtaMed ArthroS simulators. Subsequently, each participant completed a validated simulator face validity questionnaire.

RESULTS

The ARTHRO Mentor active haptic system failed to achieve face validity with mean scores for external appearance (6.61), intra-articular appearance (4.78) and instrumentation (4.36) falling below the acceptable threshold (≥7.0). The ArthroS passive haptic simulator demonstrated satisfactory scores in all domains: external appearance (8.42), intra-articular appearance (7.65), instrumentation (7.21) and was significantly (p < 0.001) more realistic than ARTHRO Mentor for all metrics. 61% of participants gave scores ≥7.0 for questions pertaining to haptic feedback realism from intra-articular structures such as menisci and ACL/PCL for the ArthroS vs. 12% for ARTHRO Mentor. There was no difference in face-validity perception between intermediate and expert groups for either simulator (p > 0.05).

CONCLUSION

Current active haptic technology which employs motors to simulate tactile feedback fails to demonstrate sufficient face-validity or match the sophistication of passive haptic systems in high fidelity arthroscopy simulators. Textured rubber phantoms that mirror the anatomy and haptic properties of the knee joint provide a significantly more realistic training experience for both intermediate and expert arthroscopists.

Virtual reality simulation-the future of orthopaedic training? A systematic review and narrative analysis

Background

Virtual reality (VR) simulation provides users with an immersive, 3D experience that can be used to allow surgical trainees to practice skills and operations in a safe yet realistic environment. The field of orthopaedics is yet to include VR in core teaching, despite its advantages as a teaching aid, particularly against current simulation tools. This study aims to conduct a systematic review to investigate the efficacy of VR in orthopaedic training, against current methods.

Methods

A systemic review of databases Medline, Embase and the Cochrane Library for randomized controlled trials focusing on VR training against conventional training in orthopaedic surgery was performed. Data synthesis was performed through narrative analysis due to the heterogeneous nature of the data.

Results

A total of 16 studies from 140 titles were identified, across 6 specialty areas. Four hundred and thirty-one participants were included. Control groups included VR, cadaver and benchtop simulators. Forty-seven outcomes were measured, focusing on skill and proficiency assessment. No outcomes focused on patient safety. Although significance between intervention and control was not always achieved, most studies found that the intervention outperformed the control.

Conclusion

VR provides a modern and immersive teaching tool that can develop skills and give confidence to trainees. This study demonstrates the potential for VR simulation as a training aid in orthopaedics and encourages its use alongside conventional teaching methods. However, long-term analysis of the results of VR training on surgical trainees has yet to be conducted. To provide conclusive justification for its inclusion in surgical training, this study recommends that future research follows trainees using VR into the operating room, to determine that VR teaches skills that are transferable onto actual surgeries, subsequently leading to better patient outcomes.

Simulation-Based Arthroscopic Skills Using a Spaced Retraining Schedule Reduces Short-Term Task Completion Time and Camera Path Length

PURPOSE

To investigate whether acquiring basic knee arthroscopic skills via a spaced retraining schedule could prevent skills deterioration and achieve further skills improvement.

METHODS

In the learning phase, 16 residents with no previous hands-on experience in practicing arthroscopic skills were asked to perform basic arthroscopic tasks on a simulator until they attained perfect scores in each task. Immediately after completing the learning phase, a pretest was performed to assess their performance. Next, they were randomly assigned into 2 groups. The spaced retraining group, which undertook a spaced repetitive training phase with a fixed-time interval, returned on days 2, 4 and 6 to repeat the same tasks for 20 minutes per day, whereas the control group did nothing. On day 7, all participants performed a posttest. A 2 × 2 mixed analysis of variance model was used for statistical analysis.

RESULTS

Significant differences between the 2 groups were found in task completion time (P = .003) and camera path length (P = .043) but not cartilage injury (P = .186). Residents in the spaced retraining group decreased their task completion time (163.2 ± 23.9 seconds) whereas the task time in the control group increased (351.3 ± 25.5 seconds). The same pattern was found with the camera path length.

CONCLUSIONS

Implementing a spaced retraining schedule in 1 week resulted in a reduced task completion time and camera path length but no significant reduction in cartilage injury. It appears that introducing a spaced retraining schedule to retain arthroscopic skills acquired through massed learning may be advantageous.

CLINICAL RELEVANCE

In consideration of the training time available to residents and the trend toward massed learning, this spaced retraining schedule may offer a cost-effective and convenient way for residents to maintain and improve their basic arthroscopic skills with no significant increase in time invested.

Editorial Commentary: Perfecting Practice: Can We Train Residents to Be Proficient Arthroscopists Before Actually Treating Patients?

Although a spaced retraining schedule improved resident arthroscopic task completion time and camera path length on a virtual simulator, it did not improve the degree of cartilage injury. To quote former Green Bay Packer Head Coach Vince Lombardi: "Practice does not make perfect. Only perfect practice makes perfect."

Global Rating Scales for the Assessment of Arthroscopic Surgical Skills: A Systematic Review

PURPOSE

To evaluate whether sufficient validity and reliability evidence exists to support the use of global rating scales (GRS) as evaluation tools in both formative assessment and competency assessment of arthroscopic procedures.

METHODS

A search of PubMed, Embase, and Scopus was conducted for articles published between 1990 and 2018. Studies reporting measures of validity and reliability of GRS relating to arthroscopic skills were included. Procedural checklists and other assessment tools were excluded.

RESULTS

A total of 39 articles met the inclusion criteria. In total, 7 de novo GRS specific for arthroscopic education and 3 pre-existing GRS repurposed 4 times for arthroscopic education were identified in the literature. The 11 GRS were used to assess 1175 surgeons 3890 times. Three GRS tools explicitly defined an arbitrary minimum competency threshold, 6 of 11 tools demonstrated construct validity-the ability to significantly discriminate between groups of differing experience-and 5 of 11 tools assessed inter-rater reliability, but only the Arthroscopic Surgical Skills Evaluation Tool demonstrated excellent inter-rater reliability. The Arthroscopic Surgical Skills Evaluation Tool was validated by 16 articles for a total of 537 surgeons for hip, knee, shoulder, and ankle arthroscopy in both simulated and clinical environments but was found to be invalid in wrist arthroscopy. The Basic Arthroscopic Knee Skill Scoring System was validated by 15 articles for a total of 497 surgeons for knee, hip, and shoulder in both clinical and simulated environments. The remaining 9 GRS were validated by 2 or fewer studies.

CONCLUSIONS

Overall, GRS have contributed to training, feedback, and formative assessment practices. The GRS reviewed demonstrate both construct and concurrent validity as well as reliability in multiple arthroscopic procedures in multiple joints. Currently, there is sufficient evidence to use GRS as a feedback tool. However, there is insufficient evidence for its use in high-stakes examinations or as a minimum competency assessment.

LEVEL OF EVIDENCE

Level III, systematic review of level I to III studies.

A Framework for the Testing and Validation of Simulated Environments in Experimentation and Training

New computer technologies, like virtual reality (VR), have created opportunities to study human behavior and train skills in novel ways. VR holds significant promise for maximizing the efficiency and effectiveness of skill learning in a variety of settings (e.g., sport, medicine, safety-critical industries) through immersive learning and augmentation of existing training methods. In many cases the adoption of VR for training has, however, preceded rigorous testing and validation of the simulation tool. In order for VR to be implemented successfully for both training and psychological experimentation it is necessary to first establish whether the simulation captures fundamental features of the real task and environment, and elicits realistic behaviors. Unfortunately evaluation of VR environments too often confuses presentation and function, and relies on superficial visual features that are not the key determinants of successful training outcomes. Therefore evidence-based methods of establishing the fidelity and validity of VR environments are required. To this end, we outline a taxonomy of the subtypes of fidelity and validity, and propose a variety of practical methods for testing and validating VR training simulations. Ultimately, a successful VR environment is one that enables transfer of learning to the real-world. We propose that key elements of psychological, affective and ergonomic fidelity, are the real determinants of successful transfer. By adopting an evidence-based approach to VR simulation design and testing it is possible to develop valid environments that allow the potential of VR training to be maximized.

Does Arthroscopic Simulation Training Improve Triangulation and Probing Skills? A Randomized Controlled Trial✰

OBJECTIVE

To determine the effectiveness of simulator training on basic arthroscopic skills utilizing a novel, low-cost arthroscopic triangulation training system.

DESIGN

A randomized controlled trial of subjects without prior arthroscopy training was conducted, with participants randomized to receive either a fixed protocol of simulation training on a triangulation simulation model (30 minutes of training for 4 consecutive days), or no training. On Days 1 and 5, all participants were evaluated on 3 simulated arthroscopic tasks by an independent observer. Variables analyzed included how many times portals were changed, the time it took to complete the tasks, and the task completion rate.

SETTING

Arthrex Inc., Naples, FL.

PARTICIPANTS

Thirty-six participants (92% male, average 28 ± 5 years) with no prior arthroscopy training were randomized into 2 groups, with 17 in the training group (T) and 19 in the no-training group (NT).

RESULTS

On Day 1, there was no difference in rate of task completion between the T group and NT groups (41% versus 53%, p = 0.52). On Day 5, significantly more participants in the T group completed all tasks compared to the NT group (100% versus 63%, p = 0.008). Participants in the T group had significantly improved task completion times on Day 5 versus Day 1 (p < 0.05). Participants in the NT group had a significantly improved task completion time for Task 1 on Day 5 versus Day 1 (p = 0.037); no differences were found for Tasks 2 or 3. On Day 5, participants in the T group required significantly fewer portal changes compared to the NT group (2.35 ± 2.29 versus 6.95 ± 8.55, p = 0.039).

CONCLUSIONS

Simulation training on a simple, low-cost arthroscopic triangulation training system resulted in an overall improvement in arthroscopic probing and triangulation skills within 1 week of training, with significantly decreased task completion times and increased efficiency of movement.

Is actual surgical experience reflected in virtual reality simulation surgery for a femoral neck fracture?

INTRODUCTION

A virtual reality simulator developed for orthopaedic and trauma surgical training has been introduced. However, it is unclear whether the experiences of actual surgery are reflected in virtual reality simulation surgery (VRSS) using a simulator. The aim of this study is to investigate whether the results in VRSS differ between a trauma expert and a trauma novice.

METHODS

In Group A (expert), there are ten orthopaedic trauma surgeons and in Group B (novice) ten residents within 2 years after medical school graduation. VRSS for a femoral neck fracture using Hansson hook-pins (Test 1) and Hansson twin hook plate (Test 2) was performed. The parameters evaluated were total procedure time (s), fluoroscopy time (s), number of times X-ray was used (defined by the number of times the foot pedal was used), number of retries in guide placement, and final implant position.

RESULTS

In Test 1, the averages of four parameters (distance to posterior cortex (p = 0.009), distal pin distance above lesser trochanter (p = 0.015), distal pin hook angular error (p = 0.004), and distal pin tip distance to centre (lateral) (p = 0.015)) were significantly different between Groups A and B. In Test 2, no parameters in a mean were significantly different between groups, but seven parameters in a variance (guide wire distance to joint surface (p = 0.0191), twin hook length outside barrel (p = 0.011), twin hook tip distance to centre (lateral) (p = 0.042), twin hook distance to centre of lateral cortex (lateral) (p = 0.016), plate end alignment error (lateral) (p = 0.027), guide wire angle with lateral cortex (front) (p = 0.024), and 3.2-mm drill outside cortex (p = 0.000)) were significantly different between groups. In Test 1, Group B showed significantly longer fluoroscopy time than Group A (p = 0.044). In Test 2, Group B showed significantly fewer instances of X-ray use than Group A (p = 0.046).

CONCLUSIONS

Our study showed that the experiences of actual surgery are reflected in the result of VRSS using the simulator.

Can an Augmented Reality Headset Improve Accuracy of Acetabular Cup Orientation in Simulated THA? A Randomized Trial

BACKGROUND

Accurate implant orientation reduces wear and increases stability in arthroplasty but is a technically demanding skill. Augmented reality (AR) headsets overlay digital information on top of the real world. We have developed an enhanced AR headset capable of tracking bony anatomy in relation to an implant, but it has not yet been assessed for its suitability as a training tool for implant orientation.

QUESTIONS/PURPOSES

(1) In the setting of simulated THA performed by novices, does an AR headset improve the accuracy of acetabular component positioning compared with hands-on training by an expert surgeon? (2) What are trainees' perceptions of the AR headset in terms of realism of the task, acceptability of the technology, and its potential role for surgical training?

METHODS

Twenty-four study participants (medical students in their final year of school, who were applying to surgery residency programs, and who had no prior arthroplasty experience) participated in a randomized simulation trial using an AR headset and a simulated THA. Participants were randomized to two groups completing four once-weekly sessions of baseline assessment, training, and reassessment. One group trained using AR (with live holographic orientation feedback) and the other received one-on-one training from a hip arthroplasty surgeon. Demographics and baseline performance in orienting an acetabular implant to six patient-specific values on the phantom pelvis were collected before training and were comparable. The orientation error in degrees between the planned and achieved orientations was measured and was not different between groups with the numbers available (surgeon group mean error ± SD 16° ± 7° versus AR 14° ± 7°; p = 0.22). Participants trained by AR also completed a validated posttraining questionnaire evaluating their experiences.

RESULTS

During the four training sessions, participants using AR-guidance had smaller mean (± SD) errors in orientation than those receiving guidance from the surgeon: 1° ± 1° versus AR 6° ± 4°, p < 0.001. In the fourth session's assessment, participants in both groups had improved (surgeon group mean improvement 6°, 95% CI, 4-8°; p < 0.001 versus AR group 9°, 95% CI 7-10°; p < 0.001). There was no difference between participants in the surgeon-trained and AR-trained group: mean difference 1.2°, 95% CI, -1.8 to 4.2°; p = 0.281. In posttraining evaluation, 11 of 12 participants would use the AR platform as a training tool for developing visuospatial skills and 10 of 12 for procedure-specific rehearsals. Most participants (11 of 12) stated that a combination of an expert trainer for learning and AR for unsupervised training would be preferred.

CONCLUSIONS

A novel head-mounted AR platform tracked an implant in relation to bony anatomy to a clinically relevant level of accuracy during simulated THA. Learners were equally accurate, whether trained by AR or a surgeon. The platform enabled the use of real instruments and gave live feedback; AR was thus considered a feasible and valuable training tool as an adjunct to expert guidance in the operating room. Although there were no differences in accuracy between the groups trained using AR and those trained by an expert surgeon, we believe the tool may be useful in education because it demonstrates that some motor skills for arthroplasty may be learned in an unsupervised setting. Future studies will evaluate AR-training for arthroplasty skills other than cup orientation and its transfer validity to real surgery.

LEVEL OF EVIDENCE

Level I, therapeutic study.

Validation of a Virtual Reality-Based Hip Arthroscopy Simulator

PURPOSE

To assess construct and face validity of a novel virtual reality-based hip arthroscopy simulator using the previously validated Arthroscopic Surgery Skills Evaluation Tool (ASSET), metric parameters, and a questionnaire.

METHODS

Metric parameters including task completion time, camera path, and grasper path were recorded, and the ASSET score was used to assess construct validity. Face validity was evaluated using a questionnaire.

RESULTS

Nine hip arthroscopy experts, of whom the majority performed more than 200 procedures (age, 48 ± 7.3; range, 38-61 years; 8 men, 1 woman), and 33 nonexperts (age, 33 ± 7.9; range, 26-62 years; 25 men, 8 women) performed 3 individual tasks on a virtual reality-based arthroscopy simulator of a left hip. The ASSET global rating scale showed a statistically significant difference between the hip arthroscopy expert and the nonexpert group, indicating strong construct validity (25.0 in the expert group, range, 17-34, versus 15.30 in the nonexpert group, range, 8-30 [P < .001], respectively). This also applied to most metric parameters recorded by the simulator. The simulator also demonstrated high face validity. The overall impression in terms of realism was graded "completely realistic" by 17% and "close to realistic" by 62% of participants.

CONCLUSIONS

The tested simulator demonstrated high construct and face validity.

CLINICAL RELEVANCE

This study demonstrates the construct and face validity of a novel hip arthroscopy simulator. The device proved to be an adequate model for the simulation of some arthroscopic procedures of the hip.

Virtual reality hip arthroscopy simulator demonstrates sufficient face validity

PURPOSE

To test the face validity of the hip diagnostics module of a virtual reality hip arthroscopy simulator.

METHODS

A total of 25 orthopaedic surgeons, 7 faculty members and 18 orthopaedic residents, performed diagnostic supine hip arthroscopies of a healthy virtual reality hip joint using a 70° arthroscope. Twelve specific targets were visualised within the central compartment; six via the anterior portal, three via the anterolateral portal and three via the posterolateral portal. This task was immediately followed by a questionnaire regarding the realism and training capability of the system. This consisted of seven questions addressing the verisimilitude of the simulator and five questions addressing the training environment of the simulator. Each question consisted of a statement stem and 10-point Likert scale. Following similar work in surgical simulators, a rating of 7 or above was considered an acceptable level of realism.

RESULTS

The diagnostic hip arthroscopy module was found to have an acceptable level of realism in all domains apart from the tactile feedback received from the soft tissue. 23 out of 25 participants (92%) felt the simulator provided a non-threatening learning environment and 22 participants (88%) stated they enjoyed using the simulator. It was most frequently agreed that the level of trainees who would benefit most from the simulator were registrars and fellows (22 participants; 88%). Additionally, 21 of the participants (84%) agreed that this would be a beneficial training modality for foundation and core trainees, and 20 participants (80%) agreed that his would be beneficial for consultants.

CONCLUSIONS

This VR hip arthroscopy simulator was demonstrated to have a sufficient level of realism, thus establishing its face validity. These results suggest this simulator has sufficient realism for use in the acquisition of basic arthroscopic skills and supports its use in orthopaedics surgical training.

LEVEL OF EVIDENCE

I.

Energy-Based Metrics for Arthroscopic Skills Assessment

Minimally invasive skills assessment methods are essential in developing efficient surgical simulators and implementing consistent skills evaluation. Although numerous methods have been investigated in the literature, there is still a need to further improve the accuracy of surgical skills assessment. Energy expenditure can be an indication of motor skills proficiency. The goals of this study are to develop objective metrics based on energy expenditure, normalize these metrics, and investigate classifying trainees using these metrics. To this end, different forms of energy consisting of mechanical energy and work were considered and their values were divided by the related value of an ideal performance to develop normalized metrics. These metrics were used as inputs for various machine learning algorithms including support vector machines (SVM) and neural networks (NNs) for classification. The accuracy of the combination of the normalized energy-based metrics with these classifiers was evaluated through a leave-one-subject-out cross-validation. The proposed method was validated using 26 subjects at two experience levels (novices and experts) in three arthroscopic tasks. The results showed that there are statistically significant differences between novices and experts for almost all of the normalized energy-based metrics. The accuracy of classification using SVM and NN methods was between 70% and 95% for the various tasks. The results show that the normalized energy-based metrics and their combination with SVM and NN classifiers are capable of providing accurate classification of trainees. The assessment method proposed in this study can enhance surgical training by providing appropriate feedback to trainees about their level of expertise and can be used in the evaluation of proficiency.