51
|
Rahimov C, Aliyev D, Rahimov N, Farzaliyev I. Mixed reality in the reconstruction of orbital floor: An experimental and clinical evaluative study. Ann Maxillofac Surg 2022; 12:46-53. [PMID: 36199454 PMCID: PMC9527844 DOI: 10.4103/ams.ams_141_21] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/24/2021] [Revised: 06/22/2022] [Accepted: 07/21/2022] [Indexed: 11/04/2022] Open
Abstract
Introduction: Materials and Methods: Results: Discussion:
Collapse
|
52
|
Marker-free Surgical Navigation of Rod Bending using a Stereo Neural Network and Augmented Reality in Spinal Fusion. Med Image Anal 2022; 77:102365. [DOI: 10.1016/j.media.2022.102365] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/22/2021] [Revised: 11/16/2021] [Accepted: 01/10/2022] [Indexed: 11/20/2022]
|
53
|
Arnold MCA, Zhao S, Doyle RJ, Jeffers JRT, Boughton OR. Power-Tool Use in Orthopaedic Surgery: Iatrogenic Injury, Its Detection, and Technological Advances: A Systematic Review. JB JS Open Access 2021; 6:JBJSOA-D-21-00013. [PMID: 34841185 PMCID: PMC8613350 DOI: 10.2106/jbjs.oa.21.00013] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/04/2022] Open
Abstract
Power tools are an integral part of orthopaedic surgery but have the capacity to cause iatrogenic injury. With this systematic review, we aimed to investigate the prevalence of iatrogenic injury due to the use of power tools in orthopaedic surgery and to discuss the current methods that can be used to reduce injury.
Collapse
Affiliation(s)
| | - Sarah Zhao
- The MSk Lab, Imperial College London, London, United Kingdom
| | - Ruben J Doyle
- Department of Mechanical Engineering, Imperial College London, London, United Kingdom
| | - Jonathan R T Jeffers
- Department of Mechanical Engineering, Imperial College London, London, United Kingdom
| | | |
Collapse
|
54
|
HoloUS: Augmented reality visualization of live ultrasound images using HoloLens for ultrasound-guided procedures. Int J Comput Assist Radiol Surg 2021; 17:385-391. [PMID: 34817764 DOI: 10.1007/s11548-021-02526-7] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/18/2021] [Accepted: 10/20/2021] [Indexed: 10/19/2022]
Abstract
PURPOSE Microsoft HoloLens is a pair of augmented reality (AR) smart glasses that could improve the intraprocedural visualization of ultrasound-guided procedures. With the wearable HoloLens headset, an ultrasound image can be virtually rendered and registered with the ultrasound transducer and placed directly in the practitioner's field of view. METHODS A custom application, called HoloUS, was developed using the HoloLens and a portable ultrasound machine connected through a wireless network. A custom 3D-printed case with an AR-pattern for the ultrasound transducer permitted ultrasound image tracking and registration. Voice controls on the HoloLens supported the scaling and movement of the ultrasound image as desired. The ultrasound images were streamed and displayed in real-time. A user study was performed to assess the effectiveness of the HoloLens as an alternative display platform. Novices and experts were timed on tasks involving targeting simulated vessels using a needle in a custom phantom. RESULTS Technical characterization of the HoloUS app was conducted using frame rate, tracking accuracy, and latency as performance metrics. The app ran at 25 frames/s, had an 80-ms latency, and could track the transducer with an average reprojection error of 0.0435 pixels. With AR visualization, the novices' times improved by 17% but the experts' times decreased slightly by 5%, which may reflect the experts' training and experience bias. CONCLUSION The HoloUS application was found to enhance user experience and simplify hand-eye coordination. By eliminating the need to alternately observe the patient and the ultrasound images presented on a separate monitor, the proposed AR application has the potential to improve efficiency and effectiveness of ultrasound-guided procedures.
Collapse
|
55
|
Johnson PB, Jackson A, Saki M, Feldman E, Bradley J. Patient posture correction and alignment using mixed reality visualization and the HoloLens 2. Med Phys 2021; 49:15-22. [PMID: 34780068 DOI: 10.1002/mp.15349] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/18/2021] [Revised: 10/08/2021] [Accepted: 11/02/2021] [Indexed: 12/17/2022] Open
Abstract
PURPOSE The purpose of this study was to develop and preliminarily test a radiotherapy system for patient posture correction and alignment using mixed reality (MixR) visualization. The write-up of this work also provides an opportunity to introduce the concepts and technology of MixR for a medical physics audience who may be unfamiliar with the topic. METHODS A MixR application was developed for on optical-see-through head-mounted display (HoloLens 2) allowing a user to simultaneously and directly view a patient and a reference hologram derived from their simulation CT scan. The hologram provides a visual reference for the exact posture needed during treatment and is initialized in relation to the origin of a radiotherapy device using marker-based tracking. The system further provides marker-less tracking that allows the user tofreely navigate the room as they view and align the patient from various angles. The system was preliminarily tested using both a rigid (pelvis) and nonrigid (female mannequin) anthropomorphic phantom. Each phantom was aligned via hologram and accuracy quantified using CBCT and CT. RESULTS A fully realized system was developed. Rigid registration accuracy was on the order of 3.0 ± 1.5 mm based on the performance of three users repeating alignment five times each. The lateral direction showed the most variability among users and was associated with the largest off-sets (approximately 2.0 mm). For nonrigid alignment, the MixR setup outperformed a setup based on three-point alignment and setup photos, the latter of which showed a difference in arm position of 2 cm and a torso roll of 6-7°. CONCLUSIONS MixR visualization is a rapidly emerging domain that has the potential to significantly impact the field of medicine. The current application is an illustration of this and highlights the advantages of MixR for patient setup in radiation oncology. The key feature of the system is the way in which it transforms nonrigid registration into rigid registration by providing an efficient, portable, and cost-effective mechanism for reproducing patient posture without the use of ionizing radiation. Preliminary estimates of registration accuracy indicate clinical viability and form the foundation for further development and clinical testing.
Collapse
Affiliation(s)
- Perry B Johnson
- Department of Radiation Oncology, University of Florida College of Medicine, Gainesville, Florida, USA.,University of Florida Health Proton Therapy Institute, Jacksonville, Florida, USA
| | - Amanda Jackson
- Department of Radiology, University of Florida College of Medicine, Gainesville, Florida, USA
| | - Mohammad Saki
- University of Florida Health Proton Therapy Institute, Jacksonville, Florida, USA
| | - Emily Feldman
- University of Florida Health Proton Therapy Institute, Jacksonville, Florida, USA
| | - Julie Bradley
- Department of Radiation Oncology, University of Florida College of Medicine, Gainesville, Florida, USA.,University of Florida Health Proton Therapy Institute, Jacksonville, Florida, USA
| |
Collapse
|
56
|
Pan J, Yu D, Li R, Huang X, Wang X, Zheng W, Zhu B, Liu X. Multi-Modality guidance based surgical navigation for percutaneous endoscopic transforaminal discectomy. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2021; 212:106460. [PMID: 34736173 DOI: 10.1016/j.cmpb.2021.106460] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/18/2021] [Accepted: 10/06/2021] [Indexed: 06/13/2023]
Abstract
OBJECTIVE Fluoroscopic guidance is a critical step for the puncture procedure in percutaneous endoscopic transforaminal discectomy (PETD). However, two-dimensional observations of the three-dimensional anatomic structure suffer from the effects of projective simplification. To accurately assess the spatial relations between the patient vertebra tissues and puncture needle, a considerable number of fluoroscopic images from different orientations need to be acquired by the surgeons. This process significantly increases the radiation risk for both the patient and surgeons. METHODS In this paper, we propose an augmented reality (AR) surgical navigation system for PETD based on multi-modality information, which contains fluoroscopy, optical tracking, and depth camera. To register the fluoroscopic image with the intraoperative video, we design a lightweight non-invasive fiducial with markers and detect the markers based on the deep learning method. It can display the intraoperative video fused with the registered fluoroscopic images. We also present a self-adaptive calibration and transformation method between a 6-DOF optical tracking device and a depth camera, which are in different coordinate systems. RESULTS With the substantially reduced frequency of fluoroscopy imaging, the system can accurately track and superimpose the virtual puncture needle on fluoroscopy images in real-time. From operating theatre in vivo animal experiments, the results illustrate that the system average positioning accuracy can reach 1.98mm and the orientation accuracy can reach 1.19∘. From the clinical validation results, the system significantly lower the frequency of fluoroscopy imaging (42.7%) and reduce the radiation risk for both the patient and surgeons. CONCLUSION Coupled with the user study, both the quantitative and qualitative results indicate that our navigation system has the potential to be highly useful in clinical practice. Compared with the existing navigation systems, which are usually equipped with a variety of large and high-cost medical equipments, such as O-arm, cone-beam CT, and robots, our navigation system does not need special equipment and can be implemented with common equipment in the operating room, such as C-arm, desktop, etc., even in small hospitals.
Collapse
Affiliation(s)
- Junjun Pan
- State Key Laboratory of Virtual Reality Technology and Systems, Beijing Advanced Innovation Center for Biomedical Engineering, Beihang University, Beijing 100191, China; PENG CHENG Laboratory, Shenzhen 518000, China.
| | - Dongfang Yu
- State Key Laboratory of Virtual Reality Technology and Systems, Beijing Advanced Innovation Center for Biomedical Engineering, Beihang University, Beijing 100191, China
| | - Ranyang Li
- State Key Laboratory of Virtual Reality Technology and Systems, Beijing Advanced Innovation Center for Biomedical Engineering, Beihang University, Beijing 100191, China; PENG CHENG Laboratory, Shenzhen 518000, China.
| | - Xin Huang
- The Pain Medicine Center, Peking University Third Hospital, Beijing, China
| | - Xinliang Wang
- State Key Laboratory of Virtual Reality Technology and Systems, Beijing Advanced Innovation Center for Biomedical Engineering, Beihang University, Beijing 100191, China
| | - Wenhao Zheng
- State Key Laboratory of Virtual Reality Technology and Systems, Beijing Advanced Innovation Center for Biomedical Engineering, Beihang University, Beijing 100191, China
| | - Bin Zhu
- The Pain Medicine Center, Peking University Third Hospital, Beijing, China
| | - Xiaoguang Liu
- The Pain Medicine Center, Peking University Third Hospital, Beijing, China
| |
Collapse
|
57
|
Hersh A, Mahapatra S, Weber-Levine C, Awosika T, Theodore JN, Zakaria HM, Liu A, Witham TF, Theodore N. Augmented Reality in Spine Surgery: A Narrative Review. HSS J 2021; 17:351-358. [PMID: 34539277 PMCID: PMC8436352 DOI: 10.1177/15563316211028595] [Citation(s) in RCA: 12] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
Abstract
Augmented reality (AR) navigation refers to novel technologies that superimpose images, such as radiographs and navigation pathways, onto a view of the operative field. The development of AR navigation has focused on improving the safety and efficacy of neurosurgical and orthopedic procedures. In this review, the authors focus on 3 types of AR technology used in spine surgery: AR surgical navigation, microscope-mediated heads-up display, and AR head-mounted displays. Microscope AR and head-mounted displays offer the advantage of reducing attention shift and line-of-sight interruptions inherent in traditional navigation systems. With the U.S. Food and Drug Administration's recent clearance of the XVision AR system (Augmedics, Arlington Heights, IL), the adoption and refinement of AR technology by spine surgeons will only accelerate.
Collapse
Affiliation(s)
- Andrew Hersh
- Department of Neurosurgery, Johns Hopkins University School of Medicine, Baltimore, MD, USA
| | - Smruti Mahapatra
- Department of Neurosurgery, Johns Hopkins University School of Medicine, Baltimore, MD, USA
| | - Carly Weber-Levine
- Department of Neurosurgery, Johns Hopkins University School of Medicine, Baltimore, MD, USA
| | - Tolulope Awosika
- Department of Neurosurgery, Johns Hopkins University School of Medicine, Baltimore, MD, USA
| | | | - Hesham M Zakaria
- Department of Neurosurgery, Johns Hopkins University School of Medicine, Baltimore, MD, USA
| | - Ann Liu
- Department of Neurosurgery, Johns Hopkins University School of Medicine, Baltimore, MD, USA
| | - Timothy F Witham
- Department of Neurosurgery, Johns Hopkins University School of Medicine, Baltimore, MD, USA
| | - Nicholas Theodore
- Department of Neurosurgery, Johns Hopkins University School of Medicine, Baltimore, MD, USA
| |
Collapse
|
58
|
Usability of Graphical Visualizations on a Tool-Mounted Interface for Spine Surgery. J Imaging 2021; 7:jimaging7080159. [PMID: 34460795 PMCID: PMC8404910 DOI: 10.3390/jimaging7080159] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/07/2021] [Revised: 07/29/2021] [Accepted: 08/17/2021] [Indexed: 12/01/2022] Open
Abstract
Screw placement in the correct angular trajectory is one of the most intricate tasks during spinal fusion surgery. Due to the crucial role of pedicle screw placement for the outcome of the operation, spinal navigation has been introduced into the clinical routine. Despite its positive effects on the precision and safety of the surgical procedure, local separation of the navigation information and the surgical site, combined with intricate visualizations, limit the benefits of the navigation systems. Instead of a tech-driven design, a focus on usability is required in new research approaches to enable advanced and effective visualizations. This work presents a new tool-mounted interface (TMI) for pedicle screw placement. By fixing a TMI onto the surgical instrument, physical de-coupling of the anatomical target and navigation information is resolved. A total of 18 surgeons participated in a usability study comparing the TMI to the state-of-the-art visualization on an external screen. With the usage of the TMI, significant improvements in system usability (Kruskal–Wallis test p < 0.05) were achieved. A significant reduction in mental demand and overall cognitive load, measured using a NASA-TLX (p < 0.05), were observed. Moreover, a general improvement in performance was shown by means of the surgical task time (one-way ANOVA p < 0.001).
Collapse
|
59
|
Review of Microsoft HoloLens Applications over the Past Five Years. APPLIED SCIENCES-BASEL 2021. [DOI: 10.3390/app11167259] [Citation(s) in RCA: 10] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/07/2023]
Abstract
Since Microsoft HoloLens first appeared in 2016, HoloLens has been used in various industries, over the past five years. This study aims to review academic papers on the applications of HoloLens in several industries. A review was performed to summarize the results of 44 papers (dated between January 2016 and December 2020) and to outline the research trends of applying HoloLens to different industries. This study determined that HoloLens is employed in medical and surgical aids and systems, medical education and simulation, industrial engineering, architecture, civil engineering and other engineering fields. The findings of this study contribute towards classifying the current uses of HoloLens in various industries and identifying the types of visualization techniques and functions.
Collapse
|
60
|
Augmented reality-navigated pedicle screw placement: a cadaveric pilot study. EUROPEAN SPINE JOURNAL : OFFICIAL PUBLICATION OF THE EUROPEAN SPINE SOCIETY, THE EUROPEAN SPINAL DEFORMITY SOCIETY, AND THE EUROPEAN SECTION OF THE CERVICAL SPINE RESEARCH SOCIETY 2021; 30:3731-3737. [PMID: 34350487 DOI: 10.1007/s00586-021-06950-w] [Citation(s) in RCA: 12] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/08/2020] [Revised: 11/04/2020] [Accepted: 07/25/2021] [Indexed: 10/20/2022]
Abstract
PURPOSE Augmented reality (AR) is an emerging technology with great potential for surgical navigation through its ability to provide 3D holographic projection of otherwise hidden anatomical information. This pilot cadaver study investigated the feasibility and accuracy of one of the first holographic navigation techniques for lumbar pedicle screw placement. METHODS Lumbar computer tomography scans (CT) of two cadaver specimens and their reconstructed 3D models were used for pedicle screw trajectory planning. Planned trajectories and 3D models were subsequently uploaded to an AR head-mounted device. Randomly, k-wires were placed either into the left or the right pedicle of a vertebra (L1-5) with or without AR-navigation (by holographic projection of the planned trajectory). CT-scans were subsequently performed to assess accuracy of both techniques. RESULTS A total of 18 k-wires could be placed (8 navigated, 10 free hand) by two experienced spine surgeons. In two vertebrae, the AR-navigation was aborted because the registration of the preoperative plan with the intraoperative anatomy was imprecise due to a technical failure. The average differences of the screw entry points between planning and execution were 4.74 ± 2.37 mm in the freehand technique and 5.99 ± 3.60 mm in the AR-navigated technique (p = 0.39). The average deviation from the planned trajectories was 11.21° ± 7.64° in the freehand technique and 5.88° ± 3.69° in the AR-navigated technique (p = 0.09). CONCLUSION This pilot study demonstrates improved angular precision in one of the first AR-navigated pedicle screw placement studies worldwide. Technical shortcomings need to be eliminated before potential clinical applications.
Collapse
|
61
|
Yanni DS, Ozgur BM, Louis RG, Shekhtman Y, Iyer RR, Boddapati V, Iyer A, Patel PD, Jani R, Cummock M, Herur-Raman A, Dang P, Goldstein IM, Brant-Zawadzki M, Steineke T, Lenke LG. Real-time navigation guidance with intraoperative CT imaging for pedicle screw placement using an augmented reality head-mounted display: a proof-of-concept study. Neurosurg Focus 2021; 51:E11. [PMID: 34333483 DOI: 10.3171/2021.5.focus21209] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/01/2021] [Accepted: 05/17/2021] [Indexed: 11/06/2022]
Abstract
OBJECTIVE Augmented reality (AR) has the potential to improve the accuracy and efficiency of instrumentation placement in spinal fusion surgery, increasing patient safety and outcomes, optimizing ergonomics in the surgical suite, and ultimately lowering procedural costs. The authors sought to describe the use of a commercial prototype Spine AR platform (SpineAR) that provides a commercial AR head-mounted display (ARHMD) user interface for navigation-guided spine surgery incorporating real-time navigation images from intraoperative imaging with a 3D-reconstructed model in the surgeon's field of view, and to assess screw placement accuracy via this method. METHODS Pedicle screw placement accuracy was assessed and compared with literature-reported data of the freehand (FH) technique. Accuracy with SpineAR was also compared between participants of varying spine surgical experience. Eleven operators without prior experience with AR-assisted pedicle screw placement took part in the study: 5 attending neurosurgeons and 6 trainees (1 neurosurgical fellow, 1 senior orthopedic resident, 3 neurosurgical residents, and 1 medical student). Commercially available 3D-printed lumbar spine models were utilized as surrogates of human anatomy. Among the operators, a total of 192 screws were instrumented bilaterally from L2-5 using SpineAR in 24 lumbar spine models. All but one trainee also inserted 8 screws using the FH method. In addition to accuracy scoring using the Gertzbein-Robbins grading scale, axial trajectory was assessed, and user feedback on experience with SpineAR was collected. RESULTS Based on the Gertzbein-Robbins grading scale, the overall screw placement accuracy using SpineAR among all users was 98.4% (192 screws). Accuracy for attendings and trainees was 99.1% (112 screws) and 97.5% (80 screws), respectively. Accuracy rates were higher compared with literature-reported lumbar screw placement accuracy using FH for attendings (99.1% vs 94.32%; p = 0.0212) and all users (98.4% vs 94.32%; p = 0.0099). The percentage of total inserted screws with a minimum of 5° medial angulation was 100%. No differences were observed between attendings and trainees or between the two methods. User feedback on SpineAR was generally positive. CONCLUSIONS Screw placement was feasible and accurate using SpineAR, an ARHMD platform with real-time navigation guidance that provided a favorable surgeon-user experience.
Collapse
Affiliation(s)
- Daniel S Yanni
- 1Pickup Family Neurosciences Institute, Hoag Memorial Hospital Presbyterian Newport Beach; and.,2Disc Comfort, Inc., Newport Beach, California
| | - Burak M Ozgur
- 1Pickup Family Neurosciences Institute, Hoag Memorial Hospital Presbyterian Newport Beach; and
| | - Robert G Louis
- 1Pickup Family Neurosciences Institute, Hoag Memorial Hospital Presbyterian Newport Beach; and
| | - Yevgenia Shekhtman
- 3Neuroscience Institute, Hackensack Meridian JFK Medical Center, Edison; and
| | - Rajiv R Iyer
- 4Department of Orthopedic Surgery, Columbia University; and
| | | | - Asha Iyer
- 3Neuroscience Institute, Hackensack Meridian JFK Medical Center, Edison; and
| | - Purvee D Patel
- 5Department of Neurological Surgery, Rutgers New Jersey Medical School, Newark, New Jersey
| | - Raja Jani
- 5Department of Neurological Surgery, Rutgers New Jersey Medical School, Newark, New Jersey
| | - Matthew Cummock
- 5Department of Neurological Surgery, Rutgers New Jersey Medical School, Newark, New Jersey
| | - Aalap Herur-Raman
- 6George Washington University School of Medicine, Washington, DC; and
| | | | - Ira M Goldstein
- 5Department of Neurological Surgery, Rutgers New Jersey Medical School, Newark, New Jersey
| | - Michael Brant-Zawadzki
- 1Pickup Family Neurosciences Institute, Hoag Memorial Hospital Presbyterian Newport Beach; and
| | - Thomas Steineke
- 3Neuroscience Institute, Hackensack Meridian JFK Medical Center, Edison; and
| | - Lawrence G Lenke
- 4Department of Orthopedic Surgery, Columbia University; and.,8Department of Neurological Surgery, NewYork-Presbyterian/Allen Hospital, New York, New York
| |
Collapse
|
62
|
Yahanda AT, Moore E, Ray WZ, Pennicooke B, Jennings JW, Molina CA. First in-human report of the clinical accuracy of thoracolumbar percutaneous pedicle screw placement using augmented reality guidance. Neurosurg Focus 2021; 51:E10. [PMID: 34333484 DOI: 10.3171/2021.5.focus21217] [Citation(s) in RCA: 21] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/01/2021] [Indexed: 11/06/2022]
Abstract
OBJECTIVE Augmented reality (AR) is an emerging technology that has great potential for guiding the safe and accurate placement of spinal hardware, including percutaneous pedicle screws. The goal of this study was to assess the accuracy of 63 percutaneous pedicle screws placed at a single institution using an AR head-mounted display (ARHMD) system. METHODS Retrospective analyses were performed for 9 patients who underwent thoracic and/or lumbar percutaneous pedicle screw placement guided by ARHMD technology. Clinical accuracy was assessed via the Gertzbein-Robbins scale by the authors and by an independent musculoskeletal radiologist. Thoracic pedicle subanalysis was also performed to assess screw accuracy based on pedicle morphology. RESULTS Nine patients received thoracic or lumbar AR-guided percutaneous pedicle screws. The mean age at the time of surgery was 71.9 ± 11.5 years and the mean number of screws per patient was 7. Indications for surgery were spinal tumors (n = 4, 44.4%), degenerative disease (n = 3, 33.3%), spinal deformity (n = 1, 11.1%), and a combination of deformity and infection (n = 1, 11.1%). Presenting symptoms were most commonly low-back pain (n = 7, 77.8%) and lower-extremity weakness (n = 5, 55.6%), followed by radicular lower-extremity pain, loss of lower-extremity sensation, or incontinence/urinary retention (n = 3 each, 33.3%). In all, 63 screws were placed (32 thoracic, 31 lumbar). The accuracy for these screws was 100% overall; all screws were Gertzbein-Robbins grade A or B (96.8% grade A, 3.2% grade B). This accuracy was achieved in the thoracic spine regardless of pedicle cancellous bone morphology. CONCLUSIONS AR-guided surgery demonstrated a 100% accuracy rate for the insertion of 63 percutaneous pedicle screws in 9 patients (100% rate of Gertzbein-Robbins grade A or B screw placement). Using an ARHMS system for the placement of percutaneous pedicle screws showed promise, but further validation using a larger cohort of patients across multiple surgeons and institutions will help to determine the true accuracy enabled by this technology.
Collapse
Affiliation(s)
| | - Emelia Moore
- 2Wayne State University School of Medicine, Detroit, Michigan
| | | | | | - Jack W Jennings
- 3Radiology, Washington University School of Medicine in St. Louis, Missouri; and
| | | |
Collapse
|
63
|
Molina CA, Sciubba DM, Greenberg JK, Khan M, Witham T. Clinical Accuracy, Technical Precision, and Workflow of the First in Human Use of an Augmented-Reality Head-Mounted Display Stereotactic Navigation System for Spine Surgery. Oper Neurosurg (Hagerstown) 2021; 20:300-309. [PMID: 33377137 DOI: 10.1093/ons/opaa398] [Citation(s) in RCA: 44] [Impact Index Per Article: 14.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/12/2020] [Accepted: 09/13/2020] [Indexed: 12/17/2022] Open
Abstract
BACKGROUND Augmented reality mediated spine surgery is a novel technology for spine navigation. Benchmark cadaveric data have demonstrated high accuracy and precision leading to recent regulatory approval. Absence of respiratory motion in cadaveric studies may positively bias precision and accuracy results and analogous investigations are prudent in live clinical scenarios. OBJECTIVE To report a technical note, accuracy, precision analysis of the first in-human deployment of this technology. METHODS A 78-yr-old female underwent an L4-S1 decompression, pedicle screw, and rod fixation for degenerative spine disease. Six pedicle screws were inserted via AR-HMD (xvision; Augmedics, Chicago, Illinois) navigation. Intraoperative computed tomography was used for navigation registration as well as implant accuracy and precision assessment. Clinical accuracy was graded per the Gertzbein-Robbins (GS) scale by an independent neuroradiologist. Technical precision was analyzed by comparing 3-dimensional (3D) (x, y, z) virtual implant vs real implant position coordinates and reported as linear (mm) and angular (°) deviation. Present data were compared to benchmark cadaveric data. RESULTS Clinical accuracy (per the GS grading scale) was 100%. Technical precision analysis yielded a mean linear deviation of 2.07 mm (95% CI: 1.62-2.52 mm) and angular deviation of 2.41° (95% CI: 1.57-3.25°). In comparison to prior cadaveric data (99.1%, 2.03 ± 0.99 mm, 1.41 ± 0.61°; GS accuracy 3D linear and angular deviation, respectively), the present results were not significantly different (P > .05). CONCLUSION The first in human deployment of the single Food and Drug Administration approved AR-HMD stereotactic spine navigation platform demonstrated clinical accuracy and technical precision of inserted hardware comparable to previously acquired cadaveric studies.
Collapse
Affiliation(s)
- Camilo A Molina
- Department of Neurosurgery, Washington University School of Medicine, St. Louis, Missouri
- Department of Neurosurgery, Johns Hopkins University School of Medicine, Baltimore, Maryland
| | - Daniel M Sciubba
- Department of Neurosurgery, Johns Hopkins University School of Medicine, Baltimore, Maryland
| | - Jacob K Greenberg
- Department of Neurosurgery, Washington University School of Medicine, St. Louis, Missouri
| | - Majid Khan
- Department of Radiology, Johns Hopkins University School of Medicine, Baltimore, Maryland
| | - Timothy Witham
- Department of Neurosurgery, Johns Hopkins University School of Medicine, Baltimore, Maryland
| |
Collapse
|
64
|
Hu X, Baena FRY, Cutolo F. Head-Mounted Augmented Reality Platform for Markerless Orthopaedic Navigation. IEEE J Biomed Health Inform 2021; 26:910-921. [PMID: 34115600 DOI: 10.1109/jbhi.2021.3088442] [Citation(s) in RCA: 12] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
Visual augmented reality (AR) has the potential to improve the accuracy, efficiency and reproducibility of computer-assisted orthopaedic surgery (CAOS). AR Head-mounted displays (HMDs) further allow non-eye-shift target observation and egocentric view. Recently, a markerless tracking and registration (MTR) algorithm was proposed to avoid the artificial markers that are conventionally pinned into the target anatomy for tracking, as their use prolongs surgical workflow, introduces human-induced errors, and necessitates additional surgical invasion in patients. However, such an MTR-based method has neither been explored for surgical applications nor integrated into current AR HMDs, making the ergonomic HMD-based markerless AR CAOS navigation hard to achieve. To these aims, we present a versatile, device-agnostic and accurate HMD-based AR platform. Our software platform, supporting both video see-through (VST) and optical see-through (OST) modes, integrates two proposed fast calibration procedures using a specially designed calibration tool. According to the camera-based evaluation, our AR platform achieves a display error of 6.31 2.55 arcmin for VST and 7.72 3.73 arcmin for OST. A proof-of-concept markerless surgical navigation system to assist in femoral bone drilling was then developed based on the platform and Microsoft HoloLens 1. According to the user study, both VST and OST markerless navigation systems are reliable, with the OST system providing the best usability. The measured navigation error is 4.90 1.04 mm, 5.96 2.22 for VST system and 4.36 0.80 mm, 5.65 1.42 for OST system.
Collapse
|
65
|
Farshad M, Fürnstahl P, Spirig JM. First in man in-situ augmented reality pedicle screw navigation. NORTH AMERICAN SPINE SOCIETY JOURNAL 2021; 6:100065. [PMID: 35141630 PMCID: PMC8819976 DOI: 10.1016/j.xnsj.2021.100065] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 02/05/2021] [Revised: 04/17/2021] [Accepted: 04/20/2021] [Indexed: 12/29/2022]
Abstract
Background Augmented reality (AR) is a rising technology gaining increasing utility in medicine. By superimposing the surgical site and the operator's visual field with computer-generated information, it has the potential to enhance the cognitive skills of surgeons. This is the report of the first in man case with "direct holographic navigation" as part of a randomized controlled trial. Case description A pointing instrument was equipped with a sterile fiducial marker, which was used to obtain a digital representation of the intraoperative bony anatomy of the lumbar spine. Subsequently, a previously validated registration method was applied to superimpose the surgery plan with the intraoperative anatomy. The registration result is shown in situ as a 3D AR hologram of the preoperative 3D vertebra model with the planned screw trajectory and entry point for validation and approval by the surgeon. After achieving alignment with the surgery plan, a borehole is drilled and the pedicle screw placed. Postoperativ computer tomography was used to measure accuracy of this novel method for surgical navigation. Outcome Correct screw positions entirely within bone were documented with a postoperative CT, with an accuracy similar to current standard of care methods for surgical navigation. The patient was mobilized uneventfully on the first postoperative day with little pain medication and dismissed on the fourth postoperative day. Conclusion This first in man report of direct AR navigation demonstrates feasibility in vivo. The continuation of this randomized controlled study will evaluate the value of this novel technology.
Collapse
Affiliation(s)
- Mazda Farshad
- Spine Division, Balgrist University Hospital, University of Zurich, Forchstrasse 340, 8008 Zurich, Switzerland
- Corresponding author.
| | - Philipp Fürnstahl
- ROCS: Research in Orthopedic Computer Science, Balgrist University Hospital, University of Zurich, Forchstrasse 340, 8008, Zurich, Switzerland
| | - José Miguel Spirig
- Spine Division, Balgrist University Hospital, University of Zurich, Forchstrasse 340, 8008 Zurich, Switzerland
| |
Collapse
|
66
|
Augmented reality in the operating room: a clinical feasibility study. BMC Musculoskelet Disord 2021; 22:451. [PMID: 34006234 PMCID: PMC8132365 DOI: 10.1186/s12891-021-04339-w] [Citation(s) in RCA: 13] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/05/2020] [Accepted: 05/06/2021] [Indexed: 11/20/2022] Open
Abstract
Background Augmented Reality (AR) is a rapidly emerging technology finding growing acceptance and application in different fields of surgery. Various studies have been performed evaluating the precision and accuracy of AR guided navigation. This study investigates the feasibility of a commercially available AR head mounted device during orthopedic surgery. Methods Thirteen orthopedic surgeons from a Swiss university clinic performed 25 orthopedic surgical procedures wearing a holographic AR headset (HoloLens, Microsoft, Redmond, WA, USA) providing complementary three-dimensional, patient specific anatomic information. The surgeon’s experience of using the device during surgery was recorded using a standardized 58-item questionnaire grading different aspects on a 100-point scale with anchor statements. Results Surgeons were generally satisfied with image quality (85 ± 17 points) and accuracy of the virtual objects (84 ± 19 point). Wearing the AR device was rated as fairly comfortable (79 ± 13 points). Functionality of voice commands (68 ± 20 points) and gestures (66 ± 20 points) provided less favorable results. The greatest potential in the use of the AR device was found for surgical correction of deformities (87 ± 15 points). Overall, surgeons were satisfied with the application of this novel technology (78 ± 20 points) and future access to it was demanded (75 ± 22 points). Conclusion AR is a rapidly evolving technology with large potential in different surgical settings, offering the opportunity to provide a compact, low cost alternative requiring a minimum of infrastructure compared to conventional navigation systems. While surgeons where generally satisfied with image quality of the here tested head mounted AR device, some technical and ergonomic shortcomings were pointed out. This study serves as a proof of concept for the use of an AR head mounted device in a real-world sterile setting in orthopedic surgery. Supplementary Information The online version contains supplementary material available at 10.1186/s12891-021-04339-w.
Collapse
|
67
|
Gu W, Shah K, Knopf J, Navab N, Unberath M. Feasibility of image-based augmented reality guidance of total shoulder arthroplasty using microsoft HoloLens 1. COMPUTER METHODS IN BIOMECHANICS AND BIOMEDICAL ENGINEERING: IMAGING & VISUALIZATION 2021. [DOI: 10.1080/21681163.2020.1835556] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/09/2023]
Affiliation(s)
- Wenhao Gu
- Johns Hopkins University, Baltimore, USA
| | | | | | | | | |
Collapse
|
68
|
Hein J, Seibold M, Bogo F, Farshad M, Pollefeys M, Fürnstahl P, Navab N. Towards markerless surgical tool and hand pose estimation. Int J Comput Assist Radiol Surg 2021; 16:799-808. [PMID: 33881732 PMCID: PMC8134312 DOI: 10.1007/s11548-021-02369-2] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/10/2021] [Accepted: 04/06/2021] [Indexed: 01/16/2023]
Abstract
Purpose: Tracking of tools and surgical activity is becoming more and more important in the context of computer assisted surgery. In this work, we present a data generation framework, dataset and baseline methods to facilitate further research in the direction of markerless hand and instrument pose estimation in realistic surgical scenarios. Methods: We developed a rendering pipeline to create inexpensive and realistic synthetic data for model pretraining. Subsequently, we propose a pipeline to capture and label real data with hand and object pose ground truth in an experimental setup to gather high-quality real data. We furthermore present three state-of-the-art RGB-based pose estimation baselines. Results: We evaluate three baseline models on the proposed datasets. The best performing baseline achieves an average tool 3D vertex error of 16.7 mm on synthetic data as well as 13.8 mm on real data which is comparable to the state-of-the art in RGB-based hand/object pose estimation. Conclusion: To the best of our knowledge, we propose the first synthetic and real data generation pipelines to generate hand and object pose labels for open surgery. We present three baseline models for RGB based object and object/hand pose estimation based on RGB frames. Our realistic synthetic data generation pipeline may contribute to overcome the data bottleneck in the surgical domain and can easily be transferred to other medical applications. Supplementary Information The online version supplementary material available at 10.1007/s11548-021-02369-2.
Collapse
Affiliation(s)
- Jonas Hein
- Research in Orthopedic Computer Science, University Hospital Balgrist, University of Zurich, Balgrist CAMPUS, Zurich, Switzerland. .,Computer Vision and Geometry Group, ETH Zurich, Zurich, Switzerland.
| | - Matthias Seibold
- Research in Orthopedic Computer Science, University Hospital Balgrist, University of Zurich, Balgrist CAMPUS, Zurich, Switzerland. .,Computer Aided Medical Procedures, Technical University Munich, Garching, Germany.
| | - Federica Bogo
- Mixed Reality & AI Zurich Lab, Microsoft, Zurich, Switzerland
| | - Mazda Farshad
- Balgrist University Hospital, University of Zurich, Zurich, Switzerland
| | - Marc Pollefeys
- Computer Vision and Geometry Group, ETH Zurich, Zurich, Switzerland.,Mixed Reality & AI Zurich Lab, Microsoft, Zurich, Switzerland
| | - Philipp Fürnstahl
- Research in Orthopedic Computer Science, University Hospital Balgrist, University of Zurich, Balgrist CAMPUS, Zurich, Switzerland
| | - Nassir Navab
- Computer Aided Medical Procedures, Technical University Munich, Garching, Germany
| |
Collapse
|
69
|
Tu P, Gao Y, Lungu AJ, Li D, Wang H, Chen X. Augmented reality based navigation for distal interlocking of intramedullary nails utilizing Microsoft HoloLens 2. Comput Biol Med 2021; 133:104402. [PMID: 33895460 DOI: 10.1016/j.compbiomed.2021.104402] [Citation(s) in RCA: 20] [Impact Index Per Article: 6.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/31/2021] [Revised: 03/24/2021] [Accepted: 04/11/2021] [Indexed: 11/19/2022]
Abstract
BACKGROUND AND OBJECTIVE The distal interlocking of intramedullary nail remains a technically demanding procedure. Existing augmented reality based solutions still suffer from hand-eye coordination problem, prolonged operation time, and inadequate resolution. In this study, an augmented reality based navigation system for distal interlocking of intramedullary nail is developed using Microsoft HoloLens 2, the state-of-the-art optical see-through head-mounted display. METHODS A customized registration cube is designed to assist surgeons with better depth perception when performing registration procedures. During drilling, surgeons can obtain accurate and in-situ visualization of intramedullary nail and drilling path, and dynamic navigation is enabled. An intraoperative warning system is proposed to provide intuitive feedback of real-time deviations and electromagnetic disturbances. RESULTS The preclinical phantom experiment showed that the reprojection errors along the X, Y, and Z axes were 1.55 ± 0.27 mm, 1.71 ± 0.40 mm, and 2.84 ± 0.78 mm, respectively. The end-to-end evaluation method indicated the distance error was 1.61 ± 0.44 mm, and the 3D angle error was 1.46 ± 0.46°. A cadaver experiment was also conducted to evaluate the feasibility of the system. CONCLUSION Our system has potential advantages over the 2D-screen based navigation system and the pointing device based navigation system in terms of accuracy and time consumption, and has tremendous application prospects.
Collapse
Affiliation(s)
- Puxun Tu
- School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai, China
| | - Yao Gao
- School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai, China
| | - Abel J Lungu
- School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai, China
| | - Dongyuan Li
- School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai, China
| | - Huixiang Wang
- Department of Orthopedics, Shanghai Jiao Tong University Affiliated Sixth People's Hospital, Shanghai, China.
| | - Xiaojun Chen
- School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai, China; Institute of Medical Robotics, Shanghai Jiao Tong University, Shanghai, China.
| |
Collapse
|
70
|
Sakai D, Joyce K, Sugimoto M, Horikita N, Hiyama A, Sato M, Devitt A, Watanabe M. Augmented, virtual and mixed reality in spinal surgery: A real-world experience. J Orthop Surg (Hong Kong) 2021; 28:2309499020952698. [PMID: 32909902 DOI: 10.1177/2309499020952698] [Citation(s) in RCA: 21] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/16/2022] Open
Abstract
This review aims to identify the role of augmented, virtual or mixed reality (AR, VR or MR) technologies in setting of spinal surgery. The authors address the challenges surrounding the implementation of this technology in the operating room. A technical standpoint addresses the efficacy of these imaging modalities based on the current literature in the field. Ultimately, these technologies must be cost-effective to ensure widespread adoption. This may be achieved through reduced surgical times and decreased incidence of post-operative complications and revisions while maintaining equivalent safety profile to alternative surgical approaches. While current studies focus mainly on the successful placement of pedicle screws via AR-guided instrumentation, a wider scope of procedures may be assisted using AR, VR or MR technology once efficacy and safety have been validated. These emerging technologies offer a significant advantage in the guidance of complex procedures that require high precision and accuracy using minimally invasive interventions.
Collapse
Affiliation(s)
- Daisuke Sakai
- Department of Orthopaedic Surgery, Surgical Science, Tokai University School of Medicine, Isehara, Kanagawa, Japan
| | - Kieran Joyce
- SFI Research Centre for Medical Devices, National University of Ireland, Galway, Ireland.,Department of Orthopaedic Surgery, School of Medicine, National University of Ireland, Galway, Ireland
| | - Maki Sugimoto
- Innovation Lab, Teikyo University Okinaga Research Institute, Tokyo, Japan
| | - Natsumi Horikita
- Department of Orthopaedic Surgery, Surgical Science, Tokai University School of Medicine, Isehara, Kanagawa, Japan
| | - Akihiko Hiyama
- Department of Orthopaedic Surgery, Surgical Science, Tokai University School of Medicine, Isehara, Kanagawa, Japan
| | - Masato Sato
- Department of Orthopaedic Surgery, Surgical Science, Tokai University School of Medicine, Isehara, Kanagawa, Japan
| | - Aiden Devitt
- Department of Orthopaedic Surgery, School of Medicine, National University of Ireland, Galway, Ireland
| | - Masahiko Watanabe
- Department of Orthopaedic Surgery, Surgical Science, Tokai University School of Medicine, Isehara, Kanagawa, Japan
| |
Collapse
|
71
|
Casari FA, Navab N, Hruby LA, Kriechling P, Nakamura R, Tori R, de Lourdes Dos Santos Nunes F, Queiroz MC, Fürnstahl P, Farshad M. Augmented Reality in Orthopedic Surgery Is Emerging from Proof of Concept Towards Clinical Studies: a Literature Review Explaining the Technology and Current State of the Art. Curr Rev Musculoskelet Med 2021; 14:192-203. [PMID: 33544367 PMCID: PMC7990993 DOI: 10.1007/s12178-021-09699-3] [Citation(s) in RCA: 41] [Impact Index Per Article: 13.7] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 01/08/2021] [Indexed: 02/07/2023]
Abstract
PURPOSE OF REVIEW Augmented reality (AR) is becoming increasingly popular in modern-day medicine. Computer-driven tools are progressively integrated into clinical and surgical procedures. The purpose of this review was to provide a comprehensive overview of the current technology and its challenges based on recent literature mainly focusing on clinical, cadaver, and innovative sawbone studies in the field of orthopedic surgery. The most relevant literature was selected according to clinical and innovational relevance and is summarized. RECENT FINDINGS Augmented reality applications in orthopedic surgery are increasingly reported. In this review, we summarize basic principles of AR including data preparation, visualization, and registration/tracking and present recently published clinical applications in the area of spine, osteotomies, arthroplasty, trauma, and orthopedic oncology. Higher accuracy in surgical execution, reduction of radiation exposure, and decreased surgery time are major findings presented in the literature. In light of the tremendous progress of technological developments in modern-day medicine and emerging numbers of research groups working on the implementation of AR in routine clinical procedures, we expect the AR technology soon to be implemented as standard devices in orthopedic surgery.
Collapse
Affiliation(s)
- Fabio A Casari
- Department of Orthopedic Surgery, Balgrist University Hospital, University of Zurich, Zurich, Switzerland.
- ROCS, Research in Orthopedic Computer Science, Balgrist Campus, University of Zurich, Forchstrasse 340, 8008, Zürich, Switzerland.
| | - Nassir Navab
- Computer Aided Medical Procedures (CAMP), Technische Universität München, Munich, Germany
- Computer Aided Medical Procedures (CAMP), Johns Hopkins University, Baltimore, MD, USA
| | - Laura A Hruby
- Department of Orthopedic Surgery, Balgrist University Hospital, University of Zurich, Zurich, Switzerland
- Department of Orthopaedics and Trauma Surgery, Medical University of Vienna, Vienna, Austria
| | - Philipp Kriechling
- Department of Orthopedic Surgery, Balgrist University Hospital, University of Zurich, Zurich, Switzerland
| | - Ricardo Nakamura
- Computer Engineering and Digital Systems Department, Escola Politécnica, Universidade de São Paulo, São Paulo, SP, Brazil
| | - Romero Tori
- Computer Engineering and Digital Systems Department, Escola Politécnica, Universidade de São Paulo, São Paulo, SP, Brazil
| | | | - Marcelo C Queiroz
- Orthopedics and Traumatology Department, Faculty of Medical Sciences of Santa Casa de Sao Paulo, Sao Paulo, SP, Brazil
| | - Philipp Fürnstahl
- ROCS, Research in Orthopedic Computer Science, Balgrist Campus, University of Zurich, Forchstrasse 340, 8008, Zürich, Switzerland
| | - Mazda Farshad
- Department of Orthopedic Surgery, Balgrist University Hospital, University of Zurich, Zurich, Switzerland
| |
Collapse
|
72
|
Burström G, Persson O, Edström E, Elmi-Terander A. Augmented reality navigation in spine surgery: a systematic review. Acta Neurochir (Wien) 2021; 163:843-852. [PMID: 33506289 PMCID: PMC7886712 DOI: 10.1007/s00701-021-04708-3] [Citation(s) in RCA: 39] [Impact Index Per Article: 13.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/29/2020] [Accepted: 01/06/2021] [Indexed: 02/07/2023]
Abstract
BACKGROUND Conventional spinal navigation solutions have been criticized for having a negative impact on time in the operating room and workflow. AR navigation could potentially alleviate some of these concerns while retaining the benefits of navigated spine surgery. The objective of this study is to summarize the current evidence for using augmented reality (AR) navigation in spine surgery. METHODS We performed a systematic review to explore the current evidence for using AR navigation in spine surgery. PubMed and Web of Science were searched from database inception to November 27, 2020, for data on the AR navigation solutions; the reported efficacy of the systems; and their impact on workflow, radiation, and cost-benefit relationships. RESULTS In this systematic review, 28 studies were included in the final analysis. The main findings were superior workflow and non-inferior accuracy when comparing AR to free-hand (FH) or conventional surgical navigation techniques. A limited number of studies indicated decreased use of radiation. There were no studies reporting mortality, morbidity, or cost-benefit relationships. CONCLUSIONS AR provides a meaningful addition to FH surgery and traditional navigation methods for spine surgery. However, the current evidence base is limited and prospective studies on clinical outcomes and cost-benefit relationships are needed.
Collapse
|
73
|
Dennler C, Safa NA, Bauer DE, Wanivenhaus F, Liebmann F, Götschi T, Farshad M. Augmented Reality Navigated Sacral-Alar-Iliac Screw Insertion. Int J Spine Surg 2021; 15:161-168. [PMID: 33900970 DOI: 10.14444/8021] [Citation(s) in RCA: 12] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/23/2023] Open
Abstract
BACKGROUND Sacral-alar-iliac (SAI) screws are increasingly used for lumbo-pelvic fixation procedures. Insertion of SAI screws is technically challenging, and surgeons often rely on costly and time-consuming navigation systems. We investigated the accuracy and precision of an augmented reality (AR)-based and commercially available head-mounted device requiring minimal infrastructure. METHODS A pelvic sawbone model served to drill pilot holes of 80 SAI screw trajectories by 2 surgeons, randomly either freehand (FH) without any kind of navigation or with AR navigation. The number of primary pilot hole perforations, simulated screw perforation, minimal axis/outer cortical wall distance, true sagittal cranio-caudal inclination angle (tSCCIA), true axial medio-lateral angle, and maximal screw length (MSL) were measured and compared to predefined optimal values. RESULTS In total, 1/40 (2.5%) of AR-navigated screw hole trajectories showed a perforation before passing the inferior gluteal line compared to 24/40 (60%) of FH screw hole trajectories (P < .05). The differences between FH- and AR-guided holes compared to optimal values were significant for tSCCIA with -10.8° ± 11.77° and MSL -65.29 ± 15 mm vs 55.04 ± 6.76 mm (P = .001). CONCLUSIONS In this study, the additional anatomical information provided by the AR headset and the superimposed operative plan improved the precision of drilling pilot holes for SAI screws in a laboratory setting compared to the conventional FH technique. Further technical development and validation studies are currently being performed to investigate potential clinical benefits of the AR-based navigation approach described here. LEVEL OF EVIDENCE 4.
Collapse
Affiliation(s)
- Cyrill Dennler
- Department of Orthopedics, University Hospital Balgrist, University of Zürich, Zürich, Switzerland
| | - Nico Akhavan Safa
- Department of Orthopedics, University Hospital Balgrist, University of Zürich, Zürich, Switzerland
| | - David Ephraim Bauer
- Department of Orthopedics, University Hospital Balgrist, University of Zürich, Zürich, Switzerland
| | - Florian Wanivenhaus
- Department of Orthopedics, University Hospital Balgrist, University of Zürich, Zürich, Switzerland
| | - Florentin Liebmann
- Computer Assisted Research and Development Group, University Hospital Balgrist, University of Zürich, Zürich Switzerland.,Laboratory for Orthopaedic Biomechanics, ETH Zürich, Zürich, Switzerland
| | - Tobias Götschi
- Department of Orthopedics, University Hospital Balgrist, University of Zürich, Zürich, Switzerland
| | - Mazda Farshad
- Department of Orthopedics, University Hospital Balgrist, University of Zürich, Zürich, Switzerland
| |
Collapse
|
74
|
Augmented Reality Based Surgical Navigation of Complex Pelvic Osteotomies—A Feasibility Study on Cadavers. APPLIED SCIENCES-BASEL 2021. [DOI: 10.3390/app11031228] [Citation(s) in RCA: 11] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/22/2022]
Abstract
Augmented reality (AR)-based surgical navigation may offer new possibilities for safe and accurate surgical execution of complex osteotomies. In this study we investigated the feasibility of navigating the periacetabular osteotomy of Ganz (PAO), known as one of the most complex orthopedic interventions, on two cadaveric pelves under realistic operating room conditions. Preoperative planning was conducted on computed tomography (CT)-reconstructed 3D models using an in-house developed software, which allowed creating cutting plane objects for planning of the osteotomies and reorientation of the acetabular fragment. An AR application was developed comprising point-based registration, motion compensation and guidance for osteotomies as well as fragment reorientation. Navigation accuracy was evaluated on CT-reconstructed 3D models, resulting in an error of 10.8 mm for osteotomy starting points and 5.4° for osteotomy directions. The reorientation errors were 6.7°, 7.0° and 0.9° for the x-, y- and z-axis, respectively. Average postoperative error of LCE angle was 4.5°. Our study demonstrated that the AR-based execution of complex osteotomies is feasible. Fragment realignment navigation needs further improvement, although it is more accurate than the state of the art in PAO surgery.
Collapse
|
75
|
Kriechling P, Roner S, Liebmann F, Casari F, Fürnstahl P, Wieser K. Augmented reality for base plate component placement in reverse total shoulder arthroplasty: a feasibility study. Arch Orthop Trauma Surg 2021; 141:1447-1453. [PMID: 32715400 PMCID: PMC8354932 DOI: 10.1007/s00402-020-03542-z] [Citation(s) in RCA: 20] [Impact Index Per Article: 6.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/08/2020] [Accepted: 07/15/2020] [Indexed: 01/27/2023]
Abstract
BACKGROUND Accurate glenoid positioning in reverse total shoulder arthroplasty (RSA) is important to achieve satisfying functional outcome and prosthesis longevity. Optimal component placement can be challenging, especially in severe glenoid deformities. The use of patient-specific instruments (PSI) and 3D computer-assisted optical tracking navigation (NAV) are already established methods to improve surgical precision. Augmented reality technology (AR) promises similar results at low cost and ease of use. With AR, the planned component placement can be superimposed to the surgical situs and shown directly in the operating field using a head mounted display. We introduce a new navigation technique using AR via head mounted display for surgical navigation in this feasibility study, aiming to improve and enhance the surgical planning. METHODS 3D surface models of ten human scapulae were printed from computed tomography (CT) data of cadaver scapulae. Guidewire positioning of the central back of the glenoid baseplate was planned with a dedicated computer software. A hologram of the planned guidewire with dynamic navigation was then projected onto the 3D-created models of the cadaver shoulders. The registration of the plan to the anatomy was realized by digitizing the glenoid surface and the base of the coracoid with optical tracking using a fiducial marker. After navigated placement of the central guidewires, another CT imaging was recorded, and the 3D model was superimposed with the preoperative planning to analyze the deviation from the planned and executed central guides trajectory and entry point. RESULTS The mean deviation of the ten placed guidewires from the planned trajectory was 2.7° ± 1.3° (95% CI 1.9°; 3.6°). The mean deviation to the planned entry point of the ten placed guidewires measured 2.3 mm ± 1.1 mm (95% CI 1.5 mm; 3.1 mm). CONCLUSION AR may be a promising new technology for highly precise surgical execution of 3D preoperative planning in RSA.
Collapse
Affiliation(s)
- Philipp Kriechling
- Department of Orthopedics, Balgrist University Hospital, Zurich, Switzerland
| | - Simon Roner
- Department of Orthopedics, Balgrist University Hospital, Zurich, Switzerland
| | - Florentin Liebmann
- Computer Assisted Research and Development Group, Balgrist University Hospital, Zurich, Switzerland
| | - Fabio Casari
- Department of Orthopedics, Balgrist University Hospital, Zurich, Switzerland
| | - Philipp Fürnstahl
- Computer Assisted Research and Development Group, Balgrist University Hospital, Zurich, Switzerland
| | - Karl Wieser
- Department of Orthopedics, Balgrist University Hospital, Zurich, Switzerland
| |
Collapse
|
76
|
Yuk FJ, Maragkos GA, Sato K, Steinberger J. Current innovation in virtual and augmented reality in spine surgery. ANNALS OF TRANSLATIONAL MEDICINE 2021; 9:94. [PMID: 33553387 PMCID: PMC7859743 DOI: 10.21037/atm-20-1132] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Subscribe] [Scholar Register] [Indexed: 12/27/2022]
Abstract
In spinal surgery, outcomes are directly related both to patient and procedure selection, as well as the accuracy and precision of instrumentation placed. Poorly placed instrumentation can lead to spinal cord, nerve root or vascular injury. Traditionally, spine surgery was performed by open methods and placement of instrumentation under direct visualization. However, minimally invasive surgery (MIS) has seen substantial advances in spine, with an ever-increasing range of indications and procedures. For these reasons, novel methods to visualize anatomy and precisely guide surgery, such as intraoperative navigation, are extremely useful in this field. In this review, we present the recent advances and innovations utilizing simulation methods in spine surgery. The application of these techniques is still relatively new, however quickly being integrated in and outside the operating room. These include virtual reality (VR) (where the entire simulation is virtual), mixed reality (MR) (a combination of virtual and physical components), and augmented reality (AR) (the superimposition of a virtual component onto physical reality). VR and MR have primarily found applications in a teaching and preparatory role, while AR is mainly applied in hands-on surgical settings. The present review attempts to provide an overview of the latest advances and applications of these methods in the neurosurgical spine setting.
Collapse
Affiliation(s)
- Frank J Yuk
- Department of Neurosurgery, Mount Sinai Hospital, Icahn School of Medicine at Mount Sinai, New York, NY, USA
| | - Georgios A Maragkos
- Department of Neurosurgery, Mount Sinai Hospital, Icahn School of Medicine at Mount Sinai, New York, NY, USA
| | - Kosuke Sato
- Hospital for Special Surgery, New York, NY, USA
| | - Jeremy Steinberger
- Department of Neurosurgery, Mount Sinai Hospital, Icahn School of Medicine at Mount Sinai, New York, NY, USA
| |
Collapse
|
77
|
Dibble CF, Molina CA. Device profile of the XVision-spine (XVS) augmented-reality surgical navigation system: overview of its safety and efficacy. Expert Rev Med Devices 2020; 18:1-8. [PMID: 33322948 DOI: 10.1080/17434440.2021.1865795] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/22/2022]
Abstract
Introduction: The field of augmented reality mediated spine surgery is growing rapidly and holds great promise for improving surgical capabilities and patient outcomes. Augmented reality can assist with complex or atypical cases involving challenging anatomy. As neuronavigation evolves, fundamental technical limitations remain in line-of-sight interruption and operator attention shift, which this novel augmented reality technology helps to address.Areas covered: XVision is a recently FDA-approved head mounted display for intraoperative neuronavigation, compatible with all current conventional pedicle screw technology. The device is a wireless, customizable headset with an integrated surgical tracking system and transparent retinal display. This review discusses the available literature on the safety and efficacy of XVision, as well as the current state of augmented reality technology in spine surgery.Expert opinion: Augmented-reality spine surgery is an emerging technology that may increase precision, efficiency, and safety as well as decrease radiation exposure of manual and robotic computer-navigated pedicle screw insertion techniques. The initial clinical experience with XVision has shown good outcomes and it has received positive operator feedback. Now that initial clinical safety and efficacy has been demonstrated, ongoing experience must be studied to empirically validate this technology and generate further innovation in this rapidly evolving field.
Collapse
Affiliation(s)
- Christopher F Dibble
- Department of Neurosurgery, Washington University School of Medicine, Saint Louis, USA
| | - Camilo A Molina
- Department of Neurosurgery, Washington University School of Medicine, Saint Louis, USA
| |
Collapse
|
78
|
Kiarostami P, Dennler C, Roner S, Sutter R, Fürnstahl P, Farshad M, Rahm S, Zingg PO. Augmented reality-guided periacetabular osteotomy-proof of concept. J Orthop Surg Res 2020; 15:540. [PMID: 33203429 PMCID: PMC7672946 DOI: 10.1186/s13018-020-02066-x] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/21/2020] [Accepted: 11/04/2020] [Indexed: 01/06/2023] Open
Abstract
BACKGROUND The Ganz' periacetabular osteotomy (PAO) consists of four technically challenging osteotomies (OT), namely, supraacetabular (saOT), pubic (pOT), ischial (iOT), and retroacetabular OT (raOT). PURPOSE We performed a proof of concept study to test (1) the feasibility of augmented reality (AR) guidance for PAO, (2) precision of the OTs guided by AR compared to the freehand technique performed by an experienced PAO surgeon, and (3) the effect of AR on performance depending on experience. METHODS A 3D preoperative plan of a PAO was created from segmented computed tomography (CT) data of an anatomic plastic pelvis model (PPM). The plan was then embedded in a software application for an AR head-mounted device. Soft tissue coverage was imitated using foam rubber. The 3D plan was then registered onto the PPM using an anatomical landmark registration. Two surgeons (one experienced and one novice PAO surgeon) each performed 15 freehand (FH) and 15 AR-guided PAOs. The starting point distances and angulation between the planned and executed OT planes for the FH and the AR-guided PAOs were compared in post-intervention CTs. RESULTS AR guidance did not affect the performance of the expert surgeon in terms of the mean differences between the planned and executed starting points, but the raOT angle was more accurate as compared to FH PAO (p = 0.0027). AR guidance increased the accuracy of the performance of the novice surgeon for iOT (p = 0.03). An intraarticular osteotomy performed by the novice surgeon with the FH technique could be observed only once. CONCLUSION AR guidance of osteotomies for PAOs is feasible and seems to increase accuracy. The effect is more accentuated for less-experienced surgeons. CLINICAL RELEVANCE This is the first proof of concept study documenting the feasibility of AR guidance for PAO. Based on these findings, further studies are essential for elaborating on the potential merits of AR guidance to increase the accuracy of complex surgical procedures.
Collapse
Affiliation(s)
- Pascal Kiarostami
- Department of Orthopaedics, Balgrist University Hospital, University of Zürich, Forchstrasse 340, 8008 Zürich, Switzerland
| | - Cyrill Dennler
- Department of Orthopaedics, Balgrist University Hospital, University of Zürich, Forchstrasse 340, 8008 Zürich, Switzerland
| | - Simon Roner
- Department of Orthopaedics, Balgrist University Hospital, University of Zürich, Forchstrasse 340, 8008 Zürich, Switzerland
| | - Reto Sutter
- Department of Radiology, Balgrist University Hospital, University of Zürich, Forchstrasse 340, 8008 Zürich, Switzerland
| | - Philipp Fürnstahl
- Computer Assisted Research & Development Group, Balgrist University Hospital, University of Zürich, Forchstrasse 340, 8008 Zürich, Switzerland
| | - Mazda Farshad
- Department of Orthopaedics, Balgrist University Hospital, University of Zürich, Forchstrasse 340, 8008 Zürich, Switzerland
| | - Stefan Rahm
- Department of Orthopaedics, Balgrist University Hospital, University of Zürich, Forchstrasse 340, 8008 Zürich, Switzerland
| | - Patrick O. Zingg
- Department of Orthopaedics, Balgrist University Hospital, University of Zürich, Forchstrasse 340, 8008 Zürich, Switzerland
| |
Collapse
|
79
|
von Atzigen M, Liebmann F, Hoch A, Bauer DE, Snedeker JG, Farshad M, Fürnstahl P. HoloYolo: A proof-of-concept study for marker-less surgical navigation of spinal rod implants with augmented reality and on-device machine learning. Int J Med Robot 2020; 17:1-10. [PMID: 33073908 DOI: 10.1002/rcs.2184] [Citation(s) in RCA: 17] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/03/2020] [Revised: 10/12/2020] [Accepted: 10/14/2020] [Indexed: 12/14/2022]
Abstract
BACKGROUND Existing surgical navigation approaches of the rod bending procedure in spinal fusion rely on optical tracking systems that determine the location of placed pedicle screws using a hand-held marker. METHODS We propose a novel, marker-less surgical navigation proof-of-concept to bending rod implants. Our method combines augmented reality with on-device machine learning to generate and display a virtual template of the optimal rod shape without touching the instrumented anatomy. Performance was evaluated on lumbosacral spine phantoms against a pointer-based navigation benchmark approach and ground truth data obtained from computed tomography. RESULTS Our method achieved a mean error of 1.83 ± 1.10 mm compared to 1.87 ± 1.31 mm measured in the marker-based approach, while only requiring 21.33 ± 8.80 s as opposed to 36.65 ± 7.49 s attained by the pointer-based method. CONCLUSION Our results suggests that the combination of augmented reality and machine learning has the potential to replace conventional pointer-based navigation in the future.
Collapse
Affiliation(s)
- Marco von Atzigen
- Research in Orthopedic Computer Science, Balgrist University Hospital, University of Zurich, Zurich, Switzerland.,Laboratory for Orthopaedic Biomechanics, ETH Zurich, Zurich, Switzerland
| | - Florentin Liebmann
- Research in Orthopedic Computer Science, Balgrist University Hospital, University of Zurich, Zurich, Switzerland.,Laboratory for Orthopaedic Biomechanics, ETH Zurich, Zurich, Switzerland
| | - Armando Hoch
- Research in Orthopedic Computer Science, Balgrist University Hospital, University of Zurich, Zurich, Switzerland.,Orthopaedic Department, Balgrist University Hospital, University of Zurich, Zurich, Switzerland
| | - David E Bauer
- Orthopaedic Department, Balgrist University Hospital, University of Zurich, Zurich, Switzerland
| | - Jess Gerrit Snedeker
- Laboratory for Orthopaedic Biomechanics, ETH Zurich, Zurich, Switzerland.,Orthopaedic Department, Balgrist University Hospital, University of Zurich, Zurich, Switzerland
| | - Mazda Farshad
- Orthopaedic Department, Balgrist University Hospital, University of Zurich, Zurich, Switzerland
| | - Philipp Fürnstahl
- Research in Orthopedic Computer Science, Balgrist University Hospital, University of Zurich, Zurich, Switzerland
| |
Collapse
|
80
|
Gibby J, Cvetko S, Javan R, Parr R, Gibby W. Use of augmented reality for image-guided spine procedures. EUROPEAN SPINE JOURNAL : OFFICIAL PUBLICATION OF THE EUROPEAN SPINE SOCIETY, THE EUROPEAN SPINAL DEFORMITY SOCIETY, AND THE EUROPEAN SECTION OF THE CERVICAL SPINE RESEARCH SOCIETY 2020; 29:1823-1832. [DOI: 10.1007/s00586-020-06495-4] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/22/2019] [Revised: 04/07/2020] [Accepted: 05/31/2020] [Indexed: 12/14/2022]
|
81
|
Dennler C, Jaberg L, Spirig J, Agten C, Götschi T, Fürnstahl P, Farshad M. Augmented reality-based navigation increases precision of pedicle screw insertion. J Orthop Surg Res 2020; 15:174. [PMID: 32410636 PMCID: PMC7227090 DOI: 10.1186/s13018-020-01690-x] [Citation(s) in RCA: 41] [Impact Index Per Article: 10.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/01/2019] [Accepted: 04/29/2020] [Indexed: 02/07/2023] Open
Abstract
Background Precise insertion of pedicle screws is important to avoid injury to closely adjacent neurovascular structures. The standard method for the insertion of pedicle screws is based on anatomical landmarks (free-hand technique). Head-mounted augmented reality (AR) devices can be used to guide instrumentation and implant placement in spinal surgery. This study evaluates the feasibility and precision of AR technology to improve precision of pedicle screw insertion compared to the current standard technique. Methods Two board-certified orthopedic surgeons specialized in spine surgery and two novice surgeons were each instructed to drill pilot holes for 40 pedicle screws in eighty lumbar vertebra sawbones models in an agar-based gel. One hundred and sixty pedicles were randomized into two groups: the standard free-hand technique (FH) and augmented reality technique (AR). A 3D model of the vertebral body was superimposed over the AR headset. Half of the pedicles were drilled using the FH method, and the other half using the AR method. Results The average minimal distance of the drill axis to the pedicle wall (MAPW) was similar in both groups for expert surgeons (FH 4.8 ± 1.0 mm vs. AR 5.0 ± 1.4 mm, p = 0.389) but for novice surgeons (FH 3.4 mm ± 1.8 mm, AR 4.2 ± 1.8 mm, p = 0.044). Expert surgeons showed 0 primary drill pedicle perforations (PDPP) in both the FH and AR groups. Novices showed 3 (7.5%) PDPP in the FH group and one perforation (2.5%) in the AR group, respectively (p > 0.005). Experts showed no statistically significant difference in average secondary screw pedicle perforations (SSPP) between the AR and the FH set 6-, 7-, and 8-mm screws (p > 0.05). Novices showed significant differences of SSPP between most groups: 6-mm screws, 18 (45%) vs. 7 (17.5%), p = 0.006; 7-mm screws, 20 (50%) vs. 10 (25%), p = 0.013; and 8-mm screws, 22 (55%) vs. 15 (37.5%), p = 0.053, in the FH and AR group, respectively. In novices, the average optimal medio-lateral convergent angle (oMLCA) was 3.23° (STD 4.90) and 0.62° (STD 4.56) for the FH and AR set screws (p = 0.017), respectively. Novices drilled with a higher precision with respect to the cranio-caudal inclination angle (CCIA) category (p = 0.04) with AR. Conclusion In this study, the additional anatomical information provided by the AR headset superimposed to real-world anatomy improved the precision of drilling pilot holes for pedicle screws in a laboratory setting and decreases the effect of surgeon’s experience. Further technical development and validations studies are currently being performed to investigate potential clinical benefits of the herein described AR-based navigation approach.
Collapse
Affiliation(s)
- Cyrill Dennler
- Spine Division, University Hospital Balgrist, University of Zürich, Forchstrasse 340, 8008, Zurich, Switzerland.
| | - Laurenz Jaberg
- Spine Division, University Hospital Balgrist, University of Zürich, Forchstrasse 340, 8008, Zurich, Switzerland
| | - José Spirig
- Spine Division, University Hospital Balgrist, University of Zürich, Forchstrasse 340, 8008, Zurich, Switzerland
| | - Christoph Agten
- Department of Radiology, University Hospital Balgrist, University of Zürich, Zurich, Switzerland
| | - Tobias Götschi
- Laboratory for Biomechanics, University Hospital Balgrist, University of Zürich, Zurich, Switzerland
| | - Philipp Fürnstahl
- Computer Assisted Research and Development Group, University Hospital Balgrist, University of Zürich, Zurich, Switzerland
| | - Mazda Farshad
- Spine Division, University Hospital Balgrist, University of Zürich, Forchstrasse 340, 8008, Zurich, Switzerland
| |
Collapse
|
82
|
Neves CA, Vaisbuch Y, Leuze C, McNab JA, Daniel B, Blevins NH, Hwang PH. Application of holographic augmented reality for external approaches to the frontal sinus. Int Forum Allergy Rhinol 2020; 10:920-925. [PMID: 32362076 DOI: 10.1002/alr.22546] [Citation(s) in RCA: 18] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/05/2019] [Revised: 01/11/2020] [Accepted: 02/05/2020] [Indexed: 12/16/2022]
Abstract
BACKGROUND External approaches to the frontal sinus such as osteoplastic flaps are challenging because they require blind entry into the sinus, posing risks of injury to the brain or orbit. Intraoperative computed tomography (CT)-based navigation is the current standard for planning the approach, but still necessitates blind entry into the sinus. The aim of this work was to describe a novel technique for external approaches to the frontal sinus using a holographic augmented reality (AR) application. METHODS Our team developed an AR system to create a 3-dimensional (3D) hologram of key anatomical structures, based on CT scans images. Using Magic Leap AR goggles for visualization, the frontal sinus hologram was aligned to the surface anatomy in 6 fresh cadaveric heads' anatomic boundaries, and the boundaries of the frontal sinus were demarcated based on the margins of the fused image. Trephinations and osteoplastic flap approaches were performed. The specimens were re-scanned to assess the accuracy of the osteotomy with respect to the actual frontal sinus perimeter. RESULTS Registration and surgery were completed successfully in all specimens. Registration required an average of 2 minutes. The postprocedure CT showed a mean difference of 1.4 ± 4.1 mm between the contour of the osteotomy and the contour of the frontal sinus. One surgical complication (posterior table perforation) occurred (16%). CONCLUSION We describe proof of concept of a novel technique utilizing AR to enhance external approaches to the frontal sinus. Holographic AR-enhanced surgical navigation holds promise for enhanced visualization of target structures during surgical approaches to the sinuses.
Collapse
Affiliation(s)
- Caio A Neves
- Department of Otolaryngology-Head & Neck Surgery, Stanford University School of Medicine, Stanford, CA.,Faculty of Medicine, University of Brasília, Brasilia, Brazil
| | - Yona Vaisbuch
- Department of Otolaryngology-Head & Neck Surgery, Stanford University School of Medicine, Stanford, CA.,Department of Otolaryngology-Head & Neck Surgery, Rambam Medical Center, Haifa, Israel
| | - Christoph Leuze
- Department of Radiology, Stanford University School of Medicine, Stanford, CA
| | - Jennifer A McNab
- Department of Radiology, Stanford University School of Medicine, Stanford, CA
| | - Bruce Daniel
- Department of Radiology, Stanford University School of Medicine, Stanford, CA
| | - Nikolas H Blevins
- Department of Otolaryngology-Head & Neck Surgery, Stanford University School of Medicine, Stanford, CA
| | - Peter H Hwang
- Department of Otolaryngology-Head & Neck Surgery, Stanford University School of Medicine, Stanford, CA
| |
Collapse
|
83
|
Abstract
STUDY DESIGN A prospective, case-based, observational study. OBJECTIVES To investigate how microscope-based augmented reality (AR) support can be utilized in various types of spine surgery. METHODS In 42 spinal procedures (12 intra- and 8 extradural tumors, 7 other intradural lesions, 11 degenerative cases, 2 infections, and 2 deformities) AR was implemented using operating microscope head-up displays (HUDs). Intraoperative low-dose computed tomography was used for automatic registration. Nonlinear image registration was applied to integrate multimodality preoperative images. Target and risk structures displayed by AR were defined in preoperative images by automatic anatomical mapping and additional manual segmentation. RESULTS AR could be successfully applied in all 42 cases. Low-dose protocols ensured a low radiation exposure for registration scanning (effective dose cervical 0.29 ± 0.17 mSv, thoracic 3.40 ± 2.38 mSv, lumbar 3.05 ± 0.89 mSv). A low registration error (0.87 ± 0.28 mm) resulted in a reliable AR representation with a close matching of visualized objects and reality, distinctly supporting anatomical orientation in the surgical field. Flexible AR visualization applying either the microscope HUD or video superimposition, including the ability to selectively activate objects of interest, as well as different display modes allowed a smooth integration in the surgical workflow, without disturbing the actual procedure. On average, 7.1 ± 4.6 objects were displayed visualizing target and risk structures reliably. CONCLUSIONS Microscope-based AR can be applied successfully to various kinds of spinal procedures. AR improves anatomical orientation in the surgical field supporting the surgeon, as well as it offers a potential tool for education.
Collapse
Affiliation(s)
- Barbara Carl
- Department of Neurosurgery, University Marburg, Marburg, Germany
| | - Miriam Bopp
- Department of Neurosurgery, University Marburg, Marburg, Germany
- Marburg Center for Mind, Brain and Behavior (MCMBB), Marburg, Germany
| | - Benjamin Saß
- Department of Neurosurgery, University Marburg, Marburg, Germany
| | - Mirza Pojskic
- Department of Neurosurgery, University Marburg, Marburg, Germany
| | | | - Christopher Nimsky
- Department of Neurosurgery, University Marburg, Marburg, Germany
- Marburg Center for Mind, Brain and Behavior (MCMBB), Marburg, Germany
| |
Collapse
|
84
|
Müller F, Roner S, Liebmann F, Spirig JM, Fürnstahl P, Farshad M. Augmented reality navigation for spinal pedicle screw instrumentation using intraoperative 3D imaging. Spine J 2020; 20:621-628. [PMID: 31669611 DOI: 10.1016/j.spinee.2019.10.012] [Citation(s) in RCA: 51] [Impact Index Per Article: 12.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/06/2019] [Revised: 10/19/2019] [Accepted: 10/21/2019] [Indexed: 02/03/2023]
Abstract
BACKGROUND CONTEXT Due to recent developments in augmented reality with head-mounted devices, holograms of a surgical plan can be displayed directly in the surgeon's field of view. To the best of our knowledge, three dimensional (3D) intraoperative fluoroscopy has not been explored for the use with holographic navigation by head-mounted devices in spine surgery. PURPOSE To evaluate the surgical accuracy of holographic pedicle screw navigation by head-mounted device using 3D intraoperative fluoroscopy. STUDY DESIGN In this experimental cadaver study, the accuracy of surgical navigation using a head-mounted device was compared with navigation with a state-of-the-art pose-tracking system. METHODS Three lumbar cadaver spines were embedded in nontransparent agar gel, leaving only commonly visible anatomy in sight. Intraoperative registration of preoperative planning was achieved by 3D fluoroscopy and fiducial markers attached to lumbar vertebrae. Trackable custom-made drill sleeve guides enabled real-time navigation. In total, 20 K-wires were navigated into lumbar pedicles using AR-navigation, 10 K-wires by the state-of-the-art pose-tracking system. 3D models obtained from postexperimental CT scans were used to measure surgical accuracy. MF is the founder and shareholder of Incremed AG, a Balgrist University Hospital start-up focusing on the development of innovative techniques for surgical executions. The other authors declare no conflict of interest concerning the contents of this study. No external funding was received for this study. RESULTS No significant difference in accuracy was measured between AR-navigated drillings and the gold standard with pose-tracking system with mean translational errors between entry points (3D vector distance; p=.85) of 3.4±1.6 mm compared with 3.2±2.0 mm, and mean angular errors between trajectories (3D angle; p=.30) of 4.3°±2.3° compared with 3.5°±1.4°. CONCLUSIONS In conclusion, holographic navigation by use of a head-mounted device achieve accuracy comparable to the gold standard of high-end pose-tracking systems. CLINICAL SIGNIFICANCE These promising results could result in a new way of surgical navigation with minimal infrastructural requirements but now have to be confirmed in clinical studies.
Collapse
Affiliation(s)
- Fabio Müller
- Department of Orthopedics, Balgrist University Hospital, University of Zurich, Forchstrasse 340, 8008 Zurich, Switzerland.
| | - Simon Roner
- Department of Orthopedics, Balgrist University Hospital, University of Zurich, Forchstrasse 340, 8008 Zurich, Switzerland
| | - Florentin Liebmann
- Computer Assisted Research and Development Group, Balgrist University Hospital, University of Zurich, Lengghalde 5, 8008 Zurich, Switzerland; Laboratory for Orthopedic Biomechanics, ETH Zurich, Forchstrasse 328, 8008 Zurich, Switzerland
| | - José M Spirig
- Department of Orthopedics, Balgrist University Hospital, University of Zurich, Forchstrasse 340, 8008 Zurich, Switzerland
| | - Philipp Fürnstahl
- Computer Assisted Research and Development Group, Balgrist University Hospital, University of Zurich, Lengghalde 5, 8008 Zurich, Switzerland
| | - Mazda Farshad
- Department of Orthopedics, Balgrist University Hospital, University of Zurich, Forchstrasse 340, 8008 Zurich, Switzerland
| |
Collapse
|
85
|
Vadalà G, De Salvatore S, Ambrosio L, Russo F, Papalia R, Denaro V. Robotic Spine Surgery and Augmented Reality Systems: A State of the Art. Neurospine 2020; 17:88-100. [PMID: 32252158 PMCID: PMC7136092 DOI: 10.14245/ns.2040060.030] [Citation(s) in RCA: 36] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/02/2020] [Accepted: 02/24/2020] [Indexed: 12/26/2022] Open
Abstract
Instrumented spine procedures have been performed for decades to treat a wide variety of spinal disorders. New technologies have been employed to obtain a high degree of precision, to minimize risks of damage to neurovascular structures and to diminish harmful exposure of patients and the operative team to ionizing radiations. Robotic spine surgery comprehends 3 major categories: telesurgical robotic systems, robotic-assisted navigation (RAN) and virtual augmented reality (AR) systems, including AR and virtual reality. Telesurgical systems encompass devices that can be operated from a remote command station, allowing to perform surgery via instruments being manipulated by the robot. On the other hand, RAN technologies are characterized by the robotic guidance of surgeon-operated instruments based on real-time imaging. Virtual AR systems are able to show images directly on special visors and screens allowing the surgeon to visualize information about the patient and the procedure (i.e., anatomical landmarks, screw direction and inclination, distance from neurological and vascular structures etc.). The aim of this review is to focus on the current state of the art of robotics and AR in spine surgery and perspectives of these emerging technologies that hold promises for future applications.
Collapse
Affiliation(s)
- Gianluca Vadalà
- Department of Orthopaedic and Trauma Surgery, Campus Bio-Medico University of Rome, Rome, Italy
| | - Sergio De Salvatore
- Department of Orthopaedic and Trauma Surgery, Campus Bio-Medico University of Rome, Rome, Italy
| | - Luca Ambrosio
- Department of Orthopaedic and Trauma Surgery, Campus Bio-Medico University of Rome, Rome, Italy
| | - Fabrizio Russo
- Department of Orthopaedic and Trauma Surgery, Campus Bio-Medico University of Rome, Rome, Italy
| | - Rocco Papalia
- Department of Orthopaedic and Trauma Surgery, Campus Bio-Medico University of Rome, Rome, Italy
| | - Vincenzo Denaro
- Department of Orthopaedic and Trauma Surgery, Campus Bio-Medico University of Rome, Rome, Italy
| |
Collapse
|
86
|
Yu C, Ou Y, Xie C, Zhang Y, Wei J, Mu X. Pedicle screw placement in spinal neurosurgery using a 3D-printed drill guide template: a systematic review and meta-analysis. J Orthop Surg Res 2020; 15:1. [PMID: 31900192 PMCID: PMC6942326 DOI: 10.1186/s13018-019-1510-5] [Citation(s) in RCA: 54] [Impact Index Per Article: 13.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/31/2019] [Accepted: 12/05/2019] [Indexed: 12/26/2022] Open
Abstract
Background Many surgeons believe that the use of a 3D-printed drill guide template shortens operative time and reduces intraoperative blood loss compared with those of the free-hand technique. In this study, we investigated the effects of a drill guide template on the accuracy of pedicle screw placement (the screw placed completely in the pedicle), operative time, and intraoperative blood loss. Materials/Methods We systematically searched the major databases, such as Medline via PubMed, EMBASE, Ovid, Cochrane Library, and Google Scholar, regarding the accuracy of pedicle screw placement, operative time, and intraoperative blood loss. The χ2 test and I2 statistic were used to examine heterogeneity. Odds ratios (ORs) with 95% confidence intervals (CIs) were used to calculate the accuracy rate of pedicle screw placement, and weighted mean differences (WMDs) with 95% CIs were utilized to express operative time and intraoperative blood loss. Results This meta-analysis included 13 studies (seven randomized controlled trials and six prospective cohort studies) involving 446 patients and 3375 screws. The risk of research bias was considered moderate. Operative time (WMD = − 20.75, 95% CI − 33.20 ~ − 8.29, P = 0.001) and intraoperative blood loss (WMD = − 106.16, 95% CI − 185.35 ~ − 26.97, P = 0.009) in the thoracolumbar vertebrae, evaluated by a subgroup analysis, were significantly different between groups. The 3D-printed drill guide template has advantages over the free-hand technique and improves the accuracy of pedicle screw placement (OR = 2.88; 95% CI, 2.39~3.47; P = 0.000). Conclusion The 3D-printed drill guide template can improve the accuracy rate of pedicle screw placement, shorten operative time, and reduce intraoperative blood loss.
Collapse
Affiliation(s)
- Chengqiang Yu
- Department of Orthopaedics, The People's Hospital of Guangxi Zhuang Autonomous Region, Nanning, Guangxi, 530021, China
| | - Yufu Ou
- Department of Orthopaedics, The People's Hospital of Guangxi Zhuang Autonomous Region, Nanning, Guangxi, 530021, China
| | - Chengxin Xie
- Department of Orthopaedics, The People's Hospital of Guangxi Zhuang Autonomous Region, Nanning, Guangxi, 530021, China
| | - Yu Zhang
- Department of Orthopaedics, The People's Hospital of Guangxi Zhuang Autonomous Region, Nanning, Guangxi, 530021, China
| | - Jianxun Wei
- Department of Orthopaedics, The People's Hospital of Guangxi Zhuang Autonomous Region, Nanning, Guangxi, 530021, China.
| | - Xiaoping Mu
- Department of Orthopaedics, The People's Hospital of Guangxi Zhuang Autonomous Region, Nanning, Guangxi, 530021, China.
| |
Collapse
|
87
|
Carl B, Bopp M, Saß B, Pojskic M, Nimsky C. Augmented reality in intradural spinal tumor surgery. Acta Neurochir (Wien) 2019; 161:2181-2193. [PMID: 31300886 DOI: 10.1007/s00701-019-04005-0] [Citation(s) in RCA: 36] [Impact Index Per Article: 7.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/06/2019] [Accepted: 07/05/2019] [Indexed: 02/07/2023]
Abstract
BACKGROUND Microscope-based augmented reality (AR) is commonly used in cranial surgery; however, until recently, this technique was not implemented for spinal surgery. We prospectively investigated, how AR can be applied for intradural spinal tumor surgery. METHODS For ten patients with intradural spinal tumors (ependymoma, glioma, hemangioblastoma, meningioma, and metastasis), AR was provided by head-up displays (HUDs) of operating microscopes. User-independent automatic AR registration was established by low-dose intraoperative computed tomography. The objects visualized by AR were segmented in preoperative imaging data; non-linear image registration was applied to consider spine flexibility. RESULTS In all cases, AR supported surgery by visualizing the tumor outline and other relevant surrounding structures. The overall AR registration error was 0.72 ± 0.24 mm (mean ± standard deviation), a close matching of visible tumor outline and AR visualization was observed for all cases. Registration scanning resulted in a low effective dose of 0.22 ± 0.16 mSv for cervical and 1.68 ± 0.61 mSv for thoracic lesions. The mean HUD AR usage in relation to microscope time was 51.6 ± 36.7%. The HUD was switched off and turned on again in a range of 2 to 17 times (5.7 ± 4.4 times). Independent of the status of the HUD, the AR visualization was displayed on monitors throughout surgery. CONCLUSIONS Microscope-based AR can be reliably applied to intradural spinal tumor surgery. Automatic AR registration ensures high precision and provides an intuitive visualization of the extent of the tumor and surrounding structures. Given this setting, all advanced multi-modality options of cranial AR can also be applied to spinal surgery.
Collapse
|