A Wearable Augmented Reality Navigation System for Surgical Telementoring Based on Microsoft HoloLens

Integration of smart glasses in telementoring for simulated medical procedures

PURPOSE

To demonstrate the complete process of telementoring implementation using augmented-reality smart glasses for performing a simulated medical procedure.

METHODS

Fifteen participants, including physicians and medical students, were recruited to participate in a telementoring session with an educational focus during simulated thoracostomy with closed pleural drainage. A questionnaire assessing technology usability and usefulness was administered, and the results were analyzed by Cronbach's alfa and multiple correspondence analysis (MCA).

RESULTS

The mean age of the participants was 28.8 and 66.67% of them were women. The test results indicated a Cronbach's alpha of 78% and an MCA of 49.48% for the first three dimensions.

CONCLUSION

The results showed that smart glasses are easy to use and facilitate communication among the professionals involved, providing comfort and safety to participants of care simulations. Additionally, smart glasses were considered to have perceived usefulness.

Current Clinical and Educational Uses of Immersive Reality in Anesthesia: Narrative Review

BACKGROUND

The concept of immersive reality (IR), an umbrella term that encompasses virtual reality, augmented reality, and mixed reality, has been established within the health care realm as a potentially valuable tool with numerous applications in both medical education and patient care.

OBJECTIVE

This review aimed to introduce anesthesiologists to the emerging and rapidly evolving literature on IR, its use in anesthesia education, and its transferability into the clinical context.

METHODS

A review of the relevant literature was conducted using the PubMed database from inception to July 5, 2023. Additional references were identified from the reference lists of selected papers.

RESULTS

A total of 51 papers related to the use of IR in anesthesia medical education (including both technical and nontechnical skills) and 63 papers related to applications in clinical practice (eg, preprocedure planning, patient education, and pain management) were included. We present evidence supporting the use of IR in the training and clinical practice of modern anesthesiologists.

CONCLUSIONS

IR is useful for a variety of applications in anesthesia medical education and has potential advantages over existing simulation approaches. Similarly, IR has demonstrated potential improvements in patient care across several clinical contexts relevant to practicing anesthesiologists. However, many applications remain in the early stages of development, and robust trials are urgently needed to confirm clinical or educational effectiveness and to assess mechanisms, educational validity, and cost-effectiveness.

A holographic telementoring system depicting surgical instrument movements for real-time guidance in open surgeries

BACKGROUND AND OBJECTIVE

During open surgeries, telementoring serves as a valuable tool for transferring surgical knowledge from a specialist surgeon (mentor) to an operating surgeon (mentee). Depicting the intended movements of the surgical instruments over the operative field improves the understanding of the required tool-tissue interaction. The objective of this work is to develop a telementoring system tailored for open surgeries, enabling the mentor to remotely demonstrate the necessary motions of surgical instruments to the mentee.

METHODS

A remote telementoring system for open surgery was implemented. The system generates visual cues in the form of virtual surgical instrument motion augmented onto the live view of the operative field. These cues can be rendered on both conventional screens in the operating room and as dynamic holograms on a head mounted display device worn by the mentee. The technical performance of the system was evaluated, where the operating room and remote location were geographically separated and connected via the Internet. Additionally, user studies were conducted to assess the effectiveness of the system as a mentoring tool.

RESULTS

The system took 307 ± 12 ms to transmit an operative field view of 1920  ×  1080 resolution, along with depth information spanning 36 cm, from the operating room to the remote location. Conversely, it took 145 ± 14 ms to receive the motion of virtual surgical instruments from the remote location back to the operating room. Furthermore, the user studies demonstrated: (a) mentor's capability to annotate the operative field with an accuracy of 3.92 ± 2.1 mm, (b) mentee's ability to comprehend and replicate the motion of surgical instruments in real-time with an average deviation of 12.8 ± 3 mm, (c) efficacy of the rendered dynamic holograms in conveying information intended for surgical instrument motion.

CONCLUSIONS

The study demonstrates the feasibility of transmitting information over the Internet from the mentor to the mentee in the form of virtual surgical instruments' motion and projecting it as holograms onto the live view of the operative field. This holds potential to enhance real-time collaborative capabilities between the mentor and the mentee during an open surgery.

Extended and augmented reality in vascular surgery: Opportunities and challenges

Extended reality has brought new opportunities for medical imaging visualization and analysis. It regroups various subfields, including virtual reality, augmented reality, and mixed reality. Various applications have been proposed for surgical practice, as well as education and training. The aim of this review was to summarize current applications of extended reality and augmented reality in vascular surgery, highlighting potential benefits, pitfalls, limitations, and perspectives on improvement.

Application of image recognition-based tracker-less augmented reality navigation system in a series of sawbone trials

BACKGROUND

This study introduced an Augmented Reality (AR) navigation system to address limitations in conventional high tibial osteotomy (HTO). The objective was to enhance precision and efficiency in HTO procedures, overcoming challenges such as inconsistent postoperative alignment and potential neurovascular damage.

METHODS

The AR-MR (Mixed Reality) navigation system, comprising HoloLens, Unity Engine, and Vuforia software, was employed for pre-clinical trials using tibial sawbone models. CT images generated 3D anatomical models, projected via HoloLens, allowing surgeons to interact through intuitive hand gestures. The critical procedure of target tracking, essential for aligning virtual and real objects, was facilitated by Vuforia's feature detection algorithm.

RESULTS

In trials, the AR-MR system demonstrated significant reductions in both preoperative planning and intraoperative times compared to conventional navigation and metal 3D-printed surgical guides. The AR system, while exhibiting lower accuracy, exhibited efficiency, making it a promising option for HTO procedures. The preoperative planning time for the AR system was notably shorter (4 min) compared to conventional navigation (30.5 min) and metal guides (75.5 min). Intraoperative time for AR lasted 8.5 min, considerably faster than that of conventional navigation (31.5 min) and metal guides (10.5 min).

CONCLUSIONS

The AR navigation system presents a transformative approach to HTO, offering a trade-off between accuracy and efficiency. Ongoing improvements, such as the incorporation of two-stage registration and pointing devices, could further enhance precision. While the system may be less accurate, its efficiency renders it a potential breakthrough in orthopedic surgery, particularly for reducing unnecessary harm and streamlining surgical procedures.

Technologies Used for Telementoring in Open Surgery: A Scoping Review

Background: Telementoring technologies enable a remote mentor to guide a mentee in real-time during surgical procedures. This addresses challenges, such as lack of expertise and limited surgical training/education opportunities in remote locations. This review aims to provide a comprehensive account of these technologies tailored for open surgery. Methods: A comprehensive scoping review of the scientific literature was conducted using PubMed, ScienceDirect, ACM Digital Library, and IEEE Xplore databases. Broad and inclusive searches were done to identify articles reporting telementoring or teleguidance technologies in open surgery. Results: Screening of the search results yielded 43 articles describing surgical telementoring for open approach. The studies were categorized based on the type of open surgery (surgical specialty, surgical procedure, and stage of clinical trial), the telementoring technology used (information transferred between mentor and mentee, devices used for rendering the information), and assessment of the technology (experience level of mentor and mentee, study design, and assessment criteria). Majority of the telementoring technologies focused on trauma-related surgeries and mixed reality headsets were commonly used for rendering information (telestrations, surgical tools, or hand gestures) to the mentee. These technologies were primarily assessed on high-fidelity synthetic phantoms. Conclusions: Despite longer operative time, these telementoring technologies demonstrated clinical viability during open surgeries through improved performance and confidence of the mentee. In general, usage of immersive devices and annotations appears to be promising, although further clinical trials will be required to thoroughly assess its benefits.

Telemedicine-Guided Two-Incision Lower Leg Fasciotomy Performed by Combat Medics During Tactical Combat Casualty Care: A Feasibility Study

INTRODUCTION

During tactical combat casualty care, life- and limb-saving procedures might also be performed by combat medics. This study assesses whether it is feasible to use a head-mounted display (HMD) to provide telemedicine (TM) support from a consulted senior surgeon for combat medics when performing a two-incision lower leg fasciotomy.

MATERIALS AND METHODS

Nine combat medics were randomized into groups to perform a two-incision lower leg fasciotomy. One group used the Vuzix M400 and the second group used the RealWear HMT-1Z1. A third, control, group received no guidance. In the Vuzix M400 group and RealWear HMT-1Z1 group, a senior surgeon examined the results after the two-incision lower leg fasciotomy was finished to assess the release of compartments, possible collateral damage, and performance of the combat medics. In the control group, these results were examined by a surgical resident with expertise in two-incision lower leg fasciotomies. The resident's operative performance questionnaire was used to score the performance of the combat medics. The telehealth usability questionnaire was used to evaluate the usability of the HMDs as perceived by the combat medics.

RESULTS

Combat medics using an HMD were considered competent in performing a two-incision lower leg fasciotomy (Vuzix: median 3 [range 0], RealWear: median 3 [range 1]). These combat medics had a significantly better score in their ability to adapt to anatomical variances compared to the control group (Vuzix: median 3 [range 0], RealWear: median 3 [range 0], control: median 1 [range 0]; P = .018). Combat medics using an HMD were faster than combat medics in the control group (Vuzix: mean 14:14 [SD 3:41], RealWear: mean 15:42 [SD 1:58], control: mean 17:45 [SD 2:02]; P = .340). The overall satisfaction with both HMDs was 5 out of 7 (Vuzix: median 5 [range 0], RealWear: median 5 [range 1]; P = .317).

CONCLUSIONS

This study shows that it is feasible to use an HMD to provide TM support performance from a consulted senior surgeon for combat medics when performing a two-incision lower leg fasciotomy. The results of this study suggest that TM support might be useful for combat medics during tactical combat casualty care when performing life- and limb-saving procedures.

HoloLens® platform for healthcare professionals simulation training, teaching, and its urological applications: an up-to-date review

The advancements of technological devices and software are putting mixed reality in the frontline of teaching medical personnel. The Microsoft® HoloLens 2® offers a unique 3D visualization of a hologram in a physical, real environment and allows the urologists to interact with it. This review provides a state-of-the-art analysis of the applications of the HoloLens® in a medical and healthcare context of teaching through simulation designed for medical students, nurses, residents especially in urology. Our objective has been to perform a comprehensively analysis of the studies in PubMed/Medline database from January 2016 to April 2023. The identified articles that researched Microsoft HoloLens, having description of feasibility and teaching outcomes in medicine with an emphasize in urological healthcare, have been included. The qualitative analysis performed identifies an increasing use of HoloLens in a teaching setting that covers a great area of expertise in medical sciences (anatomy, anatomic pathology, biochemistry, pharmacogenomics, clinical skills, emergency medicine and nurse education, imaging), and above these urology applications (urological procedures and technique, skill improvement, perception of complex renal tumors, accuracy of calyx puncture guidance in percutaneous nephrolithotomy and targeted biopsy of the prostate) can mostly benefit from it. The future potential of HoloLens technology in teaching is immense. So far, studies have focused on feasibility, applicability, perception, comparisons with traditional methods, and limitations. Moving forward, research should also prioritize the development of applications specifically for urology. This will require validation of needs and the creation of adequate protocols to standardize future research efforts.

Exploring the Potential of Three-Dimensional Imaging, Printing, and Modeling in Pediatric Surgical Oncology: A New Era of Precision Surgery

Pediatric surgical oncology is a technically challenging field that relies on CT and MRI as the primary imaging tools for surgical planning. However, recent advances in 3D reconstructions, including Cinematic Rendering, Volume Rendering, 3D modeling, Virtual Reality, Augmented Reality, and 3D printing, are increasingly being used to plan complex cases bringing new insights into pediatric tumors to guide therapeutic decisions and prognosis in different pediatric surgical oncology areas and locations including thoracic, brain, urology, and abdominal surgery. Despite this, challenges to their adoption remain, especially in soft tissue-based specialties such as pediatric surgical oncology. This work explores the main innovative imaging reconstruction techniques, 3D modeling technologies (CAD, VR, AR), and 3D printing applications through the analysis of three real cases of the most common and surgically challenging pediatric tumors: abdominal neuroblastoma, thoracic inlet neuroblastoma, and a bilateral Wilms tumor candidate for nephron-sparing surgery. The results demonstrate that these new imaging and modeling techniques offer a promising alternative for planning complex pediatric oncological cases. A comprehensive analysis of the advantages and limitations of each technique has been carried out to assist in choosing the optimal approach.

Augmented Reality in Real-time Telemedicine and Telementoring: Scoping Review

BACKGROUND

Over the last decade, augmented reality (AR) has emerged in health care as a tool for visualizing data and enhancing simulation learning. AR, which has largely been explored for communication and collaboration in nonhealth contexts, could play a role in shaping future remote medical services and training. This review summarized existing studies implementing AR in real-time telemedicine and telementoring to create a foundation for health care providers and technology developers to understand future opportunities in remote care and education.

OBJECTIVE

This review described devices and platforms that use AR for real-time telemedicine and telementoring, the tasks for which AR was implemented, and the ways in which these implementations were evaluated to identify gaps in research that provide opportunities for further study.

METHODS

We searched PubMed, Scopus, Embase, and MEDLINE to identify English-language studies published between January 1, 2012, and October 18, 2022, implementing AR technology in a real-time interaction related to telemedicine or telementoring. The search terms were "augmented reality" OR "AR" AND "remote" OR "telemedicine" OR "telehealth" OR "telementoring." Systematic reviews, meta-analyses, and discussion-based articles were excluded from analysis.

RESULTS

A total of 39 articles met the inclusion criteria and were categorized into themes of patient evaluation, medical intervention, and education. In total, 20 devices and platforms using AR were identified, with common features being the ability for remote users to annotate, display graphics, and display their hands or tools in the local user's view. Common themes across the studies included consultation and procedural education, with surgery, emergency, and hospital medicine being the most represented specialties. Outcomes were most often measured using feedback surveys and interviews. The most common objective measures were time to task completion and performance. Long-term outcome and resource cost measurements were rare. Across the studies, user feedback was consistently positive for perceived efficacy, feasibility, and acceptability. Comparative trials demonstrated that AR-assisted conditions had noninferior reliability and performance and did not consistently extend procedure times compared with in-person controls.

CONCLUSIONS

Studies implementing AR in telemedicine and telementoring demonstrated the technology's ability to enhance access to information and facilitate guidance in multiple health care settings. However, AR's role as an alternative to current telecommunication platforms or even in-person interactions remains to be validated, with many disciplines and provider-to-nonprovider uses still lacking robust investigation. Additional studies comparing existing methods may offer more insight into this intersection, but the early stage of technical development and the lack of standardized tools and adoption have hindered the conduct of larger longitudinal and randomized controlled trials. Overall, AR has the potential to complement and advance the capabilities of remote medical care and learning, creating unique opportunities for innovator, provider, and patient involvement.

The HoloLens in medicine: A systematic review and taxonomy

The HoloLens (Microsoft Corp., Redmond, WA), a head-worn, optically see-through augmented reality (AR) display, is the main player in the recent boost in medical AR research. In this systematic review, we provide a comprehensive overview of the usage of the first-generation HoloLens within the medical domain, from its release in March 2016, until the year of 2021. We identified 217 relevant publications through a systematic search of the PubMed, Scopus, IEEE Xplore and SpringerLink databases. We propose a new taxonomy including use case, technical methodology for registration and tracking, data sources, visualization as well as validation and evaluation, and analyze the retrieved publications accordingly. We find that the bulk of research focuses on supporting physicians during interventions, where the HoloLens is promising for procedures usually performed without image guidance. However, the consensus is that accuracy and reliability are still too low to replace conventional guidance systems. Medical students are the second most common target group, where AR-enhanced medical simulators emerge as a promising technology. While concerns about human-computer interactions, usability and perception are frequently mentioned, hardly any concepts to overcome these issues have been proposed. Instead, registration and tracking lie at the core of most reviewed publications, nevertheless only few of them propose innovative concepts in this direction. Finally, we find that the validation of HoloLens applications suffers from a lack of standardized and rigorous evaluation protocols. We hope that this review can advance medical AR research by identifying gaps in the current literature, to pave the way for novel, innovative directions and translation into the medical routine.

Mixed Reality Platforms in Telehealth Delivery: Scoping Review

BACKGROUND

The distinctive features of the digital reality platforms, namely augmented reality (AR), virtual reality (VR), and mixed reality (MR) have extended to medical education, training, simulation, and patient care. Furthermore, this digital reality technology seamlessly merges with information and communication technology creating an enriched telehealth ecosystem. This review provides a composite overview of the prospects of telehealth delivered using the MR platform in clinical settings.

OBJECTIVE

This review identifies various clinical applications of high-fidelity digital display technology, namely AR, VR, and MR, delivered using telehealth capabilities. Next, the review focuses on the technical characteristics, hardware, and software technologies used in the composition of AR, VR, and MR in telehealth.

METHODS

We conducted a scoping review using the methodological framework and reporting design using the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews) guidelines. Full-length articles in English were obtained from the Embase, PubMed, and Web of Science databases. The search protocol was based on the following keywords and Medical Subject Headings to obtain relevant results: "augmented reality," "virtual reality," "mixed-reality," "telemedicine," "telehealth," and "digital health." A predefined inclusion-exclusion criterion was developed in filtering the obtained results and the final selection of the articles, followed by data extraction and construction of the review.

RESULTS

We identified 4407 articles, of which 320 were eligible for full-text screening. A total of 134 full-text articles were included in the review. Telerehabilitation, telementoring, teleconsultation, telemonitoring, telepsychiatry, telesurgery, and telediagnosis were the segments of the telehealth division that explored the use of AR, VR, and MR platforms. Telerehabilitation using VR was the most commonly recurring segment in the included studies. AR and MR has been mainly used for telementoring and teleconsultation. The most important technical features of digital reality technology to emerge with telehealth were virtual environment, exergaming, 3D avatars, telepresence, anchoring annotations, and first-person viewpoint. Different arrangements of technology-3D modeling and viewing tools, communication and streaming platforms, file transfer and sharing platforms, sensors, high-fidelity displays, and controllers-formed the basis of most systems.

CONCLUSIONS

This review constitutes a recent overview of the evolving digital AR and VR in various clinical applications using the telehealth setup. This combination of telehealth with AR, VR, and MR allows for remote facilitation of clinical expertise and further development of home-based treatment. This review explores the rapidly growing suite of technologies available to users within the digital health sector and examines the opportunities and challenges they present.

Remote Interactive Surgery Platform (RISP): Proof of Concept for an Augmented-Reality-Based Platform for Surgical Telementoring

The “Remote Interactive Surgery Platform” (RISP) is an augmented reality (AR)-based platform for surgical telementoring. It builds upon recent advances of mixed reality head-mounted displays (MR-HMD) and associated immersive visualization technologies to assist the surgeon during an operation. It enables an interactive, real-time collaboration with a remote consultant by sharing the operating surgeon’s field of view through the Microsoft (MS) HoloLens2 (HL2). Development of the RISP started during the Medical Augmented Reality Summer School 2021 and is currently still ongoing. It currently includes features such as three-dimensional annotations, bidirectional voice communication and interactive windows to display radiographs within the sterile field. This manuscript provides an overview of the RISP and preliminary results regarding its annotation accuracy and user experience measured with ten participants.

Towards Wearable Augmented Reality in Healthcare: A Comparative Survey and Analysis of Head-Mounted Displays

Head-mounted displays (HMDs) have the potential to greatly impact the surgical field by maintaining sterile conditions in healthcare environments. Google Glass (GG) and Microsoft HoloLens (MH) are examples of optical HMDs. In this comparative survey related to wearable augmented reality (AR) technology in the medical field, we examine the current developments in wearable AR technology, as well as the medical aspects, with a specific emphasis on smart glasses and HoloLens. The authors searched recent articles (between 2017 and 2022) in the PubMed, Web of Science, Scopus, and ScienceDirect databases and a total of 37 relevant studies were considered for this analysis. The selected studies were divided into two main groups; 15 of the studies (around 41%) focused on smart glasses (e.g., Google Glass) and 22 (59%) focused on Microsoft HoloLens. Google Glass was used in various surgical specialities and preoperative settings, namely dermatology visits and nursing skill training. Moreover, Microsoft HoloLens was used in telepresence applications and holographic navigation of shoulder and gait impairment rehabilitation, among others. However, some limitations were associated with their use, such as low battery life, limited memory size, and possible ocular pain. Promising results were obtained by different studies regarding the feasibility, usability, and acceptability of using both Google Glass and Microsoft HoloLens in patient-centric settings as well as medical education and training. Further work and development of rigorous research designs are required to evaluate the efficacy and cost-effectiveness of wearable AR devices in the future.

Mixed reality technology enhances teaching of spinal blockade procedures

To investigate the efficacy of utilizing mixed reality technology-assisted teaching of a spinal medial branch nerve block. Twenty undergraduate students from a 5-year clinical medicine program in Fujian Medical University were selected. They were divided into group A and group B using a random number generator, with 10 students in each group. Group A used the traditional teaching method and Group B used the mixed reality technology-assisted teaching method. At the end of the teaching period, both groups were assessed on the blocking operation, number of punctures required, puncture time, and final error value (distance between the final position and the reference position). A questionnaire was administered to both groups to assess teaching satisfaction. The number of punctures required was 7.40 ± 1.26 and 2.10 ± 0.74 for groups A and B, respectively. The puncture time in group A was 297.80 ± 50.95 s and 65.60 ± 22.02 s in group B. All differences were significant p < 0.01. The final error of the puncture in group A was 2.24 ± 0.35 mm and 1.96 ± 0.26 mm in group B-not significant. Group B had (p < 0.01) higher evaluation scores than group A for teaching effectiveness, learning interest, initiative, and teaching satisfaction. The application of mixed reality technology in the teaching of posterior medial branch blocks of the spinal nerve is superior to previous methods. This method should be adopted wherever possible to enhance learning of this difficult technique.

Use of Extended Reality in Medical Education: An Integrative Review

Extended reality (XR) has emerged as an innovative simulation-based learning modality. An integrative review was undertaken to explore the nature of evidence, usage, and effectiveness of XR modalities in medical education. One hundred and thirty-three (N = 133) studies and articles were reviewed. XR technologies are commonly reported in surgical and anatomical education, and the evidence suggests XR may be as effective as traditional medical education teaching methods and, potentially, a more cost-effective means of curriculum delivery. Further research to compare different variations of XR technologies and best applications in medical education and training are required to advance the field.

Supplementary Information

The online version contains supplementary material available at 10.1007/s40670-022-01698-4.

Emerging trends in rhinoplasty education: accelerated adoption of digital tools and virtual learning platforms

PURPOSE OF REVIEW

The COVID-19 pandemic catalyzed the rapid adoption of digital tools and virtual learning platforms by rhinoplasty educators and trainees alike. This review provides an overview of the variety of digital software and web-based tools rhinoplasty educators have adopted and highlights the advantages and potential drawbacks of virtual learning via e-content.

RECENT FINDINGS

Medical education including subspecialty surgical training has recently undergone a dramatic digital transformation. Rhinoplasty surgeon-educators have been forced to embrace new digital tools, including videoconferencing, podcasts, virtual simulation and social media to reach and teach trainees. Recognizing the advantages of this new, limitless digital space, rhinoplasty surgeons are also engaging in virtual transcontinental collaboration and distance mentorship.

SUMMARY

The dramatic evolution in how clinical educational materials are now digitally created, curated, disseminated and consumed is likely to far outlast the COVID-19 pandemic itself. Rapid, exponential growth of this digital library, however, places increased responsibility on educators to guide trainees towards evidence-based and state-of-the-art content.

Augmented Reality in Surgery: A Scoping Review

Augmented reality (AR) is an innovative system that enhances the real world by superimposing virtual objects on reality. The aim of this study was to analyze the application of AR in medicine and which of its technical solutions are the most used. We carried out a scoping review of the articles published between 2019 and February 2022. The initial search yielded a total of 2649 articles. After applying filters, removing duplicates and screening, we included 34 articles in our analysis. The analysis of the articles highlighted that AR has been traditionally and mainly used in orthopedics in addition to maxillofacial surgery and oncology. Regarding the display application in AR, the Microsoft HoloLens Optical Viewer is the most used method. Moreover, for the tracking and registration phases, the marker-based method with a rigid registration remains the most used system. Overall, the results of this study suggested that AR is an innovative technology with numerous advantages, finding applications in several new surgery domains. Considering the available data, it is not possible to clearly identify all the fields of application and the best technologies regarding AR.

Augmented reality calibration using feature triangulation iteration-based registration for surgical navigation

BACKGROUND

Marker-based augmented reality (AR) calibration methods for surgical navigation often require a second computed tomography scan of the patient, and their clinical application is limited due to high manufacturing costs and low accuracy.

METHODS

This work introduces a novel type of AR calibration framework that combines a Microsoft HoloLens device with a single camera registration module for surgical navigation. A camera is used to gather multi-view images of a patient for reconstruction in this framework. A shape feature matching-based search method is proposed to adjust the size of the reconstructed model. The double clustering-based 3D point cloud segmentation method and 3D line segment detection method are also proposed to extract the corner points of the image marker. The corner points are the registration data of the image marker. A feature triangulation iteration-based registration method is proposed to quickly and accurately calibrate the pose relationship between the image marker and the patient in the virtual and real space. The patient model after registration is wirelessly transmitted to the HoloLens device to display the AR scene.

RESULTS

The proposed approach was used to conduct accuracy verification experiments on the phantoms and volunteers, which were compared with six advanced AR calibration methods. The proposed method obtained average fusion errors of 0.70 ± 0.16 and 0.91 ± 0.13 mm in phantom and volunteer experiments, respectively. The fusion accuracy of the proposed method is the highest among all comparison methods. A volunteer liver puncture clinical simulation experiment was also conducted to show the clinical feasibility.

CONCLUSIONS

Our experiments proved the effectiveness of the proposed AR calibration method, and revealed a considerable potential for improving surgical performance.

Stereotactic co-axial projection imaging for augmented reality neuronavigation: a proof-of-concept study

BACKGROUND

Lack of intuitiveness and poor hand-eye coordination present a major technical challenge in neurosurgical navigation.

METHODS

We developed an integrated dexterous stereotactic co-axial projection imaging (sCPI) system featuring orthotopic image projection for augmented reality (AR) neurosurgical navigation. The performance characteristics of the sCPI system, including projection resolution and navigation accuracy, were quantitatively verified. The resolution of the sCPI was tested with a USAF1951 resolution test chart. The stereotactic navigation accuracy of the sCPI was measured using a calibration panel with a 7×7 circle array pattern. In benchtop validation, the navigation accuracy of the sCPI and the BrainLab Kick Navigation Station was compared using a skull phantom with 8 intracranial targets. Finally, we demonstrated the potential clinical application of sCPI through a clinical trial.

RESULTS

The resolution test showed that the resolution of the sCPI was 1.3 mm. In a stereotactic navigation accuracy test, the maximum and minimum error of the sCPI was 2.9 and 0.3 mm, and the mean error was 1.5 mm. The stereotactic navigation accuracy test also showed that the navigation error of the sCPI would increase with the pitch and yaw angle, but there was no obvious difference in navigation errors caused by different yaw directions, which meant that the navigation error is unbiased across all directions. The benchtop validation showed that the average navigation errors for the sCPI system and the Kick Navigation Station were 1.4±0.8 and 1.8±0.7 mm, the medians were 1.3 and 1.9 mm, and the average preparation times were 3 min 24 sec and 6 min 8 sec, respectively. The clinical feasibility of sCPI-assisted neurosurgical navigation was demonstrated in a clinical study. In comparison with the BrainLab device, the sCPI system required less time for preoperative preparation and enhanced the clinician experience in intraoperative visualization and navigation.

CONCLUSIONS

The sCPI technique can be potentially used in many surgical applications for intuitive visualization of medical information and intraoperative guidance of surgical trajectories.

A Smarter Health through the Internet of Surgical Things

(1) Background: In the last few years, technological developments in the surgical field have been rapid and are continuously evolving. One of the most revolutionizing breakthroughs was the introduction of the IoT concept within surgical practice. Our systematic review aims to summarize the most important studies evaluating the IoT concept within surgical practice, focusing on Telesurgery and surgical Telementoring. (2) Methods: We conducted a systematic review of the current literature, focusing on the Internet of Surgical Things in Telesurgery and Telementoring. Forty-eight (48) studies were included in this review. As secondary research questions, we also included brief overviews of the use of IoT in image-guided surgery, and patient Telemonitoring, by systematically analyzing fourteen (14) and nineteen (19) studies, respectively. (3) Results: Data from 219 patients and 757 healthcare professionals were quantitively analyzed. Study designs were primarily observational or based on model development. Palpable advantages from the IoT incorporation mainly include less surgical hours, accessibility to high quality treatment, and safer and more effective surgical education. Despite the described technological advances, and proposed benefits of the systems presented, there are still identifiable gaps in the literature that need to be further explored in a systematic manner. (4) Conclusions: The use of the IoT concept within the surgery domain is a widely incorporated but less investigated concept. Advantages have become palpable over the past decade, yet further research is warranted.

Brain Tumor and Augmented Reality: New Technologies for the Future

In recent years, huge progress has been made in the management of brain tumors, due to the availability of imaging devices, which provide fundamental anatomical and pathological information not only for diagnostic purposes [...].

Telemedicine and Telementoring in Urology: A Glimpse of the Past and a Leap Into the Future

Telemedicine is the process of utilizing telecommunications and digital relay to perform, teach, or share medical knowledge. The digital era eased the incorporation of telemedicine to different areas of medical care, including the surgical care of Urologic patient mainly through telementoring, telesurgery, and telerobotics. Over the years, Telemedicine has played an integral part in a physicians' ability to provide high quality medical care to remote patients, as well as serve as an educational tool for trainee physicians, in the form of telementoring. During the COVID-19 pandemic, telemedicine has played a vital role in combatting the health implications of confinements. Challenges of telemedicine implementation include cost, ethical considerations, security, bandwidth, latency, legal, and licensure difficulties. Nevertheless, the future of telemedicine, specifically telementoring, promises several improvements and innovative advancements that aim to bridge the gap in technological divides of urologic care. In this review, we build on what is already known about telemedicine focusing specifically on aspects related to telementoring, telestration, and telesurgery. Furthermore, we discuss its historical role in healthcare with a special emphasis on current and future use in urology.

Co-axial Projective Imaging for Augmented Reality Telementoring in Skin Cancer Surgery

Telemedicine has the potential to overcome the unequal distribution of medical resources worldwide. In this study, we report the second-generation co-axial projective imaging (CPI-2) system featured with orthotopic image projection for augmented reality surgical telementoring. The CPI-2 system can acquire surgical scene images from the local site, transmit them wirelessly to the remote site, and project the virtual annotations drawn by a remote expert with great accuracy to the surgical field. The performance characteristics of the CPI-2 system are quantitatively verified in benchtop experiments. The ex vivo study that compares the CPI-2 system and a monitor-based telementoring system shows that the CPI-2 system can reduce the focus shift and avoid subjective mapping of the instructions from a monitor to the real-world scene, thereby saving operation time and achieving precise teleguidance. The clinical feasibility of the CPI-2 system is validated in teleguided skin cancer surgery. Our ex vivo and in vivo experiment results imply the improved performance of surgical telementoring, and the clinical utility of deploying the CPI-2 system for surgical interventions in resource-limited settings. The CPI-2 system has the potential to reduce healthcare disparities in remote areas with limited resources.

Overcoming the Impact of COVID-19 on Surgical Mentorship: A Scoping Review of Long-distance Mentorship in Surgery

BACKGROUND

Mentorship in the surgical field has been increasingly recognized as a crucial component of career success. Distance mentorship models may be utilized to overcome geographic limitations, increase mentorship access, and strengthen mentoring relationships in surgery.

OBJECTIVE

This review aimed to identify the scope of literature on distance mentoring in surgery, the range of its application, its effectiveness, and any gaps in the literature that should be addressed in order to enhance mentorship in the surgical field.

DESIGN

A comprehensive PubMed review was performed in January 2021 on distance mentorship of students, trainees, and surgeons in the surgical field. Reviews, replies, and non-English articles were excluded. Data was extracted regarding publication year, author's country, specialty, subjects, aim of mentorship model, and efficacy.

RESULTS

134 total studies met inclusion and exclusion criteria. Most studies were published in 2020, written by authors in the United States, from general surgery, and featured an expert surgeon paired with a more junior fully trained surgeon. In all, 93.3% of studies utilized distance mentorship to enhance surgical skill through telementoring and only 4.5% were focused on mentorship to enhance careers through professional development. The remaining studies utilized distance mentorship models to increase surgical research (0.7%) and clinical knowledge (1.5%).

CONCLUSION

The results of this review suggest successful implementation of distance mentoring in surgery through telementoring, but a lack of professionally aimed distance mentorship programs. Amidst COVID-19, distance mentorship is particularly important because of decreased face-to-face opportunity. Future studies in the surgical field should investigate distance mentoring as a means of increasing mentorship for professional development.

Emerging simulation technologies in global craniofacial surgical training

The last few decades have seen an exponential growth in the development and adoption of novel technologies in medical and surgical training of residents globally. Simulation is an active and innovative teaching method, and can be achieved via physical or digital models. Simulation allows the learners to repeatedly practice without the risk of causing any error in an actual patient and enhance their surgical skills and efficiency. Simulation may also allow the clinical instructor to objectively test the ability of the trainee to carry out the clinical procedure competently and independently prior to trainee's completion of the program. This review aims to explore the role of emerging simulation technologies globally in craniofacial training of students and residents in improving their surgical knowledge and skills. These technologies include 3D printed biomodels, virtual and augmented reality, use of google glass, hololens and haptic feedback, surgical boot camps, serious games and escape games and how they can be implemented in low and middle income countries. Craniofacial surgical training methods will probably go through a sea change in the coming years, with the integration of these new technologies in the surgical curriculum, allowing learning in a safe environment with a virtual patient, through repeated exercise. In future, it may also be used as an assessment tool to perform any specific procedure, without putting the actual patient on risk. Although these new technologies are being enthusiastically welcomed by the young surgeons, they should only be used as an addition to the actual curriculum and not as a replacement to the conventional tools, as the mentor-mentee relationship can never be replaced by any technology.

Integration of Extended Reality and a High-Fidelity Simulator in Team-Based Simulations for Emergency Scenarios

Wearable devices such as smart glasses are considered promising assistive tools for information exchange in healthcare settings. We aimed to evaluate the usability and feasibility of smart glasses for team-based simulations constructed using a high-fidelity simulator. Two scenarios of patients with arrhythmia were developed to establish a procedure for interprofessional interactions via smart glasses using 15-h simulation training. Three to four participants formed a team and played the roles of remote supporter or bed-side trainee with smart glasses. Usability, attitudes towards the interprofessional health care team and learning satisfaction were assessed. Using a 5-point Likert scale, from 1 (strongly disagree) to 5 (strongly agree), 31 participants reported that the smart glasses were easy to use (3.61 ± 0.95), that they felt confident during use (3.90 ± 0.87), and that that responded positively to long-term use (3.26 ± 0.89) and low levels of physical discomfort (1.96 ± 1.06). The learning satisfaction was high (4.65 ± 0.55), and most (84%) participants found the experience favorable. Key challenges included an unstable internet connection, poor resolution and display, and physical discomfort while using the smart glasses with accessories. We determined the feasibility and acceptability of smart glasses for interprofessional interactions within a team-based simulation. Participants responded favorably toward a smart glass-based simulation learning environment that would be applicable in clinical settings.

Opportunities and Challenges of Smartglass-Assisted Interactive Telementoring

The widespread adoption of wearables, extended reality, and metaverses has accelerated the diverse configurations of remote collaboration and telementoring systems. This paper explores the opportunities and challenges of interactive telementoring, especially for wearers of smartglasses. In particular, recent relevant studies are reviewed to derive the needs and trends of telementoring technology. Based on this analysis, we define what can be integrated into smartglass-enabled interactive telementoring. To further illustrate this type of special use case for telementoring, we present five illustrative and descriptive scenarios. We expect our specialized use case to support various telementoring applications beyond medical and surgical telementoring, while harmoniously fostering cooperation using the smart devices of mentors and mentees at different scales for collocated, distributed, and remote collaboration.

The Effectiveness of Using Augmented Reality for Training in the Medical Professions: A Meta Analysis (Preprint)

Background

Augmented reality (AR) is an interactive technology that uses persuasive digital data and real-world surroundings to expand the user's reality, wherein objects are produced by various computer applications. It constitutes a novel advancement in medical care, education, and training.

Objective

The aim of this work was to assess how effective AR is in training medical students when compared to other educational methods in terms of skills, knowledge, confidence, performance time, and satisfaction.

Methods

We performed a meta-analysis on the effectiveness of AR in medical training that was constructed by using the Cochrane methodology. A web-based literature search was performed by using the Cochrane Library, Web of Science, PubMed, and Embase databases to find studies that recorded the effect of AR in medical training up to April 2021. The quality of the selected studies was assessed by following the Cochrane criteria for risk of bias evaluations.

Results

In total, 13 studies with a total of 654 participants were included in the meta-analysis. The findings showed that using AR in training can improve participants' performance time (I²=99.9%; P<.001), confidence (I²=97.7%; P=.02), and satisfaction (I²=99.8%; P=.006) more than what occurs under control conditions. Further, AR did not have any effect on the participants’ knowledge (I²=99.4%; P=.90) and skills (I²=97.5%; P=.10). The meta-regression plot shows that there has been an increase in the number of articles discussing AR over the years and that there is no publication bias in the studies used for the meta-analysis.

Conclusions

The findings of this work suggest that AR can effectively improve performance time, satisfaction, and confidence in medical training but is not very effective in areas such as knowledge and skill. Therefore, more AR technologies should be implemented in the field of medical training and education. However, to confirm these findings, more meticulous research with more participants is needed.