Virtual reality three-dimensional echocardiographic imaging for planning surgical atrioventricular valve repair

Surgeon perspectives on a virtual reality platform for preoperative planning in complex bone sarcomas

Background and objectives

Treatment of primary bone and soft tissue sarcomas typically includes complete surgical resection with or without adjunctive modalities. Despite best efforts, for the most challenging clinical scenarios such as axial or pelvic sarcoma, five-year survival rates are reported to be between 27 and 40 %. Since quality of resection is a key determinant of oncologic outcomes, it is critical to preoperatively plan the surgical approach to improve resection accuracy, ensure sufficient surgical margins, and reduce the risk of local or metastatic recurrence. The computer conversion of 2-dimensional (2D) computerized tomography (CT) and magnetic resonance imaging (MRI) to a three-dimensional (3D) virtual reality (VR) avatar image may allow improved preoperative estimation of tumor size, location, adjacent anatomy, and spatial understanding of the tumor without relying on surgeon experience, memory, and imagination. The purpose of this study is to investigate the utility of a virtual reality platform in preoperative planning and surgical approach in a retrospective cohort of pelvic bone sarcoma cases.

Methods

The histopathology database at our institution was queried for all historical cases of bone and soft tissue sarcoma with surgical resection failure, defined as positive gross or microscopic margins. Four cases of pelvic bone sarcoma were chosen for retrospective review by fellowship-trained orthopedic tumor specialists. For each case, participants first studied conventional 2D preoperative CT images and answered a questionnaire pertaining to objective case parameters. Participants then interacted with case-specific 3D models while wearing a VR headset and answered the same questionnaire. The VR 'avatar' was created with custom-developed software. After using both modalities, participants completed a Likert-scale survey aiming to evaluate the VR technology's subjective impact on understanding tumor environment, surgical plan confidence, and its ability to improve communication with colleagues and patients. Four attending orthopedic oncologists, one orthopedic oncology fellow, and one senior orthopedic oncology resident participated in the study.

Results

Four cases of failed resection were evaluated by a group of both attending surgeons and a group of trainees composed of both residents and fellows. Tumor borders were clearly delineated in 0 % and 66.6 % cases when evaluating with conventional 2D imaging and VR, respectively. Participants changed adjacent structure involvement grade 22.2 % of the time after assessing involvement grade on the VR technology, with adjacent ligamentous structure grading changed most frequently in 55.5 % of cases. Users reported they would change the surgical approach or margins 44.4 % of the time after reviewing with VR technology. Initial 6 plane resection plans were changed in every user case. Subjective responses indicated that surgeons expressed more confidence in their approach, confidence with obtaining negative margins, and provided more detail regarding structures to be resected in specific planes.

Conclusion

Pelvic tumors present unique surgical challenges due to complex 3D anatomy, the proximity of vital structures, consistency of the tumor, and the need to alter patient position during resection procedures. Using examples of failed pelvic bone sarcoma resections, our study found that VR imaging increased understanding of the tumor environment, characteristics, and ability to communicate with patients and colleagues.

Use of Three-Dimensional (3D) Airway Modeling and Virtual Reality for Diagnosis, Communication, and Surgical Planning of Complex Airway Stenosis

OBJECTIVE

We hypothesized that a method to segment human airways from clinical cases and import them into a case presentation environment in Virtual Reality (VR) could be developed to model and visualize complex airway stenosis for efficient surgical planning.

METHODS

One normal and two pathological airways modeled from CT scans at a slice thickness of 0.625 mm were processed. A multidisciplinary team composed of airway surgeons, VR engineers, educators, and radiologists collaborated to create a clinically relevant VR rendering and explanatory Narrations of the three clinical cases. Segmentation and postprocessing were completed in the Mimics Innovation Suite v24 from Materialize. Structures were segmented from the level of bifurcation of common carotid arteries to the level of bifurcation of the main bronchi, including cartilaginous and bony airway structures, vessels, and soft tissues. They were then postprocessed into 3D image volumes and imported into syGlass (IstoVisio Inc.), a VR software.

RESULTS

Direct visualization and free manipulation of these 3D airway models within the VR environment provided improved geometrical and anatomical details compared to traditional two-dimensional (2D) CT. Then, specialized presentation and active learning tools developed for scientific communication using the VR environment permitted the creation of VR Narrations to explain pathological cases.

CONCLUSION

The method to segment human airways from clinical cases used in this paper, combined with intuitive VR tools to overlay segmentation and image data in an active learning environment, shows potential in the use of 3D airway modeling and VR in clinical practice for the description and surgical planning of complex airways. Further work is needed to validate the use of these models in clinical practice and patient education.

LEVEL OF EVIDENCE

Level 4.

How to Use Virtual Reality from Echocardiographic-Acquired Images

PURPOSE OF REVIEW

Our goal of this review is to describe the current evidence and future applications of virtual reality (VR) from echocardiographic acquired images to enhance diagnostic accuracy and improve the visualization of cardiac structures by providing immersive 3-dimensional (3D) environments for better interpretation, communication between healthcare providers, families and trainees.

RECENT FINDINGS

Virtual reality is a powerful tool for education, procedural planning and simulation. Using readily available echocardiographic images for this purpose is feasible, effective and safe. This document highlights the increasing role of VR in procedural planning, education, and intraprocedural guidance using three-dimensional echocardiography (3DE). Key advancements include the development of smaller 3D transthoracic and pediatric transesophageal probes with better temporal resolution, which enable enhanced imaging for preoperative planning. VR allows surgeons to visualize dynamic 3D cardiac models created from echocardiographic or Computed Tomography (CT) data, enabling better pre-surgical rehearsal, device selection, and landing zone visualization. It also provides immersive training tools for echocardiographers and offers potential for intraprocedural guidance in interventions such as atrial septal defect closures. These developments underscore the growing synergy between VR and 3DE for refining diagnosis and procedural strategies.

Augmented and virtual reality imaging for collaborative planning of structural cardiovascular interventions: a proof-of-concept and validation study

Purpose

Virtual reality (VR) and augmented reality (AR) have led to significant advancements in cardiac preoperative planning, shaping the world in profound ways. A noticeable gap exists in the availability of a comprehensive multi-user, multi-device mixed reality application that can be used in a multidisciplinary team meeting.

Approach

A multi-user, multi-device mixed reality application was developed, supporting AR and VR implementations. Technical validation involved a standardized testing protocol and comparison of AR and VR measurements regarding absolute error and time. Preclinical validation engaged experts in interventional cardiology, evaluating the clinical applicability prior to clinical validation. Clinical validation included patient-specific measurements for five patients in VR compared with standard computed tomography (CT) for preoperative planning. Questionnaires were used at all stages for subjective evaluation.

Results

Technical validation, including 106 size measurements, demonstrated an absolute median error of 0.69 mm (0.25 to 1.18 mm) compared with ground truth. The time to complete the entire task was 892 ± 407    s on average, with VR measurements being faster than AR ( 804 ± 483 versus 957 ± 257    s , P = 0.045 ). On clinical validation of five preoperative patients, there was no statistically significant difference between paired CT and VR measurements (0.58 [95% CI, - 1.58 to 2.74], P = 0.586 ). Questionnaires showcased unanimous agreement on the user-friendly nature, effectiveness, and clinical value.

Conclusions

The mixed reality application, validated through technical, preclinical, and clinical assessments, demonstrates precision and user-friendliness. Further research of our application is needed to validate the generalizability and impact on patient outcomes.

Applications and advances of immersive technology in cardiology

Different forms of immersive technology, such as Virtual Reality (VR) and Augmented Reality (AR), are getting increasingly invested in medicine. Advances in head-mounted display technology, processing, and rendering power have demonstrated the increasing utility of immersive technology in medicine and the healthcare environment. There are a growing number of publications on using immersive technology in cardiology. We reviewed the articles published within the last decade that reported case studies or research that uses or investigates the application of immersive technology in the broad field of cardiology - from education to preoperative planning and intraoperative guidance. We summarized the advantages and disadvantages of using AR and VR for various application categories. Our review highlights the need for a robust assessment of the effectiveness of the methods and discusses the technical limitations that hinder the complete integration of AR and VR in cardiology, including cost-effectiveness and regulatory compliance. Despite the limitations and gaps that have inhibited us from benefiting from immersive technologies' full potential in cardiology settings to date, its promising, impactful future for standard cardiovascular care is undoubted.

A System for Mixed-Reality Holographic Overlays of Real-Time Rendered 3D-Reconstructed Imaging Using a Video Pass-through Head-Mounted Display-A Pathway to Future Navigation in Chest Wall Surgery

Background: Three-dimensional reconstructions of state-of-the-art high-resolution imaging are progressively being used more for preprocedural assessment in thoracic surgery. It is a promising tool that aims to improve patient-specific treatment planning, for example, for minimally invasive or robotic-assisted lung resections. Increasingly available mixed-reality hardware based on video pass-through technology enables the projection of image data as a hologram onto the patient. We describe the novel method of real-time 3D surgical planning in a mixed-reality setting by presenting three representative cases utilizing volume rendering. Materials: A mixed-reality system was set up using a high-performance workstation running a video pass-through-based head-mounted display. Image data from computer tomography were imported and volume-rendered in real-time to be customized through live editing. The image-based hologram was projected onto the patient, highlighting the regions of interest. Results: Three oncological cases were selected to explore the potentials of the mixed-reality system. Two of them presented large tumor masses in the thoracic cavity, while a third case presented an unclear lesion of the chest wall. We aligned real-time rendered 3D holographic image data onto the patient allowing us to investigate the relationship between anatomical structures and their respective body position. Conclusions: The exploration of holographic overlay has proven to be promising in improving preprocedural surgical planning, particularly for complex oncological tasks in the thoracic surgical field. Further studies on outcome-related surgical planning and navigation should therefore be conducted. Ongoing technological progress of extended reality hardware and intelligent software features will most likely enhance applicability and the range of use in surgical fields within the near future.

Mitral valve surgery assisted by virtual and augmented reality: Cardiac surgery at the front of innovation

BACKGROUND

Given the variety in mitral valve (MV) pathology and associated surgical techniques, extended reality (XR) holds great potential to assist MV surgeons. This review aims to systematically evaluate the currently available evidence investigating the use of XR and associated technologies in MV surgery.

METHODS

A systematic database search was conducted of original articles and case reports that explored the use of XR and MV surgery in EMBASE, MEDLINE, Cochrane database and Google Scholar, from inception to February 2022.

RESULTS

Our search yielded 171 articles, of which 15 studies were included in this review, featuring 328 patients. Two main areas of application were identified: (i) pre-operative planning and (ii) predicting post-operative outcomes. The articles reporting outcomes relating to pre-operative planning were further categorised as exploring themes relevant to (i) mitral annular assessment; (ii) training; (iii) evaluation of surgical technique; (iv) surgical approach or plan and (v) selecting ring size or type. Preoperatively, XR has been shown to evaluate mitral annular pathology more accurately than echocardiography, informing the surgeon about the optimal surgical technique, approach and plan for a particular patient's MV pathology. Furthermore, XR could simulate and aid ring size/type selection for MV annuloplasty, creating a personalized surgical plan. Additionally, XR could estimate the postoperative MV biomechanical and physiological characteristics, predicting and pre-empting post-operative complications.

CONCLUSION

XR demonstrated promising applications for assisting MV surgery, enhancing outcomes and patient-centred care, nevertheless, there remain the need for randomized studies to ascertain its feasibility, safety, and validity in clinical practice.

Advancing paediatric cardiac imaging: a comprehensive analysis of the feasibility and accuracy of a novel 3D paediatric transoesophageal probe

Aims

Pediatric transoesophageal echocardiography (TOE) probes have remained two-dimensional (2D) limiting their use compared to adults. While critical in pediatrics for interventions and post-surgery assessments, technological advancements introduced a three-dimensional (3D) pediatric TOE probe. This study assessed the new 3D pediatric TOE probe (GE 9VT-D) for feasibility, handling, and imaging quality.

Methods and results

At Children's Hospital of Toulouse, 2-month prospective study enrolled children undergoing TOE with the new probe. All imaging modalities were rated by 2 operators using a 5-point Likert-type scale from 1 (very poor) to 5 (very good) quality. Forty-five children, median age 3.7 (range: 2 months-14.7 years) median weight 7.8 kg (range: 4.3-48 kg) underwent 60 TOEs: 25% pre-surgery, 45% post-surgery, 28% during percutaneous procedures, and 2% in intensive care. Probe handling was "very easy" in all cases without adverse events. The median score of 2D, 2D colour, pulsed Doppler and 3D were noted 5 out of 5 and continuous Doppler and 3D colour 4 out of 5. The 3D image quality remained consistent irrespective of the patient weighing above or below 7.8 kg (p = 0.72). Postoperative TOEs identified two cases needing further interventions, emphasizing its value in evaluating surgical outcomes and also for guiding percutaneous interventions.

Conclusion

Our comprehensive evaluation demonstrates that the new 3D pediatric TOE probe is feasible and provides high-quality imaging in pediatric patients. The successful integration of this novel probe into clinical practice has the potential to enhance diagnostic accuracy and procedural planning, ultimately optimizing patient outcomes in pediatric cardiac care.

Artificial Intelligence in Congenital Heart Disease: Current State and Prospects

The current era of big data offers a wealth of new opportunities for clinicians to leverage artificial intelligence to optimize care for pediatric and adult patients with a congenital heart disease. At present, there is a significant underutilization of artificial intelligence in the clinical setting for the diagnosis, prognosis, and management of congenital heart disease patients. This document is a call to action and will describe the current state of artificial intelligence in congenital heart disease, review challenges, discuss opportunities, and focus on the top priorities of artificial intelligence-based deployment in congenital heart disease.

Side-firing intraoperative ultrasound applied to resection of pituitary macroadenomas and giant adenomas: A single-center retrospective case-control study

Introduction

Multiple intraoperative navigation and imaging modalities are currently available as an adjunct to endoscopic transsphenoidal resection of pituitary adenomas, including intraoperative CT and MRI, fluorescence guidance, and neuronavigation. However, these imaging techniques have several limitations, including intraoperative tissue shift, lack of availability in some centers, and the increased cost and time associated with their use. The side-firing intraoperative ultrasound (IOUS) probe is a relatively new technology in endoscopic endonasal surgery that may help overcome these obstacles.

Methods

A retrospective analysis was performed on patients admitted for resection of pituitary adenomas by a single surgeon at the University of Mississippi Medical Center. The control (non-ultrasound) group consisted of twelve (n=12) patients who received surgery without IOUS guidance, and the IOUS group was composed of fifteen (n=15) patients who underwent IOUS-guided surgery. Outcome measures used to assess the side-firing IOUS were the extent of tumor resection, postoperative complications, length of hospital stay (LOS) in days, operative time, and self-reported surgeon confidence in estimating the extent of resection intraoperatively.

Results

Preoperative data analysis showed no significant differences in patient demographics or presenting symptoms between the two groups. Postoperative data revealed no significant difference in the rate of gross total resection between the groups (p = 0.716). Compared to the non-US group, surgeon confidence was significantly higher (p < 0.001), and operative time was significantly lower for the US group in univariate analysis (p = 0.011). Multivariate analysis accounting for tumor size, surgeon confidence, and operative time confirmed these findings. Interestingly, we noted a trend for a lower incidence of postoperative diabetes insipidus in the US group, although this did not quite reach our threshold for statistical significance.

Conclusion

Incorporating IOUS as an aid for endonasal resection of pituitary adenomas provides real-time image guidance that increases surgeon confidence in intraoperative assessment of the extent of resection and decreases operative time without posing additional risk to the patient. Additionally, we identified a trend for reduced diabetes insipidus with IOUS.

3D Approaches in Complex CHD: Where Are We? Funny Printing and Beautiful Images, or a Useful Tool?

Echocardiography, CT and MRI have a crucial role in the management of congenital heart disease (CHD) patients. All of these modalities can be presented in a 2D or a 3D rendered format. The aim of this paper is to review the key advantages and potential limitations, as well as the future challenges of a 3D approach in each imaging modality. The focus of this review is on anatomic rather than functional assessment. Conventional 2D echocardiography presents limitations when imaging complex lesions, whereas 3D imaging depicts the anatomy in all dimensions. CT and MRI can visualise extracardiac vasculature and guide complex biventricular repair. Three-dimensional printed models can be used in depicting complex intracardiac relationships and defining the surgical strategy in specific lesions. Extended reality imaging retained dynamic cardiac motion holds great potential for planning surgical and catheter procedures. Overall, the use of 3D imaging has resulted in a better understanding of anatomy, with a direct impact on the surgical and catheter approach, particularly in more complex cases.

Artificial intelligence in pediatric cardiology: taking baby steps in the big world of data

PURPOSE OF REVIEW

Artificial intelligence (AI) has changed virtually every aspect of modern life, and medicine is no exception. Pediatric cardiology is both a perceptual and a cognitive subspecialty that involves complex decision-making, so AI is a particularly attractive tool for this medical discipline. This review summarizes the foundational work and incremental progress made as AI applications have emerged in pediatric cardiology since 2020.

RECENT FINDINGS

AI-based algorithms can be useful for pediatric cardiology in many areas, including: (1) clinical examination and diagnosis, (2) image processing, (3) planning and management of cardiac interventions, (4) prognosis and risk stratification, (5) omics and precision medicine, and (6) fetal cardiology. Most AI initiatives showcased in medical journals seem to work well in silico, but progress toward implementation in actual clinical practice has been more limited. Several barriers to implementation are identified, some encountered throughout medicine generally, and others specific to pediatric cardiology.

SUMMARY

Despite barriers to acceptance in clinical practice, AI is already establishing a durable role in pediatric cardiology. Its potential remains great, but to fully realize its benefits, substantial investment to develop and refine AI for pediatric cardiology applications will be necessary to overcome the challenges of implementation.

A Virtual Reality System for Improved Image-Based Planning of Complex Cardiac Procedures

The intricate nature of congenital heart disease requires understanding of the complex, patient-specific three-dimensional dynamic anatomy of the heart, from imaging data such as three-dimensional echocardiography for successful outcomes from surgical and interventional procedures. Conventional clinical systems use flat screens, and therefore, display remains two-dimensional, which undermines the full understanding of the three-dimensional dynamic data. Additionally, the control of three-dimensional visualisation with two-dimensional tools is often difficult, so used only by imaging specialists. In this paper, we describe a virtual reality system for immersive surgery planning using dynamic three-dimensional echocardiography, which enables fast prototyping for visualisation such as volume rendering, multiplanar reformatting, flow visualisation and advanced interaction such as three-dimensional cropping, windowing, measurement, haptic feedback, automatic image orientation and multiuser interactions. The available features were evaluated by imaging and nonimaging clinicians, showing that the virtual reality system can help improve the understanding and communication of three-dimensional echocardiography imaging and potentially benefit congenital heart disease treatment.

Non-invasive Imaging in the Evaluation of Cardiac Shunts for Interventional Closure

Multimodality imaging provides important information to guide patient selection and pre-procedural decision making for shunt lesions in CHD. While echocardiography, CT, and CMR are well-established, 3D printing and now virtual reality imaging are beginning to show promise.

Clinical Value of Virtual Reality versus 3D Printing in Congenital Heart Disease

Both three-dimensional (3D) printing and virtual reality (VR) are reported as being superior to the current visualization techniques in conveying more comprehensive visualization of congenital heart disease (CHD). However, little is known in terms of their clinical value in diagnostic assessment, medical education, and preoperative planning of CHD. This cross-sectional study aims to address these by involving 35 medical practitioners to subjectively evaluate VR visualization of four selected CHD cases in comparison with the corresponding 3D printed heart models (3DPHM). Six questionnaires were excluded due to incomplete sections, hence a total of 29 records were included for the analysis. The results showed both VR and 3D printed heart models were comparable in terms of the degree of realism. VR was perceived as more useful in medical education and preoperative planning compared to 3D printed heart models, although there was no significant difference in the ratings (p = 0.54 and 0.35, respectively). Twenty-one participants (72%) indicated both the VR and 3DPHM provided additional benefits compared to the conventional medical imaging visualizations. This study concludes the similar clinical value of both VR and 3DPHM in CHD, although further research is needed to involve more cardiac specialists for their views on the usefulness of these tools.