A Novel Virtual Reality Medical Image Display System for Group Discussions of Congenital Heart Disease: Development and Usability Testing

Comparing assisting technologies for proficiency in cardiac morphology: 3D printing and mixed reality versus CT slice images for morphological understanding of congenital heart defects by medical students

Learning cardiac morphology largely involves spatial abilities and studies indicate benefits from innovative 3D visualization technologies that speed up and increase the learning output. Studies comparing these teaching tools and their educational output are rare and few studies include complex congenital heart defects. This study compared the effects of 3D prints, mixed reality (MR) viewing of 3D meshes and standard cardiac CT slice images on medical students' understanding of complex congenital heart defect morphology, measuring both objective level of understanding and subjective educational experience. The objective of this study was to compare morphological understanding and user experiences of 3D printed models, MR 3D visualization and axial 2D CT slices, in medical students examining morphological details in complex congenital heart defects. Medical students in the median 4th year of study (range 2nd to 6th) examined three of five different complex congenital heart defects by three different modalities: 3D printed model, MR viewed 3D mesh, and cardiac CT slices, answering a questionnaire on morphology and user experience. Time to complete task, diagnostic accuracy, and user experience data were collected and compared on group level. Task times were similar for all modalities. The percentage of correct answers was higher with MR visualization, which was also the preferred modality overall. Medical students both prefer and better understand the morphology of complex congenital heart disease with 3D models viewed using MR, without spending more time than with 3D prints or 2D CT images.

Greater than the sum of its parts: multimodality imaging in adults with congenital heart disease

As the population of adults with congenital heart disease (ACHD) grows, there also grows an expanded need for non-invasive surveillance methods to guide management and intervention. A multimodal imaging approach layers complementary insights from echocardiography, computed tomography (CT), magnetic resonance imaging (MRI), and other modalities into a clinician's view of patient physiology. Merely applying strategies from acquired adult cardiac disease would be inadequate and potentially misleading. As data amasses in this small but growing population, investigators in the field of ACHD have discovered population-specific imaging biomarkers that identify deterioration and pivotal time points where intervention may reduce morbidity and mortality. Moreover, due to the variety of physiologies and the modest number of ACHD patients relative to that of adults with acquired heart disease, multicenter registries will be key in advancing research. The integration of well-defined imaging variables into these databases can help identify important biomarkers. Emerging technologies like computational fluid dynamics (CFD) and artificial intelligence (AI) are also primed to enhance imaging capabilities and clinical workflows, though require careful adaption as ACHD patients are not meaningfully represented in the training data for these technologies. Ultimately, a multimodal imaging approach is essential for optimizing care for ACHD patients, enabling personalized medicine where interventions can be performed before clinical deterioration occurs.

Role of eXtended Reality use in medical imaging interpretation for pre-surgical planning and intraoperative augmentation

Purpose

eXtended Reality (XR) technology, including virtual reality (VR), augmented reality (AR), and mixed reality (MR), is a growing field in healthcare. Each modality offers unique benefits and drawbacks for medical education, simulation, and clinical care. We review current studies to understand how XR technology uses medical imaging to enhance surgical diagnostics, planning, and performance. We also highlight current limitations and future directions.

Approach

We reviewed the literature on immersive XR technologies for surgical planning and intraoperative augmentation, excluding studies on telemedicine and 2D video-based training. We cited publications highlighting XR's advantages and limitations in these categories.

Results

A review of 556 papers on XR for medical imaging in surgery yielded 155 relevant papers reviewed utilizing the aid of chatGPT. XR technology may improve procedural times, reduce errors, and enhance surgical workflows. It aids in preoperative planning, surgical navigation, and real-time data integration, improving surgeon ergonomics and enabling remote collaboration. However, adoption faces challenges such as high costs, infrastructure needs, and regulatory hurdles. Despite these, XR shows significant potential in advancing surgical care.

Conclusions

Immersive technologies in healthcare enhance visualization and understanding of medical conditions, promising better patient outcomes and innovative treatments but face adoption challenges such as cost, technological constraints, and regulatory hurdles. Addressing these requires strategic collaborations and improvements in image quality, hardware, integration, and training.

Investigation of the Clinical Value of Four Visualization Modalities for Congenital Heart Disease

Diagnosing congenital heart disease (CHD) remains challenging because of its complex morphology. Representing the intricate structures of CHD on conventional two-dimensional flat screens is difficult owing to wide variations in the pathologies. Technological advancements, such as three-dimensional-printed heart models (3DPHMs) and virtual reality (VR), could potentially address the limitations of viewing complex structures using conventional methods. This study aimed to investigate the usefulness and clinical value of four visualization modalities across three different cases of CHD, including ventricular septal defect, double-outlet right ventricle, and tetralogy of Fallot. Seventeen cardiac specialists were invited to participate in this study, which was aimed at assessing the usefulness and clinical value of four visualization modalities, namely, digital imaging and communications in medicine (DICOM) images, 3DPHM, VR, and 3D portable document format (PDF). Out of these modalities, 76.4% of the specialists ranked VR as the best for understanding the spatial associations between cardiac structures and for presurgical planning. Meanwhile, 94.1% ranked 3DPHM as the best modality for communicating with patients and their families. Of the various visualization modalities, VR was the best tool for assessing anatomical locations and vessels, comprehending the spatial relationships between cardiac structures, and presurgical planning. The 3DPHM models were the best tool for medical education as well as communication. In summary, both 3DPHM and VR have their own advantages and outperform the other two modalities, i.e., DICOM images and 3D PDF, in terms of visualizing and managing CHD.

Use of Virtual Reality and 3D Models in Contemporary Practice of Cardiology

PURPOSE OF REVIEW

To provide an overview of the impact of virtual and augmented reality in contemporary cardiovascular medical practice.

RECENT FINDINGS

The utilization of virtual and augmented reality has emerged as an innovative technique in various cardiovascular subspecialties, including interventional adult, pediatric, and adult congenital as well as structural heart disease and heart failure. In particular, electrophysiology has proven valuable for both diagnostic and therapeutic procedures. The incorporation of 3D reconstruction modeling has significantly enhanced our understanding of patient anatomy and morphology, thereby improving diagnostic accuracy and patient outcomes. The interactive modeling of cardiac structure and function within the virtual realm plays a pivotal role in comprehending complex congenital, structural, and coronary pathology. This, in turn, contributes to safer interventions and surgical procedures. Noteworthy applications include septal defect device closure, transcatheter valvular interventions, and left atrial occlusion device implantation. The implementation of virtual reality has been shown to yield cost savings in healthcare, reduce procedure time, minimize radiation exposure, lower intravenous contrast usage, and decrease the extent of anesthesia required. These benefits collectively result in a more efficient and effective approach to patient care.

Cardiovascular Computed Tomography in the Diagnosis of Cardiovascular Disease: Beyond Lumen Assessment

Cardiovascular CT is being widely used in the diagnosis of cardiovascular disease due to the rapid technological advancements in CT scanning techniques. These advancements include the development of multi-slice CT, from early generation to the latest models, which has the capability of acquiring images with high spatial and temporal resolution. The recent emergence of photon-counting CT has further enhanced CT performance in clinical applications, providing improved spatial and contrast resolution. CT-derived fractional flow reserve is superior to standard CT-based anatomical assessment for the detection of lesion-specific myocardial ischemia. CT-derived 3D-printed patient-specific models are also superior to standard CT, offering advantages in terms of educational value, surgical planning, and the simulation of cardiovascular disease treatment, as well as enhancing doctor-patient communication. Three-dimensional visualization tools including virtual reality, augmented reality, and mixed reality are further advancing the clinical value of cardiovascular CT in cardiovascular disease. With the widespread use of artificial intelligence, machine learning, and deep learning in cardiovascular disease, the diagnostic performance of cardiovascular CT has significantly improved, with promising results being presented in terms of both disease diagnosis and prediction. This review article provides an overview of the applications of cardiovascular CT, covering its performance from the perspective of its diagnostic value based on traditional lumen assessment to the identification of vulnerable lesions for the prediction of disease outcomes with the use of these advanced technologies. The limitations and future prospects of these technologies are also discussed.

The Use of Virtual Reality Learning on Transition Education in Adolescents with Congenital Heart Disease

Improvement in congenital heart disease (CHD) outcomes has created a growing population of adolescents and young adults with unique health needs that require thoughtful transition planning and eventual transfer of care to an adult provider. Often, poor health literacy and limited resources can lead to interrupted care, which places them at risk for adverse health-related consequences. In 2019, the Wisconsin Adult Congenital Heart Disease transition program partnered with Stanford Virtual Heart (SVH), a virtual reality (VR) platform, to allow young adult patients to learn about their CHD in a clinic-based setting. We completed a single-center pilot study to evaluate these patients' experience and perceptions to using VR during their transition education. At an initial transition visit, we used an immediate post-VR experience survey, scored using Likert scales of 1-5 (1 = strongly disagree, 5 = strongly agree). Twenty-two patients (13 males) between the ages of 16 and 19 participated. Lesions included pulmonary stenosis, Tetralogy of Fallot, atrial and ventricular septal defect, coarctation, aortic stenosis, hypoplastic left heart syndrome, and patent ductus arteriosus. Likert averages were 4.7 for finding VR helped with understanding their heart lesion, 4.6 for finding VR helped with understanding their heart surgery, 4.7 for enjoying the VR heart simulation, and 4.6 for finding that it was a good use of time. This study demonstrates that adolescents enjoyed using SVH and found it helpful. Clinical implementation shows promise as a plausible adjunct tool for transition education.

Comparative effectiveness of virtual reality (VR) vs 3D printed models of congenital heart disease in resident and nurse practitioner educational experience

BACKGROUND

Medical trainees frequently note that cardiac anatomy is difficult to conceive within a two dimensional framework. The specific anatomic defects and the subsequent pathophysiology in flow dynamics may become more apparent when framed in three dimensional models. Given the evidence of improved comprehension using such modeling, this study aimed to contribute further to that understanding by comparing Virtual Reality (VR) and 3D printed models (3DP) in medical education.

OBJECTIVES

We sought to systematically compare the perceived subjective effectiveness of Virtual Reality (VR) and 3D printed models (3DP) in the educational experience of residents and nurse practitioners.

METHODS

Trainees and practitioners underwent individual 15-minute teaching sessions in which features of a developmentally typical heart as well as a congenitally diseased heart were demonstrated using both Virtual Reality (VR) and 3D printed models (3DP). Participants then briefly explored each modality before filling out a short survey in which they identified which model (3DP or VR) they felt was more effective in enhancing their understanding of cardiac anatomy and associated pathophysiology. The survey included a binary summative assessment and a series of Likert scale questions addressing usefulness of each model type and degree of comfort with each modality.

RESULTS

Twenty-seven pediatric residents and 3 nurse practitioners explored models of a developmentally typical heart and tetralogy of Fallot pathology. Most participants had minimal prior exposure to VR (1.1 ± 0.4) or 3D printed models (2.1 ± 1.5). Participants endorsed a greater degree of understanding with VR models (8.5 ± 1) compared with 3D Printed models (6.3 ± 1.8) or traditional models of instruction (5.5 ± 1.5) p < 0.001. Most participants felt comfortable with modern technology (7.6 ± 2.1). 87% of participants preferred VR over 3DP.

CONCLUSIONS

Our study shows that, overall, VR was preferred over 3DP models by pediatric residents and nurse practitioners for understanding cardiac anatomy and pathophysiology.

HEARTBEAT4D: An Open-source Toolbox for Turning 4D Cardiac CT into VR/AR

Four-dimensional data sets are increasingly common in MRI and CT. While clinical visualization often focuses on individual temporal phases capturing the tissue(s) of interest, it may be possible to gain additional insight through exploring animated 3D reconstructions of physiological motion made possible by augmented or virtual reality representations of 4D patient imaging. Cardiac CT acquisitions can provide sufficient spatial resolution and temporal data to support advanced visualization, however, there are no open-source tools readily available to facilitate the transformation from raw medical images to dynamic and interactive augmented or virtual reality representations. To address this gap, we developed a workflow using free and open-source tools to process 4D cardiac CT imaging starting from raw DICOM data and ending with dynamic AR representations viewable on a phone, tablet, or computer. In addition to assembling the workflow using existing platforms (3D Slicer and Unity), we also contribute two new features: 1. custom software which can propagate a segmentation created for one cardiac phase to all others and export to surface files in a fully automated fashion, and 2. a user interface and linked code for the animation and interactive review of the surfaces in augmented reality. Validation of the surface-based areas demonstrated excellent correlation with radiologists' image-based areas (R > 0.99). While our tools were developed specifically for 4D cardiac CT, the open framework will allow it to serve as a blueprint for similar applications applied to 4D imaging of other tissues and using other modalities. We anticipate this and related workflows will be useful both clinically and for educational purposes.

Combined Expert and User-Driven Usability Assessment of Trauma Decision Support Systems Improves User-Centered Design

BACKGROUND

Trauma clinical decision support systems improve adherence with evidence-based practice but suffer from poor usability and the lack of a user-centered design. The objective of this study was to compare the effectiveness of user and expert-driven usability testing methods to detect usability issues in a rib fracture clinical decision support system and identify guiding principles for trauma clinical decision support systems.

METHODS

A user-driven and expert-driven usability investigation was conducted using a clinical decision support system developed for patients with rib fractures. The user-driven usability evaluation was as follows: 10 clinicians were selected for simulation-based usability testing using snowball sampling, and each clinician completed 3 simulations using a video-conferencing platform. End-users participated in a novel team-based approach that simulated realistic clinical workflows. The expert-driven heuristic evaluation was as follows: 2 usability experts conducted a heuristic evaluation of the clinical decision support system using 10 common usability heuristics. Usability issues were identified, cataloged, and ranked for severity using a 4-level ordinal scale. Thematic analysis was utilized to categorize the identified usability issues.

RESULTS

Seventy-nine usability issues were identified; 63% were identified by experts and 48% by end-users. Notably, 58% of severe usability issues were identified by experts alone. Only 11% of issues were identified by both methods. Five themes were identified that could guide the design of clinical decision support systems-transparency, functionality and integration into workflow, automated and noninterruptive, flexibility, and layout and appearance. Themes were preferentially identified by different methods.

CONCLUSION

We found that a dual-method usability evaluation involving usability experts and end-users drastically improved detection of usability issues over single-method alone. We identified 5 themes to guide trauma clinical decision support system design. Performing usability testing via a remote video-conferencing platform facilitated multi-site involvement despite a global pandemic.

The Virtual Reality Technology Effects and Features in Cardiology Interventions Training: A Scoping Review

Background: Virtual Reality (VR) as an emerging and developing technology has received much attention in healthcare and trained different medical groups. Implementing specialized training in cardiac surgery is one of the riskiest and most sensitive issues related to clinical training. Studies have been conducted to train cardiac residents using this technology. This study aimed to identify the effects and features of VR technology in cardiology interventions training. Methods: This scoping review was conducted in 2021 by searching PubMed, Scopus, and Web of Sciences scientific databases by combining the related keywords. A data extraction form was used for data gathering. Data analyses were done through the content analysis method, and results were reported based on the study objectives. Results: 21 studies were included; from the 777 articles found in the initial searches, seven (33.33%) were RCT studies. VR-based education studies in cardiology interventions have grown significantly in recent years. The main effects of applying VR include improved user attitude and satisfaction, improved performance after VR training, and improved training and learning. Input devices include tracking devices, point input devices, and controllers. Output devices were three main categories include graphics audios and haptic. Conclusion: The use of new technologies, especially VR, can improve the efficiency of medical training in clinical settings. It recommends that this technology train the necessary skills for heart surgery in cardiac residents before performing real surgery to reduce the potential risks and medical errors.

Virtual Simulation in Undergraduate Medical Education: A Scoping Review of Recent Practice

Virtual simulation (VS) as an emerging interactive pedagogical strategy has been paid more and more attentions in the undergraduate medical education. Because of the fast development of modern computer simulation technologies, more and more advanced and emerging VS-based instructional practices are constantly increasing to promote medical education in diverse forms. In order to describe an overview of the current trends in VS-based medical teaching and learning, this scoping review presented a worldwide analysis of 92 recently published articles of VS in the undergraduate medical teaching and learning. The results indicated that 98% of included articles were from Europe, North America, and Asia, suggesting a possible inequity in digital medical education. Half (52%) studies reported the immersive virtual reality (VR) application. Evidence for educational effectiveness of VS in medical students' knowledge or skills was sufficient as per Kirkpatrick's model of outcome evaluation. Recently, VS has been widely integrated in surgical procedural training, emergency and pediatric emergency medicine training, teaching of basic medical sciences, medical radiation and imaging, puncture or catheterization training, interprofessional medical education, and other case-based learning experiences. Some challenges, such as accessibility of VS instructional resources, lack of infrastructure, "decoupling" users from reality, as well as how to increase students' motivation and engagement, should be addressed.

Use of advanced cardiac imaging in congenital heart disease: growth, indications and innovations

PURPOSE OF REVIEW

Significant improvements in the diagnosis and management of patients with congenital heart disease (CHD) have led to improved survival. These patients require life-long noninvasive evaluation. The use of advanced imaging such as cardiac magnetic resonance imaging (CMR) and cardiac computed tomography (CCT) has increased to support this need. The purpose of this review is to discuss the basics of advanced cardiac imaging, indications and review the recent innovations.

RECENT FINDINGS

Recent literature has demonstrated the increasing reliance of advanced imaging for CHD patients. In addition, research is focusing on CMR techniques to shorten scan time and address previous limitations that made imaging younger and sicker patients more challenging. CCT research has involved demonstrating high-quality images with low radiation exposure. Advances in digital technology have impacted the interactivity of 3D imaging through the use of virtual and augmented reality platforms. With the increased reliance of advanced imaging, appropriate use criteria have been developed to address possible under or over utilization.

SUMMARY

The utilization of advanced cardiac imaging continues to increase. As CMR and CCT continue to grow, increased knowledge of these modalities and their usage will be necessary for clinicians caring for CHD patients.

Clinical Value of Virtual Reality versus 3D Printing in Congenital Heart Disease

Both three-dimensional (3D) printing and virtual reality (VR) are reported as being superior to the current visualization techniques in conveying more comprehensive visualization of congenital heart disease (CHD). However, little is known in terms of their clinical value in diagnostic assessment, medical education, and preoperative planning of CHD. This cross-sectional study aims to address these by involving 35 medical practitioners to subjectively evaluate VR visualization of four selected CHD cases in comparison with the corresponding 3D printed heart models (3DPHM). Six questionnaires were excluded due to incomplete sections, hence a total of 29 records were included for the analysis. The results showed both VR and 3D printed heart models were comparable in terms of the degree of realism. VR was perceived as more useful in medical education and preoperative planning compared to 3D printed heart models, although there was no significant difference in the ratings (p = 0.54 and 0.35, respectively). Twenty-one participants (72%) indicated both the VR and 3DPHM provided additional benefits compared to the conventional medical imaging visualizations. This study concludes the similar clinical value of both VR and 3DPHM in CHD, although further research is needed to involve more cardiac specialists for their views on the usefulness of these tools.

Virtual reality three-dimensional echocardiographic imaging for planning surgical atrioventricular valve repair

OBJECTIVES

To investigate how virtual reality (VR) imaging impacts decision-making in atrioventricular valve surgery.

METHODS

This was a single-center retrospective study involving 15 children and adolescents, median age 6 years (range, 0.33-16) requiring surgical repair of the atrioventricular valves between the years 2016 and 2019. The patients' preoperative 3-dimesnional (3D) echocardiographic data were used to create 3D visualization in a VR application. Five pediatric cardiothoracic surgeons completed a questionnaire formulated to compare their surgical decisions regarding the cases after reviewing conventionally presented 2-dimesnional and 3D echocardiographic images and again after visualization of 3D echocardiograms using the VR platform. Finally, intraoperative findings were shared with surgeons to confirm assessment of the pathology.

RESULTS

In 67% of cases presented with VR, surgeons reported having "more" or "much more" confidence in their understanding of each patient's pathology and their surgical approach. In all but one case, surgeons were at least as confident after reviewing the VR compared with standard imaging. The case where surgeons reported to be least confident on VR had the worst technical quality of data used. After viewing patient cases on VR, surgeons reported that they would have made minor modifications to surgical approach in 53% and major modifications in 7% of cases.

CONCLUSIONS

The main impact of viewing imaging on VR is the improved clarity of the anatomical structures. Surgeons reported that this would have impacted the surgical approach in the majority of cases. Poor-quality 3D echocardiographic data were associated with a negative impact of VR visualization; thus. quality assessment of imaging is necessary before projecting in a VR format.