Immersive visualisation of intracardiac blood flow in virtual reality on a patient with HLHS

Funding Acknowledgements

Type of funding sources: Other. Main funding source(s): NIHR i4i funded 3D Heart project Wellcome/EPSRC Centre for Medical Engineering [WT 203148/Z/16/Z]

onbehalf

3D Heart Project

Background/Introduction: Virtual Reality (VR) for surgical and interventional planning in the treatment of Congenital Heart Disease (CHD) is an emerging field that has the potential to improve planning. Particularly in very complex cases, VR permits enhanced visualisation and more intuitive interaction of volumetric images, compared to traditional flat-screen visualisation tools. Blood flow is severely affected by CHD and, thus, visualisation of blood flow allows direct observation of the cardiac maladaptions for surgical planning. However, blood flow is fundamentally 3D information, and viewing and interacting with it using conventional 2D displays is suboptimal. 

Purpose

To demonstrate feasibility of blood flow visualisation in VR using pressure and velocity obtained from a computational fluid dynamic (CFD) simulation of the right ventricle in a patient with hypoplastic left heart syndrome (HLHS) as a proof of concept.

Methods

 We extend an existing VR volume rendering application to include CFD rendering functionality using the Visualization Toolkit (VTK), an established visualisation library widely used in clinical software for visualising medical imaging data. Our prototype displays the mesh outline of the segmented heart, a slicing plane showing blood pressure on the plane within the heart, and streamlines of blood flow from a spherical source region. Existing user tools were extended to enable interactive positioning, rotation and scaling of the pressure plane and streamline origin, ensuring continuity between volume rendering and CFD interaction and, thus, ease of use. We evaluated if rendering and interaction times were low enough to ensure a comfortable, interactive VR experience. Our performance benchmark is a previous study showing VR is acceptable to clinical users when rendering speed is at least 90 fps.

Results

CFD simulations were successfully rendered, viewed and manipulated in VR, as shown in the Figure. Evaluating performance, we found that visualisation of the mesh and streamlines was at an acceptably high and stable frame rate, over 150fps. User interactions of moving, rotating or scaling the mesh or streamlines origin did not significantly reduce this frame rate. However, rendering the  pressure slicing plane reduced frame rate by an unacceptable degree, to less than 10fps.  

Conclusion

Visualisation of and interaction with CFD simulation data was successfully integrated into an existing VR application. This aids in surgery and intervention planning for defects heavily relying on blood flow simulation, and lays a foundation for a platform for clinicians to test interventions in VR. Pressure plane rendering performance will require significant optimisation, potentially addressed by updating the pressure plane data separately from the main, VR rendering.

Abstract Figure. An example render of CFD simulation

P1417 Acceptability of a virtual reality system for examination of congenital heart disease patients

Funding Acknowledgements

Work supported by the NIHR i4i funded 3D Heart project [II-LA-0716-20001]

Background/Introduction

Virtual Reality (VR) has recently gained great interest for examining 3D images from congenital heart disease (CHD) patients. Currently, 3D printed models of the heart may be used for particularly complex cases. These have been found to be intuitive and to positively impact clinical decision-making. Although positively received, such printed models must be segmented from the image data, generally only CT/MR may be used, the prints are static, and models do not allow for cropping / slicing or easy manipulation. Our VR system is designed to address these issues, as well as providing a simple interface compared to standard software. Building such a VR system, one with intuitive interaction which is clinically useful, requires studying user acceptance and requirements.

Purpose: We evaluate the usability of our VR system

can a prototype VR system be easily learned and used by clinicians unfamiliar with VR.

Method

We tested a VR system which can display 3D echo images and enables the user to interact with them, for instance by translating, rotating and cropping. Our system is tested on a transoesophageal echocardiogram from a patient with aortic valve disease. 13 clinicians evaluated the system including 5 imaging cardiologists, 5 physiologists, 2 surgeons and an interventionist, with their clinical experience ranging from trainee to more than 5 years’ of experience. None had used VR regularly in the past. After a brief training session, they were asked to place three anatomical landmarks and identify a particular cardiac view. They then completed a questionnaire on system ease of learning and image manipulation.

Results: Results are shown in the figure below. Learning to use the system was perceived as easy for all but one participant, who rated it as ‘Somewhat difficult’. However, once trained, all users found the system easy to use. Participants found the interaction, where objects in the scene are picked up using the controller and then track the controller’s motion in a 1:1 way, to be particularly easy to learn and use.

Conclusion

Our VR system was accepted by the vast majority of clinicians, both for ease of learning and use. Intuitiveness and the ability to interact with images in a natural way were highlighted as most useful - suggesting that such a system could become accepted for routine clinical use in the future.

Abstract P1417 Figure. VR system evaluation participant feedbac

P801 A virtual reality tool for measurement of 3D echocardiographic images

Funding Acknowledgements

Work supported by the NIHR i4i funded 3D Heart project [II-LA-0716-20001]

Background/Introduction

Cardiac measurements are clinically important and are invariably required in any clinical imaging software. The advent of Virtual Reality (VR) imaging systems is introducing intuitive and natural ways of visualising and interrogating echo images in a 3D environment. The 3D nature of the VR experience requires purpose-designed measurement tools, which may benefit from better depth perception and easier localisation of 3D landmarks.

Purpose

Comparison of the accuracy of our VR 3D linear measurement system to commercial clinical imaging software, using both multi-plane reformatting (MPR) and volume rendered views.

Method

Each virtual reality measurement was made by selecting two points in 3D, directly in the volume rendering. The participants could edit the measurements until satisfied with their accuracy. 5 expert clinicians carried out 26 measurements each - 6 measurements on a calibration phantom, and 5 anatomically meaningful measurements (for example: aortic valve, left atrium, left ventricle) on 4 datasets. The same measurements were made by all participants using our VR system (volume rendering), Philips" QLAB (MPR) and Tomtec (volume rendering). The frame number and view (for example: long axis) were consistent for each measurement across the 3 packages used.

Results

Preliminary results are shown in the figure below. MPR measurements made on Philips’ QLAB are used as a reference, as this is the most commonly used software for this purpose at our institution. We compare measurements made in Tomtec and VR, both using volume rendering, using Bland-Altman plots. Each measurement data point is the mean of all participants measurements for each dataset/measurement combination. The mean of the measurement differences for the VR system is closer to zero, compared to Tomtec. However, the variation of these differences is larger for the VR system than for Tomtec.

Conclusion

Our preliminary results suggest that the accuracy of line measurements made using volume rendering within a VR system is comparable to measurements made using approved software packages for volume rendering displayed on a 2D screen. This shows promise for more complex interrogation methods.

Abstract P801 Figure. Comparison of Tomtec and VR with QLAB

P1566 Evaluation of haptic feedback for interaction with volumetric image data in virtual reality

Funding Acknowledgements

Work supported by the NIHR i4i funded 3D Heart project [II-LA-0716-20001]

Background

3D printing is used for surgical planning of complex congenital heart disease (CHD) because it provides an intuitive 3D representation of the image data. However, the 3D print is static and it can be costly and time consuming to create. Virtual Reality (VR) is a cheaper alternative that is able to visualise volumetric images in 3D directly from the scanner, both statically (CT and MR) and dynamically (cardiac ultrasound). However, VR visualisation is not as tangible as a 3D print - this is because it lacks the haptic feedback which would make the interactions feel more natural.

Purpose

Evaluate if adding haptic feedback (vibration) to the visualisation of volume image data in VR improves measurement accuracy and user experience.

Method

We evaluated the effect of vibration haptic feedback in our VR system using a synthetic cylinder volume dataset. The cylinder was displayed in two conditions: (1) with no haptic feedback, and (2) with haptic feedback. Ten non-clinical participants volunteered in the evaluation. They were blinded to these two test conditions. The participants were asked to measure the cylinder’s diameter horizontally and vertically, and its length, in each test condition. The measurement results were compared to the ground truth to assess the measurement accuracy. Each participant also completed a questionnaire comparing their experience of the two test conditions during the experiment.

Results

The results show a marginal improvement of measurement accuracy with haptic feedback, compared to no haptics (see Figure a). However, this improvement was not statistically significant. The haptic feedback did improve the participants’ confidence about their performance and increased the ease of use in VR, hence, they preferred the haptics condition to the no haptics condition (see Figure b). Moreover, although 70% of the participants reported relying on the visual cue more than on the haptic cue, 90% found that the haptic cue was helpful for deciding where to place the measurement point. Also, 88.9% of the participants felt more immersed in the VR scene with haptic feedback.

Conclusion

Our evaluation suggests that although haptic feedback may only marginally improve measurement accuracy, participants nevertheless preferred it because it improved confidence in their performance, increased ease of use, and facilitated a more immersive user experience.

Abstract P1566 Figure.

Registration-based interpolation

A method is presented to interpolate between neighboring slices in a grey-scale tomographic data set. Spatial correspondence between adjacent slices is established using a nonrigid registration algorithm based on B-splines which optimizes the normalized mutual information similarity measure. Linear interpolation of the image intensities is then carried out along the directions calculated by the registration algorithm. The registration-based method is compared to both standard linear interpolation and shape-based interpolation in 20 tomographic data sets. Results show that the proposed method statistically significantly outperforms both linear and shape-based interpolation.

Nonlinear smoothing for reduction of systematic and random errors in diffusion tensor imaging

Calculation and sorting of the eigenvectors of diffusion using diffusion tensor imaging has previously been shown to be sensitive to noise levels in the acquired data. This sensitivity manifests as random and systematic errors in the diffusion eigenvalues and derived parameters such as indices of anisotropy. An optimized application of nonlinear smoothing techniques to diffusion data prior to calculation of the diffusion tensor is shown to reduce both random and systematic errors, while causing little blurring of anatomical structures. Conversely, filtering applied to calculated images of fractional anisotropy is shown to fail in reducing systematic errors and in recovering anatomical detail. Using both real and simulated brain data sets, it is demonstrated that this approach has the potential to allow acquisition of data that would otherwise be too noisy to be of use.