Predicate-Based Focus-and-Context Visualization for 3D Ultrasound

A Survey of Designs for Combined 2D+3D Visual Representations

We examine visual representations of data that make use of combinations of both 2D and 3D data mappings. Combining 2D and 3D representations is a common technique that allows viewers to understand multiple facets of the data with which they are interacting. While 3D representations focus on the spatial character of the data or the dedicated 3D data mapping, 2D representations often show abstract data properties and take advantage of the unique benefits of mapping to a plane. Many systems have used unique combinations of both types of data mappings effectively. Yet there are no systematic reviews of the methods in linking 2D and 3D representations. We systematically survey the relationships between 2D and 3D visual representations in major visualization publications-IEEE VIS, IEEE TVCG, and EuroVis-from 2012 to 2022. We closely examined 105 articles where 2D and 3D representations are connected visually, interactively, or through animation. These approaches are designed based on their visual environment, the relationships between their visual representations, and their possible layouts. Through our analysis, we introduce a design space as well as provide design guidelines for effectively linking 2D and 3D visual representations.

3D Echocardiogram Visualization: A New Method Based on "Focus + Context"

3D echocardiography (3DE) is the standard modality for visualizing heart valves and their surrounding anatomical structures. Commercial cardiovascular ultrasound systems commonly offer a set of parameters that allow clinical users to modify, in real time, visual aspects of the information contained in the echocardiogram. To our knowledge, there is currently no work that demonstrates if the methods currently used by commercial platforms are optimal. In addition, current platforms have limitations in adjusting the visibility of anatomical structures, such as reducing information that obstructs anatomical structures without removing essential clinical information. To overcome this, the present work proposes a new method for 3DE visualization based on "focus + context" (F+C), a concept which aims to present a detailed region of interest while preserving a less detailed overview of the surrounding context. The new method is intended to allow clinical users to modify parameter values differently within a certain region of interest, independently from the adjustment of contextual information. To validate this new method, a user study was conducted amongst clinical experts. As part of the user study, clinical experts adjusted parameters for five echocardiograms of patients with complete atrioventricular canal defect (CAVC) using both the method conventionally used by commercial platforms and the proposed method based on F+C. The results showed relevance for the F+C-based method to visualize 3DE of CAVC patients, where users chose significantly different parameter values with the F+C-based method.

Multi-scale RANSAC algorithm for needle localization in 3D ultrasound guided puncture surgery

Correct localization of the needle is of vital importance to guarantee successful puncture. The complexity of real US data increases the difficulties. A multi-scale random sample consensus (MS-RANSAC) algorithm is proposed in this paper to locate the needle in complicated 3D US data. The algorithm uses the radius difference between needle and other tubular human tissues to extract the correct needle location. The performance of classic RANSAC and MS-RANSAC are compared using three different datasets. Results show that MS-RANSAC can locate needle correctly in complicated condition where classic RANSAC cannot. A parallel framework of the algorithm is designed and implemented using CUDA, making it usable in real time and online.

Personalized, relevance-based Multimodal Robotic Imaging and augmented reality for Computer Assisted Interventions

In the last decade, many researchers in medical image computing and computer assisted interventions across the world focused on the development of the Virtual Physiological Human (VPH), aiming at changing the practice of medicine from classification and treatment of diseases to that of modeling and treating patients. These projects resulted in major advancements in segmentation, registration, morphological, physiological and biomechanical modeling based on state of art medical imaging as well as other sensory data. However, a major issue which has not yet come into the focus is personalizing intra-operative imaging, allowing for optimal treatment. In this paper, we discuss the personalization of imaging and visualization process with particular focus on satisfying the challenging requirements of computer assisted interventions. We discuss such requirements and review a series of scientific contributions made by our research team to tackle some of these major challenges.