Direct volume manipulation for visualizing intraoperative liver resection process

Evolution of Three-Dimensional Computed Tomography Imaging in Thoracic Surgery

Radiologic reconstruction technology allows the wide use of three-dimensional (3D) computed tomography (CT) images in thoracic surgery. A minimally invasive surgery has become one of the standard therapies in thoracic surgery, and therefore, the need for preoperative and intraoperative simulations has increased. Three-dimensional CT images have been extensively used, and various types of software have been developed to reconstruct 3D-CT images for surgical simulation worldwide. Several software types have been commercialized and widely used by not only radiologists and technicians, but also thoracic surgeons. Three-dimensional CT images are helpful surgical guides; however, in almost all cases, they provide only static images, different from the intraoperative views. Lungs are soft and variable organs that can easily change shape by intraoperative inflation/deflation and surgical procedures. To address this issue, we have developed a novel software called the Resection Process Map (RPM), which creates variable virtual 3D images. Herein, we introduce the RPM and its development by tracking the history of 3D CT imaging in thoracic surgery. The RPM could help develop a real-time and accurate surgical navigation system for thoracic surgery.

Image-to-Graph Convolutional Network for 2D/3D Deformable Model Registration of Low-Contrast Organs

Organ shape reconstruction based on a single-projection image during treatment has wide clinical scope, e.g., in image-guided radiotherapy and surgical guidance. We propose an image-to-graph convolutional network that achieves deformable registration of a three-dimensional (3D) organ mesh for a low-contrast two-dimensional (2D) projection image. This framework enables simultaneous training of two types of transformation: from the 2D projection image to a displacement map, and from the sampled per-vertex feature to a 3D displacement that satisfies the geometrical constraint of the mesh structure. Assuming application to radiation therapy, the 2D/3D deformable registration performance is verified for multiple abdominal organs that have not been targeted to date, i.e., the liver, stomach, duodenum, and kidney, and for pancreatic cancer. The experimental results show shape prediction considering relationships among multiple organs can be used to predict respiratory motion and deformation from digitally reconstructed radiographs with clinically acceptable accuracy.

Surface deformation analysis of collapsed lungs using model-based shape matching

Purpose

To facilitate intraoperative localization of lung nodules, this study used model-based shape matching techniques to analyze the inter-subject three-dimensional surface deformation induced by pneumothorax. Methods: Contrast- enhanced computed tomography (CT) images of the left lungs of 11 live beagle dogs were acquired at two bronchial pressures (14 and 2 cm\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\,\hbox {H}_2\hbox {O}$$\end{document}H2O). To address shape matching problems for largely deformed lung images with pixel intensity shift, a complete Laplacian-based shape matching solution that optimizes the differential displacement field was introduced.

Results

Experiments were performed to confirm the methods’ registration accuracy using CT images of lungs. Shape similarity and target displacement errors in the registered models were improved compared with those from existing shape matching methods. Spatial displacement of the whole lung’s surface was visualized with an average error of within 5 mm.

Conclusion

The proposed methods address problems with the matching of surfaces with large curvatures and deformations and achieved smaller registration errors than existing shape matching methods, even at the tip and ridge regions. The findings and inter-subject statistical representation are directly available for further research on pneumothorax deformation modeling.

Electronic supplementary material

The online version of this article (10.1007/s11548-019-02013-0) contains supplementary material, which is available to authorized users.

Mathematical Modeling and Virtual Reality Simulation of Surgical Tool Interactions With Soft Tissue: A Review and Prospective

This is an informed assessment of the state of the art and an extensive inventory of modeling approaches and methods for soft tissue/medical cutting tool interaction and of the associated medical processes and phenomena. Modeling and simulation through numerical, theoretical, computational, experimental, and other methods was discussed in comprehensive review sections each of which is concluded with a plausible prospective discussion biased toward the development of so-called virtual reality (VR) simulator environments. The finalized prospective section reflects on the future demands in the area of soft tissue cutting modeling and simulation mostly from a conceptual angle with emphasis on VR development requirements including real-time VR simulator response, cost-effective “close-to-reality” VR implementations, and other demands. The review sections that serve as the basis for the suggested prospective needs are categorized based on: (1) Major VR simulator applications including virtual surgery education, training, operation planning, intraoperative simulation, image-guided surgery, etc. and VR simulator types, e.g., generic, patient-specific and surgery-specific and (2) Available numerical, theoretical, and computational methods in terms of robustness, time effectiveness, computational cost, error control, and accuracy of modeling of certain types of virtual surgical interventions and their experimental validation, geared toward ethically driven artificial “phantom” tissue-based approaches. Digital data processing methods used in modeling of various feedback modalities in VR environments are also discussed.

Deformable resection process map for intraoperative cutting guides

In this paper, we will introduce the concept of deformable resection process mapping as a time-varying geometric guide for soft tissue tumor resection procedures. The deformable resection process map (RPM) estimates the local appearance of vascular structures after cuts as a novel guide. The RPM can be directly generated from patient-specific medical images using volumetric resampling techniques. Since user input of some cutting points is the only requirement for generating the RPM, the developed software will be directly available for clinical use to preview surgical procedures and intraoperative workflow management without time-consuming setups or additional workloads. We used the CT images of the patients with hepatic cancer for the experiments. The performance of our method in the shape representation for a curved cut surface was compared with that of conventional shape modeling methods.

Visualization of boundaries in volumetric data sets through a what material you pick is what boundary you see approach

Transfer function design is a key issue in direct volume rendering. Many sophisticated transfer functions have been proposed to visualize boundaries in volumetric data sets such as computed tomography and magnetic resonance imaging. However, it is still conventionally challenging to reliably detect boundaries. Meanwhile, the interactive strategy is complicated for new users or even experts. In this paper, we first propose the human-centric boundary extraction criteria and our boundary model. Based on the model we present a boundary visualization method through a what material you pick is what boundary you see approach. Users can pick out the material of interest to directly convey semantics. In addition, the 3-D canny edge detection is utilized to ensure the good localization of boundaries. Furthermore, we establish a point-to-material distance measure to guarantee the accuracy and integrity of boundaries. The proposed boundary visualization is intuitive and flexible for the exploration of volumetric data.