Mesh segmentation with concavity-aware fields

Improving rupture status prediction for intracranial aneurysms using wall shear stress informatics

BACKGROUND

Wall shear stress (WSS) plays a crucial role in the natural history of intracranial aneurysms (IA). However, spatial variations among WSS have rarely been utilized to correlate with IAs' natural history. This study aims to establish the feasibility of using spatial patterns of WSS data to predict IAs' rupture status (i.e., ruptured versus unruptured).

METHODS

"Patient-specific" computational fluid dynamics (CFD) simulations were performed for 112 IAs; each IA's rupture status was known from medical records. Recall that CFD-simulated hemodynamics data (wall shear stress and its derivatives) are located on unstructured meshes. Hence, we mapped WSS data from an unstructured grid onto a unit disk (i.e., a uniformly sampled polar coordinate system); data in a uniformly sampled polar system is equivalent to image data. Mapped WSS data (onto the unit disk) were readily available for Radiomics analysis to extract spatial patterns of WSS data. We named this innovative technology "WSS-informatics" (i.e., using informatics techniques to analyze WSS data); the usefulness of WSS-informatics was demonstrated during the predictive modeling of IAs' rupture status.

RESULTS

None of the conventional WSS parameters correlated to IAs' rupture status. However, WSS-informatics metrics were discriminative (p-value < 0.05) to IAs' rupture status. Furthermore, predictive models with WSS-informatics features could significantly improve the prediction performance (area under the receiver operating characteristic curve [AUROC]: 0.78 vs. 0.85; p-value < 0.01).

CONCLUSION

The proposed innovations enabled the first study to use spatial patterns of WSS data to improve the predictive modeling of IAs' rupture status.

SEG-MAT: 3D Shape Segmentation Using Medial Axis Transform

Segmenting arbitrary 3D objects into constituent parts that are structurally meaningful is a fundamental problem encountered in a wide range of computer graphics applications. Existing methods for 3D shape segmentation suffer from complex geometry processing and heavy computation caused by using low-level features and fragmented segmentation results due to the lack of global consideration. We present an efficient method, called SEG-MAT, based on the medial axis transform (MAT) of the input shape. Specifically, with the rich geometrical and structural information encoded in the MAT, we are able to develop a simple and principled approach to effectively identify the various types of junctions between different parts of a 3D shape. Extensive evaluations and comparisons show that our method outperforms the state-of-the-art methods in terms of segmentation quality and is also one order of magnitude faster.

Grasp Pose Detection Based on Shape Simplification

For robots in an unstructured work environment, grasping unknown objects that have neither model data nor RGB data is very important. The key to robotic autonomous grasping is not only in the judgment of object type but also in the shape of the object. We present a new grasping approach based on the basic compositions of objects. The simplification of complex objects is conducive to the description of object shape and provides effective ideas for the selection of grasping strategies. First, the depth camera is used to obtain partial 3D data of the target object. Then the 3D data are segmented and the segmented parts are simplified to a cylinder, a sphere, an ellipsoid, and a parallelepiped according to the geometric and semantic shape characteristics. The grasp pose is constrained according to the simplified shape feature and the core part of the object is used for grasping training using deep learning. The grasping model was evaluated in a simulation experiment and robot experiment, and the experiment result shows that learned grasp score using simplified constraints is more robust to gripper pose uncertainty than without simplified constraint.

VoxSegNet: Volumetric CNNs for Semantic Part Segmentation of 3D Shapes

Volumetric representation has been widely used for 3D deep learning in shape analysis due to its generalization ability and regular data format. However, for fine-grained tasks like part segmentation, volumetric data has not been widely adopted compared to other representations. Aiming at delivering an effective volumetric method for 3D shape part segmentation, this paper proposes a novel volumetric convolutional neural network. Our method can extract discriminative features encoding detailed information from voxelized 3D data under limited resolution. To this purpose, a spatial dense extraction (SDE) module is designed to preserve spatial resolution during feature extraction procedure, alleviating the loss of details caused by sub-sampling operations such as max pooling. An attention feature aggregation (AFA) module is also introduced to adaptively select informative features from different abstraction levels, leading to segmentation with both semantic consistency and high accuracy of details. Experimental results demonstrate that promising results can be achieved by using volumetric data, with part segmentation accuracy comparable or superior to state-of-the-art non-volumetric methods.

Scribble-Based 3D Shape Segmentation via Weakly-Supervised Learning

Shape segmentation is a fundamental problem in shape analysis. Previous research shows that prior knowledge helps to improve the segmentation accuracy and quality. However, completely labeling each 3D shape in a large training data set requires a heavy manual workload. In this paper, we propose a novel weakly-supervised algorithm for segmenting 3D shapes using deep learning. Our method jointly propagates information from scribbles to unlabeled faces and learns deep neural network parameters. Therefore, it does not rely on completely labeled training shapes and only needs a really simple and convenient scribble-based partially labeling process, instead of the extremely time-consuming and tedious fully labeling processes. Various experimental results demonstrate the proposed method's superior segmentation performance over the previous unsupervised approaches and comparable segmentation performance to the state-of-the-art fully supervised methods.

Spectral Mesh Segmentation via l0 Gradient Minimization

Mesh segmentation is a process of partitioning a mesh model into meaningful parts - a fundamental problem in various disciplines. This paper introduces a novel mesh segmentation method inspired by sparsity pursuit. Based on the local geometric and topological information of a given mesh, we build a Laplacian matrix whose Fiedler vector is used to characterize the uniformity among elements of the same segment. By analyzing the Fiedler vector, we reformulate the mesh segmentation problem as a l₀ gradient minimization problem. To solve this problem efficiently, we adopt a coarse-to-fine strategy. A fast heuristic algorithm is first devised to find a rational coarse segmentation, and then an optimization algorithm based on the alternating direction method of multiplier (ADMM) is proposed to refine the segment boundaries within their local regions. To extract the inherent hierarchical structure of the given mesh, our method performs segmentation in a recursive way. Experimental results demonstrate that the presented method outperforms the state-of-the-art segmentation methods when evaluated on the Princeton Segmentation Benchmark, the LIFL/LIRIS Segmentation Benchmark and a number of other complex meshes.

Sparse Graph Regularized Mesh Color Edit Propagation

Mesh color edit propagation aims to propagate the color from a few color strokes to the whole mesh, which is useful for mesh colorization, color enhancement and color editing, etc. Compared with image edit propagation, luminance information is not available for 3D mesh data, so the color edit propagation is more difficult on 3D meshes than images, with far less research carried out. This paper proposes a novel solution based on sparse graph regularization. Firstly, a few color strokes are interactively drawn by the user, and then the color will be propagated to the whole mesh by minimizing a sparse graph regularized nonlinear energy function. The proposed method effectively measures geometric similarity over shapes by using a set of complementary multiscale feature descriptors, and effectively controls color bleeding via a sparse ℓ1 optimization rather than quadratic minimization used in existing work. The proposed framework can be applied for the task of interactive mesh colorization, mesh color enhancement and mesh color editing. Extensive qualitative and quantitative experiments show that the proposed method outperforms the state-of-the-art methods.

3D Tooth Segmentation and Labeling Using Deep Convolutional Neural Networks

In this paper, we present a novel approach for 3D dental model segmentation via deep Convolutional Neural Networks (CNNs). Traditional geometry-based methods tend to receive undesirable results due to the complex appearance of human teeth (e.g., missing/rotten teeth, feature-less regions, crowding teeth, extra medical attachments, etc.). Furthermore, labeling of individual tooth is hardly enabled in traditional tooth segmentation methods. To address these issues, we propose to learn a generic and robust segmentation model by exploiting deep Neural Networks, namely NNs. The segmentation task is achieved by labeling each mesh face. We extract a set of geometry features as face feature representations. In the training step, the network is fed with those features, and produces a probability vector, of which each element indicates the probability a face belonging to the corresponding model part. To this end, we extensively experiment with various network structures, and eventually arrive at a 2-level hierarchical CNNs structure for tooth segmentation: one for teeth-gingiva labeling and the other for inter-teeth labeling. Further, we propose a novel boundary-aware tooth simplification method to significantly improve efficiency in the stage of feature extraction. After CNNs prediction, we do graph-based label optimization and further refine the boundary with an improved version of fuzzy clustering. The accuracy of our mesh labeling method exceeds that of the state-of-art geometry-based methods, reaching 99.06 percent measured by area which is directly applicable in orthodontic CAD systems. It is also robust to any possible foreign matters on model surface, e.g., air bubbles, dental accessories, and many more.

Toward Support-free 3D Printing: A Skeletal Approach for Partitioning Models

Minimizing support structures is crucial in reducing 3D printing material and time. Partition-based methods are efficient means in realizing this objective. Although some algorithms exist for support-free fabrication of solid models, no algorithm ever considers the problem of support-free fabrication for shell models (i.e., hollowed meshes). In this paper, we present a skeleton-based algorithm for partitioning a 3D surface model into the least number of parts for 3D printing without using any support structure. To achieve support-free fabrication while minimizing the effect of the seams and cracks that are inevitably induced by the partition, which affect the aesthetics and strength of the final assembled surface, we put forward an optimization system with the minimization of the number of partitions and the total length of the cuts, under the constraints of support-free printing angle. Our approach is particularly tailored for shell models, and it can be applicable to solid models as well. We first rigorously show that the optimization problem is NP-hard and then propose a stochastic method to find an optimal solution to the objectives. We propose a polynomial-time algorithm for a special case when the skeleton graph satisfies the requirement that the number of partitioned parts and the degree of each node are bounded by a small constant. We evaluate our partition method on a number of 3D models and validate our method by 3D printing experiments.

Semi-automatic segmentation of femur based on harmonic barrier

BACKGROUND AND OBJECTIVE

Segmentation of the femur from the hip joint in computed tomography (CT) is an important preliminary step in hip surgery planning and simulation. However, this is a time-consuming and challenging task due to the weak boundary, the varying topology of the hip joint, and the extremely narrow or blurred space between the femoral head and the acetabulum. To address these problems, this study proposed a semi-automatic segmentation framework based on harmonic fields for accurate segmentation.

METHODS

The proposed method comprises three steps. First, with high-level information provided by the user, shape information provided by neighboring slices as well as the statistical information in the mask, a region selection method is proposed to effectively locate joint space for the harmonic field. Second, incorporated with an improved gradient, the harmonic field is used to adaptively extract a curve as the barrier that separates the femoral head from the acetabulum accurately. Third, a divide and conquer segmentation strategy based on the harmonic barrier is used to combine the femoral head part and body part as the final segmentation result.

RESULTS

We have tested 40 hips with considerately narrow or disappeared joint spaces. The experimental results are evaluated based on Jaccard, Dice, directional cut discrepancy (DCD) and receiver operating characteristic (ROC), and we achieve the higher Jaccard of 84.02%, Dice of 85.96%, area under curve (AUC) of 89.3%, and the lower error with DCD of 0.52mm. The effective ratio of our method is 79.1% even for cases with severe malformation. The results show that our method performs best in terms of effectiveness and accuracy on the whole data set.

CONCLUSIONS

The proposed method is efficient to segment femurs with narrow joint space. The accurate segmentation results can assist the physicians for osteoarthritis diagnosis in future.

Unsupervised Spectral Mesh Segmentation Driven by Heterogeneous Graphs

A fully automatic mesh segmentation scheme using heterogeneous graphs is presented. We introduce a spectral framework where local geometry affinities are coupled with surface patch affinities. A heterogeneous graph is constructed combining two distinct graphs: a weighted graph based on adjacency of patches of an initial over-segmentation, and the weighted dual mesh graph. The partitioning relies on processing each eigenvector of the heterogeneous graph Laplacian individually, taking into account the nodal set and nodal domain theory. Experiments on standard datasets show that the proposed unsupervised approach outperforms the state-of-the-art unsupervised methodologies and is comparable to the best supervised approaches.

Generalized Local-to-Global Shape Feature Detection Based on Graph Wavelets

Informative and discriminative feature descriptors are vital in qualitative and quantitative shape analysis for a large variety of graphics applications. Conventional feature descriptors primarily concentrate on discontinuity of certain differential attributes at different orders that naturally give rise to their discriminative power in depicting point, line, small patch features, etc. This paper seeks novel strategies to define generalized, user-specified features anywhere on shapes. Our new region-based feature descriptors are constructed primarily with the powerful spectral graph wavelets (SGWs) that are both multi-scale and multi-level in nature, incorporating both local (differential) and global (integral) information. To our best knowledge, this is the first attempt to organize SGWs in a hierarchical way and unite them with the bi-harmonic diffusion field towards quantitative region-based shape analysis. Furthermore, we develop a local-to-global shape feature detection framework to facilitate a host of graphics applications, including partial matching without point-wise correspondence, coarse-to-fine recognition, model recognition, etc. Through the extensive experiments and comprehensive comparisons with the state-of-the-art, our framework has exhibited many attractive advantages such as being geometry-aware, robust, discriminative, isometry-invariant, etc.

Interactive Tooth Separation from Dental Model Using Segmentation Field

Tooth segmentation on dental model is an essential step of computer-aided-design systems for orthodontic virtual treatment planning. However, fast and accurate identifying cutting boundary to separate teeth from dental model still remains a challenge, due to various geometrical shapes of teeth, complex tooth arrangements, different dental model qualities, and varying degrees of crowding problems. Most segmentation approaches presented before are not able to achieve a balance between fine segmentation results and simple operating procedures with less time consumption. In this article, we present a novel, effective and efficient framework that achieves tooth segmentation based on a segmentation field, which is solved by a linear system defined by a discrete Laplace-Beltrami operator with Dirichlet boundary conditions. A set of contour lines are sampled from the smooth scalar field, and candidate cutting boundaries can be detected from concave regions with large variations of field data. The sensitivity to concave seams of the segmentation field facilitates effective tooth partition, as well as avoids obtaining appropriate curvature threshold value, which is unreliable in some case. Our tooth segmentation algorithm is robust to dental models with low quality, as well as is effective to dental models with different levels of crowding problems. The experiments, including segmentation tests of varying dental models with different complexity, experiments on dental meshes with different modeling resolutions and surface noises and comparison between our method and the morphologic skeleton segmentation method are conducted, thus demonstrating the effectiveness of our method.

A semi-automatic computer-aided method for surgical template design

This paper presents a generalized integrated framework of semi-automatic surgical template design. Several algorithms were implemented including the mesh segmentation, offset surface generation, collision detection, ruled surface generation, etc., and a special software named TemDesigner was developed. With a simple user interface, a customized template can be semi- automatically designed according to the preoperative plan. Firstly, mesh segmentation with signed scalar of vertex is utilized to partition the inner surface from the input surface mesh based on the indicated point loop. Then, the offset surface of the inner surface is obtained through contouring the distance field of the inner surface, and segmented to generate the outer surface. Ruled surface is employed to connect inner and outer surfaces. Finally, drilling tubes are generated according to the preoperative plan through collision detection and merging. It has been applied to the template design for various kinds of surgeries, including oral implantology, cervical pedicle screw insertion, iliosacral screw insertion and osteotomy, demonstrating the efficiency, functionality and generality of our method.

Automatic Tooth Segmentation of Dental Mesh Based on Harmonic Fields

An important preprocess in computer-aided orthodontics is to segment teeth from the dental models accurately, which should involve manual interactions as few as possible. But fully automatic partition of all teeth is not a trivial task, since teeth occur in different shapes and their arrangements vary substantially from one individual to another. The difficulty is exacerbated when severe teeth malocclusion and crowding problems occur, which is a common occurrence in clinical cases. Most published methods in this area either are inaccurate or require lots of manual interactions. Motivated by the state-of-the-art general mesh segmentation methods that adopted the theory of harmonic field to detect partition boundaries, this paper proposes a novel, dental-targeted segmentation framework for dental meshes. With a specially designed weighting scheme and a strategy of a priori knowledge to guide the assignment of harmonic constraints, this method can identify teeth partition boundaries effectively. Extensive experiments and quantitative analysis demonstrate that the proposed method is able to partition high-quality teeth automatically with robustness and efficiency.

Interactive tooth partition of dental mesh base on tooth-target harmonic field

The accurate tooth partition of dental mesh is a crucial step in computer-aided orthodontics. However, tooth boundary identification is not a trivial task for tooth partition, since different shapes and their arrangements vary substantially among common clinical cases. Though curvature field is traditionally used for identifying boundaries, it is normally not reliable enough. Other methods may improve the accuracy, but require intensive user interaction. Motivated by state-of-the-art general interactive mesh segmentation methods, this paper proposes a novel tooth-target partition framework that employs harmonic fields to partition teeth accurately and effectively. In addition, a refining strategy is introduced to successfully segment teeth from the complicated dental model with indistinctive tooth boundaries on its lingual side surface, addressing an issue that had not been solved properly before. To utilise high-level information provided by the user, smart and intuitive user interfaces are also proposed with minimum interaction. In fact, most published interactive methods specifically designed for tooth partition are lacking efficient user interfaces. Extensive experiments and quantitative analyses show that our tooth partition method outperforms the state-of-the-art approaches in terms of accuracy, robustness and efficiency.

Anisotropic elliptic PDEs for feature classification

The extraction and classification of multitype (point, curve, patch) features on manifolds are extremely challenging, due to the lack of rigorous definition for diverse feature forms. This paper seeks a novel solution of multitype features in a mathematically rigorous way and proposes an efficient method for feature classification on manifolds. We tackle this challenge by exploring a quasi-harmonic field (QHF) generated by elliptic PDEs, which is the stable state of heat diffusion governed by anisotropic diffusion tensor. Diffusion tensor locally encodes shape geometry and controls velocity and direction of the diffusion process. The global QHF weaves points into smooth regions separated by ridges and has superior performance in combating noise/holes. Our method's originality is highlighted by the integration of locally defined diffusion tensor and globally defined elliptic PDEs in an anisotropic manner. At the computational front, the heat diffusion PDE becomes a linear system with Dirichlet condition at heat sources (called seeds). Our new algorithms afford automatic seed selection, enhanced by a fast update procedure in a high-dimensional space. By employing diffusion probability, our method can handle both manufactured parts and organic objects. Various experiments demonstrate the flexibility and high performance of our method.

Interactive decomposition and mapping of saccular cerebral aneurysms using harmonic functions: its first application with "patient-specific" computational fluid dynamics (CFD) simulations

Recent developments in medical imaging and advanced computer modeling simulations) now enable studies designed to correlate either simulated or measured "patient-specific" parameters with the natural history of intracranial aneurysm i.e., ruptured or unruptured. To achieve significance, however, these studies require rigorous comparison of large amounts of data from large numbers of aneurysms, many of which are quite dissimilar anatomically. In this study, we present a method that can likely facilitate such studies as its application could potentially simplify an objective comparison of surface-based parameters of interest such as wall shear stress and blood pressure using large multi-patient, multi-institutional data sets. Based on the concept of harmonic function/field, we present a unified and simple approach for mapping the surface of an aneurysm onto a unit disc. Requiring minimal human interactions the algorithm first decomposes the vessel geometry into 1) target aneurysm and 2) parent artery and any adjacent branches; it, then, maps the segmented aneurysm surface onto a unit disk. In particular, the decomposition of the vessel geometry quantitatively exploits the unique combination of three sets of information regarding the shape of the relevant vasculature: 1) a distance metric defining the spatially varying deviation from a tubular characteristic (i.e., cylindrical structure) of a normal parent artery, 2) local curvatures and 3) local concavities at the junction/interface between an aneurysm and its parent artery. These three sets of resultant shape/geometrical data are then combined to construct a linear system of the Laplacian equation with a novel shape-sensitive weighting scheme. The solution to such a linear system is a shape-sensitive harmonic function/field whose iso-lines will densely gather at the border between the normal parent artery and the aneurysm. Finally, a simple ranking system is utilized to select the best candidate among all possible iso-lines. Quantitative analysis using “patient-specific” aneurysm geometries taken from our internal database demonstrated that the technique is robust. Similar results were obtained from aneurysms having widely different geometries (bifurcation, terminal and lateral aneurysms). Application of our method should allow for meaningful, reliable and reproducible model-to-model comparisons of surface-based physiological and hemodynamic parameters.

Dot scissor: a single-click interface for mesh segmentation

This paper presents a very easy-to-use interactive tool, which we call dot scissor, for mesh segmentation. The user's effort is reduced to placing only a single click where a cut is desired. Such a simple interface is made possible by a directional search strategy supported by a concavity-aware harmonic field and a robust voting scheme that selects the best isoline as the cut. With a concavity-aware weighting scheme, the harmonic fields gather dense isolines along concave regions which are natural boundaries of semantic components. The voting scheme relies on an isoline-face scoring mechanism that considers both shape geometry and user intent. We show by extensive experiments and quantitative analysis that our tool advances the state-of-the-art segmentation methods in both simplicity of use and segmentation quality.