Learning on 3D Meshes With Laplacian Encoding and Pooling

Geometric deep learning with adaptive full-band spatial diffusion for accurate, efficient, and robust cortical parcellation

Cortical parcellation delineates the cerebral cortex into distinct regions according to their distinctiveness in anatomy and/or function, which is a fundamental preprocess in brain cortex analysis and can influence the accuracy and specificity of subsequent neuroscientific research and clinical diagnosis. Conventional methods for cortical parcellation involve spherical mapping and multiple morphological feature computation, which are time-consuming and prone to error due to the spherical mapping process. Recent geometric learning approaches have attempted to automate this process by replacing the registration-based parcellation with deep learning-based methods. However, they have not fully addressed spherical mapping and cortical features quantification, making them sensitive to variations in mesh structures. In this work, to directly parcellate original surfaces in individual space with minimal preprocessing, we present a full-band spectral-accelerated spatial diffusion strategy for stable information propagation on highly folded cortical surfaces, contributing to adaptive learning of fine-grained geometric representations and the construction of a compact deep network (termed Cortex-Diffusion) for fully automatic parcellation. Using only raw 3D vertex coordinates and having merely 0.49 MB of learnable parameters, it demonstrates state-of-the-art parcellation accuracy, efficiency, and superior robustness to mesh resolutions and discretization patterns in both the cases of infant and adult brain imaging datasets.

Supervertex Sampling Network: A Geodesic Differential SLIC Approach for 3D Mesh

The analysis of 3D meshes with deep learning has become prevalent in computer graphics. As an essential structure, hierarchical representation is critical for mesh pooling in multiscale analysis. Existing clustering-based mesh hierarchy construction methods involve nonlinear discretization optimization operations, making them nondifferential and challenging to embed in other trainable networks for learning. Inspired by deep superpixel learning methods in image processing, we extend them from 2D images to 3D meshes by proposing a novel differentiable chart-based segmentation method named geodesic differential supervertex (GDSV). The key to the GDSV method is to ensure that the geodesic position updates are differentiable while satisfying the constraint that the renewed supervertices lie on the manifold surface. To this end, in addition to using the differential SLIC clustering algorithm to update the nonpositional features of the supervertices, a reparameterization trick, the Gumbel-Softmax trick, is employed to renew the geodesic positions of the supervertices. Therefore, the geodesic position update problem is converted into a linear matrix multiplication issue. The GDSV method can be an independent module for chart-based segmentation tasks. Meanwhile, it can be combined with the front-end feature learning network and the back-end task-specific network as a plug-in-plug-out module for training; and be applied to tasks such as shape classification, part segmentation, and 3D scene understanding. Experimental results show the excellent performance of our proposed algorithm on a range of datasets.

Laplacian2Mesh: Laplacian-Based Mesh Understanding

Geometric deep learning has sparked a rising interest in computer graphics to perform shape understanding tasks, such as shape classification and semantic segmentation. When the input is a polygonal surface, one has to suffer from the irregular mesh structure. Motivated by the geometric spectral theory, we introduce Laplacian2Mesh, a novel and flexible convolutional neural network (CNN) framework for coping with irregular triangle meshes (vertices may have any valence). By mapping the input mesh surface to the multi-dimensional Laplacian-Beltrami space, Laplacian2Mesh enables one to perform shape analysis tasks directly using the mature CNNs, without the need to deal with the irregular connectivity of the mesh structure. We further define a mesh pooling operation such that the receptive field of the network can be expanded while retaining the original vertex set as well as the connections between them. Besides, we introduce a channel-wise self-attention block to learn the individual importance of feature ingredients. Laplacian2Mesh not only decouples the geometry from the irregular connectivity of the mesh structure but also better captures the global features that are central to shape classification and segmentation. Extensive tests on various datasets demonstrate the effectiveness and efficiency of Laplacian2Mesh, particularly in terms of the capability of being vulnerable to noise to fulfill various learning tasks.

NGD-Transformer: Navigation Geodesic Distance Positional Encoding with Self-Attention Pooling for Graph Transformer on 3D Triangle Mesh

Following the significant success of the transformer in NLP and computer vision, this paper attempts to extend it to 3D triangle mesh. The aim is to determine the shape’s global representation using the transformer and capture the inherent manifold information. To this end, this paper proposes a novel learning framework named Navigation Geodesic Distance Transformer (NGD-Transformer) for 3D mesh. Specifically, this approach combined farthest point sampling with the Voronoi segmentation algorithm to spawn uniform and non-overlapping manifold patches. However, the vertex number of these patches was inconsistent. Therefore, self-attention graph pooling is employed for sorting the vertices on each patch and screening out the most representative nodes, which were then reorganized according to their scores to generate tokens and their raw feature embeddings. To better exploit the manifold properties of the mesh, this paper further proposed a novel positional encoding called navigation geodesic distance positional encoding (NGD-PE), which encodes the geodesic distance between vertices relatively and spatial symmetrically. Subsequently, the raw feature embeddings and positional encodings were summed as input embeddings fed to the graph transformer encoder to determine the global representation of the shape. Experiments on several datasets were conducted, and the experimental results show the excellent performance of our proposed method.