RGB-D Point Cloud Registration Based on Salient Object Detection

DMESH: A Structure-Preserving Diffusion Model for 3-D Mesh Denoising

Denoising diffusion models have shown a powerful capacity for generating high-quality image samples by progressively removing noise. Inspired by this, we present a diffusion-based mesh denoiser that progressively removes noise from mesh. In general, the iterative algorithm of diffusion models attempts to manipulate the overall structure and fine details of target meshes simultaneously. For this reason, it is difficult to apply the diffusion process to a mesh denoising task that removes artifacts while maintaining a structure. To address this, we formulate a structure-preserving diffusion process. Instead of diffusing the mesh vertices to be distributed as zero-centered isotopic Gaussian distribution, we diffuse each vertex into a specific noise distribution, in which the entire structure can be preserved. In addition, we propose a topology-agnostic mesh diffusion model by projecting the vertex into multiple 2-D viewpoints to efficiently learn the diffusion using a deep network. This enables the proposed method to learn the diffusion of arbitrary meshes that have an irregular topology. Finally, the denoised mesh can be obtained via refinement based on 2-D projections obtained from reverse diffusion. Through extensive experiments, we demonstrate that our method outperforms the state-of-the-art mesh denoising methods in both quantitative and qualitative evaluations.

Pose Estimation of Coil Workpieces by Automated Overhead Cranes Using an Improved Point Pair Features Algorithm

To facilitate the automation of crane operations for grabbing coil stacks in port storage areas, thereby streamlining the processes of warehousing, stacking, and transshipment for enhanced operational efficiency, this paper utilizes algorithms related to 3D point clouds for the pose estimation of coil workpieces. To overcome the limitations of the traditional point pair feature (PPF) algorithm, a novel point cloud registration algorithm is introduced. This algorithm harnesses the advantages of the PPF algorithm in describing local features and integrates it with the Generalized Iterative Closest Point (GICP) algorithm to enhance the robustness and applicability of registration. Finally, comparative experiments demonstrate that the proposed algorithm delivers superior performance. The average pose estimation errors for one, two, and three coils are 1.1%, 1.1%, and 1.2% of the coil size, respectively, with total processing times of 3.6 s, 3.4 s, and 4.7 s, meeting the practical application requirements in terms of accuracy and timing.

Norest-Net: Normal Estimation Neural Network for 3-D Noisy Point Clouds

The widely deployed ways to capture a set of unorganized points, e.g., merged laser scans, fusion of depth images, and structure-from- , usually yield a 3-D noisy point cloud. Accurate normal estimation for the noisy point cloud makes a crucial contribution to the success of various applications. However, the existing normal estimation wisdoms strive to meet a conflicting goal of simultaneously performing normal filtering and preserving surface features, which inevitably leads to inaccurate estimation results. We propose a normal estimation neural network (Norest-Net), which regards normal filtering and feature preservation as two separate tasks, so that each one is specialized rather than traded off. For full noise removal, we present a normal filtering network (NF-Net) branch by learning from the noisy height map descriptor (HMD) of each point to the ground-truth (GT) point normal; for surface feature recovery, we construct a normal refinement network (NR-Net) branch by learning from the bilaterally defiltered point normal descriptor (B-DPND) to the GT point normal. Moreover, NR-Net is detachable to be incorporated into the existing normal estimation methods to boost their performances. Norest-Net shows clear improvements over the state of the arts in both feature preservation and noise robustness on synthetic and real-world captured point clouds.

Point Cloud Registration-Driven Robust Feature Matching for 3-D Siamese Object Tracking

Learning robust feature matching between the template and search area is crucial for 3-D Siamese tracking. The core of Siamese feature matching is how to assign high feature similarity to the corresponding points between the template and the search area for precise object localization. In this article, we propose a novel point cloud registration-driven Siamese tracking framework, with the intuition that spatially aligned corresponding points (via 3-D registration) tend to achieve consistent feature representations. Specifically, our method consists of two modules, including a tracking-specific nonlocal registration (TSNR) module and a registration-aided Sinkhorn template-feature aggregation module. The registration module targets the precise spatial alignment between the template and the search area. The tracking-specific spatial distance constraint is proposed to refine the cross-attention weights in the nonlocal module for discriminative feature learning. Then, we use the weighted singular value decomposition (SVD) to compute the rigid transformation between the template and the search area and align them to achieve the desired spatially aligned corresponding points. For the feature aggregation model, we formulate the feature matching between the transformed template and the search area as an optimal transport problem and utilize the Sinkhorn optimization to search for the outlier-robust matching solution. Also, a registration-aided spatial distance map is built to improve the matching robustness in indistinguishable regions (e.g., smooth surfaces). Finally, guided by the obtained feature matching map, we aggregate the target information from the template into the search area to construct the target-specific feature, which is then fed into a CenterPoint-like detection head for object localization. Extensive experiments on KITTI, NuScenes, and Waymo datasets verify the effectiveness of our proposed method.

PlaneSeg: Building a Plug-In for Boosting Planar Region Segmentation

Existing methods in planar region segmentation suffer the problems of vague boundaries and failure to detect small-sized regions. To address these, this study presents an end-to-end framework, named PlaneSeg, which can be easily integrated into various plane segmentation models. Specifically, PlaneSeg contains three modules, namely, the edge feature extraction module, the multiscale module, and the resolution-adaptation module. First, the edge feature extraction module produces edge-aware feature maps for finer segmentation boundaries. The learned edge information acts as a constraint to mitigate inaccurate boundaries. Second, the multiscale module combines feature maps of different layers to harvest spatial and semantic information from planar objects. The multiformity of object information can help recognize small-sized objects to produce more accurate segmentation results. Third, the resolution-adaptation module fuses the feature maps produced by the two aforementioned modules. For this module, a pairwise feature fusion is adopted to resample the dropped pixels and extract more detailed features. Extensive experiments demonstrate that PlaneSeg outperforms other state-of-the-art approaches on three downstream tasks, including plane segmentation, 3-D plane reconstruction, and depth prediction. Code is available at https://github.com/nku-zhichengzhang/PlaneSeg.

Dual-Graph Attention Convolution Network for 3-D Point Cloud Classification

Three-dimensional point cloud classification is fundamental but still challenging in 3-D vision. Existing graph-based deep learning methods fail to learn both low-level extrinsic and high-level intrinsic features together. These two levels of features are critical to improving classification accuracy. To this end, we propose a dual-graph attention convolution network (DGACN). The idea of DGACN is to use two types of graph attention convolution operations with a feedback graph feature fusion mechanism. Specifically, we exploit graph geometric attention convolution to capture low-level extrinsic features in 3-D space. Furthermore, we apply graph embedding attention convolution to learn multiscale low-level extrinsic and high-level intrinsic fused graph features together. Moreover, the points belonging to different parts in real-world 3-D point cloud objects are distinguished, which results in more robust performance for 3-D point cloud classification tasks than other competitive methods, in practice. Our extensive experimental results show that the proposed network achieves state-of-the-art performance on both the synthetic ModelNet40 and real-world ScanObjectNN datasets.

Coarse Alignment Methodology of Point Cloud Based on Camera Position/Orientation Estimation Model

This study presents a methodology for the coarse alignment of light detection and ranging (LiDAR) point clouds, which involves estimating the position and orientation of each station using the pinhole camera model and a position/orientation estimation algorithm. Ground control points are obtained using LiDAR camera images and the point clouds are obtained from the reference station. The estimated position and orientation vectors are used for point cloud registration. To evaluate the accuracy of the results, the positions of the LiDAR and the target were measured using a total station, and a comparison was carried out with the results of semi-automatic registration. The proposed methodology yielded an estimated mean LiDAR position error of 0.072 m, which was similar to the semi-automatic registration value of 0.070 m. When the point clouds of each station were registered using the estimated values, the mean registration accuracy was 0.124 m, while the semi-automatic registration accuracy was 0.072 m. The high accuracy of semi-automatic registration is due to its capability for performing both coarse alignment and refined registration. The comparison between the point cloud with refined alignment using the proposed methodology and the point-to-point distance analysis revealed that the average distance was measured at 0.0117 m. Moreover, 99% of the points exhibited distances within the range of 0.0696 m.

Affine Iterative Closest Point Algorithm Based on Color Information and Correntropy for Precise Point Set Registration

In this paper, we propose a novel affine iterative closest point algorithm based on color information and correntropy, which can effectively deal with the registration problems with a large number of noise and outliers and small deformations in RGB-D datasets. Firstly, to alleviate the problem of low registration accuracy for data with weak geometric structures, we consider introducing color features into traditional affine algorithms to establish more accurate and reliable correspondences. Secondly, we introduce the correntropy measurement to overcome the influence of a large amount of noise and outliers in the RGB-D datasets, thereby further improving the registration accuracy. Experimental results demonstrate that the proposed registration algorithm has higher registration accuracy, with error reduction of almost 10 times, and achieves more stable robustness than other advanced algorithms.

An adaptive registration algorithm for zebrafish larval brain images

BACKGROUND AND OBJECTIVE

Zebrafish (Danio rerio) in their larval stages have grown increasingly popular as excellent vertebrate models for neurobiological research. Researchers can apply various tools in order to decode the neural structure patterns which can aid the understanding of vertebrate brain development. In order to do so, it is essential to map the gene expression patterns to an anatomical reference precisely. However, high accuracy in sample registration is sometimes difficult to achieve due to laboratory- or protocol-dependent variabilities.

METHODS

In this paper, we propose an accurate adaptive registration algorithm for volumetric zebrafish larval image datasets using a synergistic combination of attractive Free-Form-Deformation (FFD) and diffusive Demons algorithms. A coarse registration is achieved first for 3D volumetric data using a 3D affine transformation. A localized registration algorithm in form of a B-splines based FFD is applied next on the coarsely registered volume. Finally, the Demons algorithm is applied on this FFD registered volume for achieving fine registration by making the solution noise resilient.

RESULTS

Results Experimental procedures are carried out on a number of 72 hpf (hours post fertilization) 3D confocal zebrafish larval datasets. Comparisons with state-of-the-art methods including some ablation studies clearly demonstrate the effectiveness of the proposed method.

CONCLUSIONS

Our adaptive registration algorithm significantly aids Zebrafish imaging analysis over current methods for gene expression anatomical mapping, such as Vibe-Z. We believe the proposed solution would be able to overcome the requirement of high quality images which currently limits the applicability of Zebrafish in neuroimaging research.

Robust affine registration method using line/surface normals and correntropy criterion

The problem of matching point clouds is an efficient way of registration, which is significant for many research fields including computer vision, machine learning, and robotics. There may be linear or non-linear transformation between point clouds, but determining the affine relation is more challenging among linear cases. Various methods have been presented to overcome this problem in the literature and one of them is the affine variant of the iterative closest point (ICP) algorithm. However, traditional affine ICP variants are highly sensitive to effects such as noises, deformations, and outliers; the least-square metric is substituted with the correntropy criterion to increase the robustness of ICPs to such effects. Correntropy-based robust affine ICPs available in the literature use point-to-point metric to estimate transformation between point clouds. Conversely, in this study, a line/surface normal that examines point-to-curve or point-to-plane distances is employed together with the correntropy criterion for affine point cloud registration problems. First, the maximum correntropy criterion measure is built for line/surface normal conditions. Then, the closed-form solution that maximizes the similarity between point sets is achieved for 2D registration and extended for 3D registration. Finally, the application procedure of the developed robust affine ICP method is given and its registration performance is examined through extensive experiments on 2D and 3D point sets. The results achieved highlight that our method can align point clouds more robustly and precisely than the state-of-the-art methods in the literature, while the registration time of the process remains at reasonable levels.

A Robust Rigid Registration Framework of 3D Indoor Scene Point Clouds Based on RGB-D Information

Rigid registration of 3D indoor scenes is a fundamental yet vital task in various fields that include remote sensing (e.g., 3D reconstruction of indoor scenes), photogrammetry measurement, geometry modeling, etc. Nevertheless, state-of-the-art registration approaches still have defects when dealing with low-quality indoor scene point clouds derived from consumer-grade RGB-D sensors. The major challenge is accurately extracting correspondences between a pair of low-quality point clouds when they contain considerable noise, outliers, or weak texture features. To solve the problem, we present a point cloud registration framework in view of RGB-D information. First, we propose a point normal filter for effectively removing noise and simultaneously maintaining sharp geometric features and smooth transition regions. Second, we design a correspondence extraction scheme based on a novel descriptor encoding textural and geometry information, which can robustly establish dense correspondences between a pair of low-quality point clouds. Finally, we propose a point-to-plane registration technology via a nonconvex regularizer, which can further diminish the influence of those false correspondences and produce an exact rigid transformation between a pair of point clouds. Compared to existing state-of-the-art techniques, intensive experimental results demonstrate that our registration framework is excellent visually and numerically, especially for dealing with low-quality indoor scenes.