A Recurrent Deep Network for Estimating the Pose of Real Indoor Images from Synthetic Image Sequences

CT-Loc: Cross-domain visual localization with a channel-wise transformer

We tackle the cross-domain visual localization problem of estimating camera position and orientation from real images without three-dimensional (3D) spatial mapping or modeling. Recent studies have shown suboptimal performance in this task owing to the photometric and geometric differences between synthetic and real images. In this study, we present a deep learning approach that uses a channel-wise transformer localization (CT-Loc) framework. Inspired by the human behavior of looking for structural landmarks to estimate one's location, CT-Loc encodes the most salient features of task-relevant objects in target scenes. To evaluate the efficacy of the proposed method in a real-world application, we built a complex and large-scale dataset of the interior of the mechanical room during operations and conducted extensive performance comparisons with the publicly available state-of-the-art University of Melbourne Corridor and Virtual KITTI 2 datasets. Compared with the otherwise best-performing BIM-PoseNet indoor camera localization model, our method significantly reduces position and orientation errors through the application of attention weights and saliency maps while also learning only the visual structural patterns (e.g., floors and doors) that are most relevant to localization tasks. Our model successfully ignores uninformative objects. This approach yields higher-level robust camera-pose regression localization results without requiring prebuilt maps. The code is available at https://github.com/kdaeho27/CT-Loc.

Single RGB Image 6D Object Grasping System Using Pixel-Wise Voting Network

A robotic system that can autonomously recognize object and grasp it in a real scene with heavy occlusion would be desirable. In this paper, we integrate the techniques of object detection, pose estimation and grasping plan on Kinova Gen3 (KG3), a 7 degrees of freedom (DOF) robotic arm with a low-performance native camera sensor, to implement an autonomous real-time 6 dimensional (6D) robotic grasping system. To estimate the object 6D pose, the pixel-wise voting network (PV-net), is applied in the grasping system. However, the PV-net method can not distinguish the object from its photo through only RGB image input. To meet the demands of a real industrial environment, a rapid analytical method on a point cloud is developed to judge whether the detected object is real or not. In addition, our system shows a stable and robust performance in different installation positions with heavily cluttered scenes.