Deep Bayesian-Assisted Keypoint Detection for Pose Estimation in Assembly Automation

Pose estimation is crucial for automating assembly tasks, yet achieving sufficient accuracy for assembly automation remains challenging and part-specific. This paper presents a novel, streamlined approach to pose estimation that facilitates automation of assembly tasks. Our proposed method employs deep learning on a limited number of annotated images to identify a set of keypoints on the parts of interest. To compensate for network shortcomings and enhance accuracy we incorporated a Bayesian updating stage that leverages our detailed knowledge of the assembly part design. This Bayesian updating step refines the network output, significantly improving pose estimation accuracy. For this purpose, we utilized a subset of network-generated keypoint positions with higher quality as measurements, while for the remaining keypoints, the network outputs only serve as priors. The geometry data aid in constructing likelihood functions, which in turn result in enhanced posterior distributions of keypoint pixel positions. We then employed the maximum a posteriori (MAP) estimates of keypoint locations to obtain a final pose, allowing for an update to the nominal assembly trajectory. We evaluated our method on a 14-point snap-fit dash trim assembly for a Ford Mustang dashboard, demonstrating promising results. Our approach does not require tailoring to new applications, nor does it rely on extensive machine learning expertise or large amounts of training data. This makes our method a scalable and adaptable solution for the production floors.

Comparison of 3D Sensors for Automating Bolt-Tightening Operations in the Automotive Industry

Machine vision systems are widely used in assembly lines for providing sensing abilities to robots to allow them to handle dynamic environments. This paper presents a comparison of 3D sensors for evaluating which one is best suited for usage in a machine vision system for robotic fastening operations within an automotive assembly line. The perception system is necessary for taking into account the position uncertainty that arises from the vehicles being transported in an aerial conveyor. Three sensors with different working principles were compared, namely laser triangulation (SICK TriSpector1030), structured light with sequential stripe patterns (Photoneo PhoXi S) and structured light with infrared speckle pattern (Asus Xtion Pro Live). The accuracy of the sensors was measured by computing the root mean square error (RMSE) of the point cloud registrations between their scans and two types of reference point clouds, namely, CAD files and 3D sensor scans. Overall, the RMSE was lower when using sensor scans, with the SICK TriSpector1030 achieving the best results (0.25 mm ± 0.03 mm), the Photoneo PhoXi S having the intermediate performance (0.49 mm ± 0.14 mm) and the Asus Xtion Pro Live obtaining the higher RMSE (1.01 mm ± 0.11 mm). Considering the use case requirements, the final machine vision system relied on the SICK TriSpector1030 sensor and was integrated with a collaborative robot, which was successfully deployed in an vehicle assembly line, achieving 94% success in 53,400 screwing operations.

Evaluation of Collaborative Robot Sustainable Integration in Manufacturing Assembly by Using Process Time Savings

Collaborative robots are enablers of flexibility in the current dynamic and uncertain manufacturing environment. Decision making on its implementation requires technical feasibility, involving productivity and workforce implications that should be faced in an integrated perspective in processes where many components of different materials are assembled in products of increasing diversity and complexity. This study introduces two new parametric models for collaborative robotics, formulated in order to evaluate the differential cost of assembly (economic dimension) and the differential income from taxes that supports short-term workforce displacement (social dimension) in cobot implementation. Updated techno-economical parameters are selected for assessing feasibility ranges of application in different production scenarios. Next, the influence curves of productivity gain for a feasible implementation of cobot establish thresholds for decision making under both criteria. The results show the need for productivity gains that are significantly lower in high-wage scenarios than in low-wage scenarios; however, in a joint approach, breakeven productivity gain is always higher for the social dimension threshold than for the economic requirement of cost-effective manufacturing, with a higher gap in low-wage cases. The detailed analysis of a real case study of cobot implementation for assembly demonstrates the practical application of models and potential for future research.