Hand-Eye Calibration in Visually-Guided Robot Grinding

Joint Calibration Method for Robot Measurement Systems

Robot measurement systems with a binocular planar structured light camera (3D camera) installed on a robot end-effector are often used to measure workpieces' shapes and positions. However, the measurement accuracy is jointly influenced by the robot kinematics, camera-to-robot installation, and 3D camera measurement errors. Incomplete calibration of these errors can result in inaccurate measurements. This paper proposes a joint calibration method considering these three error types to achieve overall calibration. In this method, error models of the robot kinematics and camera-to-robot installation are formulated using Lie algebra. Then, a pillow error model is proposed for the 3D camera based on its error distribution and measurement principle. These error models are combined to construct a joint model based on homogeneous transformation. Finally, the calibration problem is transformed into a stepwise optimization problem that minimizes the sum of the relative position error between the calibrator and robot, and analytical solutions for the calibration parameters are derived. Simulation and experiment results demonstrate that the joint calibration method effectively improves the measurement accuracy, reducing the mean positioning error from over 2.5228 mm to 0.2629 mm and the mean distance error from over 0.1488 mm to 0.1232 mm.

Puncture site decision method for venipuncture robot based on near-infrared vision and multiobjective optimization

Venipuncture robots have superior perception and stability to humans and are expected to replace manual venipuncture. However, their use is greatly restricted because they cannot make decisions regarding the puncture sites. Thus, this study presents a multi-information fusion method for determining puncture sites for venipuncture robots to improve their autonomy in the case of limited resources. Here, numerous images have been gathered and processed to establish an image dataset of human forearms for training the U-Net with the soft attention mechanism (SAU-Net) for vein segmentation. Then, the veins are segmented from the images, feature information is extracted based on near-infrared vision, and a multiobjective optimization model for puncture site decision is provided by considering the depth, diameter, curvature, and length of the vein to determine the optimal puncture site. Experiments demonstrate that the method achieves a segmentation accuracy of 91.2% and a vein extraction rate of 86.7% while achieving the Pareto solution set (average time: 1.458 s) and optimal results for each vessel. Finally, a near-infrared camera is applied to the venipuncture robot to segment veins and determine puncture sites in real time, with the results transmitted back to the robot for an attitude adjustment. Consequently, this method can enhance the autonomy of venipuncture robots if implemented dramatically.

Two-Stream Spatial-Temporal Graph Convolutional Networks for Driver Drowsiness Detection

Convolutional neural networks (CNNs) have achieved remarkable performance in driver drowsiness detection based on the extraction of deep features of drivers' faces. However, the performance of driver drowsiness detection methods decreases sharply when complications, such as illumination changes in the cab, occlusions and shadows on the driver's face, and variations in the driver's head pose, occur. In addition, current driver drowsiness detection methods are not capable of distinguishing between driver states, such as talking versus yawning or blinking versus closing eyes. Therefore, technical challenges remain in driver drowsiness detection. In this article, we propose a novel and robust two-stream spatial-temporal graph convolutional network (2s-STGCN) for driver drowsiness detection to solve the above-mentioned challenges. To take advantage of the spatial and temporal features of the input data, we use a facial landmark detection method to extract the driver's facial landmarks from real-time videos and then obtain the driver drowsiness detection result by 2s-STGCN. Unlike existing methods, our proposed method uses videos rather than consecutive video frames as processing units. This is the first effort to exploit these processing units in the field of driver drowsiness detection. Moreover, the two-stream framework not only models both the spatial and temporal features but also models both the first-order and second-order information simultaneously, thereby notably improving driver drowsiness detection. Extensive experiments have been performed on the yawn detection dataset (YawDD) and the National TsingHua University drowsy driver detection (NTHU-DDD) dataset. The experimental results validate the feasibility of the proposed method. This method achieves an average accuracy of 93.4% on the YawDD dataset and an average accuracy of 92.7% on the evaluation set of the NTHU-DDD dataset.

A novel selected force controlling method for improving robotic grinding accuracy of complex curved blade

Nonlinear time-varying contact state is a crucial factor to prevent the traditional robotic belt grinding method from precision machining of blade. In this case, a novel selected force controlling method (SFC) with consideration of regional division (RD) based on machining allowance is proposed for improving robotic grinding accuracy of complex curved blade, on basis of the self-developed adaptive impedance controller. Ideal normal grinding force at each cutter-contact (CC) point is calculated by principal curvature radius and regional allowance of blade surface. Then, the CC points with similar ideal normal grinding force are divided into one region along grinding path based on the force threshold. Furthermore, an adaptive impedance controller with neural network online compensation algorithm (AICNN) is developed, and the verification test results of grinding four profile areas of intake side, exhaust side, convex and concave, indicate that the force control accuracy with AICNN has increased by 80.33%, 50.58%, 82.65% and 69.01% than that without the controller, respectively. Based on this, the grinding experiment of typical turbine blade is conducted with SFC, and the surface profile accuracy values at the four profile areas have evidently improved by 48.79%, 35.67%, 59.54%, and 66.90% than that with conventional grinding (CG), respectively.

Hand-eye calibration method with a three-dimensional-vision sensor considering the rotation parameters of the robot pose

Hand-eye calibration is a fundamental step for a robot equipped with vision systems. However, this problem usually interacts with robot calibration because robot geometric parameters are not very precise. In this article, a new calibration method considering the rotation parameters of the robot pose is proposed. First, a constraint least square model is established assuming that each spherical center measurement of standard ball is equal in the robot base frame, which provides an initial solution. To further improve the solution accuracy, a nonlinear calibration model in the sensor frame is established. Since it can reduce one error accumulation process, a more accurate reference point can be used for optimization. Then, the rotation parameters of the robot pose whose slight errors cause large disturbance to the solution are selected by analyzing the coefficient matrices of the error items. Finally, the hand-eye transformation parameters are refined together with the rotation parameters in the nonlinear optimization solution. Some comparative simulations are performed between the modified least square method, constrained least square method, and the proposed method. The experiments are conducted on a 5-axis hybrid robot named TriMule to demonstrate the superior accuracy of the proposed method.

A Robust Adaptive Trajectory Tracking Algorithm Using SMC and Machine Learning for FFSGRs with Actuator Dead Zones

The actuator dead zone of free-form surface grinding robots (FFSGRs) is very common in the grinding process and has a great impact on the grinding quality of a workpiece. In this paper, an improved trajectory tracking algorithm for an FFSGR with an asymmetric actuator dead zone was proposed with consideration of friction forces, model uncertainties, and external disturbances. The presented control algorithm was based on the machine learning and sliding mode control (SMC) methods. The control compensator used neural networks to estimate the actuator’s dead zone and eliminate its effects. The robust SMC compensator acted as an auxiliary controller to guarantee the system’s stability and robustness under circumstances with model uncertainties, approximation errors, and friction forces. The stability of the closed-loop system and the asymptotic convergence of tracking errors were evaluated using Lyapunov theory. The simulation results showed that the dead zone’s non-linearity can be estimated correctly, and satisfactory trajectory tracking performance can be obtained in this way, since the influences of the actuator’s dead zone were eliminated. The convergence time of the system was reduced from 1.1 to 0.8 s, and the maximum steady-state error was reduced from 0.06 to 0.015 rad. In the grinding experiment, the joint steady-state error decreased by 21%, which proves the feasibility and effectiveness of the proposed control method.