Monitoring of Corroded and Loosened Bolts in Steel Structures via Deep Learning and Hough Transforms

Autonomous Image-Based Corrosion Detection in Steel Structures Using Deep Learning

Steel structures are susceptible to corrosion due to their exposure to the environment. Currently used non-destructive techniques require inspector involvement. Inaccessibility of the defective part may lead to unnoticed corrosion, allowing the corrosion to propagate and cause catastrophic structural failure over time. Autonomous corrosion detection is essential for mitigating these problems. This study investigated the effect of the type of encoder-decoder neural network and the training strategy that works the best to automate the segmentation of corroded pixels in visual images. Models using pre-trained DesnseNet121 and EfficientNetB7 backbones yielded 96.78% and 98.5% average pixel-level accuracy, respectively. Deeper EffiecientNetB7 performed the worst, with only 33% true-positive values, which was 58% less than ResNet34 and the original UNet. ResNet 34 successfully classified the corroded pixels, with 2.98% false positives, whereas the original UNet predicted 8.24% of the non-corroded pixels as corroded when tested on a specific set of images exclusive to the investigated training dataset. Deep networks were found to be better for transfer learning than full training, and a smaller dataset could be one of the reasons for performance degradation. Both fully trained conventional UNet and ResNet34 models were tested on some external images of different steel structures with different colors and types of corrosion, with the ResNet 34 backbone outperforming conventional UNet.

Image-Based Bolt-Loosening Detection Using a Checkerboard Perspective Correction Method

In this paper, a new image-correction method for flange joint bolts is proposed. A checkerboard is arranged on the side of a flange node bolt, and the homography matrix can be estimated using more than four feature points, which include the checkerboard corner points. Then, the perspective distortion of the captured image and the deviation of the camera position angle are corrected using the estimated homography matrix. Due to the use of more feature points, the stability of homography matrix identification is effectively improved. Simultaneously, the influence of the number of feature points, camera lens distance, and light intensities are analyzed. Finally, based on a bolt image taken using an iPhone 12, the prototype structure of the flange joint in the laboratory is verified. The results show that the proposed method can effectively correct image distortion and camera position angle deviation. The use of more than four correction points not only effectively improves the stability of bolt image correction but also improves the stability and accuracy of bolt-loosening detection. The analysis of influencing factors shows that the proposed method is still effective when the number of checkerboard correction points is reduced to nine, and the average error of the bolt-loosening detection result is less than 1.5 degrees. Moreover, the recommended camera shooting distance range is 20 cm to 60 cm, and the method exhibits low sensitivity to light intensity.

Nut Geometry Inspection Using Improved Hough Line and Circle Methods

Nuts are the cornerstone of human industrial construction, especially A-grade nuts that can only be used in power plants, precision instruments, aircraft, and rockets. However, the traditional nuts inspection method is to manually operate the measuring instrument for conducting an inspection, so the quality of the A-grade nut cannot be guaranteed. In this work, a machine vision-based inspection system was proposed, which performs a real-time geometric inspection of the nuts before and after tapping on the production line. In order to automatically screen out A-Grade nuts on the production line, there are 7 inspections within this proposed nut inspection system. The measurements of parallel, opposite side length, straightness, radius, roundness, concentricity, and eccentricity were proposed. To shorten the overall detection time regarding nut production, the program needed to be accurate and uncomplicated. By modifying the Hough line and Hough circle, the algorithm became faster and more suitable for nut detection. The optimized Hough line and Hough circle can be used for all measures in the testing process.

Corroded Bolt Identification Using Mask Region-Based Deep Learning Trained on Synthesized Data

The performance of a neural network depends on the availability of datasets, and most deep learning techniques lack accuracy and generalization when they are trained using limited datasets. Using synthesized training data is one of the effective ways to overcome the above limitation. Besides, the previous corroded bolt detection method has focused on classifying only two classes, clean and fully rusted bolts, and its performance for detecting partially rusted bolts is still questionable. This study presents a deep learning method to identify corroded bolts in steel structures using a mask region-based convolutional neural network (Mask-RCNN) trained on synthesized data. The Resnet50 integrated with a feature pyramid network is used as the backbone for feature extraction in the Mask-RCNN-based corroded bolt detector. A four-step data synthesis procedure is proposed to autonomously generate the training datasets of corroded bolts with different severities. Afterwards, the proposed detector is trained by the synthesized datasets, and its robustness is demonstrated by detecting corroded bolts in a lab-scale steel structure under varying capturing distances and perspectives. The results show that the proposed method has detected corroded bolts well and identified their corrosion levels with the most desired overall accuracy rate = 96.3% for a 1.0 m capturing distance and 97.5% for a 15° perspective angle.