Research Progress of Automated Visual Surface Defect Detection for Industrial Metal Planar Materials

Inclusion Detection in Injection-Molded Parts with the Use of Edge Masking

The algorithm and prototype presented in the article are part of a quality control system for plastic objects coming from injection-molding machines. Some objects contain a flaw called inclusion, which is usually observed as a local discoloration and disqualifies the object. The objects have complex, irregular geometry with many edges. This makes inclusion detection difficult, because local changes in the image at inclusions are much less significant than grayscale changes at the edges. In order to exclude edges from calculations, the presented method first classifies the object and then matches it with the corresponding mask of edges, which is prepared off-line and stored in the database. Inclusions are detected based on the analysis of local variations in the surface grayscale in the unmasked part of the image under inspection. Experiments were performed on real objects rejected from production by human quality controllers. The proposed approach allows tuning the algorithm to achieve very high sensitivity without false detections at edges. Based on input from the controllers, the algorithm was tuned to detect all the inclusions. At 100% recall, 87% precision was achieved, which is acceptable for industrial applications.

Defect-engineered chiral metal-organic frameworks

Chirality has an important impact on chemical and biological research, as most active substances are chiral. In recent decades, metal-organic frameworks (MOFs), which are assembled from metal ions or clusters and organic linkers via metal-ligand bonding, have attracted considerable scientific interest due to their high crystallinity, exceptional porosity and tunable pore sizes, high modularity, and diverse functionalities. Since the discovery of the first functional chiral metal-organic frameworks (CMOFs), CMOFs have been involved in a variety of disciplines such as chemistry, physics, optics, medicine, and pharmacology. The introduction of defect engineering theory into CMOFs allows the construction of a class of defective CMOFs with high hydrothermal stability and multi-stage pore structure. The introduction of defects not only increases the active sites but also enlarges the pore sizes of the materials, which improves chiral recognition, separation, and catalytic reactions, and has been widely investigated in various fields. This review describes the design and synthesis of various defective CMOFs, their characterization, and applications. Finally, the development of the materials is summarized, and an outlook is given. This review should provide researchers with an insight into the design and study of complex defective CMOFs.

Research on Surface Defect Detection of Strip Steel Based on Improved YOLOv7

Surface defect detection of strip steel is an important guarantee for improving the production quality of strip steel. However, due to the diverse types, scales, and texture structures of surface defects on strip steel, as well as the irregular distribution of defects, it is difficult to achieve rapid and accurate detection of strip steel surface defects with existing methods. This article proposes a real-time and high-precision surface defect detection algorithm for strip steel based on YOLOv7. Firstly, Partial Conv is used to replace the conventional convolution blocks of the backbone network to reduce the size of the network model and improve the speed of detection; Secondly, The CA attention mechanism module is added to the ELAN module to enhance the ability of the network to extract detect features and improve the effectiveness of detect detection in complex environments; Finally, The SPD convolution module is introduced at the output end to improve the detection performance of small targets with surface defects on steel. The experimental results on the NEU-DET dataset indicate that the mean average accuracy (mAP@IoU = 0.5) is 80.4%, which is 4.0% higher than the baseline network. The number of parameters is reduced by 8.9%, and the computational load is reduced by 21.9% (GFLOPs). The detection speed reaches 90.9 FPS, which can well meet the requirements of real-time detection.

Metal surface defect detection based on improved YOLOv5

During the production of metal material, various complex defects may come into being on the surface, together with large amount of background texture information, causing false or missing detection in the process of small defect detection. To resolve those problems, this paper introduces a new model which combines the advantages of CSPlayer module and Global Attention Enhancement Mechanism based on the YOLOv5s model. First of all, we replace C3 module with CSPlayer module to augment the neural network model, so as to improve its flexibility and adaptability. Then, we introduce the Global Attention Mechanism (GAM) and build the generalized additive model. In the meanwhile, the attention weights of all dimensions are weighted and averaged as output to promote the detection speed and accuracy. The results of the experiment in which the GC10-DET augmented dataset is involved, show that the improved algorithm model performs better than YOLOv5s in precision, mAP@0.5 and mAP@0.5: 0.95 by 5.3%, 1.4% and 1.7% respectively, and it also has a higher reasoning speed.

Phase Deflectometry for Defect Detection of High Reflection Objects

A method for detecting the surface defects of high reflection objects using phase deflection is proposed. The abrupt change in the surface gradient at the defect leads to the change in the fringe phase. Therefore, Gray code combined with a four-step phase-shift method was employed to obtain the surface gradients to characterize the defects. Then, through the double surface illumination model, the relationship between illumination intensity and phase was established. The causes of periodic error interference were analyzed, and the method of adjusting the fringe width to eliminate it was proposed. Finally, experimental results showed the effectiveness of the proposed method.

Video-Based Two-Stage Network for Optical Glass Sub-Millimeter Defect Detection

Since tiny optical glass is the key component in various optical instruments, more and more researchers have paid attention to automatic defect detection on tiny optical glass in recent years. It remains a challenging problem, as the defects are extremely small. In this paper, we propose a video-based two-stage defect detection network to improve detection accuracy for small defects. Specifically, the detection process is carried out in a coarse-to-fine manner to improve the detection precision. First, the optical glass area is located on the down-sampled version of the input image, and then defects are detected only within the optical glass area with a higher resolution version, which can significantly reduce the false alarming rate. Since the defects may exist on any place of the optical glass, we fuse the results of multiple video frames captured from various perspectives to promote recall rates of the defects. Additionally, we propose an image quality evaluation module based on a clustering algorithm to select video frames with high quality for improving both detection recall and precision. We contribute a new dataset called OGD-DET for tiny-scale optical glass surface defect detection experiments. The datasets consist of 3415 images from 40 videos, and the size of the defect area ranges from 0.1 mm to 0.53 mm, 2 to 7 pixels on images with a resolution of 1536 × 1024 pixels. Extensive experiments show that the proposed method outperforms the state-of-the-art methods in terms of both accuracy and computation cost.

A sparrow search algorithm with intelligent weight factors and mutation operators and applications to image classification

The surface inspection of strip steel defects plays a vital role in the industry, and it has attracted widespread attention in the industry. In this paper, an improved sparrow search algorithm (WMR-SSA) with intelligent weighting factors and mutation operators is proposed, WMR-SSA can balance the development capability of the algorithm based on the number of iterations. In addition, WMR-SSA enhances the local search capability of the algorithm through mutation operators. At the same time, the algorithm determines the initial position of the population by random walk to enhance the diversity of the population. The WMR-SSA algorithm is compared with GA, PSO, CS, GWO, BSA, and original SSA, and the experiment proves that the WMR-SSA algorithm is better than other algorithms. In this study, WMR-SSA is combined with BP neural network and implemented for the classification of defective strip images. The accuracy and stability of WMR-SSA-BP are effectively demonstrated experimentally by comparing it with classifiers optimized by other intelligent algorithms.

Photometric Stereo-Based Defect Detection System for Steel Components Manufacturing Using a Deep Segmentation Network

This paper presents an automatic system for the quality control of metallic components using a photometric stereo-based sensor and a customized semantic segmentation network. This system is designed based on interoperable modules, and allows capturing the knowledge of the operators to apply it later in automatic defect detection. A salient contribution is the compact representation of the surface information achieved by combining photometric stereo images into a RGB image that is fed to a convolutional segmentation network trained for surface defect detection. We demonstrate the advantage of this compact surface imaging representation over the use of each photometric imaging source of information in isolation. An empirical analysis of the performance of the segmentation network on imaging samples of materials with diverse surface reflectance properties is carried out, achieving Dice performance index values above 0.83 in all cases. The results support the potential of photometric stereo in conjunction with our semantic segmentation network.

AI Landing for Sheet Metal-Based Drawer Box Defect Detection Using Deep Learning (ALDB-DL)

Sheet metal-based products serve as a major portion of the furniture market and maintain higher quality standards by being competitive. During industrial processes, while converting a sheet metal to an end product, new defects are observed and thus need to be identified carefully. Recent studies have shown scratches, bumps, and pollution/dust are identified, but orange peel defects present overall a new challenge. So our model identifies scratches, bumps, and dust by using computer vision algorithms, whereas orange peel defect detection with deep learning have a better performance. The goal of this paper was to resolve artificial intelligence (AI) as an AI landing challenge faced in identifying various kinds of sheet metal-based product defects by ALDB-DL process automation. Therefore, our system model consists of multiple cameras from two different angles to capture the defects of the sheet metal-based drawer box. The aim of this paper was to solve multiple defects detection as design and implementation of Industrial process integration with AI by Automated Optical Inspection (AOI) for sheet metal-based drawer box defect detection, stated as AI Landing for sheet metal-based Drawer Box defect detection using Deep Learning (ALDB-DL). Therefore, the scope was given as achieving higher accuracy using multi-camera-based image feature extraction using computer vision and deep learning algorithm for defect classification in AOI. We used SHapley Additive exPlanations (SHAP) values for pre-processing, LeNet with a (1 × 1) convolution filter, and a Global Average Pooling (GAP) Convolutional Neural Network (CNN) algorithm to achieve the best results. It has applications for sheet metal-based product industries with improvised quality control for edge and surface detection. The results were competitive as the precision, recall, and area under the curve were 1.00, 0.99, and 0.98, respectively. Successively, the discussion section presents a detailed insight view about the industrial functioning with ALDB-DL experience sharing.

An Improved VGG19 Transfer Learning Strip Steel Surface Defect Recognition Deep Neural Network Based on Few Samples and Imbalanced Datasets

The surface defects’ region of strip steel is small, and has various defect types and, complex gray structures. There tend to be a large number of false defects and edge light interference, which lead traditional machine vision algorithms to be unable to detect defects for various types of strip steel. Image detection techniques based on deep learning require a large number of images to train a network. However, for a dataset with few samples with category imbalanced defects, common deep learning neural network training tasks cannot be carried out. Based on rapid image preprocessing algorithms (improved gray projection algorithm, ROI image augmentation algorithm) and transfer learning theory, this paper proposes a set of processes for complete strip steel defect detection. These methods achieved surface rapid screening, defect feature extraction, sample dataset’s category balance, data augmentation, defect detection, and classification. Through verification of the mixed dataset, composed of the NEU surface dataset and dataset in this paper, the recognition accuracy of the improved VGG19 network in this paper reached 97.8%. The improved VGG19 network performs slightly better than the baseline VGG19 in six types of defects, but the improved VGG19 performs significantly better in the surface seams defects. The convergence speed and accuracy of the improved VGG19 network were taken into account, and the detection rate was greatly improved with few samples and imbalanced datasets. This paper also has practical value in terms of extending its method of strip steel defect detection to other products.