Using Deep Learning to Detect Defects in Manufacturing: A Comprehensive Survey and Current Challenges

Machine Learning-Driven Prediction of Composite Materials Properties Based on Experimental Testing Data

The growing demand for high-performance and cost-effective composite materials necessitates advanced computational approaches for optimizing their composition and properties. This study aimed at the application of machine learning for the prediction and optimization of the functional properties of composites based on a thermoplastic matrix with various fillers (two types of fibrous, four types of dispersed, and two types of nano-dispersed fillers). The experimental methods involved material production through powder metallurgy, further microstructural analysis, and mechanical and tribological testing. The microstructural analysis revealed distinct structural modifications and interfacial interactions influencing their functional properties. The key findings indicate that optimal filler selection can significantly enhance wear resistance while maintaining adequate mechanical strength. Carbon fibers at 20 wt. % significantly improved wear resistance (by 17-25 times) while reducing tensile strength and elongation. Basalt fibers at 10 wt. % provided an effective balance between reinforcement and wear resistance (by 11-16 times). Kaolin at 2 wt. % greatly enhanced wear resistance (by 45-57 times) with moderate strength reduction. Coke at 20 wt. % maximized wear resistance (by 9-15 times) while maintaining acceptable mechanical properties. Graphite at 10 wt. % ensured a balance between strength and wear, as higher concentrations drastically decreased mechanical properties. Sodium chloride at 5 wt. % offered moderate wear resistance improvement (by 3-4 times) with minimal impact on strength. Titanium dioxide at 3 wt. % enhanced wear resistance (by 11-12.5 times) while slightly reducing tensile strength. Ultra-dispersed PTFE at 1 wt. % optimized both strength and wear properties. The work analyzed in detail the effect of PTFE content and filler content on composite properties based on machine learning-driven prediction. Regression models demonstrated high R-squared values (0.74 for density, 0.67 for tensile strength, 0.80 for relative elongation, and 0.79 for wear intensity), explaining up to 80% of the variability in composite properties. Despite its efficiency, the limitations include potential multicollinearity, a lack of consideration of external factors, and the need for further validation under real-world conditions. Thus, the machine learning approach reduces the need for extensive experimental testing, minimizing material waste and production costs, contributing to SDG 9. This study highlights the potential use of machine learning in polymer composite design, offering a data-driven framework for the rational choice of fillers, thereby contributing to sustainable industrial practices.

A deep learning approach to perform defect classification of freeze-dried product

Cosmetic inspection of freeze-dried products is an important part of the post-manufacturing quality control process. Traditionally done by human visual inspection, this method poses typical challenges and shortcomings that can be addressed with innovative techniques. While many cosmetic defects can occur, some are considered more critical than others as they can be harmful to the patient or affect the drug's efficacy. With the rise of artificial intelligence and computer vision technology, faster and more reproducible quality control is possible, allowing real-time monitoring on a continuous manufacturing line. In this study, several continuously freeze-dried samples were prepared using formulations and process settings that lead deliberately to specific defects faced in freeze-drying as well as defect-free samples. Two approaches (i.e. patch-based approach and multi-label classification) capable of handling high-resolution images based on Convolutional Neural Networks were developed and compared to select the optimal one. Additional visualization techniques were used to enhance model understanding further. The best approach achieved perfect precision and recall on critical defects, with a prediction time of less than 50 ms to make a decision on the acceptance or rejection of vials generated.

Deep learning based gasket fault detection: a CNN approach

Gasket inspection is a critical step in the quality control of a product. The proposed method automates the detection of misaligned or incorrectly fitting gaskets, ensuring timely repair action. The suggested method uses deep learning approaches to recognize and evaluate radiator images, with a focus on identifying misaligned or incorrectly installed gaskets. Deep learning algorithms are specific for feature extraction and classification together with a convolutional neural network (CNN) module that allows for seamless connection. A gasket inspection system based on a CNN architecture is developed in this work. The system consists of two sets of convolution layers, followed by two sets of batch normalization layer, two sets of RELU layer, max pooling layer and finally fully connected layer for classification of gasket images. The obtained results indicate that our system has great potential for practical applications in the manufacturing industry. Moreover, our system provides a reliable and efficient mechanism for quality control, which can help reduce the risk of defects and ensure product reliability.

A study on the detection of conductor quantity in cable cores based on YOLO-cable

The quantity of cable conductors is a crucial parameter in cable manufacturing, and accurately detecting the number of conductors can effectively promote the digital transformation of the cable manufacturing industry. Challenges such as high density, adhesion, and knife mark interference in cable conductor images make intelligent detection of conductor quantity particularly difficult. To address these challenges, this study proposes the YOLO-cable model, which is an improvement made upon the YOLOv10 model. Specifically, the Focal loss function is introduced, the C2F structure in the backbone is optimized, the Focal NeXt module is added, and a multi-scale feature (MSF) module is incorporated in the Neck section. Comparative experiments with various YOLO series models demonstrate that the YOLO-cable model significantly outperformed the baseline YOLOv10s model as it achieves recall, mAP0.5, and mAP scores of 0.982, 0.994, and 0.952, respectively. Further visualization analysis shows that the overlap of YOLO-cable detection boxes with manually labeled samples reaches 90.9% in length and 95.7% in height, indicating high data consistency. The IOU threshold adopted by the model enables it to effectively filter out false detection, thus ensuring detection accuracy. In short, the proposed model excels in detecting the number of cable conductors, enhancing quality control in cable production. This study provides new insights and technical support for the application of deep learning in industrial inspections.

Deep learning-based anomaly detection using one-dimensional convolutional neural networks (1D CNN) in machine centers (MCT) and computer numerical control (CNC) machines

Computer numerical control (CNC) and machine center (MCT) machines are mechanical devices that manipulate different tools using computer programming as inputs. Predicting failures in CNC and MCT machines before their actual failure time is crucial to reduce maintenance costs and increase productivity. This study is centered around a novel deep learning-based model using a 1D convolutional neural network (CNN) for early fault detection in MCT machines. We collected sensor-based data from CNC/MCT machines and applied various preprocessing techniques to prepare the dataset. Our experimental results demonstrate that the 1D-CNN model achieves a higher accuracy of 91.57% compared to traditional machine learning classifiers and other deep learning models, including Random Forest (RF) at 89.71%, multi-layer perceptron (MLP) at 87.45%, XGBoost at 89.67%, logistic regression (LR) at 75.93%, support vector machine (SVM) at 75.96%, K-nearest neighbors (KNN) at 82.93%, decision tree at 88.36%, naïve Bayes at 68.31%, long short-term memory (LSTM) at 90.80%, and a hybrid 1D CNN + LSTM model at 88.51%. Moreover, our proposed 1D CNN model outperformed all other mentioned models in precision, recall, and F-1 scores, with 91.87%, 91.57%, and 91.63%, respectively. These findings highlight the efficacy of the 1D CNN model in providing optimal performance with an MCT machine's dataset, making it particularly suitable for small manufacturing companies seeking to automate early fault detection and classification in CNC and MCT machines. This approach enhances productivity and aids in proactive maintenance and safety measures, demonstrating its potential to revolutionize the manufacturing industry.

A Novel YOLOv5_ES based on lightweight small object detection head for PCB surface defect detection

In the manufacturing process of printed circuit boards (PCBs), surface defects have a significant negative impact on product quality. Considering that traditional object detection algorithms have low accuracy in handling PCB images with complex backgrounds, various types, and small-sized defects, this paper proposes a PCB defect detection algorithm based on a novel YOLOv5 multi-scale attention mechanism(EMA) spatial pyramid dilated Convolution (SPD-Conv) (YOLOv5_ES) network improved YOLOv5s framework. Firstly, the detection head is optimized by removing medium and large detection layers, fully leveraging the capability of the small detection head to identify minor target defects. This approach not only improves model accuracy but also achieves lightweighting. Secondly, in order to further reduce the number of parameters and computational costs, the SPD-Conv is introduced to improve the feature extraction capability by reducing information loss. Thirdly, a EMA module is introduced to fuse context information of different scales, enhancing the model's generalization ability. Compared to the YOLOv5s model, there is a 3.1% improvement in mean average precision (mAP0.5), a 55.8% reduction in model parameters, and a 4.8% reduction in giga floating-point operations per second (GFLOPs). These results demonstrate a significant improvement in both accuracy and model parameter efficiency.

Automatic deep learning detection of overhanging restorations in bitewing radiographs

OBJECTIVES

This study aimed to assess the effectiveness of deep convolutional neural network (CNN) algorithms for the detecting and segmentation of overhanging dental restorations in bitewing radiographs.

METHODS

A total of 1160 anonymized bitewing radiographs were used to progress the artificial intelligence (AI) system for the detection and segmentation of overhanging restorations. The data were then divided into three groups: 80% for training (930 images, 2399 labels), 10% for validation (115 images, 273 labels), and 10% for testing (115 images, 306 labels). A CNN model known as You Only Look Once (YOLOv5) was trained to detect overhanging restorations in bitewing radiographs. After utilizing the remaining 115 radiographs to evaluate the efficacy of the proposed CNN model, the accuracy, sensitivity, precision, F1 score, and area under the receiver operating characteristic curve (AUC) were computed.

RESULTS

The model demonstrated a precision of 90.9%, a sensitivity of 85.3%, and an F1 score of 88.0%. Furthermore, the model achieved an AUC of 0.859 on the receiver operating characteristic (ROC) curve. The mean average precision (mAP) at an intersection over a union (IoU) threshold of 0.5 was notably high at 0.87.

CONCLUSIONS

The findings suggest that deep CNN algorithms are highly effective in the detection and diagnosis of overhanging dental restorations in bitewing radiographs. The high levels of precision, sensitivity, and F1 score, along with the significant AUC and mAP values, underscore the potential of these advanced deep learning techniques in revolutionizing dental diagnostic procedures.

Application of Event Cameras and Neuromorphic Computing to VSLAM: A Survey

Simultaneous Localization and Mapping (SLAM) is a crucial function for most autonomous systems, allowing them to both navigate through and create maps of unfamiliar surroundings. Traditional Visual SLAM, also commonly known as VSLAM, relies on frame-based cameras and structured processing pipelines, which face challenges in dynamic or low-light environments. However, recent advancements in event camera technology and neuromorphic processing offer promising opportunities to overcome these limitations. Event cameras inspired by biological vision systems capture the scenes asynchronously, consuming minimal power but with higher temporal resolution. Neuromorphic processors, which are designed to mimic the parallel processing capabilities of the human brain, offer efficient computation for real-time data processing of event-based data streams. This paper provides a comprehensive overview of recent research efforts in integrating event cameras and neuromorphic processors into VSLAM systems. It discusses the principles behind event cameras and neuromorphic processors, highlighting their advantages over traditional sensing and processing methods. Furthermore, an in-depth survey was conducted on state-of-the-art approaches in event-based SLAM, including feature extraction, motion estimation, and map reconstruction techniques. Additionally, the integration of event cameras with neuromorphic processors, focusing on their synergistic benefits in terms of energy efficiency, robustness, and real-time performance, was explored. The paper also discusses the challenges and open research questions in this emerging field, such as sensor calibration, data fusion, and algorithmic development. Finally, the potential applications and future directions for event-based SLAM systems are outlined, ranging from robotics and autonomous vehicles to augmented reality.

Stable 3D Deep Convolutional Autoencoder Method for Ultrasonic Testing of Defects in Polymer Composites

Ultrasonic testing is widely used for defect detection in polymer composites owing to advantages such as fast processing speed, simple operation, high reliability, and real-time monitoring. However, defect information in ultrasound images is not easily detectable because of the influence of ultrasound echoes and noise. In this study, a stable three-dimensional deep convolutional autoencoder (3D-DCA) was developed to identify defects in polymer composites. Through 3D convolutional operations, it can synchronously learn the spatiotemporal properties of the data volume. Subsequently, the depth receptive field (RF) of the hidden layer in the autoencoder maps the defect information to the original depth location, thereby mitigating the effects of the defect surface and bottom echoes. In addition, a dual-layer encoder was designed to improve the hidden layer visualization results. Consequently, the size, shape, and depth of the defects can be accurately determined. The feasibility of the method was demonstrated through its application to defect detection in carbon-fiber-reinforced polymers.

Quantifying the effect of X-ray scattering for data generation in real-time defect detection

BACKGROUND

X-ray imaging is widely used for the non-destructive detection of defects in industrial products on a conveyor belt. In-line detection requires highly accurate, robust, and fast algorithms. Deep Convolutional Neural Networks (DCNNs) satisfy these requirements when a large amount of labeled data is available. To overcome the challenge of collecting these data, different methods of X-ray image generation are considered.

OBJECTIVE

Depending on the desired degree of similarity to real data, different physical effects should either be simulated or can be ignored. X-ray scattering is known to be computationally expensive to simulate, and this effect can greatly affect the accuracy of a generated X-ray image. We aim to quantitatively evaluate the effect of scattering on defect detection.

METHODS

Monte-Carlo simulation is used to generate X-ray scattering distribution. DCNNs are trained on the data with and without scattering and applied to the same test datasets. Probability of Detection (POD) curves are computed to compare their performance, characterized by the size of the smallest detectable defect.

RESULTS

We apply the methodology to a model problem of defect detection in cylinders. When trained on data without scattering, DCNNs reliably detect defects larger than 1.3 mm, and using data with scattering improves performance by less than 5%. If the analysis is performed on the cases with large scattering-to-primary ratio (1 < SPR < 5), the difference in performance could reach 15% (approx. 0.4 mm).

CONCLUSION

Excluding the scattering signal from the training data has the largest effect on the smallest detectable defects, and the difference decreases for larger defects. The scattering-to-primary ratio has a significant effect on detection performance and the required accuracy of data generation.

Leveraging Deep Neural Networks for Estimating Vickers Hardness from Nanoindentation Hardness

This research presents a comprehensive analysis of deep neural network models (DNNs) for the precise prediction of Vickers hardness (HV) in nitrided and carburized M50NiL steel samples, with hardness values spanning from 400 to 1000 HV. By conducting rigorous experimentation and obtaining corresponding nanoindentation data, we evaluated the performance of four distinct neural network architectures: Multilayer Perceptron (MLP), Convolutional Neural Network (CNN), Long Short-Term Memory network (LSTM), and Transformer. Our findings reveal that MLP and LSTM models excel in predictive accuracy and efficiency, with MLP showing exceptional iteration efficiency and predictive precision. The study validates models for broad application in various steel types and confirms nanoindentation as an effective direct measure for HV hardness in thin films and gradient-variable regions. This work contributes a validated and versatile approach to the hardness assessment of thin-film materials and those with intricate microstructures, enhancing material characterization and potential application in advanced material engineering.

Development of microcatheter tube extrusion angle estimation system using convolutional neural network segmentation

This study presents a deep learning-based monitoring system for estimating extrusion angles in the manufacturing process of microcatheter tubes. Given the critical nature of these tubes, which are directly inserted into the human body, strict quality control is imperative. To mitigate potential quality variations stemming from operator actions, a system utilizing a convolutional neural network to precisely measure the extrusion angle-a parameter with profound implications for tube quality-is developed. Until now, there has been no method to estimate the extrusion angle of resin being extruded in real-time. In this study, for the first time, a method using deep learning to estimate the angle was proposed. This innovative system comprises two RGB cameras capturing both front and side perspectives. The acquired images undergo segmentation via a meticulously trained convolutional neural network. Subsequently, the extrusion angle is accurately estimated through the application of principal component analysis on the segmented image. The usefulness of the proposed system was rigorously confirmed through comprehensive validation measures, including mean intersection over union (mIoU), mean absolute angle error (MAE), and inference time, using a real-world dataset. The attained metrics, with an mIoU of 0.8848, MAE of 0.5968, and an inference time of 0.0546, unequivocally affirm the system's suitability for enhancing the catheter tube extrusion process.

Quality Control of Carbon Look Components via Surface Defect Classification with Deep Neural Networks

Many "Industry 4.0" applications rely on data-driven methodologies such as Machine Learning and Deep Learning to enable automatic tasks and implement smart factories. Among these applications, the automatic quality control of manufacturing materials is of utmost importance to achieve precision and standardization in production. In this regard, most of the related literature focused on combining Deep Learning with Nondestructive Testing techniques, such as Infrared Thermography, requiring dedicated settings to detect and classify defects in composite materials. Instead, the research described in this paper aims at understanding whether deep neural networks and transfer learning can be applied to plain images to classify surface defects in carbon look components made with Carbon Fiber Reinforced Polymers used in the automotive sector. To this end, we collected a database of images from a real case study, with 400 images to test binary classification (defect vs. no defect) and 1500 for the multiclass classification (components with no defect vs. recoverable vs. non-recoverable). We developed and tested ten deep neural networks as classifiers, comparing ten different pre-trained CNNs as feature extractors. Specifically, we evaluated VGG16, VGG19, ResNet50 version 2, ResNet101 version 2, ResNet152 version 2, Inception version 3, MobileNet version 2, NASNetMobile, DenseNet121, and Xception, all pre-trainined with ImageNet, combined with fully connected layers to act as classifiers. The best classifier, i.e., the network based on DenseNet121, achieved a 97% accuracy in classifying components with no defects, recoverable components, and non-recoverable components, demonstrating the viability of the proposed methodology to classify surface defects from images taken with a smartphone in varying conditions, without the need for dedicated settings. The collected images and the source code of the experiments are available in two public, open-access repositories, making the presented research fully reproducible.

Research on Internal Shape Anomaly Inspection Technology for Pipeline Girth Welds Based on Alternating Excitation Detection

Abnormal formation of girth weld is a major threat to the safe operation of pipelines, which may lead to serious accidents. Therefore, regular inspection and maintenance of girth weld are essential for accident prevention and energy security. This paper presents a novel method for inspecting abnormal girth weld formation in oil and gas pipelines using alternating excitation detection technology. The method is based on the analysis of the microscopic magnetic variations in the welded area under alternating magnetic fields. An internal inspection probe and electronic system for detecting abnormal girth weld formation were designed and developed. The system's capability to identify misalignment, undercutting, root concavity, and abnormal formation height of girth weld was tested by numerical simulation and experimental study. The results show that the detection system can effectively identify a minimum misalignment of 0.5 mm at a lift-off height of 15 mm. The proposed method offers several advantages, such as rapid response, low cost, non-contact operation, and high sensitivity to surface flaws in ferromagnetic pipelines.

Implementation of a Sequence-to-Sequence Stacked Sparse Long Short-Term Memory Autoencoder for Anomaly Detection on Multivariate Timeseries Data of Industrial Blower Ball Bearing Units

The advent of Industry 4.0 introduced new ways for businesses to evolve by implementing maintenance policies leading to advancements in terms of productivity, efficiency, and financial performance. In line with the growing emphasis on sustainability, industries implement predictive techniques based on Artificial Intelligence for the purpose of mitigating machine and equipment failures by predicting anomalies during their production process. In this work, a new dataset that was made publicly available, collected from an industrial blower, is presented, analyzed and modeled using a Sequence-to-Sequence Stacked Sparse Long Short-Term Memory Autoencoder. Specifically the right and left mounted ball bearing units were measured during several months of normal operational condition as well as during an encumbered operational state. An anomaly detection model was developed for the purpose of analyzing the operational behavior of the two bearing units. A stacked sparse Long Short-Term Memory Autoencoder was successfully trained on the data obtained from the left unit under normal operating conditions, learning the underlying patterns and statistical connections of the data. The model was evaluated by means of the Mean Squared Error using data from the unit's encumbered state, as well as using data collected from the right unit. The model performed satisfactorily throughout its evaluation on all collected datasets. Also, the model proved its capability for generalization along with adaptability on assessing the behavior of equipment similar to the one it was trained on.

Defect detection of gear parts in virtual manufacturing

Gears play an important role in virtual manufacturing systems for digital twins; however, the image of gear tooth defects is difficult to acquire owing to its non-convex shape. In this study, a deep learning network is proposed to detect gear defects based on their point cloud representation. This approach mainly consists of three steps: (1) Various types of gear defects are classified into four cases (fracture, pitting, glue, and wear); A 3D gear dataset was constructed with 10000 instances following the aforementioned classification. (2) Gear-PCNet+ + introduces a novel Combinational Convolution Block, proposed based on the gear dataset for gear defect detection to effectively extract the local gear information and identify its complex topology; (3) Compared with other methods, experiments show that this method can achieve better recognition results for gear defects with higher efficiency and practicability.

Artificial Intelligence-Based Smart Quality Inspection for Manufacturing

In today's era, monitoring the health of the manufacturing environment has become essential in order to prevent unforeseen repairs, shutdowns, and to be able to detect defective products that could incur big losses. Data-driven techniques and advancements in sensor technology with Internet of the Things (IoT) have made real-time tracking of systems a reality. The health of a product can also be continuously assessed throughout the manufacturing lifecycle by using Quality Control (QC) measures. Quality inspection is one of the critical processes in which the product is evaluated and deemed acceptable or rejected. The visual inspection or final inspection process involves a human operator sensorily examining the product to ascertain its status. However, there are several factors that impact the visual inspection process resulting in an overall inspection accuracy of around 80% in the industry. With the goal of 100% inspection in advanced manufacturing systems, manual visual inspection is both time-consuming and costly. Computer Vision (CV) based algorithms have helped in automating parts of the visual inspection process, but there are still unaddressed challenges. This paper presents an Artificial Intelligence (AI) based approach to the visual inspection process by using Deep Learning (DL). The approach includes a custom Convolutional Neural Network (CNN) for inspection and a computer application that can be deployed on the shop floor to make the inspection process user-friendly. The inspection accuracy for the proposed model is 99.86% on image data of casting products.

Impact and Assessment of Suspension Stiffness on Vibration Propagation into Vehicle

The impact of transport-induced vibrations on people is a particularly important problem. Sudden or intensifying vibration phenomena of a local nature may compromise safety, especially in transport. The paper addresses the results of research on the impact of spring stiffness parameters on the propagation of vibrations in the vehicle structure using simple amplitude and frequency measures. The use of the developed method of selective multi-criteria analysis of frequency bands made it possible to compare the vibrations recorded in the vehicle with a new or worn coil spring. The results of the present study allow the development of a large data base in which all signals are classified by the exploitation parameters and location of the propagation of vibration in the vehicle. The most important findings and achievements of the presented study are the testing of actual suspension components with real damage under controlled conditions, the identification of the vibration propagation path from the wheel to the driver and passenger feet, the quantitative comparison of vibrations affecting humans in the vehicle (through the feet), and the frequency decomposition of vibration for selected bands. These findings improve the proper interpretation of the developed measures and, as a result, the difficulties in using this knowledge at the engineering level, for example, in the design and construction improvement stage. Therefore, innovation points and engineering significances are a method of selective multi-criteria analysis of frequency bands and have potential applications in diagnostics and the design of suspension systems and in terms of passengers' comfort.

Using the AraBERT Model for Customer Satisfaction Classification of Telecom Sectors in Saudi Arabia

Customer satisfaction and loyalty are essential for every business. Feedback prediction and social media classification are crucial and play a key role in accurately identifying customer satisfaction. This paper presents sentiment analysis-based customer feedback prediction based on Twitter Arabic datasets of telecommunications companies in Saudi Arabia. The human brain, which contains billions of neurons, provides feedback based on the current and past experience provided by the services and other related stakeholders. Artificial Intelligent (AI) based methods, parallel to human brain processing methods such as Deep Learning (DL) algorithms, are famous for classifying and analyzing such datasets. Comparing the Arabic Dataset to English, it is pretty challenging for typical methods to outperform in the classification or prediction tasks. Therefore, the Arabic Bidirectional Encoder Representations from Transformers (AraBERT) model was used and analyzed with various parameters such as activation functions and topologies and simulated customer satisfaction prediction takes using Arabic Twitter datasets. The prediction results were compared with two famous DL algorithms: Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN). Results show that these methods have been successfully applied and obtained highly accurate classification results. AraBERT achieved the best prediction accuracy among the three ML methods, especially with Mobily and STC datasets.

Evolving deep convolutional neutral network by hybrid sine-cosine and extreme learning machine for real-time COVID19 diagnosis from X-ray images

The COVID19 pandemic globally and significantly has affected the life and health of many communities. The early detection of infected patients is effective in fighting COVID19. Using radiology (X-Ray) images is, perhaps, the fastest way to diagnose the patients. Thereby, deep Convolutional Neural Networks (CNNs) can be considered as applicable tools to diagnose COVID19 positive cases. Due to the complicated architecture of a deep CNN, its real-time training and testing become a challenging problem. This paper proposes using the Extreme Learning Machine (ELM) instead of the last fully connected layer to address this deficiency. However, the parameters' stochastic tuning of ELM's supervised section causes the final model unreliability. Therefore, to cope with this problem and maintain network reliability, the sine-cosine algorithm was utilized to tune the ELM's parameters. The designed network is then benchmarked on the COVID-Xray-5k dataset, and the results are verified by a comparative study with canonical deep CNN, ELM optimized by cuckoo search, ELM optimized by genetic algorithm, and ELM optimized by whale optimization algorithm. The proposed approach outperforms comparative benchmarks with a final accuracy of 98.83% on the COVID-Xray-5k dataset, leading to a relative error reduction of 2.33% compared to a canonical deep CNN. Even more critical, the designed network's training time is only 0.9421 ms and the overall detection test time for 3100 images is 2.721 s.

Gigapixel imaging with a novel multi-camera array microscope

The dynamics of living organisms are organized across many spatial scales. However, current cost-effective imaging systems can measure only a subset of these scales at once. We have created a scalable multi-camera array microscope (MCAM) that enables comprehensive high-resolution recording from multiple spatial scales simultaneously, ranging from structures that approach the cellular scale to large-group behavioral dynamics. By collecting data from up to 96 cameras, we computationally generate gigapixel-scale images and movies with a field of view over hundreds of square centimeters at an optical resolution of 18 µm. This allows us to observe the behavior and fine anatomical features of numerous freely moving model organisms on multiple spatial scales, including larval zebrafish, fruit flies, nematodes, carpenter ants, and slime mold. Further, the MCAM architecture allows stereoscopic tracking of the z-position of organisms using the overlapping field of view from adjacent cameras. Overall, by removing the bottlenecks imposed by single-camera image acquisition systems, the MCAM provides a powerful platform for investigating detailed biological features and behavioral processes of small model organisms across a wide range of spatial scales.

Real-time batch inspection system for surface defects on circular optical filters

Optical filters, one of the essential parts of many optical instruments, are used to select a specific radiation band of optical devices. There are specifications for the surface quality of the optical filter in order to ensure the instrument's regular operation. The traditional machine learning techniques for examining the optical filter surface quality mentioned in the current studies primarily rely on the manual extraction of feature data, which restricts their ability to detect optical filter surfaces with multiple defects. In order to solve the problems of low detection efficiency and poor detection accuracy caused by defects too minor and too numerous types of defects, this paper proposes a real-time batch optical filter surface quality inspection method based on deep learning and image processing techniques. The first part proposes an optical filter surface defect detection and identification method for seven typical defects. A deep learning model is trained for defect detection and recognition by constructing a dataset. The second part uses image processing techniques to locate the accurate position of the defect, determine whether the defect is located within the effective aperture, and analyze the critical eigenvalue data of the defect. The experimental results show that the method improves productivity and product quality and reduces the manual workload by 90%. The proposed model and method also compare the results of surface defect detection with the actual measurement data in the field, verifying that the method has good recognition accuracy while improving efficiency.

Multilayer Feature Extraction of AGCN on Surface Defect Detection of Steel Plates

The development of industry is inseparable from the support of steel materials, and the modern industry has increasingly high requirements for the quality of steel plates. But the process of steel plate production produces many types of defects, such as roll marks, scratches, and scars. These defects will directly affect the quality and performance of the steel plate, so it is necessary to effectively detect them. Steel plate surface defects are characterized by their types, shape, and size: the same defect can have different morphologies, and similarities can exist between different defects. In this paper, industrial steel plate surface defect samples are analyzed, and a sample set is established by screening the collected defect images. Then, annotation and classification are performed. A multilayer feature extraction framework is developed in experiments to train a neural network on the sample set of defects. To address the problems of low automation, slow detection speed, and low accuracy of the traditional defect detection methods, the attention graph convolution network (AGCN) is investigated in this paper. Firstly, faster R-CNN is used as the basic network model for defect detection, and the visual features are jointly refined by combining attention mechanism and graph convolution neural network. The latter network enriches the contextual information in the visual features of steel plates and explores the semantic association between vision and defect types for different kinds of defects using the attention mechanism to achieve intelligent detection of defects, thus enabling our method to meet the practical needs of steel plate production.

A Sensitive Frequency Range Method Based on Laser Ultrasounds for Micro-Crack Depth Determination

The laser ultrasonic method using the characteristics of transmitted Rayleigh waves in the frequency domain to determine micro-crack depth is proposed. A low-pass filter model based on the interaction between Rayleigh waves and surface cracks is built and shows that the stop band, called the sensitive frequency range, is sensitive to the depth of surface cracks. The sum of transmission coefficients in the sensitive frequency range is defined as an evaluated parameter to determine crack depth. Moreover, the effects of the sensitive frequency range and measured distance on the evaluated results are analyzed by the finite-element method to validate the robustness of this depth-evaluating method. The estimated results of surface cracks with depths ranging from 0.08 mm to ~0.5 mm on the FEM models and aluminum-alloy samples demonstrate that the laser ultrasounds using the characteristics of Rayleigh waves in the frequency domain do work for quantitative crack depth.

Design of the Automated Calibration Process for an Experimental Laser Inspection Stand

This paper deals with the concept of the automated calibration design for inspection systems using laser sensors. The conceptual solution is based on using a laser sensor and its ability to scan 3D surfaces of inspected objects in order to create a representative point cloud. Problems of scanning are briefly discussed. The automated calibration procedure for solving problems of errors due to non-precise adjustment of the mechanical arrangement, possible tolerances in assembly, and their following elimination is proposed. The main goal is to develop a system able to measure and quantify the quality of produced objects in the environment of Industry 4.0. Laboratory measurements on the experimental stand, including the principal software solution for automated calibration of laser sensors suitable for gear wheel inspection systems are presented. There is described design of compensation eccentricity by Fourier transform and sinusoidal fitting to identify and suppress the first harmonic component in the data with high precision measuring.

A System for a Real-Time Electronic Component Detection and Classification on a Conveyor Belt

The presented research addresses the real-time object detection problem with small and moving objects, specifically the surface-mount component on a conveyor. Detecting and counting small moving objects on the assembly line is a challenge. In order to meet the requirements of real-time applications, state-of-the-art electronic component detection and classification algorithms are implemented into powerful hardware systems. This work proposes a low-cost system with an embedded microcomputer to detect surface-mount components on a conveyor belt in real time. The system detects moving, packed, and unpacked surface-mount components. The system’s performance was experimentally investigated by implementing several object-detection algorithms. The system’s performance with different algorithm implementations was compared using mean average precision and inference time. The results of four different surface-mount components showed average precision scores of 97.3% and 97.7% for capacitor and resistor detection. The findings suggest that the system with the implemented YOLOv4-tiny algorithm on the Jetson Nano 4 GB microcomputer achieves a mean average precision score of 88.03% with an inference time of 56.4 ms and 87.98% mean average precision with 11.2 ms inference time on the Tesla P100 16 GB platform.

Defect Detection for Metal Base of TO-Can Packaged Laser Diode Based on Improved YOLO Algorithm

Defect detection is an important part of the manufacturing process of mechanical products. In order to detect the appearance defects quickly and accurately, a method of defect detection for the metal base of TO-can packaged laser diode (metal TO-base) based on the improved You Only Look Once (YOLO) algorithm named YOLO-SO is proposed in this study. Firstly, convolutional block attention mechanism (CBAM) module was added to the convolutional layer of the backbone network. Then, a random-paste-mosaic (RPM) small object data augmentation module was proposed on the basis of Mosaic algorithm in YOLO-V5. Finally, the K-means++ clustering algorithm was applied to reduce the sensitivity to the initial clustering center, making the positioning more accurate and reducing the network loss. The proposed YOLO-SO model was compared with other object detection algorithms such as YOLO-V3, YOLO-V4, and Faster R-CNN. Experimental results demonstrated that the YOLO-SO model reaches 84.0% mAP, 5.5% higher than the original YOLO-V5 algorithm. Moreover, the YOLO-SO model had clear advantages in terms of the smallest weight size and detection speed of 25 FPS. These advantages make the YOLO-SO model more suitable for the real-time detection of metal TO-base appearance defects.

An Intelligent Solution for Automatic Garment Measurement Using Image Recognition Technologies

Global digitization trends and the application of high technology in the garment market are still too slow to integrate, despite the increasing demand for automated solutions. The main challenge is related to the extraction of garment information-general clothing descriptions and automatic dimensional extraction. In this paper, we propose the garment measurement solution based on image processing technologies, which is divided into two phases, garment segmentation and key points extraction. UNet as a backbone network has been used for mask retrieval. Separate algorithms have been developed to identify both general and specific garment key points from which the dimensions of the garment can be calculated by determining the distances between them. Using this approach, we have resulted in an average 1.27 cm measurement error for the prediction of the basic measurements of blazers, 0.747 cm for dresses and 1.012 cm for skirts.

Industry 4.0 and Digitalisation in Healthcare

Industry 4.0 in healthcare involves use of a wide range of modern technologies including digitisation, artificial intelligence, user response data (ergonomics), human psychology, the Internet of Things, machine learning, big data mining, and augmented reality to name a few. The healthcare industry is undergoing a paradigm shift thanks to Industry 4.0, which provides better user comfort through proactive intervention in early detection and treatment of various diseases. The sector is now ready to make its next move towards Industry 5.0, but certain aspects that motivated this review paper need further consideration. As a fruitful outcome of this review, we surveyed modern trends in this arena of research and summarised the intricacies of new features to guide and prepare the sector for an Industry 5.0-ready healthcare system.

Concatenated Network Fusion Algorithm (CNFA) Based on Deep Learning: Improving the Detection Accuracy of Surface Defects for Ceramic Tile

The low accuracy of detection algorithms is one impediment in detecting ceramic tile’s surface defects online utilizing intelligent detection instead of human inspection. The purpose of this paper is to present a CNFA for resolving the obstacle. Firstly, a negative sample set is generated online by non-defective images of ceramic tiles, and a comparator based on a modified VGG16 extracts a reference image from it. Disguised rectangle boxes, including defective and non-defective, are acquired from the image to be inspected by a detector. A reference rectangle box most similar to the disguised rectangle box is extracted from the reference image. A discriminator is constituted with a modified MobileNetV3 network serving as the backbone and a metric learning loss function strengthening feature recognition, distinguishing the true and false of disguised and reference rectangle boxes. Results exhibit that the discriminator appears to have an accuracy of 98.02%, 13% more than other algorithms. Furthermore, the CNFA performs an average accuracy of 98.19%, and the consumption time of a single image extends by only 64.35 ms, which has little influence on production efficiency. It provides a theoretical and practical reference for surface defect detection of products with complex and changeable textures in industrial environments.

Recognition of Human Face Regions under Adverse Conditions—Face Masks and Glasses—In Thermographic Sanitary Barriers through Learning Transfer from an Object Detector

The COVID-19 pandemic has detrimentally affected people’s lives and the economies of many countries, causing disruption in the health, education, transport, and other sectors. Several countries have implemented sanitary barriers at airports, bus and train stations, company gates, and other shared spaces to detect patients with viral symptoms in an effort to contain the spread of the disease. As fever is one of the most recurrent disease symptoms, the demand for devices that measure skin (body surface) temperature has increased. The thermal imaging camera, also known as a thermal imager, is one such device used to measure temperature. It employs a technology known as infrared thermography and is a noninvasive, fast, and objective tool. This study employed machine learning transfer using You Only Look Once (YOLO) to detect the hottest temperatures in the regions of interest (ROIs) of the human face in thermographic images, allowing the identification of a febrile state in humans. The algorithms detect areas of interest in the thermographic images, such as the eyes, forehead, and ears, before analyzing the temperatures in these regions. The developed software achieved excellent performance in detecting the established areas of interest, adequately indicating the maximum temperature within each region of interest, and correctly choosing the maximum temperature among them.

Smart Pothole Detection Using Deep Learning Based on Dilated Convolution

Roads make a huge contribution to the economy and act as a platform for transportation. Potholes in roads are one of the major concerns in transportation infrastructure. A lot of research has proposed using computer vision techniques to automate pothole detection that include a wide range of image processing and object detection algorithms. There is a need to automate the pothole detection process with adequate accuracy and speed and implement the process easily and with low setup cost. In this paper, we have developed efficient deep learning convolution neural networks (CNNs) to detect potholes in real-time with adequate accuracy. To reduce the computational cost and improve the training results, this paper proposes a modified VGG16 (MVGG16) network by removing some convolution layers and using different dilation rates. Moreover, this paper uses the MVGG16 as a backbone network for the Faster R-CNN. In addition, this work compares the performance of YOLOv5 (Large (Yl), Medium (Ym), and Small (Ys)) models with ResNet101 backbone and Faster R-CNN with ResNet50(FPN), VGG16, MobileNetV2, InceptionV3, and MVGG16 backbones. The experimental results show that the Ys model is more applicable for real-time pothole detection because of its speed. In addition, using the MVGG16 network as the backbone of the Faster R-CNN provides better mean precision and shorter inference time than using VGG16, InceptionV3, or MobilNetV2 backbones. The proposed MVGG16 succeeds in balancing the pothole detection accuracy and speed.

Traditional Artificial Neural Networks Versus Deep Learning in Optimization of Material Aspects of 3D Printing

3D printing of assistive devices requires optimization of material selection, raw materials formulas, and complex printing processes that have to balance a high number of variable but highly correlated variables. The performance of patient-specific 3D printed solutions is still limited by both the increasing number of available materials with different properties (including multi-material printing) and the large number of process features that need to be optimized. The main purpose of this study is to compare the optimization of 3D printing properties toward the maximum tensile force of an exoskeleton sample based on two different approaches: traditional artificial neural networks (ANNs) and a deep learning (DL) approach based on convolutional neural networks (CNNs). Compared with the results from the traditional ANN approach, optimization based on DL decreased the speed of the calculations by up to 1.5 times with the same print quality, improved the quality, decreased the MSE, and a set of printing parameters not previously determined by trial and error was also identified. The above-mentioned results show that DL is an effective tool with significant potential for wide application in the planning and optimization of material properties in the 3D printing process. Further research is needed to apply low-cost but more computationally efficient solutions to multi-tasking and multi-material additive manufacturing.

Study on Meso-Structure Evolution in Granular Matters Based on the Contact Loop Recognition and Determination Technique

On the mesoscopic scale, granular matter is tessellated into contact loops by a contact network. The stability of granular matter is highly dependent on the evolution of contact loops, including the number and area evolutions of contact loops with different geometric shapes (which can reflect the mechanical variables in the macroscale). For the features of numerous loops with complex geometry shapes in contact network images, a contact loop recognition and determination technique was developed in this study. Then, numerical biaxial compression tests were performed by the discrete element method (DEM) to investigate how the meso-structural indexes evolve along with the macro-mechanical indexes. The results show that the proposed Q-Y algorithm is effective in determining the geometric types of contact loops from contact network images. The evolution of contact loops is most active in the hardening stage, during which the number percentages of L₃ (loops with three sides) and L₆₊ (loops with six or more sides) show opposite evolution patterns. For the area percentage, only L₆₊ increases while others decrease. Considering the meso-structural indexes (number percentage and area percentage of loops) are sensitive to the change of macro-mechanical indexes (deviatoric stress, axial strain, and volumetric strain) in the hardening stage. Multivariate models were established to build a bridge between the meso-structure and the macro-mechanics.

Prediction of Mechanical Properties of Artificially Weathered Wood by Color Change and Machine Learning

Color parameters were used in this study to develop a machine learning model for predicting the mechanical properties of artificially weathered fir, alder, oak, and poplar wood. A CIELAB color measuring system was employed to study the color changes in wood samples. The color parameters were fed into a decision tree model for predicting the MOE and MOR values of the wood samples. The results indicated a reduction in the mechanical properties of the samples, where fir and alder were the most and least degraded wood under weathering conditions, respectively. The mechanical degradation was correlated with the color change, where the most resistant wood to color change exhibited less reduction in the mechanical properties. The predictive machine learning model estimated the MOE and MOR values with a maximum R² of 0.87 and 0.88, respectively. Thus, variations in the color parameters of wood can be considered informative features linked to the mechanical properties of small-sized and clear wood. Further research could study the effectiveness of the model when analyzing large-sized timber.

Object Detection Method for Grasping Robot Based on Improved YOLOv5

In the industrial field, the anthropomorphism of grasping robots is the trend of future development, however, the basic vision technology adopted by the grasping robot at this stage has problems such as inaccurate positioning and low recognition efficiency. Based on this practical problem, in order to achieve more accurate positioning and recognition of objects, an object detection method for grasping robot based on improved YOLOv5 was proposed in this paper. Firstly, the robot object detection platform was designed, and the wooden block image data set is being proposed. Secondly, the Eye-In-Hand calibration method was used to obtain the relative three-dimensional pose of the object. Then the network pruning method was used to optimize the YOLOv5 model from the two dimensions of network depth and network width. Finally, the hyper parameter optimization was carried out. The simulation results show that the improved YOLOv5 network proposed in this paper has better object detection performance. The specific performance is that the recognition precision, recall, mAP value and F1 score are 99.35%, 99.38%, 99.43% and 99.41% respectively. Compared with the original YOLOv5s, YOLOv5m and YOLOv5l models, the mAP of the YOLOv5_ours model has increased by 1.12%, 1.2% and 1.27%, respectively, and the scale of the model has been reduced by 10.71%, 70.93% and 86.84%, respectively. The object detection experiment has verified the feasibility of the method proposed in this paper.

Intelligent Optimization Algorithm-Based Path Planning for a Mobile Robot

The purpose of mobile robot path planning is to produce the optimal safe path. However, mobile robots have poor real-time obstacle avoidance in local path planning and longer paths in global path planning. In order to improve the accuracy of real-time obstacle avoidance prediction of local path planning, shorten the path length of global path planning, reduce the path planning time, and then obtain a better safe path, we propose a real-time obstacle avoidance decision model based on machine learning (ML) algorithms, an improved smooth rapidly exploring random tree (S-RRT) algorithm, and an improved hybrid genetic algorithm-ant colony optimization (HGA-ACO). Firstly, in local path planning, the machine learning algorithms are used to train the datasets, the real-time obstacle avoidance decision model is established, and cross validation is performed. Secondly, in global path planning, the greedy algorithm idea and B-spline curve are introduced into the RRT algorithm, redundant nodes are removed, and the reverse iteration is performed to generate a smooth path. Then, in path planning, the fitness function and genetic operation method of genetic algorithm are optimized, the pheromone update strategy and deadlock elimination strategy of ant colony algorithm are optimized, and the genetic-ant colony fusion strategy is used to fuse the two algorithms. Finally, the optimized path planning algorithm is used for simulation experiment. Comparative simulation experiments show that the random forest has the highest real-time obstacle avoidance prediction accuracy in local path planning, and the S-RRT algorithm can effectively shorten the total path length generated by the RRT algorithm in global path planning. The HGA-ACO algorithm can reduce the iteration number reasonably, reduce the search time effectively, and obtain the optimal solution in path planning.

Imbalance Modelling for Defect Detection in Ceramic Substrate by Using Convolutional Neural Network

The complexity of defect detection in a ceramic substrate causes interclass and intraclass imbalance problems. Identifying flaws in ceramic substrates has traditionally relied on aberrant material occurrences and characteristic quantities. However, defect substrates in ceramic are typically small and have a wide variety of defect distributions, thereby making defect detection more challenging and difficult. Thus, we propose a method for defect detection based on unsupervised learning and deep learning. First, the proposed method conducts K-means clustering for grouping instances according to their inherent complex characteristics. Second, the distribution of rarely occurring instances is balanced by using augmentation filters. Finally, a convolutional neural network is trained by using the balanced dataset. The effectiveness of the proposed method was validated by comparing the results with those of other methods. Experimental results show that the proposed method outperforms other methods.

A Versatile Punch Stroke Correction Model for Trial V-Bending of Sheet Metals Based on Data-Driven Method

During air bending of sheet metals, the correction of punch stroke for springback control is always implemented through repeated trial bending until achieving the forming accuracy of bending parts. In this study, a modelling method for correction of punch stroke is presented for guiding trial bending based on a data-driven technique. Firstly, the big data for the model are mainly generated from a large number of finite element simulations, considering many variables, e.g., material parameters, dimensions of V-dies and blanks, and processing parameters. Based on the big data, two punch stroke correction models are developed via neural network and dimensional analysis, respectively. The analytic comparison shows that the neural network model is more suitable for guiding trial bending of sheet metals than the dimensional analysis model, which has mechanical significance. The actual trial bending tests prove that the neural-network-based punch stroke correction model presents great versatility and accuracy in the guidance of trial bending, leading to a reduction in the number of trial bends and an improvement in the production efficiency of air bending.

Surface Defect Detection Methods for Industrial Products: A Review

The comprehensive intelligent development of the manufacturing industry puts forward new requirements for the quality inspection of industrial products. This paper summarizes the current research status of machine learning methods in surface defect detection, a key part in the quality inspection of industrial products. First, according to the use of surface features, the application of traditional machine vision surface defect detection methods in industrial product surface defect detection is summarized from three aspects: texture features, color features, and shape features. Secondly, the research status of industrial product surface defect detection based on deep learning technology in recent years is discussed from three aspects: supervised method, unsupervised method, and weak supervised method. Then, the common key problems and their solutions in industrial surface defect detection are systematically summarized; the key problems include real-time problem, small sample problem, small target problem, unbalanced sample problem. Lastly, the commonly used datasets of industrial surface defects in recent years are more comprehensively summarized, and the latest research methods on the MVTec AD dataset are compared, so as to provide some reference for the further research and development of industrial surface defect detection technology.

Deep-Learning-Based Approach to Anomaly Detection Techniques for Large Acoustic Data in Machine Operation

As the workforce shrinks, the demand for automatic, labor-saving, anomaly detection technology that can perform maintenance on advanced equipment such as vehicles has been increasing. In a vehicular environment, noise in the cabin, which directly affects users, is considered an important factor in lowering the emotional satisfaction of the driver and/or passengers in the vehicles. In this study, we provide an efficient method that can collect acoustic data, measured using a large number of microphones, in order to detect abnormal operations inside the machine via deep learning in a quick and highly accurate manner. Unlike most current approaches based on Long Short-Term Memory (LSTM) or autoencoders, we propose an anomaly detection (AD) algorithm that can overcome the limitations of noisy measurement and detection system anomalies via noise signals measured inside the mechanical system. These features are utilized to train a variety of anomaly detection models for demonstration in noisy environments with five different errors in machine operation, achieving an accuracy of approximately 90% or more.