1
|
Zhukov A, Rivero A, Benois-Pineau J, Zemmari A, Mosbah M. A Hybrid System for Defect Detection on Rail Lines through the Fusion of Object and Context Information. SENSORS (BASEL, SWITZERLAND) 2024; 24:1171. [PMID: 38400331 PMCID: PMC10892099 DOI: 10.3390/s24041171] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/21/2023] [Revised: 01/30/2024] [Accepted: 02/06/2024] [Indexed: 02/25/2024]
Abstract
Defect detection on rail lines is essential for ensuring safe and efficient transportation. Current image analysis methods with deep neural networks (DNNs) for defect detection often focus on the defects themselves while ignoring the related context. In this work, we propose a fusion model that combines both a targeted defect search and a context analysis, which is seen as a multimodal fusion task. Our model performs rule-based decision-level fusion, merging the confidence scores of multiple individual models to classify rail-line defects. We call the model "hybrid" in the sense that it is composed of supervised learning components and rule-based fusion. We first propose an improvement to existing vision-based defect detection methods by incorporating a convolutional block attention module (CBAM) in the you only look once (YOLO) versions 5 (YOLOv5) and 8 (YOLOv8) architectures for the detection of defects and contextual image elements. This attention module is applied at different detection scales. The domain-knowledge rules are applied to fuse the detection results. Our method demonstrates improvements over baseline models in vision-based defect detection. The model is open for the integration of modalities other than an image, e.g., sound and accelerometer data.
Collapse
Affiliation(s)
- Alexey Zhukov
- Univ. Bordeaux, CNRS, Bordeaux INP, INRIA, LaBRI, UMR 5800, 33400 Talence, France; (J.B.-P.); (A.Z.); (M.M.)
- Ferrocampus, 18 bd Guillet-Maillet, 17100 Saintes, France
| | - Alain Rivero
- SNCF-Réseau, Direction Générale Industrielle et Ingénierie, Département IP3M DM MATRICE, 6 Avenue François Mitterrand, 93574 Paris, France;
| | - Jenny Benois-Pineau
- Univ. Bordeaux, CNRS, Bordeaux INP, INRIA, LaBRI, UMR 5800, 33400 Talence, France; (J.B.-P.); (A.Z.); (M.M.)
| | - Akka Zemmari
- Univ. Bordeaux, CNRS, Bordeaux INP, INRIA, LaBRI, UMR 5800, 33400 Talence, France; (J.B.-P.); (A.Z.); (M.M.)
| | - Mohamed Mosbah
- Univ. Bordeaux, CNRS, Bordeaux INP, INRIA, LaBRI, UMR 5800, 33400 Talence, France; (J.B.-P.); (A.Z.); (M.M.)
| |
Collapse
|
2
|
Cumbajin E, Rodrigues N, Costa P, Miragaia R, Frazão L, Costa N, Fernández-Caballero A, Carneiro J, Buruberri LH, Pereira A. A Real-Time Automated Defect Detection System for Ceramic Pieces Manufacturing Process Based on Computer Vision with Deep Learning. SENSORS (BASEL, SWITZERLAND) 2023; 24:232. [PMID: 38203095 PMCID: PMC10781230 DOI: 10.3390/s24010232] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/02/2023] [Revised: 12/19/2023] [Accepted: 12/28/2023] [Indexed: 01/12/2024]
Abstract
Defect detection is a key element of quality control in today's industries, and the process requires the incorporation of automated methods, including image sensors, to detect any potential defects that may occur during the manufacturing process. While there are various methods that can be used for inspecting surfaces, such as those of metal and building materials, there are only a limited number of techniques that are specifically designed to analyze specialized surfaces, such as ceramics, which can potentially reveal distinctive anomalies or characteristics that require a more precise and focused approach. This article describes a study and proposes an extended solution for defect detection on ceramic pieces within an industrial environment, utilizing a computer vision system with deep learning models. The solution includes an image acquisition process and a labeling platform to create training datasets, as well as an image preprocessing technique, to feed a machine learning algorithm based on convolutional neural networks (CNNs) capable of running in real time within a manufacturing environment. The developed solution was implemented and evaluated at a leading Portuguese company that specializes in the manufacturing of tableware and fine stoneware. The collaboration between the research team and the company resulted in the development of an automated and effective system for detecting defects in ceramic pieces, achieving an accuracy of 98.00% and an F1-Score of 97.29%.
Collapse
Affiliation(s)
- Esteban Cumbajin
- Computer Science and Communications Research Centre, School of Technology and Management, Polytechnic of Leiria, 2411-901 Leiria, Portugal; (E.C.); (N.R.); (P.C.); (R.M.); (L.F.); (N.C.)
| | - Nuno Rodrigues
- Computer Science and Communications Research Centre, School of Technology and Management, Polytechnic of Leiria, 2411-901 Leiria, Portugal; (E.C.); (N.R.); (P.C.); (R.M.); (L.F.); (N.C.)
| | - Paulo Costa
- Computer Science and Communications Research Centre, School of Technology and Management, Polytechnic of Leiria, 2411-901 Leiria, Portugal; (E.C.); (N.R.); (P.C.); (R.M.); (L.F.); (N.C.)
| | - Rolando Miragaia
- Computer Science and Communications Research Centre, School of Technology and Management, Polytechnic of Leiria, 2411-901 Leiria, Portugal; (E.C.); (N.R.); (P.C.); (R.M.); (L.F.); (N.C.)
| | - Luís Frazão
- Computer Science and Communications Research Centre, School of Technology and Management, Polytechnic of Leiria, 2411-901 Leiria, Portugal; (E.C.); (N.R.); (P.C.); (R.M.); (L.F.); (N.C.)
| | - Nuno Costa
- Computer Science and Communications Research Centre, School of Technology and Management, Polytechnic of Leiria, 2411-901 Leiria, Portugal; (E.C.); (N.R.); (P.C.); (R.M.); (L.F.); (N.C.)
| | - Antonio Fernández-Caballero
- Instituto de Investigación en Informática de Albacete, 02071 Albacete, Spain;
- Departamento de Sistemas Informáticos, Universidad de Castilla-La Mancha, 02071 Albacete, Spain
| | - Jorge Carneiro
- Grestel-Produtos Cerâmicos S.A, Zona Industrial de Vagos-Lote 78, 3840-385 Vagos, Portugal; (J.C.); (L.H.B.)
| | - Leire H. Buruberri
- Grestel-Produtos Cerâmicos S.A, Zona Industrial de Vagos-Lote 78, 3840-385 Vagos, Portugal; (J.C.); (L.H.B.)
| | - António Pereira
- Computer Science and Communications Research Centre, School of Technology and Management, Polytechnic of Leiria, 2411-901 Leiria, Portugal; (E.C.); (N.R.); (P.C.); (R.M.); (L.F.); (N.C.)
- INOV INESC Inovação, Institute of New Technologies, Leiria Office, 2411-901 Leiria, Portugal
| |
Collapse
|