|
1
|
Al-Hammadi M, Bencherif MA, Alsulaiman M, Muhammad G, Mekhtiche MA, Abdul W, Alohali YA, Alrayes TS, Mathkour H, Faisal M, Algabri M, Altaheri H, Alfakih T, Ghaleb H. Spatial Attention-Based 3D Graph Convolutional Neural Network for Sign Language Recognition. Sensors (Basel) 2022; 22:s22124558. [PMID: 35746341 PMCID: PMC9227856 DOI: 10.3390/s22124558] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 05/20/2022] [Revised: 06/10/2022] [Accepted: 06/13/2022] [Indexed: 02/04/2023]
Abstract
Sign language is the main channel for hearing-impaired people to communicate with others. It is a visual language that conveys highly structured components of manual and non-manual parameters such that it needs a lot of effort to master by hearing people. Sign language recognition aims to facilitate this mastering difficulty and bridge the communication gap between hearing-impaired people and others. This study presents an efficient architecture for sign language recognition based on a convolutional graph neural network (GCN). The presented architecture consists of a few separable 3DGCN layers, which are enhanced by a spatial attention mechanism. The limited number of layers in the proposed architecture enables it to avoid the common over-smoothing problem in deep graph neural networks. Furthermore, the attention mechanism enhances the spatial context representation of the gestures. The proposed architecture is evaluated on different datasets and shows outstanding results.
Collapse
Affiliation(s)
- Muneer Al-Hammadi
- Centre of Smart Robotics Research (CS2R), King Saud University, Riyadh 11543, Saudi Arabia; (M.A.-H.); (M.A.); (M.A.M.); (W.A.); (Y.A.A.); (H.M.); (M.F.); (M.A.); (H.A.); (T.A.); (H.G.)
- Department of Civil and Environmental Engineering, Faculty of Engineering, Norwegian University of Science and Technology, Høgskoleringen 1, 7034 Trondheim, Norway
| | - Mohamed A. Bencherif
- Centre of Smart Robotics Research (CS2R), King Saud University, Riyadh 11543, Saudi Arabia; (M.A.-H.); (M.A.); (M.A.M.); (W.A.); (Y.A.A.); (H.M.); (M.F.); (M.A.); (H.A.); (T.A.); (H.G.)
- Computer Engineering Department, College of Computer and Information Sciences, King Saud University, Riyadh 11543, Saudi Arabia
- Correspondence: (M.A.B.); (G.M.)
| | - Mansour Alsulaiman
- Centre of Smart Robotics Research (CS2R), King Saud University, Riyadh 11543, Saudi Arabia; (M.A.-H.); (M.A.); (M.A.M.); (W.A.); (Y.A.A.); (H.M.); (M.F.); (M.A.); (H.A.); (T.A.); (H.G.)
- Computer Engineering Department, College of Computer and Information Sciences, King Saud University, Riyadh 11543, Saudi Arabia
| | - Ghulam Muhammad
- Centre of Smart Robotics Research (CS2R), King Saud University, Riyadh 11543, Saudi Arabia; (M.A.-H.); (M.A.); (M.A.M.); (W.A.); (Y.A.A.); (H.M.); (M.F.); (M.A.); (H.A.); (T.A.); (H.G.)
- Computer Engineering Department, College of Computer and Information Sciences, King Saud University, Riyadh 11543, Saudi Arabia
- Correspondence: (M.A.B.); (G.M.)
| | - Mohamed Amine Mekhtiche
- Centre of Smart Robotics Research (CS2R), King Saud University, Riyadh 11543, Saudi Arabia; (M.A.-H.); (M.A.); (M.A.M.); (W.A.); (Y.A.A.); (H.M.); (M.F.); (M.A.); (H.A.); (T.A.); (H.G.)
- Computer Engineering Department, College of Computer and Information Sciences, King Saud University, Riyadh 11543, Saudi Arabia
| | - Wadood Abdul
- Centre of Smart Robotics Research (CS2R), King Saud University, Riyadh 11543, Saudi Arabia; (M.A.-H.); (M.A.); (M.A.M.); (W.A.); (Y.A.A.); (H.M.); (M.F.); (M.A.); (H.A.); (T.A.); (H.G.)
- Computer Engineering Department, College of Computer and Information Sciences, King Saud University, Riyadh 11543, Saudi Arabia
| | - Yousef A. Alohali
- Centre of Smart Robotics Research (CS2R), King Saud University, Riyadh 11543, Saudi Arabia; (M.A.-H.); (M.A.); (M.A.M.); (W.A.); (Y.A.A.); (H.M.); (M.F.); (M.A.); (H.A.); (T.A.); (H.G.)
- Computer Science Department, College of Computer and Information Sciences, King Saud University, Riyadh 11543, Saudi Arabia
| | - Tareq S. Alrayes
- Department of Special Education, College of Education, King Saud University, Riyadh 11543, Saudi Arabia;
| | - Hassan Mathkour
- Centre of Smart Robotics Research (CS2R), King Saud University, Riyadh 11543, Saudi Arabia; (M.A.-H.); (M.A.); (M.A.M.); (W.A.); (Y.A.A.); (H.M.); (M.F.); (M.A.); (H.A.); (T.A.); (H.G.)
- Computer Science Department, College of Computer and Information Sciences, King Saud University, Riyadh 11543, Saudi Arabia
| | - Mohammed Faisal
- Centre of Smart Robotics Research (CS2R), King Saud University, Riyadh 11543, Saudi Arabia; (M.A.-H.); (M.A.); (M.A.M.); (W.A.); (Y.A.A.); (H.M.); (M.F.); (M.A.); (H.A.); (T.A.); (H.G.)
- Center of AI & Robotics, Kuwait College of Science and Technology (KCST), Kuwait City 35004, Kuwait
| | - Mohammed Algabri
- Centre of Smart Robotics Research (CS2R), King Saud University, Riyadh 11543, Saudi Arabia; (M.A.-H.); (M.A.); (M.A.M.); (W.A.); (Y.A.A.); (H.M.); (M.F.); (M.A.); (H.A.); (T.A.); (H.G.)
- Computer Science Department, College of Computer and Information Sciences, King Saud University, Riyadh 11543, Saudi Arabia
| | - Hamdi Altaheri
- Centre of Smart Robotics Research (CS2R), King Saud University, Riyadh 11543, Saudi Arabia; (M.A.-H.); (M.A.); (M.A.M.); (W.A.); (Y.A.A.); (H.M.); (M.F.); (M.A.); (H.A.); (T.A.); (H.G.)
- Computer Engineering Department, College of Computer and Information Sciences, King Saud University, Riyadh 11543, Saudi Arabia
| | - Taha Alfakih
- Centre of Smart Robotics Research (CS2R), King Saud University, Riyadh 11543, Saudi Arabia; (M.A.-H.); (M.A.); (M.A.M.); (W.A.); (Y.A.A.); (H.M.); (M.F.); (M.A.); (H.A.); (T.A.); (H.G.)
| | - Hamid Ghaleb
- Centre of Smart Robotics Research (CS2R), King Saud University, Riyadh 11543, Saudi Arabia; (M.A.-H.); (M.A.); (M.A.M.); (W.A.); (Y.A.A.); (H.M.); (M.F.); (M.A.); (H.A.); (T.A.); (H.G.)
- Software Engineering Department, College of Computer and Information Sciences, King Saud University, Riyadh 11543, Saudi Arabia
| |
Collapse
|