|
1
|
Luo W, Zhao Y, Shao Q, Li X, Wang D, Zhang T, Liu F, Duan L, He Y, Wang Y, Zhang G, Wang X, Yu Z. Procapra Przewalskii Tracking Autonomous Unmanned Aerial Vehicle Based on Improved Long and Short-Term Memory Kalman Filters. SENSORS (BASEL, SWITZERLAND) 2023; 23:3948. [PMID: 37112289 PMCID: PMC10144096 DOI: 10.3390/s23083948] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 03/20/2023] [Revised: 04/02/2023] [Accepted: 04/10/2023] [Indexed: 06/19/2023]
Abstract
This paper presents an autonomous unmanned-aerial-vehicle (UAV) tracking system based on an improved long and short-term memory (LSTM) Kalman filter (KF) model. The system can estimate the three-dimensional (3D) attitude and precisely track the target object without manual intervention. Specifically, the YOLOX algorithm is employed to track and recognize the target object, which is then combined with the improved KF model for precise tracking and recognition. In the LSTM-KF model, three different LSTM networks (f, Q, and R) are adopted to model a nonlinear transfer function to enable the model to learn rich and dynamic Kalman components from the data. The experimental results disclose that the improved LSTM-KF model exhibits higher recognition accuracy than the standard LSTM and the independent KF model. It verifies the robustness, effectiveness, and reliability of the autonomous UAV tracking system based on the improved LSTM-KF model in object recognition and tracking and 3D attitude estimation.
Collapse
Affiliation(s)
- Wei Luo
- North China Institute of Aerospace Engineering, Langfang 065000, China
- Key Laboratory of Land Surface Pattern and Simulation, Institute of Geographic Sciences and Natural Resources Research, Chinese Academy of Sciences, Beijing 100101, China
- Aerospace Remote Sensing Information Processing and Application Collaborative Innovation Center of Hebei Province, Langfang 065000, China
- National Joint Engineering Research Center of Space Remote Sensing Information Application Technology, Langfang 065000, China
- Agricultural Information Institute of Chinese Academy of Agricultural Sciences, Key Laboratory of Agricultural Monitoring and Early Warning Technology, Ministry of Agriculture and Rural Affairs, Beijing 100081, China
| | - Yongxiang Zhao
- North China Institute of Aerospace Engineering, Langfang 065000, China
| | - Quanqin Shao
- Key Laboratory of Land Surface Pattern and Simulation, Institute of Geographic Sciences and Natural Resources Research, Chinese Academy of Sciences, Beijing 100101, China
- University of Chinese Academy of Sciences, Beijing 101407, China
| | - Xiaoliang Li
- North China Institute of Aerospace Engineering, Langfang 065000, China
| | - Dongliang Wang
- Key Laboratory of Land Surface Pattern and Simulation, Institute of Geographic Sciences and Natural Resources Research, Chinese Academy of Sciences, Beijing 100101, China
| | - Tongzuo Zhang
- University of Chinese Academy of Sciences, Beijing 101407, China
- Key Laboratory of Adaptation and Evolution of Plateau Biota, Northwest Institute of Plateau Biology, Chinese Academy of Sciences, Xining 810001, China
| | - Fei Liu
- Intelligent Garden and Ecohealth Laboratory (iGE), College of Biosystems Engineering and Food Science, Zhejiang University, 866 Yuhangtang Road, Hangzhou 310058, China
| | - Longfang Duan
- North China Institute of Aerospace Engineering, Langfang 065000, China
- Aerospace Remote Sensing Information Processing and Application Collaborative Innovation Center of Hebei Province, Langfang 065000, China
- National Joint Engineering Research Center of Space Remote Sensing Information Application Technology, Langfang 065000, China
| | - Yuejun He
- North China Institute of Aerospace Engineering, Langfang 065000, China
- Aerospace Remote Sensing Information Processing and Application Collaborative Innovation Center of Hebei Province, Langfang 065000, China
- National Joint Engineering Research Center of Space Remote Sensing Information Application Technology, Langfang 065000, China
| | - Yancang Wang
- North China Institute of Aerospace Engineering, Langfang 065000, China
- Aerospace Remote Sensing Information Processing and Application Collaborative Innovation Center of Hebei Province, Langfang 065000, China
- National Joint Engineering Research Center of Space Remote Sensing Information Application Technology, Langfang 065000, China
| | - Guoqing Zhang
- North China Institute of Aerospace Engineering, Langfang 065000, China
| | - Xinghui Wang
- North China Institute of Aerospace Engineering, Langfang 065000, China
- Aerospace Remote Sensing Information Processing and Application Collaborative Innovation Center of Hebei Province, Langfang 065000, China
- National Joint Engineering Research Center of Space Remote Sensing Information Application Technology, Langfang 065000, China
| | - Zhongde Yu
- North China Institute of Aerospace Engineering, Langfang 065000, China
| |
Collapse
|
|
2
|
Tse KW, Pi R, Sun Y, Wen CY, Feng Y. A Novel Real-Time Autonomous Crack Inspection System Based on Unmanned Aerial Vehicles. SENSORS (BASEL, SWITZERLAND) 2023; 23:3418. [PMID: 37050478 PMCID: PMC10098570 DOI: 10.3390/s23073418] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 02/14/2023] [Revised: 03/16/2023] [Accepted: 03/22/2023] [Indexed: 06/19/2023]
Abstract
Traditional methods on crack inspection for large infrastructures require a number of structural health inspection devices and instruments. They usually use the signal changes caused by physical deformations from cracks to detect the cracks, which is time-consuming and cost-ineffective. In this work, we propose a novel real-time crack inspection system based on unmanned aerial vehicles for real-world applications. The proposed system successfully detects and classifies various types of cracks. It can accurately find the crack positions in the world coordinate system. Our detector is based on an improved YOLOv4 with an attention module, which produces 90.02% mean average precision (mAP) and outperforms the YOLOv4-original by 5.23% in terms of mAP. The proposed system is low-cost and lightweight. Moreover, it is not restricted by navigation trajectories. The experimental results demonstrate the robustness and effectiveness of our system in real-world crack inspection tasks.
Collapse
Affiliation(s)
- Kwai-Wa Tse
- Department of Aeronautical and Aviation Engineering, The Hong Kong Polytechnic University, Kowloon 999077, Hong Kong; (K.-W.T.)
| | - Rendong Pi
- Department of Mechanical Engineering, The Hong Kong Polytechnic University, Kowloon 999077, Hong Kong
| | - Yuxiang Sun
- Department of Mechanical Engineering, The Hong Kong Polytechnic University, Kowloon 999077, Hong Kong
| | - Chih-Yung Wen
- Department of Aeronautical and Aviation Engineering, The Hong Kong Polytechnic University, Kowloon 999077, Hong Kong; (K.-W.T.)
| | - Yurong Feng
- Department of Aeronautical and Aviation Engineering, The Hong Kong Polytechnic University, Kowloon 999077, Hong Kong; (K.-W.T.)
| |
Collapse
|
|
3
|
Chang CW, Lo LY, Cheung HC, Feng Y, Yang AS, Wen CY, Zhou W. Proactive Guidance for Accurate UAV Landing on a Dynamic Platform: A Visual-Inertial Approach. SENSORS (BASEL, SWITZERLAND) 2022; 22:404. [PMID: 35009946 PMCID: PMC8749553 DOI: 10.3390/s22010404] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 12/17/2021] [Revised: 12/27/2021] [Accepted: 01/01/2022] [Indexed: 12/29/2022]
Abstract
This work aimed to develop an autonomous system for unmanned aerial vehicles (UAVs) to land on moving platforms such as an automobile or a marine vessel, providing a promising solution for a long-endurance flight operation, a large mission coverage range, and a convenient recharging ground station. Unlike most state-of-the-art UAV landing frameworks that rely on UAV onboard computers and sensors, the proposed system fully depends on the computation unit situated on the ground vehicle/marine vessel to serve as a landing guidance system. Such a novel configuration can therefore lighten the burden of the UAV, and the computation power of the ground vehicle/marine vessel can be enhanced. In particular, we exploit a sensor fusion-based algorithm for the guidance system to perform UAV localization, whilst a control method based upon trajectory optimization is integrated. Indoor and outdoor experiments are conducted, and the results show that precise autonomous landing on a 43 cm × 43 cm platform can be performed.
Collapse
Affiliation(s)
- Ching-Wei Chang
- Department of Mechanical Engineering, The Hong Kong Polytechnic University, Kowloon, Hong Kong; (C.-W.C.); (H.C.C.)
| | - Li-Yu Lo
- Department of Aeronautical and Aviation Engineering, The Hong Kong Polytechnic University, Kowloon, Hong Kong; (L.-Y.L.); (Y.F.); (C.-Y.W.)
| | - Hiu Ching Cheung
- Department of Mechanical Engineering, The Hong Kong Polytechnic University, Kowloon, Hong Kong; (C.-W.C.); (H.C.C.)
| | - Yurong Feng
- Department of Aeronautical and Aviation Engineering, The Hong Kong Polytechnic University, Kowloon, Hong Kong; (L.-Y.L.); (Y.F.); (C.-Y.W.)
| | - An-Shik Yang
- Department of Energy and Refrigerating Air-Conditioning Engineering, National Taipei University of Technology, Taipei 10608, Taiwan;
| | - Chih-Yung Wen
- Department of Aeronautical and Aviation Engineering, The Hong Kong Polytechnic University, Kowloon, Hong Kong; (L.-Y.L.); (Y.F.); (C.-Y.W.)
| | - Weifeng Zhou
- School of Professional Education and Executive Development, The Hong Kong Polytechnic University, Kowloon, Hong Kong
| |
Collapse
|
|
4
|
Dynamic Object Tracking on Autonomous UAV System for Surveillance Applications. SENSORS 2021; 21:s21237888. [PMID: 34883913 PMCID: PMC8659946 DOI: 10.3390/s21237888] [Citation(s) in RCA: 10] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 09/27/2021] [Revised: 11/23/2021] [Accepted: 11/25/2021] [Indexed: 11/16/2022]
Abstract
The ever-burgeoning growth of autonomous unmanned aerial vehicles (UAVs) has demonstrated a promising platform for utilization in real-world applications. In particular, a UAV equipped with a vision system could be leveraged for surveillance applications. This paper proposes a learning-based UAV system for achieving autonomous surveillance, in which the UAV can be of assistance in autonomously detecting, tracking, and following a target object without human intervention. Specifically, we adopted the YOLOv4-Tiny algorithm for semantic object detection and then consolidated it with a 3D object pose estimation method and Kalman filter to enhance the perception performance. In addition, UAV path planning for a surveillance maneuver is integrated to complete the fully autonomous system. The perception module is assessed on a quadrotor UAV, while the whole system is validated through flight experiments. The experiment results verified the robustness, effectiveness, and reliability of the autonomous object tracking UAV system in performing surveillance tasks. The source code is released to the research community for future reference.
Collapse
|