1
|
Harandi N, Vandenberghe B, Vankerschaver J, Depuydt S, Van Messem A. How to make sense of 3D representations for plant phenotyping: a compendium of processing and analysis techniques. PLANT METHODS 2023; 19:60. [PMID: 37353846 DOI: 10.1186/s13007-023-01031-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Subscribe] [Scholar Register] [Received: 10/18/2022] [Accepted: 05/19/2023] [Indexed: 06/25/2023]
Abstract
Computer vision technology is moving more and more towards a three-dimensional approach, and plant phenotyping is following this trend. However, despite its potential, the complexity of the analysis of 3D representations has been the main bottleneck hindering the wider deployment of 3D plant phenotyping. In this review we provide an overview of typical steps for the processing and analysis of 3D representations of plants, to offer potential users of 3D phenotyping a first gateway into its application, and to stimulate its further development. We focus on plant phenotyping applications where the goal is to measure characteristics of single plants or crop canopies on a small scale in research settings, as opposed to large scale crop monitoring in the field.
Collapse
Affiliation(s)
- Negin Harandi
- Center for Biosystems and Biotech Data Science, Ghent University Global Campus, 119 Songdomunhwa-ro, Yeonsu-gu, Incheon, South Korea
- Department of Applied Mathematics, Computer Science and Statistics, Ghent University, Krijgslaan 281, S9, Ghent, Belgium
| | | | - Joris Vankerschaver
- Center for Biosystems and Biotech Data Science, Ghent University Global Campus, 119 Songdomunhwa-ro, Yeonsu-gu, Incheon, South Korea
- Department of Applied Mathematics, Computer Science and Statistics, Ghent University, Krijgslaan 281, S9, Ghent, Belgium
| | - Stephen Depuydt
- Erasmus Applied University of Sciences and Arts, Campus Kaai, Nijverheidskaai 170, Anderlecht, Belgium
| | - Arnout Van Messem
- Department of Mathematics, Université de Liège, Allée de la Découverte 12, Liège, Belgium.
| |
Collapse
|
2
|
Liu Y, Yuan H, Zhao X, Fan C, Cheng M. Fast reconstruction method of three-dimension model based on dual RGB-D cameras for peanut plant. PLANT METHODS 2023; 19:17. [PMID: 36843020 PMCID: PMC9969713 DOI: 10.1186/s13007-023-00998-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 07/31/2022] [Accepted: 02/20/2023] [Indexed: 06/18/2023]
Abstract
BACKGROUND Plant shape and structure are important factors in peanut breeding research. Constructing a three-dimension (3D) model can provide an effective digital tool for comprehensive and quantitative analysis of peanut plant structure. Fast and accurate are always the goals of the plant 3D model reconstruction research. RESULTS We proposed a 3D reconstruction method based on dual RGB-D cameras for the peanut plant 3D model quickly and accurately. The two Kinect v2 were mirror symmetry placed on both sides of the peanut plant, and the point cloud data obtained were filtered twice to remove noise interference. After rotation and translation based on the corresponding geometric relationship, the point cloud acquired by the two Kinect v2 was converted to the same coordinate system and spliced into the 3D structure of the peanut plant. The experiment was conducted at various growth stages based on twenty potted peanuts. The plant traits' height, width, length, and volume were calculated through the reconstructed 3D models, and manual measurement was also carried out during the experiment processing. The accuracy of the 3D model was evaluated through a synthetic coefficient, which was generated by calculating the average accuracy of the four traits. The test result showed that the average accuracy of the reconstructed peanut plant 3D model by this method is 93.42%. A comparative experiment with the iterative closest point (ICP) algorithm, a widely used 3D modeling algorithm, was additionally implemented to test the rapidity of this method. The test result shows that the proposed method is 2.54 times faster with approximated accuracy compared to the ICP method. CONCLUSIONS The reconstruction method for the 3D model of the peanut plant described in this paper is capable of rapidly and accurately establishing a 3D model of the peanut plant while also meeting the modeling requirements for other species' breeding processes. This study offers a potential tool to further explore the 3D model for improving traits and agronomic qualities of plants.
Collapse
Affiliation(s)
- Yadong Liu
- College of Mechanical and Electrical Engineering, Hebei Agricultural University, Baoding, 071001, China
| | - Hongbo Yuan
- College of Mechanical and Electrical Engineering, Hebei Agricultural University, Baoding, 071001, China
| | - Xin Zhao
- College of Mechanical and Electrical Engineering, Hebei Agricultural University, Baoding, 071001, China
| | - Caihu Fan
- College of Mechanical and Electrical Engineering, Hebei Agricultural University, Baoding, 071001, China
| | - Man Cheng
- College of Mechanical and Electrical Engineering, Hebei Agricultural University, Baoding, 071001, China.
| |
Collapse
|
3
|
Han B, Li Y, Bie Z, Peng C, Huang Y, Xu S. MIX-NET: Deep Learning-Based Point Cloud Processing Method for Segmentation and Occlusion Leaf Restoration of Seedlings. PLANTS (BASEL, SWITZERLAND) 2022; 11:plants11233342. [PMID: 36501381 PMCID: PMC9739940 DOI: 10.3390/plants11233342] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/12/2022] [Revised: 11/19/2022] [Accepted: 11/24/2022] [Indexed: 05/27/2023]
Abstract
In this paper, a novel point cloud segmentation and completion framework is proposed to achieve high-quality leaf area measurement of melon seedlings. In particular, the input of our algorithm is the point cloud data collected by an Azure Kinect camera from the top view of the seedlings, and our method can enhance measurement accuracy from two aspects based on the acquired data. On the one hand, we propose a neighborhood space-constrained method to effectively filter out the hover points and outlier noise of the point cloud, which can enhance the quality of the point cloud data significantly. On the other hand, by leveraging the purely linear mixer mechanism, a new network named MIX-Net is developed to achieve segmentation and completion of the point cloud simultaneously. Different from previous methods that separate these two tasks, the proposed network can better balance these two tasks in a more definite and effective way, leading to satisfactory performance on these two tasks. The experimental results prove that our methods can outperform other competitors and provide more accurate measurement results. Specifically, for the seedling segmentation task, our method can obtain a 3.1% and 1.7% performance gain compared with PointNet++ and DGCNN, respectively. Meanwhile, the R2 of leaf area measurement improved from 0.87 to 0.93 and MSE decreased from 2.64 to 2.26 after leaf shading completion.
Collapse
Affiliation(s)
- Binbin Han
- College of Engineering, Huazhong Agricultural University, Wuhan 430070, China
- Key Laboratory of Agricultural Equipment for the Middle and Lower Reaches of the Yangtze River, Ministry of Agriculture, Wuhan 430070, China
- Shenzhen Institute of Nutrition and Health, Huazhong Agricultural University, Shenzhen 518000, China
- Shenzhen Branch, Guangdong Laboratory for Lingnan Modern Agriculture, Genome Analysis Laboratory of the Ministry of Agriculture, Agricultural Genomics Institute at Shenzhen, Chinese Academy of Agricultural Sciences, Shenzhen 518000, China
- School of Mathematics and Computer Science, Wuhan Polytechnic University, Wuhan 430023, China
| | - Yaqin Li
- School of Mathematics and Computer Science, Wuhan Polytechnic University, Wuhan 430023, China
| | - Zhilong Bie
- College of Horticulture and Forestry Sciences, Huazhong Agricultural University, Wuhan 430070, China
- Key Laboratory of Horticultural Plant Biology, Ministry of Education, Wuhan 430070, China
| | - Chengli Peng
- Electronic Information School, Wuhan University, Wuhan 430072, China
| | - Yuan Huang
- College of Horticulture and Forestry Sciences, Huazhong Agricultural University, Wuhan 430070, China
- Key Laboratory of Horticultural Plant Biology, Ministry of Education, Wuhan 430070, China
| | - Shengyong Xu
- College of Engineering, Huazhong Agricultural University, Wuhan 430070, China
- Key Laboratory of Agricultural Equipment for the Middle and Lower Reaches of the Yangtze River, Ministry of Agriculture, Wuhan 430070, China
- Shenzhen Institute of Nutrition and Health, Huazhong Agricultural University, Shenzhen 518000, China
- Shenzhen Branch, Guangdong Laboratory for Lingnan Modern Agriculture, Genome Analysis Laboratory of the Ministry of Agriculture, Agricultural Genomics Institute at Shenzhen, Chinese Academy of Agricultural Sciences, Shenzhen 518000, China
| |
Collapse
|
4
|
Kot T, Bobovský Z, Heczko D, Vysocký A, Virgala I, Prada E. Using Virtual Scanning to Find Optimal Configuration of a 3D Scanner Turntable for Scanning of Mechanical Parts. SENSORS 2021; 21:s21165343. [PMID: 34450785 PMCID: PMC8400365 DOI: 10.3390/s21165343] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 07/05/2021] [Revised: 07/30/2021] [Accepted: 08/03/2021] [Indexed: 11/16/2022]
Abstract
The article describes a method of simulated 3D scanning of triangle meshes based on ray casting which is used to find the optimal configuration of a real 3D scanner turntable. The configuration include the number of scanners, their elevation above the rotary table and the number of required rotation steps. The evaluation is based on the percentage of the part surface covered by the resulting point cloud, which determines the ability to capture all details of the shape. Principal component analysis is used as a secondary criterion to also evaluate the ability to capture the overall general proportions of the model.
Collapse
Affiliation(s)
- Tomáš Kot
- Department of Robotics, Faculty of Mechanical Engineering, VSB-Technical University of Ostrava, 17. Listopadu 2172/15, 708 00 Ostrava-Poruba, Czech Republic; (Z.B.); (D.H.); (A.V.)
- Correspondence:
| | - Zdenko Bobovský
- Department of Robotics, Faculty of Mechanical Engineering, VSB-Technical University of Ostrava, 17. Listopadu 2172/15, 708 00 Ostrava-Poruba, Czech Republic; (Z.B.); (D.H.); (A.V.)
| | - Dominik Heczko
- Department of Robotics, Faculty of Mechanical Engineering, VSB-Technical University of Ostrava, 17. Listopadu 2172/15, 708 00 Ostrava-Poruba, Czech Republic; (Z.B.); (D.H.); (A.V.)
| | - Aleš Vysocký
- Department of Robotics, Faculty of Mechanical Engineering, VSB-Technical University of Ostrava, 17. Listopadu 2172/15, 708 00 Ostrava-Poruba, Czech Republic; (Z.B.); (D.H.); (A.V.)
| | - Ivan Virgala
- Faculty of Mechanical Engineering, Technical University of Košice, 04200 Košice, Slovakia; (I.V.); (E.P.)
| | - Erik Prada
- Faculty of Mechanical Engineering, Technical University of Košice, 04200 Košice, Slovakia; (I.V.); (E.P.)
| |
Collapse
|
5
|
Teng X, Zhou G, Wu Y, Huang C, Dong W, Xu S. Three-Dimensional Reconstruction Method of Rapeseed Plants in the Whole Growth Period Using RGB-D Camera. SENSORS 2021; 21:s21144628. [PMID: 34300368 PMCID: PMC8309581 DOI: 10.3390/s21144628] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 05/19/2021] [Revised: 06/21/2021] [Accepted: 06/29/2021] [Indexed: 11/16/2022]
Abstract
The three-dimensional reconstruction method using RGB-D camera has a good balance in hardware cost and point cloud quality. However, due to the limitation of inherent structure and imaging principle, the acquired point cloud has problems such as a lot of noise and difficult registration. This paper proposes a 3D reconstruction method using Azure Kinect to solve these inherent problems. Shoot color images, depth images and near-infrared images of the target from six perspectives by Azure Kinect sensor with black background. Multiply the binarization result of the 8-bit infrared image with the RGB-D image alignment result provided by Microsoft corporation, which can remove ghosting and most of the background noise. A neighborhood extreme filtering method is proposed to filter out the abrupt points in the depth image, by which the floating noise point and most of the outlier noise will be removed before generating the point cloud, and then using the pass-through filter eliminate rest of the outlier noise. An improved method based on the classic iterative closest point (ICP) algorithm is presented to merge multiple-views point clouds. By continuously reducing both the size of the down-sampling grid and the distance threshold between the corresponding points, the point clouds of each view are continuously registered three times, until get the integral color point cloud. Many experiments on rapeseed plants show that the success rate of cloud registration is 92.5% and the point cloud accuracy obtained by this method is 0.789 mm, the time consuming of a integral scanning is 302 s, and with a good color restoration. Compared with a laser scanner, the proposed method has considerable reconstruction accuracy and a significantly ahead of the reconstruction speed, but the hardware cost is much lower when building a automatic scanning system. This research shows a low-cost, high-precision 3D reconstruction technology, which has the potential to be widely used for non-destructive measurement of rapeseed and other crops phenotype.
Collapse
Affiliation(s)
- Xiaowen Teng
- College of Engineering, Huazhong Agricultural University, Wuhan 430070, China; (X.T.); (Y.W.); (C.H.); (W.D.)
- Key Laboratory of Agricultural Equipment for the Middle and Lower Reaches of the Yangtze River, Ministry of Agriculture, Wuhan 430070, China
| | - Guangsheng Zhou
- College of Plant Science & Technology, Huazhong Agricultural University, Wuhan 430070, China;
| | - Yuxuan Wu
- College of Engineering, Huazhong Agricultural University, Wuhan 430070, China; (X.T.); (Y.W.); (C.H.); (W.D.)
- Key Laboratory of Agricultural Equipment for the Middle and Lower Reaches of the Yangtze River, Ministry of Agriculture, Wuhan 430070, China
| | - Chenglong Huang
- College of Engineering, Huazhong Agricultural University, Wuhan 430070, China; (X.T.); (Y.W.); (C.H.); (W.D.)
- Key Laboratory of Agricultural Equipment for the Middle and Lower Reaches of the Yangtze River, Ministry of Agriculture, Wuhan 430070, China
| | - Wanjing Dong
- College of Engineering, Huazhong Agricultural University, Wuhan 430070, China; (X.T.); (Y.W.); (C.H.); (W.D.)
- Key Laboratory of Agricultural Equipment for the Middle and Lower Reaches of the Yangtze River, Ministry of Agriculture, Wuhan 430070, China
| | - Shengyong Xu
- College of Engineering, Huazhong Agricultural University, Wuhan 430070, China; (X.T.); (Y.W.); (C.H.); (W.D.)
- Key Laboratory of Agricultural Equipment for the Middle and Lower Reaches of the Yangtze River, Ministry of Agriculture, Wuhan 430070, China
- Correspondence: ; Tel.: +86-134-7629-3548
| |
Collapse
|