Improved Field-Based Soybean Seed Counting and Localization with Feature Level Considered

Robust soybean seed yield estimation using high-throughput ground robot videos

We present a novel method for soybean [Glycine max (L.) Merr.] yield estimation leveraging high-throughput seed counting via computer vision and deep learning techniques. Traditional methods for collecting yield data are labor-intensive, costly, and prone to equipment failures at critical data collection times and require transportation of equipment across field sites. Computer vision, the field of teaching computers to interpret visual data, allows us to extract detailed yield information directly from images. By treating it as a computer vision task, we report a more efficient alternative, employing a ground robot equipped with fisheye cameras to capture comprehensive videos of soybean plots from which images are extracted in a variety of development programs. These images are processed through the P2PNet-Yield model, a deep learning framework, where we combined a feature extraction module (the backbone of the P2PNet-Soy) and a yield regression module to estimate seed yields of soybean plots. Our results are built on 2 years of yield testing plot data-8,500 plots in 2021 and 650 plots in 2023. With these datasets, our approach incorporates several innovations to further improve the accuracy and generalizability of the seed counting and yield estimation architecture, such as the fisheye image correction and data augmentation with random sensor effects. The P2PNet-Yield model achieved a genotype ranking accuracy score of up to 83%. It demonstrates up to a 32% reduction in time to collect yield data as well as costs associated with traditional yield estimation, offering a scalable solution for breeding programs and agricultural productivity enhancement.

Adaptive spatial-channel feature fusion and self-calibrated convolution for early maize seedlings counting in UAV images

Accurate counting of crop plants is essential for agricultural science, particularly for yield forecasting, field management, and experimental studies. Traditional methods are labor-intensive and prone to errors. Unmanned Aerial Vehicle (UAV) technology offers a promising alternative; however, varying UAV altitudes can impact image quality, leading to blurred features and reduced accuracy in early maize seedling counts. To address these challenges, we developed RC-Dino, a deep learning methodology based on DINO, specifically designed to enhance the precision of seedling counts from UAV-acquired images. RC-Dino introduces two innovative components: a novel self-calibrating convolutional layer named RSCconv and an adaptive spatial feature fusion module called ASCFF. The RSCconv layer improves the representation of early maize seedlings compared to non-seedling elements within feature maps by calibrating spatial domain features. The ASCFF module enhances the discriminability of early maize seedlings by adaptively fusing feature maps extracted from different layers of the backbone network. Additionally, transfer learning was employed to integrate pre-trained weights with RSCconv, facilitating faster convergence and improved accuracy. The efficacy of our approach was validated using the Early Maize Seedlings Dataset (EMSD), comprising 1,233 annotated images of early maize seedlings, totaling 83,404 individual annotations. Testing on this dataset demonstrated that RC-Dino outperformed existing models, including DINO, Faster R-CNN, RetinaNet, YOLOX, and Deformable DETR. Specifically, RC-Dino achieved improvements of 16.29% in Average Precision (AP) and 8.19% in Recall compared to the DINO model. Our method also exhibited superior coefficient of determination (R²) values across different datasets for seedling counting. By integrating RSCconv and ASCFF into other detection frameworks such as Faster R-CNN, RetinaNet, and Deformable DETR, we observed enhanced detection and counting accuracy, further validating the effectiveness of our proposed method. These advancements make RC-Dino particularly suitable for accurate early maize seedling counting in the field. The source code for RSCconv and ASCFF is publicly available at https://github.com/collapser-AI/RC-Dino, promoting further research and practical applications.

Multi-Scale Attention Network for Vertical Seed Distribution in Soybean Breeding Fields

The increase in the global population is leading to a doubling of the demand for protein. Soybean (Glycine max), a key contributor to global plant-based protein supplies, requires ongoing yield enhancements to keep pace with increasing demand. Precise, on-plant seed counting and localization may catalyze breeding selection of shoot architectures and seed localization patterns related to superior performance in high planting density and contribute to increased yield. Traditional manual counting and localization methods are labor-intensive and prone to error, necessitating more efficient approaches for yield prediction and seed distribution analysis. To solve this, we propose MSANet: a novel deep learning framework tailored for counting and localization of soybean seeds on mature field-grown soy plants. A multi-scale attention map mechanism was applied to maximize model performance in seed counting and localization in soybean breeding fields. We compared our model with a previous state-of-the-art model using the benchmark dataset and an enlarged dataset, including various soybean genotypes. Our model outperforms previous state-of-the-art methods on all datasets across various soybean genotypes on both counting and localization tasks. Furthermore, our model also performed well on in-canopy 360° video, dramatically increasing data collection efficiency. We also propose a technique that enables previously inaccessible insights into the phenotypic and genetic diversity of single plant vertical seed distribution, which may accelerate the breeding process. To accelerate further research in this domain, we have made our dataset and software publicly available: https://github.com/UTokyo-FieldPhenomics-Lab/MSANet.

DEKR-SPrior: An Efficient Bottom-Up Keypoint Detection Model for Accurate Pod Phenotyping in Soybean

The pod and seed counts are important yield-related traits in soybean. High-precision soybean breeders face the major challenge of accurately phenotyping the number of pods and seeds in a high-throughput manner. Recent advances in artificial intelligence, especially deep learning (DL) models, have provided new avenues for high-throughput phenotyping of crop traits with increased precision. However, the available DL models are less effective for phenotyping pods that are densely packed and overlap in in situ soybean plants; thus, accurate phenotyping of the number of pods and seeds in soybean plant is an important challenge. To address this challenge, the present study proposed a bottom-up model, DEKR-SPrior (disentangled keypoint regression with structural prior), for in situ soybean pod phenotyping, which considers soybean pods and seeds analogous to human people and joints, respectively. In particular, we designed a novel structural prior (SPrior) module that utilizes cosine similarity to improve feature discrimination, which is important for differentiating closely located seeds from highly similar seeds. To further enhance the accuracy of pod location, we cropped full-sized images into smaller and high-resolution subimages for analysis. The results on our image datasets revealed that DEKR-SPrior outperformed multiple bottom-up models, viz., Lightweight-OpenPose, OpenPose, HigherHRNet, and DEKR, reducing the mean absolute error from 25.81 (in the original DEKR) to 21.11 (in the DEKR-SPrior) in pod phenotyping. This paper demonstrated the great potential of DEKR-SPrior for plant phenotyping, and we hope that DEKR-SPrior will help future plant phenotyping.

PlantSR: Super-Resolution Improves Object Detection in Plant Images

Recent advancements in computer vision, especially deep learning models, have shown considerable promise in tasks related to plant image object detection. However, the efficiency of these deep learning models heavily relies on input image quality, with low-resolution images significantly hindering model performance. Therefore, reconstructing high-quality images through specific techniques will help extract features from plant images, thus improving model performance. In this study, we explored the value of super-resolution technology for improving object detection model performance on plant images. Firstly, we built a comprehensive dataset comprising 1030 high-resolution plant images, named the PlantSR dataset. Subsequently, we developed a super-resolution model using the PlantSR dataset and benchmarked it against several state-of-the-art models designed for general image super-resolution tasks. Our proposed model demonstrated superior performance on the PlantSR dataset, indicating its efficacy in enhancing the super-resolution of plant images. Furthermore, we explored the effect of super-resolution on two specific object detection tasks: apple counting and soybean seed counting. By incorporating super-resolution as a pre-processing step, we observed a significant reduction in mean absolute error. Specifically, with the YOLOv7 model employed for apple counting, the mean absolute error decreased from 13.085 to 5.71. Similarly, with the P2PNet-Soy model utilized for soybean seed counting, the mean absolute error decreased from 19.159 to 15.085. These findings underscore the substantial potential of super-resolution technology in improving the performance of object detection models for accurately detecting and counting specific plants from images. The source codes and associated datasets related to this study are available at Github.

Accurate and fast implementation of soybean pod counting and localization from high-resolution image

Introduction

Soybean pod count is one of the crucial indicators of soybean yield. Nevertheless, due to the challenges associated with counting pods, such as crowded and uneven pod distribution, existing pod counting models prioritize accuracy over efficiency, which does not meet the requirements for lightweight and real-time tasks.

Methods

To address this goal, we have designed a deep convolutional network called PodNet. It employs a lightweight encoder and an efficient decoder that effectively decodes both shallow and deep information, alleviating the indirect interactions caused by information loss and degradation between non-adjacent levels.

Results

We utilized a high-resolution dataset of soybean pods from field harvesting to evaluate the model's generalization ability. Through experimental comparisons between manual counting and model yield estimation, we confirmed the effectiveness of the PodNet model. The experimental results indicate that PodNet achieves an R2 of 0.95 for the prediction of soybean pod quantities compared to ground truth, with only 2.48M parameters, which is an order of magnitude lower than the current SOTA model YOLO POD, and the FPS is much higher than YOLO POD.

Discussion

Compared to advanced computer vision methods, PodNet significantly enhances efficiency with almost no sacrifice in accuracy. Its lightweight architecture and high FPS make it suitable for real-time applications, providing a new solution for counting and locating dense objects.

Point clouds segmentation of rapeseed siliques based on sparse-dense point clouds mapping

In this study, we propose a high-throughput and low-cost automatic detection method based on deep learning to replace the inefficient manual counting of rapeseed siliques. First, a video is captured with a smartphone around the rapeseed plants in the silique stage. Feature point detection and matching based on SIFT operators are applied to the extracted video frames, and sparse point clouds are recovered using epipolar geometry and triangulation principles. The depth map is obtained by calculating the disparity of the matched images, and the dense point cloud is fused. The plant model of the whole rapeseed plant in the silique stage is reconstructed based on the structure-from-motion (SfM) algorithm, and the background is removed by using the passthrough filter. The downsampled 3D point cloud data is processed by the DGCNN network, and the point cloud is divided into two categories: sparse rapeseed canopy siliques and rapeseed stems. The sparse canopy siliques are then segmented from the original whole rapeseed siliques point cloud using the sparse-dense point cloud mapping method, which can effectively save running time and improve efficiency. Finally, Euclidean clustering segmentation is performed on the rapeseed canopy siliques, and the RANSAC algorithm is used to perform line segmentation on the connected siliques after clustering, obtaining the three-dimensional spatial position of each silique and counting the number of siliques. The proposed method was applied to identify 1457 siliques from 12 rapeseed plants, and the experimental results showed a recognition accuracy greater than 97.80%. The proposed method achieved good results in rapeseed silique recognition and provided a useful example for the application of deep learning networks in dense 3D point cloud segmentation.