A Survey of Robotic Harvesting Systems and Enabling Technologies

Gripping Success Metric for Robotic Fruit Harvesting

Recently, computer vision methods have been widely applied to agricultural tasks, such as robotic harvesting. In particular, fruit harvesting robots often rely on object detection or segmentation to identify and localize target fruits. During the model selection process for object detection, the average precision (AP) score typically provides the de facto standard. However, AP is not intuitive for determining which model is most efficient for robotic harvesting. It is based on the intersection-over-union (IoU) of bounding boxes, which reflects only regional overlap. IoU alone cannot reliably predict the success of robotic gripping, as identical IoU scores may yield different results depending on the overlapping shape of the boxes. In this paper, we propose a novel evaluation metric for robotic harvesting. To assess gripping success, our metric uses the center coordinates of bounding boxes and a margin hyperparameter that accounts for the gripper's specifications. We conducted evaluation about popular object detection models on peach and apple datasets. The experimental results showed that the proposed gripping success metric is much more intuitive and helpful in interpreting the performance data.

LESA-Net: Semantic segmentation of multi-type road point clouds in complex agroforestry environment

Point-cloud semantic segmentation is a visual task essential for agricultural robots to comprehend natural agroforestry environments. However, owing to the extremely large amount of point-cloud data in agroforestry environments, learning effective features for semantic segmentation from large-scale point clouds is challenging. Therefore, to address this issue and achieve accurate semantic segmentation of different types of road-surface point clouds in large-scale agroforestry environments, this study proposes a point-cloud semantic segmentation network framework based on double-distance self-attention. First, a point-cloud local feature enhancement module is proposed. This module primarily extends the receptive field and enhances the generalizability of multidimensional features by incorporating reflection intensity information and a spatial feature-encoding block that is enhanced with contextual semantic information. Second, we introduce a dual-distance attention pooling (DDAPS) block based on the self-attention mechanism. This block initially learns the feature representation of the local neighborhood of each point through the self-attention mechanism. Then, it uses the DDAPS block to aggregate more discriminative local neighborhood point features. Finally, extensive experimental results on large-scale point-cloud datasets, SemanticKITTI and RELLIS-3D, demonstrate that our algorithm outperforms similar algorithms in large-scale agroforestry environments.

Application of Machine Vision Techniques in Low-Cost Devices to Improve Efficiency in Precision Farming

In the context of recent technological advancements driven by distributed work and open-source resources, computer vision stands out as an innovative force, transforming how machines interact with and comprehend the visual world around us. This work conceives, designs, implements, and operates a computer vision and artificial intelligence method for object detection with integrated depth estimation. With applications ranging from autonomous fruit-harvesting systems to phenotyping tasks, the proposed Depth Object Detector (DOD) is trained and evaluated using the Microsoft Common Objects in Context dataset and the MinneApple dataset for object and fruit detection, respectively. The DOD is benchmarked against current state-of-the-art models. The results demonstrate the proposed method's efficiency for operation on embedded systems, with a favorable balance between accuracy and speed, making it well suited for real-time applications on edge devices in the context of the Internet of things.

Rapid Determination of Wine Grape Maturity Level from pH, Titratable Acidity, and Sugar Content Using Non-Destructive In Situ Infrared Spectroscopy and Multi-Head Attention Convolutional Neural Networks

In the pursuit of enhancing the wine production process through the utilization of new technologies in viticulture, this study presents a novel approach for the rapid assessment of wine grape maturity levels using non-destructive, in situ infrared spectroscopy and artificial intelligence techniques. Building upon our previous work focused on estimating sugar content (∘Brix) from the visible and near-infrared (VNIR) and short-wave infrared (SWIR) regions, this research expands its scope to encompass pH and titratable acidity, critical parameters determining the grape maturity degree, and in turn, wine quality, offering a more representative estimation pathway. Data were collected from four grape varieties-Chardonnay, Malagouzia, Sauvignon Blanc, and Syrah-during the 2023 harvest and pre-harvest phenological stages in the vineyards of Ktima Gerovassiliou, northern Greece. A comprehensive spectral library was developed, covering the VNIR-SWIR spectrum (350-2500 nm), with measurements performed in situ. Ground truth data for pH, titratable acidity, and sugar content were obtained using conventional laboratory methods: total soluble solids (TSS) (∘Brix) by refractometry, titratable acidity by titration (expressed as mg tartaric acid per liter of must) and pH by a pH meter, analyzed at different maturation stages in the must samples. The maturity indicators were predicted from the point hyperspectral data by employing machine learning algorithms, including Partial Least Squares regression (PLS), Random Forest regression (RF), Support Vector Regression (SVR), and Convolutional Neural Networks (CNN), in conjunction with various pre-processing techniques. Multi-output models were also considered to simultaneously predict all three indicators to exploit their intercorrelations. A novel multi-input-multi-output CNN model was also proposed, incorporating a multi-head attention mechanism and enabling the identification of the spectral regions it focuses on, and thus having a higher interpretability degree. Our results indicate high accuracy in the estimation of sugar content, pH, and titratable acidity, with the best models yielding mean R2 values of 0.84, 0.76, and 0.79, respectively, across all properties. The multi-output models did not improve the prediction results compared to the best single-output models, and the proposed CNN model was on par with the next best model. The interpretability analysis highlighted that the CNN model focused on spectral regions associated with the presence of sugars (i.e., glucose and fructose) and of the carboxylic acid group. This study underscores the potential of portable spectrometry for real-time, non-destructive assessments of wine grape maturity, thereby providing valuable tools for informed decision making in the wine production industry. By integrating pH and titratable acidity into the analysis, our approach offers a holistic view of grape quality, facilitating more comprehensive and efficient viticultural practices.

Human-Robot Interaction in Agriculture: A Systematic Review

In the pursuit of optimizing the efficiency, flexibility, and adaptability of agricultural practices, human-robot interaction (HRI) has emerged in agriculture. Enabled by the ongoing advancement in information and communication technologies, this approach aspires to overcome the challenges originating from the inherent complex agricultural environments. Τhis paper systematically reviews the scholarly literature to capture the current progress and trends in this promising field as well as identify future research directions. It can be inferred that there is a growing interest in this field, which relies on combining perspectives from several disciplines to obtain a holistic understanding. The subject of the selected papers is mainly synergistic target detection, while simulation was the main methodology. Furthermore, melons, grapes, and strawberries were the crops with the highest interest for HRI applications. Finally, collaboration and cooperation were the most preferred interaction modes, with various levels of automation being examined. On all occasions, the synergy of humans and robots demonstrated the best results in terms of system performance, physical workload of workers, and time needed to execute the performed tasks. However, despite the associated progress, there is still a long way to go towards establishing viable, functional, and safe human-robot interactive systems.

A High-Certainty Visual Servo Control Method for a Space Manipulator with Flexible Joints

This paper introduces a novel high-certainty visual servo algorithm for a space manipulator with flexible joints, which consists of a kinematic motion planner and a Lyapunov dynamics model reference adaptive controller. To enhance kinematic certainty, a three-stage motion planner is proposed in Cartesian space to control the intermediate states and minimize the relative position error between the manipulator and the target. Moreover, a planner in joint space based on the fast gradient descent algorithm is proposed to optimize the joint's deviation from the centrality. To improve dynamic certainty, an adaptive control algorithm based on Lyapunov stability analysis is used to enhance the system's anti-disturbance capability. As to the basic PBVS (position-based visual servo methods) algorithm, the proposed method aims to increase the certainty of the intermediate states to avoid collision. A physical experiment is designed to validate the effectiveness of the algorithm. The experiment shows that the visual servo motion state in Cartesian space is basically consistent with the planned three-stage motion state, the average joint deviation index from the centrality is less than 40%, and the motion trajectory consistency exceeds 90% under different inertial load disturbances. Overall, this method reduces the risk of collision by enhancing the certainty of the basic PBVS algorithm.