Tomato Recognition for Harvesting Robots Considering Overlapping Leaves and Stems

In recent years, the declining and aging population of farmers has become a serious problem. Smart agriculture has been promoted to solve these problems. It is a type of agriculture that utilizes robotics, and information and communication technology to promote labor saving, precision, and realization of high-quality production. In this research, we focused on robots that can harvest tomatoes. Tomatoes are delicate vegetables with a thin skin and a relatively large yield. During automatic harvesting of tomatoes, to ensure the operation of the harvesting arm, an input by image processing is crucial to determine the color of the tomatoes at the time of harvesting. Research on robot image processing technology is indispensable for accurate operation of the arm. In an environment where tomatoes are harvested, obstacles such as leaves, stems, and unripe tomatoes should be taken into consideration. Therefore, in this research, we propose a method of image processing to provide an appropriate route for the arm to ensure easy harvesting, considering the surrounding obstacles.

Tomato Growth State Map for the Automation of Monitoring and Harvesting

To realize smart agriculture, we engaged in its systematization, from monitoring to harvesting tomato fruits using robots. In this paper, we explain a method of generating a map of the tomato growth states to monitor the various stages of tomato fruits and decide a harvesting strategy for the robots. The tomato growth state map visualizes the relationship between the maturity stage, harvest time, and yield. We propose a generation method of the tomato growth state map, a recognition method of tomato fruits, and an estimation method of the growth states (maturity stages and harvest times). For tomato fruit recognition, we demonstrate that a simple machine learning method using a limited learning dataset and the optical properties of tomato fruits on infrared images exceeds more complex convolutional neural network, although the results depend on how the training dataset is created. For the estimation of the growth states, we conducted a survey of experienced farmers to quantify the maturity stages into six classifications and harvest times into three terms. The growth states were estimated based on the survey results. To verify the tomato growth state map, we conducted experiments in an actual tomato greenhouse and herein report the results.

Pallet Handling System with an Autonomous Forklift for Outdoor Fields

In Japan, the aging and depopulation of its workforce are issues. Therefore, the development of autonomous agricultural robots is required for saving manpower and labor. In this paper, we described an autonomous pallet handling system for forklift, which can automatically unload and convey pallets for harvesting vegetables outdoors. Because of inserting the forks into a narrow pallet hole, accurate pallet posture estimation and accurate control of a forklift and the forks are required. The system can detect the pallet by deep learning based object detection from an image. Based on the results of object detection and measurement by horizontal 3D light detection and ranging (LiDAR), the system accurately estimates a distance as well as horizontal and vertical deviation between the forklift and the pallet in the outside field. The forklift is controlled by sliding mode control (SMC) which is robust to disturbances. Furthermore, the vertical LiDAR scans the pallet for precisely adjusting the height of the fork. The system requires the environment with no or little preparation for the automation process. We confirmed the effectiveness of the system through an experiment. The experiment is assumed that the forklift unloads the pallet from the vehicle as the real task of agriculture. The experimental results indicated the suitability of the system in real agricultural environments.

Cutting Point Detection Using a Robot with Point Clouds for Tomato Harvesting

This paper proposes a method to detect cutting points on tomato peduncles using a harvesting robot. The main objective of this study was to develop automated harvesting robots. The harvesting robot was equipped with an RGB-D (Red, Blue, Green, and Depth) camera to detect peduncles and an end effector to harvest tomatoes. Robots must be able to detect where to cut crops during harvesting. The proposed method was used to detect the cutting points on peduncles using a point cloud captured by the RGB-D camera. Our robot was used to identify the cutting points on target tomato peduncles at an actual farm to demonstrate the effectiveness of our approach experimentally. Using the proposed method, the harvesting robot could detect the cutting points on tomatoes.