AmodalAppleSize_RGB-D dataset: RGB-D images of apple trees annotated with modal and amodal segmentation masks for fruit detection, visibility and size estimation

The present dataset comprises a collection of RGB-D apple tree images that can be used to train and test computer vision-based fruit detection and sizing methods. This dataset encompasses two distinct sets of data obtained from a Fuji and an Elstar apple orchards. The Fuji apple orchard sub-set consists of 3925 RGB-D images containing a total of 15,335 apples annotated with both modal and amodal apple segmentation masks. Modal masks denote the visible portions of the apples, whereas amodal masks encompass both visible and occluded apple regions. Notably, this dataset is the first public resource to incorporate on-tree fruit amodal masks. This pioneering inclusion addresses a critical gap in existing datasets, enabling the development of robust automatic fruit sizing methods and accurate fruit visibility estimation, particularly in the presence of partial occlusions. Besides the fruit segmentation masks, the dataset also includes the fruit size (calliper) ground truth for each annotated apple. The second sub-set comprises 2731 RGB-D images capturing five Elstar apple trees at four distinct growth stages. This sub-set includes mean diameter information for each tree at every growth stage and serves as a valuable resource for evaluating fruit sizing methods trained with the first sub-set. The present data was employed in the research paper titled "Looking behind occlusions: a study on amodal segmentation for robust on-tree apple fruit size estimation" [1].

Multi-omics assists genomic prediction of maize yield with machine learning approaches

With the improvement of high-throughput technologies in recent years, large multi-dimensional plant omics data have been produced, and big-data-driven yield prediction research has received increasing attention. Machine learning offers promising computational and analytical solutions to interpret the biological meaning of large amounts of data in crops. In this study, we utilized multi-omics datasets from 156 maize recombinant inbred lines, containing 2496 single nucleotide polymorphisms (SNPs), 46 image traits (i-traits) from 16 developmental stages obtained through an automatic phenotyping platform, and 133 primary metabolites. Based on benchmark tests with different types of prediction models, some machine learning methods, such as Partial Least Squares (PLS), Random Forest (RF), and Gaussian process with Radial basis function kernel (GaussprRadial), achieved better prediction for maize yield, albeit slight difference for method preferences among i-traits, genomic, and metabolic data. We found that better yield prediction may be caused by various capabilities in ranking and filtering data features, which is found to be linked with biological meaning such as photosynthesis-related or kernel development-related regulations. Finally, by integrating multiple omics data with the RF machine learning approach, we can further improve the prediction accuracy of grain yield from 0.32 to 0.43. Our research provides new ideas for the application of plant omics data and artificial intelligence approaches to facilitate crop genetic improvements.

Supplementary Information

The online version contains supplementary material available at 10.1007/s11032-024-01454-z.

MetaRF: attention-based random forest for reaction yield prediction with a few trails

Artificial intelligence has deeply revolutionized the field of medicinal chemistry with many impressive applications, but the success of these applications requires a massive amount of training samples with high-quality annotations, which seriously limits the wide usage of data-driven methods. In this paper, we focus on the reaction yield prediction problem, which assists chemists in selecting high-yield reactions in a new chemical space only with a few experimental trials. To attack this challenge, we first put forth MetaRF, an attention-based random forest model specially designed for the few-shot yield prediction, where the attention weight of a random forest is automatically optimized by the meta-learning framework and can be quickly adapted to predict the performance of new reagents while given a few additional samples. To improve the few-shot learning performance, we further introduce a dimension-reduction based sampling method to determine valuable samples to be experimentally tested and then learned. Our methodology is evaluated on three different datasets and acquires satisfactory performance on few-shot prediction. In high-throughput experimentation (HTE) datasets, the average yield of our methodology's top 10 high-yield reactions is relatively close to the results of ideal yield selection.

Identifying and mapping individual medicinal plant Lamiophlomis rotata at high elevations by using unmanned aerial vehicles and deep learning

BACKGROUND

The identification and enumeration of medicinal plants at high elevations is an important part of accurate yield calculations. However, the current assessment of medicinal plant reserves continues to rely on field sampling surveys, which are cumbersome and time-consuming. Recently, unmanned aerial vehicle (UAV) remote sensing and deep learning (DL) have provided ultrahigh-resolution imagery and high-accuracy object recognition techniques, respectively, providing an excellent opportunity to improve the current manual surveying of plants. However, accurate segmentation of individual plants from drone images remains a significant challenge due to the large variation in size, geometry, and distribution of medicinal plants.

RESULTS

In this study, we proposed a new pipeline for wild medicinal plant detection and yield assessment based on UAV and DL that was specifically designed for detecting wild medicinal plants in an orthomosaic. We used a drone to collect panoramic images of Lamioplomis rotata Kudo (LR) in high-altitude areas. Then, we annotated and cropped these images into equally sized sub-images and used a DL model Mask R-CNN for object detection and segmentation of LR. Finally, on the basis of the segmentation results, we accurately counted the number and yield of LRs. The results showed that the Mask R-CNN model based on the ResNet-101 backbone network was superior to ResNet-50 in all evaluation indicators. The average identification precision of LR by Mask R-CNN based on the ResNet-101 backbone network was 89.34%, while that of ResNet-50 was 88.32%. The cross-validation results showed that the average accuracy of ResNet-101 was 78.73%, while that of ResNet-50 was 71.25%. According to the orthomosaic, the average number and yield of LR in the two sample sites were 19,376 plants and 57.93 kg and 19,129 plants and 73.5 kg respectively.

CONCLUSIONS

The combination of DL and UAV remote sensing reveals significant promise in medicinal plant detection, counting, and yield prediction, which will benefit the monitoring of their populations for conservation assessment and management, among other applications.

Exploring the potential role of environmental and multi-source satellite data in crop yield prediction across Northeast China

Developing an accurate crop yield predicting system at a large scale is of paramount importance for agricultural resource management and global food security. Earth observation provides a unique source of information to monitor crops from a diversity of spectral ranges. However, the integrated use of these data and their values in crop yield prediction is still understudied. Here we proposed the combination of environmental data (climate, soil, geography, and topography) with multiple satellite data (optical-based vegetation indices, solar-induced fluorescence (SIF), land surface temperature (LST), and microwave vegetation optical depth (VOD)) into the framework to estimate crop yield for maize, rice, and soybean in northeast China, and their unique value and relative influence on yield prediction was assessed. Two linear regression methods, three machine learning (ML) methods, and one ML ensemble model were adopted to build yield prediction models. Results showed that the individual ML methods outperformed the linear regression methods, the ML ensemble model further improved the single ML models. Moreover, models with more inputs achieved better performance, the combination of satellite data with environmental data, which explained 72%, 69%, and 57% of maize, rice, and soybean yield variability, respectively, demonstrated higher yield prediction performance than individual inputs. While satellite data contributed to crop yield prediction mainly at the early-peak of the growing season, climate data offered extra information mainly at the peak-late season. We also found that the combined use of EVI, LST and SIF has improved the model accuracy compared to the benchmark EVI model. However, the optical-based vegetation indices shared similar information and did not provide much extra information beyond EVI. The within-season yield forecasting showed that crop yields can be satisfactorily forecasted at two to three months prior to harvest. Geography, topography, VOD, EVI, soil hydraulic and nutrient parameters are more important for crop yield prediction.

The potential of dynamic physiological traits in young tomato plants to predict field-yield performance

To address the challenge of predicting tomato yields in the field, we used whole-plant functional phenotyping to evaluate water relations under well-irrigated and drought conditions. The genotypes tested are known to exhibit variability in their yields in wet and dry fields. The examined lines included two lines with recessive mutations that affect carotenoid biosynthesis, zeta z²⁰⁸³ and tangerine t³⁴⁰⁶, both isogenic to the processing tomato variety M82. The two mutant lines were reciprocally grafted onto M82, and multiple physiological characteristics were measured continuously, before, during and after drought treatment in the greenhouse. A comparative analysis of greenhouse and field yields showed that the whole-canopy stomatal conductance (g_sc) in the morning and cumulative transpiration (CT) were strongly correlated with field measurements of total yield (TY: r² = 0.9 and 0.77, respectively) and plant vegetative weight (PW: r² = 0.6 and 0.94, respectively). Furthermore, the minimum CT during drought and the rate of recovery when irrigation was resumed were both found to predict resilience.

Yield prediction of "Thermal-dissolution based carbon enrichment" treatment on biomass wastes through coupled model of artificial neural network and AdaBoost

The "Thermal-dissolution based carbon enrichment" was proven as an efficient and homogenizing treatment method in converting biomass wastes into similar high-quality carbon materials. However, their yields varied significantly with respect to the different experimental parameters employed. It is therefore imperative to establish the correlation between product yield and experimental parameters for material selection and condition optimization. In this study, Adaboost was coupled with an artificial neural network algorithm to precisely describe the abovementioned correlation. The results demonstrated the effectiveness of this model through its outstanding predicting performance for all the products, especially, the coefficient of determination in predicting the yield of Residue was as high as 0.97. Additionally, the coupling effect of temperature and time was observed. This study not only validates a close correlation between selected experimental parameters and product yields, but also provides a quick and reliable way for material selection and condition optimization.

Effects of fallow tillage on winter wheat yield and predictions under different precipitation types

In northern China, precipitation that is primarily concentrated during the fallow period is insufficient for the growth stage, creates a moisture shortage, and leads to low, unstable yields. Yield prediction in the early growth stages significantly informs field management decisions for winter wheat (Triticum aestivum L.). A 10-year field experiment carried out in the Loess Plateau area tested how three tillage practices (deep ploughing (DP), subsoiling (SS), and no tillage (NT)) influenced cultivation and yield across different fallow periods. The experiment used the random forest (RF) algorithm to construct a prediction model of yields and yield components. Our results revealed that tillage during the fallow period was more effective than NT in improving yield in dryland wheat. Under drought condition, DP during the fallow period achieved a higher yield than SS, especially in drought years; DP was 16% higher than SS. RF was deemed fit for yield prediction across different precipitation years. An RF model was developed using meteorological factors for fixed variables and soil water storage after tillage during a fallow period for a control variable. Small error values existed in the prediction yield, spike number, and grains number per spike. Additionally, the relative error of crop yield under fallow tillage (5.24%) was smaller than that of NT (6.49%). The prediction error of relative meteorological yield was minimum and optimal, indicating that the model is suitable to explain the influence of meteorological factors on yield.

Remote sensing techniques and stable isotopes as phenotyping tools to assess wheat yield performance: Effects of growing temperature and vernalization

This study compares distinct phenotypic approaches to assess wheat performance under different growing temperatures and vernalization needs. A set of 38 (winter and facultative) wheat cultivars were planted in Valladolid (Spain) under irrigation and two contrasting planting dates: normal (late autumn), and late (late winter). The late plating trial exhibited a 1.5 °C increase in average crop temperature. Measurements with different remote sensing techniques were performed at heading and grain filling, as well as carbon isotope composition (δ¹³C) and nitrogen content analysis. Multispectral and RGB vegetation indices and canopy temperature related better to grain yield (GY) across the whole set of genotypes in the normal compared with the late planting, with indices (such as the RGB indices Hue, a* and the spectral indices NDVI, EVI and CCI) measured at grain filling performing the best. Aerially assessed remote sensing indices only performed better than ground-acquired ones at heading. Nitrogen content and δ¹³C correlated with GY at both planting dates. Correlations within winter and facultative genotypes were much weaker, particularly in the facultative subset. For both planting dates, the best GY prediction models were achieved when combining remote sensing indices with δ¹³C and nitrogen of mature grains. Implications for phenotyping in the context of increasing temperatures are further discussed.

Fuji-SfM dataset: A collection of annotated images and point clouds for Fuji apple detection and location using structure-from-motion photogrammetry

The present dataset contains colour images acquired in a commercial Fuji apple orchard (Malus domestica Borkh. cv. Fuji) to reconstruct the 3D model of 11 trees by using structure-from-motion (SfM) photogrammetry. The data provided in this article is related to the research article entitled "Fruit detection and 3D location using instance segmentation neural networks and structure-from-motion photogrammetry" [1]. The Fuji-SfM dataset includes: (1) a set of 288 colour images and the corresponding annotations (apples segmentation masks) for training instance segmentation neural networks such as Mask-RCNN; (2) a set of 582 images defining a motion sequence of the scene which was used to generate the 3D model of 11 Fuji apple trees containing 1455 apples by using SfM; (3) the 3D point cloud of the scanned scene with the corresponding apple positions ground truth in global coordinates. With that, this is the first dataset for fruit detection containing images acquired in a motion sequence to build the 3D model of the scanned trees with SfM and including the corresponding 2D and 3D apple location annotations. This data allows the development, training, and test of fruit detection algorithms either based on RGB images, on coloured point clouds or on the combination of both types of data.

Fuji-SfM dataset: A collection of annotated images and point clouds for Fuji apple detection and location using structure-from-motion photogrammetry

The present dataset contains colour images acquired in a commercial Fuji apple orchard (Malus domestica Borkh. cv. Fuji) to reconstruct the 3D model of 11 trees by using structure-from-motion (SfM) photogrammetry. The data provided in this article is related to the research article entitled "Fruit detection and 3D location using instance segmentation neural networks and structure-from-motion photogrammetry" [1]. The Fuji-SfM dataset includes: (1) a set of 288 colour images and the corresponding annotations (apples segmentation masks) for training instance segmentation neural networks such as Mask-RCNN; (2) a set of 582 images defining a motion sequence of the scene which was used to generate the 3D model of 11 Fuji apple trees containing 1455 apples by using SfM; (3) the 3D point cloud of the scanned scene with the corresponding apple positions ground truth in global coordinates. With that, this is the first dataset for fruit detection containing images acquired in a motion sequence to build the 3D model of the scanned trees with SfM and including the corresponding 2D and 3D apple location annotations. This data allows the development, training, and test of fruit detection algorithms either based on RGB images, on coloured point clouds or on the combination of both types of data.

LFuji-air dataset: Annotated 3D LiDAR point clouds of Fuji apple trees for fruit detection scanned under different forced air flow conditions

This article presents the LFuji-air dataset, which contains LiDAR based point clouds of 11 Fuji apples trees and the corresponding apples location ground truth. A mobile terrestrial laser scanner (MTLS) comprised of a LiDAR sensor and a real-time kinematics global navigation satellite system was used to acquire the data. The MTLS was mounted on an air-assisted sprayer used to generate different air flow conditions. A total of 8 scans per tree were performed, including scans from different LiDAR sensor positions (multi-view approach) and under different air flow conditions. These variability of the scanning conditions allows to use the LFuji-air dataset not only for training and testing new fruit detection algorithms, but also to study the usefulness of the multi-view approach and the application of forced air flow to reduce the number of fruit occlusions. The data provided in this article is related to the research article entitled “Fruit detection, yield prediction and canopy geometric characterization using LiDAR with forced air flow” [1].

Principal variable selection to explain grain yield variation in winter wheat from features extracted from UAV imagery

BACKGROUND

Automated phenotyping technologies are continually advancing the breeding process. However, collecting various secondary traits throughout the growing season and processing massive amounts of data still take great efforts and time. Selecting a minimum number of secondary traits that have the maximum predictive power has the potential to reduce phenotyping efforts. The objective of this study was to select principal features extracted from UAV imagery and critical growth stages that contributed the most in explaining winter wheat grain yield. Five dates of multispectral images and seven dates of RGB images were collected by a UAV system during the spring growing season in 2018. Two classes of features (variables), totaling to 172 variables, were extracted for each plot from the vegetation index and plant height maps, including pixel statistics and dynamic growth rates. A parametric algorithm, LASSO regression (the least angle and shrinkage selection operator), and a non-parametric algorithm, random forest, were applied for variable selection. The regression coefficients estimated by LASSO and the permutation importance scores provided by random forest were used to determine the ten most important variables influencing grain yield from each algorithm.

RESULTS

Both selection algorithms assigned the highest importance score to the variables related with plant height around the grain filling stage. Some vegetation indices related variables were also selected by the algorithms mainly at earlier to mid growth stages and during the senescence. Compared with the yield prediction using all 172 variables derived from measured phenotypes, using the selected variables performed comparable or even better. We also noticed that the prediction accuracy on the adapted NE lines (r = 0.58-0.81) was higher than the other lines (r = 0.21-0.59) included in this study with different genetic backgrounds.

CONCLUSIONS

With the ultra-high resolution plot imagery obtained by the UAS-based phenotyping we are now able to derive more features, such as the variation of plant height or vegetation indices within a plot other than just an averaged number, that are potentially very useful for the breeding purpose. However, too many features or variables can be derived in this way. The promising results from this study suggests that the selected set from those variables can have comparable prediction accuracies on the grain yield prediction than the full set of them but possibly resulting in a better allocation of efforts and resources on phenotypic data collection and processing.

Bayesian inference of environmental effects on seaweed production in Japan via a production-environmental suitability model

BACKGROUND

Both natural and human-induced disturbances are commonly responsible for an overall decrease of the world's seaweed. Along Japan's coastal areas, edible seaweed production has been decreasing for decades. In this study, a production-environmental suitability model to estimate the impacts of environmental factors on seaweed production was developed. The developed model not only estimates human-induced disturbances but also quantifies the impacts of environmental factors responsible for the decline of annual seaweed production. The model estimated the temporal variation in human-induced disturbances and the effects of environmental factors (i.e., rainfall, CO2 concentrations, temperature, typhoons, solar radiation, water nutrient levels, and water quality) on edible seaweeds in Japan from 1985 to 2012.

RESULTS

The environmental suitability for seaweed production in Japan was about 4.6 times greater in 1992 than in 2011, meanwhile as a result of human activities, human-induced disturbances of seaweed increased at a rate of 4.9 times faster during the period of 1998-2012 than the period of 1985-1997. The ratio of decreased production to decreased environmental suitability for seaweed production in Japan increased by 15.2% during the study years, which means that seaweed production has become more sensitive to environmental disturbances, including climatic factors and human activities in 1998-2012.

CONCLUSIONS

The results are novel in demonstrating temporal variations in the level of environmental suitability to seaweed production by using a simple mathematical model. The production-environmental suitability model successfully predicted seaweed production by reflecting the 28-year temporal variation of the observed seaweed production in Japan.

Evaluation of grain yield based on digital images of rice canopy

BACKGROUND

Rice canopy changes are associated with changes in the red light (R), green light (G), and blue light (B) value parameters of digital images. To rapidly diagnose the responses of rice to nitrogen (N) fertilizer application and planting density, a simple model based on digital images was developed for predicting and evaluating rice yield.

RESULTS

N application rate and planting density had significant effects on rice yield. Rice yield first increased and then decreased with increasing of N rates, while the rice yield always increased significantly with increasing planting density. The normalized redness intensity (NRI), normalized greenness intensity (NGI), and normalized blueness intensity (NBI) values of the rice canopy varied among stages; however, they were primarily affected by N fertilizer rates, while planting density had no significant effects. Furthermore, the significant relationships of grain yield with NRI and NBI at the late filling stage could be fitted by quadratic equations, but there was no significant relationship observed between grain yield and NGI across all stages. In addition, a field validation experiment showed that the predicted yield based on the fitted quadratic equations was consistent with the measured yield.

CONCLUSION

The NRI, NGI, and NBI values of rice canopy were mainly affected by N fertilizer rates, while the planting density had no significant effect. The significant relationships between grain yield with NRI and NBI at the late filling stage could be fitted by quadratic equations. Therefore, the canopy NRI and NBI at the late filling stage as measured by digital photography could be used to predict grain yield in southern China.