A Direct Comparison of Remote Sensing Approaches for High-Throughput Phenotyping in Plant Breeding

Simulation of Automatically Annotated Visible and Multi-/Hyperspectral Images Using the Helios 3D Plant and Radiative Transfer Modeling Framework

Deep learning and multimodal remote and proximal sensing are widely used for analyzing plant and crop traits, but many of these deep learning models are supervised and necessitate reference datasets with image annotations. Acquiring these datasets often demands experiments that are both labor-intensive and time-consuming. Furthermore, extracting traits from remote sensing data beyond simple geometric features remains a challenge. To address these challenges, we proposed a radiative transfer modeling framework based on the Helios 3-dimensional (3D) plant modeling software designed for plant remote and proximal sensing image simulation. The framework has the capability to simulate RGB, multi-/hyperspectral, thermal, and depth cameras, and produce associated plant images with fully resolved reference labels such as plant physical traits, leaf chemical concentrations, and leaf physiological traits. Helios offers a simulated environment that enables generation of 3D geometric models of plants and soil with random variation, and specification or simulation of their properties and function. This approach differs from traditional computer graphics rendering by explicitly modeling radiation transfer physics, which provides a critical link to underlying plant biophysical processes. Results indicate that the framework is capable of generating high-quality, labeled synthetic plant images under given lighting scenarios, which can lessen or remove the need for manually collected and annotated data. Two example applications are presented that demonstrate the feasibility of using the model to enable unsupervised learning by training deep learning models exclusively with simulated images and performing prediction tasks using real images.

Reviewing the essential roles of remote phenotyping, GWAS and explainable AI in practical marker-assisted selection for drought-tolerant winter wheat breeding

Marker-assisted selection (MAS) plays a crucial role in crop breeding improving the speed and precision of conventional breeding programmes by quickly and reliably identifying and selecting plants with desired traits. However, the efficacy of MAS depends on several prerequisites, with precise phenotyping being a key aspect of any plant breeding programme. Recent advancements in high-throughput remote phenotyping, facilitated by unmanned aerial vehicles coupled to machine learning, offer a non-destructive and efficient alternative to traditional, time-consuming, and labour-intensive methods. Furthermore, MAS relies on knowledge of marker-trait associations, commonly obtained through genome-wide association studies (GWAS), to understand complex traits such as drought tolerance, including yield components and phenology. However, GWAS has limitations that artificial intelligence (AI) has been shown to partially overcome. Additionally, AI and its explainable variants, which ensure transparency and interpretability, are increasingly being used as recognised problem-solving tools throughout the breeding process. Given these rapid technological advancements, this review provides an overview of state-of-the-art methods and processes underlying each MAS, from phenotyping, genotyping and association analyses to the integration of explainable AI along the entire workflow. In this context, we specifically address the challenges and importance of breeding winter wheat for greater drought tolerance with stable yields, as regional droughts during critical developmental stages pose a threat to winter wheat production. Finally, we explore the transition from scientific progress to practical implementation and discuss ways to bridge the gap between cutting-edge developments and breeders, expediting MAS-based winter wheat breeding for drought tolerance.

Prediction of biomass accumulation and tolerance of wheat seedlings to drought and elevated temperatures using hyperspectral imaging

Early prediction of important agricultural traits in wheat opens up broad prospects for the development of approaches to accelerate the selection of genotypes for further breeding trials. This study is devoted to the search for predictors of biomass accumulation and tolerance of wheat to abiotic stressors. Hyperspectral (HS) and chlorophyll fluorescence (ChlF) parameters were analyzed as predictors under laboratory conditions. The predictive ability of reflectance and normalized difference indices (NDIs), as well as their relationship with parameters of photosynthetic activity, which is a key process influencing organic matter production and crop yields, were analyzed. HS parameters calculated using the wavelengths in Red (R) band and the spectral range next to the red edge (FR-NIR) were found to be correlated with biomass accumulation. The same ranges showed potential for predicting wheat tolerance to elevated temperatures. The relationship of HS predictors with biomass accumulation and heat tolerance were of opposite sign. A number of ChlF parameters also showed statistically significant correlation with biomass accumulation and heat tolerance. A correlation between HS and ChlF parameters, that demonstrated potential for predicting biomass accumulation and tolerance, has been shown. No predictors of drought tolerance were found among the HS and ChlF parameters analyzed.

Assessment of genetic variation among wheat genotypes for drought tolerance utilizing microsatellite markers and morpho-physiological characteristics

Drought is a major abiotic stress that severely limits sustainable wheat (Triticum aestivum L.) productivity via morphological and physio-biochemical alterations of cellular processes. The complex nature and polygenic control of drought tolerance traits make breeding tolerant genotypes quite challenging. However, naturally occurring variabilities among wheat germplasm resources could potentially help combating drought. The present study was conducted to assess the drought tolerance of 18 Bangladeshi hexaploid wheat genotypes, focusing on the identification of potent sources of diversity by combining microsatellite markers, also known as single sequence repeat markers, and morpho-physiological characteristics that might help accelerating wheat crop improvement programs. Initially, the genotypes were evaluated using 25 microsatellite markers followed by an on-field evaluation of 7 morphological traits (plant height, spike number, spike length, grains per spike, 1000-grain weight, grain yield, biological yield) and 6 physiological traits (SPAD value, membrane stability index, leaf relative water content, proline content, canopy temperature depression, and leaf K+ ion content). The field-trial was conducted in a factorial fashion of 18 wheat genotypes and two water regimes (control and drought) following a split-plot randomized complete block design. Regardless of genotype, drought was significantly damaging for all the tested traits; however, substantial variability in drought stress tolerance was evident among the genotypes. Spike length, 1000-grain weight, SPAD value, leaf relative water content, canopy temperature depression, proline content, and potassium (K+) ion content were the most representative of drought-induced growth and yield impairments and also correlated well with the contrasting ability of genotypic tolerance. Microsatellite markers amplified 244 alleles exhibiting 79% genetic diversity. Out of 25 markers, 23 was highly polymorphic showing 77% average polymorphism. Morpho-physiological trait-based hierarchical clustering and microsatellite marker-based neighbor-jointing clustering both revealed three genotypic clusters with 71% co-linearity between them. In both cases, the genotypes Kanchan, BAW-1147, BINA Gom 1, BARI Gom 22, BARI Gom 26, and BARI Gom 33 were found to be comparatively more tolerant than the other tested genotypes, showing potential for cultivation in water-deficit environments. The findings of this study would contribute to the present understanding of drought tolerance in wheat and would provide a basis for future genotype selection for drought-tolerant wheat breeding programs.

Data-driven approaches to improve water-use efficiency and drought resistance in crop plants

With the increasing population, there lies a pressing demand for food, feed and fibre, while the changing climatic conditions pose severe challenges for agricultural production worldwide. Water is the lifeline for crop production; thus, enhancing crop water-use efficiency (WUE) and improving drought resistance in crop varieties are crucial for overcoming these challenges. Genetically-driven improvements in yield, WUE and drought tolerance traits can buffer the worst effects of climate change on crop production in dry areas. While traditional crop breeding approaches have delivered impressive results in increasing yield, the methods remain time-consuming and are often limited by the existing allelic variation present in the germplasm. Significant advances in breeding and high-throughput omics technologies in parallel with smart agriculture practices have created avenues to dramatically speed up the process of trait improvement by leveraging the vast volumes of genomic and phenotypic data. For example, individual genome and pan-genome assemblies, along with transcriptomic, metabolomic and proteomic data from germplasm collections, characterised at phenotypic levels, could be utilised to identify marker-trait associations and superior haplotypes for crop genetic improvement. In addition, these omics approaches enable the identification of genes involved in pathways leading to the expression of a trait, thereby providing an understanding of the genetic, physiological and biochemical basis of trait variation. These data-driven gene discoveries and validation approaches are essential for crop improvement pipelines, including genomic breeding, speed breeding and gene editing. Herein, we provide an overview of prospects presented using big data-driven approaches (including artificial intelligence and machine learning) to harness new genetic gains for breeding programs and develop drought-tolerant crop varieties with favourable WUE and high-yield potential traits.

The potential of UAV and very high-resolution satellite imagery for yellow and stem rust detection and phenotyping in Ethiopia

Very high (spatial and temporal) resolution satellite (VHRS) and high-resolution unmanned aerial vehicle (UAV) imagery provides the opportunity to develop new crop disease detection methods at early growth stages with utility for early warning systems. The capability of multispectral UAV, SkySat and Pleiades imagery as a high throughput phenotyping (HTP) and rapid disease detection tool for wheat rusts is assessed. In a randomized trial with and without fungicide control, six bread wheat varieties with differing rust resistance were monitored using UAV and VHRS. In total, 18 spectral features served as predictors for stem and yellow rust disease progression and associated yield loss. Several spectral features demonstrated strong predictive power for the detection of combined wheat rust diseases and the estimation of varieties' response to disease stress and grain yield. Visible spectral (VIS) bands (Green, Red) were more useful at booting, shifting to VIS-NIR (near-infrared) vegetation indices (e.g., NDVI, RVI) at heading. The top-performing spectral features for disease progression and grain yield were the Red band and UAV-derived RVI and NDVI. Our findings provide valuable insight into the upscaling capability of multispectral sensors for disease detection, demonstrating the possibility of upscaling disease detection from plot to regional scales at early growth stages.

Field phenotyping for African crops: overview and perspectives

Improvements in crop productivity are required to meet the dietary demands of the rapidly-increasing African population. The development of key staple crop cultivars that are high-yielding and resilient to biotic and abiotic stresses is essential. To contribute to this objective, high-throughput plant phenotyping approaches are important enablers for the African plant science community to measure complex quantitative phenotypes and to establish the genetic basis of agriculturally relevant traits. These advances will facilitate the screening of germplasm for optimum performance and adaptation to low-input agriculture and resource-constrained environments. Increasing the capacity to investigate plant function and structure through non-invasive technologies is an effective strategy to aid plant breeding and additionally may contribute to precision agriculture. However, despite the significant global advances in basic knowledge and sensor technology for plant phenotyping, Africa still lags behind in the development and implementation of these systems due to several practical, financial, geographical and political barriers. Currently, field phenotyping is mostly carried out by manual methods that are prone to error, costly, labor-intensive and may come with adverse economic implications. Therefore, improvements in advanced field phenotyping capabilities and appropriate implementation are key factors for success in modern breeding and agricultural monitoring. In this review, we provide an overview of the current state of field phenotyping and the challenges limiting its implementation in some African countries. We suggest that the lack of appropriate field phenotyping infrastructures is impeding the development of improved crop cultivars and will have a detrimental impact on the agricultural sector and on food security. We highlight the prospects for integrating emerging and advanced low-cost phenotyping technologies into breeding protocols and characterizing crop responses to environmental challenges in field experimentation. Finally, we explore strategies for overcoming the barriers and maximizing the full potential of emerging field phenotyping technologies in African agriculture. This review paper will open new windows and provide new perspectives for breeders and the entire plant science community in Africa.

Remote sensing continuity: a comparison of HTP platforms and potential challenges with field applications

In an era of climate change and increased environmental variability, breeders are looking for tools to maintain and increase genetic gain and overall efficiency. In recent years the field of high throughput phenotyping (HTP) has received increased attention as an option to meet this need. There are many platform options in HTP, but ground-based handheld and remote aerial systems are two popular options. While many HTP setups have similar specifications, it is not always clear if data from different systems can be treated interchangeably. In this research, we evaluated two handheld radiometer platforms, Cropscan MSR16R and Spectra Vista Corp (SVC) HR-1024i, as well as a UAS-based system with a Sentera Quad Multispectral Sensor. Each handheld radiometer was used for two years simultaneously with the unoccupied aircraft systems (UAS) in collecting winter wheat breeding trials between 2018-2021. Spectral reflectance indices (SRI) were calculated for each system. SRI heritability and correlation were analyzed in evaluating the platform and SRI usability for breeding applications. Correlations of SRIs were low against UAS SRI and grain yield while using the Cropscan system in 2018 and 2019. Dissimilarly, the SVC system in 2020 and 2021 produced moderate correlations across UAS SRI and grain yield. UAS SRI were consistently more heritable, with broad-sense heritability ranging from 0.58 to 0.80. Data standardization and collection windows are important to consider in ensuring reliable data. Furthermore, practical aspects and best practices for these HTP platforms, relative to applied breeding applications, are highlighted and discussed. The findings of this study can be a framework to build upon when considering the implementation of HTP technology in an applied breeding program.

Identification of new cold tolerant Zoysia grass species using high-resolution RGB and multi-spectral imaging

Zoysia grass (Zoysia spp.) is the most widely used warm-season turf grass in Korea due to its durability and resistance to environmental stresses. To develop new longer-period greenness cultivars, it is essential to screen germplasm which maintains the greenness at a lower temperature. Conventional methods are time-consuming, laborious, and subjective. Therefore, in this study, we demonstrate an objective and efficient method to screen maintaining longer greenness germplasm using RGB and multispectral images. From August to December, time-series data were acquired and we calculated green cover percentage (GCP), Normalized Difference Vegetation Index (NDVI), Normalized Difference Red Edge Index (NDRE), Soil-adjusted Vegetation Index (SAVI), and Enhanced Vegetation Index (EVI) values of germplasm from RGB and multispectral images by applying vegetation indexs. The result showed significant differences in GCP, NDVI, NDRE, SAVI, and EVI among germplasm (p < 0.05). The GCP, which evaluated the quantity of greenness by counting pixels of the green area from RGB images, exhibited maintenance of greenness over 90% for August and September but, sharply decrease from October. The study found significant differences in GCP and NDVI among germplasm. san208 exhibiting over 90% GCP and high NDVI values during 153 days. In addition, we also conducted assessments using various vegetation indexes, namely NDRE, SAVI, and EVI. san208 exhibited NDRE levels exceeding 3% throughout this period. As for SAVI, it initially started at approximately 38% and gradually decreased to around 4% over the course of these days. Furthermore, for the month of August, it recorded approximately 6%, but experienced a decline from about 9% to 1% between September and October. The complementary use of both indicators could be an efficient method for objectively assessing the greenness of turf both quantitatively and qualitatively.

Atmospheric correction of vegetation reflectance with simulation-trained deep learning for ground-based hyperspectral remote sensing

BACKGROUND

Vegetation spectral reflectance obtained with hyperspectral imaging (HSI) offer non-invasive means for the non-destructive study of their physiological status. The light intensity at visible and near-infrared wavelengths (VNIR, 0.4-1.0µm) captured by the sensor are composed of mixtures of spectral components that include the vegetation reflectance, atmospheric attenuation, top-of-atmosphere solar irradiance, and sensor artifacts. Common methods for the extraction of spectral reflectance from the at-sensor spectral radiance offer a trade-off between explicit knowledge of atmospheric conditions and concentrations, computational efficiency, and prediction accuracy, and are generally geared towards nadir pointing platforms. Therefore, a method is needed for the accurate extraction of vegetation reflectance from spectral radiance captured by ground-based remote sensors with a side-facing orientation towards the target, and a lack of knowledge of the atmospheric parameters.

RESULTS

We propose a framework for obtaining the vegetation spectral reflectance from at-sensor spectral radiance, which relies on a time-dependent Encoder-Decoder Convolutional Neural Network trained and tested using simulated spectra generated from radiative transfer modeling. Simulated at-sensor spectral radiance are produced from combining 1440 unique simulated solar angles and atmospheric absorption profiles, and 1000 different spectral reflectance curves of vegetation with various health indicator values, together with sensor artifacts. Creating an ensemble of 10 models, each trained and tested on a separate 10% of the dataset, results in the prediction of the vegetation spectral reflectance with a testing r² of 98.1% (±0.4). This method produces consistently high performance with accuracies >90% for spectra with resolutions as low as 40 channels in VNIR each with 40 nm full width at half maximum (FWHM) and greater, and remains viable with accuracies >80% down to a resolution of 10 channels with 60 nm FWHM. When applied to real sensor obtained spectral radiance data, the predicted spectral reflectance curves showed general agreement and consistency with those corrected by the Compound Ratio method.

CONCLUSIONS

We propose a method that allows for the accurate estimation of the vegetation spectral reflectance from ground-based HSI platforms with sufficient spectral resolution. It is capable of extracting the vegetation spectral reflectance at high accuracy in the absence of knowledge of the exact atmospheric compositions and conditions at time of capture, and the lack of available sensor-measured spectral radiance and their true ground-truth spectral reflectance profiles.

Row selection in remote sensing from four-row plots of maize and sorghum based on repeatability and predictive modeling

Remote sensing enables the rapid assessment of many traits that provide valuable information to plant breeders throughout the growing season to improve genetic gain. These traits are often extracted from remote sensing data on a row segment (rows within a plot) basis enabling the quantitative assessment of any row-wise subset of plants in a plot, rather than a few individual representative plants, as is commonly done in field-based phenotyping. Nevertheless, which rows to include in analysis is still a matter of debate. The objective of this experiment was to evaluate row selection and plot trimming in field trials conducted using four-row plots with remote sensing traits extracted from RGB (red-green-blue), LiDAR (light detection and ranging), and VNIR (visible near infrared) hyperspectral data. Uncrewed aerial vehicle flights were conducted throughout the growing seasons of 2018 to 2021 with data collected on three years of a sorghum experiment and two years of a maize experiment. Traits were extracted from each plot based on all four row segments (RS) (RS1234), inner rows (RS23), outer rows (RS14), and individual rows (RS1, RS2, RS3, and RS4). Plot end trimming of 40 cm was an additional factor tested. Repeatability and predictive modeling of end-season yield were used to evaluate performance of these methodologies. Plot trimming was never shown to result in significantly different outcomes from non-trimmed plots. Significant differences were often observed based on differences in row selection. Plots with more row segments were often favorable for increasing repeatability, and excluding outer rows improved predictive modeling. These results support long-standing principles of experimental design in agronomy and should be considered in breeding programs that incorporate remote sensing.

Satellite imagery for high-throughput phenotyping in breeding plots

Advances in breeding efforts to increase the rate of genetic gains and enhance crop resilience to climate change have been limited by the procedure and costs of phenotyping methods. The recent rapid development of sensors, image-processing technology, and data-analysis has provided opportunities for multiple scales phenotyping methods and systems, including satellite imagery. Among these platforms, satellite imagery may represent one of the ultimate approaches to remotely monitor trials and nurseries planted in multiple locations while standardizing protocols and reducing costs. However, the deployment of satellite-based phenotyping in breeding trials has largely been limited by low spatial resolution of satellite images. The advent of a new generation of high-resolution satellites may finally overcome these limitations. The SkySat constellation started offering multispectral images at a 0.5 m resolution since 2020. In this communication we present a case study on the use of time series SkySat images to estimate NDVI from wheat and maize breeding plots encompassing different sizes and spacing. We evaluated the reliability of the calculated NDVI and tested its capacity to detect seasonal changes and genotypic differences. We discuss the advantages, limitations, and perspectives of this approach for high-throughput phenotyping in breeding programs.

Opportunity and challenges of phenotyping plant salt tolerance

Salinity is a key factor limiting agricultural production worldwide. Recent advances in field phenotyping have enabled the recording of the environmental history and dynamic response of plants by considering both genotype × environment (G×E) interactions and envirotyping. However, only a few studies have focused on plant salt tolerance phenotyping. Therefore, we analyzed the potential opportunities and major challenges in improving plant salt tolerance using advanced field phenotyping technologies. RGB imaging and spectral and thermal sensors are the most useful and important sensing techniques for assessing key morphological and physiological traits of plant salt tolerance. However, field phenotyping faces challenges owing to its practical applications and high costs, limiting its use in early generation breeding and in developing countries.

Exotic alleles contribute to heat tolerance in wheat under field conditions

Global warming poses a major threat to food security and necessitates the development of crop varieties that are resilient to future climatic instability. By evaluating 149 spring wheat lines in the field under yield potential and heat stressed conditions, we demonstrate how strategic integration of exotic material significantly increases yield under heat stress compared to elite lines, with no significant yield penalty under favourable conditions. Genetic analyses reveal three exotic-derived genetic loci underlying this heat tolerance which together increase yield by over 50% and reduce canopy temperature by approximately 2 °C. We identified an Ae. tauschii introgression underlying the most significant of these associations and extracted the introgressed Ae. tauschii genes, revealing candidates for further dissection. Incorporating these exotic alleles into breeding programmes could serve as a pre-emptive strategy to produce high yielding wheat cultivars that are resilient to the effects of future climatic uncertainty.

Remote sensing for field pea yield estimation: A study of multi-scale data fusion approaches in phenomics

Introduction

Remote sensing using unmanned aerial systems (UAS) are prevalent for phenomics and precision agricultural applications. The high-resolution data for these applications can provide useful spectral characteristics of crops associated with performance traits such as seed yield. With the recent availability of high-resolution satellite imagery, there has been growing interest in using this technology for plot-scale remote sensing applications, particularly those related to breeding programs. This study compared the features extracted from high-resolution satellite and UAS multispectral imagery (visible and near-infrared) to predict the seed yield from two diverse plot-scale field pea yield trials (advanced breeding and variety testing) using the random forest model.

Methods

The multi-modal (spectral and textural features) and multi-scale (satellite and UAS) data fusion approaches were evaluated to improve seed yield prediction accuracy across trials and time points. These approaches included both image fusion, such as pan-sharpening of satellite imagery with UAS imagery using intensity-hue-saturation transformation and additive wavelet luminance proportional approaches, and feature fusion, which involved integrating extracted spectral features. In addition, we also compared the image fusion approach to high-definition satellite data with a resolution of 0.15 m/pixel. The effectiveness of each approach was evaluated with data at both individual and combined time points.

Results and discussion

The major findings can be summarized as follows: (1) the inclusion of the texture features did not improve the model performance, (2) the performance of the model using spectral features from satellite imagery at its original resolution can provide similar results as UAS imagery, with variation depending on the field pea yield trial under study and the growth stage, (3) the model performance improved after applying multi-scale, multiple time point feature fusion, (4) the features extracted from the pan-sharpened satellite imagery using intensity-hue-saturation transformation (image fusion) showed better model performance than those with original satellite imagery or high definition imagery, and (5) the green normalized difference vegetation index and transformed triangular vegetation index were identified as key features contributing to high model performance across trials and time points. These findings demonstrate the potential of high-resolution satellite imagery and data fusion approaches for plot-scale phenomics applications.

Hyperspectral remote sensing to assess weed competitiveness in maize farmland ecosystems

Weed competition causes serious economic losses to maize production. Timely and accurate assessment of pressure from competition is crucial for ecological weed management. In this work, we apply hyperspectral remote sensing (HRS) technology to conduct a competitive experiment in Harbin, China, in 2021, with 5-leaf maize as the study target. A weed competition assessment method that combines comprehensive competition indices (CCI) and deep learning is proposed. For the comprehensive competition assessment, the relationship between different weed competitive pressures (Levels 1-5) and changes in the structural and physiological information of maize was analyzed. The accumulative/transient competition indices CCI-A and CCI-T were designed for accurate quantification. The results showed that parameters such as plant height, stalk thickness and nutrient elements of maize decreased with increasing competition level. Parameters, such as stomatal conductance and transpiration rate, showed a fluctuating change of increasing and then decreasing with increasing competition level. Compared with the traditional relative competitive intensity (RCI), the standard deviation of CCI is 0.303 and 0.499. The dispersion effect of CCI is better and more suitable for quantifying the competition response. HRS images combined with 3D-CNN model were then applied to reveal the spectral response to different weed competition pressures (Levels 1-5) and to make early predictions of weed competition. The first-order derivative showed that the spectral reflectance exhibited significant differences at 520-525 nm peak, 570-655 nm trough, and near 700 nm red edge. For hyperspectral spatial-spectral features, the 3D-CNN model is proposed for prediction of competing indices CCI. In addition, the VIP method is used to select the characteristic wavelengths. The 3D-CNN model achieves a prediction accuracy of RMSE = 0.106 and 0.152 using 13 feature bands, which can accurately quantify the subtle changes in competition indices. Overall, this study shows that the combination of CCI and deep learning can provide a multivariate and comprehensive assessment of weed competition pressure.

Capturing crop adaptation to abiotic stress using image-based technologies

Farmers and breeders aim to improve crop responses to abiotic stresses and secure yield under adverse environmental conditions. To achieve this goal and select the most resilient genotypes, plant breeders and researchers rely on phenotyping to quantify crop responses to abiotic stress. Recent advances in imaging technologies allow researchers to collect physiological data non-destructively and throughout time, making it possible to dissect complex plant responses into quantifiable traits. The use of image-based technologies enables the quantification of crop responses to stress in both controlled environmental conditions and field trials. This paper summarizes phenotyping imaging technologies (RGB, multispectral and hyperspectral sensors, among others) that have been used to assess different abiotic stresses including salinity, drought and nitrogen deficiency, while discussing their advantages and drawbacks. We present a detailed review of traits involved in abiotic tolerance, which have been quantified by a range of imaging sensors under high-throughput phenotyping facilities or using unmanned aerial vehicles in the field. We also provide an up-to-date compilation of spectral tolerance indices and discuss the progress and challenges in machine learning, including supervised and unsupervised models as well as deep learning.

Plant Genotype to Phenotype Prediction Using Machine Learning

Genomic prediction tools support crop breeding based on statistical methods, such as the genomic best linear unbiased prediction (GBLUP). However, these tools are not designed to capture non-linear relationships within multi-dimensional datasets, or deal with high dimension datasets such as imagery collected by unmanned aerial vehicles. Machine learning (ML) algorithms have the potential to surpass the prediction accuracy of current tools used for genotype to phenotype prediction, due to their capacity to autonomously extract data features and represent their relationships at multiple levels of abstraction. This review addresses the challenges of applying statistical and machine learning methods for predicting phenotypic traits based on genetic markers, environment data, and imagery for crop breeding. We present the advantages and disadvantages of explainable model structures, discuss the potential of machine learning models for genotype to phenotype prediction in crop breeding, and the challenges, including the scarcity of high-quality datasets, inconsistent metadata annotation and the requirements of ML models.

Hyperspectral Monitoring Driven by Machine Learning Methods for Grassland Above-Ground Biomass

Above-ground biomass (AGB) is a key indicator for studying grassland productivity and evaluating carbon sequestration capacity; it is also a key area of interest in hyperspectral ecological remote sensing. In this study, we use data from a typical alpine meadow in the Qinghai–Tibet Plateau during the main growing season (July–September), compare the results of various feature selection algorithms to extract an optimal subset of spectral variables, and use machine learning methods and data mining techniques to build an AGB prediction model and realize the optimal inversion of above-ground grassland biomass. The results show that the Lasso and RFE_SVM band filtering machine learning models can effectively select the global optimal feature and improve the prediction effect of the model. The analysis also compares the support vector machine (SVM), least squares regression boosting (LSB), and Gaussian process regression (GPR) AGB inversion models; our findings show that the results of the three models are similar, with the GPR machine learning model achieving the best outcomes. In addition, through the analysis of different data combinations, it is found that the accuracy of AGB inversion can be significantly improved by combining the spectral characteristics with the growing season. Finally, by constructing a machine learning interpretable model to analyze the specific role of features, it was found that the same band plays different roles in different records, and the related results can provide a scientific basis for the research of grassland resource monitoring and estimation.

Preharvest phenotypic prediction of grain quality and yield of durum wheat using multispectral imaging

Durum wheat is an important cereal that is widely grown in the Mediterranean basin. In addition to high yield, grain quality traits are of high importance for farmers. The strong influence of climatic conditions makes the improvement of grain quality traits, like protein content, vitreousness, and test weight, a challenging task. Evaluation of quality traits post-harvest is time- and labor-intensive and requires expensive equipment, such as near-infrared spectroscopes or hyperspectral imagers. Predicting not only yield but also important quality traits in the field before harvest is of high value for breeders aiming to optimize resource allocation. Implementation of efficient approaches for trait prediction, such as the use of high-resolution spectral data acquired by a multispectral camera mounted on unmanned aerial vehicles (UAVs), needs to be explored. In this study, we have acquired multispectral image data with an 11-band multispectral camera mounted on a UAV and analyzed the data with machine learning (ML) models to predict grain yield and important quality traits in breeding micro-plots. Combining 11-band multispectral data for 34 cultivars and 16 environments allowed to develop ML models with good prediction capability. Applying the trained models to test sets explained a considerable degree of phenotypic variance with good accuracy showing r squared values of 0.84, 0.69, 0.64, and 0.61 and normalized root mean squared errors of 0.17, 0.07, 0.14, and 0.03 for grain yield, protein content, vitreousness, and test weight, respectively.

Advances in optical phenotyping of cereal crops

Optical sensors and sensing-based phenotyping techniques have become mainstream approaches in high-throughput phenotyping for improving trait selection and genetic gains in crops. We review recent progress and contemporary applications of optical sensing-based phenotyping (OSP) techniques in cereal crops and highlight optical sensing principles for spectral response and sensor specifications. Further, we group phenotypic traits determined by OSP into four categories - morphological, biochemical, physiological, and performance traits - and illustrate appropriate sensors for each extraction. In addition to the current status, we discuss the challenges of OSP and provide possible solutions. We propose that optical sensing-based traits need to be explored further, and that standardization of the language of phenotyping and worldwide collaboration between phenotyping researchers and other fields need to be established.

Entailing the Next-Generation Sequencing and Metabolome for Sustainable Agriculture by Improving Plant Tolerance

Crop production is a serious challenge to provide food for the 10 billion individuals forecasted to live across the globe in 2050. The scientists' emphasize establishing an equilibrium among diversity and quality of crops by enhancing yield to fulfill the increasing demand for food supply sustainably. The exploitation of genetic resources using genomics and metabolomics strategies can help generate resilient plants against stressors in the future. The innovation of the next-generation sequencing (NGS) strategies laid the foundation to unveil various plants' genetic potential and help us to understand the domestication process to unmask the genetic potential among wild-type plants to utilize for crop improvement. Nowadays, NGS is generating massive genomic resources using wild-type and domesticated plants grown under normal and harsh environments to explore the stress regulatory factors and determine the key metabolites. Improved food nutritional value is also the key to eradicating malnutrition problems around the globe, which could be attained by employing the knowledge gained through NGS and metabolomics to achieve suitability in crop yield. Advanced technologies can further enhance our understanding in defining the strategy to obtain a specific phenotype of a crop. Integration among bioinformatic tools and molecular techniques, such as marker-assisted, QTLs mapping, creation of reference genome, de novo genome assembly, pan- and/or super-pan-genomes, etc., will boost breeding programs. The current article provides sequential progress in NGS technologies, a broad application of NGS, enhancement of genetic manipulation resources, and understanding the crop response to stress by producing plant metabolites. The NGS and metabolomics utilization in generating stress-tolerant plants/crops without deteriorating a natural ecosystem is considered a sustainable way to improve agriculture production. This highlighted knowledge also provides useful research that explores the suitable resources for agriculture sustainability.

Comparison of Multi-Methods for Identifying Maize Phenology Using PhenoCams

Accurately identifying the phenology of summer maize is crucial for both cultivar breeding and fertilizer controlling in precision agriculture. In this study, daily RGB images covering the entire growth of summer maize were collected using phenocams at sites in Shangqiu (2018, 2019 and 2020) and Nanpi (2020) in China. Four phenological dates, including six leaves, booting, heading and maturity of summer maize, were pre-defined and extracted from the phenocam-based images. The spectral indices, textural indices and integrated spectral and textural indices were calculated using the improved adaptive feature-weighting method. The double logistic function, harmonic analysis of time series, Savitzky–Golay and spline interpolation were applied to filter these indices and pre-defined phenology was identified and compared with the ground observations. The results show that the DLF achieved the highest accuracy, with the coefficient of determination (R2) and the root-mean-square error (RMSE) being 0.86 and 9.32 days, respectively. The new index performed better than the single usage of spectral and textural indices, of which the R2 and RMSE were 0.92 and 9.38 days, respectively. The phenological extraction using the new index and double logistic function based on the PhenoCam data was effective and convenient, obtaining high accuracy. Therefore, it is recommended the adoption of the new index by integrating the spectral and textural indices for extracting maize phenology using PhenoCam data.

Opportunities and limits of controlled-environment plant phenotyping for climate response traits

Rising temperatures and changing precipitation patterns will affect agricultural production substantially, exposing crops to extended and more intense periods of stress. Therefore, breeding of varieties adapted to the constantly changing conditions is pivotal to enable a quantitatively and qualitatively adequate crop production despite the negative effects of climate change. As it is not yet possible to select for adaptation to future climate scenarios in the field, simulations of future conditions in controlled-environment (CE) phenotyping facilities contribute to the understanding of the plant response to special stress conditions and help breeders to select ideal genotypes which cope with future conditions. CE phenotyping facilities enable the collection of traits that are not easy to measure under field conditions and the assessment of a plant's phenotype under repeatable, clearly defined environmental conditions using automated, non-invasive, high-throughput methods. However, extrapolation and translation of results obtained under controlled environments to field environments is ambiguous. This review outlines the opportunities and challenges of phenotyping approaches under controlled environments complementary to conventional field trials. It gives an overview on general principles and introduces existing phenotyping facilities that take up the challenge of obtaining reliable and robust phenotypic data on climate response traits to support breeding of climate-adapted crops.

Statistical Methods for the Quantitative Genetic Analysis of High-Throughput Phenotyping Data

The advent of plant phenomics, coupled with the wealth of genotypic data generated by next-generation sequencing technologies, provides exciting new resources for investigations into and improvement of complex traits. However, these new technologies also bring new challenges in quantitative genetics, namely, a need for the development of robust frameworks that can accommodate these high-dimensional data. In this chapter, we describe methods for the statistical analysis of high-throughput phenotyping (HTP) data with the goal of enhancing the prediction accuracy of genomic selection (GS). Following the Introduction in Sec. 1, Sec. 2 discusses field-based HTP, including the use of unoccupied aerial vehicles and light detection and ranging, as well as how we can achieve increased genetic gain by utilizing image data derived from HTP. Section 3 considers extending commonly used GS models to integrate HTP data as covariates associated with the principal trait response, such as yield. Particular focus is placed on single-trait, multi-trait, and genotype by environment interaction models. One unique aspect of HTP data is that phenomics platforms often produce large-scale data with high spatial and temporal resolution for capturing dynamic growth, development, and stress responses. Section 4 discusses the utility of a random regression model for performing longitudinal modeling. The chapter concludes with a discussion of some standing issues.

Factors Influencing Temperature Measurements from Miniaturized Thermal Infrared (TIR) Cameras: A Laboratory-Based Approach

The workflow for estimating the temperature in agricultural fields from multiple sensors needs to be optimized upon testing each type of sensor’s actual user performance. In this sense, readily available miniaturized UAV-based thermal infrared (TIR) cameras can be combined with proximal sensors in measuring the surface temperature. Before the two types of cameras can be operationally used in the field, laboratory experiments are needed to fully understand their capabilities and all the influencing factors. We present the measurement results of laboratory experiments of UAV-borne WIRIS 2nd GEN and handheld FLIR E8-XT cameras. For these uncooled sensors, it took 30 to 60 min for the measured signal to stabilize and the sensor temperature drifted continuously. The drifting sensor temperature was strongly correlated to the measured signal. Specifically for WIRIS, the automated non-uniformity correction (NUC) contributed to extra uncertainty in measurements. Another problem was the temperature measurement dependency on various ambient environmental parameters. An increase in the measuring distance resulted in the underestimation of surface temperature, though the degree of change may also come from reflected radiation from neighboring objects, water vapor absorption, and the object size in the field of view (FOV). Wind and radiation tests suggested that these factors can contribute to the uncertainty of several Celsius degrees in measured results. Based on these indoor experiment results, we provide a list of suggestions on the potential practices for deriving accurate temperature data from radiometric miniaturized TIR cameras in actual field practices for (agro-)environmental research.

Semi-Automatic Spectral Image Stitching for a Compact Hybrid Linescan Hyperspectral Camera towards Near Field Remote Monitoring of Potato Crop Leaves

The miniaturization of hyperspectral cameras has opened a new path to capture spectral information. One such camera, called the hybrid linescan camera, requires accurate control of its movement. Contrary to classical linescan cameras, where one line is available for every band in one shot, the latter asks for multiple shots to fill a line with multiple bands. Unfortunately, the reconstruction is corrupted by a parallax effect, which affects each band differently. In this article, we propose a two-step procedure, which first reconstructs an approximate datacube in two different ways, and second, performs a corrective warping on each band based on a multiple homography framework. The second step combines different stitching methods to perform this reconstruction. A complete synthetic and experimental comparison is performed by using geometric indicators of reference points. It appears throughout the course of our experimentation that misalignment is significantly reduced but remains non-negligible at the potato leaf scale.

Genomic prediction modeling of soybean biomass using UAV-based remote sensing and longitudinal model parameters

The application of remote sensing in plant breeding can provide rich information about the growth processes of plants, which leads to better understanding concerning crop yield. It has been shown that traits measured by remote sensing were also beneficial for genomic prediction (GP) because the inclusion of remote sensing data in multitrait models improved prediction accuracies of target traits. However, the present multitrait GP model cannot incorporate high-dimensional remote sensing data due to the difficulty in the estimation of a covariance matrix among the traits, which leads to failure in improving its prediction accuracy. In this study, we focused on growth models to express growth patterns using remote sensing data with a few parameters and investigated whether a multitrait GP model using these parameters could derive better prediction accuracy of soybean [Glycine max (L.) Merr.] biomass. A total of 198 genotypes of soybean germplasm were cultivated in experimental fields, and longitudinal changes of their canopy height and area were measured continuously via remote sensing with an unmanned aerial vehicle. Growth parameters were estimated by applying simple growth models and incorporated into the GP of biomass. By evaluating heritability and correlation, we showed that the estimated growth parameters appropriately represented the observed growth curves. Also, the use of these growth parameters in the multitrait GP model contributed to successful biomass prediction. We conclude that the growth models could describe the genetic variation of soybean growth curves based on several growth parameters. These dimension-reduction growth models will be indispensable for extracting useful information from remote sensing data and using this data in GP and plant breeding.

Potential Phenotyping Methodologies to Assess Inter- and Intravarietal Variability and to Select Grapevine Genotypes Tolerant to Abiotic Stress

Plant phenotyping is an emerging science that combines multiple methodologies and protocols to measure plant traits (e.g., growth, morphology, architecture, function, and composition) at multiple scales of organization. Manual phenotyping remains as a major bottleneck to the advance of plant and crop breeding. Such constraint fostered the development of high throughput plant phenotyping (HTPP), which is largely based on imaging approaches and automatized data retrieval and processing. Field phenotyping still poses major challenges and the progress of HTPP for field conditions can be relevant to support selection and breeding of grapevine. The aim of this review is to discuss potential and current methods to improve field phenotyping of grapevine to support characterization of inter- and intravarietal diversity. Vitis vinifera has a large genetic diversity that needs characterization, and the availability of methods to support selection of plant material (polyclonal or clonal) able to withstand abiotic stress is paramount. Besides being time consuming, complex and expensive, field experiments are also affected by heterogeneous and uncontrolled climate and soil conditions, mostly due to the large areas of the trials and to the high number of traits to be observed in a number of individuals ranging from hundreds to thousands. Therefore, adequate field experimental design and data gathering methodologies are crucial to obtain reliable data. Some of the major challenges posed to grapevine selection programs for tolerance to water and heat stress are described herein. Useful traits for selection and related field phenotyping methodologies are described and their adequacy for large scale screening is discussed.

Resources for image-based high-throughput phenotyping in crops and data sharing challenges

High-throughput phenotyping (HTP) platforms are capable of monitoring the phenotypic variation of plants through multiple types of sensors, such as red green and blue (RGB) cameras, hyperspectral sensors, and computed tomography, which can be associated with environmental and genotypic data. Because of the wide range of information provided, HTP datasets represent a valuable asset to characterize crop phenotypes. As HTP becomes widely employed with more tools and data being released, it is important that researchers are aware of these resources and how they can be applied to accelerate crop improvement. Researchers may exploit these datasets either for phenotype comparison or employ them as a benchmark to assess tool performance and to support the development of tools that are better at generalizing between different crops and environments. In this review, we describe the use of image-based HTP for yield prediction, root phenotyping, development of climate-resilient crops, detecting pathogen and pest infestation, and quantitative trait measurement. We emphasize the need for researchers to share phenotypic data, and offer a comprehensive list of available datasets to assist crop breeders and tool developers to leverage these resources in order to accelerate crop breeding.

Combining UAV-RGB high-throughput field phenotyping and genome-wide association study to reveal genetic variation of rice germplasms in dynamic response to drought stress

Accurate and high-throughput phenotyping of the dynamic response of a large rice population to drought stress in the field is a bottleneck for genetic dissection and breeding of drought resistance. Here, high-efficiency and high-frequent image acquisition by an unmanned aerial vehicle (UAV) was utilized to quantify the dynamic drought response of a rice population under field conditions. Deep convolutional neural networks (DCNNs) and canopy height models were applied to extract highly correlated phenotypic traits including UAV-based leaf-rolling score (LRS_uav), plant water content (PWC_uav) and a new composite trait, drought resistance index by UAV (DRI_uav). The DCNNs achieved high accuracy (correlation coefficient R = 0.84 for modeling set and R = 0.86 for test set) to replace manual leaf-rolling rating. PWC_uav values were precisely estimated (correlation coefficient R = 0.88) and DRI_uav was modeled to monitor the drought resistance of rice accessions dynamically and comprehensively. A total of 111 significantly associated loci were detected by genome-wide association study for the three dynamic traits, and 30.6% of them were not detected in previous mapping studies using nondynamic drought response traits. Unmanned aerial vehicle and deep learning are confirmed effective phenotyping techniques for more complete genetic dissection of rice dynamic responses to drought and exploration of valuable alleles for drought resistance improvement.

Uncovering candidate genes involved in photosynthetic capacity using unexplored genetic variation in Spring Wheat

To feed an ever-increasing population we must leverage advances in genomics and phenotyping to harness the variation in wheat breeding populations for traits like photosynthetic capacity which remains unoptimized. Here we survey a diverse set of wheat germplasm containing elite, introgression and synthetic derivative lines uncovering previously uncharacterized variation. We demonstrate how strategic integration of exotic material alleviates the D genome genetic bottleneck in wheat, increasing SNP rate by 62% largely due to Ae. tauschii synthetic wheat donors. Across the panel, 67% of the Ae. tauschii donor genome is represented as introgressions in elite backgrounds. We show how observed genetic variation together with hyperspectral reflectance data can be used to identify candidate genes for traits relating to photosynthetic capacity using association analysis. This demonstrates the value of genomic methods in uncovering hidden variation in wheat and how that variation can assist breeding efforts and increase our understanding of complex traits.

High-Throughput Phenotyping Methods for Breeding Drought-Tolerant Crops

Drought is a main factor limiting crop yields. Modern agricultural technologies such as irrigation systems, ground mulching, and rainwater storage can prevent drought, but these are only temporary solutions. Understanding the physiological, biochemical, and molecular reactions of plants to drought stress is therefore urgent. The recent rapid development of genomics tools has led to an increasing interest in phenomics, i.e., the study of phenotypic plant traits. Among phenomic strategies, high-throughput phenotyping (HTP) is attracting increasing attention as a way to address the bottlenecks of genomic and phenomic studies. HTP provides researchers a non-destructive and non-invasive method yet accurate in analyzing large-scale phenotypic data. This review describes plant responses to drought stress and introduces HTP methods that can detect changes in plant phenotypes in response to drought.

Prediction of plant-level tomato biomass and yield using machine learning with unmanned aerial vehicle imagery

BACKGROUND

The objective of this study is twofold. First, ascertain the important variables that predict tomato yields from plant height (PH) and vegetation index (VI) maps. The maps were derived from images taken by unmanned aerial vehicles (UAVs). Second, examine the accuracy of predictions of tomato fresh shoot masses (SM), fruit weights (FW), and the number of fruits (FN) from multiple machine learning algorithms using selected variable sets. To realize our objective, ultra-high-resolution RGB and multispectral images were collected by a UAV on ten days in 2020's tomato growing season. From these images, 756 total variables, including first- (e.g., average, standard deviation, skewness, range, and maximum) and second-order (e.g., gray-level co-occurrence matrix features and growth rates of PH and VIs) statistics for each plant, were extracted. Several selection algorithms (i.e., Boruta, DALEX, genetic algorithm, least absolute shrinkage and selection operator, and recursive feature elimination) were used to select the variable sets useful for predicting SM, FW, and FN. Random forests, ridge regressions, and support vector machines were used to predict the yield using the top five selected variable sets.

RESULTS

First-order statistics of PH and VIs collected during the early to mid-fruit formation periods, about one month prior to harvest, were important variables for predicting SM. Similar to the case for SM, variables collected approximately one month prior to harvest were important for predicting FW and FN. Furthermore, variables related to PH were unimportant for prediction. Compared with predictions obtained using only first-order statistics, those obtained using the second-order statistics of VIs were more accurate for FW and FN. The prediction accuracy of SM, FW, and FN by models constructed from all variables (rRMSE = 8.8-28.1%) was better than that from first-order statistics (rRMSE = 10.0-50.1%).

CONCLUSIONS

In addition to basic statistics (e.g., average and standard deviation), we derived second-order statistics of PH and VIs at the plant level using the ultra-high resolution UAV images. Our findings indicated that our variable selection method reduced the number variables needed for tomato yield prediction, improving the efficiency of phenotypic data collection and assisting with the selection of high-yield lines within breeding programs.

Prediction of plant-level tomato biomass and yield using machine learning with unmanned aerial vehicle imagery

BACKGROUND

The objective of this study is twofold. First, ascertain the important variables that predict tomato yields from plant height (PH) and vegetation index (VI) maps. The maps were derived from images taken by unmanned aerial vehicles (UAVs). Second, examine the accuracy of predictions of tomato fresh shoot masses (SM), fruit weights (FW), and the number of fruits (FN) from multiple machine learning algorithms using selected variable sets. To realize our objective, ultra-high-resolution RGB and multispectral images were collected by a UAV on ten days in 2020's tomato growing season. From these images, 756 total variables, including first- (e.g., average, standard deviation, skewness, range, and maximum) and second-order (e.g., gray-level co-occurrence matrix features and growth rates of PH and VIs) statistics for each plant, were extracted. Several selection algorithms (i.e., Boruta, DALEX, genetic algorithm, least absolute shrinkage and selection operator, and recursive feature elimination) were used to select the variable sets useful for predicting SM, FW, and FN. Random forests, ridge regressions, and support vector machines were used to predict the yield using the top five selected variable sets.

RESULTS

First-order statistics of PH and VIs collected during the early to mid-fruit formation periods, about one month prior to harvest, were important variables for predicting SM. Similar to the case for SM, variables collected approximately one month prior to harvest were important for predicting FW and FN. Furthermore, variables related to PH were unimportant for prediction. Compared with predictions obtained using only first-order statistics, those obtained using the second-order statistics of VIs were more accurate for FW and FN. The prediction accuracy of SM, FW, and FN by models constructed from all variables (rRMSE = 8.8-28.1%) was better than that from first-order statistics (rRMSE = 10.0-50.1%).

CONCLUSIONS

In addition to basic statistics (e.g., average and standard deviation), we derived second-order statistics of PH and VIs at the plant level using the ultra-high resolution UAV images. Our findings indicated that our variable selection method reduced the number variables needed for tomato yield prediction, improving the efficiency of phenotypic data collection and assisting with the selection of high-yield lines within breeding programs.

High night temperature effects on wheat and rice: Current status and way forward

Rapid increases in minimum night temperature than in maximum day temperature is predicted to continue, posing significant challenges to crop productivity. Rice and wheat are two major staples that are sensitive to high night-temperature (HNT) stress. This review aims to (i) systematically compare the grain yield responses of rice and wheat exposed to HNT stress across scales, and (ii) understand the physiological and biochemical responses that affect grain yield and quality. To achieve this, we combined a synthesis of current literature on HNT effects on rice and wheat with information from a series of independent experiments we conducted across scales, using a common set of genetic materials to avoid confounding our findings with differences in genetic background. In addition, we explored HNT-induced alterations in physiological mechanisms including carbon balance, source-sink metabolite changes and reactive oxygen species. Impacts of HNT on grain developmental dynamics focused on grain-filling duration, post-flowering senescence, changes in grain starch and protein composition, starch metabolism enzymes and chalk formation in rice grains are summarized. Finally, we highlight the need for high-throughput field-based phenotyping facilities for improved assessment of large-diversity panels and mapping populations to aid breeding for increased resilience to HNT in crops.

Convolutional Neural Networks to Estimate Dry Matter Yield in a Guineagrass Breeding Program Using UAV Remote Sensing

Forage dry matter is the main source of nutrients in the diet of ruminant animals. Thus, this trait is evaluated in most forage breeding programs with the objective of increasing the yield. Novel solutions combining unmanned aerial vehicles (UAVs) and computer vision are crucial to increase the efficiency of forage breeding programs, to support high-throughput phenotyping (HTP), aiming to estimate parameters correlated to important traits. The main goal of this study was to propose a convolutional neural network (CNN) approach using UAV-RGB imagery to estimate dry matter yield traits in a guineagrass breeding program. For this, an experiment composed of 330 plots of full-sib families and checks conducted at Embrapa Beef Cattle, Brazil, was used. The image dataset was composed of images obtained with an RGB sensor embedded in a Phantom 4 PRO. The traits leaf dry matter yield (LDMY) and total dry matter yield (TDMY) were obtained by conventional agronomic methodology and considered as the ground-truth data. Different CNN architectures were analyzed, such as AlexNet, ResNeXt50, DarkNet53, and two networks proposed recently for related tasks named MaCNN and LF-CNN. Pretrained AlexNet and ResNeXt50 architectures were also studied. Ten-fold cross-validation was used for training and testing the model. Estimates of DMY traits by each CNN architecture were considered as new HTP traits to compare with real traits. Pearson correlation coefficient r between real and HTP traits ranged from 0.62 to 0.79 for LDMY and from 0.60 to 0.76 for TDMY; root square mean error (RSME) ranged from 286.24 to 366.93 kg·ha-1 for LDMY and from 413.07 to 506.56 kg·ha-1 for TDMY. All the CNNs generated heritable HTP traits, except LF-CNN for LDMY and AlexNet for TDMY. Genetic correlations between real and HTP traits were high but varied according to the CNN architecture. HTP trait from ResNeXt50 pretrained achieved the best results for indirect selection regardless of the dry matter trait. This demonstrates that CNNs with remote sensing data are highly promising for HTP for dry matter yield traits in forage breeding programs.

Breeding for drought and heat tolerance in wheat

Many approaches have been adopted to enhance the heat and drought tolerance of wheat with mixed success. An assessment of the relative merits of different strategies is presented. Wheat is the most widely grown crop globally and plays a key role in human nutrition. However, it is grown in environments that are prone to heat and drought stress, resulting in severely reduced yield in some seasons. Increased climate variability is expected to have a particularly adverse effect of wheat production. Breeding for stable yield across both good and bad seasons while maintaining high yield under optimal conditions is a high priority for most wheat breeding programs and has been a focus of research activities. Multiple strategies have been explored to enhance the heat and drought tolerance of wheat including extensive genetic analysis and modify the expression of genes involved in stress responses, targeting specific physiological traits and direct selection under a range of stress scenarios. These approaches have been combined with improvements in phenotyping, the development of genetic and genomic resources, and extended screening and analysis techniques. The results have greatly expanded our knowledge and understanding of the factors that influence yield under stress, but not all have delivered the hoped-for progress. Here, we provide an overview of the different strategies and an assessment of the most promising approaches.

Scaling up high-throughput phenotyping for abiotic stress selection in the field

High-throughput phenotyping (HTP) is in its infancy for deployment in large-scale breeding programmes. With the ability to measure correlated traits associated with physiological ideotypes, in-field phenotyping methods are available for screening of abiotic stress responses. As cropping environments become more hostile and unpredictable due to the effects of climate change, the need to characterise variability across spatial and temporal scales will become increasingly important. The sensor technologies that have enabled HTP from macroscopic through to satellite sensors may also be utilised here to complement spatial characterisation using envirotyping, which can improve estimations of genotypic performance across environments by better accounting for variation at the plot, trial and inter-trial levels. Climate change is leading to increased variation at all physical and temporal scales in the cropping environment. Maintaining yield stability under circumstances with greater levels of abiotic stress while capitalising upon yield potential in good years, requires approaches to plant breeding that target the physiological limitations to crop performance in specific environments. This requires dynamic modelling of conditions within target populations of environments, GxExM predictions, clustering of environments so breeding trajectories can be defined, and the development of screens that enable selection for genetic gain to occur. High-throughput phenotyping (HTP), combined with related technologies used for envirotyping, can help to address these challenges. Non-destructive analysis of the morphological, biochemical and physiological qualities of plant canopies using HTP has great potential to complement whole-genome selection, which is becoming increasingly common in breeding programmes. A range of novel analytic techniques, such as machine learning and deep learning, combined with a widening range of sensors, allow rapid assessment of large breeding populations that are repeatable and objective. Secondary traits underlying radiation use efficiency and water use efficiency can be screened with HTP for selection at the early stages of a breeding programme. HTP and envirotyping technologies can also characterise spatial variability at trial and within-plot levels, which can be used to correct for spatial variations that confound measurements of genotypic values. This review explores HTP for abiotic stress selection through a physiological trait lens and additionally investigates the use of envirotyping and EC to characterise spatial variability at all physical scales in METs.

Recent developments and potential of robotics in plant eco-phenotyping

Automated acquisition of plant eco-phenotypic information can serve as a decision-making basis for precision agricultural management and can also provide detailed insights into plant growth status, pest management, water and fertilizer management for plant breeders and plant physiologists. Because the microscopic components and macroscopic morphology of plants will be affected by the ecological environment, research on plant eco-phenotyping is more meaningful than the study of single-plant phenotyping. To achieve high-throughput acquisition of phenotyping information, the combination of high-precision sensors and intelligent robotic platforms have become an emerging research focus. Robotic platforms and automated systems are the important carriers of phenotyping monitoring sensors that enable large-scale screening. Through the diverse design and flexible systems, an efficient operation can be achieved across a range of experimental and field platforms. The combination of robot technology and plant phenotyping monitoring tools provides the data to inform novel artificial intelligence (AI) approaches that will provide steppingstones for new research breakthroughs. Therefore, this article introduces robotics and eco-phenotyping and examines research significant to this novel domain of plant eco-phenotyping. Given the monitoring scenarios of phenotyping information at different scales, the used intelligent robot technology, efficient automation platform, and advanced sensor equipment are summarized in detail. We further discuss the challenges posed to current research as well as the future developmental trends in the application of robot technology and plant eco-phenotyping. These include the use of collected data for AI applications and high-bandwidth data transfer, and large well-structured (meta) data storage approaches in plant sciences and agriculture.

Advancing crop genomics from lab to field

Crop genomics remains a key element in ensuring scientific progress to secure global food security. It has been two decades since the sequence of the first plant genome, that of Arabidopsis thaliana, was released, and soon after that the draft sequencing of the rice genome was completed. Since then, the genomes of more than 100 crops have been sequenced, plant genome research has expanded across multiple fronts and the next few years promise to bring further advances spurred by the advent of new technologies and approaches. We are likely to see continued innovations in crop genome sequencing, genetic mapping and the acquisition of multiple levels of biological data. There will be exciting opportunities to integrate genome-scale information across multiple scales of biological organization, leading to advances in our mechanistic understanding of crop biological processes, which will, in turn, provide greater impetus for translation of laboratory results to the field.

Field-based remote sensing models predict radiation use efficiency in wheat

Wheat yields are stagnating or declining in many regions, requiring efforts to improve the light conversion efficiency, known as radiation use efficiency (RUE). RUE is a key trait in plant physiology because it links light capture and primary metabolism with biomass accumulation and yield, but its measurement is time consuming and this has limited its use in fundamental research and large-scale physiological breeding. In this study, high-throughput plant phenotyping (HTPP) approaches were used among a population of field-grown wheat with variation in RUE and photosynthetic traits to build predictive models of RUE, biomass, and intercepted photosynthetically active radiation (IPAR). Three approaches were used: best combination of sensors; canopy vegetation indices; and partial least squares regression. The use of remote sensing models predicted RUE with up to 70% accuracy compared with ground truth data. Water indices and canopy greenness indices [normalized difference vegetation index (NDVI), enhanced vegetation index (EVI)] are the better option to predict RUE, biomass, and IPAR, and indices related to gas exchange, non-photochemical quenching [photochemical reflectance index (PRI)] and senescence [structural-insensitive pigment index (SIPI)] are better predictors for these traits at the vegetative and grain-filling stages, respectively. These models will be instrumental to explain canopy processes, improve crop growth and yield modelling, and potentially be used to predict RUE in different crops or ecosystems.

Crop Performance Evaluation of Chickpea and Dry Pea Breeding Lines Across Seasons and Locations Using Phenomics Data

The Pacific Northwest is an important pulse production region in the United States. Currently, pulse crop (chickpea, lentil, and dry pea) breeders rely on traditional phenotyping approaches to collect performance and agronomic data to support decision making. Traditional phenotyping poses constraints on data availability (e.g., number of locations and frequency of data acquisition) and throughput. In this study, phenomics technologies were applied to evaluate the performance and agronomic traits in two pulse (chickpea and dry pea) breeding programs using data acquired over multiple seasons and locations. An unmanned aerial vehicle-based multispectral imaging system was employed to acquire image data of chickpea and dry pea advanced yield trials from three locations during 2017-2019. The images were analyzed semi-automatically with custom image processing algorithm and features were extracted, such as canopy area and summary statistics associated with vegetation indices. The study demonstrated significant correlations (P < 0.05) between image-based features (e.g., canopy area and sum normalized difference vegetation index) with yield (r up to 0.93 and 0.85 for chickpea and dry pea, respectively), days to 50% flowering (r up to 0.76 and 0.85, respectively), and days to physiological maturity (r up to 0.58 and 0.84, respectively). Using image-based features as predictors, seed yield was estimated using least absolute shrinkage and selection operator regression models, during which, coefficients of determination as high as 0.91 and 0.80 during model testing for chickpea and dry pea, respectively, were achieved. The study demonstrated the feasibility to monitor agronomic traits and predict seed yield in chickpea and dry pea breeding trials across multiple locations and seasons using phenomics tools. Phenomics technologies can assist plant breeders to evaluate the performance of breeding materials more efficiently and accelerate breeding programs.

High Throughput Field Phenotyping for Plant Height Using UAV-Based RGB Imagery in Wheat Breeding Lines: Feasibility and Validation

Plant height (PH) is an essential trait in the screening of most crops. While in crops such as wheat, medium stature helps reduce lodging, tall plants are preferred to increase total above-ground biomass. PH is an easy trait to measure manually, although it can be labor-intense depending on the number of plots. There is an increasing demand for alternative approaches to estimate PH in a higher throughput mode. Crop surface models (CSMs) derived from dense point clouds generated via aerial imagery could be used to estimate PH. This study evaluates PH estimation at different phenological stages using plot-level information from aerial imaging-derived 3D CSM in wheat inbred lines during two consecutive years. Multi-temporal and high spatial resolution images were collected by fixed-wing (P l a t F W ) and multi-rotor (P l a t M R ) unmanned aerial vehicle (UAV) platforms over two wheat populations (50 and 150 lines). The PH was measured and compared at four growth stages (GS) using ground-truth measurements (PHground) and UAV-based estimates (PHaerial). The CSMs generated from the aerial imagery were validated using ground control points (GCPs) as fixed reference targets at different heights. The results show that PH estimations using P l a t F W were consistent with those obtained from P l a t M R , showing some slight differences due to image processing settings. The GCPs heights derived from CSM showed a high correlation and low error compared to their actual heights (R 2 ≥ 0.90, RMSE ≤ 4 cm). The coefficient of determination (R 2) between PHground and PHaerial at different GS ranged from 0.35 to 0.88, and the root mean square error (RMSE) from 0.39 to 4.02 cm for both platforms. In general, similar and higher heritability was obtained using PHaerial across different GS and years and ranged according to the variability, and environmental error of the PHground observed (0.06-0.97). Finally, we also observed high Spearman rank correlations (0.47-0.91) and R 2 (0.63-0.95) of PHaerial adjusted and predicted values against PHground values. This study provides an example of the use of UAV-based high-resolution RGB imagery to obtain time-series estimates of PH, scalable to tens-of-thousands of plots, and thus suitable to be applied in plant wheat breeding trials.

A novel way to validate UAS-based high-throughput phenotyping protocols using in silico experiments for plant breeding purposes

It is possible to make inferences regarding the feasibility and applicability of plant high-throughput phenotyping via computer simulations. Protocol validation has been a key challenge to the establishment of high-throughput phenotyping (HTP) in breeding programs. We add to this matter by proposing an innovative way for designing and validating aerial imagery-based HTP approaches with in silico 3D experiments for plant breeding purposes. The algorithm is constructed following a pipeline composed of the simulation of phenotypic values, three-dimensional modeling of trials, and image rendering. Our tool is exemplified by testing a set of experimental setups that are of interest in the context of maize breeding using a comprehensive case study. We report on how the choice of (percentile of) points in dense clouds, the experimental repeatability (heritability), the treatment variance (genetic variability), and the flight altitude affect the accuracy of high-throughput plant height estimation based on conventional structure-from-motion (SfM) and multi-view stereo (MVS) pipelines. The evaluation of both the algorithm and the case study was driven by comparisons of the computer-simulated (ground truth) and the HTP-estimated values using correlations, regressions, and similarity indices. Our results showed that the 3D experiments can be adequately reconstructed, enabling inference-making. Moreover, it suggests that treatment variance, repeatability, and the choice of the percentile of points are highly influential over the accuracy of HTP. Conversely, flight altitude influenced the quality of reconstruction but not the accuracy of plant height estimation. Therefore, we believe that our tool can be of high value, enabling the promotion of new insights and further understanding of the events underlying the practice of high-throughput phenotyping.

Development of a Miniaturized Mobile Mapping System for In-Row, Under-Canopy Phenotyping

This paper focuses on the development of a miniaturized mobile mapping platform with advantages over current agricultural phenotyping systems in terms of acquiring data that facilitate under-canopy plant trait extraction. The system is based on an unmanned ground vehicle (UGV) for in-row, under-canopy data acquisition to deliver accurately georeferenced 2D and 3D products. The paper addresses three main aspects pertaining to the UGV development: (a) architecture of the UGV mobile mapping system (MMS), (b) quality assessment of acquired data in terms of georeferencing information as well as derived 3D point cloud, and (c) ability to derive phenotypic plant traits using data acquired by the UGV MMS. The experimental results from this study demonstrate the ability of the UGV MMS to acquire dense and accurate data over agricultural fields that would facilitate highly accurate plant phenotyping (better than above-canopy platforms such as unmanned aerial systems and high-clearance tractors). Plant centers and plant count with an accuracy in the 90% range have been achieved.

Entropy Weight Ensemble Framework for Yield Prediction of Winter Wheat Under Different Water Stress Treatments Using Unmanned Aerial Vehicle-Based Multispectral and Thermal Data

Crop breeding programs generally perform early field assessments of candidate selection based on primary traits such as grain yield (GY). The traditional methods of yield assessment are costly, inefficient, and considered a bottleneck in modern precision agriculture. Recent advances in an unmanned aerial vehicle (UAV) and development of sensors have opened a new avenue for data acquisition cost-effectively and rapidly. We evaluated UAV-based multispectral and thermal images for in-season GY prediction using 30 winter wheat genotypes under 3 water treatments. For this, multispectral vegetation indices (VIs) and normalized relative canopy temperature (NRCT) were calculated and selected by the gray relational analysis (GRA) at each growth stage, i.e., jointing, booting, heading, flowering, grain filling, and maturity to reduce the data dimension. The elastic net regression (ENR) was developed by using selected features as input variables for yield prediction, whereas the entropy weight fusion (EWF) method was used to combine the predicted GY values from multiple growth stages. In our results, the fusion of dual-sensor data showed high yield prediction accuracy [coefficient of determination (R ²) = 0.527-0.667] compared to using a single multispectral sensor (R ² = 0.130-0.461). Results showed that the grain filling stage was the optimal stage to predict GY with R ² = 0.667, root mean square error (RMSE) = 0.881 t ha^-1, relative root-mean-square error (RRMSE) = 15.2%, and mean absolute error (MAE) = 0.721 t ha^-1. The EWF model outperformed at all the individual growth stages with R ² varying from 0.677 to 0.729. The best prediction result (R ² = 0.729, RMSE = 0.831 t ha^-1, RRMSE = 14.3%, and MAE = 0.684 t ha^-1) was achieved through combining the predicted values of all growth stages. This study suggests that the fusion of UAV-based multispectral and thermal IR data within an ENR-EWF framework can provide a precise and robust prediction of wheat yield.

Phenomics-Assisted Selection for Herbage Accumulation in Alfalfa (Medicago sativa L.)

The application of remote sensing in plant breeding is becoming a routine method for fast and non-destructive high-throughput phenotyping (HTP) using unmanned aerial vehicles (UAVs) equipped with sensors. Alfalfa (Medicago sativa L.) is a perennial forage legume grown in more than 30 million hectares worldwide. Breeding alfalfa for herbage accumulation (HA) requires frequent and multiple phenotyping efforts, which is laborious and costly. The objective of this study was to assess the efficiency of UAV-based imagery and spatial analysis in the selection of alfalfa for HA. The alfalfa breeding population was composed of 145 full-sib and 34 half-sib families, and the experimental design was a row-column with augmented representation of controls. The experiment was established in November 2017, and HA was harvested four times between August 2018 and January 2019. A UAV equipped with a multispectral camera was used for HTP before each harvest. Four vegetation indices (VIs) were calculated from the UAV-based images: NDVI, NDRE, GNDVI, and GRVI. All VIs showed a high correlation with HA, and VIs predicted HA with moderate accuracy. HA and NDVI were used for further analyses to calculate the genetic parameters using linear mixed models. The spatial analysis had a significant effect in both dimensions (rows and columns) for HA and NDVI, resulting in improvements in the estimation of genetic parameters. Univariate models for NDVI and HA, and bivariate models, were fit to predict family performance for scenarios with various levels of HA data (simulated in silico by assigning missing values to full dataset). The bivariate models provided higher correlation among predicted values, higher coincidence for selection, and higher genetic gain even for scenarios with only 30% of HA data. Hence, HTP is a reliable and efficient method to aid alfalfa phenotyping to improve HA. Additionally, the use of spatial analysis can also improve the accuracy of selection in breeding trials.