Leveraging Image Analysis for High-Throughput Plant Phenotyping

The role of phenomics and genomics in delineating the genetic basis of complex traits in millets

Millets, comprising a diverse group of small-seeded grains, have emerged as vital crops with immense nutritional, environmental, and economic significance. The comprehension of complex traits in millets, influenced by multifaceted genetic determinants, presents a compelling challenge and opportunity in agricultural research. This review delves into the transformative roles of phenomics and genomics in deciphering these intricate genetic architectures. On the phenomics front, high-throughput platforms generate rich datasets on plant morphology, physiology, and performance in diverse environments. This data, coupled with field trials and controlled conditions, helps to interpret how the environment interacts with genetics. Genomics provides the underlying blueprint for these complex traits. Genome sequencing and genotyping technologies have illuminated the millet genome landscape, revealing diverse gene pools and evolutionary relationships. Additionally, different omics approaches unveil the intricate information of gene expression, protein function, and metabolite accumulation driving phenotypic expression. This multi-omics approach is crucial for identifying candidate genes and unfolding the intricate pathways governing complex traits. The review highlights the synergy between phenomics and genomics. Genomically informed phenotyping targets specific traits, reducing the breeding size and cost. Conversely, phenomics identifies promising germplasm for genomic analysis, prioritizing variants with superior performance. This dynamic interplay accelerates breeding programs and facilitates the development of climate-smart, nutrient-rich millet varieties and hybrids. In conclusion, this review emphasizes the crucial roles of phenomics and genomics in unlocking the genetic enigma of millets.

Stress phenotyping analysis leveraging autofluorescence image sequences with machine learning

Background

Autofluorescence-based imaging has the potential to non-destructively characterize the biochemical and physiological properties of plants regulated by genotypes using optical properties of the tissue. A comparative study of stress tolerant and stress susceptible genotypes of Brassica rapa with respect to newly introduced stress-based phenotypes using machine learning techniques will contribute to the significant advancement of autofluorescence-based plant phenotyping research.

Methods

Autofluorescence spectral images have been used to design a stress detection classifier with two classes, stressed and non-stressed, using machine learning algorithms. The benchmark dataset consisted of time-series image sequences from three Brassica rapa genotypes (CC, R500, and VT), extreme in their morphological and physiological traits captured at the high-throughput plant phenotyping facility at the University of Nebraska-Lincoln, USA. We developed a set of machine learning-based classification models to detect the percentage of stressed tissue derived from plant images and identified the best classifier. From the analysis of the autofluorescence images, two novel stress-based image phenotypes were computed to determine the temporal variation in stressed tissue under progressive drought across different genotypes, i.e., the average percentage stress and the moving average percentage stress.

Results

The study demonstrated that both the computed phenotypes consistently discriminated against stressed versus non-stressed tissue, with oilseed type (R500) being less prone to drought stress relative to the other two Brassica rapa genotypes (CC and VT).

Conclusion

Autofluorescence signals from the 365/400 nm excitation/emission combination were able to segregate genotypic variation during a progressive drought treatment under a controlled greenhouse environment, allowing for the exploration of other meaningful phenotypes using autofluorescence image sequences with significance in the context of plant science.

Phenotyping of Potato Plants Using Morphological and Physiological Tools

Potato (Solanum tuberosum L.) is one of the main non-grain agricultural crops and one of the main sources of food for humanity. Currently, growing potatoes requires new approaches and methods for cultivation and breeding. Phenotyping is one of the important tools for assessing the characteristics of a potato variety. In this work, 29 potato varieties of different ripeness groups were studied. Linear leaf dimensions, leaf mass area, number of stems, number of tubers per plant, average tuber weight, signs of virus infection, dry weight, pigment content, and number of stomata per unit leaf area were used as phenotyping tools. The strongest positive relationship was found between yield and bush area in the stage of full shoots (R = 0.77, p = 0.001), linear dimensions of a complex leaf (R = 0.44, p = 0.002; R = 0.40, p = 0.003), number of stems (R = 0.36, p = 0.05), and resistance to viruses X (R = 0.42, p = 0.03) and S (R = 0.43, p = 0.02). An inverse relationship was found between growth dynamics and yield (R = -0.29, p = 0.05). Thus, the use of morphological and physiological phenotyping tools in the field is informative for predicting key agricultural characteristics such as yield and/or stress resistance.

Root Development Monitoring under Different Water Supply Levels in Processing Tomato Plants

Managing crop yields and optimizing water use is a global challenge, as fresh water supply decreases rapidly and demand remains high. Therefore, understanding how plants react to varying water levels is crucial for efficient water usage. This study evaluates how tomato plants adapt to varying water levels (100%, 50% of crop evapotranspiration, and non-irrigated control) over two growing seasons in 2020 and 2021. Root images were captured weekly during an 8-week monitoring period in 2020 and 6 weeks in 2021 using a non-destructive CI-600 in-situ root imager at depths between 10 and 70 cm. Under water stress, plants developed deeper, more extensive root systems to maximize water uptake, consistent with prior research. Root depth and architecture varied with soil depth and the severity of water stress. Year-to-year variations were also found, likely due to changes in irrigation levels and environmental conditions such as temperature. SPAD values were higher under control conditions, especially in the 2021 growing season, suggesting reduced chlorophyll degradation, while no significant differences were observed in chlorophyll fluorescence (Fv/Fm) between treatments, suggesting stable photosynthetic efficiency under varied water stress conditions. These findings contribute to our understanding of root zone optimization and drought-resilient cultivar breeding, contributing to more sustainable agricultural practices.

Image-based phenotyping of seed architectural traits and prediction of seed weight using machine learning models in soybean

Among seed attributes, weight is one of the main factors determining the soybean harvest index. Recently, the focus of soybean breeding has shifted to improving seed size and weight for crop optimization in terms of seed and oil yield. With recent technological advancements, there is an increasing application of imaging sensors that provide simple, real-time, non-destructive, and inexpensive image data for rapid image-based prediction of seed traits in plant breeding programs. The present work is related to digital image analysis of seed traits for the prediction of hundred-seed weight (HSW) in soybean. The image-based seed architectural traits (i-traits) measured were area size (AS), perimeter length (PL), length (L), width (W), length-to-width ratio (LWR), intersection of length and width (IS), seed circularity (CS), and distance between IS and CG (DS). The phenotypic investigation revealed significant genetic variability among 164 soybean genotypes for both i-traits and manually measured seed weight. Seven popular machine learning (ML) algorithms, namely Simple Linear Regression (SLR), Multiple Linear Regression (MLR), Random Forest (RF), Support Vector Regression (SVR), LASSO Regression (LR), Ridge Regression (RR), and Elastic Net Regression (EN), were used to create models that can predict the weight of soybean seeds based on the image-based novel features derived from the Red-Green-Blue (RGB)/visual image. Among the models, random forest and multiple linear regression models that use multiple explanatory variables related to seed size traits (AS, L, W, and DS) were identified as the best models for predicting seed weight with the highest prediction accuracy (coefficient of determination, R2=0.98 and 0.94, respectively) and the lowest prediction error, i.e., root mean square error (RMSE) and mean absolute error (MAE). Finally, principal components analysis (PCA) and a hierarchical clustering approach were used to identify IC538070 as a superior genotype with a larger seed size and weight. The identified donors/traits can potentially be used in soybean improvement programs.

Phenomics as an approach to Comparative Developmental Physiology

The dynamic nature of developing organisms and how they function presents both opportunity and challenge to researchers, with significant advances in understanding possible by adopting innovative approaches to their empirical study. The information content of the phenotype during organismal development is arguably greater than at any other life stage, incorporating change at a broad range of temporal, spatial and functional scales and is of broad relevance to a plethora of research questions. Yet, effectively measuring organismal development, and the ontogeny of physiological regulations and functions, and their responses to the environment, remains a significant challenge. "Phenomics", a global approach to the acquisition of phenotypic data at the scale of the whole organism, is uniquely suited as an approach. In this perspective, we explore the synergies between phenomics and Comparative Developmental Physiology (CDP), a discipline of increasing relevance to understanding sensitivity to drivers of global change. We then identify how organismal development itself provides an excellent model for pushing the boundaries of phenomics, given its inherent complexity, comparably smaller size, relative to adult stages, and the applicability of embryonic development to a broad suite of research questions using a diversity of species. Collection, analysis and interpretation of whole organismal phenotypic data are the largest obstacle to capitalising on phenomics for advancing our understanding of biological systems. We suggest that phenomics within the context of developing organismal form and function could provide an effective scaffold for addressing grand challenges in CDP and phenomics.

Image-Based High-Throughput Phenotyping in Horticultural Crops

Plant phenotyping is the primary task of any plant breeding program, and accurate measurement of plant traits is essential to select genotypes with better quality, high yield, and climate resilience. The majority of currently used phenotyping techniques are destructive and time-consuming. Recently, the development of various sensors and imaging platforms for rapid and efficient quantitative measurement of plant traits has become the mainstream approach in plant phenotyping studies. Here, we reviewed the trends of image-based high-throughput phenotyping methods applied to horticultural crops. High-throughput phenotyping is carried out using various types of imaging platforms developed for indoor or field conditions. We highlighted the applications of different imaging platforms in the horticulture sector with their advantages and limitations. Furthermore, the principles and applications of commonly used imaging techniques, visible light (RGB) imaging, thermal imaging, chlorophyll fluorescence, hyperspectral imaging, and tomographic imaging for high-throughput plant phenotyping, are discussed. High-throughput phenotyping has been widely used for phenotyping various horticultural traits, which can be morphological, physiological, biochemical, yield, biotic, and abiotic stress responses. Moreover, the ability of high-throughput phenotyping with the help of various optical sensors will lead to the discovery of new phenotypic traits which need to be explored in the future. We summarized the applications of image analysis for the quantitative evaluation of various traits with several examples of horticultural crops in the literature. Finally, we summarized the current trend of high-throughput phenotyping in horticultural crops and highlighted future perspectives.

A Synthetic Review of Various Dimensions of Non-Destructive Plant Stress Phenotyping

Non-destructive plant stress phenotyping begins with traditional one-dimensional (1D) spectroscopy, followed by two-dimensional (2D) imaging, three-dimensional (3D) or even temporal-three-dimensional (T-3D), spectral-three-dimensional (S-3D), and temporal-spectral-three-dimensional (TS-3D) phenotyping, all of which are aimed at observing subtle changes in plants under stress. However, a comprehensive review that covers all these dimensional types of phenotyping, ordered in a spatial arrangement from 1D to 3D, as well as temporal and spectral dimensions, is lacking. In this review, we look back to the development of data-acquiring techniques for various dimensions of plant stress phenotyping (1D spectroscopy, 2D imaging, 3D phenotyping), as well as their corresponding data-analyzing pipelines (mathematical analysis, machine learning, or deep learning), and look forward to the trends and challenges of high-performance multi-dimension (integrated spatial, temporal, and spectral) phenotyping demands. We hope this article can serve as a reference for implementing various dimensions of non-destructive plant stress phenotyping.

Benchmarking Self-Supervised Contrastive Learning Methods for Image-Based Plant Phenotyping

The rise of self-supervised learning (SSL) methods in recent years presents an opportunity to leverage unlabeled and domain-specific datasets generated by image-based plant phenotyping platforms to accelerate plant breeding programs. Despite the surge of research on SSL, there has been a scarcity of research exploring the applications of SSL to image-based plant phenotyping tasks, particularly detection and counting tasks. We address this gap by benchmarking the performance of 2 SSL methods-momentum contrast (MoCo) v2 and dense contrastive learning (DenseCL)-against the conventional supervised learning method when transferring learned representations to 4 downstream (target) image-based plant phenotyping tasks: wheat head detection, plant instance detection, wheat spikelet counting, and leaf counting. We studied the effects of the domain of the pretraining (source) dataset on the downstream performance and the influence of redundancy in the pretraining dataset on the quality of learned representations. We also analyzed the similarity of the internal representations learned via the different pretraining methods. We find that supervised pretraining generally outperforms self-supervised pretraining and show that MoCo v2 and DenseCL learn different high-level representations compared to the supervised method. We also find that using a diverse source dataset in the same domain as or a similar domain to the target dataset maximizes performance in the downstream task. Finally, our results show that SSL methods may be more sensitive to redundancy in the pretraining dataset than the supervised pretraining method. We hope that this benchmark/evaluation study will guide practitioners in developing better SSL methods for image-based plant phenotyping.

A Systematic Review of Effective Hardware and Software Factors Affecting High-Throughput Plant Phenotyping

Plant phenotyping studies the complex characteristics of plants, with the aim of evaluating and assessing their condition and finding better exemplars. Recently, a new branch emerged in the phenotyping field, namely, high-throughput phenotyping (HTP). Specifically, HTP exploits modern data sampling techniques to gather a high amount of data that can be used to improve the effectiveness of phenotyping. Hence, HTP combines the knowledge derived from the phenotyping domain with computer science, engineering, and data analysis techniques. In this scenario, machine learning (ML) and deep learning (DL) algorithms have been successfully integrated with noninvasive imaging techniques, playing a key role in automation, standardization, and quantitative data analysis. This study aims to systematically review two main areas of interest for HTP: hardware and software. For each of these areas, two influential factors were identified: for hardware, platforms and sensing equipment were analyzed; for software, the focus was on algorithms and new trends. The study was conducted following the PRISMA protocol, which allowed the refinement of the research on a wide selection of papers by extracting a meaningful dataset of 32 articles of interest. The analysis highlighted the diffusion of ground platforms, which were used in about 47% of reviewed methods, and RGB sensors, mainly due to their competitive costs, high compatibility, and versatility. Furthermore, DL-based algorithms accounted for the larger share (about 69%) of reviewed approaches, mainly due to their effectiveness and the focus posed by the scientific community over the last few years. Future research will focus on improving DL models to better handle hardware-generated data. The final aim is to create integrated, user-friendly, and scalable tools that can be directly deployed and used on the field to improve the overall crop yield.

A deep learning approach to track Arabidopsis seedlings' circumnutation from time-lapse videos

BACKGROUND

Circumnutation (Darwin et al., Sci Rep 10(1):1-13, 2000) is the side-to-side movement common among growing plant appendages but the purpose of circumnutation is not always clear. Accurately tracking and quantifying circumnutation can help researchers to better study its underlying purpose.

RESULTS

In this paper, a deep learning-based model is proposed to track the circumnutating flowering apices in the plant Arabidopsis thaliana from time-lapse videos. By utilizing U-Net to segment the apex, and combining it with the model update mechanism, pre- and post- processing steps, the proposed model significantly improves the tracking time and accuracy over other baseline tracking methods. Additionally, we evaluate the computational complexity of the proposed model and further develop a method to accelerate the inference speed of the model. The fast algorithm can track the apices in real-time on a computer without a dedicated GPU.

CONCLUSION

We demonstrate that the accuracy of tracking the flowering apices in the plant Arabidopsis thaliana can be improved with our proposed deep learning-based model in terms of both the racking success rate and the tracking error. We also show that the improvement in the tracking accuracy is statistically significant. The time-lapse video dataset of Arabidopsis is also provided which can be used for future studies on Arabidopsis in various takes.

EmergeNet: A novel deep-learning based ensemble segmentation model for emergence timing detection of coleoptile

The emergence timing of a plant, i.e., the time at which the plant is first visible from the surface of the soil, is an important phenotypic event and is an indicator of the successful establishment and growth of a plant. The paper introduces a novel deep-learning based model called EmergeNet with a customized loss function that adapts to plant growth for coleoptile (a rigid plant tissue that encloses the first leaves of a seedling) emergence timing detection. It can also track its growth from a time-lapse sequence of images with cluttered backgrounds and extreme variations in illumination. EmergeNet is a novel ensemble segmentation model that integrates three different but promising networks, namely, SEResNet, InceptionV3, and VGG19, in the encoder part of its base model, which is the UNet model. EmergeNet can correctly detect the coleoptile at its first emergence when it is tiny and therefore barely visible on the soil surface. The performance of EmergeNet is evaluated using a benchmark dataset called the University of Nebraska-Lincoln Maize Emergence Dataset (UNL-MED). It contains top-view time-lapse images of maize coleoptiles starting before the occurrence of their emergence and continuing until they are about one inch tall. EmergeNet detects the emergence timing with 100% accuracy compared with human-annotated ground-truth. Furthermore, it significantly outperforms UNet by generating very high-quality segmented masks of the coleoptiles in both natural light and dark environmental conditions.

A comprehensive review on acquisition of phenotypic information of Prunoideae fruits: Image technology

Fruit phenotypic information reflects all the physical, physiological, biochemical characteristics and traits of fruit. Accurate access to phenotypic information is very necessary and meaningful for post-harvest storage, sales and deep processing. The methods of obtaining phenotypic information include traditional manual measurement and damage detection, which are inefficient and destructive. In the field of fruit phenotype research, image technology is increasingly mature, which greatly improves the efficiency of fruit phenotype information acquisition. This review paper mainly reviews the research on phenotypic information of Prunoideae fruit based on three imaging techniques (RGB imaging, hyperspectral imaging, multispectral imaging). Firstly, the classification was carried out according to the image type. On this basis, the review and summary of previous studies were completed from the perspectives of fruit maturity detection, fruit quality classification and fruit disease damage identification. Analysis of the advantages and disadvantages of various types of images in the study, and try to give the next research direction for improvement.

PhenoBot: an automated system for leaf area analysis using deep learning

A low-cost dynamic image capturing and analysis pipeline using color-based deep learning segmentation was developed for direct leaf area estimation of multiple crop types in a commercial environment. Crop yield is largely driven by intercepted radiation of the leaf canopy, making the leaf area index (LAI) a critical parameter for estimating yields. The growth rate of leaves at different growth stages determines the overall LAI, which is used by crop growth models (CGM) for simulating yield. Consequently, precise phenotyping of the leaves can help elucidate phenological processes relating to resource capturing. A stable system for acquiring images and a strong data processing backend play a vital role in reducing throughput time and increasing accuracy of calculations, compared to manual analysis. However, most available solutions are not dynamic, as they use color-based segmentation, which fails to capture leaves with varying shades and shapes. We have developed a system that uses a low-cost setup to acquire images and an automated pipeline to manage the data storage on the device and in the cloud. The system is powered by virtual machines that run multiple custom-trained deep learning models to segment out leaves, calculate leaf area (LA) for the whole set and at the individual leaf level, overlays important information on the images, and appends them on a compatible file used for CGMs with very high accuracy. The pipeline is dynamic and can be used for multiple crops. The use of open-source hardware, platforms, and algorithms makes this system affordable and reproducible.

Implication of high variance in germplasm characteristics

The beauty of conserving germplasm is the securement of genetic resources with numerous important traits, which could be utilized whenever they need to be incorporated into current cultivars. However, it would not be as useful as expected if the proper information was not given to breeders and researchers. In this study, we demonstrated that there is a large variation, both among and within germplasm, using a low-cost image-based phenotyping method; this could be valuable for improving gene banks' screening systems and for crop breeding. Using the image analyses of 507 accessions of buckwheat, we identified a wide range of variations per trait between germplasm accessions and within an accession. Since this implies a similarity with other important agronomic traits, we suggest that the variance of the presented traits should be checked and provided for better germplasm enhancement.

Image-based phenotyping to estimate anthocyanin concentrations in lettuce

Anthocyanins provide blue, red, and purple color to fruits, vegetables, and flowers. Due to their benefits for human health and aesthetic appeal, anthocyanin content in crops affects consumer preference. Rapid, low-cost, and non-destructive phenotyping of anthocyanins is not well developed. Here, we introduce the normalized difference anthocyanin index (NDAI), which is based on the optical properties of anthocyanins: high absorptance in the green and low absorptance in the red part of the spectrum. NDAI is determined as (I_red - I_green)/(I_red + I_green), where I is the pixel intensity, a measure of reflectance. To test NDAI, leaf discs of two red lettuce (Lactuca sativa) cultivars 'Rouxai' and 'Teodore' with wide range of anthocyanin concentrations were imaged using a multispectral imaging system and the red and green images were used to calculate NDAI. NDAI and other commonly used indices for anthocyanin quantification were evaluated by comparing to with the measured anthocyanin concentration (n = 50). Statistical results showed that NDAI has advantages over other indices in terms of prediction of anthocyanin concentrations. Canopy NDAI, obtained using multispectral canopy imaging, was correlated (n = 108, R² = 0.73) with the anthocyanin concentrations of the top canopy layer, which is visible in the images. Comparison of canopy NDAI from multispectral images and RGB images acquired using a Linux-based microcomputer with color camera, showed similar results in the prediction of anthocyanin concentration. Thus, a low-cost microcomputer with a camera can be used to build an automated phenotyping system for anthocyanin content.

Rapid online plant leaf area change detection with high-throughput plant image data

High-throughput plant phenotyping (HTPP) has become an emerging technique to study plant traits due to its fast, labor-saving, accurate and non-destructive nature. It has wide applications in plant breeding and crop management. However, the resulting massive image data has raised a challenge associated with efficient plant traits prediction and anomaly detection. In this paper, we propose a two-step image-based online detection framework for monitoring and quick change detection of the individual plant leaf area via real-time imaging data. Our proposed method is able to achieve a smaller detection delay compared with some baseline methods under some predefined false alarm rate constraint. Moreover, it does not need to store all past image information and can be implemented in real time. The efficiency of the proposed framework is validated by a real data analysis.

Leaf Count Aided Novel Framework for Rice (Oryza sativa L.) Genotypes Discrimination in Phenomics: Leveraging Computer Vision and Deep Learning Applications

Drought is a detrimental factor to gaining higher yields in rice (Oryza sativa L.), especially amid the rising occurrence of drought across the globe. To combat this situation, it is essential to develop novel drought-resilient varieties. Therefore, screening of drought-adaptive genotypes is required with high precision and high throughput. In contemporary emerging science, high throughput plant phenotyping (HTPP) is a crucial technology that attempts to break the bottleneck of traditional phenotyping. In traditional phenotyping, screening significant genotypes is a tedious task and prone to human error while measuring various plant traits. In contrast, owing to the potential advantage of HTPP over traditional phenotyping, image-based traits, also known as i-traits, were used in our study to discriminate 110 genotypes grown for genome-wide association study experiments under controlled (well-watered), and drought-stress (limited water) conditions, under a phenomics experiment in a controlled environment with RGB images. Our proposed framework non-destructively estimated drought-adaptive plant traits from the images, such as the number of leaves, convex hull, plant-aspect ratio (plant spread), and similarly associated geometrical and morphological traits for analyzing and discriminating genotypes. The results showed that a single trait, the number of leaves, can also be used for discriminating genotypes. This critical drought-adaptive trait was associated with plant size, architecture, and biomass. In this work, the number of leaves and other characteristics were estimated non-destructively from top view images of the rice plant for each genotype. The estimation of the number of leaves for each rice plant was conducted with the deep learning model, YOLO (You Only Look Once). The leaves were counted by detecting corresponding visible leaf tips in the rice plant. The detection accuracy was 86-92% for dense to moderate spread large plants, and 98% for sparse spread small plants. With this framework, the susceptible genotypes (MTU1010, PUSA-1121 and similar genotypes) and drought-resistant genotypes (Heera, Anjali, Dular and similar genotypes) were grouped in the core set with a respective group of drought-susceptible and drought-tolerant genotypes based on the number of leaves, and the leaves' emergence during the peak drought-stress period. Moreover, it was found that the number of leaves was significantly associated with other pertinent morphological, physiological and geometrical traits. Other geometrical traits were measured from the RGB images with the help of computer vision.

Evaluation of Effective Class-Balancing Techniques for CNN-Based Assessment of Aphanomyces Root Rot Resistance in Pea (Pisum sativum L.)

Aphanomyces root rot (ARR) is a devastating disease that affects the production of pea. The plants are prone to infection at any growth stage, and there are no chemical or cultural controls. Thus, the development of resistant pea cultivars is important. Phenomics technologies to support the selection of resistant cultivars through phenotyping can be valuable. One such approach is to couple imaging technologies with deep learning algorithms that are considered efficient for the assessment of disease resistance across a large number of plant genotypes. In this study, the resistance to ARR was evaluated through a CNN-based assessment of pea root images. The proposed model, DeepARRNet, was designed to classify the pea root images into three classes based on ARR severity scores, namely, resistant, intermediate, and susceptible classes. The dataset consisted of 1581 pea root images with a skewed distribution. Hence, three effective data-balancing techniques were identified to solve the prevalent problem of unbalanced datasets. Random oversampling with image transformations, generative adversarial network (GAN)-based image synthesis, and loss function with class-weighted ratio were implemented during the training process. The result indicated that the classification F1-score was 0.92 ± 0.03 when GAN-synthesized images were added, 0.91 ± 0.04 for random resampling, and 0.88 ± 0.05 when class-weighted loss function was implemented, which was higher than when an unbalanced dataset without these techniques were used (0.83 ± 0.03). The systematic approaches evaluated in this study can be applied to other image-based phenotyping datasets, which can aid the development of deep-learning models with improved performance.

Capturing crop adaptation to abiotic stress using image-based technologies

Farmers and breeders aim to improve crop responses to abiotic stresses and secure yield under adverse environmental conditions. To achieve this goal and select the most resilient genotypes, plant breeders and researchers rely on phenotyping to quantify crop responses to abiotic stress. Recent advances in imaging technologies allow researchers to collect physiological data non-destructively and throughout time, making it possible to dissect complex plant responses into quantifiable traits. The use of image-based technologies enables the quantification of crop responses to stress in both controlled environmental conditions and field trials. This paper summarizes phenotyping imaging technologies (RGB, multispectral and hyperspectral sensors, among others) that have been used to assess different abiotic stresses including salinity, drought and nitrogen deficiency, while discussing their advantages and drawbacks. We present a detailed review of traits involved in abiotic tolerance, which have been quantified by a range of imaging sensors under high-throughput phenotyping facilities or using unmanned aerial vehicles in the field. We also provide an up-to-date compilation of spectral tolerance indices and discuss the progress and challenges in machine learning, including supervised and unsupervised models as well as deep learning.

Anatomics: High-throughput phenotyping of plant anatomy

Anatomics is a novel phenotyping strategy focused on high-throughput imaging and quantification of plant anatomy from field-grown plants. Here we highlight its potential applications for genetic and physiological analysis of plant anatomical phenotypes.

Using breeding and quantitative genetics to understand the C4 pathway

Reducing photorespiration in C3 crops could significantly increase rates of photosynthesis and yield. One method to achieve this would be to integrate C4 photosynthesis into C3 species. This objective is challenging as it involves engineering incompletely understood traits into C3 leaves, including complex changes to their biochemistry, cell biology, and anatomy. Quantitative genetics and selective breeding offer underexplored routes to identify regulators of these processes. We first review examples of natural intraspecific variation in C4 photosynthesis as well as the potential for hybridization between C3 and C4 species. We then discuss how quantitative genetic approaches including artificial selection and genome-wide association could be used to better understand the C4 syndrome and in so doing guide the engineering of the C4 pathway into C3 crops.

A Segmentation-Guided Deep Learning Framework for Leaf Counting

Deep learning-based methods have recently provided a means to rapidly and effectively extract various plant traits due to their powerful ability to depict a plant image across a variety of species and growth conditions. In this study, we focus on dealing with two fundamental tasks in plant phenotyping, i.e., plant segmentation and leaf counting, and propose a two-steam deep learning framework for segmenting plants and counting leaves with various size and shape from two-dimensional plant images. In the first stream, a multi-scale segmentation model using spatial pyramid is developed to extract leaves with different size and shape, where the fine-grained details of leaves are captured using deep feature extractor. In the second stream, a regression counting model is proposed to estimate the number of leaves without any pre-detection, where an auxiliary binary mask from segmentation stream is introduced to enhance the counting performance by effectively alleviating the influence of complex background. Extensive pot experiments are conducted CVPPP 2017 Leaf Counting Challenge dataset, which contains images of Arabidopsis and tobacco plants. The experimental results demonstrate that the proposed framework achieves a promising performance both in plant segmentation and leaf counting, providing a reference for the automatic analysis of plant phenotypes.

An Intelligent Analysis Method for 3D Wheat Grain and Ventral Sulcus Traits Based on Structured Light Imaging

The wheat grain three-dimensional (3D) phenotypic characters are of great significance for final yield and variety breeding, and the ventral sulcus traits are the important factors to the wheat flour yield. The wheat grain trait measurements are necessary; however, the traditional measurement method is still manual, which is inefficient, subjective, and labor intensive; moreover, the ventral sulcus traits can only be obtained by destructive measurement. In this paper, an intelligent analysis method based on the structured light imaging has been proposed to extract the 3D wheat grain phenotypes and ventral sulcus traits. First, the 3D point cloud data of wheat grain were obtained by the structured light scanner, and then, the specified point cloud processing algorithms including single grain segmentation and ventral sulcus location have been designed; finally, 28 wheat grain 3D phenotypic characters and 4 ventral sulcus traits have been extracted. To evaluate the best experimental conditions, three-level orthogonal experiments, which include rotation angle, scanning angle, and stage color factors, were carried out on 125 grains of 5 wheat varieties, and the results demonstrated that optimum conditions of rotation angle, scanning angle, and stage color were 30°, 37°, black color individually. Additionally, the results also proved that the mean absolute percentage errors (MAPEs) of wheat grain length, width, thickness, and ventral sulcus depth were 1.83, 1.86, 2.19, and 4.81%. Moreover, the 500 wheat grains of five varieties were used to construct and validate the wheat grain weight model by 32 phenotypic traits, and the cross-validation results showed that the R 2 of the models ranged from 0.77 to 0.83. Finally, the wheat grain phenotype extraction and grain weight prediction were integrated into the specialized software. Therefore, this method was demonstrated to be an efficient and effective way for wheat breeding research.

Common Latent Space Exploration for Calibration Transfer across Hyperspectral Imaging-Based Phenotyping Systems

Hyperspectral imaging has increasingly been used in high-throughput plant phenotyping systems. Rapid advancement in the field of phenotyping has resulted in a wide array of hyperspectral imaging systems. However, sharing the plant feature prediction models between different phenotyping facilities becomes challenging due to the differences in imaging environments and imaging sensors. Calibration transfer between imaging facilities is crucially important to cope with such changes. Spectral space adjustment methods including direct standardization (DS), its variants (PDS, DPDS) and spectral scale transformation (SST) require the standard samples to be imaged in different facilities. However, in real-world scenarios, imaging the standard samples is practically unattractive. Therefore, in this study, we presented three methods (TCA, c-PCA, and di-PLSR) to transfer the calibration models without requiring the standard samples. In order to compare the performance of proposed approaches, maize plants were imaged in two greenhouse-based HTPP systems using two pushbroom-style hyperspectral cameras covering the visible near-infrared range. We tested the proposed methods to transfer nitrogen content (N) and relative water content (RWC) calibration models. The results showed that prediction R2 increased by up to 14.50% and 42.20%, while the reduction in RMSEv was up to 74.49% and 76.72% for RWC and N, respectively. The di-PLSR achieved the best results for almost all the datasets included in this study, with TCA being second. The performance of c-PCA was not at par with the di-PLSR and TCA. Our results showed that the di-PLSR helped to recover the performance of RWC, and N models plummeted due to the differences originating from new imaging systems (sensor type, spectrograph, lens system, spatial resolution, spectral resolution, field of view, bit-depth, frame rate, and exposure time) or lighting conditions. The proposed approaches can alleviate the requirement of developing a new calibration model for a new phenotyping facility or to resort to the spectral space adjustment using the standard samples.

Statistical Growth Analysis of Rice Plants in Chhattisgarh Region Using Automated Pixel-Based Mapping Technique

The statistical growth analysis of field crop has become a great challenge in agriculture. Analyzing the growth of crop through automation provides extensive significance to the farmers for getting information about the problem arising in plants due to irregular growth monitoring. The idea behind this work is the importance of mapping with pixel-based clustering technique for growth analysis in terms of height calculation of rice crop (rice variety is MTU-1010). Height measurement plays a vital role in regular assessment for a healthy crop, and the approach proposed in this work achieves 97.58% accuracy of 14 sampled datasets taken from Indira Gandhi Agriculture University of Raipur, Chhattisgarh; a real-time dataset has been prepared. Proposed work is used for analyzing vertical as well as horizontal scaling technique. Vertical mapping provides the height of a single plant whereas horizontal mapping using k-means clustering provides an average height of the whole field. This work uses machine learning, and image processing techniques are used for this work.

Potential Phenotyping Methodologies to Assess Inter- and Intravarietal Variability and to Select Grapevine Genotypes Tolerant to Abiotic Stress

Plant phenotyping is an emerging science that combines multiple methodologies and protocols to measure plant traits (e.g., growth, morphology, architecture, function, and composition) at multiple scales of organization. Manual phenotyping remains as a major bottleneck to the advance of plant and crop breeding. Such constraint fostered the development of high throughput plant phenotyping (HTPP), which is largely based on imaging approaches and automatized data retrieval and processing. Field phenotyping still poses major challenges and the progress of HTPP for field conditions can be relevant to support selection and breeding of grapevine. The aim of this review is to discuss potential and current methods to improve field phenotyping of grapevine to support characterization of inter- and intravarietal diversity. Vitis vinifera has a large genetic diversity that needs characterization, and the availability of methods to support selection of plant material (polyclonal or clonal) able to withstand abiotic stress is paramount. Besides being time consuming, complex and expensive, field experiments are also affected by heterogeneous and uncontrolled climate and soil conditions, mostly due to the large areas of the trials and to the high number of traits to be observed in a number of individuals ranging from hundreds to thousands. Therefore, adequate field experimental design and data gathering methodologies are crucial to obtain reliable data. Some of the major challenges posed to grapevine selection programs for tolerance to water and heat stress are described herein. Useful traits for selection and related field phenotyping methodologies are described and their adequacy for large scale screening is discussed.

Identification and utilization of genetic determinants of trait measurement errors in image-based, high-throughput phenotyping

The accuracy of trait measurements greatly affects the quality of genetic analyses. During automated phenotyping, trait measurement errors, i.e. differences between automatically extracted trait values and ground truth, are often treated as random effects that can be controlled by increasing population sizes and/or replication number. In contrast, there is some evidence that trait measurement errors may be partially under genetic control. Consistent with this hypothesis, we observed substantial nonrandom, genetic contributions to trait measurement errors for five maize (Zea mays) tassel traits collected using an image-based phenotyping platform. The phenotyping accuracy varied according to whether a tassel exhibited "open" versus. "closed" branching architecture, which is itself under genetic control. Trait-associated SNPs (TASs) identified via genome-wide association studies (GWASs) conducted on five tassel traits that had been phenotyped both manually (i.e. ground truth) and via feature extraction from images exhibit little overlap. Furthermore, identification of TASs from GWASs conducted on the differences between the two values indicated that a fraction of measurement error is under genetic control. Similar results were obtained in a sorghum (Sorghum bicolor) plant height dataset, demonstrating that trait measurement error is genetically determined in multiple species and traits. Trait measurement bias cannot be controlled by increasing population size and/or replication number.

Comparing Machine Learning Methods for Classifying Plant Drought Stress from Leaf Reflectance Spectra in Arabidopsis thaliana

Plant breeders and plant physiologists are deeply committed to high throughput plant phenotyping for drought tolerance. A combination of artificial intelligence with reflectance spectroscopy was tested, as a non-invasive method, for the automatic classification of plant drought stress. Arabidopsis thaliana plants (ecotype Col-0) were subjected to different levels of slowly imposed dehydration (S0, control; S1, moderate stress; S2, severe stress). The reflectance spectra of fully expanded leaves were recorded with an Ocean Optics USB4000 spectrometer and the soil water content (SWC, %) of each pot was determined. The entire data set of the reflectance spectra (intensity vs. wavelength) was given to different machine learning (ML) algorithms, namely decision trees, random forests and extreme gradient boosting. The performance of different methods in classifying the plants in one of the three drought stress classes (S0, S1 and S2) was measured and compared. All algorithms produced very high evaluation scores (F1 > 90%) and agree on the features with the highest discriminative power (reflectance at ~670 nm). Random forests was the best performing method and the most robust to random sampling of training data, with an average F1-score of 0.96 ± 0.05. This classification method is a promising tool to detect plant physiological responses to drought using high-throughput pipelines.

Robotic Technologies for High-Throughput Plant Phenotyping: Contemporary Reviews and Future Perspectives

Phenotyping plants is an essential component of any effort to develop new crop varieties. As plant breeders seek to increase crop productivity and produce more food for the future, the amount of phenotype information they require will also increase. Traditional plant phenotyping relying on manual measurement is laborious, time-consuming, error-prone, and costly. Plant phenotyping robots have emerged as a high-throughput technology to measure morphological, chemical and physiological properties of large number of plants. Several robotic systems have been developed to fulfill different phenotyping missions. In particular, robotic phenotyping has the potential to enable efficient monitoring of changes in plant traits over time in both controlled environments and in the field. The operation of these robots can be challenging as a result of the dynamic nature of plants and the agricultural environments. Here we discuss developments in phenotyping robots, and the challenges which have been overcome and others which remain outstanding. In addition, some perspective applications of the phenotyping robots are also presented. We optimistically anticipate that autonomous and robotic systems will make great leaps forward in the next 10 years to advance the plant phenotyping research into a new era.

Extension of Plant Phenotypes by the Foliar Microbiome

The foliar microbiome can extend the host plant phenotype by expanding its genomic and metabolic capabilities. Despite increasing recognition of the importance of the foliar microbiome for plant fitness, stress physiology, and yield, the diversity, function, and contribution of foliar microbiomes to plant phenotypic traits remain largely elusive. The recent adoption of high-throughput technologies is helping to unravel the diversityand spatiotemporal dynamics of foliar microbiomes, but we have yet to resolve their functional importance for plant growth, development, and ecology. Here, we focus on the processes that govern the assembly of the foliar microbiome and the potential mechanisms involved in extended plant phenotypes. We highlight knowledge gaps and provide suggestions for new research directions that can propel the field forward. These efforts will be instrumental in maximizing the functional potential of the foliar microbiome for sustainable crop production.

Recent developments and potential of robotics in plant eco-phenotyping

Automated acquisition of plant eco-phenotypic information can serve as a decision-making basis for precision agricultural management and can also provide detailed insights into plant growth status, pest management, water and fertilizer management for plant breeders and plant physiologists. Because the microscopic components and macroscopic morphology of plants will be affected by the ecological environment, research on plant eco-phenotyping is more meaningful than the study of single-plant phenotyping. To achieve high-throughput acquisition of phenotyping information, the combination of high-precision sensors and intelligent robotic platforms have become an emerging research focus. Robotic platforms and automated systems are the important carriers of phenotyping monitoring sensors that enable large-scale screening. Through the diverse design and flexible systems, an efficient operation can be achieved across a range of experimental and field platforms. The combination of robot technology and plant phenotyping monitoring tools provides the data to inform novel artificial intelligence (AI) approaches that will provide steppingstones for new research breakthroughs. Therefore, this article introduces robotics and eco-phenotyping and examines research significant to this novel domain of plant eco-phenotyping. Given the monitoring scenarios of phenotyping information at different scales, the used intelligent robot technology, efficient automation platform, and advanced sensor equipment are summarized in detail. We further discuss the challenges posed to current research as well as the future developmental trends in the application of robot technology and plant eco-phenotyping. These include the use of collected data for AI applications and high-bandwidth data transfer, and large well-structured (meta) data storage approaches in plant sciences and agriculture.

High-Throughput Plant Phenotyping Platform (HT3P) as a Novel Tool for Estimating Agronomic Traits From the Lab to the Field

Food scarcity, population growth, and global climate change have propelled crop yield growth driven by high-throughput phenotyping into the era of big data. However, access to large-scale phenotypic data has now become a critical barrier that phenomics urgently must overcome. Fortunately, the high-throughput plant phenotyping platform (HT3P), employing advanced sensors and data collection systems, can take full advantage of non-destructive and high-throughput methods to monitor, quantify, and evaluate specific phenotypes for large-scale agricultural experiments, and it can effectively perform phenotypic tasks that traditional phenotyping could not do. In this way, HT3Ps are novel and powerful tools, for which various commercial, customized, and even self-developed ones have been recently introduced in rising numbers. Here, we review these HT3Ps in nearly 7 years from greenhouses and growth chambers to the field, and from ground-based proximal phenotyping to aerial large-scale remote sensing. Platform configurations, novelties, operating modes, current developments, as well the strengths and weaknesses of diverse types of HT3Ps are thoroughly and clearly described. Then, miscellaneous combinations of HT3Ps for comparative validation and comprehensive analysis are systematically present, for the first time. Finally, we consider current phenotypic challenges and provide fresh perspectives on future development trends of HT3Ps. This review aims to provide ideas, thoughts, and insights for the optimal selection, exploitation, and utilization of HT3Ps, and thereby pave the way to break through current phenotyping bottlenecks in botany.

High-throughput phenotyping analysis of maize at the seedling stage using end-to-end segmentation network

Image processing technologies are available for high-throughput acquisition and analysis of phenotypes for crop populations, which is of great significance for crop growth monitoring, evaluation of seedling condition, and cultivation management. However, existing methods rely on empirical segmentation thresholds, thus can have insufficient accuracy of extracted phenotypes. Taking maize as an example crop, we propose a phenotype extraction approach from top-view images at the seedling stage. An end-to-end segmentation network, named PlantU-net, which uses a small amount of training data, was explored to realize automatic segmentation of top-view images of a maize population at the seedling stage. Morphological and color related phenotypes were automatic extracted, including maize shoot coverage, circumscribed radius, aspect ratio, and plant azimuth plane angle. The results show that the approach can segment the shoots at the seedling stage from top-view images, obtained either from the UAV or tractor-based high-throughput phenotyping platform. The average segmentation accuracy, recall rate, and F1 score are 0.96, 0.98, and 0.97, respectively. The extracted phenotypes, including maize shoot coverage, circumscribed radius, aspect ratio, and plant azimuth plane angle, are highly correlated with manual measurements (R2 = 0.96-0.99). This approach requires less training data and thus has better expansibility. It provides practical means for high-throughput phenotyping analysis of early growth stage crop populations.

WinRoots: A High-Throughput Cultivation and Phenotyping System for Plant Phenomics Studies Under Soil Stress

Soil stress, such as salinity, is a primary cause of global crop yield reduction. Existing crop phenotyping platforms cannot fully meet the specific needs of phenomics studies of plant response to soil stress in terms of throughput, environmental controllability, or root phenotypic acquisition. Here, we report the WinRoots, a low-cost and high-throughput plant soil cultivation and phenotyping system that can provide uniform, controlled soil stress conditions and accurately quantify the whole-plant phenome, including roots. Using soybean seedlings exposed to salt stress as an example, we demonstrate the uniformity and controllability of the soil environment in this system. A high-throughput multiple-phenotypic assay among 178 soybean cultivars reveals that the cotyledon character can serve as a non-destructive indicator of the whole-seedling salt tolerance. Our results demonstrate that WinRoots is an effective tool for high-throughput plant cultivation and soil stress phenomics studies.

Leveraging Image Analysis to Compute 3D Plant Phenotypes Based on Voxel-Grid Plant Reconstruction

High throughput image-based plant phenotyping facilitates the extraction of morphological and biophysical traits of a large number of plants non-invasively in a relatively short time. It facilitates the computation of advanced phenotypes by considering the plant as a single object (holistic phenotypes) or its components, i.e., leaves and the stem (component phenotypes). The architectural complexity of plants increases over time due to variations in self-occlusions and phyllotaxy, i.e., arrangements of leaves around the stem. One of the central challenges to computing phenotypes from 2-dimensional (2D) single view images of plants, especially at the advanced vegetative stage in presence of self-occluding leaves, is that the information captured in 2D images is incomplete, and hence, the computed phenotypes are inaccurate. We introduce a novel algorithm to compute 3-dimensional (3D) plant phenotypes from multiview images using voxel-grid reconstruction of the plant (3DPhenoMV). The paper also presents a novel method to reliably detect and separate the individual leaves and the stem from the 3D voxel-grid of the plant using voxel overlapping consistency check and point cloud clustering techniques. To evaluate the performance of the proposed algorithm, we introduce the University of Nebraska-Lincoln 3D Plant Phenotyping Dataset (UNL-3DPPD). A generic taxonomy of 3D image-based plant phenotypes are also presented to promote 3D plant phenotyping research. A subset of these phenotypes are computed using computer vision algorithms with discussion of their significance in the context of plant science. The central contributions of the paper are (a) an algorithm for 3D voxel-grid reconstruction of maize plants at the advanced vegetative stages using images from multiple 2D views; (b) a generic taxonomy of 3D image-based plant phenotypes and a public benchmark dataset, i.e., UNL-3DPPD, to promote the development of 3D image-based plant phenotyping research; and (c) novel voxel overlapping consistency check and point cloud clustering techniques to detect and isolate individual leaves and stem of the maize plants to compute the component phenotypes. Detailed experimental analyses demonstrate the efficacy of the proposed method, and also show the potential of 3D phenotypes to explain the morphological characteristics of plants regulated by genetic and environmental interactions.

Tomato Fruit Detection and Counting in Greenhouses Using Deep Learning

Accurately detecting and counting fruits during plant growth using imaging and computer vision is of importance not only from the point of view of reducing labor intensive manual measurements of phenotypic information, but also because it is a critical step toward automating processes such as harvesting. Deep learning based methods have emerged as the state-of-the-art techniques in many problems in image segmentation and classification, and have a lot of promise in challenging domains such as agriculture, where they can deal with the large variability in data better than classical computer vision methods. This paper reports results on the detection of tomatoes in images taken in a greenhouse, using the MaskRCNN algorithm, which detects objects and also the pixels corresponding to each object. Our experimental results on the detection of tomatoes from images taken in greenhouses using a RealSense camera are comparable to or better than the metrics reported by earlier work, even though those were obtained in laboratory conditions or using higher resolution images. Our results also show that MaskRCNN can implicitly learn object depth, which is necessary for background elimination.

Voxel carving-based 3D reconstruction of sorghum identifies genetic determinants of light interception efficiency

Changes in canopy architecture traits have been shown to contribute to yield increases. Optimizing both light interception and light interception efficiency of agricultural crop canopies will be essential to meeting the growing food needs. Canopy architecture is inherently three-dimensional (3D), but many approaches to measuring canopy architecture component traits treat the canopy as a two-dimensional (2D) structure to make large scale measurement, selective breeding, and gene identification logistically feasible. We develop a high throughput voxel carving strategy to reconstruct 3D representations of sorghum from a small number of RGB photos. Our approach builds on the voxel carving algorithm to allow for fully automatic reconstruction of hundreds of plants. It was employed to generate 3D reconstructions of individual plants within a sorghum association population at the late vegetative stage of development. Light interception parameters estimated from these reconstructions enabled the identification of known and previously unreported loci controlling light interception efficiency in sorghum. The approach is generalizable and scalable, and it enables 3D reconstructions from existing plant high throughput phenotyping datasets. We also propose a set of best practices to increase 3D reconstructions' accuracy.

Rapid Screening of Nitrogen Use Efficiency in Perennial Ryegrass (Lolium perenne L.) Using Automated Image-Based Phenotyping

Perennial ryegrass (Lolium perenne L.) is a dominant species in temperate Australian pastures. Currently, nitrogenous fertilizers are used to support herbage production for pasture and fodder. Increasing the nitrogen use efficiency (NUE) of pasture grasses could decrease the amount of fertilizer application and reduce nitrogen (N) leaching into the environment. NUE, defined as units of dry matter production per unit of supplied nitrogen, is a complex trait in which genomic selection may provide a promising strategy in breeding. Our objective was to develop a rapid, high-throughput screening method to enable genomic selection for y -60NUE in perennial ryegrass. NUE of 76 genotypes of perennial ryegrass from a breeding population were screened in a greenhouse using an automated image-based phenomics platform under low (0.5 mM) and moderate (5 mM) N levels over 3 consecutive harvests. Significant (p < 0.05) genotype, treatment, and genotype by treatment interactions for dry matter yield and NUE were observed. NUE under low and moderate N treatments was significantly correlated. Of the seven plant architecture features directly extracted from image analysis and four secondarily derived measures, mean projected plant area (MPPA) from the two side view images had the highest correlation with dry matter yield (r = 0.94). Automated digital image-based phenotyping enables temporal plant growth responses to N to be measured efficiently and non-destructively. The method developed in this study would be suitable for screening large populations of perennial ryegrass growth in response to N for genomic selection purposes.

Super resolution for root imaging

PREMISE

High-resolution cameras are very helpful for plant phenotyping as their images enable tasks such as target vs. background discrimination and the measurement and analysis of fine above-ground plant attributes. However, the acquisition of high-resolution images of plant roots is more challenging than above-ground data collection. An effective super-resolution (SR) algorithm is therefore needed for overcoming the resolution limitations of sensors, reducing storage space requirements, and boosting the performance of subsequent analyses.

METHODS

We propose an SR framework for enhancing images of plant roots using convolutional neural networks. We compare three alternatives for training the SR model: (i) training with non-plant-root images, (ii) training with plant-root images, and (iii) pretraining the model with non-plant-root images and fine-tuning with plant-root images. The architectures of the SR models were based on two state-of-the-art deep learning approaches: a fast SR convolutional neural network and an SR generative adversarial network.

RESULTS

In our experiments, we observed that the SR models improved the quality of low-resolution images of plant roots in an unseen data set in terms of the signal-to-noise ratio. We used a collection of publicly available data sets to demonstrate that the SR models outperform the basic bicubic interpolation, even when trained with non-root data sets.

DISCUSSION

The incorporation of a deep learning-based SR model in the imaging process enhances the quality of low-resolution images of plant roots. We demonstrate that SR preprocessing boosts the performance of a machine learning system trained to separate plant roots from their background. Our segmentation experiments also show that high performance on this task can be achieved independently of the signal-to-noise ratio. We therefore conclude that the quality of the image enhancement depends on the desired application.

Image Phenotyping of Spring Barley (Hordeum vulgare L.) RIL Population Under Drought: Selection of Traits and Biological Interpretation

Image-based phenotyping is a non-invasive method that permits the dynamic evaluation of plant features during growth, which is especially important for understanding plant adaptation and temporal dynamics of responses to environmental cues such as water deficit or drought. The aim of the present study was to use high-throughput imaging in order to assess the variation and dynamics of growth and development during drought in a spring barley population and to investigate associations between traits measured in time and yield-related traits measured after harvesting. Plant material covered recombinant inbred line population derived from a cross between European and Syrian cultivars. After placing the plants on the platform (28th day after sowing), drought stress was applied for 2 weeks. Top and side cameras were used to capture images daily that covered the visible range of the light spectrum, fluorescence signals, and the near infrared spectrum. The image processing provided 376 traits that were subjected to analysis. After 32 days of image phenotyping, the plants were cultivated in the greenhouse under optimal watering conditions until ripening, when several architecture and yield-related traits were measured. The applied data analysis approach, based on the clustering of image-derived traits into groups according to time profiles of statistical and genetic parameters, permitted to select traits representative for inference from the experiment. In particular, drought effects for 27 traits related to convex hull geometry, texture, proportion of brown pixels and chlorophyll intensity were found to be highly correlated with drought effects for spike traits and thousand grain weight.

Generating segmentation masks of herbarium specimens and a data set for training segmentation models using deep learning

PREMISE

Digitized images of herbarium specimens are highly diverse with many potential sources of visual noise and bias. The systematic removal of noise and minimization of bias must be achieved in order to generate biological insights based on the plants rather than the digitization and mounting practices involved. Here, we develop a workflow and data set of high-resolution image masks to segment plant tissues in herbarium specimen images and remove background pixels using deep learning.

METHODS AND RESULTS

We generated 400 curated, high-resolution masks of ferns using a combination of automatic and manual tools for image manipulation. We used those images to train a U-Net-style deep learning model for image segmentation, achieving a final Sørensen-Dice coefficient of 0.96. The resulting model can automatically, efficiently, and accurately segment massive data sets of digitized herbarium specimens, particularly for ferns.

CONCLUSIONS

The application of deep learning in herbarium sciences requires transparent and systematic protocols for generating training data so that these labor-intensive resources can be generalized to other deep learning applications. Segmentation ground-truth masks are hard-won data, and we share these data and the model openly in the hopes of furthering model training and transfer learning opportunities for broader herbarium applications.

Image-Based Assessment of Drought Response in Grapevines

Many plants can modify their leaf profile rapidly in response to environmental stress. Image-based data are increasingly used to retrieve reliable information on plant water status in a non-contact manner that has the potential to be scaled to high-throughput and repeated through time. This paper examined the variation of leaf angle as measured by both 3D images and goniometer in progressively drought stressed grapevine. Grapevines, grown in pots, were subjected to a 21-day period of drought stress receiving 100% (CTRL), 60% (IRR 60%) and 30% (IRR 30%) of maximum soil available water capacity. Leaf angle was (i) measured manually (goniometer) and (ii) computed by a 3D reconstruction method (multi-view stereo and structure from motion). Stomatal conductance, leaf water potential, fluorescence (F v /F m ), leaf area and 2D RGB data were simultaneously collected during drought imposition. Throughout the experiment, values of leaf water potential ranged from -0.4 (CTRL) to -1.1 MPa (IRR 30%) and it linearly influenced the leaf angle when measured manually (R 2 = 0.86) and with 3D image (R 2 = 0.73). Drought was negatively related to stomatal conductance and leaf area growth particularly in IRR 30% while photosynthetic parameters (i.e., F v /F m ) were not impaired by water restriction. A model for leaf area estimation based on the number of pixels of 2D RGB images developed at a different phenotyping robotized platform in a closely related experiment was successfully employed (R 2 = 0.78). At the end of the experiment, top view 2D RGB images showed a ∼50% reduction of greener fraction (GGF) in CTRL and IRR 60% vines compared to initial values, while GGF in IRR 30% increased by approximately 20%.

Skeletonization of Plant Point Cloud Data in Stochastic Optimization Framework.. [DOI: 10.1101/2020.02.15.950519] [Citation(s) in RCA: 0] [Impact Index Per Article: 0]

Skeleton extraction from 3D plant point cloud data is an essential prior for myriads of phenotyping studies. Although skeleton extraction from 3D shapes have been studied extensively in the computer vision and graphics literature, handling the case of plants is still an open problem. Drawbacks of the existing approaches include the zigzag structure of the skeleton, nonuniform density of skeleton points, lack of points in the areas having complex geometry structure, and most importantly the lack of biological relevance. With the aim to improve existing skeleton structures of state-of-the-art, we propose a stochastic framework which is supported by the biological structure of the original plant (we consider plants without any leaves). Initially we estimate the branching structure of the plant by the notion of β-splines to form a curve tree defined as a finite set of curves joined in a tree topology with certain level of smoothness. In the next phase, we force the discrete points in the curve tree to move towards the original point cloud by treating each point in the curve tree as a center of Gaussian, and points in the input cloud data as observations from the Gaussians. The task is to find the correct locations of the Gaussian centroids by maximizing a likelihood. The optimization technique is iterative and is based on the Expectation Maximization (EM) algorithm. The E-step estimates which Gaussian the observed point cloud was sampled from, and the M-step maximizes the negative log-likelihood that the observed points were sampled from the Gaussian Mixture Model (GMM) with respect to the model parameters. We experiment with several real world and synthetic datasets and demonstrate the robustness of the approach over the state-of-the-art.

Deep learning-based detection of seedling development

BACKGROUND

Monitoring the timing of seedling emergence and early development via high-throughput phenotyping with computer vision is a challenging topic of high interest in plant science. While most studies focus on the measurements of leaf area index or detection of specific events such as emergence, little attention has been put on the identification of kinetics of events of early seedling development on a seed to seed basis.

RESULT

Imaging systems screened the whole seedling growth process from the top view. Precise annotation of emergence out of the soil, cotyledon opening, and appearance of first leaf was conducted. This annotated data set served to train deep neural networks. Various strategies to incorporate in neural networks, the prior knowledge of the order of the developmental stages were investigated. Best results were obtained with a deep neural network followed with a long short term memory cell, which achieves more than 90% accuracy of correct detection.

CONCLUSION

This work provides a full pipeline of image processing and machine learning to classify three stages of plant growth plus soil on the different accessions of two species of red clover and alfalfa but which could easily be extended to other crops and other stages of development.

Skeletonization of Plant Point Cloud Data Using Stochastic Optimization Framework

Skeleton extraction from 3D plant point cloud data is an essential prior for myriads of phenotyping studies. Although skeleton extraction from 3D shapes have been studied extensively in the computer vision and graphics literature, handling the case of plants is still an open problem. Drawbacks of the existing approaches include the zigzag structure of the skeleton, nonuniform density of skeleton points, lack of points in the areas having complex geometry structure, and most importantly the lack of biological relevance. With the aim to improve existing skeleton structures of state-of-the-art, we propose a stochastic framework which is supported by the biological structure of the original plant (we consider plants without any leaves). Initially we estimate the branching structure of the plant by the notion of β-splines to form a curve tree defined as a finite set of curves joined in a tree topology with certain level of smoothness. In the next phase, we force the discrete points in the curve tree to move toward the original point cloud by treating each point in the curve tree as a center of Gaussian, and points in the input cloud data as observations from the Gaussians. The task is to find the correct locations of the Gaussian centroids by maximizing a likelihood. The optimization technique is iterative and is based on the Expectation Maximization (EM) algorithm. The E-step estimates which Gaussian the observed point cloud was sampled from, and the M-step maximizes the negative log-likelihood that the observed points were sampled from the Gaussian Mixture Model (GMM) with respect to the model parameters. We experiment with several real world and synthetic datasets and demonstrate the robustness of the approach over the state-of-the-art.

Applications and Trends of Machine Learning in Genomics and Phenomics for Next-Generation Breeding

Crops are the major source of food supply and raw materials for the processing industry. A balance between crop production and food consumption is continually threatened by plant diseases and adverse environmental conditions. This leads to serious losses every year and results in food shortages, particularly in developing countries. Presently, cutting-edge technologies for genome sequencing and phenotyping of crops combined with progress in computational sciences are leading a revolution in plant breeding, boosting the identification of the genetic basis of traits at a precision never reached before. In this frame, machine learning (ML) plays a pivotal role in data-mining and analysis, providing relevant information for decision-making towards achieving breeding targets. To this end, we summarize the recent progress in next-generation sequencing and the role of phenotyping technologies in genomics-assisted breeding toward the exploitation of the natural variation and the identification of target genes. We also explore the application of ML in managing big data and predictive models, reporting a case study using microRNAs (miRNAs) to identify genes related to stress conditions.