A novel 3D imaging system for strawberry phenotyping

Three-dimensional fruit growth analysis clarifies developmental mechanisms underlying complex shape diversity in persimmon fruit

The determination of fruit size and shape are of considerable interest in horticulture and developmental biology. Fruit typically exhibits three-dimensional structures characterized by geometric features that are dependent on the genotype. Although minor developmental variations have been recognized, few studies have fully visualized and measured these variations throughout fruit growth. Here, a high-resolution 3D scanner was used to investigate the fruit development of 51 persimmon (Diospyros kaki) cultivars with various complex shapes. We obtained 2380 3D models that fully represented fruit appearance, and enabled precise and automated measurements of shape features throughout fruit development, including horizontal and vertical grooves, length-to-width ratio, and roundness. The 3D fruit model analysis identified key stages that determined the shape attributes at maturity. Typically, genetic diversity was found in vertical groove development, and these grooves could be filled by tissue expansion in the carpel fusion zone during fruit development. In addition, transcriptome analysis of fruit tissues from groove and non-groove tissues revealed gene co-expression networks that were highly associated with groove depth variation. The presence of YABBY homologs was most closely associated with groove depth and indicated the possibility that this pathway is a key molecular contributor to vertical groove depth variation. Overall, our results revealed deterministic patterns of complex shape traits in persimmon fruit and showed that different growth patterns among tissues are the main factor contributing to the shape of both vertical and horizontal grooves.

Extraction of soybean plant trait parameters based on SfM-MVS algorithm combined with GRNN

Soybean is an important grain and oil crop worldwide and is rich in nutritional value. Phenotypic morphology plays an important role in the selection and breeding of excellent soybean varieties to achieve high yield. Nowadays, the mainstream manual phenotypic measurement has some problems such as strong subjectivity, high labor intensity and slow speed. To address the problems, a three-dimensional (3D) reconstruction method for soybean plants based on structure from motion (SFM) was proposed. First, the 3D point cloud of a soybean plant was reconstructed from multi-view images obtained by a smartphone based on the SFM algorithm. Second, low-pass filtering, Gaussian filtering, Ordinary Least Square (OLS) plane fitting, and Laplacian smoothing were used in fusion to automatically segment point cloud data, such as individual plants, stems, and leaves. Finally, Eleven morphological traits, such as plant height, minimum bounding box volume per plant, leaf projection area, leaf projection length and width, and leaf tilt information, were accurately and nondestructively measured by the proposed an algorithm for leaf phenotype measurement (LPM). Moreover, Support Vector Machine (SVM), Back Propagation Neural Network (BP), and Back Propagation Neural Network (GRNN) prediction models were established to predict and identify soybean plant varieties. The results indicated that, compared with the manual measurement, the root mean square error (RMSE) of plant height, leaf length, and leaf width were 0.9997, 0.2357, and 0.2666 cm, and the mean absolute percentage error (MAPE) were 2.7013%, 1.4706%, and 1.8669%, and the coefficients of determination (R2) were 0.9775, 0.9785, and 0.9487, respectively. The accuracy of predicting plant species according to the six leaf parameters was highest when using GRNN, reaching 0.9211, and the RMSE was 18.3263. Based on the phenotypic traits of plants, the differences between C3, 47-6 and W82 soybeans were analyzed genetically, and because C3 was an insect-resistant line, the trait parametes (minimum box volume per plant, number of leaves, minimum size of single leaf box, leaf projection area).The results show that the proposed method can effectively extract the 3D phenotypic structure information of soybean plants and leaves without loss which has the potential using ability in other plants with dense leaves.

Towards smart and sustainable development of modern berry cultivars in Europe

Fresh berries are a popular and important component of the human diet. The demand for high-quality berries and sustainable production methods is increasing globally, challenging breeders to develop modern berry cultivars that fulfill all desired characteristics. Since 1994, research projects have characterized genetic resources, developed modern tools for high-throughput screening, and published data in publicly available repositories. However, the key findings of different disciplines are rarely linked together, and only a limited range of traits and genotypes has been investigated. The Horizon2020 project BreedingValue will address these challenges by studying a broader panel of strawberry, raspberry and blueberry genotypes in detail, in order to recover the lost genetic diversity that has limited the aroma and flavor intensity of recent cultivars. We will combine metabolic analysis with sensory panel tests and surveys to identify the key components of taste, flavor and aroma in berries across Europe, leading to a high-resolution map of quality requirements for future berry cultivars. Traits linked to berry yields and the effect of environmental stress will be investigated using modern image analysis methods and modeling. We will also use genetic analysis to determine the genetic basis of complex traits for the development and optimization of modern breeding technologies, such as molecular marker arrays, genomic selection and genome-wide association studies. Finally, the results, raw data and metadata will be made publicly available on the open platform Germinate in order to meet FAIR data principles and provide the basis for sustainable research in the future.

Introducing Three-Dimensional Scanning for Phenotyping of Olive Fruits Based on an Extensive Germplasm Survey

Morphological characterization of olive (Olea europaea L.) varieties to detect desirable traits has been based on the training of expert panels and implementation of laborious multiyear measurements with limitations in accuracy and throughput of measurements. The present study compares two- and three-dimensional imaging systems for phenotyping a large dataset of 50 olive varieties maintained in the National Germplasm Depository of Greece, employing this technology for the first time in olive fruit and endocarps. The olive varieties employed for the present study exhibited high phenotypic variation, particularly for the endocarp shadow area, which ranged from 0.17−3.34 cm2 as evaluated via 2D and 0.32−2.59 cm2 as determined by 3D scanning. We found significant positive correlations (p < 0.001) between the two methods for eight quantitative morphological traits using the Pearson correlation coefficient. The highest correlation between the two methods was detected for the endocarp length (r = 1) and width (r = 1) followed by the fruit length (r = 0.9865), mucro length (r = 0.9631), fruit shadow area (r = 0.9573), fruit width (r = 0.9480), nipple length (r = 0.9441), and endocarp area (r = 0.9184). The present study unraveled novel morphological indicators of olive fruits and endocarps such as volume, total area, up- and down-skin area, and center of gravity using 3D scanning. The highest volume and area regarding both endocarp and fruit were observed for ‘Gaidourelia’. This methodology could be integrated into existing olive breeding programs, especially when the speed of scanning increases. Another potential future application could be assessing olive fruit quality on the trees or in the processing facilities.

Response of Population Canopy Color Gradation Skewed Distribution Parameters of the RGB Model to Micrometeorology Environment in Begonia Fimbristipula Hance

The high quality and efficient production of greenhouse vegetation depend on micrometeorology environmental adjusting such as system warming and illumination supplement. In order to improve the quantity, quality, and efficiency of greenhouse vegetation, it is necessary to figure out the relationship between the crop growth conditions and environmental meteorological factors, which could give constructive suggestions for precise control of the greenhouse environment and reduce the running costs. The parameters from the color information of the plant canopy reflect the internal physiological conditions, thus, the RGB model has been widely used in the color analysis of digital pictures of leaves. We take photographs of Begonia Fimbristipula Hance (BFH) growing in the greenhouse at a fixed time every day and measure the meteorological factors. The results showed that the color scale for the single leaf, single plant, and the populated canopy of the BFH photographs all have skewed cumulative distribution histograms. The color gradation skewness-distribution (CGSD) parameters of the RGB model were increased from 4 to 20 after the skewness analysis, which greatly expanded the canopy leaf color information and could simultaneously describe the depth and distribution characteristics of the canopy color. The 20 CGSD parameters were sensitive to the micrometeorology factors, especially to the radiation and temperature accumulation. The multiple regression models of mean, median, mode, and kurtosis parameters to microclimate factors were established, and the spatial models of skewness parameters were optimized. The models can well explain the response of canopy color to microclimate factors and can be used to monitor the variation of plant canopy color under different micrometeorology.

Development of Multiple UAV Collaborative Driving Systems for Improving Field Phenotyping

Unmanned aerial vehicle-based remote sensing technology has recently been widely applied to crop monitoring due to the rapid development of unmanned aerial vehicles, and these technologies have considerable potential in smart agriculture applications. Field phenotyping using remote sensing is mostly performed using unmanned aerial vehicles equipped with RGB cameras or multispectral cameras. For accurate field phenotyping for precision agriculture, images taken from multiple perspectives need to be simultaneously collected, and phenotypic measurement errors may occur due to the movement of the drone and plants during flight. In this study, to minimize measurement error and improve the digital surface model, we proposed a collaborative driving system that allows multiple UAVs to simultaneously acquire images from different viewpoints. An integrated navigation system based on MAVSDK is configured for the attitude control and position control of unmanned aerial vehicles. Based on the leader–follower-based swarm driving algorithm and a long-range wireless network system, the follower drone cooperates with the leader drone to maintain a constant speed, direction, and image overlap ratio, and to maintain a rank to improve their phenotyping. A collision avoidance algorithm was developed because different UAVs can collide due to external disturbance (wind) when driving in groups while maintaining a rank. To verify and optimize the flight algorithm developed in this study in a virtual environment, a GAZEBO-based simulation environment was established. Based on the algorithm that has been verified and optimized in the previous simulation environment, some unmanned aerial vehicles were flown in the same flight path in a real field, and the simulation and the real field were compared. As a result of the comparative experiment, the simulated flight accuracy (RMSE) was 0.36 m and the actual field flight accuracy was 0.46 m, showing flight accuracy like that of a commercial program.

Estimation of Vegetative Growth in Strawberry Plants Using Mobile LiDAR Laser Scanner

Monitoring of plant vegetative growth can provide the basis for precise crop management. In this study, a 2D light detection and ranging (LiDAR) laser scanner, mounted on a linear conveyor, was used to acquire multi-temporal three-dimensional (3D) data from strawberry plants (‘Honeoye’ and ‘Malling Centenary’) 14–77 days after planting (DAP). Canopy geometrical variables, i.e., points per plant, height, ground projected area, and canopy volume profile, were extracted from 3D point cloud. The manually measured leaf area exhibited a linear relationship with LiDAR-derived parameters (R2 = 0.98, 0.90, 0.93, and 0.96 with number of points per plant, volume, height, and projected canopy area, respectively). However, the measuring uncertainty was high in the dense canopies. Particularly, the canopy volume estimation was adapted to the plant habitus to remove gaps and empty spaces in the canopy point cloud. The parametric values for maximum point to point distance (Dmax) = 0.15 cm and slice height (S) = 0.10 cm resulted in R² = 0.80 and RMSPE = 26.93% for strawberry plant volume estimation considering actual volume measured by water displacement. The vertical volume profiling provided growth data for cultivars ‘Honeoye’ and ‘Malling Centenary’ being 51.36 cm³ at 77 DAP and 42.18 cm3 at 70 DAP, respectively. The results contribute an approach for estimating plant geometrical features and particularly strawberry canopy volume profile based on LiDAR point cloud for tracking plant growth.

A miniaturized phenotyping platform for individual plants using multi-view stereo 3D reconstruction

Plant phenotyping is essential in plant breeding and management. High-throughput data acquisition and automatic phenotypes extraction are common concerns in plant phenotyping. Despite the development of phenotyping platforms and the realization of high-throughput three-dimensional (3D) data acquisition in tall plants, such as maize, handling small-size plants with complex structural features remains a challenge. This study developed a miniaturized shoot phenotyping platform MVS-Pheno V2 focusing on low plant shoots. The platform is an improvement of MVS-Pheno V1 and was developed based on multi-view stereo 3D reconstruction. It has the following four components: Hardware, wireless communication and control, data acquisition system, and data processing system. The hardware sets the rotation on top of the platform, separating plants to be static while rotating. A novel local network was established to realize wireless communication and control; thus, preventing cable twining. The data processing system was developed to calibrate point clouds and extract phenotypes, including plant height, leaf area, projected area, shoot volume, and compactness. This study used three cultivars of wheat shoots at four growth stages to test the performance of the platform. The mean absolute percentage error of point cloud calibration was 0.585%. The squared correlation coefficient R ² was 0.9991, 0.9949, and 0.9693 for plant height, leaf length, and leaf width, respectively. The root mean squared error (RMSE) was 0.6996, 0.4531, and 0.1174 cm for plant height, leaf length, and leaf width. The MVS-Pheno V2 platform provides an alternative solution for high-throughput phenotyping of low individual plants and is especially suitable for shoot architecture-related plant breeding and management studies.

Identification and utilization of genetic determinants of trait measurement errors in image-based, high-throughput phenotyping

The accuracy of trait measurements greatly affects the quality of genetic analyses. During automated phenotyping, trait measurement errors, i.e. differences between automatically extracted trait values and ground truth, are often treated as random effects that can be controlled by increasing population sizes and/or replication number. In contrast, there is some evidence that trait measurement errors may be partially under genetic control. Consistent with this hypothesis, we observed substantial nonrandom, genetic contributions to trait measurement errors for five maize (Zea mays) tassel traits collected using an image-based phenotyping platform. The phenotyping accuracy varied according to whether a tassel exhibited "open" versus. "closed" branching architecture, which is itself under genetic control. Trait-associated SNPs (TASs) identified via genome-wide association studies (GWASs) conducted on five tassel traits that had been phenotyped both manually (i.e. ground truth) and via feature extraction from images exhibit little overlap. Furthermore, identification of TASs from GWASs conducted on the differences between the two values indicated that a fraction of measurement error is under genetic control. Similar results were obtained in a sorghum (Sorghum bicolor) plant height dataset, demonstrating that trait measurement error is genetically determined in multiple species and traits. Trait measurement bias cannot be controlled by increasing population size and/or replication number.

Identification for surrogate drought tolerance in maize inbred lines utilizing high-throughput phenomics approach

Screening for drought tolerance requires precise techniques like phonemics, which is an emerging science aimed at non-destructive methods allowing large-scale screening of genotypes. Large-scale screening complements genomic efforts to identify genes relevant for crop improvement. Thirty maize inbred lines from various sources (exotic and indigenous) maintained at Dryland Agriculture Research Station were used in the current study. In the automated plant transport and imaging systems (LemnaTec Scanalyzer system for large plants), top and side view images were taken of the VIS (visible) and NIR (near infrared) range of the light spectrum to capture phenes. All images were obtained with a thermal imager. All sensors were used to collect images one day after shifting the pots from the greenhouse for 11 days. Image processing was done using pre-processing, segmentation and flowered by features' extraction. Different surrogate traits such as pixel area, plant aspect ratio, convex hull ratio and calliper length were estimated. A strong association was found between canopy temperature and above ground biomass under stress conditions. Promising lines in different surrogates will be utilized in breeding programmes to develop mapping populations for traits of interest related to drought resilience, in terms of improved tissue water status and mapping of genes/QTLs for drought traits.

Multivariate Analysis Approaches for Dimension and Shape Discrimination of Vitis vinifera Varieties

In this study, berry dimensions and shape traits, which are important for the design of the grape processing system and the classification of 10 different grape varieties grown in same ecological conditions (‘Ata Sarısı’, ‘Barış’, ‘Dımışkı’, ‘Hatun Parmağı’, ‘Helvani’, ‘Horoz Karası’, ‘Hönüsü’, ‘İtalia’, ‘Mevlana Sarısı’, and ‘Red Globe’) were determined; differences between the varieties were identified with the use of discriminant analysis. The largest grape varieties were identified as ‘Ata Sarısı’ and ‘Red Globe’. The ‘Red Globe’ and ‘Helvani’ varieties had geometrically sphere-like shape. The ‘Barış’ variety had the lowest size averages. According to elliptic Fourier analysis, the primary source of shape variation was ellipse and sphere-looking varieties. However, shape variation was seen due to the existence of a small number of drop-like varieties. According to discriminant analysis, shape differences of the varieties were defined by two discriminant functions. Based on these discriminant functions, the greatest classification performance was achieved for ‘Mevlana Sarısı’ and ‘Dımışkı’. In scatter plots, three shape definitions (sphere, ellipse, and drop) were made for grape varieties. Cluster analysis revealed 4 sub-groups. The first sub-group included the ‘Mevlana Sarısı’ variety; the second sub-group included the ‘Hönüsü’, ‘Hatun Parmağı’, ‘Dımışkı’, and ‘Horoz Karası’ varieties; the third sub-group included the ‘Ata Sarısı’ variety; the fourth sub-group included the ‘Barış’, ‘Helvani’, ‘İtalia’, and ‘Red Globe’ varieties. The variety in the first group had a geometrically ellipse-like shape, the largest length, and the smallest width. The size data were the smallest for the second sub-group. The third sub-group, with the ellipse-like shape, had the large size data. The grape varieties the closest to the sphere were classified in the fourth group, and these varieties had the large sizes.

Novel 3D Imaging Systems for High-Throughput Phenotyping of Plants

The use of 3D plant models for high-throughput phenotyping is increasingly becoming a preferred method for many plant science researchers. Numerous camera-based imaging systems and reconstruction algorithms have been developed for the 3D reconstruction of plants. However, it is still challenging to build an imaging system with high-quality results at a low cost. Useful comparative information for existing imaging systems and their improvements is also limited, making it challenging for researchers to make data-based selections. The objective of this study is to explore the possible solutions to address these issues. We introduce two novel systems for plants of various sizes, as well as a pipeline to generate high-quality 3D point clouds and meshes. The higher accuracy and efficiency of the proposed systems make it a potentially valuable tool for enhancing high-throughput phenotyping by integrating 3D traits for increased resolution and measuring traits that are not amenable to 2D imaging approaches. The study shows that the phenotype traits derived from the 3D models are highly correlated with manually measured phenotypic traits (R2 > 0.91). Moreover, we present a systematic analysis of different settings of the imaging systems and a comparison with the traditional system, which provide recommendations for plant scientists to improve the accuracy of 3D construction. In summary, our proposed imaging systems are suggested for 3D reconstruction of plants. Moreover, the analysis results of the different settings in this paper can be used for designing new customized imaging systems and improving their accuracy.

Automatic Fruit Morphology Phenome and Genetic Analysis: An Application in the Octoploid Strawberry

Automatizing phenotype measurement will decisively contribute to increase plant breeding efficiency. Among phenotypes, morphological traits are relevant in many fruit breeding programs, as appearance influences consumer preference. Often, these traits are manually or semiautomatically obtained. Yet, fruit morphology evaluation can be enhanced using fully automatized procedures and digital images provide a cost-effective opportunity for this purpose. Here, we present an automatized pipeline for comprehensive phenomic and genetic analysis of morphology traits extracted from internal and external strawberry (Fragaria x ananassa) images. The pipeline segments, classifies, and labels the images and extracts conformation features, including linear (area, perimeter, height, width, circularity, shape descriptor, ratio between height and width) and multivariate (Fourier elliptical components and Generalized Procrustes) statistics. Internal color patterns are obtained using an autoencoder to smooth out the image. In addition, we develop a variational autoencoder to automatically detect the most likely number of underlying shapes. Bayesian modeling is employed to estimate both additive and dominance effects for all traits. As expected, conformational traits are clearly heritable. Interestingly, dominance variance is higher than the additive component for most of the traits. Overall, we show that fruit shape and color can be quickly and automatically evaluated and are moderately heritable. Although we study strawberry images, the algorithm can be applied to other fruits, as shown in the GitHub repository.

Strawberry fruit shape: quantification by image analysis and QTL detection by genome-wide association analysis

Fruit shape of cultivated strawberry (Fragaria × ananassa Duch.) is an important breeding target. To detect genomic regions associated with this trait, its quantitative evaluation is needed. Previously we created a multi-parent advanced-generation inter-cross (MAGIC) strawberry population derived from six founder parents. In this study, we used this population to quantify fruit shape. Elliptic Fourier descriptors (EFDs) were generated from 2 969 two-dimensional binarized fruit images, and principal component (PC) scores were calculated on the basis of the EFD coefficients. PC1-PC3 explained 96% of variation in shape and thus adequately quantified it. In genome-wide association study, the PC scores were used as phenotypes. Genome wide association study using mixed linear models revealed 2 quantitative trait loci (QTLs) for fruit shape. Our results provide a novel and effective method to analyze strawberry fruit morphology; the detected QTLs and presented method can support marker-assisted selection in practical breeding programs to improve fruit shape.

Remote Sensing and Machine Learning in Crop Phenotyping and Management, with an Emphasis on Applications in Strawberry Farming

Measurement of plant characteristics is still the primary bottleneck in both plant breeding and crop management. Rapid and accurate acquisition of information about large plant populations is critical for monitoring plant health and dissecting the underlying genetic traits. In recent years, high-throughput phenotyping technology has benefitted immensely from both remote sensing and machine learning. Simultaneous use of multiple sensors (e.g., high-resolution RGB, multispectral, hyperspectral, chlorophyll fluorescence, and light detection and ranging (LiDAR)) allows a range of spatial and spectral resolutions depending on the trait in question. Meanwhile, computer vision and machine learning methodology have emerged as powerful tools for extracting useful biological information from image data. Together, these tools allow the evaluation of various morphological, structural, biophysical, and biochemical traits. In this review, we focus on the recent development of phenomics approaches in strawberry farming, particularly those utilizing remote sensing and machine learning, with an eye toward future prospects for strawberries in precision agriculture. The research discussed is broadly categorized according to strawberry traits related to (1) fruit/flower detection, fruit maturity, fruit quality, internal fruit attributes, fruit shape, and yield prediction; (2) leaf and canopy attributes; (3) water stress; and (4) pest and disease detection. Finally, we present a synthesis of the potential research opportunities and directions that could further promote the use of remote sensing and machine learning in strawberry farming.

Application of Hyperspectral Imaging for Maturity and Soluble Solids Content Determination of Strawberry With Deep Learning Approaches

Maturity degree and quality evaluation are important for strawberry harvest, trade, and consumption. Deep learning has been an efficient artificial intelligence tool for food and agro-products. Hyperspectral imaging coupled with deep learning was applied to determine the maturity degree and soluble solids content (SSC) of strawberries with four maturity degrees. Hyperspectral image of each strawberry was obtained and preprocessed, and the spectra were extracted from the images. One-dimension residual neural network (1D ResNet) and three-dimension (3D) ResNet were built using 1D spectra and 3D hyperspectral image as inputs for maturity degree evaluation. Good performances were obtained for maturity identification, with the classification accuracy over 84% for both 1D ResNet and 3D ResNet. The corresponding saliency maps showed that the pigments related wavelengths and image regions contributed more to the maturity identification. For SSC determination, 1D ResNet model was also built, with the determination of coefficient (R ²) over 0.55 of the training, validation, and testing sets. The saliency maps of 1D ResNet for the SSC determination were also explored. The overall results showed that deep learning could be used to identify strawberry maturity degree and determine SSC. More efforts were needed to explore the use of 3D deep learning methods for the SSC determination. The close results of 1D ResNet and 3D ResNet for classification indicated that more samples might be used to improve the performances of 3D ResNet. The results in this study would help to develop 1D and 3D deep learning models for fruit quality inspection and other researches using hyperspectral imaging, providing efficient analysis approaches of fruit quality inspection using hyperspectral imaging.

Leveraging Image Analysis to Compute 3D Plant Phenotypes Based on Voxel-Grid Plant Reconstruction

High throughput image-based plant phenotyping facilitates the extraction of morphological and biophysical traits of a large number of plants non-invasively in a relatively short time. It facilitates the computation of advanced phenotypes by considering the plant as a single object (holistic phenotypes) or its components, i.e., leaves and the stem (component phenotypes). The architectural complexity of plants increases over time due to variations in self-occlusions and phyllotaxy, i.e., arrangements of leaves around the stem. One of the central challenges to computing phenotypes from 2-dimensional (2D) single view images of plants, especially at the advanced vegetative stage in presence of self-occluding leaves, is that the information captured in 2D images is incomplete, and hence, the computed phenotypes are inaccurate. We introduce a novel algorithm to compute 3-dimensional (3D) plant phenotypes from multiview images using voxel-grid reconstruction of the plant (3DPhenoMV). The paper also presents a novel method to reliably detect and separate the individual leaves and the stem from the 3D voxel-grid of the plant using voxel overlapping consistency check and point cloud clustering techniques. To evaluate the performance of the proposed algorithm, we introduce the University of Nebraska-Lincoln 3D Plant Phenotyping Dataset (UNL-3DPPD). A generic taxonomy of 3D image-based plant phenotypes are also presented to promote 3D plant phenotyping research. A subset of these phenotypes are computed using computer vision algorithms with discussion of their significance in the context of plant science. The central contributions of the paper are (a) an algorithm for 3D voxel-grid reconstruction of maize plants at the advanced vegetative stages using images from multiple 2D views; (b) a generic taxonomy of 3D image-based plant phenotypes and a public benchmark dataset, i.e., UNL-3DPPD, to promote the development of 3D image-based plant phenotyping research; and (c) novel voxel overlapping consistency check and point cloud clustering techniques to detect and isolate individual leaves and stem of the maize plants to compute the component phenotypes. Detailed experimental analyses demonstrate the efficacy of the proposed method, and also show the potential of 3D phenotypes to explain the morphological characteristics of plants regulated by genetic and environmental interactions.

Reconstruction method and optimum range of camera-shooting angle for 3D plant modeling using a multi-camera photography system

BACKGROUND

Measurement of plant structure is useful in monitoring plant conditions and understanding the responses of plants to environmental changes. 3D imaging technologies, especially the passive-SfM (Structure from Motion) algorithm combined with a multi-camera photography (MCP) system has been studied to measure plant structure due to its low-cost, close-range, and rapid image capturing ability. However, reconstruction of 3D plant models with complex structure is a time-consuming process and some systems have failed to reconstruct 3D models properly. Therefore, an MCP based SfM system was developed and an appropriate reconstruction method and optimal range of camera-shooting angles were investigated.

RESULTS

An MCP system which utilized 10 cameras and a rotary table for plant was developed. The 3D mesh model of a single leaf reconstruction using a set of images taken at each viewing zenith angle (VZA) from 12° (C2 camera) to 60° (C6 camera) by the MCP based SfM system had less undetected or unstable regions in comparison with other VZAs. The 3D mesh model of a whole plant, which merged 3D dense point cloud models built from a set of images taken at each appropriate VZA (Method 1), had high accuracy. The Method 1 error percentages for leaf area, leaf length, leaf width, stem height, and stem width are in the range of 2.6-4.4%, 0.2-2.2%, 1.0-4.9%, 1.9-2.8%, and 2.6-5.7% respectively. Also, the error of the leaf inclination angle was less than 5°. Conversely, the 3D mesh model of a whole plant built directly from a set of images taken at all appropriate VZAs (Method 2) had lower accuracy than that of Method 1. For Method 2, the error percentages of leaf area, leaf length, and leaf width are in the range of 3.1-13.3%, 0.4-3.3%, and 1.6-8.6%, respectively. It was difficult to obtain the error percentages of stem height and stem width because some information was missing in this model. In addition, the calculation time for Method 2 was 1.97 times longer computational time in comparison to Method 1.

CONCLUSIONS

In this study, we determined the optimal shooting angles on the MCP based SfM system developed. We found that it is better in terms of computational time and accuracy to merge partial 3D models from images taken at each appropriate VZA, then construct complete 3D model (Method 1), rather than to construct 3D model by using images taken at all appropriate VZAs (Method 2). This is because utilization of incorporation of incomplete images to match feature points could result in reduced accuracy in 3D models and the increase in computational time for 3D model reconstruction.

Defining strawberry shape uniformity using 3D imaging and genetic mapping

Strawberry shape uniformity is a complex trait, influenced by multiple genetic and environmental components. To complicate matters further, the phenotypic assessment of strawberry uniformity is confounded by the difficulty of quantifying geometric parameters 'by eye' and variation between assessors. An in-depth genetic analysis of strawberry uniformity has not been undertaken to date, due to the lack of accurate and objective data. Nonetheless, uniformity remains one of the most important fruit quality selection criteria for the development of a new variety. In this study, a 3D-imaging approach was developed to characterise berry shape uniformity. We show that circularity of the maximum circumference had the closest predictive relationship with the manual uniformity score. Combining five or six automated metrics provided the best predictive model, indicating that human assessment of uniformity is highly complex. Furthermore, visual assessment of strawberry fruit quality in a multi-parental QTL mapping population has allowed the identification of genetic components controlling uniformity. A "regular shape" QTL was identified and found to be associated with three uniformity metrics. The QTL was present across a wide array of germplasm, indicating a potential candidate for marker-assisted breeding, while the potential to implement genomic selection is explored. A greater understanding of berry uniformity has been achieved through the study of the relative impact of automated metrics on human perceived uniformity. Furthermore, the comprehensive definition of strawberry shape uniformity using 3D imaging tools has allowed precision phenotyping, which has improved the accuracy of trait quantification and unlocked the ability to accurately select for uniform berries.

MVS-Pheno: A Portable and Low-Cost Phenotyping Platform for Maize Shoots Using Multiview Stereo 3D Reconstruction

Plant phenotyping technologies play important roles in plant research and agriculture. Detailed phenotypes of individual plants can guide the optimization of shoot architecture for plant breeding and are useful to analyze the morphological differences in response to environments for crop cultivation. Accordingly, high-throughput phenotyping technologies for individual plants grown in field conditions are urgently needed, and MVS-Pheno, a portable and low-cost phenotyping platform for individual plants, was developed. The platform is composed of four major components: a semiautomatic multiview stereo (MVS) image acquisition device, a data acquisition console, data processing and phenotype extraction software for maize shoots, and a data management system. The platform's device is detachable and adjustable according to the size of the target shoot. Image sequences for each maize shoot can be captured within 60-120 seconds, yielding 3D point clouds of shoots are reconstructed using MVS-based commercial software, and the phenotypic traits at the organ and individual plant levels are then extracted by the software. The correlation coefficient (R 2) between the extracted and manually measured plant height, leaf width, and leaf area values are 0.99, 0.87, and 0.93, respectively. A data management system has also been developed to store and manage the acquired raw data, reconstructed point clouds, agronomic information, and resulting phenotypic traits. The platform offers an optional solution for high-throughput phenotyping of field-grown plants, which is especially useful for large populations or experiments across many different ecological regions.

PI-Plat: a high-resolution image-based 3D reconstruction method to estimate growth dynamics of rice inflorescence traits

BACKGROUND

Recent advances in image-based plant phenotyping have improved our capability to study vegetative stage growth dynamics. However, more complex agronomic traits such as inflorescence architecture (IA), which predominantly contributes to grain crop yield are more challenging to quantify and hence are relatively less explored. Previous efforts to estimate inflorescence-related traits using image-based phenotyping have been limited to destructive end-point measurements. Development of non-destructive inflorescence phenotyping platforms could accelerate the discovery of the phenotypic variation with respect to inflorescence dynamics and mapping of the underlying genes regulating critical yield components.

RESULTS

The major objective of this study is to evaluate post-fertilization development and growth dynamics of inflorescence at high spatial and temporal resolution in rice. For this, we developed the Panicle Imaging Platform (PI-Plat) to comprehend multi-dimensional features of IA in a non-destructive manner. We used 11 rice genotypes to capture multi-view images of primary panicle on weekly basis after the fertilization. These images were used to reconstruct a 3D point cloud of the panicle, which enabled us to extract digital traits such as voxel count and color intensity. We found that the voxel count of developing panicles is positively correlated with seed number and weight at maturity. The voxel count from developing panicles projected overall volumes that increased during the grain filling phase, wherein quantification of color intensity estimated the rate of panicle maturation. Our 3D based phenotyping solution showed superior performance compared to conventional 2D based approaches.

CONCLUSIONS

For harnessing the potential of the existing genetic resources, we need a comprehensive understanding of the genotype-to-phenotype relationship. Relatively low-cost sequencing platforms have facilitated high-throughput genotyping, while phenotyping, especially for complex traits, has posed major challenges for crop improvement. PI-Plat offers a low cost and high-resolution platform to phenotype inflorescence-related traits using 3D reconstruction-based approach. Further, the non-destructive nature of the platform facilitates analyses of the same panicle at multiple developmental time points, which can be utilized to explore the genetic variation for dynamic inflorescence traits in cereals.

Measuring crops in 3D: using geometry for plant phenotyping

Using 3D sensing for plant phenotyping has risen within the last years. This review provides an overview on 3D traits for the demands of plant phenotyping considering different measuring techniques, derived traits and use-cases of biological applications. A comparison between a high resolution 3D measuring device and an established measuring tool, the leaf meter, is shown to categorize the possible measurement accuracy. Furthermore, different measuring techniques such as laser triangulation, structure from motion, time-of-flight, terrestrial laser scanning or structured light approaches enable the assessment of plant traits such as leaf width and length, plant size, volume and development on plant and organ level. The introduced traits were shown with respect to the measured plant types, the used measuring technique and the link to their biological use case. These were trait and growth analysis for measurements over time as well as more complex investigation on water budget, drought responses and QTL (quantitative trait loci) analysis. The used processing pipelines were generalized in a 3D point cloud processing workflow showing the single processing steps to derive plant parameters on plant level, on organ level using machine learning or over time using time series measurements. Finally the next step in plant sensing, the fusion of different sensor types namely 3D and spectral measurements is introduced by an example on sugar beet. This multi-dimensional plant model is the key to model the influence of geometry on radiometric measurements and to correct it. This publication depicts the state of the art for 3D measuring of plant traits as they were used in plant phenotyping regarding how the data is acquired, how this data is processed and what kind of traits is measured at the single plant, the miniplot, the experimental field and the open field scale. Future research will focus on highly resolved point clouds on the experimental and field scale as well as on the automated trait extraction of organ traits to track organ development at these scales.

Crop Phenomics: Current Status and Perspectives

Reliable, automatic, multifunctional, and high-throughput phenotypic technologies are increasingly considered important tools for rapid advancement of genetic gain in breeding programs. With the rapid development in high-throughput phenotyping technologies, research in this area is entering a new era called 'phenomics.' The crop phenotyping community not only needs to build a multi-domain, multi-level, and multi-scale crop phenotyping big database, but also to research technical systems for phenotypic traits identification and develop bioinformatics technologies for information extraction from the overwhelming amounts of omics data. Here, we provide an overview of crop phenomics research, focusing on two parts, from phenotypic data collection through various sensors to phenomics analysis. Finally, we discussed the challenges and prospective of crop phenomics in order to provide suggestions to develop new methods of mining genes associated with important agronomic traits, and propose new intelligent solutions for precision breeding.

Leveraging Image Analysis for High-Throughput Plant Phenotyping

The complex interaction between a genotype and its environment controls the biophysical properties of a plant, manifested in observable traits, i.e., plant's phenome, which influences resources acquisition, performance, and yield. High-throughput automated image-based plant phenotyping refers to the sensing and quantifying plant traits non-destructively by analyzing images captured at regular intervals and with precision. While phenomic research has drawn significant attention in the last decade, extracting meaningful and reliable numerical phenotypes from plant images especially by considering its individual components, e.g., leaves, stem, fruit, and flower, remains a critical bottleneck to the translation of advances of phenotyping technology into genetic insights due to various challenges including lighting variations, plant rotations, and self-occlusions. The paper provides (1) a framework for plant phenotyping in a multimodal, multi-view, time-lapsed, high-throughput imaging system; (2) a taxonomy of phenotypes that may be derived by image analysis for better understanding of morphological structure and functional processes in plants; (3) a brief discussion on publicly available datasets to encourage algorithm development and uniform comparison with the state-of-the-art methods; (4) an overview of the state-of-the-art image-based high-throughput plant phenotyping methods; and (5) open problems for the advancement of this research field.

PI-Plat: a high-resolution image-based 3D reconstruction method to estimate growth dynamics of rice inflorescence traits

BACKGROUND

Recent advances in image-based plant phenotyping have improved our capability to study vegetative stage growth dynamics. However, more complex agronomic traits such as inflorescence architecture (IA), which predominantly contributes to grain crop yield are more challenging to quantify and hence are relatively less explored. Previous efforts to estimate inflorescence-related traits using image-based phenotyping have been limited to destructive end-point measurements. Development of non-destructive inflorescence phenotyping platforms could accelerate the discovery of the phenotypic variation with respect to inflorescence dynamics and mapping of the underlying genes regulating critical yield components.

RESULTS

The major objective of this study is to evaluate post-fertilization development and growth dynamics of inflorescence at high spatial and temporal resolution in rice. For this, we developed the Panicle Imaging Platform (PI-Plat) to comprehend multi-dimensional features of IA in a non-destructive manner. We used 11 rice genotypes to capture multi-view images of primary panicle on weekly basis after the fertilization. These images were used to reconstruct a 3D point cloud of the panicle, which enabled us to extract digital traits such as voxel count and color intensity. We found that the voxel count of developing panicles is positively correlated with seed number and weight at maturity. The voxel count from developing panicles projected overall volumes that increased during the grain filling phase, wherein quantification of color intensity estimated the rate of panicle maturation. Our 3D based phenotyping solution showed superior performance compared to conventional 2D based approaches.

CONCLUSIONS

For harnessing the potential of the existing genetic resources, we need a comprehensive understanding of the genotype-to-phenotype relationship. Relatively low-cost sequencing platforms have facilitated high-throughput genotyping, while phenotyping, especially for complex traits, has posed major challenges for crop improvement. PI-Plat offers a low cost and high-resolution platform to phenotype inflorescence-related traits using 3D reconstruction-based approach. Further, the non-destructive nature of the platform facilitates analyses of the same panicle at multiple developmental time points, which can be utilized to explore the genetic variation for dynamic inflorescence traits in cereals.

Prunus genetics and applications after de novo genome sequencing: achievements and prospects

Prior to the availability of whole-genome sequences, our understanding of the structural and functional aspects of Prunus tree genomes was limited mostly to molecular genetic mapping of important traits and development of EST resources. With public release of the peach genome and others that followed, significant advances in our knowledge of Prunus genomes and the genetic underpinnings of important traits ensued. In this review, we highlight key achievements in Prunus genetics and breeding driven by the availability of these whole-genome sequences. Within the structural and evolutionary contexts, we summarize: (1) the current status of Prunus whole-genome sequences; (2) preliminary and ongoing work on the sequence structure and diversity of the genomes; (3) the analyses of Prunus genome evolution driven by natural and man-made selection; and (4) provide insight into haploblocking genomes as a means to define genome-scale patterns of evolution that can be leveraged for trait selection in pedigree-based Prunus tree breeding programs worldwide. Functionally, we summarize recent and ongoing work that leverages whole-genome sequences to identify and characterize genes controlling 22 agronomically important Prunus traits. These include phenology, fruit quality, allergens, disease resistance, tree architecture, and self-incompatibility. Translationally, we explore the application of sequence-based marker-assisted breeding technologies and other sequence-guided biotechnological approaches for Prunus crop improvement. Finally, we present the current status of publically available Prunus genomics and genetics data housed mainly in the Genome Database for Rosaceae (GDR) and its updated functionalities for future bioinformatics-based Prunus genetics and genomics inquiry.

Protocol for Producing Three-Dimensional Infrared Video of Freezing in Plants

Freezing in plants can be monitored using infrared (IR) thermography, because when water freezes, it gives off heat. However, problems with color contrast make 2-dimensions (2D) infrared images somewhat difficult to interpret. Viewing an IR image or the video of plants freezing in 3 dimensions (3D) would allow a more accurate identification of sites for ice nucleation as well as the progression of freezing. In this paper, we demonstrate a relatively simple means to produce a 3D infrared video of a strawberry plant freezing. Strawberry is an economically important crop that is subjected to unexpected spring freeze events in many areas of the world. An accurate understanding of the freezing in strawberry will provide both breeders and growers with more economical ways to prevent any damage to plants during freezing conditions. The technique involves a positioning of two IR cameras at slightly different angles to film the strawberry freezing. The two video streams will be precisely synchronized using a screen capture software that records both cameras simultaneously. The recordings will then be imported into the imaging software and processed using an anaglyph technique. Using red-blue glasses, the 3D video will make it easier to determine the precise site of ice nucleation on leaf surfaces.