Optimizing Gaussian Process Regression for Image Time Series Gap-Filling and Crop Monitoring

"How sweet are your strawberries?": Predicting sugariness using non-destructive and affordable hardware

Global soft fruit supply chains rely on trustworthy descriptions of product quality. However, crucial criteria such as sweetness and firmness cannot be accurately established without destroying the fruit. Since traditional alternatives are subjective assessments by human experts, it is desirable to obtain quality estimations in a consistent and non-destructive manner. The majority of research on fruit quality measurements analyzed fruits in the lab with uniform data collection. However, it is laborious and expensive to scale up to the level of the whole yield. The "harvest-first, analysis-second" method also comes too late to decide to adjust harvesting schedules. In this research, we validated our hypothesis of using in-field data acquirable via commodity hardware to obtain acceptable accuracies. The primary instance that the research concerns is the sugariness of strawberries, described by the juice's total soluble solid (TSS) content (unit: °Brix or Brix). We benchmarked the accuracy of strawberry Brix prediction using convolutional neural networks (CNN), variational autoencoders (VAE), principal component analysis (PCA), kernelized ridge regression (KRR), support vector regression (SVR), and multilayer perceptron (MLP), based on fusions of image data, environmental records, and plant load information, etc. Our results suggest that: (i) models trained by environment and plant load data can perform reliable prediction of aggregated Brix values, with the lowest RMSE at 0.59; (ii) using image data can further supplement the Brix predictions of individual fruits from (i), from 1.27 to as low up to 1.10, but they by themselves are not sufficiently reliable.

A Review of Hybrid Approaches for Quantitative Assessment of Crop Traits Using Optical Remote Sensing: Research Trends and Future Directions

Remote sensing technology allows to provide information about biochemical and biophysical crop traits and monitor their spatiotemporal dynamics of agriculture ecosystems. Among multiple retrieval techniques, hybrid approaches have been found to provide outstanding accuracy, for instance, for the inference of leaf area index (LAI), fractional vegetation cover (fCover), and leaf and canopy chlorophyll content (LCC and CCC). The combination of radiative transfer models (RTMs) and data-driven models creates an advantage in the use of hybrid methods. Through this review paper, we aim to provide state-of-the-art hybrid retrieval schemes and theoretical frameworks. To achieve this, we reviewed and systematically analyzed publications over the past 22 years. We identified two hybrid-based parametric and hybrid-based nonparametric regression models and evaluated their performance for each variable of interest. From the results of our extensive literature survey, most research directions are now moving towards combining RTM and machine learning (ML) methods in a symbiotic manner. In particular, the development of ML will open up new ways to integrate innovative approaches such as integrating shallow or deep neural networks with RTM using remote sensing data to reduce errors in crop trait estimations and improve control of crop growth conditions in very large areas serving precision agriculture applications.

Gaussian Processes for Signal Processing and Representation in Control Engineering

The Gaussian process is an increasingly well-known type of stochastic process, which is a generalization of the Gaussian probability distribution. It allows us to model complex functions thanks to its flexibility, which would not be possible with the use of other tools. Gaussian processes also have a couple of other features that are used in various branches of automation with positive results, ranging from industrial processes to image processing. There are also many ways of setting up the Gaussian processes, which required knowledge on the topic and depend on the presented problem. Considerations on these topics lead to the conclusion that the current state of practical usefulness of Gaussian processes increases significantly, therefore the deepening of knowledge about the ways of its use is highly suggested. In this review, we present selected technical applications of Gaussian Processes allowing an understanding of their broad applicability.

Multi-Season Phenology Mapping of Nile Delta Croplands Using Time Series of Sentinel-2 and Landsat 8 Green LAI

Space-based cropland phenology monitoring substantially assists agricultural managing practices and plays an important role in crop yield predictions. Multitemporal satellite observations allow analyzing vegetation seasonal dynamics over large areas by using vegetation indices or by deriving biophysical variables. The Nile Delta represents about half of all agricultural lands of Egypt. In this region, intensifying farming systems are predominant and multi-cropping rotations schemes are increasing, requiring a high temporal and spatial resolution monitoring for capturing successive crop growth cycles. This study presents a workflow for cropland phenology characterization and mapping based on time series of green Leaf Area Index (LAI) generated from NASA’s Harmonized Landsat 8 (L8) and Sentinel-2 (S2) surface reflectance dataset from 2016 to 2019. LAI time series were processed for each satellite dataset, which were used separately and combined to identify seasonal dynamics for a selection of crop types (wheat, clover, maize and rice). For the combination of L8 with S2 LAI products, we proposed two time series smoothing and fitting methods: (1) the Savitzky–Golay (SG) filter and (2) the Gaussian Processes Regression (GPR) fitting function. Single-sensor and L8-S2 combined LAI time series were used for the calculation of key crop Land Surface Phenology (LSP) metrics (start of season, end of season, length of season), whereby the detection of cropland growing seasons was based on two established threshold methods, i.e., a seasonal or a relative amplitude value. Overall, the developed phenology extraction scheme enabled identifying up to two successive crop cycles within a year, with a superior performance observed for the seasonal than for the relative threshold method, in terms of consistency and cropland season detection capability. Differences between the time series collections were analyzed by comparing the phenology metrics per crop type and year. Results suggest that L8-S2 combined LAI data streams with GPR led to a more precise detection of the start and end of growing seasons for most crop types, reaching an overall detection of 74% over the total planted crops versus 69% with S2 and 63% with L8 alone. Finally, the phenology mapping allowed us to evaluate the spatial and temporal evolution of the croplands over the agroecosystem in the Nile Delta.

Quantifying Fundamental Vegetation Traits over Europe Using the Sentinel-3 OLCI Catalogue in Google Earth Engine

Thanks to the emergence of cloud-computing platforms and the ability of machine learning methods to solve prediction problems efficiently, this work presents a workflow to automate spatiotemporal mapping of essential vegetation traits from Sentinel-3 (S3) imagery. The traits included leaf chlorophyll content (LCC), leaf area index (LAI), fraction of absorbed photosynthetically active radiation (FAPAR), and fractional vegetation cover (FVC), being fundamental for assessing photosynthetic activity on Earth. The workflow involved Gaussian process regression (GPR) algorithms trained on top-of-atmosphere (TOA) radiance simulations generated by the coupled canopy radiative transfer model (RTM) SCOPE and the atmospheric RTM 6SV. The retrieval models, named to S3-TOA-GPR-1.0, were directly implemented in Google Earth Engine (GEE) to enable the quantification of the traits from TOA data as acquired from the S3 Ocean and Land Colour Instrument (OLCI) sensor.Following good to high theoretical validation results with normalized root mean square error (NRMSE) ranging from 5% (FAPAR) to 19% (LAI), a three fold evaluation approach over diverse sites and land cover types was pursued: (1) temporal comparison against LAI and FAPAR products obtained from Moderate Resolution Imaging Spectroradiometer (MODIS) for the time window 2016-2020, (2) spatial difference mapping with Copernicus Global Land Service (CGLS) estimates, and (3) direct validation using interpolated in situ data from the VALERI network. For all three approaches, promising results were achieved. Selected sites demonstrated coherent seasonal patterns compared to LAI and FAPAR MODIS products, with differences between spatially averaged temporal patterns of only 6.59%. In respect of the spatial mapping comparison, estimates provided by the S3-TOA-GPR-1.0 models indicated highest consistency with FVC and FAPAR CGLS products. Moreover, the direct validation of our S3-TOA-GPR-1.0 models against VALERI estimates indicated with regard to jurisdictional claims in good retrieval performance for LAI, FAPAR and FVC. We conclude that our retrieval workflow of spatiotemporal S3 TOA data processing into GEE opens the path towards global monitoring of fundamental vegetation traits, accessible to the whole research community.

Monitoring Cropland Phenology on Google Earth Engine Using Gaussian Process Regression

Monitoring cropland phenology from optical satellite data remains a challenging task due to the influence of clouds and atmospheric artifacts. Therefore, measures need to be taken to overcome these challenges and gain better knowledge of crop dynamics. The arrival of cloud computing platforms such as Google Earth Engine (GEE) has enabled us to propose a Sentinel-2 (S2) phenology end-to-end processing chain. To achieve this, the following pipeline was implemented: (1) the building of hybrid Gaussian Process Regression (GPR) retrieval models of crop traits optimized with active learning, (2) implementation of these models on GEE (3) generation of spatiotemporally continuous maps and time series of these crop traits with the use of gap-filling through GPR fitting, and finally, (4) calculation of land surface phenology (LSP) metrics such as the start of season (SOS) or end of season (EOS). Overall, from good to high performance was achieved, in particular for the estimation of canopy-level traits such as leaf area index (LAI) and canopy chlorophyll content, with normalized root mean square errors (NRMSE) of 9% and 10%, respectively. By means of the GPR gap-filling time series of S2, entire tiles were reconstructed, and resulting maps were demonstrated over an agricultural area in Castile and Leon, Spain, where crop calendar data were available to assess the validity of LSP metrics derived from crop traits. In addition, phenology derived from the normalized difference vegetation index (NDVI) was used as reference. NDVI not only proved to be a robust indicator for the calculation of LSP metrics, but also served to demonstrate the good phenology quality of the quantitative trait products. Thanks to the GEE framework, the proposed workflow can be realized anywhere in the world and for any time window, thus representing a shift in the satellite data processing paradigm. We anticipate that the produced LSP metrics can provide meaningful insights into crop seasonal patterns in a changing environment that demands adaptive agricultural production.

Comparison of Crop Trait Retrieval Strategies Using UAV-Based VNIR Hyperspectral Imaging

Hyperspectral cameras onboard unmanned aerial vehicles (UAVs) have recently emerged for monitoring crop traits at the sub-field scale. Different physical, statistical, and hybrid methods for crop trait retrieval have been developed. However, spectra collected from UAVs can be confounded by various issues, including illumination variation throughout the crop growing season, the effect of which on the retrieval performance is not well understood at present. In this study, four retrieval methods are compared, in terms of retrieving the leaf area index (LAI), fractional vegetation cover (fCover), and canopy chlorophyll content (CCC) of potato plants over an agricultural field for six dates during the growing season. We analyzed: (1) The standard look-up table method (LUTstd), (2) an improved (regularized) LUT method that involves variable correlation (LUTreg), (3) hybrid methods, and (4) random forest regression without (RF) and with (RFexp) the exposure time as an additional explanatory variable. The Soil–Leaf–Canopy (SLC) model was used in association with the LUT-based inversion and hybrid methods, while the statistical modelling methods (RF and RFexp) relied entirely on in situ data. The results revealed that RFexp was the best-performing method, yielding the highest accuracies, in terms of the normalized root mean square error (NRMSE), for LAI (5.36%), fCover (5.87%), and CCC (15.01%). RFexp was able to reduce the effects of illumination variability and cloud shadows. LUTreg outperformed the other two retrieval methods (hybrid methods and LUTstd), with an NRMSE of 9.18% for LAI, 10.46% for fCover, and 12.16% for CCC. Conversely, LUTreg led to lower accuracies than those derived from RF for LAI (5.51%) and for fCover (6.23%), but not for CCC (16.21%). Therefore, the machine learning approaches—in particular, RF—appear to be the most promising retrieval methods for application to UAV-based hyperspectral data.

Green LAI Mapping and Cloud Gap-Filling Using Gaussian Process Regression in Google Earth Engine

For the last decade, Gaussian process regression (GPR) proved to be a competitive machine learning regression algorithm for Earth observation applications, with attractive unique properties such as band relevance ranking and uncertainty estimates. More recently, GPR also proved to be a proficient time series processor to fill up gaps in optical imagery, typically due to cloud cover. This makes GPR perfectly suited for large-scale spatiotemporal processing of satellite imageries into cloud-free products of biophysical variables. With the advent of the Google Earth Engine (GEE) cloud platform, new opportunities emerged to process local-to-planetary scale satellite data using advanced machine learning techniques and convert them into gap-filled vegetation properties products. However, GPR is not yet part of the GEE ecosystem. To circumvent this limitation, this work proposes a general adaptation of GPR formulation to parallel processing framework and its integration into GEE. To demonstrate the functioning and utility of the developed workflow, a GPR model predicting green leaf area index (LAIG) from Sentinel-2 imagery was imported. Although by running this GPR model into GEE any corner of the world can be mapped into LAIG at a resolution of 20 m, here we show some demonstration cases over western Europe with zoom-ins over Spain. Thanks to the computational power of GEE, the mapping takes place on-the-fly. Additionally, a GPR-based gap filling strategy based on pre-optimized kernel hyperparameters is also put forward for the generation of multi-orbit cloud-free LAIG maps with an unprecedented level of detail, and the extraction of regularly-sampled LAIG time series at a pixel level. The ability to plugin a locally-trained GPR model into the GEE framework and its instant processing opens up a new paradigm of remote sensing image processing.