Graph-Based Data Fusion Applied to: Change Detection and Biomass Estimation in Rice Crops

Commonality Feature Representation Learning for Unsupervised Multimodal Change Detection

The main challenge of multimodal change detection (MCD) is that multimodal bitemporal images (MBIs) cannot be compared directly to identify changes. To overcome this problem, this paper proposes a novel commonality feature representation learning (CFRL) and constructs a CFRL-based unsupervised MCD framework. The CFRL is composed of a Siamese-based encoder and two decoders. First, the Siamese-based encoder can map original MBIs in the same feature space for extracting the representative features of each modality. Then, the two decoders are used to reconstruct the original MBIs by regressing themselves, respectively. Meanwhile, we swap the decoders to reconstruct the pseudo-MBIs to conduct modality alignment. Subsequently, all reconstructed images are input to the Siamese-based encoder again to map them in a same feature space, by which representative features are obtained. On this basis, latent commonality features between MBIs can be extracted by minimizing the distance between these representative features. These latent commonality features are comparable and can be used to identify changes. Notably, the proposed CFRL can be performed simultaneously in two modalities corresponding to MBIs. Therefore, two change magnitude images (CMIs) can be generated simultaneously by measuring the difference between the commonality features of MBIs. Finally, a simple threshold algorithm or a clustering algorithm can be employed to divide CMIs into binary change maps. Extensive experiments on six publicly available MCD datasets show that the proposed CFRL-based framework can achieve superior performance compared with other state-of-the-art approaches.

Estimating potassium in potato plants based on multispectral images acquired from unmanned aerial vehicles

Plant potassium content (PKC) is a crucial indicator of crop potassium nutrient status and is vital in making informed fertilization decisions in the field. This study aims to enhance the accuracy of PKC estimation during key potato growth stages by using vegetation indices (VIs) and spatial structure features derived from UAV-based multispectral sensors. Specifically, the fraction of vegetation coverage (FVC), gray-level co-occurrence matrix texture, and multispectral VIs were extracted from multispectral images acquired at the potato tuber formation, tuber growth, and starch accumulation stages. Linear regression and stepwise multiple linear regression analyses were conducted to investigate how VIs, both individually and in combination with spatial structure features, affect potato PKC estimation. The findings lead to the following conclusions: (1) Estimating potato PKC using multispectral VIs is feasible but necessitates further enhancements in accuracy. (2) Augmenting VIs with either the FVC or texture features makes potato PKC estimation more accurate than when using single VIs. (3) Finally, integrating VIs with both the FVC and texture features improves the accuracy of potato PKC estimation, resulting in notable R 2 values of 0.63, 0.84, and 0.80 for the three fertility periods, respectively, with corresponding root mean square errors of 0.44%, 0.29%, and 0.25%. Overall, these results highlight the potential of integrating canopy spectral information and spatial-structure information obtained from multispectral sensors mounted on unmanned aerial vehicles for monitoring crop growth and assessing potassium nutrient status. These findings thus have significant implications for agricultural management.

Estimation of potato above-ground biomass based on unmanned aerial vehicle red-green-blue images with different texture features and crop height

Obtaining crop above-ground biomass (AGB) information quickly and accurately is beneficial to farmland production management and the optimization of planting patterns. Many studies have confirmed that, due to canopy spectral saturation, AGB is underestimated in the multi-growth period of crops when using only optical vegetation indices. To solve this problem, this study obtains textures and crop height directly from ultrahigh-ground-resolution (GDS) red-green-blue (RGB) images to estimate the potato AGB in three key growth periods. Textures include a grayscale co-occurrence matrix texture (GLCM) and a Gabor wavelet texture. GLCM-based textures were extracted from seven-GDS (1, 5, 10, 30, 40, 50, and 60 cm) RGB images. Gabor-based textures were obtained from magnitude images on five scales (scales 1-5, labeled S1-S5, respectively). Potato crop height was extracted based on the generated crop height model. Finally, to estimate potato AGB, we used (i) GLCM-based textures from different GDS and their combinations, (ii) Gabor-based textures from different scales and their combinations, (iii) all GLCM-based textures combined with crop height, (iv) all Gabor-based textures combined with crop height, and (v) two types of textures combined with crop height by least-squares support vector machine (LSSVM), extreme learning machine, and partial least squares regression techniques. The results show that (i) potato crop height and AGB first increase and then decrease over the growth period; (ii) GDS and scales mainly affect the correlation between GLCM- and Gabor-based textures and AGB; (iii) to estimate AGB, GLCM-based textures of GDS1 and GDS30 work best when the GDS is between 1 and 5 cm and 10 and 60 cm, respectively (however, estimating potato AGB based on Gabor-based textures gradually deteriorates as the Gabor convolution kernel scale increases); (iv) the AGB estimation based on a single-type texture is not as good as estimates based on multi-resolution GLCM-based and multiscale Gabor-based textures (with the latter being the best); (v) different forms of textures combined with crop height using the LSSVM technique improved by 22.97, 14.63, 9.74, and 8.18% (normalized root mean square error) compared with using only all GLCM-based textures, all Gabor-based textures, the former combined with crop height, and the latter combined with crop height, respectively. Therefore, different forms of texture features obtained from RGB images acquired from unmanned aerial vehicles and combined with crop height improve the accuracy of potato AGB estimates under high coverage.

Change Detection Techniques with Synthetic Aperture Radar Images: Experiments with Random Forests and Sentinel-1 Observations

This work aims to clarify the potential of incoherent and coherent change detection (CD) approaches for detecting and monitoring ground surface changes using sequences of synthetic aperture radar (SAR) images. Nowadays, the growing availability of remotely sensed data collected by the twin Sentinel-1A/B sensors of the European (EU) Copernicus constellation allows fast mapping of damage after a disastrous event using radar data. In this research, we address the role of SAR (amplitude) backscattered signal variations for CD analyses when a natural (e.g., a fire, a flash flood, etc.) or a human-induced (disastrous) event occurs. Then, we consider the additional pieces of information that can be recovered by comparing interferometric coherence maps related to couples of SAR images collected between a principal disastrous event date. This work is mainly concerned with investigating the capability of different coherent/incoherent change detection indices (CDIs) and their mutual interactions for the rapid mapping of “changed” areas. In this context, artificial intelligence (AI) algorithms have been demonstrated to be beneficial for handling the different information coming from coherent/incoherent CDIs in a unique corpus. Specifically, we used CDIs that synthetically describe ground surface changes associated with a disaster event (i.e., the pre-, cross-, and post-disaster phases), based on the generation of sigma nought and InSAR coherence maps. Then, we trained a random forest (RF) to produce CD maps and study the impact on the final binary decision (changed/unchanged) of the different layers representing the available synthetic CDIs. The proposed strategy was effective for quickly assessing damage using SAR data and can be applied in several contexts. Experiments were conducted to monitor wildfire’s effects in the 2021 summer season in Italy, considering two case studies in Sardinia and Sicily. Another experiment was also carried out on the coastal city of Houston, Texas, the US, which was affected by a large flood in 2017; thus, demonstrating the validity of the proposed integrated method for fast mapping of flooded zones using SAR data.

Data Fusion in Agriculture: Resolving Ambiguities and Closing Data Gaps

Acquiring useful data from agricultural areas has always been somewhat of a challenge, as these are often expansive, remote, and vulnerable to weather events. Despite these challenges, as technologies evolve and prices drop, a surge of new data are being collected. Although a wealth of data are being collected at different scales (i.e., proximal, aerial, satellite, ancillary data), this has been geographically unequal, causing certain areas to be virtually devoid of useful data to help face their specific challenges. However, even in areas with available resources and good infrastructure, data and knowledge gaps are still prevalent, because agricultural environments are mostly uncontrolled and there are vast numbers of factors that need to be taken into account and properly measured for a full characterization of a given area. As a result, data from a single sensor type are frequently unable to provide unambiguous answers, even with very effective algorithms, and even if the problem at hand is well defined and limited in scope. Fusing the information contained in different sensors and in data from different types is one possible solution that has been explored for some decades. The idea behind data fusion involves exploring complementarities and synergies of different kinds of data in order to extract more reliable and useful information about the areas being analyzed. While some success has been achieved, there are still many challenges that prevent a more widespread adoption of this type of approach. This is particularly true for the highly complex environments found in agricultural areas. In this article, we provide a comprehensive overview on the data fusion applied to agricultural problems; we present the main successes, highlight the main challenges that remain, and suggest possible directions for future research.

Novel Feature-Extraction Methods for the Estimation of Above-Ground Biomass in Rice Crops

Traditional methods to measure spatio-temporal variations in above-ground biomass dynamics (AGBD) predominantly rely on the extraction of several vegetation-index features highly associated with AGBD variations through the phenological crop cycle. This work presents a comprehensive comparison between two different approaches for feature extraction for non-destructive biomass estimation using aerial multispectral imagery. The first method is called GFKuts, an approach that optimally labels the plot canopy based on a Gaussian mixture model, a Montecarlo-based K-means, and a guided image filtering for the extraction of canopy vegetation indices associated with biomass yield. The second method is based on a Graph-Based Data Fusion (GBF) approach that does not depend on calculating vegetation-index image reflectances. Both methods are experimentally tested and compared through rice growth stages: vegetative, reproductive, and ripening. Biomass estimation correlations are calculated and compared against an assembled ground-truth biomass measurements taken by destructive sampling. The proposed GBF-Sm-Bs approach outperformed competing methods by obtaining biomass estimation correlation of 0.995 with R2=0.991 and RMSE=45.358 g. This result increases the precision in the biomass estimation by around 62.43% compared to previous works.

Knowledge and Spatial Pyramid Distance-Based Gated Graph Attention Network for Remote Sensing Semantic Segmentation

The pixel-based semantic segmentation methods take pixels as recognitions units, and are restricted by the limited range of receptive fields, so they cannot carry richer and higher-level semantics. These reduce the accuracy of remote sensing (RS) semantic segmentation to a certain extent. Comparing with the pixel-based methods, the graph neural networks (GNNs) usually use objects as input nodes, so they not only have relatively small computational complexity, but also can carry richer semantic information. However, the traditional GNNs are more rely on the context information of the individual samples and lack geographic prior knowledge that reflects the overall situation of the research area. Therefore, these methods may be disturbed by the confusion of “different objects with the same spectrum” or “violating the first law of geography” in some areas. To address the above problems, we propose a remote sensing semantic segmentation model called knowledge and spatial pyramid distance-based gated graph attention network (KSPGAT), which is based on prior knowledge, spatial pyramid distance and a graph attention network (GAT) with gating mechanism. The model first uses superpixels (geographical objects) to form the nodes of a graph neural network and then uses a novel spatial pyramid distance recognition algorithm to recognize the spatial relationships. Finally, based on the integration of feature similarity and the spatial relationships of geographic objects, a multi-source attention mechanism and gating mechanism are designed to control the process of node aggregation, as a result, the high-level semantics, spatial relationships and prior knowledge can be introduced into a remote sensing semantic segmentation network. The experimental results show that our model improves the overall accuracy by 4.43% compared with the U-Net Network, and 3.80% compared with the baseline GAT network.