A transferable spectroscopic diagnosis model for predicting arsenic contamination in soil

Would it be better for partition prediction of heavy metal concentration in soils based on the fusion of XRF and Vis-NIR data?

Heavy metal (HM) contamination in soil necessitates effective methods to diagnose suspected contaminated areas and control rehabilitation processes. The synergistic use of proximal sensors demonstrates significant potential for rapid detection via accurate surveys of soil HM pollution at large scales and high sampling densities, and necessitates the selection of appropriate data mining and modeling methods for early diagnosis of soil pollution. The aim of this study is to evaluate the performance of a subarea model based on geographically partitioned and global models based on high-precision energy dispersive X-ray fluorescence (HD-XRF) and visible near-infrared (vis-NIR) spectra using a random forest model for predicting soil Cu and Pb concentrations. A total of 166 soil samples are acquired from a contaminated plot in Baiyin, Gansu Province, China. The soil samples are subjected to HM analysis and proximal sensor scanning in a laboratory. Vis-NIR spectral data are preprocessed using the Savitzky Golay (SG) and first-order derivative with Savitzky Golay (SGFD) methods. The results show that for predicting Cu and Pb concentrations in soil, the subarea models performs better than the global models in terms of quantitative prediction, based solely on individual HD-XRF data. For the subarea and global models, the R2 values are 0.961 and 0.981, respectively; the RMSE values are 27.8 and 79.6, respectively; and the RPD values are 4.96 and 7.38, respectively. However, making use of the random forest algorithm trained with data fusion obtained from the HD-XRF and vis-NIR sensors, the global model achieves the best predictions for Cu and Pb concentrations via HD-XRF + vis-NIR (SGFD) and HD-XRF + vis-NIR (SG), respectively. The results will provide a new perspective for modeling approaches to rapidly invert HM concentrations based on proximal sensor data fusion within a large scope of the study area.

Fast Detection of Heavy Metal Content in Fritillaria thunbergii by Laser-Induced Breakdown Spectroscopy with PSO-BP and SSA-BP Analysis

Fast detection of heavy metals is important to ensure the quality and safety of herbal medicines. In this study, laser-induced breakdown spectroscopy (LIBS) was applied to detect the heavy metal content (Cd, Cu, and Pb) in Fritillaria thunbergii. Quantitative prediction models were established using a back-propagation neural network (BPNN) optimized using the particle swarm optimization (PSO) algorithm and sparrow search algorithm (SSA), called PSO-BP and SSA-BP, respectively. The results revealed that the BPNN models optimized by PSO and SSA had better accuracy than the BPNN model without optimization. The performance evaluation metrics of the PSO-BP and SSA-BP models were similar. However, the SSA-BP model had two advantages: it was faster and had higher prediction accuracy at low concentrations. For the three heavy metals Cd, Cu and Pb, the prediction correlation coefficient (R_p²) values for the SSA-BP model were 0.972, 0.991 and 0.956; the prediction root mean square error (RMSEP) values were 5.553, 7.810 and 12.906 mg/kg; and the prediction relative percent deviation (RPD) values were 6.04, 10.34 and 4.94, respectively. Therefore, LIBS could be considered a constructive tool for the quantification of Cd, Cu and Pb contents in Fritillaria thunbergii.

Combination of spectral index and transfer learning strategy for glyphosate-resistant cultivar identification

Glyphosate is one of the most widely used non-selective herbicides, and the creation of glyphosate-resistant cultivars solves the problem of limited spraying area. Therefore, it is of great significance to quickly identify resistant cultivars without destruction during the development of superior cultivars. This work took maize seedlings as the experimental object, and the spectral indices of leaves were calculated to construct a model with good robustness that could be used in different experiments. Compared with no transfer strategies, transferability of support vector machine learning model was improved by randomly selecting 14% of source domain from target domain to train and applying transfer component analysis algorithm, the accuracy on target domain reached 83% (increased by 71%), recall increased from 10 to 100%, and F1-score increased from 0.17 to 0.86. The overall results showed that both transfer component analysis algorithm and updating source domain could improve the transferability of model among experiments, and these two transfer strategies could complement each other's advantages to achieve the best classification performance. Therefore, this work is beneficial to timely understanding of the physiological status of plants, identifying glyphosate resistant cultivars, and ultimately provides theoretical basis and technical support for new cultivar creation and high-throughput selection.

Retrieving soil heavy metals concentrations based on GaoFen-5 hyperspectral satellite image at an opencast coal mine, Inner Mongolia, China

Soil heavy metals pollution has been becoming one of the severely environmental issues globally. Previous studies reported laboratory-measured spectra could be used to infer soil heavy metals concentrations to some extent. However, using field-obtained spectra to estimate soil heavy metals concentrations is still a great challenge due to the low precision and weak efficiency at large scales. The present study collected 110 topsoil samples from an Opencast Coal Mine of Ordos, Inner Mongolia, China. Then, the spectra and soil heavy metals concentrations of samples were measured under laboratory conditions. The direct standardization (DS) algorithm was introduced to calibrate the Gaofen-5 (GF-5) hyperspectral image based on the measured spectra of samples. The spectral reflectance of the GF-5 hyperspectral image was reconstructed using continuous wavelet transform (CWT) at different scales. The characteristic bands of GF-5 for estimating heavy metals concentrations were selected by the Boruta algorithm. Finally, the random forest (RF), the extreme learning machine (ELM), the support vector machine (SVM), and the back-propagation neural network (BPNN) algorithms were used to predict the heavy metals concentrations. Some findings were achieved. First, CWT can effectively eliminate the noise of satellite hyperspectral data. The characteristic bands of Zn (480-677, 827-1029, 1241-1334, 1435-1797, and 1949-2500 nm), Ni (514-630, 835-985, 1258-1325, 1460-1578, and 1949-2319 nm), and Cu (822-831; 1029-1300, 1486-1595, and 1730-2294 nm) can be effectively retrieved via the Boruta algorithm. Second, the estimation accuracy was significantly improved by using the DS algorithm. For zinc (Zn), nickel (Ni), and copper (Cu), the determination coefficients of the validation dataset (R_v²) were 0.77 (RF), 0.62 (RF), and 0.56 (ELM), respectively. Third, the distribution trends of heavy metals were almost consistent with the results of actual ground measurements. This paper revealed that the GF-5 can be one of the reliable satellite hyperspectral imagery for mapping soil heavy metals.

Identification of Soil Arsenic Contamination in Rice Paddy Field Based on Hyperspectral Reflectance Approach

Toxic heavy metals in soil negatively impact soil’s physical, biological, and chemical characteristics, and also human wellbeing. The traditional approach of chemical analysis procedures for assessing soil toxicant element concentration is time-consuming and expensive. Due to accessibility, reliability, and rapidity at a high temporal and spatial resolution, hyperspectral remote sensing within the Vis-NIR region is an indispensable and widely used approach in today’s world for monitoring broad regions and controlling soil arsenic (As) pollution in agricultural land. This study investigates the effectiveness of hyperspectral reflectance approaches in different regions for assessing soil As pollutants, as well as a basic review of space-borne earth observation hyperspectral sensors. Multivariate and various regression models were developed to avoid collinearity and improve prediction capabilities using spectral bands with the perfect correlation coefficients to access the soil As contamination in previous studies. This review highlights some of the most significant factors to consider when developing a remote sensing approach for soil As contamination in the future, as well as the potential limits of employing spectroscopy data.

Meta-Learner Hybrid Models to Classify Hyperspectral Images

Hyperspectral (HS) images are adjacent band images that are generally used in remote-sensing applications. They have numerous spatial and spectral information bands that are extremely useful for material detection in various fields. However, their high dimensionality is a big challenge that affects their overall performance. A new data normalization method was developed to enhance the variations and data distribution using the output of principal component analysis (PCA) and quantile transformation, called QPCA. This paper also proposes a novel HS images classification framework using the meta-learner technique to train multi-class and multi-size datasets by concatenating and training the hybrid and multi-size kernel of convolutional neural networks (CNN). The high-level model works to combine the output of the lower-level models and train them with the new input data, called meta-learner hybrid models (MLHM). The proposed MLHM framework with our external normalization (QPCA) improves the accuracy and outperforms other approaches using three well-known benchmark datasets. Moreover, the evaluation outcomes showed that the QPCA enhanced the framework accuracy by 13% for most models and datasets and others by more than 25%, and MLHM provided the best performance.

Hyperspectral Image Classification Based on Two-Branch Spectral–Spatial-Feature Attention Network

Although most of deep-learning-based hyperspectral image (HSI) classification methods achieve great performance, there still remains a challenge to utilize small-size training samples to remarkably enhance the classification accuracy. To tackle this challenge, a novel two-branch spectral–spatial-feature attention network (TSSFAN) for HSI classification is proposed in this paper. Firstly, two inputs with different spectral dimensions and spatial sizes are constructed, which can not only reduce the redundancy of the original dataset but also accurately explore the spectral and spatial features. Then, we design two parallel 3DCNN branches with attention modules, in which one focuses on extracting spectral features and adaptively learning the more discriminative spectral channels, and the other focuses on exploring spatial features and adaptively learning the more discriminative spatial structures. Next, the feature attention module is constructed to automatically adjust the weights of different features based on their contributions for classification to remarkably improve the classification performance. Finally, we design the hybrid architecture of 3D–2DCNN to acquire the final classification result, which can significantly decrease the sophistication of the network. Experimental results on three HSI datasets indicate that our presented TSSFAN method outperforms several of the most advanced classification methods.

Retrieving zinc concentrations in topsoil with reflectance spectroscopy at Opencast Coal Mine sites

Heavy metals contaminations in mining areas aroused wide concerns globally. Efficient evaluation of its pollution status is a basis for further soil reclamation. Visible and near-infrared reflectance (Vis-NIR) spectroscopy has been diffusely used for retrieving heavy metals concentrations. However, the reliability and feasibility of calibrated models were still doubtful. The present study estimated zinc (Zn) concentrations via the random forest (RF) and partial least squares regression (PLSR) using ground in-situ Zn concentrations as well as soil spectral reflectance at an Opencast Coal Mine of Ordos, China in February 2020. The coefficient of determination (R2), root mean square error (RMSE), mean absolute error (MAE), and the ratio of performance to deviation (RPD) were selected to assess the robustness of the methods in estimating Zn contents. Moreover, the characteristic bands were chosen by Pearson correlation analysis and Boruta Algorithm. Finally, the comparison between RF and PLSR combined with eight spectral reflectance transformation methods was conducted for four concentration groups to determine the optimal model. The results indicated that: (1) Zn contents represented a skewed distribution (coefficient of variation (CV) = 33%); (2) the spectral reflectance tended to decrease with the increase of Zn contents during 580-1850 nm based on Savitzky-Golay smoothing (SG); (3) the continuous wavelet transform (CWT) demonstrated higher effectiveness than other spectral reflectance transformation methods in enhancing spectral responses, the R2 between Zn contents and the soil spectral reflectance achieved the highest (R2 = 0.71) by using CWT; (4) the RF combined with CWT exhibited the best performance than other methods in the current study (R2 = 0.97, RPD = 3.39, RMSE = 1.05 mg kg-1, MAE = 0.79 mg kg-1). The current study supplied a scientific scheme and theoretical support for predicting heavy metals concentrations via the Vis-NIR spectral method in possible contaminated areas such as coal mines and metallic mineral deposit areas.

A Near Standard Soil Samples Spectra Enhanced Modeling Strategy for Cd Concentration Prediction

Visible and near-infrared (VNIR) spectroscopy technology for soil heavy metal (HM) concentration prediction has been widely studied. However, its spectral response characteristics are still uncertain. In this study, a near standard soil Cd samples (NSSCd) spectra enhanced modeling strategy was developed in order to to reveal the soil cadmium (Cd) spectral response characteristics and predict its concentration. NSSCd were produced by adding the quantitative Cd solution into background soil. Then, prior spectral bands (i.e., the bands with higher variable importance in projection (VIP) score in NSSCd spectra) were used for predicting Cd concentration in soil samples collected from the Hengyang mining area and Baoding agriculture area. The partial least squares (PLS) and competitive adaptive reweighted sampling-partial least squares (CARS-PLS) were used for validation. Compared to using entire VNIR spectral ranges, the new modeling strategy performed very well, with the coefficient of determination (R2) and the ratio of prediction to deviation (RPD) showing an improvement from 0.63 and 1.72 to 0.71 and 1.95 in Hengyang and from 0.54 and 1.57 to 0.76 and 2.19 in Baoding. These results suggest that NSS prior spectral bands are critical for soil HM prediction. Our results represent an exciting finding for the future design of remote sensing sensors for soil HM detection.

Transfer learning strategy for plastic pollution detection in soil: Calibration transfer from high-throughput HSI system to NIR sensor

Rapid detection tasks in soil environment are generally implemented by various spectrometers and chemometric models. To reduce costs for model construction, calibration transfer from laboratory spectral instruments to portable devices has recently received extensive attention. In different application cases of model transference, most conventional methods require extra time to tune hyperparameters and to select calibration transfer techniques. Based on the near-infrared (NIR) analytical technique, this work aimed at exploring a transfer learning strategy to detect plastic pollution levels in the soil by transferring the model from a high-throughput hyperspectral image (HSI) system to an ultra-portable NIR sensor. Transfer learning was explored to diagnose the proper calibration transfer algorithm and construct the transferable model. For transferable model construction, conventional calibration transfer algorithms (Direct Standardization (DS) or Repeatability file (Repfile)) served as a pre-processing step, and non-parametric transfer learning algorithm (Easy Transfer Learning (EasyTL)) was explored in the modeling step. Supporting vector machine (SVM) was carried out as a typical modeling algorithm for comparison. For transformation algorithms selection, a distance metric algorithm, maximum mean discrepancy (MMD), was performed on spectral feature matrices before and after DS or Repfile transformation. On three transfer tasks, the results indicated that the Repfile-EasyTL model was a promising solution with higher accuracy, much lower time costs, less parameters, and dependency on the increase of standard samples than other models (SVM, DS-SVM, Repfile-SVM, EasyTL, DS-EasyTL). Moreover, MMD distance presented the great potential to serve as an indicator to vote the optimal calibration transfer algorithm before the modeling step.

Classification of Rice Yield Using UAV-Based Hyperspectral Imagery and Lodging Feature

High-yield rice cultivation is an effective way to address the increasing food demand worldwide. Correct classification of high-yield rice is a key step of breeding. However, manual measurements within breeding programs are time consuming and have high cost and low throughput, which limit the application in large-scale field phenotyping. In this study, we developed an accurate large-scale approach and presented the potential usage of hyperspectral data for rice yield measurement using the XGBoost algorithm to speed up the rice breeding process for many breeders. In total, 13 japonica rice lines in regional trials in northern China were divided into different categories according to the manual measurement of yield. Using an Unmanned Aerial Vehicle (UAV) platform equipped with a hyperspectral camera to capture images over multiple time series, a rice yield classification model based on the XGBoost algorithm was proposed. Four comparison experiments were carried out through the intraline test and the interline test considering lodging characteristics at the midmature stage or not. The result revealed that the degree of lodging in the midmature stage was an important feature affecting the classification accuracy of rice. Thus, we developed a low-cost, high-throughput phenotyping and nondestructive method by combining UAV-based hyperspectral measurements and machine learning for estimation of rice yield to improve rice breeding efficiency.

Transfer learning method for plastic pollution evaluation in soil using NIR sensor

Plastic debris are ubiquitous in soil and bring severe threatening to environment and ecosystem. It is of great significance to extensively investigate the plastic pollution level in soil. An ultra-portable Near-infrared (NIR) sensor was used to detect plastic pollution level in soil. Soil samples were collected from three different regions and artificially polluted in two degrees (10-1.5% and 0.7-0.15%). Here, instead of constructing detection models for specific soil region, transfer learning approaches were explored to build classification model which could evaluate plastic pollution level in different soil regions simultaneously. The transfer learning algorithms, Manifold Embedded Distribution Alignment (MEDA) and Transfer Component Analysis (TCA), were employed for transfer learning model construction. Supporting Vector Machine (SVM) models were calibrated for transferability analysis and comparison. MEDA transferable models achieved the average classification accuracy of 97.78% in source soil regions and 79.52% in target soil regions. The average accuracy of TCA based models and conventional SVM models were equivalent to each other and lower than MEDA models. Besides, the average running time of MEDA method (0.70 s) was much lower than TCA based method (21.90 s) and conventional SVM models (41.38 s). Overall, the results indicated that transfer learning approaches especially MEDA method could work in a more efficient manner than that of conventional multivariate analysis. The ultra-portable NIR sensor in combination with MEDA transfer learning algorithm as modelling method was a promising solution for low-cost and efficient field detection of plastic contaminated level in soil.

Multispectral Models from Bare Soil Composites for Mapping Topsoil Properties over Europe

Reflectance of light across the visible, near-infrared and shortwave infrared (VIS-NIR-SWIR, 0.4–2.5 µm) spectral region is very useful for investigating mineralogical, physical and chemical properties of soils, which can reduce the need for traditional wet chemistry analyses. As many collections of multispectral satellite data are available for environmental studies, a large extent with medium resolution mapping could be benefited from the spectral measurements made from remote sensors. In this paper, we explored the use of bare soil composites generated from the large historical collections of Landsat images for mapping cropland topsoil attributes across the European extent. For this task, we used the Geospatial Soil Sensing System (GEOS3) for generating two bare soil composites of 30 m resolution (named synthetic soil images, SYSI), which were employed to represent the median topsoil reflectance of bare fields. The first (framed SYSI) was made with multitemporal images (2006–2012) framed to the survey time of the Land-Use/Land-Cover Area Frame Survey (LUCAS) soil dataset (2009), seeking to be more compatible to the soil condition upon the sampling campaign. The second (full SYSI) was generated from the full collection of Landsat images (1982–2018), which although displaced to the field survey, yields a higher proportion of bare areas for soil mapping. For evaluating the two SYSIs, we used the laboratory spectral data as a reference of topsoil reflectance to calculate the Spearman correlation coefficient. Furthermore, both SYSIs employed machine learning for calibrating prediction models of clay, sand, soil organic carbon (SOC), calcium carbonates (CaCO3), cation exchange capacity (CEC), and pH determined in water, using the gradient boosting regression algorithm. The original LUCAS laboratory spectra and a version of the data resampled to the Landsat multispectral bands were also used as reference of prediction performance using VIS-NIR-SWIR multispectral data. Our results suggest that generating a bare soil composite displaced to the survey time of soil observations did not improve the quality of topsoil reflectance, and consequently, the prediction performance of soil attributes. Despite the lower spectral resolution and the variability of soils in Europe, a SYSI calculated from the full collection of Landsat images can be employed for topsoil prediction of clay and CaCO3 contents with a moderate performance (testing R2, root mean square error (RMSE) and ratio of performance to interquartile range (RPIQ) of 0.44, 9.59, 1.77, and 0.36, 13.99, 1.54, respectively). Thus, this study shows that although there exist some constraints due to the spatial and temporal variation of soil exposures and among the Landsat sensors, it is possible to use bare soil composites for mapping key soil attributes of croplands across the European extent.

Multisource spectral-integrated estimation of cadmium concentrations in soil using a direct standardization and Spiking algorithm

Benefiting from the advantages of a wide spatial sampling range and strong continuity, hyperspectral analysis provides a potential way to detect heavy metals in soil. However, it is still a great challenge to identify the spectral response characteristics of heavy metals from naturally polluted soil samples. This paper innovatively produces near standard soil samples for exploring the exact spectral response of cadmium (Cd) in soil and presents a novel method by combining the direct standardization (DS) and Spiking algorithms for integrating multisource spectra to improve the accuracy of Cd concentration estimation. A total of 46 naturally polluted soil samples were collected from a known Cd-contaminated mining area in Xiangjiang River Basin, China. The soil spectra of the naturally polluted soil samples were synchronously measured in the field. Moreover, clean soils with low heavy metal contaminants were collected to produce 65 near standard soil samples with known Cd levels. Then, the spectra and Cd concentrations of all 111 soil samples were measured under laboratory conditions. The principle component stepwise regression (PCSR) analysis results illustrated that the reflectance at all the wavelengths (380-2460 nm) is indicative of the differences in the soil Cd concentrations. Among these, the sensitivity of the spectral reflectance is the strongest at approximately 400 nm, 1000 nm and above 2300 nm. Additionally, the integrated multisource spectra significantly improved the accuracy of soil Cd concentration estimation (coefficient of determination, R² = 0.96; root mean square error, RMSE = 0.29; ratio of prediction to deviation, RPD = 1.21) when 30 transfer samples and 15 training samples were simultaneously implemented in the combined DS and Spiking algorithm. This will provide a feasible scheme for exploration of spectral response characteristics of multiple soil heavy metals, and highlight the potential of developing low-level and satellite remote sensing on a large scale.

A Back Propagation Neural Network Model Optimized by Mind Evolutionary Algorithm for Estimating Cd, Cr, and Pb Concentrations in Soils Using Vis-NIR Diffuse Reflectance Spectroscopy

Visible and near infrared spectroscopy is an effective method for monitoring the content of heavy metals in soil. However, due to the difference between polluted soil with phytoremediation and without phytoremediation, the common estimation model cannot meet accuracy requirements. To solve this problem, combined with an ecological restoration experiment for soil contamination using the plant Neyraudia reynaudiana, this study explored the feasibility of using a hyperspectral technology to estimate the heavy metal content (Cd, Cr, and Pb) of soil under phytoremediation. A total of 108 surface soil samples (from depths of 0–20 cm) were collected. Inversion models were established using partial least squares regression (PLSR) and the back propagation neural network optimized by a mind evolutionary algorithm (MEA-BPNN). The results revealed that: (1) modeling with derivative-transformed spectra can effectively enhance the correlation between soil spectral reflectance and heavy metal content. (2) Compared with the BP neural network model, the estimation accuracy (R2) was improved from 0.728, 0.737, and 0.675 to 0.873, 0.884, and 0.857 using the MEA-BP neural network model. The residual prediction deviation (RPD) values for the three heavy metals Cd, Cr, and Pb using the MEA-BPNN model were 2.114, 3.000, and 2.560, respectively. Among them, the estimated model of Cd was an excellent prediction. (3) Compared with PLSR, the model prediction results established by the MEA-BP neural network had higher estimation accuracy. In summary, the use of diffuse reflectance spectroscopy to predict heavy metal content provides a theoretical basis for further study of the large-scale monitoring of soil heavy-metal pollution and its remediation evaluation in the polluted area, which is of great significance.

Spectral-Spatial Attention Networks for Hyperspectral Image Classification

Many deep learning models, such as convolutional neural network (CNN) and recurrent neural network (RNN), have been successfully applied to extracting deep features for hyperspectral tasks. Hyperspectral image classification allows distinguishing the characterization of land covers by utilizing their abundant information. Motivated by the attention mechanism of the human visual system, in this study, we propose a spectral-spatial attention network for hyperspectral image classification. In our method, RNN with attention can learn inner spectral correlations within a continuous spectrum, while CNN with attention is designed to focus on saliency features and spatial relevance between neighboring pixels in the spatial dimension. Experimental results demonstrate that our method can fully utilize the spectral and spatial information to obtain competitive performance.