Hyperspectral and Fluorescence Imaging Approaches for Nondestructive Detection of Rice Chlorophyll

Estimating and monitoring chlorophyll content is a critical step in crop spectral image analysis. The quick, non-destructive assessment of chlorophyll content in rice leaves can optimize nitrogen fertilization, benefit the environment and economy, and improve rice production management and quality. In this research, spectral analysis of rice leaves is performed using hyperspectral and fluorescence spectroscopy for the detection of chlorophyll content in rice leaves. This study generated ninety experimental spectral datasets by collecting rice leaf samples from a farm in Sichuan Province, China. By implementing a feature extraction algorithm, this study compresses redundant spectral bands and subsequently constructs machine learning models to reveal latent correlations among the extracted features. The prediction capabilities of six feature extraction methods and four machine learning algorithms in two types of spectral data are examined, and an accurate method of predicting chlorophyll concentration in rice leaves was devised. The IVSO-IVISSA (Iteratively Variable Subset Optimization-Interval Variable Iterative Space Shrinkage Approach) quadratic feature combination approach, based on fluorescence spectrum data, has the best prediction performance among the CNN+LSTM (Convolutional Neural Network Long Short-Term Memory) algorithms, with corresponding RMSE-Train (Root Mean Squared Error), RMSE-Test, and RPD (Ratio of standard deviation of the validation set to standard error of prediction) indexes of 0.26, 0.29, and 2.64, respectively. We demonstrated in this study that hyperspectral and fluorescence spectroscopy, when analyzed with feature extraction and machine learning methods, provide a new avenue for rapid and non-destructive crop health monitoring, which is critical to the advancement of smart and precision agriculture.

HyperFace: A Deep Fusion Model for Hyperspectral Face Recognition

Face recognition has been well studied under visible light and infrared (IR) in both intra-spectral and cross-spectral cases. However, how to fuse different light bands for face recognition, i.e., hyperspectral face recognition, is still an open research problem, which has the advantages of richer information retention and all-weather functionality over single-band face recognition. Thus, in this research, we revisit the hyperspectral recognition problem and provide a deep learning-based approach. A new fusion model (named HyperFace) is proposed to address this problem. The proposed model features a pre-fusion scheme, a Siamese encoder with bi-scope residual dense learning, a feedback-style decoder, and a recognition-oriented composite loss function. Experiments demonstrate that our method yields a much higher recognition rate than face recognition using only visible light or IR data. Moreover, our fusion model is shown to be superior to other general-purpose image fusion methods that are either traditional or deep learning-based, including state-of-the-art methods, in terms of both image quality and recognition performance.

Inside-out: Synergising leaf biochemical traits with stomatal-regulated water fluxes to enhance transpiration modelling during abiotic stress

As the global climate continues to change, plants will increasingly experience abiotic stress(es). Stomata on leaf surfaces are the gatekeepers to plant interiors, regulating gaseous exchanges that are crucial for both photosynthesis and outward water release. To optimise future crop productivity, accurate modelling of how stomata govern plant-environment interactions will be crucial. Here, we synergise optical and thermal imaging data to improve modelled transpiration estimates during water and/or nutrient stress (where leaf N is reduced). By utilising hyperspectral data and partial least squares regression analysis of six plant traits and fluxes in wheat (Triticum aestivum), we develop a new spectral vegetation index; the Combined Nitrogen and Drought Index (CNDI), which can be used to detect both water stress and/or nitrogen deficiency. Upon full stomatal closure during drought, CNDI shows a strong relationship with leaf water content (r2 = 0.70), with confounding changes in leaf biochemistry. By incorporating CNDI transformed with a sigmoid function into thermal-based transpiration modelling, we have increased the accuracy of modelling water fluxes during abiotic stress. These findings demonstrate the potential of using combined optical and thermal remote sensing-based modelling approaches to dynamically model water fluxes to improve both agricultural water usage and yields.

Corrigendum: Proximal hyperspectral imaging detects diurnal and drought-induced changes in maize physiology

[This corrects the article DOI: 10.3389/fpls.2021.640914.].

Research Progress on Quality Detection of Livestock and Poultry Meat Based on Machine Vision, Hyperspectral and Multi-Source Information Fusion Technologies

Presently, the traditional methods employed for detecting livestock and poultry meat predominantly involve sensory evaluation conducted by humans, chemical index detection, and microbial detection. While these methods demonstrate commendable accuracy in detection, their application becomes more challenging when applied to large-scale production by enterprises. Compared with traditional detection methods, machine vision and hyperspectral technology can realize real-time online detection of large throughput because of their advantages of high efficiency, accuracy, and non-contact measurement, so they have been widely concerned by researchers. Based on this, in order to further enhance the accuracy of online quality detection for livestock and poultry meat, this article presents a comprehensive overview of methods based on machine vision, hyperspectral, and multi-sensor information fusion technologies. This review encompasses an examination of the current research status and the latest advancements in these methodologies while also deliberating on potential future development trends. The ultimate objective is to provide pertinent information and serve as a valuable research resource for the non-destructive online quality detection of livestock and poultry meat.

Monitoring of Nitrogen Concentration in Soybean Leaves at Multiple Spatial Vertical Scales Based on Spectral Parameters

Nitrogen is a fundamental component for building amino acids and proteins, playing a crucial role in the growth and development of plants. Leaf nitrogen concentration (LNC) serves as a key indicator for assessing plant growth and development. Monitoring LNC provides insights into the absorption and utilization of nitrogen from the soil, offering valuable information for rational nutrient management. This, in turn, contributes to optimizing nutrient supply, enhancing crop yields, and minimizing adverse environmental impacts. Efficient and non-destructive estimation of crop LNC is of paramount importance for on-field crop management. Spectral technology, with its advantages of repeatability and high-throughput observations, provides a feasible method for obtaining LNC data. This study explores the responsiveness of spectral parameters to soybean LNC at different vertical scales, aiming to refine nitrogen management in soybeans. This research collected hyperspectral reflectance data and LNC data from different leaf layers of soybeans. Three types of spectral parameters, nitrogen-sensitive empirical spectral indices, randomly combined dual-band spectral indices, and "three-edge" parameters, were calculated. Four optimal spectral index selection strategies were constructed based on the correlation coefficients between the spectral parameters and LNC for each leaf layer. These strategies included empirical spectral index combinations (Combination 1), randomly combined dual-band spectral index combinations (Combination 2), "three-edge" parameter combinations (Combination 3), and a mixed combination (Combination 4). Subsequently, these four combinations were used as input variables to build LNC estimation models for soybeans at different vertical scales using partial least squares regression (PLSR), random forest (RF), and a backpropagation neural network (BPNN). The results demonstrated that the correlation coefficients between the LNC and spectral parameters reached the highest values in the upper soybean leaves, with most parameters showing significant correlations with the LNC (p < 0.05). Notably, the reciprocal difference index (VI6) exhibited the highest correlation with the upper-layer LNC at 0.732, with a wavelength combination of 841 nm and 842 nm. In constructing the LNC estimation models for soybeans at different leaf layers, the accuracy of the models gradually improved with the increasing height of the soybean plants. The upper layer exhibited the best estimation performance, with a validation set coefficient of determination (R2) that was higher by 9.9% to 16.0% compared to other layers. RF demonstrated the highest accuracy in estimating the upper-layer LNC, with a validation set R2 higher by 6.2% to 8.8% compared to other models. The RMSE was lower by 2.1% to 7.0%, and the MRE was lower by 4.7% to 5.6% compared to other models. Among different input combinations, Combination 4 achieved the highest accuracy, with a validation set R2 higher by 2.3% to 13.7%. In conclusion, by employing Combination 4 as the input, the RF model achieved the optimal estimation results for the upper-layer LNC, with a validation set R2 of 0.856, RMSE of 0.551, and MRE of 10.405%. The findings of this study provide technical support for remote sensing monitoring of soybean LNCs at different spatial scales.

HyperTRCSS: A hyperspectral time-resolved compressive sensing spectrometer for depth-sensitive monitoring of cytochrome-c-oxidase and blood oxygenation

Significance

Hyperspectral time-resolved (TR) near-infrared spectroscopy offers the potential to monitor cytochrome-c-oxidase (oxCCO) and blood oxygenation in the adult brain with minimal scalp/skull contamination. We introduce a hyperspectral TR spectrometer that uses compressive sensing to minimize acquisition time without compromising spectral range or resolution and demonstrate oxCCO and blood oxygenation monitoring in deep tissue.

Aim

Develop a hyperspectral TR compressive sensing spectrometer and use it to monitor oxCCO and blood oxygenation in deep tissue.

Approach

Homogeneous tissue-mimicking phantom experiments were conducted to confirm the spectrometer's sensitivity to oxCCO and blood oxygenation. Two-layer phantoms were used to evaluate the spectrometer's sensitivity to oxCCO and blood oxygenation in the bottom layer through a 10 mm thick static top layer.

Results

The spectrometer was sensitive to oxCCO and blood oxygenation changes in the bottom layer of the two-layer phantoms, as confirmed by concomitant measurements acquired directly from the bottom layer. Measures of oxCCO and blood oxygenation by the spectrometer were highly correlated with "gold standard" measures in the homogeneous and two-layer phantom experiments.

Conclusions

The results show that the hyperspectral TR compressive sensing spectrometer is sensitive to changes in oxCCO and blood oxygenation in deep tissue through a thick static top layer.

Research on the Detection Method of Organic Matter in Tea Garden Soil Based on Image Information and Hyperspectral Data Fusion

Soil organic matter is an important component that reflects soil fertility and promotes plant growth. The soil of typical Chinese tea plantations was used as the research object in this work, and by combining soil hyperspectral data and image texture characteristics, a quantitative prediction model of soil organic matter based on machine vision and hyperspectral imaging technology was built. Three methods, standard normalized variate (SNV), multisource scattering correction (MSC), and smoothing, were first used to preprocess the spectra. After that, random frog (RF), variable combination population analysis (VCPA), and variable combination population analysis and iterative retained information variable (VCPA-IRIV) algorithms were used to extract the characteristic bands. Finally, the quantitative prediction model of nonlinear support vector regression (SVR) and linear partial least squares regression (PLSR) for soil organic matter was established by combining nine color features and five texture features of hyperspectral images. The outcomes demonstrate that, in comparison to single spectral data, fusion data may greatly increase the performance of the prediction model, with MSC + VCPA-IRIV + SVR (R2C = 0.995, R2P = 0.986, RPD = 8.155) being the optimal approach combination. This work offers excellent justification for more investigation into nondestructive methods for determining the amount of organic matter in soil.

Discrimination of Hover Fly Species and Sexes by Wing Interference Signals

Remote automated surveillance of insect abundance and diversity is poised to revolutionize insect decline studies. The study reveals spectral analysis of thin-film wing interference signals (WISs) can discriminate free-flying insects beyond what can be accomplished by machine vision. Detectable by photonic sensors, WISs are robust indicators enabling species and sex identification. The first quantitative survey of insect wing thickness and modulation through shortwave-infrared hyperspectral imaging of 600 wings from 30 hover fly species is presented. Fringy spectral reflectance of WIS can be explained by four optical parameters, including membrane thickness. Using a Naïve Bayes Classifier with five parameters that can be retrieved remotely, 91% is achieved accuracy in identification of species and sexes. WIS-based surveillance is therefore a potent tool for remote insect identification and surveillance.

Integrating spectral and image information for prediction of cottonseed vitality

Cotton plays a significant role in people's lives, and cottonseeds serve as a vital assurance for successful cotton cultivation and production. Premium-quality cottonseeds can significantly enhance the germination rate of cottonseeds, resulting in increased cotton yields. The vitality of cottonseeds is a crucial metric that reflects the quality of the seeds. However, currently, the industry lacks a non-destructive method to directly assess cottonseed vitality without compromising the integrity of the seeds. To address this challenge, this study employed a hyperspectral imaging acquisition system to gather hyperspectral data on cottonseeds. This system enables the simultaneous collection of hyperspectral data from 25 cottonseeds. This study extracted spectral and image information from the hyperspectral data of cottonseeds to predict their vitality. SG, SNV, and MSC methods were utilized to preprocess the spectral data of cottonseeds. Following this preprocessing step, feature wavelength points of the cottonseeds were extracted using SPA and CARS algorithms. Subsequently, GLCM was employed to extract texture features from images corresponding to these feature wavelength points, including attributes such as Contrast, Correlation, Energy, and Entropy. Finally, the vitality of cottonseeds was predicted using PLSR, SVR, and a self-built 1D-CNN model. For spectral data analysis, the 1D-CNN model constructed after MSC+CARS preprocessing demonstrated the highest performance, achieving a test set correlation coefficient of 0.9214 and an RMSE of 0.7017. For image data analysis, the 1D-CNN model constructed after SG+CARS preprocessing outperformed the others, yielding a test set correlation coefficient of 0.8032 and an RMSE of 0.9683. In the case of fused spectral and image data, the 1D-CNN model built after SG+SPA preprocessing displayed the best performance, attaining a test set correlation coefficient of 0.9427 and an RMSE of 0.6872. These findings highlight the effectiveness of the 1D-CNN model and the fusion of spectral and image features for cottonseed vitality prediction. This research contributes significantly to the development of automated detection devices for assessing cottonseed vitality.

Classification models for Tobacco Mosaic Virus and Potato Virus Y using hyperspectral and machine learning techniques

Tobacco Mosaic Virus (TMV) and Potato Virus Y (PVY) pose significant threats to crop production. Non-destructive and accurate surveillance is crucial to effective disease control. In this study, we propose the adoption of hyperspectral and machine learning technologies to discern the type and severity of tobacco leaves affected by PVY and TMV infection. Initially, we applied three preprocessing methods - Multivariate Scattering Correction (MSC), Standard Normal Variate (SNV), and Savitzky-Golay smoothing filter (SavGol) - to corrected the leaf full-length spectral sheet data (350-2500nm). Subsequently, we employed two classifiers, support vector machine (SVM) and random forest (RF), to establish supervised classification models, including binary classification models (healthy/diseased leaves or PVY/TMV infected leaves) and six-class classification models (healthy and various severity levels of diseased leaves). Based on the core evaluation index, our models achieved accuracies in the range of 91-100% in the binary classification. In general, SVM demonstrated superior performance compared to RF in distinguishing leaves infected with PVY and TMV. Different combinations of preprocessing methods and classifiers have distinct capabilities in the six-class classification. Notably, SavGol united with SVM gave an excellent performance in the identification of different PVY severity levels with 98.1% average precision, and also achieved a high recognition rate (96.2%) in the different TMV severity level classifications. The results further highlighted that the effective wavelengths captured by SVM, 700nm and 1800nm, would be valuable for estimating disease severity levels. Our study underscores the efficacy of integrating hyperspectral technology and machine learning, showcasing their potential for accurate and non-destructive monitoring of plant viral diseases.

Combining Vis-NIR and NIR Spectral Imaging Techniques with Data Fusion for Rapid and Nondestructive Multi-Quality Detection of Cherry Tomatoes

Firmness, soluble solid content (SSC) and titratable acidity (TA) are characteristic substances for evaluating the quality of cherry tomatoes. In this paper, a hyper spectral imaging (HSI) system using visible/near-infrared (Vis-NIR) and near-infrared (NIR) was proposed to detect the key qualities of cherry tomatoes. The effects of individual spectral information and fused spectral information in the detection of different qualities were compared for firmness, SSC and TA of cherry tomatoes. Data layer fusion combined with multiple machine learning methods including principal component regression (PCR), partial least squares regression (PLSR), support vector regression (SVR) and back propagation neural network (BP) is used for model training. The results show that for firmness, SSC and TA, the determination coefficient R2 of the multi-quality prediction model established by Vis-NIR spectra is higher than that of NIR spectra. The R2 of the best model obtained by SSC and TA fusion band is greater than 0.9, and that of the best model obtained by the firmness fusion band is greater than 0.85. It is better to use the spectral bands after information fusion for nondestructive quality detection of cherry tomatoes. This study shows that hyperspectral imaging technology can be used for the nondestructive detection of multiple qualities of cherry tomatoes, and the method based on the fusion of two spectra has a better prediction effect for the rapid detection of multiple qualities of cherry tomatoes compared with a single spectrum. This study can provide certain technical support for the rapid nondestructive detection of multiple qualities in other melons and fruits.

Pancreatic Islet Viability Assessment Using Hyperspectral Imaging of Autofluorescence

Islets prepared for transplantation into type 1 diabetes patients are exposed to compromising intrinsic and extrinsic factors that contribute to early graft failure, necessitating repeated islet infusions for clinical insulin independence. A lack of reliable pre-transplant measures to determine islet viability severely limits the success of islet transplantation and will limit future beta cell replacement strategies. We applied hyperspectral fluorescent microscopy to determine whether we could non-invasively detect islet damage induced by oxidative stress, hypoxia, cytokine injury, and warm ischaemia, and so predict transplant outcomes in a mouse model. In assessing islet spectral signals for NAD(P)H, flavins, collagen-I, and cytochrome-C in intact islets, we distinguished islets compromised by oxidative stress (ROS) (AUC = 1.00), hypoxia (AUC = 0.69), cytokine exposure (AUC = 0.94), and warm ischaemia (AUC = 0.94) compared to islets harvested from pristine anaesthetised heart-beating mouse donors. Significantly, with unsupervised assessment we defined an autofluorescent score for ischaemic islets that accurately predicted the restoration of glucose control in diabetic recipients following transplantation. Similar results were obtained for islet single cell suspensions, suggesting translational utility in the context of emerging beta cell replacement strategies. These data show that the pre-transplant hyperspectral imaging of islet autofluorescence has promise for predicting islet viability and transplant success.

Determining the Spectral Requirements for Cyanobacteria Detection for the CyanoSat Hyperspectral Imager with Machine Learning

This study determines an optimal spectral configuration for the CyanoSat imager for the discrimination and retrieval of cyanobacterial pigments using a simulated dataset with machine learning (ML). A minimum viable spectral configuration with as few as three spectral bands enabled the determination of cyanobacterial pigments phycocyanin (PC) and chlorophyll-a (Chl-a) but may not be suitable for determining cyanobacteria composition. A spectral configuration with about nine ideally positioned spectral bands enabled estimation of the cyanobacteria-to-algae ratio (CAR) and pigment concentrations with almost the same accuracy as using all 300 spectral channels. A narrower spectral band full-width half-maximum (FWHM) did not provide improved performance compared to the nominal 12 nm configuration. In conclusion, continuous sampling of the visible spectrum is not a requirement for cyanobacterial detection, provided that a multi-spectral configuration with ideally positioned, narrow bands is used. The spectral configurations identified here could be used to guide the selection of bands for future ocean and water color radiometry sensors.

Emerging clinical applications in oncology for non-invasive multi- and hyperspectral imaging of cell and tissue autofluorescence

Hyperspectral and multispectral imaging of cell and tissue autofluorescence is an emerging technology in which fluorescence imaging is applied to biological materials across multiple spectral channels. This produces a stack of images where each matched pixel contains information about the sample's spectral properties at that location. This allows precise collection of molecularly specific data from a broad range of native fluorophores. Importantly, complex information, directly reflective of biological status, is collected without staining and tissues can be characterised in situ, without biopsy. For oncology, this can spare the collection of biopsies from sensitive regions and enable accurate tumour mapping. For in vivo tumour analysis, the greatest focus has been on oral cancer, whereas for ex vivo assessment head-and-neck cancers along with colon cancer have been the most studied, followed by oral and eye cancer. This review details the scope and progress of research undertaken towards clinical translation in oncology.

Rapid and Non-Destructive Estimation of Moisture Content in Caragana Korshinskii Pellet Feed Using Hyperspectral Imaging

Moisture content is an important parameter for estimating the quality of pellet feed, which is vital in nutrition, storage, and taste. The ranges of moisture content serve as an index for factors such as safe storage and nutrition stability. A rapid and non-destructive model for the measurement of moisture content in pellet feed was developed. To achieve this, 144 samples of Caragana korshinskii pellet feed from various regions in Inner Mongolia Autonomous Region underwent separate moisture content control, measurement using standard methods, and captured their images using a hyperspectral imaging (HSI) system in the spectral range of 935.5-2539 nm. The Monte Carlo cross validation (MCCV) was used to eliminate abnormal sample data from the spectral data for better model accuracy, and a global model of moisture content was built by using partial least squares regression (PLSR) with seven preprocessing techniques and two spectral feature extraction techniques. The results showed that the regression model developed by PLSR based on second derivative (SD) and competitive adaptive reweighted sampling (CARS) resulted in better performance for moisture content. The model showed predictive abilities for moisture content with a coefficient of determination of 0.9075 and a root mean square error (RMSE) of 0.4828 for the training set; and a coefficient of determination of 0.907 and a root mean square error (RMSE) of 0.5267 for the test set; and a relative prediction error of 3.3 and the standard error of 0.307.

Low-Temperature Emission Dynamics of Methylammonium Lead Bromide Hybrid Perovskite Thin Films at the Sub-Micrometer Scale

We study the low-temperature (T = 4.7 K) emission dynamics of a thin film of methylammonium lead bromide (MAPbBr3), prepared via the anti-solvent method. Using intensity-dependent (over 5 decades) hyperspectral microscopy under quasi-resonant (532 nm) continuous wave excitation, we revealed spatial inhomogeneities in the thin film emission. This was drastically different at the band-edge (∼550 nm, sharp peaks) than in the emission tail (∼568 nm, continuum of emission). We are able to observe regions of the film at the micrometer scale where emission is dominated by excitons, in between regions of trap emission. Varying the density of absorbed photons by the MAPbBr3 thin films, two-color fluorescence lifetime imaging microscopy unraveled the emission dynamics: a fast, resolution-limited (∼200 ps) monoexponential tangled with a stretched exponential decay. We associate the first to the relaxation of excitons and the latter to trap emission dynamics. The obtained stretching exponents can be interpreted as the result of a two-dimensional electron diffusion process: Förster resonant transfer mechanism. Furthermore, the non-vanishing fast monoexponential component even in the tail of the MAPbBr3 emission indicates the subsistence of localized excitons. Finally, we estimate the density of traps in MAPbBr3 thin films prepared using the anti-solvent method at n∼1017 cm-3.

Study on the Adsorption Deformation of a Substrate via Spin Coating Based on the 3D-DIC Method and Its Effect on the Homogeneity of Perovskite Films

The physical and chemical stability of perovskite films has always been a key issue for their industrialization, which has been extensively studied in terms of materials, environment, and encapsulation. Spin coating is one of the most commonly used methods for the preparation of perovskite films in research. However, little attention has been paid to the deformation state of the substrate when it is fixed by means of adsorption and its impact. In this work, the three-dimensional digital image correlation (3D-DIC) method and hyperspectral technology are used to acquire and analyze the adsorption deformation characteristics of the substrate during spin coating, as well as the resulting inhomogeneity. Plastic and four different thicknesses of float glass (0.2, 0.5, 0.7, 1.1 mm) were selected as substrates, and they were tested separately on two suction cups with different structures. The results show that the plastic and 0.2 mm specimens exhibit obvious strain localization behavior. The distribution and magnitude of the strain are affected by the size of the sucker structure, especially the width of the groove. For glass specimens, this effect shows a nonlinear decrease with increasing substrate thickness. Compared to the strain value, the irregularity of local deformation has a greater impact on the non-uniform distribution of materials. Finally, inhomogeneities in the perovskite films were observed through optical lens and hyperspectral data. Obviously, the deformation of the substrate caused by adsorption should attract the attention of researchers, especially for flexible or rigid substrates with low thickness. This may affect the centrifugal diffusion path of the precursor, causing microstructure inhomogeneity and residual stress, etc.

The prediction model of nitrogen nutrition in cotton canopy leaves based on hyperspectral visible-near infrared band feature fusion

Hyperspectral remote sensing technology is becoming increasingly popular in various fields due to its ability to provide detailed information about crop growth and nutritional status. The use of hyperspectral technology to predict SPAD (Soil and Plant Analyzer Development) values during cotton growth and adopt precise fertilization management measures is crucial for achieving high yield and fertilizer efficiency. To detect the nitrogen nutrition in cotton canopy leaves quickly, a non-destructive nitrogen nutrition retrieval model was proposed based on the spectral fusion features of the cotton canopy. The hyperspectral vegetation index and multifractal features were fused to predict the SPAD value and identify the amount of fertilizer applied at different levels. The random decision forest algorithm was used as the model predictor and classifier. A method was introduced which was widely used in the fields of finance and stocks (MF-DFA) into the field of agriculture to extract fractal features of cotton spectral reflectance. Comparing the fusion feature with multi-fractal feature and vegetation index, the results showed that the fusion feature parameters had higher accuracy and better stability than using a single feature or feature combination. The R2 was as high as 0.8363, and the RMSE was 1.8767%. Our intelligent model provides a new idea for detecting nitrogen nutrition in cotton canopy leaves rapidly.

[Origin identification of Polygonatum cyrtonema based on hyperspectral data]

In this study, visual-near infrared(VNIR), short-wave infrared(SWIR), and VNIR + SWIR fusion hyperspectral data of Polygonatum cyrtonema from different geographical origins were collected and preprocessed by first derivative(FD), second derivative(SD), Savitzky-Golay smoothing(S-G), standard normalized variate(SNV), multiplicative scatter correction(MSC), FD+S-G, and SD+S-G. Three algorithms, namely random forest(RF), linear support vector classification(LinearSVC), and partial least squares discriminant analysis(PLS-DA), were used to establish the identification models of P. cyrtonema origin from three spatial scales, i.e., province, county, and township, respectively. Successive projection algorithm(SPA) and competitive adaptive reweighted sampling(CARS) were used to screen the characteristic bands, and the P. cyrtonema origin identification models were established according to the selected characteristic bands. The results showed that(1)after FD preprocessing of VNIR+SWIR fusion hyperspectral data, the accuracy of recognition models established using LinearSVC was the highest, reaching 99.97% and 99.82% in the province origin identification model, 100.00% and 99.46% in the county origin identification model, and 99.62% and 98.39% in the township origin identification model. The accuracy of province, county, and township origin identification models reached more than 98.00%.(2)Among the 26 characteristic bands selected by CARS, after FD pretreatment, the accuracy of origin identification models of different spatial scales was the highest using LinearSVC, reaching 98.59% and 97.05% in the province origin identification model, 97.79% and 94.75% in the county origin identification model, and 90.13% and 87.95% in the township origin identification model. The accuracy of identification models of different spatial scales established by 26 characteristic bands reached more than 87.00%. The results show that hyperspectral imaging technology can realize accurate identification of P. cyrtonema origin from different spatial scales.

Effect of Spatial and Spectral Scaling on Joint Characterization of the Spectral Mixture Residual: Comparative Analysis of AVIRIS and WorldView-3 SWIR for Geologic Mapping in Anza-Borrego Desert State Park

A geologic map is both a visual depiction of the lithologies and structures occurring at the Earth's surface and a representation of a conceptual model for the geologic history in a region. The work needed to capture such multifaced information in an accurate geologic map is time consuming. Remote sensing can complement traditional primary field observations, geochemistry, chronometry, and subsurface geophysical data in providing useful information to assist with the geologic mapping process. Two novel sources of remote sensing data are particularly relevant for geologic mapping applications: decameter-resolution imaging spectroscopy (spectroscopic imaging) and meter-resolution multispectral shortwave infrared (SWIR) imaging. Decameter spectroscopic imagery can capture important mineral absorptions but is frequently unable to spatially resolve important geologic features. Meter-resolution multispectral SWIR images are better able to resolve fine spatial features but offer reduced spectral information. Such disparate but complementary datasets can be challenging to integrate into the geologic mapping process. Here, we conduct a comparative analysis of spatial and spectral scaling for two such datasets: one Airborne Visible/Infrared Imaging Spectrometer-Classic (AVIRIS-classic) flightline, and one WorldView-3 (WV3) scene, for a geologically complex landscape in Anza-Borrego Desert State Park, California. To do so, we use a two-stage framework that synthesizes recent advances in the spectral mixture residual and joint characterization. The mixture residual uses the wavelength-explicit misfit of a linear spectral mixture model to capture low variance spectral signals. Joint characterization utilizes nonlinear dimensionality reduction (manifold learning) to visualize spectral feature space topology and identify clusters of statistically similar spectra. For this study area, the spectral mixture residual clearly reveals greater spectral dimensionality in AVIRIS than WorldView (99% of variance in 39 versus 5 residual dimensions). Additionally, joint characterization shows more complex spectral feature space topology for AVIRIS than WorldView, revealing information useful to the geologic mapping process in the form of mineralogical variability both within and among mapped geologic units. These results illustrate the potential of recent and planned imaging spectroscopy missions to complement high-resolution multispectral imagery-along with field and lab observations-in planning, collecting, and interpreting the results from geologic field work.

Row selection in remote sensing from four-row plots of maize and sorghum based on repeatability and predictive modeling

Remote sensing enables the rapid assessment of many traits that provide valuable information to plant breeders throughout the growing season to improve genetic gain. These traits are often extracted from remote sensing data on a row segment (rows within a plot) basis enabling the quantitative assessment of any row-wise subset of plants in a plot, rather than a few individual representative plants, as is commonly done in field-based phenotyping. Nevertheless, which rows to include in analysis is still a matter of debate. The objective of this experiment was to evaluate row selection and plot trimming in field trials conducted using four-row plots with remote sensing traits extracted from RGB (red-green-blue), LiDAR (light detection and ranging), and VNIR (visible near infrared) hyperspectral data. Uncrewed aerial vehicle flights were conducted throughout the growing seasons of 2018 to 2021 with data collected on three years of a sorghum experiment and two years of a maize experiment. Traits were extracted from each plot based on all four row segments (RS) (RS1234), inner rows (RS23), outer rows (RS14), and individual rows (RS1, RS2, RS3, and RS4). Plot end trimming of 40 cm was an additional factor tested. Repeatability and predictive modeling of end-season yield were used to evaluate performance of these methodologies. Plot trimming was never shown to result in significantly different outcomes from non-trimmed plots. Significant differences were often observed based on differences in row selection. Plots with more row segments were often favorable for increasing repeatability, and excluding outer rows improved predictive modeling. These results support long-standing principles of experimental design in agronomy and should be considered in breeding programs that incorporate remote sensing.

Capturing patterns of evolutionary relatedness with reflectance spectra to model and monitor biodiversity

Biogeographic history can set initial conditions for vegetation community assemblages that determine their climate responses at broad extents that land surface models attempt to forecast. Numerous studies have indicated that evolutionarily conserved biochemical, structural, and other functional attributes of plant species are captured in visible-to-short wavelength infrared, 400 to 2,500 nm, reflectance properties of vegetation. Here, we present a remotely sensed phylogenetic clustering and an evolutionary framework to accommodate spectra, distributions, and traits. Spectral properties evolutionarily conserved in plants provide the opportunity to spatially aggregate species into lineages (interpreted as "lineage functional types" or LFT) with improved classification accuracy. In this study, we use Airborne Visible/Infrared Imaging Spectrometer data from the 2013 Hyperspectral Infrared Imager campaign over the southern Sierra Nevada, California flight box, to investigate the potential for incorporating evolutionary thinking into landcover classification. We link the airborne hyperspectral data with vegetation plot data from 1372 surveys and a phylogeny representing 1,572 species. Despite temporal and spatial differences in our training data, we classified plant lineages with moderate reliability (Kappa = 0.76) and overall classification accuracy of 80.9%. We present an assessment of classification error and detail study limitations to facilitate future LFT development. This work demonstrates that lineage-based methods may be a promising way to leverage the new-generation high-resolution and high return-interval hyperspectral data planned for the forthcoming satellite missions with sparsely sampled existing ground-based ecological data.

Cuticle architecture and mechanical properties: a functional relationship delineated through correlated multimodal imaging

Cuticles are multifunctional hydrophobic biocomposites that protect the aerial organs of plants. During plant development, plant cuticles must accommodate different mechanical constraints combining extensibility and stiffness, and the corresponding relationships with their architecture are unknown. Recent data showed a fine-tuning of cuticle architecture during fruit development, with several chemical clusters which raise the question of how they impact the mechanical properties of cuticles. We investigated the in-depth nanomechanical properties of tomato (Solanum lycopersicum) fruit cuticle from early development to ripening, in relation to chemical and structural heterogeneities by developing a correlative multimodal imaging approach. Unprecedented sharps heterogeneities were evidenced including an in-depth mechanical gradient and a 'soft' central furrow that were maintained throughout the plant development despite the overall increase in elastic modulus. In addition, we demonstrated that these local mechanical areas are correlated to chemical and structural gradients. This study shed light on fine-tuning of mechanical properties of cuticles through the modulation of their architecture, providing new insight for our understanding of structure-function relationships of plant cuticles and for the design of bioinspired material.

Evolutionary history explains foliar spectral differences between arbuscular and ectomycorrhizal plant species

Leaf spectra are integrated foliar phenotypes that capture a range of traits and can provide insight into ecological processes. Leaf traits, and therefore leaf spectra, may reflect belowground processes such as mycorrhizal associations. However, evidence for the relationship between leaf traits and mycorrhizal association is mixed, and few studies account for shared evolutionary history. We conduct partial least squares discriminant analysis to assess the ability of spectra to predict mycorrhizal type. We model the evolution of leaf spectra for 92 vascular plant species and use phylogenetic comparative methods to assess differences in spectral properties between arbuscular mycorrhizal and ectomycorrhizal plant species. Partial least squares discriminant analysis classified spectra by mycorrhizal type with 90% (arbuscular) and 85% (ectomycorrhizal) accuracy. Univariate models of principal components identified multiple spectral optima corresponding with mycorrhizal type due to the close relationship between mycorrhizal type and phylogeny. Importantly, we found that spectra of arbuscular mycorrhizal and ectomycorrhizal species do not statistically differ from each other after accounting for phylogeny. While mycorrhizal type can be predicted from spectra, enabling the use of spectra to identify belowground traits using remote sensing, this is due to evolutionary history and not because of fundamental differences in leaf spectra due to mycorrhizal type.

Intra- and interspecific variation in spectral properties of dominant Sphagnum moss species in boreal peatlands

Boreal peatlands store ~25 % of global soil organic carbon and host many endangered species; however, they face degradation due to climate change and anthropogenic drainage. In boreal peatlands, vegetation indicates ecohydrological conditions of the ecosystem. Applying remote sensing would enable spatially and temporally continuous monitoring of peatland vegetation. New multi- and hyperspectral satellite data offer promising approaches for understanding the spectral properties of peatland vegetation at high temporal and spectral resolutions. However, using spectral satellite data to their fullest potential requires detailed spectral analyses of dominant species in peatlands. A dominant feature of peatland vegetation is the genus Sphagnum mosses. We investigated how the reflectance spectra of common boreal Sphagnum mosses, collected from waterlogged natural conditions after snowmelt, change when the mosses are desiccated. We conducted a laboratory experiment where the reflectance spectra (350-2500 nm) and the mass of 90 moss samples (representing nine species) were measured repetitively. Furthermore, we examined (i) their inter- and intraspecific spectral differences and (ii) whether the species or their respective habitats could be identified based on their spectral signatures in varying states of drying. Our findings show that the most informative spectral regions to retrieve information about the Sphagnum species and their state of desiccation are in the shortwave infrared region. Furthermore, the visible and near-infrared spectral regions contain less information on species and moisture content. Our results also indicate that hyperspectral data can, to a limited extent, be used to separate mosses belonging to meso- and ombrotrophic habitats. Overall, this study demonstrates the importance of including data especially from the shortwave infrared region (1100-2500 nm) in remote sensing applications of boreal peatlands. The spectral library of Sphagnum mosses collected in this study is available as open data and can be used to develop new methods for remote monitoring of boreal peatlands.

Phenomic and genomic prediction of yield on multiple locations in winter wheat

Genomic selection has recently become an established part of breeding strategies in cereals. However, a limitation of linear genomic prediction models for complex traits such as yield is that these are unable to accommodate Genotype by Environment effects, which are commonly observed over trials on multiple locations. In this study, we investigated how this environmental variation can be captured by the collection of a large number of phenomic markers using high-throughput field phenotyping and whether it can increase GS prediction accuracy. For this purpose, 44 winter wheat (Triticum aestivum L.) elite populations, comprising 2,994 lines, were grown on two sites over 2 years, to approximate the size of trials in a practical breeding programme. At various growth stages, remote sensing data from multi- and hyperspectral cameras, as well as traditional ground-based visual crop assessment scores, were collected with approximately 100 different data variables collected per plot. The predictive power for grain yield was tested for the various data types, with or without genome-wide marker data sets. Models using phenomic traits alone had a greater predictive value (R² = 0.39-0.47) than genomic data (approximately R² = 0.1). The average improvement in predictive power by combining trait and marker data was 6%-12% over the best phenomic-only model, and performed best when data from one full location was used to predict the yield on an entire second location. The results suggest that genetic gain in breeding programmes can be increased by utilisation of large numbers of phenotypic variables using remote sensing in field trials, although at what stage of the breeding cycle phenomic selection could be most profitably applied remains to be answered.

A Vicarious Technique for Understanding and Diagnosing Hyperspectral Spatial Misregistration

Pushbroom hyperspectral imaging (HSI) systems intrinsically measure our surroundings by leveraging 1D spatial imaging, where each pixel contains a unique spectrum of the observed materials. Spatial misregistration is an important property of HSI systems because it defines the spectral integrity of spatial pixels and requires characterization. The IEEE P4001 Standards Association committee has defined laboratory-based methods to test the ultimate limit of HSI systems but negates any impacts from mounting and flying the instruments on airborne platforms such as unmanned aerial vehicles (UAV's) or drones. Our study was designed to demonstrate a novel vicarious technique using convex mirrors to bridge the gap between laboratory and field-based HSI performance testing with a focus on extracting hyperspectral spatial misregistration. A fast and simple extraction technique is proposed for estimating the sampled Point Spread Function's width, along with keystone, as a function of wavelength for understanding the key contributors to hyperspectral spatial misregistration. With the ease of deploying convex mirrors, off-axis spatial misregistration is assessed and compared with on-axis behavior, where the best performance is often observed. In addition, convex mirrors provide an easy methodology to exploit ortho-rectification errors related to fixed pushbroom HSI systems, which we will show. The techniques discussed in this study are not limited to drone-based systems but can be easily applied to other airborne or satellite-based systems.

Hyperspectral Characteristic Band Selection and Estimation Content of Soil Petroleum Hydrocarbon Based on GARF-PLSR

With continuous improvements in oil production, the environmental problems caused by oil exploitation are becoming increasingly serious. Rapid and accurate estimation of soil petroleum hydrocarbon content is of great significance to the investigation and restoration of environments in oil-producing areas. In this study, the content of petroleum hydrocarbon and the hyperspectral data of soil samples collected from an oil-producing area were measured. For the hyperspectral data, spectral transforms, including continuum removal (CR), first- and second-order differential (CR-FD, CR-SD), and Napierian logarithm (CR-LN), were applied to eliminate background noise. At present, there are some shortcomings in the method of feature band selection, such as large quantity, time of calculation, and unclear importance of each feature band obtained. Meanwhile, redundant bands easily exist in the feature set, which seriously affects the accuracy of the inversion algorithm. In order to solve the above problems, a new method (GARF) for hyperspectral characteristic band selection was proposed. It combined the advantage that the grouping search algorithm can effectively reduce the calculation time with the advantage that the point-by-point search algorithm can determine the importance of each band, which provided a clearer direction for further spectroscopic research. The 17 selected bands were used as the input data of partial least squares regression (PLSR) and K-nearest neighbor (KNN) algorithms to estimate soil petroleum hydrocarbon content, and the leave-one-out method was used for cross-validation. The root mean squared error (RMSE) and coefficient of determination (R²) of the estimation result were 3.52 and 0.90, which implemented a high accuracy with only 8.37% of the entire bands. The results showed that compared with the traditional characteristic band selection methods, GARF can effectively reduce the redundant bands and screen out the optimal characteristic bands in the hyperspectral data of soil petroleum hydrocarbon with the method of importance assessment, which retained the physical meaning. It provided a new idea for the research of other substances in soil.

Hybrid Multi-Dimensional Attention U-Net for Hyperspectral Snapshot Compressive Imaging Reconstruction

In order to capture the spatial-spectral (x,y,λ) information of the scene, various techniques have been proposed. Different from the widely used scanning-based methods, spectral snapshot compressive imaging (SCI) utilizes the idea of compressive sensing to compressively capture the 3D spatial-spectral data-cube in a single-shot 2D measurement and thus it is efficient, enjoying the advantages of high-speed and low bandwidth. However, the reconstruction process, i.e., to retrieve the 3D cube from the 2D measurement, is an ill-posed problem and it is challenging to reconstruct high quality images. Previous works usually use 2D convolutions and preliminary attention to address this challenge. However, these networks and attention do not exactly extract spectral features. On the other hand, 3D convolutions can extract more features in a 3D cube, but increase computational cost significantly. To balance this trade-off, in this paper, we propose a hybrid multi-dimensional attention U-Net (HMDAU-Net) to reconstruct hyperspectral images from the 2D measurement in an end-to-end manner. HMDAU-Net integrates 3D and 2D convolutions in an encoder-decoder structure to fully utilize the abundant spectral information of hyperspectral images with a trade-off between performance and computational cost. Furthermore, attention gates are employed to highlight salient features and suppress the noise carried by the skip connections. Our proposed HMDAU-Net achieves superior performance over previous state-of-the-art reconstruction algorithms.

A novel transfer learning framework for sorghum biomass prediction using UAV-based remote sensing data and genetic markers

Yield for biofuel crops is measured in terms of biomass, so measurements throughout the growing season are crucial in breeding programs, yet traditionally time- and labor-consuming since they involve destructive sampling. Modern remote sensing platforms, such as unmanned aerial vehicles (UAVs), can carry multiple sensors and collect numerous phenotypic traits with efficient, non-invasive field surveys. However, modeling the complex relationships between the observed phenotypic traits and biomass remains a challenging task, as the ground reference data are very limited for each genotype in the breeding experiment. In this study, a Long Short-Term Memory (LSTM) based Recurrent Neural Network (RNN) model is proposed for sorghum biomass prediction. The architecture is designed to exploit the time series remote sensing and weather data, as well as static genotypic information. As a large number of features have been derived from the remote sensing data, feature importance analysis is conducted to identify and remove redundant features. A strategy to extract representative information from high-dimensional genetic markers is proposed. To enhance generalization and minimize the need for ground reference data, transfer learning strategies are proposed for selecting the most informative training samples from the target domain. Consequently, a pre-trained model can be refined with limited training samples. Field experiments were conducted over a sorghum breeding trial planted in multiple years with more than 600 testcross hybrids. The results show that the proposed LSTM-based RNN model can achieve high accuracies for single year prediction. Further, with the proposed transfer learning strategies, a pre-trained model can be refined with limited training samples from the target domain and predict biomass with an accuracy comparable to that from a trained-from-scratch model for both multiple experiments within a given year and across multiple years.

Spectral Measurements of Muzzle Flash with a Temporally and Spatially Modulated LWIR-Imaging Fourier Transform Spectrometer

It is important to obtain information on an instantaneous target. A high-speed camera can capture a picture of an immediate scene, but spectral information about the object cannot be retrieved. Spectrographic analysis is a key tool for identifying chemicals. Detecting dangerous gas quickly can help ensure personal safety. In this paper, a temporally and spatially modulated long-wave infrared (LWIR)-imaging Fourier transform spectrometer was used to realize hyperspectral imaging. The spectral range was 700~1450 cm^-1 (7~14.5 μm). The frame rate of infrared imaging was 200 Hz. The muzzle-flash area of guns with calibers of 5.56 mm, 7.62 mm, and 14.5 mm were detected. LWIR images of muzzle flash were obtained. Spectral information on muzzle flash was obtained using instantaneous interferograms. The main peak of the spectrum of the muzzle flash appeared at 970 cm^-1 (10.31 μm). Two secondary peaks near 930 cm^-1 (10.75 μm) and 1030 cm^-1 (9.71 μm) were observed. Radiance and brightness temperature were also measured. The spatiotemporal modulation of the LWIR-imaging Fourier transform spectrometer provides a new method for rapid spectral detection. The high-speed identification of hazardous gas leakage can ensure personal safety.

Recognition of Abnormal-Laying Hens Based on Fast Continuous Wavelet and Deep Learning Using Hyperspectral Images

The egg production of laying hens is crucial to breeding enterprises in the laying hen breeding industry. However, there is currently no systematic or accurate method to identify low-egg-production-laying hens in commercial farms, and the majority of these hens are identified by breeders based on their experience. In order to address this issue, we propose a method that is widely applicable and highly precise. First, breeders themselves separate low-egg-production-laying hens and normal-laying hens. Then, under a halogen lamp, hyperspectral images of the two different types of hens are captured via hyperspectral imaging equipment. The vertex component analysis (VCA) algorithm is used to extract the cockscomb end member spectrum to obtain the cockscomb spectral feature curves of low-egg-production-laying hens and normal ones. Next, fast continuous wavelet transform (FCWT) is employed to analyze the data of the feature curves in order to obtain the two-dimensional spectral feature image dataset. Finally, referring to the two-dimensional spectral image dataset of the low-egg-production-laying hens and normal ones, we developed a deep learning model based on a convolutional neural network (CNN). When we tested the model's accuracy by using the prepared dataset, we found that it was 0.975 percent accurate. This outcome demonstrates our identification method, which combines hyperspectral imaging technology, an FCWT data analysis method, and a CNN deep learning model, and is highly effective and precise in laying-hen breeding plants. Furthermore, the attempt to use FCWT for the analysis and processing of hyperspectral data will have a significant impact on the research and application of hyperspectral technology in other fields due to its high efficiency and resolution characteristics for data signal analysis and processing.

Study on Hyperspectral Monitoring Model of Total Flavonoids and Total Phenols in Tartary Buckwheat Grains

Tartary buckwheat is a common functional food. Its grains are rich in flavonoids and phenols. The rapid measurement of flavonoids and phenols in buckwheat grains is of great significance in promoting the development of the buckwheat industry. This study, based on multiple scattering correction (MSC), standardized normal variate (SNV), reciprocal logarithm (Lg), first-order derivative (FD), second-order derivative (SD), and fractional-order derivative (FOD) preprocessing spectra, constructed hyperspectral monitoring models of total flavonoids content and total phenols content in tartary buckwheat grains. The results showed that SNV, Lg, FD, SD, and FOD preprocessing had different effects on the original spectral reflectance and that FOD can also reflect the change process from the original spectrum to the integer-order derivative spectrum. Compared with the original spectrum, MSC, SNV, Lg, FD, and SD transformation spectra can improve the correlation between spectral data and total flavonoids and total phenols in varying degrees, while the correlation between FOD spectra of different orders and total flavonoids and total phenols in grains was different. The monitoring models of total flavonoids and total phenols in grains based on MSC, SNV, Lg, FD, and SD transformation spectra achieved the best accuracy under SD and FD transformation, respectively. Therefore, this study further constructed monitoring models of total flavonoids and total phenols content in grains based on the FOD spectrum and achieved the best accuracy under 1.6 and 0.6 order derivative preprocessing, respectively. The R²_c, RMSE_c, R²_v, RMSE_v, and RPD were 0.8731, 0.1332, 0.8384, 0.1448, and 2.4475 for the total flavonoids model, and 0.8296, 0.2025, 0.6535, 0.1740, and 1.6713 for the total phenols model. The model can realize the rapid measurement of total flavonoids content and total phenols content in tartary buckwheat grains, respectively.

Study on the nitrogen content estimation model of cotton leaves based on "image-spectrum-fluorescence" data fusion

OBJECTIVE

Precise monitoring of cotton leaves' nitrogen content is important for increasing yield and reducing fertilizer application. Spectra and images are used to monitor crop nitrogen information. However, the information expressed using nitrogen monitoring based on a single data source is limited and cannot consider the expression of various phenotypic and physiological parameters simultaneously, which can affect the accuracy of inversion. Introducing a multi-source data-fusion mechanism can improve the accuracy and stability of cotton nitrogen content monitoring from the perspective of information complementarity.

METHODS

Five nitrogen treatments were applied to the test crop, Xinluzao No. 53 cotton, grown indoors. Cotton leaf hyperspectral, chlorophyll fluorescence, and digital image data were collected and screened. A multilevel data-fusion model combining multiple machine learning and stacking integration learning was built from three dimensions: feature-level fusion, decision-level fusion, and hybrid fusion.

RESULTS

The determination coefficients (R²) of the feature-level fusion, decision-level fusion, and hybrid-fusion models were 0.752, 0.771, and 0.848, and the root-mean-square errors (RMSE) were 3.806, 3.558, and 2.898, respectively. Compared with the nitrogen estimation models of the three single data sources, R² increased by 5.0%, 6.8%, and 14.6%, and the RMSE decreased by 3.2%, 9.5%, and 26.3%, respectively.

CONCLUSION

The multilevel fusion model can improve accuracy to varying degrees, and the accuracy and stability were highest with the hybrid-fusion model; these results provide theoretical and technical support for optimizing an accurate method of monitoring cotton leaf nitrogen content.

UAV Multisensory Data Fusion and Multi-Task Deep Learning for High-Throughput Maize Phenotyping

Recent advances in unmanned aerial vehicles (UAV), mini and mobile sensors, and GeoAI (a blend of geospatial and artificial intelligence (AI) research) are the main highlights among agricultural innovations to improve crop productivity and thus secure vulnerable food systems. This study investigated the versatility of UAV-borne multisensory data fusion within a framework of multi-task deep learning for high-throughput phenotyping in maize. UAVs equipped with a set of miniaturized sensors including hyperspectral, thermal, and LiDAR were collected in an experimental corn field in Urbana, IL, USA during the growing season. A full suite of eight phenotypes was in situ measured at the end of the season for ground truth data, specifically, dry stalk biomass, cob biomass, dry grain yield, harvest index, grain nitrogen utilization efficiency (Grain NutE), grain nitrogen content, total plant nitrogen content, and grain density. After being funneled through a series of radiometric calibrations and geo-corrections, the aerial data were analytically processed in three primary approaches. First, an extended version normalized difference spectral index (NDSI) served as a simple arithmetic combination of different data modalities to explore the correlation degree with maize phenotypes. The extended NDSI analysis revealed the NIR spectra (750-1000 nm) alone in a strong relation with all of eight maize traits. Second, a fusion of vegetation indices, structural indices, and thermal index selectively handcrafted from each data modality was fed to classical machine learning regressors, Support Vector Machine (SVM) and Random Forest (RF). The prediction performance varied from phenotype to phenotype, ranging from R² = 0.34 for grain density up to R² = 0.85 for both grain nitrogen content and total plant nitrogen content. Further, a fusion of hyperspectral and LiDAR data completely exceeded limitations of single data modality, especially addressing the vegetation saturation effect occurring in optical remote sensing. Third, a multi-task deep convolutional neural network (CNN) was customized to take a raw imagery data fusion of hyperspectral, thermal, and LiDAR for multi-predictions of maize traits at a time. The multi-task deep learning performed predictions comparably, if not better in some traits, with the mono-task deep learning and machine learning regressors. Data augmentation used for the deep learning models boosted the prediction accuracy, which helps to alleviate the intrinsic limitation of a small sample size and unbalanced sample classes in remote sensing research. Theoretical and practical implications to plant breeders and crop growers were also made explicit during discussions in the studies.

Non-destructive detection of kiwifruit soluble solid content based on hyperspectral and fluorescence spectral imaging

The soluble solid content (SSC) is one of the important parameters depicting the quality, maturity and taste of fruits. This study explored hyperspectral imaging (HSI) and fluorescence spectral imaging (FSI) techniques, as well as suitable chemometric techniques to predict the SSC in kiwifruit. 90 kiwifruit samples were divided into 70 calibration sets and 20 prediction sets. The hyperspectral images of samples in the spectral range of 387 nm~1034 nm and the fluorescence spectral images in the spectral range of 400 nm~1000 nm were collected, and their regions of interest were extracted. Six spectral pre-processing techniques were used to pre-process the two spectral data, and the best pre-processing method was selected after comparing it with the predicted results. Then, five primary and three secondary feature extraction algorithms were used to extract feature variables from the pre-processed spectral data. Subsequently, three regression prediction models, i.e., the extreme learning machines (ELM), the partial least squares regression (PLSR) and the particle swarm optimization - least square support vector machine (PSO-LSSVM), were established. The prediction results were analyzed and compared further. MASS-Boss-ELM, based on fluorescence spectral imaging technique, exhibited the best prediction performance for the kiwifruit SSC, with the R p 2 , R c 2 and RPD of 0.8894, 0.9429 and 2.88, respectively. MASS-Boss-PLSR based on the hyperspectral imaging technique showed a slightly lower prediction performance, with the R p 2 , R c 2 , and RPD of 0.8717, 0.8747, and 2.89, respectively. The outcome presents that the two spectral imaging techniques are suitable for the non-destructive prediction of fruit quality. Among them, the FSI technology illustrates better prediction, providing technical support for the non-destructive detection of intrinsic fruit quality.

Non-destructive Detection of Fatty Acid Content of Camellia Seed Based on Hyperspectral

As a unique traditional vegetable oil in China, camellia seed oil has very high edible value. Camellia seed kernel is mainly composed of fatty acids, which not only determines the oil yield of camellia seed, but also exert an important impact on the storage performance of camellia seed. In order to quickly and accurately determine the fatty acid content of camellia seed, this paper took camellia seed as the research object, used hyperspectral technology to determine the fatty acid content of camellia seed, and establishes a spectral model. 8 pretreatment methods, such as Savitzky-Golay smoothing, normalization, baseline correction, multivariate scattering correction, standard normal variable transformation, detrending algorithm, first derivative and second derivative, were adopted in this paper. The spectral prediction model of fatty acid content in camellia seed was established by combining 4 modeling methods: principal components regression (PCR), partial least square regression (PLSR), back propagation neural network (BP), radial basis function neural network (RBF). The optimal prediction model was selected by comparing the coefficient of determination (R²) and root mean square error (RMSE) of various models. The results showed that the spectral sensitive bands with high correlation coefficients (r) were 410-420 nm, 450-460 nm, 490-510 nm, 545-580 nm, 845-870 nm and 905-925 nm, respectively. The r obtained by MSC pretreatment of spectral data was the largest. The data obtained by 8 different pretreatment methods combined with RBF neural network model was the best, in which the average value of coefficient of determination (R_C²) in the calibration set was 0.8654, and the root mean square error of calibration (RMSEC) was 0.0777; the average value of coefficient of determination (R_P²) and root mean square error of prediction (RMSEP) in the prediction set model were 0.8437 and 0.0827, respectively. It could be seen that the best accuracy could be achieved by MSC pretreatment combined with RBF neural network modeling. This paper can provide reference for rapid nondestructive detection of fatty acid content in camellia seed by hyperspectral technology.

Initial Performance Analysis of the At-Altitude Radiance Ratio Method for Reflectance Conversion of Hyperspectral Remote Sensing Data

In remote sensing, the conversion of at-sensor radiance to surface reflectance for each pixel in a scene is an essential component of many analysis tasks. The empirical line method (ELM) is the most used technique among remote sensing practitioners due to its reliability and production of accurate reflectance measurements. However, the at-altitude radiance ratio (AARR), a more recently proposed methodology, is attractive as it allows reflectance conversion to be carried out in real time throughout data collection, does not require calibrated samples of pre-measured reflectance to be placed in scene, and can account for changes in illumination conditions. The benefits of AARR can substantially reduce the level of effort required for collection setup and subsequent data analysis, and provide a means for large-scale automation of remote sensing data collection, even in atypical flight conditions. In this study, an onboard, downwelling irradiance spectrometer integrated onto a small unmanned aircraft system (sUAS) is utilized to characterize the performance of AARR-generated reflectance from hyperspectral radiance data under a variety of challenging illumination conditions. The observed error introduced by AARR is often on par with ELM and acceptable depending on the application requirements and natural variation in the reflectance of the targets of interest. Additionally, a number of radiometric and atmospheric corrections are proposed that could increase the accuracy of the method in future trials, warranting further research.

Wheat yield estimation using remote sensing data based on machine learning approaches

Accurate predictions of wheat yields are essential to farmers'production plans and to the international trade in wheat. However, only poor approximations of the productivity of wheat crops in China can be obtained using traditional linear regression models based on vegetation indices and observations of the yield. In this study, Sentinel-2 (multispectral data) and ZY-1 02D (hyperspectral data) were used together with 15709 gridded yield data (with a resolution of 5 m × 5 m) to predict the winter wheat yield. These estimates were based on four mainstream data-driven approaches: Long Short-Term Memory (LSTM), Random Forest (RF), Gradient Boosting Decision Tree (GBDT), and Support Vector Regression (SVR). The method that gave the best estimate of the winter wheat yield was determined, and the accuracy of the estimates based on multispectral and hyperspectral data were compared. The results showed that the LSTM model, for which the RMSE of the estimates was 0.201 t/ha, performed better than the RF (RMSE = 0.260 t/ha), GBDT (RMSE = 0.306 t/ha), and SVR (RMSE = 0.489 t/ha) methods. The estimates based on the ZY-1 02D hyperspectral data were more accurate than those based on the 30-m Sentinel-2 data: RMSE = 0.237 t/ha for the ZY-1 02D data, which is about a 5% improvement on the RSME of 0.307 t/ha for the 30-m Sentinel-2 data. However, the 10-m Sentinel-2 data performed even better, giving an RMSE of 0.219 t/ha. In addition, it was found that the greenness vegetation index SR (simple ratio index) outperformed the traditional vegetation indices. The results highlight the potential of the shortwave infrared bands to replace the visible and near-infrared bands for predicting crop yields Our study demonstrates the advantages of the deep learning method LSTM over machine learning methods in terms of its ability to make accurate estimates of the winter wheat yield.

Application of convolutional neural network in fusion and classification of multi-source remote sensing data

Introduction

Through remote sensing images, we can understand and observe the terrain, and its application scope is relatively large, such as agriculture, military, etc.

Methods

In order to achieve more accurate and efficient multi-source remote sensing data fusion and classification, this study proposes DB-CNN algorithm, introduces SVM algorithm and ELM algorithm, and compares and verifies their performance through relevant experiments.

Results

From the results, we can find that for the dual branch CNN network structure, hyperspectral data and laser mines joint classification of data can achieve higher classification accuracy. On different data sets, the global classification accuracy of the joint classification method is 98.46%. DB-CNN model has the highest training accuracy and fastest speed in training and testing. In addition, the DB-CNN model has the lowest test error, about 0.026, 0.037 lower than the ELM model and 0.056 lower than the SVM model. The AUC value corresponding to the ROC curve of its model is about 0.922, higher than that of the other two models.

Discussion

It can be seen that the method used in this paper can significantly improve the effect of multi-source remote sensing data fusion and classification, and has certain practical value.

Prediction of Apple Hybrid Offspring Aroma Based on Hyperspectral

Used Random forest algorithm to construct a prediction model of aroma components based on the hybrid offspring of 'Honeycrisp' × 'Maodi', and different preprocessing methods were tried (Standardization (SS), First-order Derivative (D1) and Standard normal variate (SNV)). The aroma composition and content were determined by gas chromatography-mass spectrometry (GC-MS), and the main aroma components of apples were classified according to compound categories, including ester, aldehyde, ketone, alcohol. Taking the chemical groups as the research objects, the characteristic wavelengths were selected by grid search algorithm, and the characteristic wavelength-aroma chemical group model was established, and the same method was used to construct the model for single aroma components. The results show: SNV has the best noise removal effect among the five preprocessing methods. Under the SNV treatment, aroma chemical groups of apples showed a good correlation with the spectrum. The number of characteristic spectra of ester are 413, 493, 512, 551, 592, 600, 721, 727, 729, 733 nm, all in the visible light range. The determination coefficient (R2), the root mean square error (RMSE) and the ratio of the standard deviation values (RPD) of validation were 0.90, 4936.16 and 1.13. The characteristic spectrum of alcohols is 519, 562, 570, 571, 660, 676, 737, 738 nm, the range is close to that of ester. The R2 and RMSE of alcohol validation are 0.92 and 83.21, and RPD is 1.30. The number of characteristic spectra of aldehyde is 20, and the most important band is 1000 nm, which is outside the visible light range. The number of characteristic spectra of ketone is 15, and also has some distribution outside the visible light range. The R2 of aldehyde and ketone validation are 0.84 and 0.86. Except for cyclooctanol, the R2 of single aroma compound prediction model performed poorly. Based on the models, we tried to visualize alcohol, which can roughly represent their distribution on apple. Their distributions all show significant differences in the center and edge of apple, but the results are still rough due to the accuracy of models. In conclusion, the study can preliminarily prove that hyperspectral imaging technology (HSI) can perform non-destructive detection of aroma in apple hybrid offspring.

HyScreen: A Ground-Based Imaging System for High-Resolution Red and Far-Red Solar-Induced Chlorophyll Fluorescence

Solar-induced chlorophyll fluorescence (SIF) is used as a proxy of photosynthetic efficiency. However, interpreting top-of-canopy (TOC) SIF in relation to photosynthesis remains challenging due to the distortion introduced by the canopy's structural effects (i.e., fluorescence re-absorption, sunlit-shaded leaves, etc.) and sun-canopy-sensor geometry (i.e., direct radiation infilling). Therefore, ground-based, high-spatial-resolution data sets are needed to characterize the described effects and to be able to downscale TOC SIF to the leafs where the photosynthetic processes are taking place. We herein introduce HyScreen, a ground-based push-broom hyperspectral imaging system designed to measure red (F687) and far-red (F760) SIF and vegetation indices from TOC with single-leaf spatial resolution. This paper presents measurement protocols, the data processing chain and a case study of SIF retrieval. Raw data from two imaging sensors were processed to top-of-canopy radiance by dark-current correction, radiometric calibration, and empirical line correction. In the next step, the improved Fraunhofer line descrimination (iFLD) and spectral-fitting method (SFM) were used for SIF retrieval, and vegetation indices were calculated. With the developed protocol and data processing chain, we estimated a signal-to-noise ratio (SNR) between 50 and 200 from reference panels with reflectance from 5% to 95% and noise equivalent radiance (NER) of 0.04 (5%) to 0.18 (95%) mW m-2 sr-1 nm-1. The results from the case study showed that non-vegetation targets had SIF values close to 0 mW m-2 sr-1 nm-1, whereas vegetation targets had a mean F687 of 1.13 and F760 of 1.96 mW m-2 sr-1 nm-1 from the SFM method. HyScreen showed good performance for SIF retrievals at both F687 and F760; nevertheless, we recommend further adaptations to correct for the effects of noise, varying illumination and sensor optics. In conclusion, due to its high spatial resolution, Hyscreen is a promising tool for investigating the relationship between leafs and TOC SIF as well as their relationship with plants' photosynthetic capacity.

Continuous Monitoring of Suspended Particulate Matter in Tropical Inland Waters by High-Frequency, Above-Water Radiometry

Water and sediment discharges can change rapidly, and low-frequency measurement devices might not be sufficient to elucidate existing dynamics. As such, above-water radiometry might enhance monitoring of suspended particulate matter (SPM) dynamics in inland waters. However, it has been barely applied for continuous monitoring, especially under partially cloudy sky conditions. In this study, an in situ, high-frequency (30 s timestep), above-water radiometric dataset, collected over 18 days in a tropical reservoir, is analyzed for the purpose of continuous monitoring of SPM concentration. Different modalities to retrieve reflectance spectra, as well as SPM inversion algorithms, were applied and evaluated. We propose a sequence of processing that achieved an average unsigned percent difference (UPD) of 10.4% during cloudy conditions and 4.6% during clear-sky conditions for Rrs (665 nm), compared to the respective UPD values of 88.23% and 13.17% when using a simple calculation approach. SPM retrieval methods were also evaluated and, depending on the methods used, we show that the coefficient of variation (CV) of the SPM concentration varied from 69.5% down to 2.7% when using a semi-analytical approach. As such, the proposed processing approach is effective at reducing unwanted variability in the resulting SPM concentration assessed from above-water radiometry, and our work paves the way towards the use of this noninvasive technique for high-frequency monitoring of SPM concentrations in streams and lakes.

Reflectance Measurement Method Based on Sensor Fusion of Frame-Based Hyperspectral Imager and Time-of-Flight Depth Camera

Hyperspectral imaging and distance data have previously been used in aerial, forestry, agricultural, and medical imaging applications. Extracting meaningful information from a combination of different imaging modalities is difficult, as the image sensor fusion requires knowing the optical properties of the sensors, selecting the right optics and finding the sensors' mutual reference frame through calibration. In this research we demonstrate a method for fusing data from Fabry-Perot interferometer hyperspectral camera and a Kinect V2 time-of-flight depth sensing camera. We created an experimental application to demonstrate utilizing the depth augmented hyperspectral data to measure emission angle dependent reflectance from a multi-view inferred point cloud. We determined the intrinsic and extrinsic camera parameters through calibration, used global and local registration algorithms to combine point clouds from different viewpoints, created a dense point cloud and determined the angle dependent reflectances from it. The method could successfully combine the 3D point cloud data and hyperspectral data from different viewpoints of a reference colorchecker board. The point cloud registrations gained 0.29-0.36 fitness for inlier point correspondences and RMSE was approx. 2, which refers a quite reliable registration result. The RMSE of the measured reflectances between the front view and side views of the targets varied between 0.01 and 0.05 on average and the spectral angle between 1.5 and 3.2 degrees. The results suggest that changing emission angle has very small effect on the surface reflectance intensity and spectrum shapes, which was expected with the used colorchecker.

Predicting F v /F m and evaluating cotton drought tolerance using hyperspectral and 1D-CNN

The chlorophyll fluorescence parameter F_v/F_m is significant in abiotic plant stress. Current acquisition methods must deal with the dark adaptation of plants, which cannot achieve rapid, real-time, and high-throughput measurements. However, increased inputs on different genotypes based on hyperspectral model recognition verified its capabilities of handling large and variable samples. F_v/F_m is a drought tolerance index reflecting the best drought tolerant cotton genotype. Therefore, F_v/F_m hyperspectral prediction of different cotton varieties, and drought tolerance evaluation, are worth exploring. In this study, 80 cotton varieties were studied. The hyperspectral cotton data were obtained during the flowering, boll setting, and boll opening stages under normal and drought stress conditions. Next, One-dimensional convolutional neural networks (1D-CNN), Categorical Boosting (CatBoost), Light Gradient Boosting Machines (LightBGM), eXtreme Gradient Boosting (XGBoost), Decision Trees (DT), Random Forests (RF), Gradient elevation decision trees (GBDT), Adaptive Boosting (AdaBoost), Extra Trees (ET), and K-Nearest Neighbors (KNN) were modeled with F _v /F _m. The Savitzky-Golay + 1D-CNN model had the best robustness and accuracy (RMSE = 0.016, MAE = 0.009, MAPE = 0.011). In addition, the F _v /F _m prediction drought tolerance coefficient and the manually measured drought tolerance coefficient were similar. Therefore, cotton varieties with different drought tolerance degrees can be monitored using hyperspectral full band technology to establish a 1D-CNN model. This technique is non-destructive, fast and accurate in assessing the drought status of cotton, which promotes smart-scale agriculture.

Linear Support Vector Machine Classification of Plant Stress From Soybean Aphid (Hemiptera: Aphididae) Using Hyperspectral Reflectance

Spectral remote sensing has the potential to improve scouting and management of soybean aphid (Aphis glycines Matsumura), which can cause yield losses of over 40% in the North Central Region of the United States. We used linear support vector machines (SVMs) to determine 1) whether hyperspectral samples could be classified into treat/no-treat classes based on the economic threshold (250 aphids per plant) and 2) how many wavelengths or features are needed to generate an accurate model without overfitting the data. A range of aphid infestation levels on soybean was created using caged field plots in 2013, 2014, 2017, and 2018 in Minnesota and in 2017 and 2018 in Iowa. Hyperspectral measurements of soybean canopies in each plot were recorded with a spectroradiometer. SVM training and testing were performed using 15 combinations of normalized canopy reflectance at wavelengths of 720, 750, 780, and 1,010 nm. Pairwise Bonferroni-adjusted t-tests of Cohen's kappa values showed four wavelength combinations were optimal, namely model 1 (780 nm), model 2 (780 and 1,010 nm), model 3 (780, 1,010, and 720 nm), and model 4 (780, 1,010, 720, and 750 nm). Model 2 showed the best overall performance, with an accuracy of 89.4%, a sensitivity of 81.2%, and a specificity of 91.6%. The findings from this experiment provide the first documentation of successful classification of remotely sensed spectral data of soybean aphid-induced stress into threshold-based classes.

Hyperspectral imaging for tumor segmentation on pigmented skin lesions

SIGNIFICANCE

Malignant skin tumors, which include melanoma and nonmelanoma skin cancers, are the most prevalent type of malignant tumor. Gross pathology of pigmented skin lesions (PSL) remains manual, time-consuming, and heavily dependent on the expertise of the medical personnel. Hyperspectral imaging (HSI) can assist in the detection of tumors and evaluate the status of tumor margins by their spectral signatures.

AIM

Tumor segmentation of medical HSI data is a research field. The goal of this study is to propose a framework for HSI-based tumor segmentation of PSL.

APPROACH

An HSI dataset of 28 PSL was prepared. Two frameworks for data preprocessing and tumor segmentation were proposed. Models based on machine learning and deep learning were used at the core of each framework.

RESULTS

Cross-validation performance showed that pixel-wise processing achieves higher segmentation performance, in terms of the Jaccard coefficient. Simultaneous use of spatio-spectral features produced more comprehensive tumor masks. A three-dimensional Xception-based network achieved performance similar to state-of-the-art networks while allowing for more detailed detection of the tumor border.

CONCLUSIONS

Good performance was achieved for melanocytic lesions, but margins were difficult to detect in some cases of basal cell carcinoma. The frameworks proposed in this study could be further improved for robustness against different pathologies and detailed delineation of tissue margins to facilitate computer-assisted diagnosis during gross pathology.

Comparison of preprocessing techniques to reduce nontissue-related variations in hyperspectral reflectance imaging

SIGNIFICANCE

Hyperspectral reflectance imaging can be used in medicine to identify tissue types, such as tumor tissue. Tissue classification algorithms are developed based on, e.g., machine learning or principle component analysis. For the development of these algorithms, data are generally preprocessed to remove variability in data not related to the tissue itself since this will improve the performance of the classification algorithm. In hyperspectral imaging, the measured spectra are also influenced by reflections from the surface (glare) and height variations within and between tissue samples.

AIM

To compare the ability of different preprocessing algorithms to decrease variations in spectra induced by glare and height differences while maintaining contrast based on differences in optical properties between tissue types.

APPROACH

We compare eight preprocessing algorithms commonly used in medical hyperspectral imaging: standard normal variate, multiplicative scatter correction, min-max normalization, mean centering, area under the curve normalization, single wavelength normalization, first derivative, and second derivative. We investigate conservation of contrast stemming from differences in: blood volume fraction, presence of different absorbers, scatter amplitude, and scatter slope-while correcting for glare and height variations. We use a similarity metric, the overlap coefficient, to quantify contrast between spectra. We also investigate the algorithms for clinical datasets from the colon and breast.

CONCLUSIONS

Preprocessing reduces the overlap due to glare and distance variations. In general, the algorithms standard normal variate, min-max, area under the curve, and single wavelength normalization are the most suitable to preprocess data used to develop a classification algorithm for tissue classification. The type of contrast between tissue types determines which of these four algorithms is most suitable.