Visualization of vibrational spectroscopy for agro-food samples using t-Distributed Stochastic Neighbor Embedding

Quantitative determination of acid value in palm oil during thermal oxidation using Raman spectroscopy combined with deep learning models

Accurate monitoring of acid value (AV) is critical for edible oil quality control, yet traditional chemometric methods often face limitations in handling complex spectral data. This study combines Raman spectroscopy with deep learning, including Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM), and Transformer, to explore their potential in improving the accuracy and efficiency of AV quantification during the thermal oxidation of palm oil. The results showed that all three deep learning models outperformed traditional chemometric methods in predictive accuracy. The CNN-LSTM model achieved the best performance, with a predicted coefficient of determination (Rp2) of 0.9978, a mean square error of prediction (RMSEP) of 0.0015, and a residual predictive deviation (RPD) of 21.21. This method demonstrates the effectiveness of Raman spectroscopy-driven deep learning for precise AV monitoring and holds promise for further validation with more diverse indicator datasets, providing a novel technical reference for edible oil quality control.

Discrimination of unsound soybeans using hyperspectral imaging: A deep learning method based on dual-channel feature fusion strategy and attention mechanism

The application of high-level data fusion in the detection of agricultural products still presents a significant challenge. In this study, dual-channel feature fusion model (DCFFM) with attention mechanism was proposed to optimize the utilization of both one-dimensional spectral data and two-dimensional image data in the hyperspectral images for achieving high-level data fusion. A comparative analysis of support vector machine (SVM), convolutional neural network (CNN) with DCFFM, demonstrated that DCFFM exhibited superior results, achieving the accuracy, precision, recall, specificity, and F1-score of 95.13 %, 95.49 %, 94.83 %, 98.97 %, 95.12 % in the visible and near-infrared (Vis-NIR), and 94.00 %, 94.43 %, 94.16 %, 98.67 %, 94.27 % in the short-wave infrared (SWIR). This also indicated that Vis-NIR was more suitable for identifying unsound soybeans than SWIR. Furthermore, visualization was employed to demonstrate classification outcomes, thereby illustrating the generalization capacity of DCFFM through model inversion. In summary, this study is to explore a modeling framework that is capable of the comprehensive acquisition of spectra and images in the hyperspectral images, allowing for high-level data fusion, thereby achieving enhanced levels of accuracy.

A novel technology for rapid identification of hemp fibers by terahertz spectroscopy

A novel method for the rapid identification of hemp fibers is proposed in this paper, utilizing terahertz time-domain spectroscopy (THz-TDS) combined with the LargeVis (LV) dimensionality reduction technique. This approach takes advantage of the strengths of THz-TDS while enhancing classification accuracy through LV. To verify the efficacy of this method, terahertz absorption spectral data from three types of hemp fibers were processed. The THz absorption spectra were initially preprocessed using Hanning filtering. Following this, the filtered data underwent dimensionality reduction through three distinct methods: linear Principal Component Analysis (PCA), nonlinear t-Distributed Stochastic Neighbor Embedding (t-SNE), and the LV method. This sequence of steps resulted in a two-dimensional feature data matrix derived from the THz source spectral data. The resultant feature data matrices were then input into both K-Nearest Neighbors (KNN) and Decision Tree (DT) classifiers for analysis. The classification accuracies of six models were evaluated, revealing that the LV-KNN model achieved a 86.67% accuracy rate for the three hemp fiber types. Impressively, the LV-DT model achieved a perfect 100.00% accuracy rate for the same fibers. The LV-DT model, when integrated with THz spectroscopy technology, offers a quick and precise method for identifying various types of hemp fibers. This development introduces an innovative optical measurement scheme for the characterization of textile materials.

Progress of machine learning-based biosensors for the monitoring of food safety: A review

Rapid urbanization and growing food demand caused people to be concerned about food safety. Biosensors have gained considerable attention for assessing food safety due to selectivity, and sensitivity but poor stability inherently limits their application. The emergence of machine learning (ML) has enhanced the efficiency of different sensors for food safety assessment. The ML combined with various noninvasive biosensors has been implemented efficiently to monitor food safety by considering the stability of bio-recognition molecules. This review comprehensively summarizes the application of ML-powered biosensors to investigate food safety. Initially, different detector-based biosensors using biological molecules with their advantages and disadvantages and biosensor-related various ML algorithms for food safety monitoring have been discussed. Next, the application of ML-powered biosensors to detect antibiotics, foodborne microorganisms, mycotoxins, pesticides, heavy metals, anions, and persistent organic pollutants has been highlighted for the last five years. The challenges and prospects have also been deliberated. This review provides a new prospect in developing various biosensors for multi-food contaminants powered by suitable ML algorithms to monitor in-situ food safety.

Injectable chitosan-polyvinylpyrrolidone composite thermosensitive hydrogels with sustained submucosal lifting for endoscopic submucosal dissection

To develop a submucosal injection material with sustained submucosal lifting for endoscopic submucosal dissection (ESD), this study designed and prepared a novel composite thermosensitive hydrogel system with high pH chitosan-polyvinylpyrrolidone-β-glycerophosphate (HpHCS-PVP-GP). HpHCS improved the injectability of the hydrogels and retained the rapid gelation ability at low concentrations. The modification of PVP significantly improved the stability of low-temperature hydrogel precursor solutions and the integrity of hydrogels formed at 37 °C through hydrogen bonds between PVP and HpHCS. A mathematical model was established using response surface methodology (RSM) to evaluate the synergistic effect of HpHCS, GP, and PVP concentrations on gelation time. This RSM model and submucosal lifting evaluation using in vitro pig esophageal models were used to determine the optimal formula of HpHCS-PVP-GP hydrogels. Although the higher PVP concentration (5 % (w/v)) prolonged gelation time, it improved hydrogel mechanical strength, resulting in better submucosal lifting performance. The experiments of Bama mini pigs showed that the heights of the cushions elevated by the HpHCS-5%PVP-GP hydrogel remained about 80 % 1 h after injection. Repeated injections were avoided, and the hydrogel had no cytotoxicity after electric cutting. Therefore, the HpHCS-PVP-GP thermosensitive hydrogel might be a promising submucosal injection material for ESD.

Nondestructive detection of SSC in multiple pear (Pyrus pyrifolia Nakai) cultivars using Vis-NIR spectroscopy coupled with the Grad-CAM method

Vis-NIR spectroscopy coupled with chemometric models is frequently used for pear soluble solid content (SSC) prediction. However, the model robustness is challenged by the variations in pear cultivars. This study explored the feasibility of developing universal models for predicting SSC of multiple pear varieties to improve the model's generalizability. The mature fruits of 6 pear cultivars with green skin (Pyrus pyrifolia Nakai cv. 'Cuiyu', 'Sucui No.1' and 'Cuiguan') and brown skin (Pyrus pyrifolia Nakai cv. 'Hosui','Syusui' and 'Wakahikari') were used to establish single-cultivar models and multi-cultivar universal models using convolutional neural network (CNN), partial least square (PLS), and support vector regression (SVR) approaches. Multi-cultivar universal models were built using full spectra and important variables extracted by gradient-weighted class activation mapping (Grad-CAM), respectively. The universal models based on important variables obtained satisfactory performances with RMSEPs of 0.76, 0.59, 0.80, 1.64, 0.98, and 1.03°Brix on 6 cultivars, respectively.

Determination of seawater COD spectra using double-loop contraction and sorted frog optimization

This study develops a novel double-loop contraction and C value sorting selection-based shrinkage frog-leaping algorithm (double-contractive cognitive random field [DC-CRF]) to mitigate the interference of complex salts and ions in seawater on the ultraviolet-visible (UV-Vis) absorbance spectra for chemical oxygen demand (COD) quantification. The key innovations of DC-CRF are introducing variable importance evaluation via C value to guide wavelength selection and accelerate convergence; a double-loop structure integrating random frog (RF) leaping and contraction attenuation to dynamically balance convergence speed and efficiency. Utilizing seawater samples from Jiaozhou Bay, DC-CRF-partial least squares regression (PLSR) reduced the input variables by 97.5% after 1,600 iterations relative to full-spectrum PLSR, RF-PLSR, and CRF-PLSR. It achieved a test R2 of 0.943 and root mean square error of 1.603, markedly improving prediction accuracy and efficiency. This work demonstrates the efficacy of DC-CRF-PLSR in enhancing UV-Vis spectroscopy for rapid COD analysis in intricate seawater matrices, providing an efficient solution for optimizing seawater spectra.

Qualitative and quantitative studies of phthalates in extra virgin olive oil (EVOO) by surface-enhanced Raman spectroscopy (SERS) combined with long short term memory (LSTM) neural network

Phthalates are commonly used plasticizers in the plastics industry, and have received extensive attention due to their reproductive toxicity. Since phthalates are lipophilic solutions, phthalates can easily migrate from packaging to edible oils. This study synthesized stable and sensitive Gold Nanostars as SERS substrates to conduct qualitative and quantitative analysis of two common phthalates, dibutyl phthalate and di(2-ethylhexyl) phthalate. Two ethanol standard solutions and actual oil solutions of phthalates at different concentrations (10, 5, 1, 0.5, 0.1, 0.02 mg/kg) were prepared. After dimension reduction, LSTM achieved the accuracy of 98% for pure EVOO and EVOO adulterated with different types of phthalates. In terms of quantification, LSTM demonstrates great predictive performance with Rp2 greater than 0.97 and the ratio of performance to deviation greater than 5. These results have certain guiding significance for the analysis of plasticizers in edible oil.

Rapid and Accurate Authentication of Porcini Mushroom Species Using Fourier Transform Near-Infrared Spectra Combined with Machine Learning and Chemometrics

Porcini mushrooms have high nutritional value and great potential, but different species are easily confused, so it is essential to identify them rapidly and precisely. The diversity of nutrients in stipe and cap will lead to differences in spectral information. In this research, Fourier transform near-infrared (FT-NIR) spectral information about imparity species of porcini mushroom stipe and cap was collected and combined into four data matrices. FT-NIR spectra of four data sets were combined with chemometric methods and machine learning for accurate evaluation and identification of different porcini mushroom species. From the results: (1) improved visualization level of t-distributed stochastic neighbor embedding (t-SNE) results after the second derivative preprocessing compared with raw spectra; (2) after using multiple pretreatment combinations to process the four data matrices, the model accuracies based on support vector machine and partial least-square discriminant analysis (PLS-DA) under the best preprocessing method were 98.73-99.04% and 98.73-99.68%, respectively; (3) by comparing the modeling results of FT-NIR spectra with different data matrices, it was found that the PLS-DA model based on low-level data fusion has the highest accuracy (99.68%), but residual neural network (ResNet) model based on the stipe, cap, and average spectral data matrix worked better (100% accuracy). The above results suggest that distinct models should be selected for dissimilar spectral data matrices of porcini mushrooms. Additionally, FT-NIR spectra have the advantages of being nondevastate and fast; this method is expected to be a promising analytical tool in food safety control.

FAPD: An Astringency Threshold and Astringency Type Prediction Database for Flavonoid Compounds Based on Machine Learning

Astringency is a puckering or velvety sensation mainly derived from flavonoid compounds in food. The traditional experimental approach for astringent compound discovery was labor-intensive and cost-consuming, while machine learning (ML) can greatly accelerate this procedure. Herein, we propose the Flavonoid Astringency Prediction Database (FAPD) based on ML. First, the Molecular Fingerprint Similarities (MFSs) and thresholds of flavonoid compounds were hierarchically clustering analyzed. For the astringency threshold prediction, four regressions models (i.e., Gaussian Process Regression (GPR), Support Vector Regression (SVR), Random Forest (RF), and Gradient Boosted Decision Tree (GBDT)) were established, and the best model was RF which was interpreted by the SHapley Additive exPlanations (SHAP) approach. For the astringency type prediction, six classification models (i.e., RF, GBDT, Gaussian Naive Bayes (GNB), Support Vector Machine (SVM), k-Nearest Neighbor (kNN), and Stochastic Gradient Descent (SGD)) were established, and the best model was SGD. Furthermore, over 1200 natural flavonoid compounds were discovered and built into the customized FAPD. In FAPD, the astringency thresholds were achieved by RF; the astringency types were distinguished by SGD, and the real and predicted astringency types were verified by t-Distributed Stochastic Neighbor Embedding (t-SNE). Therefore, ML models can be used to predict the astringency threshold and astringency type of flavonoid compounds, which provides a new paradigm to research the molecular structure-flavor property relationship of food components.

Enhancing the classification metrics of spectroscopy spectrums using neural network based low dimensional space

Spectroscopy is a methodology for gaining knowledge of particles, especially biomolecules, by quantifying the interactions between matter and light. By examining the level of light absorbed, reflected or released by a specimen, its constituents, properties, and volume can be determined. Spectra obtained through spectroscopy procedures are quick, harmless and contactless; hence nowadays preferred in chemometrics. Due to the high dimensional nature of the spectra, it is challenging to build a robust classifier with good performance metrics. Many linear and nonlinear dimensionality reduction-based classification models have been previously implemented to overcome this issue. However, they lack in capturing the subtle details of the spectra into the low dimension space or cannot efficiently handle the nonlinearity present in the spectral data. We propose a graph-based neural network embedding approach to extract appropriate features into latent space and circumvent the spectrums' nonlinearity problem. Our approach performs dimensionality reduction into two phases: constructing a nearest neighbor graph and producing almost linear embedding using a fully connected neural network. Further, the low dimensional embedding is subjected to classification using the Random Forest algorithm. In this paper, we have implemented and compared our technique with four nonlinear dimensionality techniques widely used for spectral data analysis. In this study, we have considered five different spectral datasets belonging to specific applications. The various classification performance metrics of all the techniques are evaluated. The proposed approach is able to perform competitively well on six different low-dimensional spaces for each dataset with an accuracy score above 95% and Matthew's correlation coefficient value close to 1. The trustworthiness score of almost 1 show that the presented dimensionality reduction approach preserves the closest neighbor structure of high dimensional spectral inputs into latent space.

Rapid identification of the geographic origin of Taiping Houkui green tea using near-infrared spectroscopy combined with a variable selection method

BACKGROUND

Most studies focus on the geographically larger production areas in tea traceability. However, famous high-quality tea is often produced in a narrow range of origins, which makes traceability a challenge. In this study, Taiping Houkui (TPHK) green tea of narrow geographical origin was rapidly identified using Fourier-transform near-infrared (FT-NIR) spectroscopy.

RESULTS

First, spectral information of 114 TPHK samples from four production areas was acquired. Second, the synthetic minority over-sampling technique (SMOTE) was used to balance the sample data set, and three different spectral pre-processing methods were compared. Third, three feature variable selection algorithms were used to obtain the pre-processed spectral features. Finally, extreme learning machine (ELM) models based on the variables obtained from the selected features were established to trace the TPHK origin. The optimized ELM model achieves 95.35% classification accuracy in the test set.

CONCLUSION

The present study demonstrates that the optimized variable selection method in combination with NIR spectroscopy represents a suitable strategy for tea traceability in narrow regions. © 2022 Society of Chemical Industry.

Estimation of chemical oxygen demand in different water systems by near-infrared spectroscopy

To monitor environmental water pollution effectively and meet human water needs, it is crucial to develop a fast, simple, and accurate method for monitoring chemical oxygen demand (COD) in various water systems. In this study, COD prediction models for different water systems were developed by combining near-infrared (NIR) spectroscopy with partial least squares regression (PLSR). Samples of wastewater, surface water, and seawater were collected from Guangzhou, Guangdong Province, China. Three pretreatment methods were used to preprocess the spectra in order to improve the accuracy and minimalism of the model. We investigate the performance of two variable selection algorithms, namely, binary gray wolf optimization (BGWO) and competitive adaptive reweighting sampling (CARS). The results show that both BGWO and CARS improved the performance of the model in terms of higher accuracy and less wavelength input; both of the combined model performances were better than that of PLSR alone, and CARS-PLSR achieved the best results. Using CARS-PLSR, surface water, wastewater, and seawater model inputs were reduced by 96 %, 96 %, and 82 % as compared to the PLSR results, respectively, and the testing sets R² reached 0.860, 0.815, and 0.692, respectively. The spectral variable selection algorithm could identify the important spectral variables between COD content and NIR spectra in three water systems, thereby improving the accuracy and simplicity of the PLSR model for COD prediction. Our results have important practical value for predicting COD content in different water systems by NIR spectroscopy.

Systematic investigation of keywords selection and processing strategy on search engine forecasting: a case of tourist volume in Beijing

The timeliness, precision, and low cost of search data have great potential for projecting tourist volume. Obtaining valuable information for decision-making, particularly for predicting, is hampered by the vast amount of search data. A systematic investigation of keyword selection and processing has been conducted. Using Beijing tourist volume as an example, 11 different feature extraction algorithms were selected and combined with long short-term memory (LSTM), random forest (RF) and fuzzy time series (FTS) for forecasting tourist volume. A total of 1612 keywords were retrieved from Baidu Index demand mapping using the direct word extraction method, range word extraction method and empirical selection method. The remaining 813 keywords were subjected to feature extraction. Based on the forecasting results of medium and short-term (1-day, 7-days and 10-days), the forecasting results of Kernel principal component analysis (KPCA) and locally linear embedding (LLE) are relatively stable when the dimensionality is reduced to 5 dimensions. The forecasting results of t-stochastic neighbor embedding (t-SNE), isometric mapping (IsoMap) and locally linear embedding (LLE), locality preserving projections (LPP), independent component correlation (ICA) are relatively stable when the dimensionality is reduced to 10 dimensions. Accurately forecasting many factors (transportation, attraction, food, lodging, travel, tips, tickets, and weather) provides a solid foundation for tourism demand optimization and scientific management and a resource for tourists' holistic vacation planning.

Detection of lipid efflux from foam cell models using a label-free infrared method

Synchrotron-based microFTIR spectroscopy was used to study the process of lipid efflux in a foam cell model. The anti-atherosclerotic drug, atorvastatin, removed low-density lipoprotein from the foam cells in a dose, and time dependent manner.