New data preprocessing trends based on ensemble of multiple preprocessing techniques

Comprehensive assessment of the role of spectral data pre-processing in spectroscopy-based liquid biopsy

Spectroscopic data often contain artifacts or noise related to the sample characteristics, instrumental variations, or experimental design flaws. Therefore, classifying the raw data is not recommended and might lead to biased results. Nevertheless, most issues may be addressed through appropriate data pre-processing. Effective pre-processing is particularly crucial in critical applications like liquid biopsy for disease detection, where even minor performance improvements may impact patient outcomes. Unfortunately, there is no consensus regarding optimal pre-processing, complicating cross-study comparisons. This study presents a comprehensive evaluation of various pre-processing methods and their combinations to assess their influence on classification results. The goal was to identify whether some pre-processing methods are associated with higher classification outcomes and find an optimal strategy for the given data. Data from Raman optical activity and infrared and Raman spectroscopy were processed, applying tens of thousands of possible pre-processing pipelines. The resulting data were classified using three algorithms to distinguish between subjects with liver cirrhosis and those who had developed hepatocellular carcinoma. Results highlighted that some specific pre-processing methods often ranked among the best classification results, such as the Rolling Ball for correcting the baseline of Raman spectra or the Doubly Reweighted Penalized Least Squares and Mixture model in the case of Raman optical activity. On the other hand, the selection of filtering and/or normalization approach usually did not have a significant impact. Nonetheless, the pre-processing of top-scoring pipelines also depended on the classifier utilized. The best pipelines yielded an AUROC of 0.775-0.823, varying with the evaluated spectroscopic data and classifier.

Functional data analysis, a comprehensive framework for processing non-quadrilinear and low-selective data provided by four-way liquid chromatography analysis

The chemometric treatment of higher-order chromatographic (LC) data often crashes due to two main effects: the chromatographic band shifts/warpings across samples and quasi-full overlaps between component signals, affecting their analytical selectivities. From a chemometric point of view, these phenomena have independent effects, although they can jointly contribute to the failure of the algorithms: 1) data multilinearity breaking, leading to the poor performance of multilinear decomposition algorithms, and 2) linear dependence between the analytes signals, causing the failure of folded models. Under this scenario, making chemometric processing feasible involves defining specific experimental conditions that minimize these effects or increasing the number of instrumental ways to deal with selectivity lost. This work presents the Functional Aligned of Pure Vectors (FAPV) algorithm for restoring four-way chromatographic data multilinearity and bearing the spectral overlap trouble. Simulated and experimental four-way data were used to test the FAPV analytical efficiency, covering a wide range of chromatographic artifacts. Based on a multi-injection procedure, the experimental case implied the chromatographic determination of two analytes with uncalibrated interferents in aqueous samples. Both data systems were subjected to FAPV and then processed by PARAFAC. Therefore, a comprehensive comparison was made with the most widely used chemometric models for non-multilinear chromatographic data (MCR-ALS and PARAFAC2). Moreover, the performance of the FAPV approach was compared with commonly used alignment procedures, e.g., correlation-optimized warping. The results (c.a. REPs of 10 % in both analytes from the experimental case) show the efficiency of the FAPV algorithm in solving the troubles observed in chromatographic/spectral data.

Innovative novel regularized memory graph attention capsule network for financial fraud detection

Financial fraud detection (FFD) is crucial for ensuring the safety and efficiency of financial transactions. This article presents the Regularised Memory Graph Attention Capsule Network (RMGACNet), an original architecture aiming at improving fraud detection using Bidirectional Long Short-Term Memory (BiLSTM) networks combined with advanced feature extraction and classification algorithms. The model is tested on two reliable datasets: the European Cardholder (ECH) transactions dataset, which contains 284,807 transactions and 492 fraud instances, and the IEEE-CIS dataset, which has more than 1 million transactions. Our approach enhances comparison to existing methods of feature selection and classification accuracy. On the ECH dataset, RMGACNet achieves an accuracy of 0.9772, a precision of 0.9768, and an F1 score of 0.9770 measures; on the IEEE-CIS dataset, it achieves an accuracy of 0.9882, a precision of 0.9876 and an F1 score of 0.9879. The findings indicate that RMGACNet routinely surpasses existing models' efficiency and accuracy while ensuring strong execution time performance, especially when handling large-scale datasets. The suggested model demonstrates scalability and stability, making it suitable for real-time financial systems.

A story of falsification and authentication: Authenticity control of phytopharmaceuticals and herbal remedies

Natural remedies and phytopharmaceuticals are gaining popularity among both consumers and patients who seek a healthier and more environmentally friendly lifestyle. Healthcare professionals are also increasingly incorporating phytopharmaceuticals into therapeutic practices, recognizing their potential benefits for overall health and well-being. This trend is further driven by a growing distrust of synthetic compounds and dissatisfaction with conventional medical treatments, prompting many patients to explore natural alternatives. As a result, the demand for phytopharmaceuticals, essential oils, and nutraceuticals continues to rise steadily. However, falsified natural remedies can pose serious risks to human health, making quality assurance a critical priority. Sophisticated and reliable analytical strategies are essential to ensure product authenticity and to protect consumers and patients from potentially harmful adulteration. This review provides an overview of documented cases of adulteration and the corresponding countermeasures developed to detect such incidents. Commonly used analytical techniques for authenticity verification of phytopharmaceuticals, essential oils, and other natural remedies are presented. The review discusses the application of spectroscopic methods, mass spectrometry, chromatographic separation techniques, and DNA-based profiling, with reference to current scientific literature. In addition, selected chemometric tools are introduced to illustrate how meaningful information can be extracted from complex analytical data. The potential of multidimensional data analysis, combining complementary insights from various analytical platforms, is also explored. Finally, key considerations for generating and analysing reliable data are highlighted.

Identifying the Geographical Origin of Wolfberry Using Near-Infrared Spectroscopy and Stacking-Orthogonal Linear Discriminant Analysis

The geographical origin identification of wolfberry is key to ensuring its medicinal and edible quality. To accurately identify the geographical origin, the Stacking-Orthogonal Linear Discriminant Analysis (OLDA) algorithm was proposed by combining OLDA with the Stacking ensemble learning framework. In this study, Savitzky-Golay (SG) + Multiplicative Scatter Correction (MSC) served as the optimal preprocessing method. Four classifiers-K-Nearest Neighbors (KNN), Decision Tree, Support Vector Machine (SVM), and Naive Bayes-were used to explore 12 stacked combinations on 400 samples from five regions in Gansu: Zhangye, Yumen, Wuwei, Baiyin, and Dunhuang. When Principal Component Analysis (PCA), PCA + Linear Discriminant Analysis (LDA), and OLDA were used for feature extraction, Stacking-OLDA achieved the highest average identification accuracy of 99%. The overall accuracy of stacked combinations was generally higher than that of single-classifier models. This study also assessed the role of different classifiers in different combinations, finding that Stacking-OLDA combined with KNN as the meta-classifier achieved the highest accuracy. Experimental results demonstrate that Stacking-OLDA has excellent classification performance, providing an effective approach for the accurate classification of wolfberry origins and offering an innovative solution for quality control in the food industry.

Digital Restoration of Separations Data

In experimental separations, the acquired signal can have four common but unwanted elements: (i) high-frequency noise, (ii) drift, (iii) occasional periodic and pink noise, and (iv) a mixture of components traveling with close velocities. Additionally, separation scientists observe partially resolved peaks in chromatographic or capillary electrophoresis detector output. A protocol for digitally recovering the "true data" is provided with clear-cut details and open-access codes for formulating denoising, baseline correction, and peak resolution problems as linear or nonlinear optimizations. The smoother of Bohlmann-Whittaker denoises and preserves the total area of the signal. Fourier transform methods to detect and remove periodic noise are also proposed for erratic periodic noise problems. A peak width preserving filter, called the bilateral filter, is also given. This procedure is followed by two reliable baseline correction algorithms based on asymmetric reweighted least squares or curvature-based weights to deal with low-frequency signal variations. Once low- and high-frequency noise components are removed, a convenient peak mode location method is introduced, relying upon numerically stable second derivative calculation from Savitzky-Golay (SG) filter. Peak locations, area, height, and higher statistical moments can be derived from iterative curve fitting. The protocol shows two peak models that can model left or right skewed peaks. The new peak models are (i) a numerically stable version of the bi-directional exponentially modified Gaussian (EMG), suitable for a wide variety of real chromatograms, and (ii) the twice generalized normal model with third- and fourth-moment parameters built into it. The nonlinear least squares regression approach uses the trust-region reflective optimization method. The algorithm converges to a solution even with multiple peaks and approximate guesses of peak parameters. The protocol can be generalized to fit more than 200 peak functions in separation sciences available in literature by minor adaptations of the provided codes in the MATLAB environment.

BERT ensemble based MBR framework for android malware detection

Predicting attacks in Android Malware (AM) devices within recommender systems-based IoT is challenging. A novel framework is presented in this study for AM Detection (AMD) using BERT Ensemble (MBR) and MobileNetV2. The MBR model uses a threat analysis technique to assess Android apps by using a subset of 100 permissions from 329 Android application-based permissions, together with a refined feature set. Using MCADS, DroidRL, CNN, FAGnet, GAN, and FEDriod, the MBR model performs exceptionally well, achieving 98% accuracy, 96% precision, 98% recall, 97% F1-score, and a log loss of 0.058. By leveraging their strengths, the MBR model introduces significant innovation. By using ensemble methods on static data, the MBR framework not only provides a reliable malware detection solution but also presents a novel strategy. This research highlights the potential for significant applications in this dynamic and evolving field by addressing user privacy and system security issues, despite the growing Android malware risks in IoT.

Quantitative analysis of safflower seed oil adulteration based on near-infrared spectroscopy combined with improved sparrow algorithm optimization model ISSA-ELM

Safflower seed oil is expensive, and there are issues in the market such as the adulteration with cheaper edible oils, leading to inferior products. Near-infrared spectroscopy (NIR) is a non-destructive analytical method with the advantages of being fast, non-destructive, and pollution-free. However, in the case of low-concentration adulterated oil, the main components and their contents are nearly identical to those in pure oil, and the spectral feature peaks of the samples are similar, making it difficult for conventional analysis methods to select effective characteristic variables. Extreme Learning Machine (ELM), as a single-hidden-layer feedforward neural network model, possesses strong feature extraction and model representation capabilities. However, its random initialization of weights and biases leads to the issue of blind training. The Sparrow Search Algorithm (SSA) can effectively optimize the random initialization problem in ELM, but it is prone to issues such as being trapped in local optima and slower convergence. This study proposes a novel quantitative adulteration analysis model for safflower seed oil (ISSA-ELM), which combines ELM with an improved Sparrow Search Algorithm (ISSA). In the experiment, safflower seed oil was used as the base oil, and peanut oil, corn oil, and soybean oil were gradually added to prepare adulterated oil samples. To ensure the resolution of experimental samples and accurately capture spectral variations in the low-concentration range, a concentration gradient of 2% was used for adulteration levels between 2% and 70%, while a 5% gradient was applied for concentrations exceeding 70%. Each type of adulterant had 42 gradient concentrations, with 6 samples per concentration, totaling 252 samples. Original spectral data were collected, and the samples were randomly divided into a training set (70%) and a testing set (30%). Different preprocessing methods and feature extraction techniques were combined to discuss the results of three adulteration analysis models: PLS, SSA-ELM and ISSA-ELM. The final experimental results show that compared to the safflower seed oil adulteration prediction model based on SSA-ELM (R2 = 0.8938, RMSE = 0.0835, RPD = 2.9018)and PLS(RP2 = 0.9015, RMSEP = 0.0876, RPD = 3.0128), the model based on ISSA-ELM (RP2 = 0.9934, RMSEP = 0.0207, RPD = 10.1457) offers higher prediction accuracy and stability, and the error detection limit of ISSA-ELM can reach 2%. The ISSA optimized the SSA's tendency to reduce population diversity, slow convergence, and get stuck in local optima in the later stages of iteration, greatly enhancing the global optimization ability of the algorithm. Therefore, ISSA-ELM can effectively identify adulterated safflower seed oil, providing a technological pathway and basis for research into the adulteration of safflower seed oil.

Deep Ensembling of Multiband Images for Earth Remote Sensing and Foramnifera Data

The classification of multiband images captured by advanced sensors, such as satellite-mounted imaging systems, is a critical task in remote sensing and environmental monitoring. These sensors provide high-dimensional data that encapsulate a wealth of spectral and spatial information, enabling detailed analyses of the Earth's surface features. However, the complexity of these data poses significant challenges for accurate and efficient classification. Our study describes and highlights methods for creating ensembles of neural networks for handling multiband images. Two applications are illustrated in this work: (1) satellite image classification tested on the EuroSAT and LCZ42 datasets and (2) a species-level identification of planktic foraminifera. Multichannel images are fed into an ensemble of Convolutional Neural Networks (CNNs) (ResNet50, MobileNetV2, and DenseNet201), where each network is trained using three channels obtained from the multichannel images, and two custom networks (one based on ResNet50 and the other one based on attention) where the input is a multiband image. The ensemble learning framework harnesses these variations to improve classification accuracy, surpassing other advanced methods. The proposed system, implemented in MATLAB 2024b and PyTorch 2.6, is shown to achieve higher classification accuracy than those of human experts for species-level identification of planktic foraminifera (>92% vs. 83%) and state-of-the-art performance on the tested planktic foraminifera, the EuroSAT and LCZ42 datasets.

Research on Pine Wilt Disease Spread Prediction Based on an Improved Light Gradient Boosting Machine Model

Pine wilt disease has caused significant damage to China's ecological and financial resources. To prevent its further spread across the country, proactive control measures are necessary. Given the low accuracy of traditional models, we have employed an enhanced light gradient boosting machine (LGBM) model to predict the development trend of pine wilt disease in China, providing a theoretical basis for its monitoring and prevention. We collected and organized data on the occurrence points of pine wilt disease at the county level in China. By incorporating anthropogenic factors such as the volume of pine wood imports from 2017 to 2022, the density of graded roads, the number of adjacent counties, and the presence of wood processing factories, as well as natural factors such as temperature, humidity, and wind speed, we employed Pearson correlation and LGBM model's feature importance analysis to select the 17 most significant influencing factors. Spatial analysis was conducted on the epidemic subcompartments (a divisional unit smaller than a township) of pine wilt disease for 2022 and 2023, revealing the distribution patterns of epidemic subcompartments within 2 km of roads and the spatial relationships between new and old epidemic subcompartments. We improved the LGBM model using a Bayesian algorithm, sparrow search algorithm, and hunter-prey optimization algorithm. By comparison, the enhanced model was validated to outperform in terms of accuracy, precision, recall, sensitivity, and specificity. Based on the results of correlation analysis and spatial analysis, an enhanced model was used to predict the emergence of pine wilt disease in new counties and districts in the future. Currently, pine wilt disease is primarily concentrated in the central-southern and northeastern provinces of China. Predictions indicate that the disease will further spread to the northeastern and southern regions of the country in the future.

Cooking loss estimation of semispinalis capitis muscle of pork butt using a deep neural network on hyperspectral data

This study evaluated the performance of a deep-learning-based model that predicted cooking loss in the semispinalis capitis (SC) muscle of pork butts using hyperspectral images captured 24 h postmortem. To overcome low-scale samples, 70 pork butts were used with pixel-based data augmentation. Principal component regression (PCR) and partial least squares regression (PLSR) models for predicting cooking loss in SC muscle showed higher R2 values with multiplicative signal correction, while the first derivative resulted in a lower root mean square error (RMSE). The deep learning-based model outperformed the PCR and PLSR models. The classification accuracy of the models for cooking loss grade classification decreased as the number of grades increased, with the models with three grades achieving the highest classification accuracy. The deep learning model exhibited the highest classification accuracy (0.82). Cooking loss in the SC muscle was visualized using a deep learning model. The pH and cooking loss of the SC muscle were significantly correlated with the cooking loss of pork butt slices (-0.54 and 0.69, respectively). Therefore, a deep learning model using hyperspectral images can predict the cooking loss grade of SC muscle. This suggests that nondestructive prediction of the quality properties of pork butts can be achieved using hyperspectral images obtained from the SC muscle.

Chemometric techniques for the prediction of milk composition from MIR spectral data: A review

Chemometrics; use of statistical models to characterise and understand complex chemical systems/samples, is an advancing field. In the dairy industry, the accurate prediction of milk composition involves combining mid-infrared spectroscopy with chemometric techniques for the evaluation of major constituents of milk. The increased interest in determination of detailed composition of dairy products, alongside emerging and more-widespread use of chemometric methodologies, have generated continuous improvement in predictive models for this application. Herein the main chemometric techniques employed for the study of milk composition are described, compared and discussed. The capability of emerging technologies to improve predictive accuracy of models, over the gold standard technique Partial Least-Squares Regression, to provide recommendation for future directions in this area are particularly emphasised. This review will be of particular interest to researchers involved in milk and dairy product composition, alongside all working in application of statistical methodologies to complex chemical systems.

Characterization of rice starch changes in saline and alkaline area under different fertilization conditions based on Raman spectral recognition technology

Starch content in rice is one of the important parameters in characterizing the nutritional quality of rice, and the starch content of rice produced in saline soils under different fertilization conditions varies. In this study, Raman spectroscopy combined with three machine learning models, support vector machine (SVM), feedforward neural network, and k-nearest neighbor classification, was used to classify and evaluate the effect of different fertilizer treatments on rice. The collected rice spectral data were normalized before machine learning, then preprocessed with multiple scattering correction (MSC), standard normal variable, and Savitzky-Golay filtering algorithms to improve the quality and reliability of the data. The evaluation indexes such as the confusion matrix and the receiver operating characteristic curve comprehensively analyzed the model's performance. The research shows that the MSC preprocessing method significantly improves the classification accuracy and prediction ability in all three models, and the classification accuracy was close to 100%, while the overall performance of the SVM models after various preprocessing is the best among the three machine learning methods. The predictive coefficient of determination, predictive root mean square error, and predictive average relative error of the starch content detection model built by the SVM model after MSC preprocessing were 0.93, 0.04%, and 0.20%, respectively, which indicated that its prediction had high accuracy and low error. The results of this study used Raman spectroscopy to carry out the identification of different fertilization techniques and rice starch quality correlation characteristics, providing theoretical and experimental support for the rapid identification of rice quality.

Enhancing precision agriculture through cloud based transformative crop recommendation model

Modern agriculture relies more on technology to boost food production. It aims to improve both the quality and quantity of food. This paper introduces a novel TCRM (Transformative Crop Recommendation Model). It uses advanced machine learning and cloud platforms to give personalized crop recommendations. Unlike traditional methods, TCRM uses real-time data. It includes environmental and agronomic factors to optimize recommendations. The system has SMS alerts for remote farmers. It outperforms baseline algorithms like Logistic Regression, KNN(k-nearest neighbor), and AdaBoost. TCRM empowers farmers with actionable insights, reducing resource wastage while boosting yield. By offering region-specific recommendations, it enhances profitability and promotes sustainable agricultural practices. The model has 94% accuracy, 94.46% precision, and 94% recall. Its F1 score is 93.97%. The fivefold cross-validation score is 97.67%. These findings show that the model can improve precision farming. It can make agriculture more sustainable and efficient.

The application of vibrational spectroscopy in forensic analysis of biological evidence

Vibrational spectroscopy is a powerful analytical domain, within which Fourier transform infrared spectroscopy (FTIR) and Raman spectroscopy stand as exemplars, offering high chemical specificity and sensitivity. These methodologies have been instrumental in the characterization of chemical compounds for an extensive period. They are particularly adept at the identification and analysis of minute sample quantities. Both FTIR and Raman spectroscopy are proficient in elucidating small liquid samples and detecting nuanced molecular alterations. The application of chemometrics further augments their analytical prowess. Currently, these techniques are in the research phase within forensic medicine and have yet to be broadly implemented in examination and identification processes. Nonetheless, studies have indicated that a combined classification model utilizing FTIR and Raman spectroscopy yields exceptional results for the identification of biological fluid-related information and the determination of causes of death. The objective of this review is to delineate the current research trajectory and potential applications of these two vibrational spectroscopic techniques in the detection of body fluids and the ascertainment of causes of death within the context of forensic medicine.

From spectroscopic data variability to optimal preprocessing: leveraging multivariate error in almond powder adulteration of different grain size

Analysing samples in their original form is increasingly crucial in analytical chemistry due to the need for efficient and sustainable practices. Analytical chemists face the dual challenge of achieving accuracy while detecting minute analyte quantities in complex matrices, often requiring sample pretreatment. This necessitates the use of advanced techniques with low detection limits, but the emphasis on sensitivity can conflict with efforts to simplify procedures and reduce solvent use. This article discusses the shift towards green analytical methods, focusing on portable spectroscopic techniques in the near-infrared (NIR) region. A case study involving the prediction of adulteration in almond flour with bitter almond flour illustrates the importance of particle size and the integration between the sample and the instrument. The study emphasizes the necessity of investigating the multivariate error associated with raw data to enhance data preprocessing strategies. This research provides valuable insights for professionals in the field, presenting a methodology applicable to a broad range of analytical applications while underscoring the critical role of raw data analysis in achieving accurate and reliable results.

Distributional uniformity quantification in heterogeneous prepared dishes combined the hyperspectral imaging technology with Moran's I: A case study of pizza

Quality detection is critical in the development of prepared dishes, with distributional uniformity playing a significant role. This study used hyperspectral imaging (HSI) and Moran's I to quantify distributional uniformity, employing pizza as case. Pizza ingredients' spectra were collected, pre-processed with Detrended Fluctuation Analysis (DFA), Savitzky-Golay (SG) and Standard Normal Variate (SNV), and down-scaled with Principal Component Analysis (PCA). Subsequently, the classifiers Fine Tree, Support Vector Machine (SVM), and K-Nearest Neighbors (KNN) were utilized, where KNN based on the DFA-processed data had the greatest accuracy of 99.2 %. This best-fit model was used to create visualization maps. At last, image analysis methods containing regional statistics, Grey Level Co-occurrence Matrix (GLCM) and Moran's I were used to measure distributional uniformity. Moran's I demonstrated great distinctiveness and accuracy, making it the best tool. Therefore, HSI and Moran's I combination proved feasible to indicate distributional uniformity, ensuring the high quality of prepared dishes.

Determination of aflatoxin B1 in wheat using Raman spectroscopy combined with chemometrics

Aflatoxin B1 (AFB1) is carcinogenic and highly susceptible to production in wheat. In this study, the quantitative detection of contaminant AFB1 in wheat was investigated by Raman spectroscopy combined with chemometric method realization. Firstly, Savitzky-Golay smoothing (SG) and baseline calibration methods were used to perform the necessary preprocessing of the collected raw Raman spectra. Then, three variable optimization methods, i.e., competitive adaptive reweighted sampling (CARS), iteratively variable subset optimization (IVSO), and bootstrap soft shrinkage (BOSS), were applied to the preprocessed wheat Raman spectra. Finally, partial least squares regression (PLSR) models were developed to determine AFB1 in wheat samples. The results showed that all three variable optimization algorithms significantly improved the predictive performance of the models. The BOSS-PLSR model has strong generalization performance and robustness. Its prediction coefficient of determination (RP2) was 0.9927, the root mean square error of prediction (RMSEP) was 2.4260 μg/kg, and the relative prediction deviation (RPD) was 11.5250, respectively. In conclusion, the combination of Raman spectroscopy and chemometrics can realize the rapid quantitative detection of AFB1 in wheat.

Feasibility study on non-destructive detection of microplastic content in flour based on portable Raman spectroscopy system combined with mixed variable selection method

Microplastics, as emerging environmental pollutants, have garnered considerable attention due to their contamination of both the environment and food. Microplastics can infiltrate the human food chain through multiple pathways, potentially posing health risks to humans. Currently, non-destructive testing of microplastics in food is considered challenging. This study aims to investigate the feasibility of employing a portable Raman spectroscopy system for non-destructive detection of microplastic content (polystyrene, PS; polyethylene, PE) in flour. In this study, a portable spectrometer was used to collect flour spectra of different abundances of microplastics. To enhance the predictive performance of the partial least squares (PLS) model, a mixed variable selection strategy that combined the wavelength interval selection method (Synergy interval partial least squares, siPLS) and the wavelength point selection method (Least absolute shrinkage and selection operator, LASSO; Multiple feature-spaces ensemble by least absolute shrinkage and selection operator, MFE-LASSO) was proposed. Four regression models (PLS, siPLS, siPLS-LASSO, siPLS-MFE-LASSO) were developed and compared for detecting PS and PE content in flour. The siPLS-MFE-LASSO model exhibited the best generalization performance in the prediction set, and was considered to have the best generalization performance (PS: RP2 = 0.9889, RMSEP=0.0344 %; PE: RP2 = 0.9878, RMSEP=0.0361 %). In conclusion, this study has demonstrated the potential of using a portable Raman spectrometer in conjunction with a mixed variable selection algorithm for non-destructive detection of PS and PE content in flour, providing more possibilities for non-destructive detection of microplastic content in food.

Application of spectroscopic technology with machine learning in Chinese herbs from seeds to medicinal materials: The case of genus Paris

To ensure the safety and efficacy of Chinese herbs, it is of great significance to conduct rapid quality detection of Chinese herbs at every link of their supply chain. Spectroscopic technology can reflect the overall chemical composition and structural characteristics of Chinese herbs, with the multi-component and multitarget characteristics of Chinese herbs. This review took the genus Paris as an example, and applications of spectroscopic technology with machine learning (ML) in supply chain of the genus Paris from seeds to medicinal materials were introduced. The specific contents included the confirmation of germplasm resources, identification of growth years, cultivar, geographical origin, and original processing and processing methods. The potential application of spectroscopic technology in genus Paris was pointed out, and the prospects of combining spectroscopic technology with blockchain were proposed. The summary and prospects presented in this paper will be beneficial to the quality control of the genus Paris in all links of its supply chain, so as to rationally use the genus Paris resources and ensure the safety and efficacy of medication.

Bionic Recognition Technologies Inspired by Biological Mechanosensory Systems

Mechanical information is a medium for perceptual interaction and health monitoring of organisms or intelligent mechanical equipment, including force, vibration, sound, and flow. Researchers are increasingly deploying mechanical information recognition technologies (MIRT) that integrate information acquisition, pre-processing, and processing functions and are expected to enable advanced applications. However, this also poses significant challenges to information acquisition performance and information processing efficiency. The novel and exciting mechanosensory systems of organisms in nature have inspired us to develop superior mechanical information bionic recognition technologies (MIBRT) based on novel bionic materials, structures, and devices to address these challenges. Herein, first bionic strategies for information pre-processing are presented and their importance for high-performance information acquisition is highlighted. Subsequently, design strategies and considerations for high-performance sensors inspired by mechanoreceptors of organisms are described. Then, the design concepts of the neuromorphic devices are summarized in order to replicate the information processing functions of a biological nervous system. Additionally, the ability of MIBRT is investigated to recognize basic mechanical information. Furthermore, further potential applications of MIBRT in intelligent robots, healthcare, and virtual reality are explored with a view to solve a range of complex tasks. Finally, potential future challenges and opportunities for MIBRT are identified from multiple perspectives.

Lycopene detection in cherry tomatoes with feature enhancement and data fusion

Lycopene, a biologically active phytochemical with health benefits, is a key quality indicator for cherry tomatoes. While ultraviolet/visible/near-infrared (UV/Vis/NIR) spectroscopy holds promise for large-scale online lycopene detection, capturing its characteristic signals is challenging due to the low lycopene concentration in cherry tomatoes. This study improved the prediction accuracy of lycopene by supplementing spectral data with image information through spectral feature enhancement and spectra-image fusion. The feasibility of using UV/Vis/NIR spectra and image features to predict lycopene content was validated. By enhancing spectral bands corresponding to colors correlated with lycopene, the performance of the spectral model was improved. Additionally, direct spectra-image fusion further enhanced the prediction accuracy, achieving RP2, RMSEP, and RPD as 0.95, 8.96 mg/kg, and 4.25, respectively. Overall, this research offers valuable insights into supplementing spectral data with image information to improve the accuracy of non-destructive lycopene detection, providing practical implications for online fruit quality prediction.

Near-infrared spectroscopy combined with support vector machine for the identification of Tartary buckwheat (Fagopyrum tataricum (L.) Gaertn) adulteration using wavelength selection algorithms

The frequent occurrence of adulterating Tartary buckwheat powder with crop flours in the market necessitates an urgent need for a simple analysis method to ensure the quality of Tartary buckwheat. This study employed near-infrared spectroscopy (NIRS) for the collection of spectral data from Tartary buckwheat samples adulterated with whole wheat, oat, soybean, barley, and sorghum flours. The competitive adaptive reweighted sampling (CARS) and successive projection algorithm (SPA) were deployed to identify informative wavelengths. By integrating support vector machine (SVM) and partial least squares discriminant analysis (PLS-DA), we constructed qualitative models to discern Tartary buckwheat adulteration. The PLS-DA model exhibited prediction accuracies between 89.78 % and 94.22 %, while the mean-centering (MC)-PLS-DA model showcased impressive predictive accuracy of 93.33 %. Notably, the feature-based Autoscales-CARS-CV-SVM model achieved more excellent identification accuracy. These findings exhibit the excellent potential of chemometrics as a powerful tool for detecting food product adulteration.

Fusion of convolutional neural network with XGBoost feature extraction for predicting multi-constituents in corn using near infrared spectroscopy

Near-infrared (NIR) spectroscopy has been widely utilized to predict multi-constituents of corn in agriculture. However, directly extracting constituent information from the NIR spectra is challenging due to many issues such as broad absorption band, overlapping and non-specific nature. To solve these problems and extract implicit features from the raw data of NIR spectra to improve performance of quantitative models, a one-dimensional shallow convolutional neural network (CNN) model based on an eXtreme Gradient Boosting (XGBoost) feature extraction method was proposed in this paper. The leaf node feature information in the XGBoost was encoded and reconstructed to obtain the implicit features of raw data in the NIR spectra. A two-parametric Swish (TSwish or TS) activation function was proposed to improve the performance of CNN, and the elastic net (EN) was also applied to avoid the overfitting problem of the CNN model. Performance of the developed XGBoost-CNN-TS-EN model was evaluated using two public NIR spectroscopy datasets of corn and soil, and the obtained determination coefficients (R2) for moisture, oil, protein, and starch of the corn on test set were 0.993, 0.991, 0.998, and 0.992, respectively, with that of the soil organic matter being 0.992. The XGBoost-CNN-TS-EN model exhibits superior stability, good prediction accuracy, and generalization ability, demonstrating its great potentials for quantitative analysis of multi-constituents in spectroscopic applications.

Monitoring the Concentrations of Na, Mg, Ca, Cu, Fe, and K in Sargassum fusiforme at Different Growth Stages by NIR Spectroscopy Coupled with Chemometrics

Sargassum fusiforme, an edible seaweed, plays a crucial role in our daily lives by providing essential nutrients, including minerals, to the human body. The detection of mineral content during different growth stages of S. fusiforme benefits the goals of ensuring product quality, meeting diverse consumer needs, and achieving quality classification. Currently, the determination of minerals in S. fusiforme primarily relies on inductively coupled plasma mass spectrometry and other methods, which are time-consuming and labor-intensive. Thus, a rapid and convenient method was developed for the determination of six minerals (i.e., Na, Mg, Ca, Cu, Fe, and K) in S. fusiforme via near-infrared (NIR) spectroscopy based on chemometrics. This study investigated the variations in minerals in S. fusiforme from different growth stages. The effects of four spectral pretreatment methods and three wavelength selection methods, including the synergy interval partial least squares (SI-PLS) algorithm, genetic algorithm (GA), and competitive adaptive reweighted sampling method (CARS) on the model optimization, were evaluated. Superior CARS-PLS models were established for Na, Mg, Ca, Cu, Fe, and K with root mean square error of prediction (RMSEP) values of 0.8196 × 103 mg kg-1, 0.4370 × 103 mg kg-1, 1.544 × 103 mg kg-1, 0.9745 mg kg-1, 49.88 mg kg-1, and 7.762 × 103 mg kg-1, respectively, and coefficient of determination of prediction (RP2) values of 0.9787, 0.9371, 0.9913, 0.9909, 0.9874, and 0.9265, respectively. S. fusiforme demonstrated higher levels of Mg and Ca at the seedling stage and lower levels of Cu and Fe at the maturation stage. Additionally, S. fusiforme exhibited higher Na and lower K at the growth stage. NIR combined with CARS-PLS is a potential alternative for monitoring the concentrations of minerals in S. fusiforme at different growth stages, aiding in the convenient evaluation and further grading of the quality of S. fusiforme.

Recent advances of machine learning in the geographical origin traceability of food and agro-products: A review

The geographical origin traceability of food and agro-products has been attracted worldwide. Especially with the rise of machine learning (ML) technology, it provides cutting-edge solutions to erstwhile intractable issues to identify the origin of food and agro-products. By utilizing advanced algorithms, ML can extract feature information of food and agro-products that is closely related to origin and, more accurately, identify and trace their origins, which is of great significance to the entire food and agriculture industry. This paper provides a comprehensive overview of the state-of-the-art applications of ML in the geographical origin traceability of food and agro-products. First, commonly used ML methods are summarized. The paper then outlines the whole process of preparation for modeling, model training as well as model evaluation for building traceability models-based ML. Finally, recent applications of ML combined with different traceability techniques in the field of food and agro-products are revisited. Although ML has made many achievements in solving the geographical origin traceability problem of food and agro-products, it still has great development potential. For example, the application of ML is yet insufficient in the geographical origin traceability using DNA or computer vision techniques. The ability of ML to predict the geographical origin of food and agro-products can be further improved, for example, by increasing model interpretability, incorporating data fusion strategies, and others.

Laser Desorption-Ion Mobility Spectrometry of Explosives for Forensic and Security Applications

BACKGROUND

The detection of explosives in crime scene investigations is critical for forensic science. This study explores the application of laser desorption (LD) ion mobility spectrometry (IMS) as a novel method for this purpose utilising a new IMS prototype developed by MaSaTECH.

METHODS

The LD sampling technique employs a laser diode module to vaporise explosive traces on surfaces, allowing immediate analysis by IMS without sample preparation. Chemometric approaches, including multivariate data analysis, were utilised for data processing and interpretation, including pre-processing of raw IMS plasmagrams and various pattern recognition techniques, such as linear discriminant analysis (LDA) and support vector machines (SVMs).

RESULTS

The IMS prototype was validated through experiments with pure explosives (TNT, RDX, PETN) and explosive products (SEMTEX 1A, C4) on different materials. The study found that the pre-processing method significantly impacts classification accuracy, with the PCA-LDA model demonstrating the best performance for real-world applications.

CONCLUSIONS

The LD-IMS prototype, coupled with effective chemometric techniques, presents a promising methodology for the detection of explosives in forensic investigations, enhancing the reliability of field applications.

Combination ATR-FTIR with Multiple Classification Algorithms for Authentication of the Four Medicinal Plants from Curcuma L. in Rhizomes and Tuberous Roots

Curcumae Longae Rhizoma (CLRh), Curcumae Radix (CRa), and Curcumae Rhizoma (CRh), derived from the different medicinal parts of the Curcuma species, are blood-activating analgesics commonly used for promoting blood circulation and relieving pain. Due to their certain similarities in chemical composition and pharmacological effects, these three herbs exhibit a high risk associated with mixing and indiscriminate use. The diverse methods used for distinguishing the medicinal origins are complex, time-consuming, and limited to intraspecific differentiation, which are not suitable for rapid and systematic identification. We developed a rapid analysis method for identification of affinis and different medicinal materials using attenuated total reflection-Fourier-transform infrared spectroscopy (ATR-FTIR) combined with machine learning algorithms. The original spectroscopic data were pretreated using derivatives, standard normal variate (SNV), multiplicative scatter correction (MSC), and smoothing (S) methods. Among them, 1D + MSC + 13S emerged as the best pretreatment method. Then, t-distributed stochastic neighbor embedding (t-SNE) was applied to visualize the results, and seven kinds of classification models were constructed. The results showed that support vector machine (SVM) modeling was superior to other models and the accuracy of validation and prediction was preferable, with a modeling time of 127.76 s. The established method could be employed to rapidly and effectively distinguish the different origins and parts of Curcuma species and thus provides a technique for rapid quality evaluation of affinis species.

Quantitative Prediction of Protein Content in Corn Kernel Based on Near-Infrared Spectroscopy

Rapid and accurate detection of protein content is essential for ensuring the quality of maize. Near-infrared spectroscopy (NIR) technology faces limitations due to surface effects and sample homogeneity issues when measuring the protein content of whole maize grains. Focusing on maize grain powder can significantly improve the quality of data and the accuracy of model predictions. This study aims to explore a rapid detection method for protein content in maize grain powder based on near-infrared spectroscopy. A method for determining protein content in maize grain powder was established using near-infrared (NIR) reflectance spectra in the 940-1660 nm range. Various preprocessing techniques, including Savitzky-Golay (S-G), multiplicative scatter correction (MSC), standard normal variate (SNV), and the first derivative (1D), were employed to preprocess the raw spectral data. Near-infrared spectral data from different varieties of maize grain powder were collected, and quantitative analysis of protein content was conducted using Partial Least Squares Regression (PLSR), Support Vector Machine (SVM), and Extreme Learning Machine (ELM) models. Feature wavelengths were selected to enhance model accuracy further using the Successive Projections Algorithm (SPA) and Uninformative Variable Elimination (UVE). Experimental results indicated that the PLSR model, preprocessed with 1D + MSC, yielded the best performance, achieving a root mean square error of prediction (RMSEP) of 0.3 g/kg, a correlation coefficient (Rp) of 0.93, and a residual predictive deviation (RPD) of 3. The associated methods and theoretical foundation provide a scientific basis for the quality control and processing of maize.

Prediction and mapping of leaf water content in Populus alba var. pyramidalis using hyperspectral imagery

Leaf water content (LWC) encapsulates critical aspects of tree physiology and is considered a proxy for assessing tree drought stress and the risk of forest decline; however, its measurement relies on destructive sampling and is thus less efficient. Advancements in hyperspectral imaging technology present new prospects for noninvasively evaluating LWC and mapping drought severity across forested regions. In this study, leaf samples were obtained from Populus alba var. pyramidalis, a species widely employed for constructing farmland shelterbelts in water-limited regions of northern China but notably susceptible to drought. These samples were dehydrated to varying degrees to generate concurrent LWC measurements and hyperspectral images, enabling the development of narrow-band and multivariate spectral prediction models for LWC estimation. Two visible-spectrum narrow-band indices identified, the single-band index (R627) and the band subtraction index (R437 - R444), demonstrated a strong correlation with LWC. Despite certain influences of variable preprocessing and selection on multivariate model performance, most models exhibited robust predictive accuracy for LWC. The FDRL-UVE-PLSR combination emerged as the optimal multivariate model, with R2 values reaching 0.9925 and 0.9853 and RMSE values below 0.0124 and 0.0264 for the calibration and validation datasets, respectively. Using this optimal model, along with localized spectral smoothing, moisture distribution across leaf surfaces was visualized, revealing lower water retention at the leaf margins compared to central regions. These methodologies provide critical insights into subtle water-associated physiological processes at the leaf scale and facilitate high-frequency, large-scale assessments and monitoring of drought stress levels and the risk of drought-induced tree mortality and forest degradation in drylands.

XGBoost algorithm assisted multi-component quantitative analysis with Raman spectroscopy

To improve prediction performance and reduce artifacts in Raman spectra, we developed an eXtreme Gradient Boosting (XGBoost) preprocessing method to preprocess the Raman spectra of glucose, glycerol and ethanol mixtures. To ensure the robustness and reliability of the XGBoost preprocessing method, an X-LR model (which combined XGBoost preprocessing and a linear regression (LR) model) and a X-MLP model (which combined XGBoost preprocessing and a multilayer perceptron (MLP) model) were developed. These two models were used to quantitatively analyze the concentrations of glucose, glycerol and ethanol in the Raman spectra of mixed solutions. The proportion map of hyperparameters was firstly used to narrow down the search range of hyperparameters in the X-LR and the X-MLP models. Then the correlation coefficients (R2), root mean square of calibration (RMSEC), and root mean square error of prediction (RMSEP) were used to evaluate the models' performance. Experimental results indicated that the XGBoost preprocessing method achieved higher accuracy and generalization capability, with better performance than those of other preprocessing methods for both LR and MLP models.

Detection of Pear Quality Using Hyperspectral Imaging Technology and Machine Learning Analysis

The non-destructive detection of fruit quality is indispensable in the agricultural and food industries. This study aimed to explore the application of hyperspectral imaging (HSI) technology, combined with machine learning, for a quality assessment of pears, so as to provide an efficient technical method. Six varieties of pears were used for inspection, including 'Sucui No.1', 'Zaojinxiang', 'Huangguan', 'Akizuki', 'Yali', and 'Hongli No.1'. Spectral data within the 398~1004 nm wavelength range were analyzed to compare the predictive performance of the Least Squares Support Vector Machine (LS-SVM) models on various quality parameters, using different preprocessing methods and the selected feature wavelengths. The results indicated that the combination of Fast Detrend-Standard Normal Variate (FD-SNV) preprocessing and Competitive Adaptive Reweighted Sampling (CARS)-selected feature wavelengths yielded the best improvement in model predictive ability for forecasting key quality parameters such as firmness, soluble solids content (SSC), pH, color, and maturity degree. They could enhance the predictive capability and reduce computational complexity. Furthermore, in order to construct a quality prediction model, integrating hyperspectral data from six pear varieties resulted in an RPD (Ratio of Performance to Deviation) exceeding 2.0 for all the quality parameters, indicating that increasing the fruit sample size and variety number further strengthened the robustness of the model. The Backpropagation Neural Network (BPNN) model could accurately distinguish six distinct pear varieties, achieving prediction accuracies of above 99% for both the calibration and test sets. In summary, the combination of HSI and machine learning models enabled an efficient, rapid, and non-destructive detection of pear quality and provided a practical value for quality control and the commercial processing of pears.

Using portable visible and near-infrared spectroscopy to authenticate beef from grass, barley, and corn-fed cattle

Meat product labels including information on livestock production systems are increasingly demanded, as consumers request total traceability of the products. The aim of this study was to explore the potential of visible and near-infrared spectroscopy (Vis-NIRS) to authenticate meat and fat from steers raised under different feeding systems (barley, corn, grass-fed). In total, spectra from 45 steers were collected (380-2,500 nm) on the subcutaneous fat and intact longissimus thoracis (LT) at 72 h postmortem and, after fabrication, on the frozen-thawed ground longissimus lumborum (LL). In subcutaneous fat samples, excellent results were obtained using partial least squares-discriminant analysis (PLS-DA) with the 100 % of the samples in external Test correctly classified (Vis, NIR or Vis-NIR regions); whereas linear-support vector machine (L-SVM) discriminated 75-100 % in Test (Vis-NIR range). In intact meat samples, PLS-DA segregated 100 % of the samples in Test (Vis-NIR region). A slightly lower percentage of meat samples were correctly classified by L-SVM using the NIR region (75-100 % in Train and Test). For ground meat, 100 % of correctly classified samples in Test was achieved using Vis, NIR or Vis-NIR spectral regions with PLS-DA and the Vis with L-SVM. Variable importance in projection (VIP) reported the influence of fat and meat pigments as well as fat, fatty acids, protein, and moisture absorption for the discriminant analyses. From the results obtained with the animals and diets used in this study, NIRS technology stands out as a reliable and green analytical tool to authenticate fat and meat from different livestock production systems.

Characterization of volatile profiles and markers prediction of eleven popular edible boletes using SDE-GC-MS and FT-NIR combined with chemometric analysis

Wild edible boletes mushrooms are regarded as a delicacy in many countries and regions due to their rich nutritional contents and strong aromatic compounds. This study aimed to identify 445 samples of 11 boletes species collected from Yunnan and Sichuan provinces through molecular analysis. Using simultaneous distillation-extraction (SDE) combined with gas chromatography-mass spectrometry (GC-MS), 97 volatile compounds were identified. Chemometric methods were then applied to analyze the heterogeneity of these volatile compounds among the different species. The results showed that, 22 and 21 volatile compounds were selected using variable importance in projection (VIP > 1) and relative odor activity values (ROAV > 0.1), respectively. Partial least squares discrimnatint analysis (PLS-DA) was then employed to develop pattern recognition models for 11 species, which demonstrated strong identification performance. Furthermore, correlation heat maps, volcano plots, and Fisher linear discriminant analysis identified five volatile organic compounds, including methyl (9E)-9-octadecenoate, 2, 6-dimethylpyrazine, 1-decen-3-one, furfural, and methional as markers for distinguishing 11 boletes species. Ultimately, the rapid content prediction models of partial least squares regression (PLSR) were established by combining Fourier Transform Near-Infrared Spectroscopy (FT-NIR) with the concentrations of these five marker compounds. These findings provide a methodological strategy for the effective species identification of wild edible mushrooms and the rapid prediction of their characteristic aroma compounds.

Quantitative detection of heavy metal Cd in vegetable oils: A nondestructive method based on Raman spectroscopy combined with chemometrics

Heavy metal contaminants in vegetable oils can cause irreversible damage to human health. In this study, the quantitative detection of Cd in vegetable oils was investigated based on Raman spectroscopy combined with chemometric methods. The necessary preprocessing of the Raman signal was performed using baseline calibration and the Savitzky-Golay method. Three variable optimization methods were applied to the preprocessed Raman spectra. Namely, bootstrap soft shrinkage, multiple feature spaces ensemble strategy with least absolute shrinkage and selection operator, and competitive adaptive reweighted sampling (CARS), respectively. Partial least squares regression (PLSR) modeling for the determination of Cd in vegetable oils. The results show that three variable optimization algorithms improved the predictive performance of the model. Among them, the CARS-PLSR model has strong generalization performance and robustness. Its prediction coefficient of determination (R P 2 $R_{\mathrm{P}}^2$ ) was 0.9995, the root mean square error of prediction was 0.3533 mg/kg, and the relative prediction deviation was 44.3748, respectively. In summary, rapid quantitative analysis of Cd contamination in vegetable oils can be realized based on Raman spectroscopy combined with chemometrics.

Prediction and Visualization of Total Volatile Basic Nitrogen in Yellow Croaker (Larimichthys polyactis) Using Shortwave Infrared Hyperspectral Imaging

In the present investigation, we have devised a hyperspectral imaging (HSI) apparatus to assess the chemical characteristics and freshness of the yellow croaker (Larimichthys polyactis) throughout its storage period. This system operates within the shortwave infrared spectrum, specifically ranging from 900 to 1700 nm. A variety of spectral pre-processing techniques, including standard normal variate (SNV), multiple scatter correction, and Savitzky-Golay (SG) derivatives, were employed to augment the predictive accuracy of total volatile basic nitrogen (TVB-N)-which serves as a critical freshness parameter. Among the assessed methodologies, SG-1 pre-processing demonstrated superior predictive accuracy (Rp2 = 0.8166). Furthermore, this investigation visualized freshness indicators as concentration images to elucidate the spatial distribution of TVB-N across the samples. These results indicate that HSI, in conjunction with chemometric analysis, constitutes an efficacious instrument for the surveillance of quality and safety in yellow croakers during its storage phase. Moreover, this methodology guarantees the freshness and safety of seafood products within the aquatic food sector.

The Genetic Characteristics of FT-MIRS-Predicted Milk Fatty Acids in Chinese Holstein Cows

Fourier Transform Mid-Infrared Spectroscopy (FT-MIRS) can be used for quantitative detection of milk components. Here, milk samples of 458 Chinese Holstein cows from 11 provinces in China were collected and we established a total of 22 quantitative prediction models in milk fatty acids by FT-MIRS. The coefficient of determination of the validation set ranged from 0.59 (C18:0) to 0.76 (C4:0). The models were adopted to predict the milk fatty acids from 2138 cows and a new high-throughput computing software HiBLUP was employed to construct a multi-trait model to estimate and analyze genetic parameters in dairy cows. Finally, genome-wide association analysis was performed and seven novel SNPs significantly associated with fatty acid content were selected, investigated, and verified with the FarmCPU method, which stands for "Fixed and random model Circulating Probability Unification". The findings of this study lay a foundation and offer technical support for the study of fatty acid trait breeding and the screening and grouping of characteristic dairy cows in China with rich, high-quality fatty acids. It is hoped that in the future, the method established in this study will be able to screen milk sources rich in high-quality fatty acids.

Predicting Cd(II) adsorption capacity of biochar materials using typical machine learning models for effective remediation of aquatic environments

The screening and design of "green" biochar materials with high adsorption capacity play a pivotal role in promoting the sustainable treatment of Cd(II)-containing wastewater. In this study, six typical machine learning (ML) models, namely Linear Regression, Random Forest, Gradient Boosting Decision Tree, CatBoost, K-Nearest Neighbors, and Backpropagation Neural Network, were employed to accurately predict the adsorption capacity of Cd(II) onto biochars. A large dataset with 1051 data points was generated using 21 input variables obtained from batch adsorption experiments, including preparation conditions for biochar (2 features), physical properties of biochar (4 features), chemical composition of biochar (9 features), and adsorption experiment conditions (6 features). The rigorous evaluation and comparison of the ML models revealed that the CatBoost model exhibited the highest test R2 value (0.971) and the lowest RMSE (20.54 mg/g), significantly outperforming all other models. The feature importance analysis using Shapley Additive Explanations (SHAP) indicated that biochar chemical compositions had the greatest impact on model predictions of adsorption capacity (42.2 %), followed by adsorption conditions (37.57 %), biochar physical characteristics (12.38 %), and preparation conditions (7.85 %). The optimal experimental conditions optimized by partial dependence plots (PDP) are as follows: as high Cd(II) concentration as possible, C(%) of 33 %, N(%) of 0.3 %, adsorption time of 600 min, pyrolysis time of 50 min, biochar dosage of less than 2 g/L, O(%) of 42 %, biochar pH value of 11.2, and DBE of 1.15. This study unveils novel insights into the adsorption of Cd(II) and provides a comprehensive reference for the sustainable engineering of biochars in Cd(II) wastewater treatment.

PROSAC as a selection tool for SO-PLS regression: A strategy for multi-block data fusion

BACKGROUND

Spectral data from multiple sources can be integrated into multi-block fusion chemometric models, such as sequentially orthogonalized partial-least squares (SO-PLS), to improve the prediction of sample quality features. Pre-processing techniques are often applied to mitigate extraneous variability, unrelated to the response variables. However, the selection of suitable pre-processing methods and identification of informative data blocks becomes increasingly complex and time-consuming when dealing with a large number of blocks. The problem addressed in this work is the efficient pre-processing, selection, and ordering of data blocks for targeted applications in SO-PLS.

RESULTS

We introduce the PROSAC-SO-PLS methodology, which employs pre-processing ensembles with response-oriented sequential alternation calibration (PROSAC). This approach identifies the best pre-processed data blocks and their sequential order for specific SO-PLS applications. The method uses a stepwise forward selection strategy, facilitated by the rapid Gram-Schmidt process, to prioritize blocks based on their effectiveness in minimizing prediction error, as indicated by the lowest prediction residuals. To validate the efficacy of our approach, we showcase the outcomes of three empirical near-infrared (NIR) datasets. Comparative analyses were performed against partial-least-squares (PLS) regressions on single-block pre-processed datasets and a methodology relying solely on PROSAC. The PROSAC-SO-PLS approach consistently outperformed these methods, yielding significantly lower prediction errors. This has been evidenced by a reduction in the root-mean-squared error of prediction (RMSEP) ranging from 5 to 25 % across seven out of the eight response variables analyzed.

SIGNIFICANCE

The PROSAC-SO-PLS methodology offers a versatile and efficient technique for ensemble pre-processing in NIR data modeling. It enables the use of SO-PLS minimizing concerns about pre-processing sequence or block order and effectively manages a large number of data blocks. This innovation significantly streamlines the data pre-processing and model-building processes, enhancing the accuracy and efficiency of chemometric models.

Optimized ensemble deep learning for predictive analysis of student achievement

Education is essential for individuals to lead fulfilling lives and attain greatness by enhancing their value. It improves self-assurance and enables individuals to navigate the complexities of modern society effectively. Despite the obstacles it faces, education continues to develop. The objective of numerous pedagogical approaches is to enhance academic performance. The development of technology, especially artificial intelligence, has caused a significant change in learning. This has made instructional materials available anytime and wherever easily accessible. Higher education institutions are adding technology to conventional teaching strategies to improve learning. This work presents an innovative approach to student performance prediction in educational settings. The strategy combines the DistilBERT with LSTM (DBTM) hybrid approach with the Spotted Hyena Optimizer (SHO) to change parameters. Regarding accuracy, log loss, and execution time, the model significantly improved over earlier models. The challenges presented by the increasing volume of data in graduate and postgraduate programs are effectively addressed by the proposed method. It produces exceptional performance metrics, including a 15-25% decrease in processing time through optimization, 98.7% accuracy, and 0.03% log loss. This work additionally demonstrates the effectiveness of DBTM-SHO in administering extensive datasets and makes an important improvement to educational data mining. It provides a robust foundation for organizations facing the challenges of evaluating student achievement in the era of vast data.

Stacking and ridge regression-based spectral ensemble preprocessing method and its application in near-infrared spectral analysis

Spectral preprocessing techniques can, to a certain extent, eliminate irrelevant information, such as current noise and stray light from spectral data, thereby enhancing the performance of prediction models. However, current preprocessing techniques mostly attempt to find the best single preprocessing method or their combination, overlooking the complementary information among different preprocessing methods. These preprocessing techniques fail to maximize the utilization of useful information in spectral data and restrict the performance of prediction models. This study proposed a spectral ensemble preprocessing method based on the rapidly developing ensemble learning methods in recent years and the ridge regression (RR) model, named stacking preprocessing ridge regression (SPRR), to address the aforementioned issues. Different from conventional ensemble learning methods, the proposed SPRR method applied multiple different preprocessing techniques to the original spectral data, generating multiple preprocessed datasets. These datasets were then individually inputted into RR base models for training. Ultimately, RR still served as the meta-model, integrating the output results of each RR base model through stacking. This approach not only produced diversity in base models but also achieved higher accuracy and lower computational complexity by using a single type of base model. On the apple spectral dataset collected by our team, correlation analysis showed significant complementary information among the data produced by different preprocessing techniques. This provided robust theoretical support for the proposed SPRR method. By introducing the currently popular averaging ensemble preprocessing method in a comparative experiment, the results of applying the proposed SPRR method to six datasets (apple, meat, wheat, olive oil, tablet, and corn) demonstrated that compared to the single preprocessing method and averaging ensemble preprocessing method, SPRR yielded the best accuracy and reliability for all six datasets. Furthermore, under the same conditions of the training and test datasets, the proposed SPRR method demonstrated better performance than the four commonly used ensemble preprocessing methods.

Advances in the tea plants phenotyping using hyperspectral imaging technology

Rapid detection of plant phenotypic traits is crucial for plant breeding and cultivation. Traditional measurement methods are carried out by rich-experienced agronomists, which are time-consuming and labor-intensive. However, with the increasing demand for rapid and high-throughput testing in tea plants traits, digital breeding and smart cultivation of tea plants rely heavily on precise plant phenotypic trait measurement techniques, among which hyperspectral imaging (HSI) technology stands out for its ability to provide real-time and rich-information. In this paper, we provide a comprehensive overview of the principles of hyperspectral imaging technology, the processing methods of cubic data, and relevant algorithms in tea plant phenomics, reviewing the progress of applying hyperspectral imaging technology to obtain information on tea plant phenotypes, growth conditions, and quality indicators under environmental stress. Lastly, we discuss the challenges faced by HSI technology in the detection of tea plant phenotypic traits from different perspectives, propose possible solutions, and envision the potential development prospects of HSI technology in the digital breeding and smart cultivation of tea plants. This review aims to provide theoretical and technical support for the application of HSI technology in detecting tea plant phenotypic information, further promoting the trend of developing high quality and high yield tea leaves.

Near-infrared spectroscopy identification method of cashmere and wool fibers based on an optimized wavelength selection algorithm

Cashmere and wool fibers have similar chemical compositions, making them difficult to distinguish based on their absorption peaks and band positions in near-infrared spectroscopy. Existing studies commonly use wavelength selection or feature extraction algorithms to obtain significant spectral features, but traditional algorithms often overlook the correlations between wavelengths, resulting in weak adaptability and local optimum issues. To address this problem, this paper proposes a recognition algorithm based on optimal wavelength selection, which can remove redundant information and make the model effective in capturing patterns and key features of the data. The wavelengths are rearranged by computing the information gain ratio for each wavelength. Then, the sorted wavelengths are grouped based on equal density, which ensures that all wavelengths within each group have equal information and avoids over-focusing on individual groups. Meanwhile, the group genetic algorithm is used to find the wavelengths with highly informative and search optimal grouped combinations, in order to explore the entire spectrum wavelength. Finally, combined with a partial least squares discriminant analysis(PLS-DA) model, the recognition accuracy reached 97.3 %. The results indicate that, compared to traditional methods such as CARS, SPA, and GA, our method effectively reduces redundant information, selects fewer but more informative wavelengths, and improves classification accuracy and model adaptability.

Trends in authentication of edible oils using vibrational spectroscopic techniques

The authentication of edible oils has become increasingly important for ensuring product quality, safety, and compliance with regulatory standards. Some prevalent authenticity issues found in edible oils include blending expensive oils with cheaper substitutes or lower-grade oils, incorrect labeling regarding the oil's source or type, and falsely stating the oil's origin. Vibrational spectroscopy techniques, such as infrared (IR) and Raman spectroscopy, have emerged as effective tools for rapidly and non-destructively analyzing edible oils. This review paper offers a comprehensive overview of recent advancements in using vibrational spectroscopy for authenticating edible oils. The fundamental principles underlying vibrational spectroscopy are introduced and chemometric approaches that enhance the accuracy and reliability of edible oil authentication are summarized. Recent research trends highlighted in the review include authenticating newly introduced oils, identifying oils based on their specific origins, adopting handheld/portable spectrometers and hyperspectral imaging, and integrating modern data handling techniques into the use of vibrational spectroscopic techniques for edible oil authentication. Overall, this review provides insights into the current state-of-the-art techniques and prospects for utilizing vibrational spectroscopy in the authentication of edible oils, thereby facilitating quality control and consumer protection in the food industry.

Revolutionizing Chronic Heart Disease Management: The Role of IoT-Based Ambulatory Blood Pressure Monitoring System

Chronic heart disease (CHD) is a widespread and persistent health challenge that demands immediate attention. Early detection and accurate diagnosis are essential for effective treatment and management of this condition. To overcome this difficulty, we created a state-of-the-art IoT-Based Ambulatory Blood Pressure Monitoring System that provides real-time blood pressure readings, systolic, diastolic, and pulse rates at predefined intervals. This unique technology comes with a module that forecasts CHD's early warning score. Various machine learning algorithms employed comprise Naïve Bayes, K-Nearest Neighbors (K-NN), random forest, decision tree, and Support Vector Machine (SVM). Using Naïve Bayes, the proposed model has achieved an impressive 99.44% accuracy in predicting blood pressure, a vital aspect of real-time intensive care for CHD. This IoT-based ambulatory blood pressure monitoring (IABPM) system will provide some advancement in the field of healthcare. The system overcomes the limitations of earlier BP monitoring devices, significantly reduces healthcare costs, and efficiently detects irregularities in chronic heart diseases. By implementing this system, we can take a significant step forward in improving patient outcomes and reducing the global burden of CHD. The system's advanced features provide an accurate and reliable diagnosis that is essential for treating and managing CHD. Overall, this IoT-based ambulatory blood pressure monitoring system is an important tool for the early identification and treatment of CHD in the field of healthcare.

Determination of heavy metals in edible oils by a novel voltammetry taste sensor array

Herein, a novel voltammetry taste sensor array (VTSA) using pencil graphite electrode, screen-printed electrode, and glassy carbon electrode was used to identify heavy metals (HM) including Cad, Pb, Sn and Ni in soybean and rapeseed oils. HMs were added to edible oils at three concentrations of 0.05, 0.1 and 0.25 ppm, and then, the output of the device was classified using a chemometric classification method. According to the principal component analysis results, PG electrode explains 96% and 81% of the variance between the data in rapeseed and soybean edible oils, respectively. Additionally, the SP electrode explains 91% of the variance between the data in rapeseed and soybean oils. Moreover, the GC electrode explains 100% and 99% of the variance between the data in rapeseed and soybean edible oils, respectively. K-nearest neighbor exhibited high capability in classifying HMs in edible oils. In addition, partial least squares in the combine of VTSA shows a predict 99% in rapeseed oil. The best electrode for soybean edible oil was GC.

A Raman spectroscopy based chemometric approach to predict the derived cetane number of hydrocarbon jet fuels and their mixtures

Fuel ignition quality, measured in the form of Derived Cetane Number (DCN), is an important part of integrating fuels, including sustainable aviation fuels, in compression ignition engines. DCN has been correlated with simulated and/or real spectroscopic measurements as well as other physical and chemical properties, but rarely have these correlations developed into a pathway to application. One application of the correlations is the use of miniaturized onboard fuel sensors that could assist, by using predicted DCN, in real-time feedforward engine control. To aid in the application of developing such DCN fuel sensors, Raman spectra coupled with chemometrics and a selection of influential spectral features were investigated. In this study, the Raman spectra were obtained from a database that included jet fuels, jet fuel mixtures, pure hydrocarbon components, and their weighted mixtures. The resulting Raman spectral database from the experimental measurements included spectra of components that span a wide range of DCNs and covered all the expected chemical functional groups present in a standard jet fuel. Chemometric models were developed to associate Raman spectra with DCN in subsets of the spectral range to aid in sensor miniaturization. The models were tested on jet fuels such as National Jet Fuel Combustion Program fuels designated A-1, A-2, and A-3 along with mixtures of jet fuels that spanned a wide range of DCN, simulating fuels that could represent real-world scenarios. An Artificial Neural Network (ANN) model trained on the fingerprint region (500 cm-1 - 1800 cm-1) of the Raman spectra was able to capture the non-linearity of the association between the Raman spectra and DCN with a test R2 score of 0.926, a test MSE of 3.61, and a test MPE of 3.41. Around 97 % of the unseen test samples were predicted within 10 % of the DCN measured with an Ignition Quality Tester. One hundred features of the fingerprint region influencing DCN predictions in the optimal ANN model were extracted using a Global Surrogate (GS) model. A reduced ANN model trained on only these one hundred features performed slightly better with a test R2 score of 0.935, test MSE of 3.19, test MPE of 3.20 and with the entire set of unseen test samples predicted within 10 % of the measured DCN. For assessing applicability of real-time and online DCN sensing, the Raman spectrometer was integrated with a flow cell capable of allowing measurements of DCN in flowing fuel samples and included the optimal ANN model of the fingerprint region and the 100-feature GS-ANN model on a Raspberry Pi computer. A number of unseen F-24/alcohol-to-jet fuel mixtures composed of unknown volumes were tested using the flow cell for DCN, and all of these samples were predicted within 10 % of the measured DCN.

Estimation of the rice aboveground biomass based on the first derivative spectrum and Boruta algorithm

Aboveground biomass (AGB) is regarded as a critical variable in monitoring crop growth and yield. The use of hyperspectral remote sensing has emerged as a viable method for the rapid and precise monitoring of AGB. Due to the extensive dimensionality and volume of hyperspectral data, it is crucial to effectively reduce data dimensionality and select sensitive spectral features to enhance the accuracy of rice AGB estimation models. At present, derivative transform and feature selection algorithms have become important means to solve this problem. However, few studies have systematically evaluated the impact of derivative spectrum combined with feature selection algorithm on rice AGB estimation. To this end, at the Xiaogang Village (Chuzhou City, China) Experimental Base in 2020, this study used an ASD FieldSpec handheld 2 ground spectrometer (Analytical Spectroscopy Devices, Boulder, Colorado, USA) to obtain canopy spectral data at the critical growth stage (tillering, jointing, booting, heading, and maturity stages) of rice, and evaluated the performance of the recursive feature elimination (RFE) and Boruta feature selection algorithm through partial least squares regression (PLSR), principal component regression (PCR), support vector machine (SVM) and ridge regression (RR). Moreover, we analyzed the importance of the optimal derivative spectrum. The findings indicate that (1) as the growth stage progresses, the correlation between rice canopy spectrum and AGB shows a trend from high to low, among which the first derivative spectrum (FD) has the strongest correlation with AGB. (2) The number of feature bands selected by the Boruta algorithm is 19~35, which has a good dimensionality reduction effect. (3) The combination of FD-Boruta-PCR (FB-PCR) demonstrated the best performance in estimating rice AGB, with an increase in R² of approximately 10% ~ 20% and a decrease in RMSE of approximately 0.08% ~ 14%. (4) The best estimation stage is the booting stage, with R2 values between 0.60 and 0.74 and RMSE values between 1288.23 and 1554.82 kg/hm2. This study confirms the accuracy of hyperspectral remote sensing in estimating vegetation biomass and further explores the theoretical foundation and future direction for monitoring rice growth dynamics.

Application of Hyperspectral Technology with Machine Learning for Brix Detection of Pastry Pears

Sugar content is an essential indicator for evaluating crisp pear quality and categorization, being used for fruit quality identification and market sales prediction. In this study, we paired a support vector machine (SVM) algorithm with genetic algorithm optimization to reliably estimate the sugar content in crisp pears. We evaluated the spectral data and actual sugar content in crisp pears, then applied three preprocessing methods to the spectral data: standard normal variable transformation (SNV), multivariate scattering correction (MSC), and convolution smoothing (SG). Support vector regression (SVR) models were built using processing approaches. According to the findings, the SVM model preprocessed with convolution smoothing (SG) was the most accurate, with a correlation coefficient 0.0742 higher than that of the raw spectral data. Based on this finding, we used competitive adaptive reweighting (CARS) and the continuous projection algorithm (SPA) to select key representative wavelengths from the spectral data. Finally, we used the retrieved characteristic wavelength data to create a support vector machine model (GASVR) that was genetically tuned. The correlation coefficient of the SG-GASVR model in the prediction set was higher by 0.0321 and the root mean square prediction error (RMSEP) was lower by 0.0267 compared with those of the SG-SVR model. The SG-CARS-GASVR model had the highest correlation coefficient, at 0.8992. In conclusion, the developed SG-CARS-GASVR model provides a reliable method for detecting the sugar content in crisp pear using hyperspectral technology, thereby increasing the accuracy and efficiency of the quality assessment of crisp pear.

Illuminating the Tiny World: A Navigation Guide for Proper Raman Studies on Microorganisms

Raman spectroscopy is an emerging method for the identification of bacteria. Nevertheless, a lot of different parameters need to be considered to establish a reliable database capable of identifying real-world samples such as medical or environmental probes. In this review, the establishment of such reliable databases with the proper design in microbiological Raman studies is demonstrated, shining a light into all the parts that require attention. Aspects such as the strain selection, sample preparation and isolation requirements, the phenotypic influence, measurement strategies, as well as the statistical approaches for discrimination of bacteria, are presented. Furthermore, the influence of these aspects on spectra quality, result accuracy, and read-out are discussed. The aim of this review is to serve as a guide for the design of microbiological Raman studies that can support the establishment of this method in different fields.