Benchtop and different miniaturized near-infrared spectrometers application study: Calibration transfer and 2D-COS for in-situ analysis of moisture content in HPMC

The demand for miniaturized near-infrared (NIR) spectrometers has surged due to their potential for in-situ analysis. However, their predictive accuracy has not yet matched that of traditional benchtop instruments. This study evaluates the effectiveness of rapid quantitative moisture analysis in hydroxypropyl methylcellulose (HPMC) using a benchtop spectrometer, Antaris II from Thermo Fisher Scientific Inc., and five miniaturized spectrometers (MicroNIR 1700 from Viavi Solutions, OTO-SW2540 from OtOPhotonics, IAS DLP 1700 from Dallas, NIRONE Sensor 2.2 from Helsinki, and NIRS M1800 from Alian Optoelectronics). This study employed an Improved Principal Component Analysis (IPCA) transfer method to standardize spectra from the diverse miniaturized NIR spectrometers, facilitating calibration transfer across different spectroscopic technologies. The benchtop (Antaris II) delivered the most superior results, indicating that miniaturized spectrometers must refine their methodologies to approach the predictive performance of benchtop counterparts. Further, this work conducted a two-dimensional correlation spectroscopy (2D-COS) analysis on the spectra from various spectrometers. This analysis bolstered the partial least squares regression (PLSR) model, highlighting discrepancies between miniaturized and benchtop spectrometers and deepening understanding of the factors influencing the PLSR models. The IPCA leverages the benchtop model to enhance the precision and reliability of miniaturized NIR spectrometers. This innovative and versatile research approach aims to further optimize the performance of miniaturized NIR spectrometers for specific applications, representing a significant step forward in their development.

Outlier Removal with Weight Penalization and Aggregation: A Robust Variable Selection Method for Enhancing Near-Infrared Spectral Analysis Performance

Full-wavelength near-infrared (NIR) spectroscopy faces significant challenges due to the strong collinearity among spectral variables and the presence of variables that are highly sensitive to sample fluctuations. Additionally, not all spectral variables contribute equally to the NIR model. Weakly influential variables, although not important on their own, can provide substantial improvement when combined with stronger variables, thus increasing both model stability and prediction accuracy. Therefore, this study proposes a new variable selection method called outlier removal with weight penalization and aggregation (OR-WPA). The method begins by removing outlier spectral variables with high coefficient of variation, which enhances model stability. During the variable selection process, multiple submodels are constructed based on variable subsets, with variable weights assigned according to the absolute values of regression coefficients. A moving window is applied to average the weights, and variables with excessively high weights are penalized, promoting the selection of weakly influential variables that positively contribute to model accuracy. The variable space is iteratively reduced, and the subset of variables associated with the highest predictive accuracy is selected as the final characteristic variable combination. The OR-WPA method was evaluated on three NIR spectral data sets, involving corn, heated tobacco substrate, and flue-cured tobacco. The results were compared with three advanced variable selection methods: Monte Carlo uninformative variable elimination, competitive adaptive reweighted sampling, and bootstrapping soft shrinkage. The results indicate that OR-WPA demonstrates better predictive performance, particularly in predicting low-content components, where it significantly enhances both the accuracy and stability of the NIR model.

Statistic and Network Features of RGB and Hyperspectral Imaging for Determination of Black Root Mold Infection in Apples

Apples damaged by black root mold (BRM) lose moisture, vitamins, and minerals as well as carry dangerous toxins. Determination of the infection degree can allow for customized use of apples, reduce financial losses, and ensure food safety. In this study, red-green-blue (RGB) imaging and hyperspectral imaging (HSI) are combined to detect the infection degree of BRM in apple fruits. First, RGB and HSI images of healthy, mildly, moderately, and severely infected fruits are measured, and those with effective wavelengths (EWs) are screened from HSI by random frog. Second, the statistic and network features of images are extracted by using color moment and convolutional neural network. Meanwhile, random forest (RF), K-nearest neighbor, and support vector machine are used to construct classification models with the above two features of RGB and HSI images of EWs. Optimal results with the 100% accuracy of training set and 96% accuracy of prediction set are obtained by RF with the statistic and network features of the two images, outperforming the other cases. The proposed method furnishes an accurate and effective solution for determining the BRM infection degree in apples.

Variable selection in the chemometric treatment of food data: A tutorial review

Food analysis covers aspects of quality and detection of possible frauds to ensure the integrity of the food. The arsenal of analytical instruments available for food analysis is broad and allows the generation of a large volume of information per sample. But this instrumental information may not yet give the desired answer; it must be processed to provide a final answer for decision making. The possibility of discarding non-informative and/or redundant signals can lead to models of better accuracy, robustness, and chemical interpretability, in line with the principle of parsimony. Thus, in this tutorial review, we cover aspects of variable selection in food analysis, including definitions, theoretical aspects of variable selection, and case studies showing the advantages of variable selection-based models concerning the use of a wide range of non-informative and redundant instrumental information in the analysis of food matrices.

Reflectance spectroscopy with operator difference for determination of behenic acid in edible vegetable oils by using convolutional neural network and polynomial correction

A novel polynomial correction method, order-adaptive polynomial correction (OAPC), was proposed to correct reflectance spectra with operator differences, and convolutional neural network (CNN) was used to develop analysis model to predict behenic acid in edible oils. With application of OAPC, CNN performed well with coefficient of determination of correction (R²_cor) of 0.8843 and root mean square error of correction (RMSE_cor) of 0.1182, outperforming partial least squares regression, support vector regression and random forest with OAPC, as well as the cases without OAPC. Based on 16 effective wavelengths selected by combination of bootstrapping soft shrinkage, random frog and Pearson's correlation, CNN and OAPC exhibited excellent performance with R²_cor of 0.9560 and RMSE_cor of 0.0730. Meanwhile, only 5% correction samples were selected by Kennard-Stone for OAPC. Overall, the proposed method could alleviate the impact of operator differences on spectral analysis, thereby providing potential to correct differences from measurement instruments or environments.

An efficient variable selection method based on random frog for the multivariate calibration of NIR spectra

Variable selection is a critical step for spectrum modeling. In this study, a new method of variable interval selection based on random frog (RF), known as Interval Selection based on Random Frog (ISRF), is developed. In the ISRF algorithm, RF is used to search the most likely informative variables and then, a local search is applied to expand the interval width of the informative variables. Through multiple runs and visualization of the results, the best informative interval variables are obtained. This method was tested on three near infrared (NIR) datasets. Four variable selection methods, namely, genetic algorithm PLS (GA-PLS), random frog, interval random frog (iRF) and interval variable iterative space shrinkage approach (iVISSA) were used for comparison. The results show that the proposed method is very efficient to find the best interval variables and improve the model's prediction performance and interpretation.

Estimation of the critical quality attributes for hydroxypropyl methylcellulose with near-infrared spectroscopy and chemometrics

With the implementation of quality by design (QbD), critical attributes of raw material (drug substance and excipients) are of significantly importance in pharmaceutical manufacturing process. It is desirable for the quality control of critical material attributes (CMAs) of excipients to ensure the quality of end product. This paper explored the feasibility of an at-line method for the quantitative analysis of hydroxypropoxy group in hydroxypropyl methylcellulose (HPMC) with near infrared spectroscopy (NIRS). Hydroxypropoxy group content can be seen as a CMA of HPMC for quality control. The partial least squares (PLS) model was built with 61 samples including 47 samples as calibration set, 14 samples as validation set by sample set partitioning based on joint x-y distances (SPXY) method. Multiplicative scattering correction (MSC) combined with Savitzkye-Golay (SG) smoothing with first derivative was used as the appropriate pretreatment method. Three variable selection methods including interval partial least-squares (iPLS), competitive adaptive reweighted Sampling (CARS), and the combination of the two methods (iPLS-CARS) were performed for optimizing the model. The results indicated that NIRS could predict rapidly and effectively the content of hydroxypropoxy group in HPMC. NIRS could be a potential method for the quality control of CMAs.

A bootstrapping soft shrinkage approach for variable selection in chemical modeling

In this study, a new variable selection method called bootstrapping soft shrinkage (BOSS) method is developed. It is derived from the idea of weighted bootstrap sampling (WBS) and model population analysis (MPA). The weights of variables are determined based on the absolute values of regression coefficients. WBS is applied according to the weights to generate sub-models and MPA is used to analyze the sub-models to update weights for variables. The optimization procedure follows the rule of soft shrinkage, in which less important variables are not eliminated directly but are assigned smaller weights. The algorithm runs iteratively and terminates until the number of variables reaches one. The optimal variable set with the lowest root mean squared error of cross-validation (RMSECV) is selected. The method was tested on three groups of near infrared (NIR) spectroscopic datasets, i.e. corn datasets, diesel fuels datasets and soy datasets. Three high performing variable selection methods, i.e. Monte Carlo uninformative variable elimination (MCUVE), competitive adaptive reweighted sampling (CARS) and genetic algorithm partial least squares (GA-PLS) are used for comparison. The results show that BOSS is promising with improved prediction performance. The Matlab codes for implementing BOSS are freely available on the website: http://www.mathworks.com/matlabcentral/fileexchange/52770-boss.

Gene selection and cancer classification using Monte Carlo and nonnegative matrix factorization

Cancer classification is a key problem for identifying the genomic biomarkers and treating cancerous tumors in clinical research.

A novel variable selection approach that iteratively optimizes variable space using weighted binary matrix sampling

In this study, a new optimization algorithm called the Variable Iterative Space Shrinkage Approach (VISSA) that is based on the idea of model population analysis (MPA) is proposed for variable selection. Unlike most of the existing optimization methods for variable selection, VISSA statistically evaluates the performance of variable space in each step of optimization. Weighted binary matrix sampling (WBMS) is proposed to generate sub-models that span the variable subspace. Two rules are highlighted during the optimization procedure. First, the variable space shrinks in each step. Second, the new variable space outperforms the previous one. The second rule, which is rarely satisfied in most of the existing methods, is the core of the VISSA strategy. Compared with some promising variable selection methods such as competitive adaptive reweighted sampling (CARS), Monte Carlo uninformative variable elimination (MCUVE) and iteratively retaining informative variables (IRIV), VISSA showed better prediction ability for the calibration of NIR data. In addition, VISSA is user-friendly; only a few insensitive parameters are needed, and the program terminates automatically without any additional conditions. The Matlab codes for implementing VISSA are freely available on the website: https://sourceforge.net/projects/multivariateanalysis/files/VISSA/.

A new method for wavelength interval selection that intelligently optimizes the locations, widths and combinations of the intervals

In this study, a new algorithm for wavelength interval selection, known as interval variable iterative space shrinkage approach (iVISSA), is proposed based on the VISSA algorithm. It combines global and local searches to iteratively and intelligently optimize the locations, widths and combinations of the spectral intervals. In the global search procedure, it inherits the merit of soft shrinkage from VISSA to search the locations and combinations of informative wavelengths, whereas in the local search procedure, it utilizes the information of continuity in spectroscopic data to determine the widths of wavelength intervals. The global and local search procedures are carried out alternatively to realize wavelength interval selection. This method was tested using three near infrared (NIR) datasets. Some high-performing wavelength selection methods, such as synergy interval partial least squares (siPLS), moving window partial least squares (MW-PLS), competitive adaptive reweighted sampling (CARS), genetic algorithm PLS (GA-PLS) and interval random frog (iRF), were used for comparison. The results show that the proposed method is very promising with good results both on prediction capability and stability. The MATLAB codes for implementing iVISSA are freely available on the website: .

WITHDRAWN: Recent advances in chemometric methods for plant metabolomics: A review

This article has been withdrawn at the request of the author(s) and/or editor. The Publisher apologizes for any inconvenience this may cause. The full Elsevier Policy on Article Withdrawal can be found at http://www.elsevier.com/locate/withdrawalpolicy.