Wavelength interval selection in multicomponent spectral analysis by moving window partial least-squares regression with applications to mid-infrared and near-infrared spectroscopic data

Spectral noise-to-signal ratio priority method with application for visible and near-infrared analysis of whole blood viscosity

Whole blood viscosity (WBV) is a group of important clinical indicators of cardio-cerebral vascular diseases. Existing detection methods for WBV are complex, making them inconvenient for large population screening. Blood viscosity is closely related to the deformability and aggregation of erythrocytes, which are associated with haemoglobin. Haemoglobin has obvious near-infrared (NIR) spectral absorption. Scattering occurs when NIR light enters a viscous blood sample, and its scattering degree is correlated with blood viscosity. Based on repeated spectral measurements and spectral similarity, spectral noise-to-signal ratio (NSR) was proposed to quantify the spectral scattering effect in the blood sample. A novel selection method of piecewise-continuous wavelengths, named NSR priority-partial least squares (NSRP-PLS), was proposed and applied for visible-NIR quantitative analysis of WBV with high, medium and low shear rates [WBV(H), WBV(M), WBV(L)]. Modelling was separately performed by gender to allow for systematic gender differences in blood viscosity. For the NIR-predicted and clinically measured values of the three WBV indicators in independent validation, the root mean square errors for prediction (SEP) were 0.498, 0.222 and 0.193 (mPa·s), respectively. And the correlation coefficients (R_P) were 0.927, 0.934 and 0.927, respectively. Compared with the three current well-performing methods (MW-PLS, CARS-PLS and SPA-PLS), the proposed NSRP-PLS method achieved better predictive accuracy. Results indicated that visible-NIR spectroscopy combined with the NSRP-PLS method can be used for the quantitative analysis of WBV. The proposed analytical method is rapid, reagent-free and is scientific and meaningful for cardio-cerebral vascular diseases screening in large populations.

Rapid and non-destructive analysis for the identification of multi-grain rice seeds with near-infrared spectroscopy

The rapid and non-destructive discriminant analysis of rice seeds has great significance for large-scale agriculture. Using near-infrared (NIR) diffuse-reflectance spectroscopy with partial least squares-discriminant analysis (PLS-DA), a variety identification method of multi-grain rice seeds was developed. The equidistant combination method was adopted for large-range wavelength screening. A step-by-step phase-out method was proposed to eliminate interference wavelengths and improve the predicted effect. The optimal wavelength model was a combination of 54 wavelengths within 808-974 nm of the short-NIR region. One type of pure rice variety (Y Liangyou 900) was used for identification (negative). Positive samples included the other four pure varieties and contamination of Y Liangyou 900 by the above four varieties. The recognition-accuracy rates for positive, negative and total validation samples reached 93.1%, 95.1%, and 94.3%, respectively. In the long-NIR region, the local optimal wavelength model was a combination of 49 wavelengths within 1188-1650 nm, and the recognition-accuracy rates for positive, negative and total validation samples were 90.3%, 94.1%, and 92.5%, respectively. Results confirmed the feasibility of NIR spectroscopy for variety identification of multi-grain rice seeds. The proposed two discrete-wavelength models located in the short- and long-NIR regions can provide valuable reference to a dedicated spectrometer.

A Comparison of Sparse Partial Least Squares and Elastic Net in Wavelength Selection on NIR Spectroscopy Data

Elastic net (Enet) and sparse partial least squares (SPLS) are frequently employed for wavelength selection and model calibration in analysis of near infrared spectroscopy data. Enet and SPLS can perform variable selection and model calibration simultaneously. And they also tend to select wavelength intervals rather than individual wavelengths when the predictors are multicollinear. In this paper, we focus on comparison of Enet and SPLS in interval wavelength selection and model calibration for near infrared spectroscopy data. The results from both simulation and real spectroscopy data show that Enet method tends to select less predictors as key variables than SPLS; thus it gets more parsimony model and brings advantages for model interpretation. SPLS can obtain much lower mean square of prediction error (MSE) than Enet. So SPLS is more suitable when the attention is to get better model fitting accuracy. The above conclusion is still held when coming to performing the strongly correlated NIR spectroscopy data whose predictors present group structures, Enet exhibits more sparse property than SPLS, and the selected predictors (wavelengths) are segmentally successive.

Quality evaluation of flue-cured tobacco by near infrared spectroscopy and spectral similarity method

Herein we propose near infrared (NIR) spectroscopy as a rapid method of evaluating the quality of agricultural products. Unlike existing quantitative or qualitative models, quality similarity is characterised using spectral similarity. Key factors of the spectral similarity method were investigated, including variable selection, pre-processing and similarity measures. Sophisticated techniques were developed to ensure the reliability of similarity algorithm. The proposed method was tested by quality similarity of flue-cured tobacco samples. The results demonstrated that the quality-related factors between the target and the similar samples (determined by spectral similarity), showed high similarities. This new method has the potential to characterise product quality effectively and could be a useful new alternative to the widely used PLS models.

Multicomponent spectrometric analysis of drugs and their preparations

Pharmaceutical preparations may contain a single ingredient or multi-ingredients as well as excipients. In multicomponent systems, specific analytical methods are required to determine the concentrations of individual components in the presence of interfering substances. Ultraviolet and visible spectrometric methods have widely been developed for the analysis of drugs in mixtures and pharmaceutical preparations. These methods are based on ultraviolet and visible multicomponent analysis and chemometrics (multivariate data analysis). The commonly used chemometric methods include principal component analysis (PCA); regression involving classical least squares (CLS), partial least squares (PLS), inverse least squares (ILS), principal component regression (PCR), multiple linear regression (MLR), artificial neural networks (ANNs); soft independent modeling of class anthology (SIMCA), PLS-discriminant analysis (DA); and functional data analysis (FDA). In this chapter, the applications of multicomponent ultraviolet and visible, derivative, infrared and mass spectrometric and spectrofluorimetric methods to the analysis of multi-ingredient pharmaceutical preparations, biological samples and the kinetics of drug degradation have been reviewed. Chemometric methods provide an efficient solution to calibration problems in the analysis of spectral data for the simultaneous determination of drugs in multicomponent systems. These methods facilitate the assessment of product quality and enhance the efficiency of quality control systems.

Rapid determination of phytosterols by NIRS and chemometric methods

Phytosterols have been extensively studied because it plays essential roles in the physiology of plants and can be used as nutritional supplement to promote human health. We use a rapid method by coupling near-infrared spectroscopy (NIRS) and chemometric techniques to quickly and efficiently determine three essential phytosterols (β-sitosterol, campesterol and stigmasterol) in vegetable oils. Continuous wavelet transform (CWT) method was adopted to remove the baseline shift in the spectra. The quantitative analysis models were constructed by partial least squares (PLS) regression and randomization test (RT) method was used to further improve the models. The optimized models were used to calculate the phytosterol contents in prediction set in order to evaluate their predictability. We have found that the phytosterol contents obtained by the optimized models and Gas Chromatography/Mass Spectrometry (GC/MS) analysis are almost consistent. The root mean square error of prediction (RMSEP) and ratio of prediction to deviation (RPD) for the three phytosterols are 525.7590, 212.2245, 65.1611 and 4.0060, 4.7195 and 3.5441, respectively. The results have proved the feasibility of the proposed method for rapid and non-destructive analysis of phytosterols in edible oils.

A modification of the bootstrapping soft shrinkage approach for spectral variable selection in the issue of over-fitting, model accuracy and variable selection credibility

In this study, we proposed a new computational method stabilized bootstrapping soft shrinkage approach (SBOSS) for variable selection based on bootstrapping soft shrinkage approach (BOSS) which can enhance the analysis of chemical interest from the massive variables among the overlapped absorption bands. In SBOSS, variable is selected by the index of stability of regression coefficients instead of regression coefficients absolute value. In each loop, a weighted bootstrap sampling (WBS) is applied to generate sub-models, according to the weights update by conducting model population analysis (MPA) on the stability of regression coefficients (RC) of these sub-models. Finally, the subset with the lowest RMSECV is chosen to be the optimal variable set. The performance of the SBOSS was evaluated by one simulated dataset and three NIR datasets. The results show that SBOSS can select the fewer variables and supply the least RMSEP and latent variable number of the PLS model with the best stability comparing with methods of Monte Carlo uninformative variables elimination (MCUVE), genetic algorithm (GA), competitive reweighted sampling (CARS), stability of competitive adaptive reweighted sampling (SCARS) and BOSS.

Laboratory Visible and Near-Infrared Spectroscopy with Genetic Algorithm-Based Partial Least Squares Regression for Assessing the Soil Phosphorus Content of Upland and Lowland Rice Fields in Madagascar

As a laboratory proximal sensing technique, the capability of visible and near-infrared (Vis-NIR) diffused reflectance spectroscopy with partial least squares (PLS) regression to determine soil properties has previously been demonstrated. However, the evaluation of the soil phosphorus (P) content—a major nutrient constraint for crop production in the tropics—is still a challenging task. PLS regression with waveband selection can improve the predictive ability of a calibration model, and a genetic algorithm (GA) has been widely applied as a suitable method for selecting wavebands in laboratory calibrations. To develop a laboratory-based proximal sensing method, this study investigated the potential to use GA-PLS regression analyses to estimate oxalate-extractable P in upland and lowland soils from laboratory Vis-NIR reflectance data. In terms of predictive ability, GA-PLS regression was compared with iterative stepwise elimination PLS (ISE-PLS) regression and standard full-spectrum PLS (FS-PLS) regression using soil samples collected in 2015 and 2016 from the surface of upland and lowland rice fields in Madagascar (n = 103). Overall, the GA-PLS model using first derivative reflectance (FDR) had the best predictive accuracy (R2 = 0.796) with a good prediction ability (residual predictive deviation (RPD) = 2.211). Selected wavebands in the GA-PLS model did not perfectly match wavelengths of previously known absorption features of soil nutrients, but in most cases, the selected wavebands were within 20 nm of previously known wavelength regions. Bootstrap procedures (N = 10,000 times) using selected wavebands also confirmed the improvements in accuracy and robustness of the GA-PLS model compared to those of the ISE-PLS and FS-PLS models. These results suggest that soil oxalate-extractable P can be predicted from Vis-NIR spectroscopy and that GA-PLS regression has the advantage of tuning optimum bands for PLS regression, contributing to a better predictive ability.

Simultaneous Recognition of Species, Quality Grades, and Multivariate Calibration of Antioxidant Activities for 12 Famous Green Teas Using Mid- and Near-Infrared Spectroscopy Coupled with Chemometrics

In this paper, mid- and near-infrared spectroscopy fingerprints were combined to simultaneously discriminate 12 famous green teas and quantitatively characterize their antioxidant activities using chemometrics. A supervised pattern recognition method based on partial least square discriminant analysis (PLSDA) was adopted to classify the 12 famous green teas with different species and quality grades, and then optimized sample-weighted least-squares support vector machine (OSWLS-SVM) based on particle swarm optimization was employed to investigate the quantitative relationship between their antioxidant activities and the spectral fingerprints. As a result, 12 famous green teas can be discriminated with a recognition rate of 100% by MIR or NIR data. However, compared with individual instrumental data, data fusion was more adequate for modeling the antioxidant activities of samples with RMSEP of 0.0065. Finally, the performance of the proposed method was evaluated and validated by some statistical parameters and the elliptical joint confidence region (EJCR) test. The results indicate that fusion of mid- and near-infrared spectroscopy suggests a new avenue to discriminate the species and grades of green teas. Moreover, the proposed method also implies other promising applications with more accurate multivariate calibration of antioxidant activities.

Development of a novel wavelength selection method for the trace determination of chlorpyrifos on Au@Ag NPs substrate coupled surface-enhanced Raman spectroscopy

A novel wavelength selection method named ICPA-mRMR coupled SERS was employed for the detection of CPS residues in tea samples.

A rapid quantification method for the screening indicator for β-thalassemia with near-infrared spectroscopy

Near-infrared (NIR) spectroscopy combined with chemometrics was applied to rapidly analyse haemoglobin A2 (HbA₂) for β-thalassemia screening in human haemolysate samples. The relative content indicator HbA₂ was indirectly quantified by simultaneous analysis of two absolute content indicators (Hb and Hb∙HbA₂). According to the comprehensive prediction effect of the multiple partitioning of calibration and prediction sets, the parameters were optimized to achieve modelling stability, and the preferred models were validated using the samples not involved in modelling. Savitzky-Golay smoothing was firstly used for the spectral pretreatment. The absorbance optimization partial least squares (AO-PLS) was used to eliminate high-absorption wave-bands appropriately. The equidistant combination PLS (EC-PLS) was further used to optimize wavelength models. The selected optimal models were I=856nm, N=16, G=1 and F=6 for Hb and I=988nm, N=12, G=2 and F=5 for Hb∙HbA₂. Through independent validation, the root-mean-square errors and correlation coefficients for prediction (RMSEP, R_P) were 3.50gL^-1 and 0.977 for Hb and 0.38gL^-1 and 0.917 for Hb∙HbA₂, respectively. The predicted values of relative percentage HbA₂ were further calculated, and the calculated RMSEP and R_P were 0.31% and 0.965, respectively. The sensitivity and specificity for β-thalassemia both reached 100%. Therefore, the prediction of HbA₂ achieved high accuracy for distinguishing β-thalassemia. The local optimal models for single parameter and the optimal equivalent model sets were proposed, providing more models to match possible constraints in practical applications. The NIR analysis method for the screening indicator of β-thalassemia was successfully established. The proposed method was rapid, simple and promising for thalassemia screening in a large population.

Application of variable selection in the origin discrimination of Wolfiporia cocos (F.A. Wolf) Ryvarden & Gilb. based on near infrared spectroscopy

Dried sclerotium of Wolfiporia cocos (F.A. Wolf) Ryvarden & Gilb. is a traditional Chinese medicine. Its chemical components showed difference among geographical origins, which made it difficult to keep therapeutic potency consistent. The identification of the geographical origin of W. cocos is the fundamental prerequisite for its worldwide recognition and acceptance. Four variable selection methods were employed for near infrared spectroscopy (NIR) variable selection and the characteristic variables were screened for the establishment of Fisher function models in further identification of the origin of W. cocos from Yunnan, China. For the obvious differences between poriae cutis (fu-ling-pi in Chinese, or FLP) and the inner part (bai-fu-ling in Chinese, or BFL) of the sclerotia of W. cocos in the pattern space of principal component analysis (PCA), we established discriminant models for FLP and BFL separately. Through variable selection, the models were significant improved and also the models were simplified by using only a small part of the variables. The characteristic variables were screened (13 for BFL and 10 for FLP) to build Fisher discriminant function models and the validation results showed the models were reliable and effective. Additionally, the characteristic variables were interpreted.

Boosting the Performance of Genetic Algorithms for Variable Selection in Partial Least Squares Spectral Calibrations

A genetic algorithm (GA) for variable selection in partial least squares (PLS) regression that incorporates adaptive boosting to identify informative wavelengths in near-infrared (NIR) spectra has been developed. Three studies demonstrating the advantages of incorporating an adaptive boosting routine into a GA that employs the root mean square error of calibration as its fitness function are highlighted: (1) prediction of hydroxyl number of terpolymers from NIR diffuse reflectance spectra; (2) calibration of acetone from NIR transmission spectra of mixtures of water, acetone, t-butyl alcohol and isopropyl alcohol; and (3) determination of the active pharmaceutical ingredients in drug tablets from NIR diffuse reflectance spectra. The performance of the GA with adaptive boosting to select wavelengths was compared with one without adaptive boosting. For all three NIR data sets, variable selected PLS models developed by a GA with adaptive boosting performed better. Analysis of the wavelengths selected by the GA with adaptive boosting also demonstrate that chemical information indicative of the analyte was captured by the selected wavelengths.

A comprehensive quality evaluation method by FT-NIR spectroscopy and chemometric: Fine classification and untargeted authentication against multiple frauds for Chinese Ganoderma lucidum

The origins and authenticity against frauds are two essential aspects of food quality. In this work, a comprehensive quality evaluation method by FT-NIR spectroscopy and chemometrics were suggested to address the geographical origins and authentication of Chinese Ganoderma lucidum (GL). Classification for 25 groups of GL samples (7 common species from 15 producing areas) was performed using near-infrared spectroscopy and interval-combination One-Versus-One least squares support vector machine (IC-OVO-LS-SVM). Untargeted analysis of 4 adulterants of cheaper mushrooms was performed by one-class partial least squares (OCPLS) modeling for each of the 7 GL species. After outlier diagnosis and comparing the influences of different preprocessing methods and spectral intervals on classification, IC-OVO-LS-SVM with standard normal variate (SNV) spectra obtained a total classification accuracy of 0.9317, an average sensitivity and specificity of 0.9306 and 0.9971, respectively. With SNV or second-order derivative (D2) spectra, OCPLS could detect at least 2% or more doping levels of adulterants for 5 of the 7 GL species and 5% or more doping levels for the other 2 GL species. This study demonstrates the feasibility of using new chemometrics and NIR spectroscopy for fine classification of GL geographical origins and species as well as for untargeted analysis of multiple adulterants.

High-Speed Scanning for the Quantitative Evaluation of Glycogen Concentration in Bioethanol Feedstock Synechocystis sp. PCC6803 Using a Near-Infrared Hyperspectral Imaging System with a New Near-Infrared Spectral Camera

In the present study, the high-speed quantitative evaluation of glycogen concentration accumulated in bioethanol feedstock Synechocystis sp. PCC6803 was performed using a near-infrared (NIR) imaging system with a hyperspectral NIR spectral camera named Compovision. The NIR imaging system has a feature for high-speed and wide area monitoring and the two-dimensional scanning speed is almost 100 times faster than the general NIR imaging systems for the same pixel size. For the quantitative analysis of glycogen concentration, partial least squares regression (PLSR) and moving window PLSR (MWPLSR) were performed with the information of glycogen concentration measured by high performance liquid chromatography (HPLC) and the calibration curves for the concentration within the Synechocystis sp. PCC6803 cell were constructed. The results had high accuracy for the quantitative estimation of glycogen concentration as the best squared correlation coefficient R² was bigger than 0.99 and a root mean square error (RMSE) was less than 2.9%. The present results proved not only the potential for the applicability of NIR spectroscopy to the high-speed quantitative evaluation of glycogen concentration in the bioethanol feedstock but also the expansivity of the NIR imaging instrument to in-line or on-line product evaluation on a factory production line of bioethanol in the future.

Online Monitoring of Copper Damascene Electroplating Bath by Voltammetry: Selection of Variables for Multiblock and Hierarchical Chemometric Analysis of Voltammetric Data

The Real Time Analyzer (RTA) utilizing DC- and AC-voltammetric techniques is an in situ, online monitoring system that provides a complete chemical analysis of different electrochemical deposition solutions. The RTA employs multivariate calibration when predicting concentration parameters from a multivariate data set. Although the hierarchical and multiblock Principal Component Regression- (PCR-) and Partial Least Squares- (PLS-) based methods can handle data sets even when the number of variables significantly exceeds the number of samples, it can be advantageous to reduce the number of variables to obtain improvement of the model predictions and better interpretation. This presentation focuses on the introduction of a multistep, rigorous method of data-selection-based Least Squares Regression, Simple Modeling of Class Analogy modeling power, and, as a novel application in electroanalysis, Uninformative Variable Elimination by PLS and by PCR, Variable Importance in the Projection coupled with PLS, Interval PLS, Interval PCR, and Moving Window PLS. Selection criteria of the optimum decomposition technique for the specific data are also demonstrated. The chief goal of this paper is to introduce to the community of electroanalytical chemists numerous variable selection methods which are well established in spectroscopy and can be successfully applied to voltammetric data analysis.

Multivariate Analysis of Hemicelluloses in Bleached Kraft Pulp Using Infrared Spectroscopy

The hemicellulose composition of a pulp significantly affects its chemical and physical properties and thus represents an important process control variable. However, complicated steps of sample preparation make standard methods for the carbohydrate analysis of pulp samples, such as high performance liquid chromatography (HPLC), expensive and time-consuming. In contrast, pulp analysis by attenuated total internal reflection Fourier transform infrared spectroscopy (ATR FT-IR) requires little sample preparation. Here we show that ATR FT-IR with discrete wavelet transform (DWT) and standard normal variate (SNV) spectral preprocessing offers a convenient means for the qualitative and quantitative analysis of hemicelluloses in bleached kraft pulp and alkaline treated kraft pulp. The pulp samples investigated include bleached softwood kraft pulps, bleached hardwood kraft pulps, and their mixtures, as obtained from Canadian industry mills or blended in a lab, and bleached kraft pulp samples treated with 0-6% NaOH solutions. In the principal component analysis (PCA) of these spectra, we find the potential both to differentiate all pulps on the basis of hemicellulose compositions and to distinguish bleached hardwood pulps by species. Partial least squares (PLS) multivariate analysis gives a 0.442 wt% root mean square errors of prediction (RMSEP) for the prediction of xylan content and 0.233 wt% RMSEP for the prediction of mannan content. These data all support the idea that ATR FT-IR has a great potential to rapidly and accurately predict the content of xylan and mannan for bleached kraft pulps (softwood, hardwood, and their mixtures) in industry. However, the prediction of xylan and mannan concentrations presented a difficulty for pulp samples with modified cellulose crystalline structure.

Construction of Models for Nondestructive Prediction of Ingredient Contents in Blueberries by Near-infrared Spectroscopy Based on HPLC Measurements

Nondestructive prediction of ingredient contents of farm products is useful to ship and sell the products with guaranteed qualities. Here, near-infrared spectroscopy is used to predict nondestructively total sugar, total organic acid, and total anthocyanin content in each blueberry. The technique is expected to enable the selection of only delicious blueberries from all harvested ones. The near-infrared absorption spectra of blueberries are measured with the diffuse reflectance mode at the positions not on the calyx. The ingredient contents of a blueberry determined by high-performance liquid chromatography are used to construct models to predict the ingredient contents from observed spectra. Partial least squares regression is used for the construction of the models. It is necessary to properly select the pretreatments for the observed spectra and the wavelength regions of the spectra used for analyses. Validations are necessary for the constructed models to confirm that the ingredient contents are predicted with practical accuracies. Here we present a protocol to construct and validate the models for nondestructive prediction of ingredient contents in blueberries by near-infrared spectroscopy.

A bootstrapping soft shrinkage approach for variable selection in chemical modeling

In this study, a new variable selection method called bootstrapping soft shrinkage (BOSS) method is developed. It is derived from the idea of weighted bootstrap sampling (WBS) and model population analysis (MPA). The weights of variables are determined based on the absolute values of regression coefficients. WBS is applied according to the weights to generate sub-models and MPA is used to analyze the sub-models to update weights for variables. The optimization procedure follows the rule of soft shrinkage, in which less important variables are not eliminated directly but are assigned smaller weights. The algorithm runs iteratively and terminates until the number of variables reaches one. The optimal variable set with the lowest root mean squared error of cross-validation (RMSECV) is selected. The method was tested on three groups of near infrared (NIR) spectroscopic datasets, i.e. corn datasets, diesel fuels datasets and soy datasets. Three high performing variable selection methods, i.e. Monte Carlo uninformative variable elimination (MCUVE), competitive adaptive reweighted sampling (CARS) and genetic algorithm partial least squares (GA-PLS) are used for comparison. The results show that BOSS is promising with improved prediction performance. The Matlab codes for implementing BOSS are freely available on the website: http://www.mathworks.com/matlabcentral/fileexchange/52770-boss.

On feature selection for supervised learning problems involving high-dimensional analytical information

Feature selection for supervised learning problems involving analytical information.

Baseline Correction of Diffuse Reflection Near-Infrared Spectra Using Searching Region Standard Normal Variate (SRSNV)

An alternative baseline correction method for diffuse reflection near-infrared (NIR) spectra, searching region standard normal variate (SRSNV), was proposed. Standard normal variate (SNV) is an effective pretreatment method for baseline correction of diffuse reflection NIR spectra of powder and granular samples; however, its baseline correction performance depends on the NIR region used for SNV calculation. To search for an optimal NIR region for baseline correction using SNV, SRSNV employs moving window partial least squares regression (MWPLSR), and an optimal NIR region is identified based on the root mean square error (RMSE) of cross-validation of the partial least squares regression (PLSR) models with the first latent variable (LV). The performance of SRSNV was evaluated using diffuse reflection NIR spectra of mixture samples consisting of wheat flour and granular glucose (0-100% glucose at 5% intervals). From the obtained NIR spectra of the mixture in the 10 000-4000 cm(-1) region at 4 cm intervals (1501 spectral channels), a series of spectral windows consisting of 80 spectral channels was constructed, and then SNV spectra were calculated for each spectral window. Using these SNV spectra, a series of PLSR models with the first LV for glucose concentration was built. A plot of RMSE versus the spectral window position obtained using the PLSR models revealed that the 8680–8364 cm(-1) region was optimal for baseline correction using SNV. In the SNV spectra calculated using the 8680–8364 cm(-1) region (SRSNV spectra), a remarkable relative intensity change between a band due to wheat flour at 8500 cm(-1) and that due to glucose at 8364 cm(-1) was observed owing to successful baseline correction using SNV. A PLSR model with the first LV based on the SRSNV spectra yielded a determination coefficient (R2) of 0.999 and an RMSE of 0.70%, while a PLSR model with three LVs based on SNV spectra calculated in the full spectral region gave an R2 of 0.995 and an RMSE of 2.29%. Additional evaluation of SRSNV was carried out using diffuse reflection NIR spectra of marzipan and corn samples, and PLSR models based on SRSNV spectra showed good prediction results. These evaluation results indicate that SRSNV is effective in baseline correction of diffuse reflection NIR spectra and provides regression models with good prediction accuracy.

Variable selection in multivariate calibration based on clustering of variable concept

Recently we have proposed a new variable selection algorithm, based on clustering of variable concept (CLoVA) in classification problem. With the same idea, this new concept has been applied to a regression problem and then the obtained results have been compared with conventional variable selection strategies for PLS. The basic idea behind the clustering of variable is that, the instrument channels are clustered into different clusters via clustering algorithms. Then, the spectral data of each cluster are subjected to PLS regression. Different real data sets (Cargill corn, Biscuit dough, ACE QSAR, Soy, and Tablet) have been used to evaluate the influence of the clustering of variables on the prediction performances of PLS. Almost in the all cases, the statistical parameter especially in prediction error shows the superiority of CLoVA-PLS respect to other variable selection strategies. Finally the synergy clustering of variable (sCLoVA-PLS), which is used the combination of cluster, has been proposed as an efficient and modification of CLoVA algorithm. The obtained statistical parameter indicates that variable clustering can split useful part from redundant ones, and then based on informative cluster; stable model can be reached.

A novel variable selection approach that iteratively optimizes variable space using weighted binary matrix sampling

In this study, a new optimization algorithm called the Variable Iterative Space Shrinkage Approach (VISSA) that is based on the idea of model population analysis (MPA) is proposed for variable selection. Unlike most of the existing optimization methods for variable selection, VISSA statistically evaluates the performance of variable space in each step of optimization. Weighted binary matrix sampling (WBMS) is proposed to generate sub-models that span the variable subspace. Two rules are highlighted during the optimization procedure. First, the variable space shrinks in each step. Second, the new variable space outperforms the previous one. The second rule, which is rarely satisfied in most of the existing methods, is the core of the VISSA strategy. Compared with some promising variable selection methods such as competitive adaptive reweighted sampling (CARS), Monte Carlo uninformative variable elimination (MCUVE) and iteratively retaining informative variables (IRIV), VISSA showed better prediction ability for the calibration of NIR data. In addition, VISSA is user-friendly; only a few insensitive parameters are needed, and the program terminates automatically without any additional conditions. The Matlab codes for implementing VISSA are freely available on the website: https://sourceforge.net/projects/multivariateanalysis/files/VISSA/.

A new method for wavelength interval selection that intelligently optimizes the locations, widths and combinations of the intervals

In this study, a new algorithm for wavelength interval selection, known as interval variable iterative space shrinkage approach (iVISSA), is proposed based on the VISSA algorithm. It combines global and local searches to iteratively and intelligently optimize the locations, widths and combinations of the spectral intervals. In the global search procedure, it inherits the merit of soft shrinkage from VISSA to search the locations and combinations of informative wavelengths, whereas in the local search procedure, it utilizes the information of continuity in spectroscopic data to determine the widths of wavelength intervals. The global and local search procedures are carried out alternatively to realize wavelength interval selection. This method was tested using three near infrared (NIR) datasets. Some high-performing wavelength selection methods, such as synergy interval partial least squares (siPLS), moving window partial least squares (MW-PLS), competitive adaptive reweighted sampling (CARS), genetic algorithm PLS (GA-PLS) and interval random frog (iRF), were used for comparison. The results show that the proposed method is very promising with good results both on prediction capability and stability. The MATLAB codes for implementing iVISSA are freely available on the website: .

Iteratively variable subset optimization for multivariate calibration

IVSO is a variable selection method, which shows good prediction and stability and can eliminate uninformative variables gradually and gently.

Verification of model development technique for NIR-based real-time monitoring of ingredient concentration during blending

There has been a considerable research on the process analytical technology (PAT) and real-time monitoring based on NIR, but the model development is still an important issue and persons in charge have difficulty in building good models. In this study, to realize efficient NIR-based real-time monitoring of ingredient concentration and establish a model development method, we investigated the effect of a calibration set, spectral preprocessing, wavelengths, and other factors on the prediction error through pilot and commercial scale blending experiments. The results confirmed that the small prediction error was realized by a calibration set, including dynamic measurement spectra acquired with the target blender. In addition, the results demonstrated that locally weighted partial least squares (LW-PLS) achieved the smaller prediction error than conventional PLS. The present study has also clarified that spectral preprocessing methods and wavelengths selected to build a model affect the prediction error of ingredient concentration interactively. A wide wavelength range should be selected when the spectral preprocessing does not lessen the effect of baseline variation, while a narrow wavelength range should be selected when it strongly decreases the effect.