Wavelength interval selection in multicomponent spectral analysis by moving window partial least-squares regression with applications to mid-infrared and near-infrared spectroscopic data

Simultaneous determination of traces amounts of cadmium, zinc, and cobalt based on UV-Vis spectrometry combined with wavelength selection and partial least squares regression

The use of wavelength selection before partial least squares regression (PLSR) for simultaneous determination of divalent metal ions, cadmium, zinc and cobalt by UV-Vis spectrometry was investigated in this paper. The number of wavelengths selected by competitive adaptive reweighted sampling (CARS) for cadmium, zinc, and cobalt were 21, 13 and 7, respectively, from the 916 original wavelength points. The analytical system was based on the formation of the complexes with 2-(5-bromo-2-pyridylazo)-5-(diethylamino)phenol (Br-PADAP) in surfactant media. Compared with the results of full spectra calibration, the root mean squared error of prediction (RMSEP) reduced to 0.0110, 0.0098 and 0.0031 for cadmium, zinc and cobalt, respectively. Moreover, by using the selective wavelengths instead of the 916 original wavelengths, the latent variables of PLS models reduced to 3, 3 and 4. The results indicated that the PLS model established by selected wavelength could be used for simultaneous determination of divalent metal ions.

Selection method of quasi-continuous wavelength combination with applications to the near-infrared spectroscopic analysis of soil organic matter

Equidistant combination multiple linear regression (EC-MLR) for the quasi-continuous wavelength selection of spectroscopic analysis was proposed and successfully applied to the reagent-free determination of soil organic matter with near-infrared spectroscopy. For comparison, the continuous-mode moving window partial least squares (MWPLS) and the discrete-mode successive projections algorithm (SPA) were improved by considering the stability and applied to the same analysis object as well. All methods exhibited good effect, but the modeling accuracy, stability, and validation effect of EC-MLR were better than that of the other two methods. Compared with MWPLS, the optimal EC-MLR model contained only 16 wavelengths, and method complexity was substantially reduced. Compared with SPA-MLR, the optimal EC-MLR model could easily undergo spectral preprocessing to improve predictive capability. Moreover, appropriate equidistant discrete wavelength combination with EC-MLR corresponded to the spectral absorption band with proper resolution and can effectively overcome co-linearity interruption for the MLR model. Thus, the EC-MLR method has great potential in practical application and instrument design.

Comprehensive, quantitative bioprocess productivity monitoring using fluorescence EEM spectroscopy and chemometrics

Using fluorescence excitation-emission matrix spectroscopy and chemometric methods we demonstrate an effective and rapid method for quantitative monitoring of a mammalian cell culture based manufacturing process.

On-line application of near-infrared spectroscopy for monitoring water levels in parts per million in a manufacturing-scale distillation process

An on-line analytical method based on transmission near-infrared spectroscopy (NIRS) for the quantitative determination of water concentrations (in parts per million) was developed and applied to the manufacture of a pharmaceutical intermediate. Calibration models for water analysis, built at the development site and applied at the manufacturing site, were successfully demonstrated during six manufacturing runs at a 250-gallon scale. The water measurements will be used as a forward-processing control point following distillation of a toluene product solution prior to use in a Grignard reaction. The most significant impact of using this NIRS-based process analytical technology (PAT) to replace off-line measurements is the significant reduction in the risk of operator exposure through the elimination of sampling of a severely lachrymatory and mutagenic compound. The work described in this report illustrates the development effort from proof-of-concept phase to manufacturing implementation.

An efficient method of wavelength interval selection based on random frog for multivariate spectral calibration

Wavelength selection is a critical step for producing better prediction performance when applied to spectral data. Considering the fact that the vibrational and rotational spectra have continuous features of spectral bands, we propose a novel method of wavelength interval selection based on random frog, called interval random frog (iRF). To obtain all the possible continuous intervals, spectra are first divided into intervals by moving window of a fix width over the whole spectra. These overlapping intervals are ranked applying random frog coupled with PLS and the optimal ones are chosen. This method has been applied to two near-infrared spectral datasets displaying higher efficiency in wavelength interval selection than others. The source code of iRF can be freely downloaded for academy research at the website: http://code.google.com/p/multivariate-calibration/downloads/list.

A multi-model fusion strategy for multivariate calibration using near and mid-infrared spectra of samples from brewing industry

Near and mid-infrared (NIR/MIR) spectroscopy techniques have gained great acceptance in the industry due to their multiple applications and versatility. However, a success of application often depends heavily on the construction of accurate and stable calibration models. For this purpose, a simple multi-model fusion strategy is proposed. It is actually the combination of Kohonen self-organizing map (KSOM), mutual information (MI) and partial least squares (PLSs) and therefore named as KMICPLS. It works as follows: First, the original training set is fed into a KSOM for unsupervised clustering of samples, on which a series of training subsets are constructed. Thereafter, on each of the training subsets, a MI spectrum is calculated and only the variables with higher MI values than the mean value are retained, based on which a candidate PLS model is constructed. Finally, a fixed number of PLS models are selected to produce a consensus model. Two NIR/MIR spectral datasets from brewing industry are used for experiments. The results confirms its superior performance to two reference algorithms, i.e., the conventional PLS and genetic algorithm-PLS (GAPLS). It can build more accurate and stable calibration models without increasing the complexity, and can be generalized to other NIR/MIR applications.

Determination of amino acid nitrogen in tuber mustard using near-infrared spectroscopy with waveband selection stability

Near-infrared (NIR) spectroscopy was successfully applied to the rapid chemical-free determination of amino acid nitrogen (AAN) in tuber mustard. Moving window partial least squares, combined with Savitzky-Golay smoothing, was used for the waveband selection. Based on the various divisions in the calibration and prediction sets, an effective modeling approach with good stability was proposed. The results confirmed that the long-wave NIR region contains sufficient information for the quantification of AAN in tuber mustard. An appropriate waveband (5874-4,258 cm(-1)) located in the combination region of NIR was selected. The validation root mean square error of prediction and the validation correlation coefficients of prediction were 0.380 mg/g and 0.962, respectively, both indicating good validation effect and stability. The results provided a reliable NIR model and can serve as valuable references for designing spectroscopic instruments for quality evaluation of tuber mustard.

Improvement of spectral calibration for food analysis through multi-model fusion

Near-infrared (NIR) spectroscopy will present a more promising tool for quantitative analysis if the predictive ability of the calibration model is further improved. To achieve this goal, a new ensemble calibration method based on uninformative variable elimination (UVE)-partial least square (PLS) is proposed, which is named as ensemble PLS (EPLS), meaning a fusion of multiple PLS models. In this method, different calibration sets are first generated by bootstrap and different PLS models are obtained. Then, the UVE is used to shrink the original variable space into a specific subspace. By repeating this process, a fixed number of candidates PLS member models are obtained. Finally, a smaller part of candidate models are integrated to produce an ensemble model. In order to verify the performance of EPLS, three NIR spectral datasets from food industry were used for illustration. Both full-spectrum PLS and UVEPLS of single models were used as reference. It was found that the proposed method could lead to lower RMSEP (root mean square error of prediction) value than PLS and UVEPLS and such an improvement is statistically significant according to a paired t-test. The results showed that the method is of value to enhance the predictive ability of PLS-based calibration involving complex NIR matrices in food analysis.

Wavelength selection-based nonlinear calibration for transcutaneous blood glucose sensing using Raman spectroscopy

While Raman spectroscopy provides a powerful tool for noninvasive and real time diagnostics of biological samples, its translation to the clinical setting has been impeded by the lack of robustness of spectroscopic calibration models and the size and cumbersome nature of conventional laboratory Raman systems. Linear multivariate calibration models employing full spectrum analysis are often misled by spurious correlations, such as system drift and covariations among constituents. In addition, such calibration schemes are prone to overfitting, especially in the presence of external interferences that may create nonlinearities in the spectra-concentration relationship. To address both of these issues we incorporate residue error plot-based wavelength selection and nonlinear support vector regression (SVR). Wavelength selection is used to eliminate uninformative regions of the spectrum, while SVR is used to model the curved effects such as those created by tissue turbidity and temperature fluctuations. Using glucose detection in tissue phantoms as a representative example, we show that even a substantial reduction in the number of wavelengths analyzed using SVR lead to calibration models of equivalent prediction accuracy as linear full spectrum analysis. Further, with clinical datasets obtained from human subject studies, we also demonstrate the prospective applicability of the selected wavelength subsets without sacrificing prediction accuracy, which has extensive implications for calibration maintenance and transfer. Additionally, such wavelength selection could substantially reduce the collection time of serial Raman acquisition systems. Given the reduced footprint of serial Raman systems in relation to conventional dispersive Raman spectrometers, we anticipate that the incorporation of wavelength selection in such hardware designs will enhance the possibility of miniaturized clinical systems for disease diagnosis in the near future.

Waveband Selection for NIR Spectroscopy Analysis of Wastewater COD

Waveband selection of near-infrared (NIR) spectroscopy analysis of wastewater chemical oxygen demand (COD) by moving window partial least squares (MWPLS) method with changeable size, the optimization of PLS factor was combined with MWPLS method. According to the prediction effect, the optimal model was selected, and the corresponding waveband, number of adopted wavelengths, PLS factor, RMSEP, RP were 820-850nm, 16, 13, 25.5 mg/L, and 0.968 respectively, which was obviously superior to the optimal PLS model on the whole spectral collecting region. The result shows that the MWPLS method can improve the model prediction effect, reduce model complexity, and provide valuable reference for designing special NIR spectrometer.

Preliminary study on the application of near infrared spectroscopy and pattern recognition methods to classify different types of apple samples

In this paper, near infrared (NIR) spectroscopy combined with pattern recognition methods was used in an attempt to classify different types of apple samples. Three pattern recognition methods such as K-nearest neighbour (KNN), partial least-squares discriminant analysis (PLSDA) and moving window partial least-squares discriminant analysis (MWPLSDA) were used to classify apple samples of different geographical origins, grades and varieties. The result indicates that MWPLSDA is superior to these two conventional pattern recognition methods. Because MWPLSDA method can select narrow but informative wavelength intervals to reconstruct an efficacious classification model with high predicting accuracy. In conclusion, MWPLSDA coupled with near-infrared fibre-optic technology is proved to be an effective method for fruit classification.

An ensemble method based on uninformative variable elimination and mutual information for spectral multivariate calibration

Based on the combination of uninformative variable elimination (UVE), bootstrap and mutual information (MI), a simple ensemble algorithm, named ESPLS, is proposed for spectral multivariate calibration (MVC). In ESPLS, those uninformative variables are first removed; and then a preparatory training set is produced by bootstrap, on which a MI spectrum of retained variables is calculated. The variables that exhibit higher MI than a defined threshold form a subspace on which a candidate partial least-squares (PLS) model is constructed. This process is repeated. After a number of candidate models are obtained, a small part of models is picked out to construct an ensemble model by simple/weighted average. Four near/mid-infrared (NIR/MIR) spectral datasets concerning the determination of six components are used to verify the proposed ESPLS. The results indicate that ESPLS is superior to UVEPLS and its combination with MI-based variable selection (SPLS) in terms of both the accuracy and robustness. Besides, from the perspective of end-users, ESPLS does not increase the complexity of a calibration when enhancing its performance.

Spectral multivariate calibration with wavelength selection using variants of Tikhonov regularization

Tikhonov regularization (TR) is a general method that can be used to form a multivariate calibration model and numerous variants of it exist, including ridge regression (RR). This paper reports on the unique flexibility of TR to form a model using full wavelengths (RR), individually selected wavelengths, or multiple bands of selected wavelengths. Of these three TR variants, the one based on selection of wavelength bands is found to produce lower prediction errors. As with most wavelength selection algorithms, the model vector magnitude indicates that this error reduction comes with a potential increase in prediction uncertainty. Results are presented for near-infrared, ultraviolet-visible, and synthetic spectral data sets. While the focus of this paper is wavelength selection, the TR methods are generic and applicable to other variable-selection situations.

Construction of an efficacious model for a nondestructive identification of traditional Chinese medicines Liuwei Dihuang pills from different manufacturers using near-infrared spectroscopy and moving window partial least-squares discriminant analysis

This paper reports on the construction of an efficacious model for a non-invasive identification of traditional Chinese medicines, Liuwei Dihuang pills from different manufacturers, on the basis of near-infrared spectra (NIRS) coupled with moving window partial least-squares discriminant analysis (MWPLSDA). Considering the continuity of near-infrared spectral measurements, MWPLSDA is used to identify continuous and highly classification-related information intervals, a simple, yet effective classification model that can be developed for identifying accurate 150 Liuwei Dihuang pills from five different manufacturers. Meanwhile, the method is compared with some traditional pattern-recognition methods including principal component analysis (PCA), linear discriminant analysis (LDA) and partial least-squares discriminant analysis (PLSDA). The obtained results show that the method not only can reduce the operation time, but also significantly improves the classification accuracy. Hence, the nondestructive method can be expected to be promising for more practical applications on quality control and the discrimination of traditional Chinese medicine.

Net analyte signal-based simultaneous determination of dyes in environmental samples using moving window partial least squares regression with UV-vis spectroscopy

The multivariate calibration methods-moving window selection partial least squares regression (MWPLSR) and net analyte signal (NAS)-were employed for simultaneous determination of a mixture of C.I. Disperse Blue 183, C.I. Disperse Blue 79, C.I. Disperse Red 82, C.I. Disperse Red 65, C.I. Disperse Yellow 211 and C.I. Disperse Orange 25 by UV-vis spectrophotometry. The absorption spectra of the six disperse dyes were recorded between 320 and 680 nm. A modified changeable size moving window partial least squares (CSMWPLS) and searching combination moving window partial least squares (SCMWPLS) were proposed to search for an optimized spectral interval and an optimized combination of spectral regions from informative regions obtained by MWPLSR. Different wavelength regions were selected by taking into account different spectral parameters including the starting wavelength, the ending wavelength and wavelength interval. It was found that wavelength selection improved the performance of the corresponding net analyte signal-partial least squares (NAS-PLS) model, in terms of root mean square error (RMSE), compared with the results obtained using whole spectra or direct combination of informative regions for each dye. The importance of calibration design was also investigated by calculating the prediction and validation errors. The influence of using independent validation sets were emphasized. The proposed calibration method gave better results in combination and informative spectral regions for determination of the six disperse dyes without prior separation.

An Improved Ensemble Method for Completely Automatic Optimization of Spectral Interval Selection in Multivariate Calibration

In our recent work, Monte Carlo Cross Validation Stacked Regression (MCCVSR) is proposed to achieve automatic optimization of spectral interval selection in multivariate calibration. Though MCCVSR performs well in normal conditions, it is still necessary to improve it for more general applications. According to the well-known principle of "garbage in, garbage out (GIGO)", as a precise ensemble method, MCCVSR might be influenced by outlying and very bad submodels. In this paper, a statistical test is designed to exclude the ruinous submodels from the ensemble learning process, therefore, the combination process becomes more reliable. Though completely automated, the proposed method is adjustable according to the nature of the data analyzed, including the size of training samples, resolution of spectra and quantitative potentials of the submodels. The effectiveness of the submodel refining is demonstrated by the investigation of a real standard data.

Mutual information-induced interval selection combined with kernel partial least squares for near-infrared spectral calibration

With the aim of developing a nonlinear tool for near-infrared spectral (NIRS) calibration, an applicable algorithm, called MIKPLS, is designed based on the combination of two different strategies, i.e. mutual information (MI) for interval selection and kernel partial least squares (KPLS) for modeling. Due to the ability of capturing linear and nonlinear dependencies between variables simultaneously, mutual information between each candidate variables and target is calculated and employed to induce a continuous wavelength interval, which is subsequently applied to build a parsimonious calibration model for future use by kernel partial least squares. Through the experiments on two datasets, it seems that mutual information (MI)-induced interval selection, followed by KPLS, forms a very simple and practical tool, allowing a prediction model to be constructed using a much-reduced set of neighboring variables, but without any loss of generalizations and with improved prediction performance instead.