Classification Modeling Method for Near-Infrared Spectroscopy of Tobacco Based on Multimodal Convolution Neural Networks

Artificial neural network and convolutional neural network for prediction of dental caries

Dental caries has high prevalence among kids and adults thus it has become one of the global health concerns. The current modern dentistry focused on the preventives measures to reduce the number of dental caries cases. The employment of machine learning coupled with UV spectroscopy plays a crucial role to detect the early stage of caries. Artificial neural network with hyperparameter tuning was employed to train spectral data for the classification based on the International Caries Detection and Assesment System (ICDAS). Spectra preprocessing namely mean center (MC), autoscale (AS) and Savitzky Golay smoothing (SG) were applied on the data for spectra correction. The best performance of ANN model obtained has accuracy of 0.85 with precision of 1.00. Convolutional neural network (CNN) combined with Savitzky Golay smoothing performed on the spectral data has accuracy, precision, sensitivity and specificity for validation data of 1.00 respectively. The result obtained shows that the application of ANN and CNN capable to produce robust model to be used as an early screening of dental caries.

Exploring the Steps of Infrared (IR) Spectral Analysis: Pre-Processing, (Classical) Data Modelling, and Deep Learning

Infrared (IR) spectroscopy has greatly improved the ability to study biomedical samples because IR spectroscopy measures how molecules interact with infrared light, providing a measurement of the vibrational states of the molecules. Therefore, the resulting IR spectrum provides a unique vibrational fingerprint of the sample. This characteristic makes IR spectroscopy an invaluable and versatile technology for detecting a wide variety of chemicals and is widely used in biological, chemical, and medical scenarios. These include, but are not limited to, micro-organism identification, clinical diagnosis, and explosive detection. However, IR spectroscopy is susceptible to various interfering factors such as scattering, reflection, and interference, which manifest themselves as baseline, band distortion, and intensity changes in the measured IR spectra. Combined with the absorption information of the molecules of interest, these interferences prevent direct data interpretation based on the Beer-Lambert law. Instead, more advanced data analysis approaches, particularly artificial intelligence (AI)-based algorithms, are required to remove the interfering contributions and, more importantly, to translate the spectral signals into high-level biological/chemical information. This leads to the tasks of spectral pre-processing and data modeling, the main topics of this review. In particular, we will discuss recent developments in both tasks from the perspectives of classical machine learning and deep learning.

Intelligent Eye-Controlled Electric Wheelchair Based on Estimating Visual Intentions Using One-Dimensional Convolutional Neural Network and Long Short-Term Memory

When an electric wheelchair is operated using gaze motion, eye movements such as checking the environment and observing objects are also incorrectly recognized as input operations. This phenomenon is called the "Midas touch problem", and classifying visual intentions is extremely important. In this paper, we develop a deep learning model that estimates the user's visual intention in real time and an electric wheelchair control system that combines intention estimation and the gaze dwell time method. The proposed model consists of a 1DCNN-LSTM that estimates visual intention from feature vectors of 10 variables, such as eye movement, head movement, and distance to the fixation point. The evaluation experiments classifying four types of visual intentions show that the proposed model has the highest accuracy compared to other models. In addition, the results of the driving experiments of the electric wheelchair implementing the proposed model show that the user's efforts to operate the wheelchair are reduced and that the operability of the wheelchair is improved compared to the traditional method. From these results, we concluded that visual intentions could be more accurately estimated by learning time series patterns from eye and head movement data.

Ecological risk and machine learning based source analyses of trace metals in typical surface water

Surface water is threatened by trace metal pollution due to increasing anthropogenic activities. Therefore, an appropriate source identification was essential to reduce the ecological risk posed by the given pollutants. In this study, shallow and deep learning approaches trained by a registered environmental dataset of discharge sources were employed to classify the potential emission sources of trace metals in the Elbe River, Germany. The results showed that the overall concentration rank of the given metals was Zn (226.5 ± 526.5 μg·L^-1) > Ni (5.6 ± 4.7 μg·L^-1) > Cu (5.3 ± 5.8 μg·L^-1) > As (3.3 ± 3.7 μg·L^-1) > Pb (2.9 ± 5.2 μg·L^-1) > Cr (1.8 ± 2.5 μg·L^-1) > Cd (1.3 ± 3.1 μg·L^-1) in seven tributaries and the mainstream of the Elbe River, among which the tributary Triebisch had the highest risk quotient over 86. Random Forest outperformed other algorithms with the highest Kappa median values of 0.59 and the lowest Hamming-loss values of 0.22 in extraction of the majority voted class. Then, the source apportionment conducted by random forest suggested that wastewater disposal and metal industrial emissions were the source contributors in the tributary Triebisch (probabilities: 0.39, 0.3), upstream segment (0.45, 0.25), and downstream segment (0.32, 0.23) of the given river. Additional sources of mineral industry emissions were found in the upstream segment (0.21) and downstream segment (0.22). The data provided herein suggest that random forest would be an effective approach to identify pollutants in aquatic environments and could assist source-oriented adaptive management.

Discrimination of Substandard and Falsified Formulations from Genuine Pharmaceuticals Using NIR Spectra and Machine Learning

Near-infrared (NIR) spectroscopy is a promising technique for field identification of substandard and falsified drugs because it is portable, rapid, nondestructive, and can differentiate many formulated pharmaceutical products. Portable NIR spectrometers rely heavily on chemometric analyses based on libraries of NIR spectra from authentic pharmaceutical samples. However, it is difficult to build comprehensive product libraries in many low- and middle-income countries due to the large numbers of manufacturers who supply these markets, frequent unreported changes in materials sourcing and product formulation by the manufacturers, and general lack of cooperation in providing authentic samples. In this work, we show that a simple library of lab-formulated binary mixtures of an active pharmaceutical ingredient (API) with two diluents gave good performance on field screening tasks, such as discriminating substandard and falsified formulations of the API. Six data analysis models, including principal component analysis and support-vector machine classification and regression methods and convolutional neural networks, were trained on binary mixtures of acetaminophen with either lactose or ascorbic acid. While the models all performed strongly in cross-validation (on formulations similar to their training set), they individually showed poor robustness for formulations outside the training set. However, a predictive algorithm based on the six models, trained only on binary samples, accurately predicts whether the correct amount of acetaminophen is present in ternary mixtures, genuine acetaminophen formulations, adulterated acetaminophen formulations, and falsified formulations containing substitute APIs. This data analytics approach may extend the utility of NIR spectrometers for analysis of pharmaceuticals in low-resource settings.

Tobacco shred varieties classification using Multi-Scale-X-ResNet network and machine vision

The primary task in calculating the tobacco shred blending ratio is identifying the four tobacco shred types: expanded tobacco silk, cut stem, tobacco silk, and reconstituted tobacco shred. The classification precision directly affects the subsequent determination of tobacco shred components. However, the tobacco shred types, especially expanded tobacco silk and tobacco silk, have no apparent differences in macro-scale characteristics. The tobacco shreds have small size and irregular shape characteristics, creating significant challenges in their recognition and classification based on machine vision. This study provides a complete set of solutions aimed at this problem for screening tobacco shred samples, taking images, image preprocessing, establishing datasets, and identifying types. A block threshold binarization method is used for image preprocessing. Parameter setting and method performance are researched to obtain the maximum number of complete samples with acceptable execution time. ResNet50 is used as the primary classification and recognition network structure. By increasing the multi-scale structure and optimizing the number of blocks and loss function, a new tobacco shred image classification method is proposed based on the MS-X-ResNet (Multi-Scale-X-ResNet) network. Specifically, the MS-ResNet network is obtained by fusing the multi-scale Stage 3 low-dimensional and Stage 4 high-dimensional features to reduce the overfitting risk. The number of blocks in Stages 1-4 are adjusted from the original 3:4:6:3 to 3:4:N:3 (A-ResNet) and 3:3:N:3 (B-ResNet) to obtain the X-ResNet network, which improves the model's classification performance with lower complexity. The focal loss function is selected to reduce the impact of identification difficulty for different sample types on the network and improve its performance. The experimental results show that the final classification accuracy of the network on a tobacco shred dataset is 96.56%. The image recognition of a single tobacco shred requires 103 ms, achieving high classification accuracy and efficiency. The image preprocessing and deep learning algorithms for tobacco shred classification and identification proposed in this study provide a new implementation approach for the actual production and quality detection of tobacco and a new way for online real-time type identification of other agricultural products.

Estimation of Soluble Solids for Stone Fruit Varieties Based on Near-Infrared Spectra Using Machine Learning Techniques

The quality control for fruit maturity inspection is a key issue in fruit packaging and international trade. The quantification of Soluble Solids (SS) in fruits gives a good approximation of the total sugar concentration at the ripe stage, and on the other hand, SS alone or in combination with acidity is highly related to the acceptability of the fruit by consumers. The non-destructive analysis based on Visible (VIS) and Near-Infrared (NIR) spectroscopy has become a popular technique for the assessment of fruit quality. To improve the accuracy of fruit maturity inspection, VIS−NIR spectra models based on machine learning techniques are proposed for the non-destructive evaluation of soluble solids in considering a range of variations associated with varieties of stones fruit species (peach, nectarine, and plum). In this work, we propose a novel approach based on a Convolutional Neural Network (CNN) for the classification of the fruits into species and then a Feedforward Neural Network (FNN) to extract the information of VIS−NIR spectra to estimate the SS content of the fruit associated to several varieties. A classification accuracy of 98.9% was obtained for the CNN classification model and a correlation coefficient of Rc>0.7109 for the SS estimation of the FNN models was obtained. The results reported show the potential of this method for a fast and on-line classification of fruits and estimation of SS concentration.

Identifying the Geographical Origin of Tobacco Leaf by Strontium and Lead Isotopic with Mineral Elemental Fingerprint

The primary aim of this paper was to identifying the geographical origin of tobacco leaves based on stable isotopic and mineral elemental fingerprint. We collected eighty-one tobacco leaf samples from Argentina, Brazil, Zimbabwe, the U.S., Zambia, and China. And nine mineral element contents and four strontium and lead isotope ratios of the tobacco leaves were determined by thermal ionization mass spectrometry (TIMS) and inductively coupled plasma mass spectrometry (ICP-MS). After variance and stepwise discriminant analysis, the discriminant functions of the tobacco leaf’s geographical origin were established. The results indicate that: (1) the contents of six mineral elements including Cu, Zn, Cr, Ni, Cd, and Pb, together with four strontium and lead isotope ratios containing 87Sr/86Sr, 208Pb/204Pb, 207Pb/204Pb, and 206Pb/204Pb, were significantly different among six countries. (2) Different countries presented some characteristic mineral elemental and isotopic fingerprint. The even contents of mineral elements from Zambian tobacco leaf were much lower than the other countries, among which four elements consisting of Zn, Cr, As, and Cd were not detected. The three average lead isotope ratios including 208Pb/204Pb, 207Pb/204Pb, and 206Pb/204Pb from Zimbabwe tobacco leaves were far higher than the other countries, and the range of which was unoverlapped. (3) The effective identification of the geographical origin of tobacco leaf was accomplished by Fisher stepwise discriminant analysis and the characteristic tracing elements consisted of Cu, Zn, Cr, Ni, Cd, Pb, 87Sr/86Sr, 207Pb/204Pb, and 206Pb/204Pb. Based on the established discriminant functions, the original and cross-validation accuracy towards different geographical origins of tobacco leaves were 98.8% and 95.1%, respectively. The study shows that the strontium and lead isotopic with mineral elemental fingerprints is a potential effective method to identify the geographical origin of tobacco leaves from different countries.

A Neural Network-Based Spectral Approach for the Assignment of Individual Trees to Genetically Differentiated Subpopulations

Studying population structure has made an essential contribution to understanding evolutionary processes and demographic history in forest ecology research. This inference process basically involves the identification of common genetic variants among individuals, then grouping the similar individuals into subpopulations. In this study, a spectral-based classification of genetically differentiated groups was carried out using a provenance–progeny trial of Eucalyptus cladocalyx. First, the genetic structure was inferred through a Bayesian analysis using single-nucleotide polymorphisms (SNPs). Then, different machine learning models were trained with foliar spectral information to assign individual trees to subpopulations. The results revealed that spectral-based classification using the multilayer perceptron method was very successful at classifying individuals into their respective subpopulations (with an average of 87% of correct individual assignments), whereas 85% and 81% of individuals were assigned to their respective classes correctly by convolutional neural network and partial least squares discriminant analysis, respectively. Notably, 93% of individual trees were assigned correctly to the class with the smallest size using the spectral data-based multi-layer perceptron classification method. In conclusion, spectral data, along with neural network models, are able to discriminate and assign individuals to a given subpopulation, which could facilitate the implementation and application of population structure studies on a large scale.

Diagnosis of dengue virus infection using spectroscopic images and deep learning

Dengue virus (DENV) infection is one of the major health issues and a substantial epidemic infectious human disease. More than two billion humans are living in dengue susceptible regions with annual infection mortality rate is about 5%-20%. At initial stages, it is difficult to differentiate dengue virus symptoms with other similar diseases. The main objective of this research is to diagnose dengue virus infection in human blood sera for better treatment and rehabilitation process. A novel and robust approach is proposed based on Raman spectroscopy and deep learning. In this regard, the ResNet101 deep learning model is modified by exploiting transfer learning (TL) concept on Raman spectroscopic data of human blood sera. Sample size was selected using standard statistical tests. The proposed model is evaluated on 2,000 Raman spectra images in which 1,200 are DENV-infected of human blood sera samples, and 800 are healthy ones. It offers 96.0% accuracy on testing data for DENV infection diagnosis. Moreover, the developed approach demonstrated minimum improvement of 6.0% and 7.0% in terms of AUC and Kappa index respectively over the other state-of-the-art techniques. The developed model offers superior performance to capture minute Raman spectral variations due to the better residual learning capability and generalization ability compared to others deep learning models. The developed model revealed that it might be applied for diagnosis of DENV infection to save precious human lives.

A Long Short-Term Memory Neural Network Based Simultaneous Quantitative Analysis of Multiple Tobacco Chemical Components by Near-Infrared Hyperspectroscopy Images

Near-infrared (NIR) spectroscopy has been widely used in agricultural operations to obtain various crop parameters, such as water content, sugar content, and different indicators of ripeness, as well as other potential information concerning crops that cannot be directly obtained by human observation. The chemical compositions of tobacco play an important role in the quality of cigarettes. The NIR spectroscopy-based chemical composition analysis has recently become one of the most effective methods in tobacco quality analysis. Existing NIR spectroscopy-related solutions either have relatively low analysis accuracy, or are only able to analyze one or two chemical components. Thus, a precise prediction model is needed to improve the analysis accuracy of NIR data. This paper proposes a tobacco chemical component analysis method based on a neural network (TCCANN) to quantitatively analyze the chemical components of tobacco leaves by using NIR spectroscopy, including nicotine, total sugar, reducing sugar, total nitrogen, potassium, chlorine, and pH value. The proposed TCCANN consists of both residual network (ResNet) and long short-term memory (LSTM) neural network. ResNet is applied to the feature extraction of high-dimension NIR spectroscopy, which can effectively avoid the gradient-disappearance issue caused by the increase of network depth. LSTM is used to quantitatively analyze the multiple chemical compositions of tobacco leaves in a simultaneous manner. LSTM selectively allows information to pass through by a gated unit, thereby comprehensively analyzing the correlation between multiple chemical components and corresponding spectroscopy. The experimental results confirm that the proposed TCCANN not only predicts the corresponding values of seven chemical components simultaneously, but also achieves better prediction performance than other existing machine learning methods.

Classification modeling method for hyperspectral stamp-pad ink data based on one-dimensional convolutional neural network

In the questioned document, the examination of stamp-pad ink is crucial scientific evidence to discern the difference between genuine and forged documents. In this study, a new method for rapid and non-destructive identification of types of stamp-pad inks by combining hyperspectral imaging (HSI) technology and deep learning was developed. Twenty stamp-pad inks of different brands and models were collected and numbered in turn, and then, each of them was sealed six times repeatedly on the A4 printing paper for the test. After that, the hyperspectral imager was used to collect the hyperspectral images and the reflectance spectral data were obtained after pixel fusion. Principal component analysis (PCA) and non-negative matrix factorization (NMF) were used to deal with the dataset, but visual results were not good. Then, back propagation neural network (BPNN) and one-dimensional convolutional neural network (1D-CNN) were constructed and their merits and drawbacks were compared. The final loss function of the BPNN of training set and validation set was stable at 0.27 and 0.42, and the classification accuracy of the training set and validation set reached 90.02% and 83.99%, respectively. Compared with the BPNN, the 1D-CNN had better stability and efficiency for the classification. The loss function of the training set and validation set was as low as 0.068 and 0.075, and the final classification accuracy reached 98.30% and 97.94%, respectively. Therefore, the combination of hyperspectral imaging technology and 1D-CNN represents a potentially simple, non-destructive, and rapid method for stamp-pad inks detection and classification.

Determination of the Geographical Origin of Coffee Beans Using Terahertz Spectroscopy Combined With Machine Learning Methods

Different geographical origins can lead to great variance in coffee quality, taste, and commercial value. Hence, controlling the authenticity of the origin of coffee beans is of great importance for producers and consumers worldwide. In this study, terahertz (THz) spectroscopy, combined with machine learning methods, was investigated as a fast and non-destructive method to classify the geographic origin of coffee beans, comparing it with the popular machine learning methods, including convolutional neural network (CNN), linear discriminant analysis (LDA), and support vector machine (SVM) to obtain the best model. The curse of dimensionality will cause some classification methods which are struggling to train effective models. Thus, principal component analysis (PCA) and genetic algorithm (GA) were applied for LDA and SVM to create a smaller set of features. The first nine principal components (PCs) with an accumulative contribution rate of 99.9% extracted by PCA and 21 variables selected by GA were the inputs of LDA and SVM models. The results demonstrate that the excellent classification (accuracy was 90% in a prediction set) could be achieved using a CNN method. The results also indicate variable selecting as an important step to create an accurate and robust discrimination model. The performances of LDA and SVM algorithms could be improved with spectral features extracted by PCA and GA. The GA-SVM has achieved 75% accuracy in a prediction set, while the SVM and PCA-SVM have achieved 50 and 65% accuracy, respectively. These results demonstrate that THz spectroscopy, together with machine learning methods, is an effective and satisfactory approach for classifying geographical origins of coffee beans, suggesting the techniques to tap the potential application of deep learning in the authenticity of agricultural products while expanding the application of THz spectroscopy.

A Review of the Discriminant Analysis Methods for Food Quality Based on Near-Infrared Spectroscopy and Pattern Recognition

Near-infrared spectroscopy (NIRS) combined with pattern recognition technique has become an important type of non-destructive discriminant method. This review first introduces the basic structure of the qualitative analysis process based on near-infrared spectroscopy. Then, the main pretreatment methods of NIRS data processing are investigated. Principles and recent developments of traditional pattern recognition methods based on NIRS are introduced, including some shallow learning machines and clustering analysis methods. Moreover, the newly developed deep learning methods and their applications of food quality analysis are surveyed, including convolutional neural network (CNN), one-dimensional CNN, and two-dimensional CNN. Finally, several applications of these pattern recognition techniques based on NIRS are compared. The deficiencies of the existing pattern recognition methods and future research directions are also reviewed.

Discrimination of Fresh Tobacco Leaves with Different Maturity Levels by Near-Infrared (NIR) Spectroscopy and Deep Learning

The maturity affects the yield, quality, and economic value of tobacco leaves. Leaf maturity level discrimination is an important step in manual harvesting. However, the maturity judgment of fresh tobacco leaves by grower visual evaluation is subjective, which may lead to quality loss and low prices. Therefore, an objective and reliable discriminant technique for tobacco leaf maturity level based on near-infrared (NIR) spectroscopy combined with a deep learning approach of convolutional neural networks (CNNs) is proposed in this study. To assess the performance of the proposed maturity discriminant model, four conventional multiclass classification approaches-K-nearest neighbor (KNN), backpropagation neural network (BPNN), support vector machine (SVM), and extreme learning machine (ELM)-were employed for a comparative analysis of three categories (upper, middle, and lower position) of tobacco leaves. Experimental results showed that the CNN discriminant models were able to precisely classify the maturity level of tobacco leaves for the above three data sets with accuracies of 96.18%, 95.2%, and 97.31%, respectively. Moreover, the CNN models with strong feature extraction and learning ability were superior to the KNN, BPNN, SVM, and ELM models. Thus, NIR spectroscopy combined with CNN is a promising alternative to overcome the limitations of sensory assessment for tobacco leaf maturity level recognition. The development of a maturity-distinguishing model can provide an accurate, reliable, and scientific auxiliary means for tobacco leaf harvesting.

Pre-dispersive near-infrared light sensing in non-destructively classifying the brix of intact pineapples

Exported fresh intact pineapples must fulfill the minimum internal quality requirement of 12 degree brix. Even though near-infrared (NIR) spectroscopic approaches are promising to non-destructively and rapidly assess the internal quality of intact pineapples, these approaches involve expensive and complex NIR spectroscopic instrumentation. Thus, this research evaluates the performance of a proposed pre-dispersive NIR light sensing approach in non-destructively classifying the Brix of pineapples using K-fold cross-validation, holdout validation, and sensitive analysis. First, the proposed pre-dispersive NIR sensing device that consisted of a light sensing element and five NIR light emitting diodes with peak wavelengths of 780, 850, 870, 910, and 940 nm, respectively, was developed. After that, the diffuse reflectance NIR light of intact pineapples was non-destructively acquired using the developed NIR sensing device before their Brix values were conventionally measured using a digital refractometer. Next, an artificial neural network (ANN) was trained and optimized to classify the Brix values of pineapples using the acquired NIR light. The results of the sensitivity analysis showed that either one wavelength that was near to the water absorbance or chlorophyll band was redundant in the classification. The performance of the trained ANN was tested using new pineapples with the optimal classification accuracy of 80.56%. This indicates that the proposed pre-dispersive NIR light sensing approach coupled with the ANN is promising to be an alternative to non-destructively classifying the internal quality of fruits.