An approximation of the Gaussian RBF kernel for efficient classification with SVMs

Variable Projection Support Vector Machines and Some Applications Using Adaptive Hermite Expansions

In this paper, we develop the so-called variable projection support vector machine (VP-SVM) algorithm that is a generalization of the classical SVM. In fact, the VP block serves as an automatic feature extractor to the SVM, which are trained simultaneously. We consider the primal form of the arising optimization task and investigate the use of nonlinear kernels. We show that by choosing the so-called adaptive Hermite function system as the basis of the orthogonal projections in our classification scheme, several real-world signal processing problems can be successfully solved. In particular, we test the effectiveness of our method in two case studies corresponding to anomaly detection. First, we consider the detection of abnormal peaks in accelerometer data caused by sensor malfunction. Then, we show that the proposed classification algorithm can be used to detect abnormalities in ECG data. Our experiments show that the proposed method produces comparable results to the state-of-the-art while retaining desired properties of SVM classification such as light weight architecture and interpretability. We implement the proposed method on a microcontroller and demonstrate its ability to be used for real-time applications. To further minimize computational cost, discrete orthogonal adaptive Hermite functions are introduced for the first time.

Multi-band spatial feature extraction and classification for motor imaging EEG signals based on OSFBCSP-GAO-SVM model : EEG signal processing

Electroencephalogram (EEG) is a non-stationary random signal with strong background noise, which makes its feature extraction difficult and recognition rate low. This paper presents a feature extraction and classification model of motor imagery EEG signals based on wavelet threshold denoising. Firstly, this paper uses the improved wavelet threshold algorithm to obtain the denoised EEG signal, divides all EEG channel data into multiple partially overlapping frequency bands, and uses the common spatial pattern (CSP) method to construct multiple spatial filters to extract the characteristics of EEG signals. Secondly, EEG signal classification and recognition are realized by the support vector machine algorithm optimized by a genetic algorithm. Finally, the dataset of the third brain-computer interface (BCI) competition and the dataset of the fourth BCI competition is selected to verify the classification effect of the algorithm. The highest accuracy of this method for two BCI competition datasets is 92.86% and 87.16%, respectively, which is obviously superior to the traditional algorithm model. The accuracy of EEG feature classification is improved. It shows that an overlapping sub-band filter banks common spatial pattern-genetic algorithms optimization-support vector machines (OSFBCSP-GAO-SVM) model is an effective model for feature extraction and classification of motor imagination EEG signals.

Application of the Gaussian Process Regression Method Based on a Combined Kernel Function in Engine Performance Prediction

At present, regression modeling methods fail to achieve higher simulation accuracy, which limits the application of simulation technology in more fields such as virtual calibration and hardware-in-the-loop real-time simulation in automotive industry. After fully considering the abruptness and complexity of engine predictions, a Gaussian process regression modeling method based on a combined kernel function is proposed and verified in this study for engine torque, emission, and temperature predictions. The comparison results with linear regression, decision tree, support vector machine (abbreviated as SVM), neural network, and other Gaussian regression methods show that the Gaussian regression method based on the combined kernel function proposed in this study can achieve higher prediction accuracy. Fitting results show that the R ² value of engine torque and exhaust gas temperature after the engine turbo (abbreviated as T4) prediction model reaches 1.00, and the R ² value of the nitrogen oxide (abbreviated as NOx) prediction model reaches 0.9999. The model generalization ability verification test results show that for a totally new world harmonized transient cycle data, the R ² value of engine torque prediction is 0.9993, the R ² value of exhaust gas temperature is 0.995, and the R ² value of NOx emission prediction result is 0.9962. The results of model generalization ability verification show that the model can achieve high prediction accuracy for performance prediction, temperature prediction, and emission prediction under steady-state and transient operating conditions.

A new prediction method of industrial atmospheric pollutant emission intensity based on pollutant emission standard quantification

Industrial emissions are the main source of atmospheric pollutants in China. Accurate and reasonable prediction of the emission of atmospheric pollutants from single enterprise can determine the exact source of atmospheric pollutants and control atmospheric pollution precisely. Based on China's coking enterprises in 2020, we proposed a quantitative method for pollutant emission standards and introduced the quantification results of pollutant emission standards (QRPES) into the construction of support vector regression (SVR) and random forest regression (RFR) prediction methods for SO₂ emission of coking enterprises in China. The results show that, affected by the types of coke ovens and regions, China's current coking enterprises have implemented a total of 21 emission standards, with marked differences. After adding QRPES, it was found that the root mean squared error (RMSE) of SVR and RFR decreased from 0.055 kt/a and 0.059 kt/a to 0.045 kt/a and 0.039 kt/a, and the R ² increased from 0.890 and 0.881 to 0.926 and 0.945, respectively. This shows that the QRPES can greatly improve the prediction accuracy, and the SO₂ emissions of each enterprise are highly correlated with the strictness of standards. The predicted result shows that 45% of SO₂ emissions from Chinese coking enterprises are concentrated in Shanxi, Shaanxi and Hebei provinces in central China. The method created in this paper fills in the blank of forecasting method of air pollutant emission intensity of single enterprise and is of great help to the accurate control of air pollutants.

ELECTRONIC SUPPLEMENTARY MATERIAL

Supplementary material is available in the online version of this article at 10.1007/s11783-023-1608-1 and is accessible for authorized users.

Monitoring the Spatiotemporal Trajectory of Urban Area Hotspots Using the SVM Regression Method Based on NPP-VIIRS Imagery

Urban area hotspots are considered to be an ideal proxy for spatial heterogeneity of human activity, which is vulnerable to urban expansion. Nighttime light (NTL) images have been extensively employed in monitoring current urbanization dynamics. However, the existing studies related to NTL images mainly concern detection of urban areas, leaving inner spatial differences in urban NTL luminosity poorly explored. In this study, we propose an innovative approach to explore the spatiotemporal trajectory of urban area hotspots using monthly Visible Infrared Imaging Radiometer Suite (VIIRS) NTL images. Firstly, multi-temporal VIIRS NTL intensity was decomposed by time-series analysis to obtain annual stable components after data preprocessing. Secondly, the support vector machine (SVM) regression model was utilized to identify urban area hotspots. In order to ensure the model accuracy, the grid search and cross-validation method was integrated to achieve the optimized model parameters. Finally, we analyzed the spatiotemporal migration trajectory of urban area hotspots by the center of gravity method (i.e., shift distance and angle of urban area hotspot centroid). The results indicate that our method successfully captured urban area hotspots with a regression coefficient over 0.8. Meanwhile, the findings give an intuitive understanding of coupling interaction between urban area hotspots and socioeconomic indicators. This study provides important insights for further decision-making regarding sustainable urban planning.

Rapid Detection of Pomelo Fruit Quality Using Near-Infrared Hyperspectral Imaging Combined With Chemometric Methods

Pomelo is an important agricultural product in southern China. Near-infrared hyperspectral imaging (NIRHI) technology is applied to the rapid detection of pomelo fruit quality. Advanced chemometric methods have been investigated for the optimization of the NIRHI spectral calibration model. The partial least squares (PLS) method is improved for non-linear regression by combining it with the kernel Gaussian radial basis function (RBF). In this study, the core parameters of the PLS latent variables and the RBF kernel width were designed for grid search selection to observe the minimum prediction error and a relatively high correlation coefficient. A deep learning architecture was proposed for the parametric scaling optimization of the RBF-PLS modeling process for NIRHI data in the spectral dimension. The RBF-PLS models were established for the quantitative prediction of the sugar (SU), vitamin C (VC), and organic acid (OA) contents in pomelo samples. Experimental results showed that the proposed RBF-PLS method performed well in the parameter deep search progress for the prediction of the target contents. The predictive errors for model training were 1.076% for SU, 41.381 mg/kg for VC, and 1.136 g/kg for OA, which were under 15% of their reference chemical measurements. The corresponding model testing results were acceptably good. Therefore, the NIRHI technology combined with the study of chemometric methods is applicable for the rapid quantitative detection of pomelo fruit quality, and the proposed algorithmic framework may be promoted for the detection of other agricultural products.

Multiquadrics without the Shape Parameter for Solving Partial Differential Equations

In this article, we present multiquadric radial basis functions (RBFs), including multiquadric (MQ) and inverse multiquadric (IMQ) functions, without the shape parameter for solving partial differential equations using the fictitious source collocation scheme. Different from the conventional collocation method that assigns the RBF at each center point coinciding with an interior point, we separated the center points from the interior points, in which the center points were regarded as the fictitious sources collocated outside the domain. The interior, boundary, and source points were therefore collocated within, on, and outside the domain, respectively. Since the radial distance between the interior point and the source point was always greater than zero, the MQ and IMQ RBFs and their derivatives in the governing equation were smooth and globally infinitely differentiable. Accordingly, the shape parameter was no longer required in the MQ and IMQ RBFs. Numerical examples with the domain in symmetry and asymmetry are presented to verify the accuracy and robustness of the proposed method. The results demonstrated that the proposed method using MQ RBFs without the shape parameter acquires more accurate results than the conventional RBF collocation method with the optimum shape parameter. Additionally, it was found that the locations of the fictitious sources were not sensitive to the accuracy.

Extended Isolation Forests for Fault Detection in Small Hydroelectric Plants

Maintenance in small hydroelectric plants is fundamental for guaranteeing the expansion of clean energy sources and supplying the energy estimated to be necessary for the coming years. Most fault diagnosis models for hydroelectric generating units, proposed so far, are based on the distance between the normal operating profile and newly observed values. The extended isolation forest model is a model, based on binary trees, that has been gaining prominence in anomaly detection applications. However, no study so far has reported the application of the algorithm in the context of hydroelectric power generation. We compared this model with the PCA and KICA-PCA models, using one-year operating data in a small hydroelectric plant with time-series anomaly detection metrics. The algorithm showed satisfactory results with less variance than the others; therefore, it is a suitable candidate for online fault detection applications in the sector.

Modeling Daily and Monthly Water Quality Indicators in a Canal Using a Hybrid Wavelet-Based Support Vector Regression Structure

Accurate prediction of water quality indicators plays an important role in the effective management of water resources. The models which studied limited water quality indicators in natural rivers may give inadequate guidance for managing a canal being used for water diversion. In this study, a hybrid structure (WA-PSO-SVR) based on wavelet analysis (WA) coupled with support vector regression (SVR) and particle swarm optimization (PSO) algorithms was developed to model three water quality indicators, chemical oxygen demand determined by KMnO4 (CODMn), ammonia nitrogen (NH3-N), and dissolved oxygen (DO), in water from the Grand Canal from Beijing to Hangzhou. Modeling was independently conducted over daily and monthly time scales. The results demonstrated that the hybrid WA-PSO-SVR model was able to effectively predict non-linear stationary and non-stationary time series and outperformed two other models (PSO-SVR and a standalone SVR), especially for extreme values prediction. Daily predictions were more accurate than monthly predictions, indicating that the hybrid model was more suitable for short-term predictions in this case. It also demonstrated that using the autocorrelation and partial autocorrelation of time series enabled the construction of appropriate models for water quality prediction. The results contribute to water quality monitoring and better management for water diversion.

Data classification based on fractional order gradient descent with momentum for RBF neural network

The weight-updating methods have played an important role in improving the performance of neural networks. To ameliorate the oscillating phenomenon in training radial basis function (RBF) neural network, a fractional order gradient descent with momentum method for updating the weights of RBF neural network (FOGDM-RBF) is proposed for data classification. Its convergence is proved. In order to speed up the convergence process, an adaptive learning rate is used to adjust the training process. The Iris data set and MNIST data set are used to test the proposed algorithm. The results verify the theoretical results of the proposed algorithm such as its monotonicity and convergence. Some non-parametric statistical tests such as Friedman test and Quade test are taken for the comparison of the proposed algorithm with other algorithms. The influence of fractional order, learning rate and batch size is analysed and compared. Error analysis shows that the algorithm can effectively accelerate the convergence speed of gradient descent method and improve its performance with high accuracy and validity.

Classification and Identification of Industrial Gases Based on Electronic Nose Technology

Rapid detection and identification of industrial gases is a challenging problem. They have a complex composition and different specifications. This paper presents a method based on the kernel discriminant analysis (KDA) algorithm to identify industrial gases. The smell prints of four typical industrial gases were collected by an electronic nose. The extracted features of the collected gases were employed for gas identification using different classification algorithms, including principal component analysis (PCA), linear discriminant analysis (LDA), PCA + LDA, and KDA. In order to obtain better classification results, we reduced the dimensions of the original high-dimensional data, and chose a good classifier. The KDA algorithm provided a high classification accuracy of 100% by selecting the offset of the kernel function c = 10 and the degree of freedom d = 5. It was found that this accuracy was 4.17% higher than the one obtained using PCA. In the case of standard deviation, the KDA algorithm has the highest recognition rate and the least time consumption.

Flow regime identification for air valves failure evaluation in water pipelines using pressure data

Air valve failure can cause air accumulation and result in a loss of carrying capacity, pipe vibration and even in some situations a catastrophic failure of water transmission pipelines. Air is most likely to accumulate in downward sloping pipes, leading to flow regime transition in these pipes. The flow regime identification can be used for fault diagnosis of air valves, but has received little attention in previous research. This paper develops a flow regime identification method that is based on support vector machines (SVMs) to evaluate the operational state of air valves in freshwater/potable pipelines using pressure signals. The laboratory experiments are set up to collect pressure data with respect to the four common flow regimes: bubbly flow, plug flow, blow-back flow and stratified flow. Two SVMs are constructed to identify bubbly and plug flows and validated based on the collected pressure data. The results demonstrate that pressure signals can be used for identifying flow regimes that represent the operational state (functioning or malfunctioning) of air valves. Among several signal features, Power Spectral Density and Short-Zero Crossing Rate are found to be the best indictors to classify flow regimes by SVMs. The sampling rate and time of pressure signals have significant influence on the performance of SVM classification. With optimal SVM features and pressure sampling parameters the identification accuracies exceeded 93% in the test cases. The findings of this study show that the SVM flow regime identification is a promising methodology for fault diagnosis of air valve failure in water pipelines.

Machine Learning for Tailoring Optoelectronic Properties of Single-Walled Carbon Nanotube Films

A machine learning technique, namely, support vector regression, is implemented to enhance single-walled carbon nanotube (SWCNT) thin-film performance for transparent and conducting applications. We collected a comprehensive data set describing the influence of synthesis parameters (temperature and CO₂ concentration) on the equivalent sheet resistance (at 90% transmittance in the visible light range) for SWCNT films obtained by a semi-industrial aerosol (floating-catalyst) CVD with CO as a carbon source and ferrocene as a catalyst precursor. The predictive model trained on the data set shows principal applicability of the method for refining synthesis conditions toward the advanced optoelectronic performance of multiparameter processes such as nanotube growth. Further doping of the improved carbon nanotube films with HAuCl₄ results in the equivalent sheet resistance of 39 Ω/□-one of the lowest values achieved so far for SWCNT films.

Intelligent Estimation of Vitrinite Reflectance of Coal from Photomicrographs Based on Machine Learning

The accurate measurement of vitrinite reflectance (especially for mean maximum vitrinite reflectance, MMVR) is an important issue in the fields of coal mining and processing. However, the application of MMVR has been somewhat hampered by the subjective and the time-consuming characteristic of manual measurements. Semi-automated methods that are oversimplified might affect the accuracy in measuring MMVR values. To address these concerns, we propose a novel MMVR measurement strategy based on machine learning (MMVRML). Considering the complex nature of coal, adaptive K-means clustering is firstly employed to automatically detect the number of clusters (i.e., maceral groups) in photomicrographs. Furthermore, comprehensive features along with a support vector machine are utilized to intelligently identify the regions with vitrinite. The largest region with vitrinite in each photomicrograph is gridded for further regression analysis. Evaluations on 78 photomicrographs show that the model based on random forest and 15 simplified grayscale features achieves the state-of-the-art root mean square error of 0.0424. In addition, to facilitate the usage of petrologists without strong expertise in the machine learning domain, we released the first non-commercial standalone software for estimating MMVR.

Novel automated vessel pattern characterization of larynx contact endoscopic video images

Purpose

Contact endoscopy (CE) is a minimally invasive procedure providing real-time information about the cellular and vascular structure of the superficial layer of laryngeal mucosa. This method can be combined with optical enhancement methods such as narrow band imaging (NBI). However, these techniques have some problems like subjective interpretation of vascular patterns and difficulty in differentiation between benign and malignant lesions. We propose a novel automated approach for vessel pattern characterization of larynx CE + NBI images in order to solve these problems.

Methods

In this approach, five indicators were computed to characterize the level of vessel’s disorder based on evaluation of consistency of gradient and two-dimensional curvature analysis and then 24 features were extracted from these indicators. The method evaluated the ability of the extracted features to classify CE + NBI images based on the vascular pattern and based on the laryngeal lesions. Four datasets were generated from 32 patients involving 1485 images. The classification scenarios were implemented using four supervised classifiers.

Results

For classification of CE + NBI images based on the vascular pattern, polykernel support vector machine (SVM), SVM with radial basis function (RBF), k-nearest neighbor (kNN), and random forest (RF) show an accuracy of 97%, 96%, 96%, and 96%, respectively. For the classification based on the histopathology, Polykernel SVM showed an accuracy of 84%, 86% and 84%, RBF SVM showed an accuracy of 81%, 87% and 83%, kNN showed an accuracy of 89%, 87%, 91%, RF showed an accuracy of 90%, 88% and 91% for classification between benign histopathologies, between malignant histopathologies and between benign and malignant lesions, respectively.

Conclusion

These promising results show that the proposed method could solve the problem of subjectivity in interpretation of vascular patterns and also support the clinicians in the early detection of benign, pre-malignant and malignant lesions.

Classification of spot-welded joint strength using ultrasonic signal time-frequency features and PSO-SVM method

Resistance spot welding (RSW) ultrasonic testing signal contains nugget size and internal defect information which can reflect the mechanical property of spot-welded joint. The mechanical property of spot-welded joint is the most direct indicator for evaluation of spot welding quality. In this paper, 100 samples of different quality spot-welded joints are detected by ultrasonic detection technology, then ultrasonic signals are processed by fast Fourier transform (FFT) and wavelet packet transform (WPT). After that, mathematical statistical methods are used to feature extraction for ultrasonic detection signals in time domain, frequency domain, and wavelet domain based on WPT. 100 samples are subjected to tensile-shear tests to obtain the maximum tensile-shear strength (MTSS) that is used as the classification identifier here. Finally, back-propagation (BP) neural network classifier and particle swarm optimization support vector machine (PSO-SVM) classifier are used to classify the MTSS of spot-welded joints and comparing the accuracy of the two classifiers with different number of features. The results show that the PSO-SVM classifier with all 9 features has a good accuracy, which verifies the feasibility and correctness of the spot welding quality classification method proposed in this paper.

A Video Based Fire Smoke Detection Using Robust AdaBoost

This work considers using camera sensors to detect fire smoke. Static features including texture, wavelet, color, edge orientation histogram, irregularity, and dynamic features including motion direction, change of motion direction and motion speed, are extracted from fire smoke to train and test with different combinations. A robust AdaBoost (RAB) classifier is proposed to improve training and classification accuracy. Extensive experiments on well known challenging datasets and application for fire smoke detection demonstrate that the proposed fire smoke detector leads to a satisfactory performance.

Multicriteria Design of Cost-Conscious Fuzzy Rule-Based Classifiers

Many real-world classification systems must comply with a series of inherent restrictions to the problem at hand such as response times, power consumptions or computational costs. This poses a fundamental limitation to traditional performance-driven classifiers and learning algorithms by restraining their applicability in cost-sensitive scenarios. Because of this, fuzzy systems are leveraged to learn cost-conscious multi-stage classifiers through multiobjective optimization to find a set of optimal tradeoffs between accuracy and any related cost. This approach allows find a suitable balance between all objectives regardless of the scenario. Experimental evaluations were done for Sound Environment Classification in modern battery-powered hearing aids by jointly optimising classification accuracy and computational costs.