2-D Stochastic Configuration Networks for Image Data Analytics

Mesothelin expression prediction in pancreatic cancer based on multimodal stochastic configuration networks

Predicting tumor biomarkers with high precision is essential for improving the diagnostic accuracy and developing more effective treatment strategies. This paper proposes a machine learning model that utilizes CT images and biopsy whole slide images (WSI) to classify mesothelin expression levels in pancreatic cancer. By combining multimodal learning and stochastic configuration networks, a radiopathomics mesothelin-prediction system named RPMSNet is developed. The system extracts radiomic and pathomic features from CT images and WSI, respectively, and sends them into stochastic configuration networks for the final prediction. Compared to traditional radiomics or pathomics, this system has the capability to capture more comprehensive image features, providing a multidimensional insight into tissue characteristics. The experiments and analyses demonstrate the accuracy and effectiveness of the system, with an area under the curve of 81.03%, an accuracy of 73.67%, a sensitivity of 71.54%, a precision of 76.78%, and a F1-score of 72.61%, surpassing both single-modality and dual-modality models. RPMSNet highlights its potential for early diagnosis and personalized treatment in precision medicine.

Deep Optimized Broad Learning System for Applications in Tabular Data Recognition

The broad learning system (BLS) is a versatile and effective tool for analyzing tabular data. However, the rapid expansion of big data has resulted in an overwhelming amount of tabular data, necessitating the development of specialized tools for effective management and analysis. This article introduces an optimized BLS (OBLS) specifically tailored for big data analysis. In addition, a deep-optimized BLS (DOBLS) network is developed further to enhance the performance and efficiency of the OBLS. The main contributions of this article are: 1) by retracing the network's error from the output space to the latent space, the OBLS adjusts parameters in the feature and enhancement node layers. This process aims to achieve more resilient representations, resulting in improved performance; 2) the DOBLS is a multilayered structure consisting of multiple OBLSs, wherein each OBLS connects to the input and output layers, enabling direct data propagation. This design helps reduce information loss between layers, ensuring an efficient flow of information throughout the network; and 3) the proposed methods demonstrate robustness across various applications, including multiview feature embedding, one-class classification (OCC), camera model identification, electroencephalogram (EEG) signal processing, and radar signal analysis. Experimental results validate the effectiveness of the proposed models. To ensure reproducibility, the source code is available at https://github.com/1027051515/OBLS_DOBLS.

Industry Image Classification Based on Stochastic Configuration Networks and Multi-Scale Feature Analysis

For industry image data, this paper proposes an image classification method based on stochastic configuration networks and multi-scale feature extraction. The multi-scale features are extracted from images of different scales using deep 2DSCN, and the hidden features of multiple layers are also connected together to obtain more informational features. The integrated features are fed into SCNs to learn a classifier which improves the recognition rate for different categories. In the experiments, a handwritten digit database and an industry hot-rolled steel strip database are used, and the comparison results demonstrate the proposed method can effectively improve the classification accuracy.

Reliable prediction intervals with directly optimized inductive conformal regression for deep learning

By generating prediction intervals (PIs) to quantify the uncertainty of each prediction in deep learning regression, the risk of wrong predictions can be effectively controlled. High-quality PIs need to be as narrow as possible, whilst covering a preset proportion of real labels. At present, many approaches to improve the quality of PIs can effectively reduce the width of PIs, but they do not ensure that enough real labels are captured. Inductive Conformal Predictor (ICP) is an algorithm that can generate effective PIs which is theoretically guaranteed to cover a preset proportion of data. However, typically ICP is not directly optimized to yield minimal PI width. In this study, we propose Directly Optimized Inductive Conformal Regression (DOICR) for neural networks that takes only the average width of PIs as the loss function and increases the quality of PIs through an optimized scheme, under the validity condition that sufficient real labels are captured in the PIs. Benchmark experiments show that DOICR outperforms current state-of-the-art algorithms for regression problems using underlying Deep Neural Network structures for both tabular and image data.

Co-Design of Adaptive Event-Triggered Mechanism and Asynchronous H∞ Control for 2-D Markov Jump Systems via Genetic Algorithm

This article concerns the co-design scheme of the adaptive event-triggered mechanism (AETM) and asynchronous H∞ control for two-dimensional (2-D) Markov jump systems. First, we introduce a hidden Markov model with the observation that the asynchronous phenomenon is inevitable between the plant mode and the controller mode. Besides, for economizing the communication times, an innovative 2-D AETM is constructed, which can dynamically regulate the event-triggered thresholds to strive for better system performance. Then, by utilizing the 2-D Lyapunov stability theory, nonlinear matrix inequalities are built to ensure the asymptotic mean-square stability with an H∞ performance for the closed-loop 2-D system. To avoid introducing any conservatism when handling the above nonlinear matrix inequalities, a binary-based genetic algorithm (BGA) is exploited to treat some variables as known, such that derive some directly solvable linear matrix inequalities. Finally, a simulation example is provided to verify the effectiveness of the proposed 2-D AETM-based asynchronous controller strategy with a BGA.

A regularized stochastic configuration network based on weighted mean of vectors for regression

The stochastic configuration network (SCN) randomly configures the input weights and biases of hidden layers under a set of inequality constraints to guarantee its universal approximation property. The SCN has demonstrated great potential for fast and efficient data modeling. However, the prediction accuracy and convergence rate of SCN are frequently impacted by the parameter settings of the model. The weighted mean of vectors (INFO) is an innovative swarm intelligence optimization algorithm, with an optimization procedure consisting of three phases: updating rule, vector combining, and a local search. This article aimed at establishing a new regularized SCN based on the weighted mean of vectors (RSCN-INFO) to optimize its parameter selection and network structure. The regularization term that combines the ridge method with the residual error feedback was introduced into the objective function in order to dynamically adjust the training parameters. Meanwhile, INFO was employed to automatically explore an appropriate four-dimensional parameter vector for RSCN. The selected parameters may lead to a compact network architecture with a faster reduction of the network residual error. Simulation results over some benchmark datasets demonstrated that the proposed RSCN-INFO showed superior performance with respect to parameter setting, fast convergence, and network compactness compared with other contrast algorithms.

Are Graph Convolutional Networks With Random Weights Feasible?

Graph Convolutional Networks (GCNs), as a prominent example of graph neural networks, are receiving extensive attention for their powerful capability in learning node representations on graphs. There are various extensions, either in sampling and/or node feature aggregation, to further improve GCNs' performance, scalability and applicability in various domains. Still, there is room for further improvements on learning efficiency because performing batch gradient descent using the full dataset for every training iteration, as unavoidable for training (vanilla) GCNs, is not a viable option for large graphs. The good potential of random features in speeding up the training phase in large-scale problems motivates us to consider carefully whether GCNs with random weights are feasible. To investigate theoretically and empirically this issue, we propose a novel model termed Graph Convolutional Networks with Random Weights (GCN-RW) by revising the convolutional layer with random filters and simultaneously adjusting the learning objective with regularized least squares loss. Theoretical analyses on the model's approximation upper bound, structure complexity, stability and generalization, are provided with rigorous mathematical proofs. The effectiveness and efficiency of GCN-RW are verified on semi-supervised node classification task with several benchmark datasets. Experimental results demonstrate that, in comparison with some state-of-the-art approaches, GCN-RW can achieve better or matched accuracies with less training time cost.

Greengage Grading Method Based on Dynamic Feature and Ensemble Networks

To overcome the deficiencies of the traditional open-loop cognition method, which lacks evaluation of the cognitive results, a novel cognitive method for greengage grading based on dynamic feature and ensemble networks is explored in this paper. First, a greengage grading architecture with an adaptive feedback mechanism based on error adjustment is constructed to imitate the human cognitive mechanism. Secondly, a dynamic representation model for convolutional feature space construction of a greengage image is established based on the entropy constraint indicators, and the bagging classification network for greengage grading is built based on stochastic configuration networks (SCNs) to realize a hierarchical representation of the greengage features and enhance the generalization of the classifier. Thirdly, an entropy-based error model of the cognitive results for greengage grading is constructed to describe the optimal cognitive problem from an information perspective, and then the criteria and mechanism for feature level and feature efficiency regulation are given out within the constraint of cognitive error entropy. Finally, numerous experiments are performed on the collected greengage images. The experimental results demonstrate the effectiveness and superiority of our method, especially for the classification of similar samples, compared with the existing open-loop algorithms.

A Lightweight Learning Method for Stochastic Configuration Networks Using Non-Inverse Solution

Stochastic configuration networks (SCNs) face time-consuming issues when dealing with complex modeling tasks that usually require a mass of hidden nodes to build an enormous network. An important reason behind this issue is that SCNs always employ the Moore–Penrose generalized inverse method with high complexity to update the output weights in each increment. To tackle this problem, this paper proposes a lightweight SCNs, called L-SCNs. First, to avoid using the Moore–Penrose generalized inverse method, a positive definite equation is proposed to replace the over-determined equation, and the consistency of their solution is proved. Then, to reduce the complexity of calculating the output weight, a low complexity method based on Cholesky decomposition is proposed. The experimental results based on both the benchmark function approximation and real-world problems including regression and classification applications show that L-SCNs are sufficiently lightweight.

Stochastic configuration networks with fast implementations

Stochastic configuration networks (SCNs) employ a supervisory mechanism to assign hidden-node parameters in the incremental construction process. SCNs offer the advantages of practical implementation, fast convergence, and better generalization performance. However, due to its high computational cost and the scalability of numerical algorithms for the least square technique, it is rather limited for dealing with enormous amounts of data. This paper proposes fast SCNs (F-SCNs), whose output weights are determined using orthogonal matrix Q and upper triangular matrix R decomposition. The network can iteratively update the output weights utilizing the output information from the predecessor node using this incremental technique. We investigated the computational complexity of SCNs and F-SCNs and demonstrated that F-SCNs are suitable for scenarios in which the hidden layer has a significant number of nodes. We evaluated the proposed method on four real-world regression datasets; experimental results show that our method has notable advantages in terms of speed and effectiveness of learning.

Why Do Big Data and Machine Learning Entail the Fractional Dynamics?

Fractional-order calculus is about the differentiation and integration of non-integer orders. Fractional calculus (FC) is based on fractional-order thinking (FOT) and has been shown to help us to understand complex systems better, improve the processing of complex signals, enhance the control of complex systems, increase the performance of optimization, and even extend the enabling of the potential for creativity. In this article, the authors discuss the fractional dynamics, FOT and rich fractional stochastic models. First, the use of fractional dynamics in big data analytics for quantifying big data variability stemming from the generation of complex systems is justified. Second, we show why fractional dynamics is needed in machine learning and optimal randomness when asking: "is there a more optimal way to optimize?". Third, an optimal randomness case study for a stochastic configuration network (SCN) machine-learning method with heavy-tailed distributions is discussed. Finally, views on big data and (physics-informed) machine learning with fractional dynamics for future research are presented with concluding remarks.

Optimal Randomness for Stochastic Configuration Network (SCN) with Heavy-Tailed Distributions

Stochastic Configuration Network (SCN) has a powerful capability for regression and classification analysis. Traditionally, it is quite challenging to correctly determine an appropriate architecture for a neural network so that the trained model can achieve excellent performance for both learning and generalization. Compared with the known randomized learning algorithms for single hidden layer feed-forward neural networks, such as Randomized Radial Basis Function (RBF) Networks and Random Vector Functional-link (RVFL), the SCN randomly assigns the input weights and biases of the hidden nodes in a supervisory mechanism. Since the parameters in the hidden layers are randomly generated in uniform distribution, hypothetically, there is optimal randomness. Heavy-tailed distribution has shown optimal randomness in an unknown environment for finding some targets. Therefore, in this research, the authors used heavy-tailed distributions to randomly initialize weights and biases to see if the new SCN models can achieve better performance than the original SCN. Heavy-tailed distributions, such as Lévy distribution, Cauchy distribution, and Weibull distribution, have been used. Since some mixed distributions show heavy-tailed properties, the mixed Gaussian and Laplace distributions were also studied in this research work. Experimental results showed improved performance for SCN with heavy-tailed distributions. For the regression model, SCN-Lévy, SCN-Mixture, SCN-Cauchy, and SCN-Weibull used less hidden nodes to achieve similar performance with SCN. For the classification model, SCN-Mixture, SCN-Lévy, and SCN-Cauchy have higher test accuracy of 91.5%, 91.7% and 92.4%, respectively. Both are higher than the test accuracy of the original SCN.

FPGA-Based Implementation of Stochastic Configuration Networks for Regression Prediction

The implementation of neural network regression prediction based on digital circuits is one of the challenging problems in the field of machine learning and cognitive recognition, and it is also an effective way to relieve the pressure of the Internet in the era of intelligence. As a nonlinear network, the stochastic configuration network (SCN) is considered to be an effective method for regression prediction due to its good performance in learning and generalization. Therefore, in this paper, we adapt the SCN to regression analysis, and design and verify the field programmable gate array (FPGA) framework to implement SCN model for the first time. In addition, in order to improve the performance of the SCN model based on the FPGA, the implementation of the nonlinear activation function on the FPGA is optimized, which effectively improves the prediction accuracy while considering the utilization rate of hardware resources. Experimental results based on the simulation data set and the real data set prove that the proposed FPGA framework successfully implements the SCN regression prediction model, and the improved SCN model has higher accuracy and a more stable performance. Compared with the extreme learning machine (ELM), the prediction performance of the proposed SCN implementation model based on the FPGA for the simulation data set and the real data set is improved by 56.37% and 17.35%, respectively.