An Online Active Broad Learning Approach for Real-Time Safety Assessment of Dynamic Systems in Nonstationary Environments

Broad Multitask Learning System With Group Sparse Regularization

The broad learning system (BLS) featuring lightweight, incremental extension, and strong generalization capabilities has been successful in its applications. Despite these advantages, BLS struggles in multitask learning (MTL) scenarios with its limited ability to simultaneously unravel multiple complex tasks where existing BLS models cannot adequately capture and leverage essential information across tasks, decreasing their effectiveness and efficacy in MTL scenarios. To address these limitations, we proposed an innovative MTL framework explicitly designed for BLS, named group sparse regularization for broad multitask learning system using related task-wise (BMtLS-RG). This framework combines a task-related BLS learning mechanism with a group sparse optimization strategy, significantly boosting BLS's ability to generalize in MTL environments. The task-related learning component harnesses task correlations to enable shared learning and optimize parameters efficiently. Meanwhile, the group sparse optimization approach helps minimize the effects of irrelevant or noisy data, thus enhancing the robustness and stability of BLS in navigating complex learning scenarios. To address the varied requirements of MTL challenges, we presented two additional variants of BMtLS-RG: BMtLS-RG with sharing parameters of feature mapped nodes (BMtLS-RGf), which integrates a shared feature mapping layer, and BMtLS-RGf and enhanced nodes (BMtLS-RGfe), which further includes an enhanced node layer atop the shared feature mapping structure. These adaptations provide customized solutions tailored to the diverse landscape of MTL problems. We compared BMtLS-RG with state-of-the-art (SOTA) MTL and BLS algorithms through comprehensive experimental evaluation across multiple practical MTL and UCI datasets. BMtLS-RG outperformed SOTA methods in 97.81% of classification tasks and achieved optimal performance in 96.00% of regression tasks, demonstrating its superior accuracy and robustness. Furthermore, BMtLS-RG exhibited satisfactory training efficiency, outperforming existing MTL algorithms by 8.04-42.85 times.

An Incremental-Self-Training-Guided Semi-Supervised Broad Learning System

The broad learning system (BLS) has recently been applied in numerous fields. However, it is mainly a supervised learning system and thus not suitable for specific practical applications with a mixture of labeled and unlabeled data. Despite a manifold regularization-based semi-supervised BLS, its performance still requires improvement, because its assumption is not always applicable. Therefore, this article proposes an incremental-self-training-guided semi-supervised BLS (ISTSS-BLS). Distinctive to traditional self-training, where all unlabeled data are labeled simultaneously, incremental self-training (IST) obtains unlabeled data incrementally from an established sorted list based on the distance between the data and their cluster center. During iterative learning, a small portion of labeled data is first used to train BLS. The system recursively self-updates its structure and meta-parameters using: 1) the double-restricted mechanism and 2) the dynamic neuron-incremental mechanism. The double-restricted mechanism is beneficial to preventing the introduction of incorrect pseudo-labeled samples, and the dynamic neuron-incremental mechanism guides the self-updating of the network structure effectively based on the training accuracy of the labeled data. These strategies guarantee a parsimonious model during the update. Besides, a novel metric, the accuracy-time ratio (A/T), is proposed to evaluate the model's performance comprehensively regarding time and accuracy. In experimental verifications, ISTSS-BLS performs outstandingly on 11 datasets. Specifically, the IST is compared with the traditional one on three scales data, saving up to 52.02% learning time. In addition, ISTSS-BLS is compared with different state-of-the-art alternatives, and all results indicate that it possesses significant advantages in performance.

Bearing Rigidity-Based Flocking Control of AUVs via Semi-Supervised Incremental Broad Learning

Flocking control of autonomous underwater vehicles (AUVs) has been regarded as the basis of many sophisticated marine coordination missions. However, there is still a research gap on the flocking of AUVs in weak communication and complex marine environment. This article attempts to fill up the above research gap from graph theory and intelligent learning perspectives. We first employ the bearing rigidity graph to describe the topology relationships of AUVs, through which an iterative gradient decent-based localization estimator is provided to obtain the position information. In order to improve the localization accuracy and energy efficiency, a min-weighted bearing rigidity graph generation strategy is developed. Along with this, we adopt the semi-supervised broad learning system (BLS) to design the model-free flocking controllers for AUVs in obstacle environment. The innovations of this article are summarized as follows: 1) the min-weighted bearing rigidity-based localization strategy can balance the localization accuracy and communication consumption as compared to the neighboring rule-based solutions and 2) the semi-supervised broad learning-based flocking controller can decrease the training time and solve the label limit over the supervised learning-based controllers. Finally, simulation and experimental studies are provided to verify the effectiveness.

Robust Incremental Broad Learning System for Data Streams of Uncertain Scale

Due to its marvelous performance and remarkable scalability, a broad learning system (BLS) has aroused a wide range of attention. However, its incremental learning suffers from low accuracy and long training time, especially when dealing with unstable data streams, making it difficult to apply in real-world scenarios. To overcome these issues and enrich its relevant research, a robust incremental BLS (RI-BLS) is proposed. In this method, the proposed weight update strategy introduces two memory matrices to store the learned information, thus the computational procedure of ridge regression is decomposed, resulting in precomputed ridge regression. During incremental learning, RI-BLS updates two memory matrices and renews weights via precomputed ridge regression efficiently. In addition, this update strategy is theoretically analyzed in error, time complexity, and space complexity compared with existing incremental BLSs. Different from Greville's method used in the original incremental BLS, its results are closer to the solution of one-shot calculation. Compared with the existing incremental BLSs, the proposed method exhibits more stable time complexity and superior space complexity. The experiments prove that RI-BLS outperforms other incremental BLSs when handling both stable and unstable data streams. Furthermore, experiments demonstrate that the proposed weight update strategy applies to other random neural networks as well.

CADM+: Confusion-Based Learning Framework With Drift Detection and Adaptation for Real-Time Safety Assessment

Real-time safety assessment (RTSA) of dynamic systems holds substantial implications across diverse fields, including industrial and electronic applications. However, the complexity and rapid flow nature of data streams, coupled with the expensive label cost and pose significant challenges. To address these issues, a novel confusion-based learning framework, termed confusion-and-detection method plus (CADM+), is proposed in this article. When drift occurs, the model is updated with uncertain samples, which may cause confusion between existing and new concepts, resulting in performance differences. The cosine similarity is used to measure the degree of such conceptual confusion in the model. Furthermore, the change of standard deviation within a fixed-size cosine similarity window is introduced as an indicator for drift detection. Theoretical demonstrations show the asymptotic increase of cosine similarity. In addition, the approximate independence of the change in standard deviation with the number of trained samples is indicated. Finally, the extreme value theory (EVT) is applied to determine the threshold of judging drifts. Several experiments are conducted to verify its effectiveness. Experimental results prove that the proposed framework is more suitable for RTSA tasks compared with state-of-the-art algorithms. The source code is available at https://github.com/THUFDD/CADM-plus.

Bidimensionally partitioned online sequential broad learning system for large-scale data stream modeling

Incremental broad learning system (IBLS) is an effective and efficient incremental learning method based on broad learning paradigm. Owing to its streamlined network architecture and flexible dynamic update scheme, IBLS can achieve rapid incremental reconstruction on the basis of the previous model without the entire retraining from scratch, which enables it adept at handling streaming data. However, two prominent deficiencies still persist in IBLS and constrain its further promotion in large-scale data stream scenarios. Firstly, IBLS needs to retain all historical data and perform associated calculations in the incremental learning process, which causes its computational overhead and storage burden to increase over time and as such puts the efficacy of the algorithm at risk for massive or unlimited data streams. Additionally, due to the random generation rule of hidden nodes, IBLS generally necessitates a large network size to guarantee approximation accuracy, and the resulting high-dimensional matrix calculation poses a greater challenge to the updating efficiency of the model. To address these issues, we propose a novel bidimensionally partitioned online sequential broad learning system (BPOSBLS) in this paper. The core idea of BPOSBLS is to partition the high-dimensional broad feature matrix bidimensionally from the aspects of instance dimension and feature dimension, and consequently decompose a large least squares problem into multiple smaller ones, which can then be solved individually. By doing so, the scale and computational complexity of the original high-order model are substantially diminished, thus significantly improving its learning efficiency and usability for large-scale complex learning tasks. Meanwhile, an ingenious recursive computation method called partitioned recursive least squares is devised to solve the BPOSBLS. This method exclusively utilizes the current online samples for iterative updating, while disregarding the previous historical samples, thereby rendering BPOSBLS a lightweight online sequential learning algorithm with consistently low computational costs and storage requirements. Theoretical analyses and simulation experiments demonstrate the effectiveness and superiority of the proposed algorithm.

Dynamic Submodular-Based Learning Strategy in Imbalanced Drifting Streams for Real-Time Safety Assessment in Nonstationary Environments

The design of real-time safety assessment (RTSA) approaches in nonstationary environments is meaningful to reduce the possibility of significant losses. However, several challenging problems are needed to be well considered. The performance of existing approaches will be negatively affected in the settings of imbalanced drifting streams. In this case, the model design with the incremental update should also be explored. Furthermore, the query strategy should also be well-designed. This article investigates a dynamic submodular-based learning strategy to address such issues. Specifically, an efficient incremental update procedure is designed with the structure of the broad learning system (BLS), which is beneficial to the detection of concept drift. Furthermore, a novel dynamic submodular-based annotation with an activation interval strategy is proposed to select valuable samples in imbalanced drifting streams. The lower bound of annotation value is also proven theoretically with a novel drift adaption mechanism. Numerous experiments are conducted with the realistic data of JiaoLong deep-sea manned submersible. The experimental results show that the proposed approach can achieve better assessment accuracy than typical existing approaches.