ProtoCLIP: Prototypical Contrastive Language Image Pretraining

On the probability of necessity and sufficiency of explaining Graph Neural Networks: A lower bound optimization approach

The explainability of Graph Neural Networks (GNNs) is critical to various GNN applications, yet it remains a significant challenge. A convincing explanation should be both necessary and sufficient simultaneously. However, existing GNN explaining approaches focus on only one of the two aspects, necessity or sufficiency, or a heuristic trade-off between the two. Theoretically, the Probability of Necessity and Sufficiency (PNS) holds the potential to identify the most necessary and sufficient explanation since it can mathematically quantify the necessity and sufficiency of an explanation. Nevertheless, the difficulty of obtaining PNS due to non-monotonicity and the challenge of counterfactual estimation limit its wide use. To address the non-identifiability of PNS, we resort to a lower bound of PNS that can be optimized via counterfactual estimation, and propose a framework of Necessary and Sufficient Explanation for GNN (NSEG) via optimizing that lower bound. Specifically, we depict the GNN as a structural causal model (SCM), and estimate the probability of counterfactual via the intervention under the SCM. Additionally, we leverage continuous masks with a sampling strategy to optimize the lower bound to enhance the scalability. Empirical results demonstrate that NSEG outperforms state-of-the-art methods, consistently generating the most necessary and sufficient explanations. The implementation of our NSEG is available at https://github.com/EthanChu7/NSEG.

Personalizing Vision-Language Models With Hybrid Prompts for Zero-Shot Anomaly Detection

Zero-shot anomaly detection (ZSAD) aims to develop a foundational model capable of detecting anomalies across arbitrary categories without relying on reference images. However, since "abnormality" is inherently defined in relation to "normality" within specific categories, detecting anomalies without reference images describing the corresponding normal context remains a significant challenge. As an alternative to reference images, this study explores the use of widely available product standards to characterize normal contexts and potential abnormal states. Specifically, this study introduces AnomalyVLM, which leverages generalized pretrained vision-language models (VLMs) to interpret these standards and detect anomalies. Given the current limitations of VLMs in comprehending complex textual information, AnomalyVLM generates hybrid prompts-comprising prompts for abnormal regions, symbolic rules, and region numbers-from the standards to facilitate more effective understanding. These hybrid prompts are incorporated into various stages of the anomaly detection process within the selected VLMs, including an anomaly region generator and an anomaly region refiner. By utilizing hybrid prompts, VLMs are personalized as anomaly detectors for specific categories, offering users flexibility and control in detecting anomalies across novel categories without the need for training data. Experimental results on four public industrial anomaly detection datasets, as well as a practical automotive part inspection task, highlight the superior performance and enhanced generalization capability of AnomalyVLM, especially in texture categories. An online demo of AnomalyVLM is available at https://github.com/caoyunkang/Segment-Any-Anomaly.

One-Shot Any-Scene Crowd Counting With Local-to-Global Guidance

Due to different installation angles, heights, and positions of the camera installation in real-world scenes, it is difficult for crowd counting models to work in unseen surveillance scenes. In this paper, we are interested in accurate crowd counting based on the data collected by any surveillance camera, that is to count the crowd from any scene given only one annotated image from that scene. To this end, we firstly pose crowd counting as a one-shot learning task. Through the metric-learning, we propose a simple yet effective method that firstly estimates crowd characteristics and then transfers them to guide the model to count the crowd. Specifically, to fully capture these crowd characteristics of the target scene, we devise the Multi-Prototype Learner to learn the prototypes of foreground and density from the limited support image using the Expectation-Maximization algorithm. To learn the adaptation capability for any unseen scene, estimated multi prototypes are proposed to guide the crowd counting of query images in a local-to-global way. CNN is utilized to activate the local features. And transformer is introduced to correlate global features. Extensive experiments on three surveillance datasets suggest that our method outperforms the SOTA methods in the few-shot crowd counting.

Dynamic event-triggering-based distributed model predictive control of heterogeneous connected vehicle platoon under DoS attacks

This paper is concerned with the distributed model predictive control (DMPC) for heterogeneous connected vehicle platoon (CVP) under denial-of-service (DoS) attacks. Firstly, a dynamic event-triggering mechanism (DETM) based on the information interaction between vehicles is proposed to reduce the communication and computational burdens. Due to the fact that the triggering moment for each vehicle cannot be synchronized and DoS attacks can break the communication between vehicles, a packet replenishment mechanism is designed to ensure the integrity and effectiveness of information interaction. Then, the effect of external disturbance is handled by adding robustness constraints to the DMPC algorithm. In addition, the recursive feasibility of the DMPC algorithm and input-to-state practical stability (ISPS) of the CVP control system are demonstrated. Finally, the effectiveness of the algorithm is verified by simulation and comparison results.

Attributed Multi-Order Graph Convolutional Network for Heterogeneous Graphs

Heterogeneous graph neural networks play a crucial role in discovering discriminative node embeddings and relations from multi-relational networks. One of the key challenges in heterogeneous graph learning lies in designing learnable meta-paths, which significantly impact the quality of learned embeddings. In this paper, we propose an Attributed Multi-Order Graph Convolutional Network (AMOGCN), which automatically explores meta-paths that involve multi-hop neighbors by aggregating multi-order adjacency matrices. The proposed model first constructs different orders of adjacency matrices from manually designed node connections. Next, AMOGCN fuses these various orders of adjacency matrices to create an intact multi-order adjacency matrix. This process is supervised by the node semantic information, which is extracted from the node homophily evaluated by attributes. Eventually, we employ a one-layer simplifying graph convolutional network with the learned multi-order adjacency matrix, which is equivalent to the cross-hop node information propagation with multi-layer graph neural networks. Substantial experiments reveal that AMOGCN achieves superior semi-supervised classification performance compared with state-of-the-art competitors.

Higher-order neurodynamical equation for simplex prediction

It is demonstrated that higher-order patterns beyond pairwise relations can significantly enhance the learning capability of existing graph-based models, and simplex is one of the primary form for graphically representing higher-order patterns. Predicting unknown (disappeared) simplices in real-world complex networks can provide us with deeper insights, thereby assisting us in making better decisions. Nevertheless, previous efforts to predict simplices suffer from two issues: (i) they mainly focus on 2- or 3-simplices, and there are few models available for predicting simplices of arbitrary orders, and (ii) they lack the ability to analyze and learn the features of simplices from the perspective of dynamics. In this paper, we present a Higher-order Neurodynamical Equation for Simplex Prediction of arbitrary order (HNESP), which is a framework that combines neural networks and neurodynamics. Specifically, HNESP simulates the dynamical coupling process of nodes in simplicial complexes through different relations (i.e., strong pairwise relation, weak pairwise relation, and simplex) to learn node-level representations, while explaining the learning mechanism of neural networks from neurodynamics. To enrich the higher-order information contained in simplices, we exploit the entropy and normalized multivariate mutual information of different sub-structures of simplices to acquire simplex-level representations. Furthermore, simplex-level representations and multi-layer perceptron are used to quantify the existence probability of simplices. The effectiveness of HNESP is demonstrated by extensive simulations on seven higher-order benchmarks. Experimental results show that HNESP improves the AUC values of the state-of-the-art baselines by an average of 8.32%. Our implementations will be publicly available at: https://github.com/jianruichen/HNESP.