MADE: Multicurvature Adaptive Embedding for Temporal Knowledge Graph Completion

Temporal knowledge graphs (TKGs) are receiving increased attention due to their time-dependent properties and the evolving nature of knowledge over time. TKGs typically contain complex geometric structures, such as hierarchical, ring, and chain structures, which can often be mixed together. However, embedding TKGs into Euclidean space, as is typically done with TKG completion (TKGC) models, presents a challenge when dealing with high-dimensional nonlinear data and complex geometric structures. To address this issue, we propose a novel TKGC model called multicurvature adaptive embedding (MADE). MADE models TKGs in multicurvature spaces, including flat Euclidean space (zero curvature), hyperbolic space (negative curvature), and hyperspherical space (positive curvature), to handle multiple geometric structures. We assign different weights to different curvature spaces in a data-driven manner to strengthen the ideal curvature spaces for modeling and weaken the inappropriate ones. Additionally, we introduce the quadruplet distributor (QD) to assist the information interaction in each geometric space. Ultimately, we develop an innovative temporal regularization to enhance the smoothness of timestamp embeddings by strengthening the correlation of neighboring timestamps. Experimental results show that MADE outperforms the existing state-of-the-art TKGC models.

Multi-modal long document classification based on Hierarchical Prompt and Multi-modal Transformer

In the realm of long document classification (LDC), previous research has predominantly focused on modeling unimodal texts, overlooking the potential of multi-modal documents incorporating images. To address this gap, we introduce an innovative approach for multi-modal long document classification based on the Hierarchical Prompt and Multi-modal Transformer (HPMT). The proposed HPMT method facilitates multi-modal interactions at both the section and sentence levels, enabling a comprehensive capture of hierarchical structural features and complex multi-modal associations of long documents. Specifically, a Multi-scale Multi-modal Transformer (MsMMT) is tailored to capture the multi-granularity correlations between sentences and images. This is achieved through the incorporation of multi-scale convolutional kernels on sentence features, enhancing the model's ability to discern intricate patterns. Furthermore, to facilitate cross-level information interaction and promote learning of specific features at different levels, we introduce a Hierarchical Prompt (HierPrompt) block. This block incorporates section-level prompts and sentence-level prompts, both derived from a global prompt via distinct projection networks. Extensive experiments are conducted on four challenging multi-modal long document datasets. The results conclusively demonstrate the superiority of our proposed method, showcasing its performance advantages over existing techniques.

Parallelly Adaptive Graph Convolutional Clustering Model

Benefiting from exploiting the data topological structure, graph convolutional network (GCN) has made considerable improvements in processing clustering tasks. The performance of GCN significantly relies on the quality of the pretrained graph, while the graph structures are often corrupted by noise or outliers. To overcome this problem, we replace the pre-trained and fixed graph in GCN by the adaptive graph learned from the data. In this article, we propose a novel end-to-end parallelly adaptive graph convolutional clustering (AGCC) model with two pathway networks. In the first pathway, an adaptive graph convolutional (AGC) module alternatively updates the graph structure and the data representation layer by layer. The updated graph can better reflect the data relationship than the fixed graph. In the second pathway, the auto-encoder (AE) module aims to extract the latent data features. To effectively connect the AGC and AE modules, we creatively propose an attention-mechanism-based fusion (AMF) module to weight and fuse the data representations of the two modules, and transfer them to the AGC module. This simultaneously avoids the over-smoothing problem of GCN. Experimental results on six public datasets show that the effectiveness of the proposed AGCC compared with multiple state-of-the-art deep clustering methods. The code is available at https://github.com/HeXiax/AGCC.

Bridging the Cross-Modality Semantic Gap in Visual Question Answering

The objective of visual question answering (VQA) is to adequately comprehend a question and identify relevant contents in an image that can provide an answer. Existing approaches in VQA often combine visual and question features directly to create a unified cross-modality representation for answer inference. However, this kind of approach fails to bridge the semantic gap between visual and text modalities, resulting in a lack of alignment in cross-modality semantics and the inability to match key visual content accurately. In this article, we propose a model called the caption bridge-based cross-modality alignment and contrastive learning model (CBAC) to address the issue. The CBAC model aims to reduce the semantic gap between different modalities. It consists of a caption-based cross-modality alignment module and a visual-caption (V-C) contrastive learning module. By utilizing an auxiliary caption that shares the same modality as the question and has closer semantic associations with the visual, we are able to effectively reduce the semantic gap by separately matching the caption with both the question and the visual to generate pre-alignment features for each, which are then used in the subsequent fusion process. We also leverage the fact that V-C pairs exhibit stronger semantic connections compared to question-visual (Q-V) pairs to employ a contrastive learning mechanism on visual and caption pairs to further enhance the semantic alignment capabilities of single-modality encoders. Extensive experiments conducted on three benchmark datasets demonstrate that the proposed model outperforms previous state-of-the-art VQA models. Additionally, ablation experiments confirm the effectiveness of each module in our model. Furthermore, we conduct a qualitative analysis by visualizing the attention matrices to assess the reasoning reliability of the proposed model.

Fine-Grained Accident Detection: Database and Algorithm

This paper presents a novel fine-grained task for traffic accident analysis. Accident detection in surveillance or dashcam videos is a common task in the field of traffic accident analysis by using videos. However, common accident detection does not analyze the specific particulars of the accident, only identifies the accident's existence or occurrence time in a video. In this paper, we define the novel fine-grained accident detection task which contains fine-grained accident classification, temporal-spatial occurrence region localization, and accident severity estimation. A transformer-based framework combining the RGB and optical flow information of videos is proposed for fine-grained accident detection. Additionally, we introduce a challenging Fine-grained Accident Detection (FAD) database that covers multiple tasks in surveillance videos which places more emphasis on the overall perspective. Experimental results demonstrate that our model could effectively extract the video features for multiple tasks, indicating that current traffic accident analysis has limitations in dealing with the FAD task and that further research is indeed needed.

Redundancy Is Not What You Need: An Embedding Fusion Graph Auto-Encoder for Self-Supervised Graph Representation Learning

Attribute graphs are a crucial data structure for graph communities. However, the presence of redundancy and noise in the attribute graph can impair the aggregation effect of integrating two different heterogeneous distributions of attribute and structural features, resulting in inconsistent and distorted data that ultimately compromises the accuracy and reliability of attribute graph learning. For instance, redundant or irrelevant attributes can result in overfitting, while noisy attributes can lead to underfitting. Similarly, redundant or noisy structural features can affect the accuracy of graph representations, making it challenging to distinguish between different nodes or communities. To address these issues, we propose the embedded fusion graph auto-encoder framework for self-supervised learning (SSL), which leverages multitask learning to fuse node features across different tasks to reduce redundancy. The embedding fusion graph auto-encoder (EFGAE) framework comprises two phases: pretraining (PT) and downstream task learning (DTL). During the PT phase, EFGAE uses a graph auto-encoder (GAE) based on adversarial contrastive learning to learn structural and attribute embeddings separately and then fuses these embeddings to obtain a representation of the entire graph. During the DTL phase, we introduce an adaptive graph convolutional network (AGCN), which is applied to graph neural network (GNN) classifiers to enhance recognition for downstream tasks. The experimental results demonstrate that our approach outperforms state-of-the-art (SOTA) techniques in terms of accuracy, generalization ability, and robustness.

Beyond low-pass filtering on large-scale graphs via Adaptive Filtering Graph Neural Networks

Graph Neural Networks (GNNs) have emerged as a crucial deep learning framework for graph-structured data. However, existing GNNs suffer from the scalability limitation, which hinders their practical implementation in industrial settings. Many scalable GNNs have been proposed to address this limitation. However, they have been proven to act as low-pass graph filters, which discard the valuable middle- and high-frequency information. This paper proposes a novel graph neural network named Adaptive Filtering Graph Neural Networks (AFGNN), which can capture all frequency information on large-scale graphs. AFGNN consists of two stages. The first stage utilizes low-, middle-, and high-pass graph filters to extract comprehensive frequency information without introducing additional parameters. This computation is a one-time task and is pre-computed before training, ensuring its scalability. The second stage incorporates a node-level attention-based feature combination, enabling the generation of customized graph filters for each node, contrary to existing spectral GNNs that employ uniform graph filters for the entire graph. AFGNN is suitable for mini-batch training, and can enhance scalability and efficiently capture all frequency information from large-scale graphs. We evaluate AFGNN by comparing its ability to capture all frequency information with spectral GNNs, and its scalability with scalable GNNs. Experimental results illustrate that AFGNN surpasses both scalable GNNs and spectral GNNs, highlighting its superiority.

Explore Contextual Information for 3D Scene Graph Generation

3D scene graph generation (SGG) has been of high interest in computer vision. Although the accuracy of 3D SGG on coarse classification and single relation label has been gradually improved, the performance of existing works is still far from being perfect for fine-grained and multi-label situations. In this article, we propose a framework fully exploring contextual information for the 3D SGG task, which attempts to satisfy the requirements of fine-grained entity class, multiple relation labels, and high accuracy simultaneously. Our proposed approach is composed of a Graph Feature Extraction module and a Graph Contextual Reasoning module, achieving appropriate information-redundancy feature extraction, structured organization, and hierarchical inferring. Our approach achieves superior or competitive performance over previous methods on the 3DSSG dataset, especially on the relationship prediction sub-task.

DR-GAN: Distribution Regularization for Text-to-Image Generation

This article presents a new text-to-image (T2I) generation model, named distribution regularization generative adversarial network (DR-GAN), to generate images from text descriptions from improved distribution learning. In DR-GAN, we introduce two novel modules: a semantic disentangling module (SDM) and a distribution normalization module (DNM). SDM combines the spatial self-attention mechanism (SSAM) and a new semantic disentangling loss (SDL) to help the generator distill key semantic information for the image generation. DNM uses a variational auto-encoder (VAE) to normalize and denoise the image latent distribution, which can help the discriminator better distinguish synthesized images from real images. DNM also adopts a distribution adversarial loss (DAL) to guide the generator to align with normalized real image distributions in the latent space. Extensive experiments on two public datasets demonstrated that our DR-GAN achieved a competitive performance in the T2I task. The code link: https://github.com/Tan-H-C/DR-GAN-Distribution-Regularization-for-Text-to-Image-Generation.

Distractor-Aware Event-Based Tracking

Event cameras, or dynamic vision sensors, have recently achieved success from fundamental vision tasks to high-level vision researches. Due to its ability to asynchronously capture light intensity changes, event camera has an inherent advantage to capture moving objects in challenging scenarios including objects under low light, high dynamic range, or fast moving objects. Thus event camera are natural for visual object tracking. However, the current event-based trackers derived from RGB trackers simply modify the input images to event frames and still follow conventional tracking pipeline that mainly focus on object texture for target distinction. As a result, the trackers may not be robust dealing with challenging scenarios such as moving cameras and cluttered foreground. In this paper, we propose a distractor-aware event-based tracker that introduces transformer modules into Siamese network architecture (named DANet). Specifically, our model is mainly composed of a motion-aware network and a target-aware network, which simultaneously exploits both motion cues and object contours from event data, so as to discover motion objects and identify the target object by removing dynamic distractors. Our DANet can be trained in an end-to-end manner without any post-processing and can run at over 80 FPS on a single V100. We conduct comprehensive experiments on two large event tracking datasets to validate the proposed model. We demonstrate that our tracker has superior performance against the state-of-the-art trackers in terms of both accuracy and efficiency.

MHSA-Net: Multihead Self-Attention Network for Occluded Person Re-Identification

This article presents a novel person reidentification model, named multihead self-attention network (MHSA-Net), to prune unimportant information and capture key local information from person images. MHSA-Net contains two main novel components: multihead self-attention branch (MHSAB) and attention competition mechanism (ACM). The MHSAB adaptively captures key local person information and then produces effective diversity embeddings of an image for the person matching. The ACM further helps filter out attention noise and nonkey information. Through extensive ablation studies, we verified that the MHSAB and ACM both contribute to the performance improvement of the MHSA-Net. Our MHSA-Net achieves competitive performance in the standard and occluded person Re-ID tasks.

Domain Adaptation as Optimal Transport on Grassmann Manifolds

Domain adaptation in the Euclidean space is a challenging task on which researchers recently have made great progress. However, in practice, there are rich data representations that are not Euclidean. For example, many high-dimensional data in computer vision are in general modeled by a low-dimensional manifold. This prompts the demand of exploring domain adaptation between non-Euclidean manifold spaces. This article is concerned with domain adaption over the classic Grassmann manifolds. An optimal transport-based domain adaptation model on Grassmann manifolds has been proposed. The model implements the adaption between datasets by minimizing the Wasserstein distances between the projected source data and the target data on Grassmann manifolds. Four regularization terms are introduced to keep task-related consistency in the adaptation process. Furthermore, to reduce the computational cost, a simplified model preserving the necessary adaption property and its efficient algorithm is proposed and tested. The experiments on several publicly available datasets prove the proposed model outperforms several relevant baseline domain adaptation methods.

Hierarchical Graph Convolutional Networks for Structured Long Document Classification

Long document classification (LDC) has been a focused interest in natural language processing (NLP) recently with the exponential increase of publications. Based on the pretrained language models, many LDC methods have been proposed and achieved considerable progression. However, most of the existing methods model long documents as sequences of text while omitting the document structure, thus limiting the capability of effectively representing long texts carrying structure information. To mitigate such limitation, we propose a novel hierarchical graph convolutional network (HGCN) for structured LDC in this article, in which a section graph network is proposed to model the macrostructure of a document and a word graph network with a decoupled graph convolutional block is designed to extract the fine-grained features of a document. In addition, an interaction strategy is proposed to integrate these two networks as a whole by propagating features between them. To verify the effectiveness of the proposed model, four structured long document datasets are constructed, and the extensive experiments conducted on these datasets and another unstructured dataset show that the proposed method outperforms the state-of-the-art related classification methods.

Global and Local Interactive Perception Network for Referring Image Segmentation

The effective modal fusion and perception between the language and the image are necessary for inferring the reference instance in the referring image segmentation (RIS) task. In this article, we propose a novel RIS network, the global and local interactive perception network (GLIPN), to enhance the quality of modal fusion between the language and the image from the local and global perspectives. The core of GLIPN is the global and local interactive perception (GLIP) scheme. Specifically, the GLIP scheme contains the local perception module (LPM) and the global perception module (GPM). The LPM is designed to enhance the local modal fusion by the correspondence between word and image local semantics. The GPM is designed to inject the global structured semantics of images into the modal fusion process, which can better guide the word embedding to perceive the whole image's global structure. Combined with the local-global context semantics fusion, extensive experiments on several benchmark datasets demonstrate the advantage of the proposed GLIPN over most state-of-the-art approaches.

Graph structure learning layer and its graph convolution clustering application

To learn the embedding representation of graph structure data corrupted by noise and outliers, existing graph structure learning networks usually follow the two-step paradigm, i.e., constructing a "good" graph structure and achieving the message passing for signals supported on the learned graph. However, the data corrupted by noise may make the learned graph structure unreliable. In this paper, we propose an adaptive graph convolutional clustering network that alternatively adjusts the graph structure and node representation layer-by-layer with back-propagation. Specifically, we design a Graph Structure Learning layer before each Graph Convolutional layer to learn the sparse graph structure from the node representations, where the graph structure is implicitly determined by the solution to the optimal self-expression problem. This is one of the first works that uses an optimization process as a Graph Network layer, which is obviously different from the function operation in traditional deep learning layers. An efficient iterative optimization algorithm is given to solve the optimal self-expression problem in the Graph Structure Learning layer. Experimental results show that the proposed method can effectively defend the negative effects of inaccurate graph structures. The code is available at https://github.com/HeXiax/SSGNN.

QDN: A Quadruplet Distributor Network for Temporal Knowledge Graph Completion

Temporal knowledge graph completion (TKGC) is an extension of the traditional static knowledge graph completion (SKGC) by introducing the timestamp. The existing TKGC methods generally translate the original quadruplet to the form of the triplet by integrating the timestamp into the entity/relation, and then use SKGC methods to infer the missing item. However, such an integrating operation largely limits the expressive ability of temporal information and ignores the semantic loss problem due to the fact that entities, relations, and timestamps are located in different spaces. In this article, we propose a novel TKGC method called the quadruplet distributor network (QDN), which independently models the embeddings of entities, relations, and timestamps in their specific spaces to fully capture the semantics and builds the QD to facilitate the information aggregation and distribution among them. Furthermore, the interaction among entities, relations, and timestamps is integrated using a novel quadruplet-specific decoder, which stretches the third-order tensor to the fourth-order to satisfy the TKGC criterion. Equally important, we design a novel temporal regularization that imposes a smoothness constraint on temporal embeddings. Experimental results show that the proposed method outperforms the existing state-of-the-art TKGC methods. The source codes of this article are available at https://github.com/QDN for Temporal Knowledge Graph Completion.git.

A Variant in the IRF6 Promoter Associated with the Risk for Orofacial Clefting

The single-nucleotide polymorphism (SNP) rs2235371 (IRF6 V274I) is associated with nonsyndromic cleft lip with or without cleft palate (NSCL/P) in Han Chinese and other populations but appears to be without a functional effect. To find the common etiologic variant or variants within the haplotype tagged by rs2235371, we carried out targeted sequencing of an interval containing IRF6 in 159 Han Chinese with NSCL/P. This study revealed that the SNP rs12403599, within the IRF6 promoter, is associated with all phenotypes of NSCL/P, especially nonsyndromic cleft lip (NSCLO) and a subphenotype of it, microform cleft lip (MCL). This association was replicated in 2 additional much larger cohorts of cases and controls from the Han Chinese. Conditional logistic analysis indicated that association of rs2235371 with NSCL/P was lost if rs12403599 was excluded. rs12403599 contributes the most risk to MCL: its G allele is responsible for 38.47% of the genetic contribution to MCL, and the odds ratios of G/C and G/G genotypes were 2.91 and 6.58, respectively, for MCL. To test if rs12403599 is functional, we carried out reporter assays in a fetal oral epithelium cells (GMSM-K). Unexpectedly, the risk allele G yielded higher promoter activity in GMSM-K. Consistent with the reporter studies, expression of IRF6 in lip tissues from NSCLO and MCL patients with the G/G phenotype was higher than in those from patients with the C/C phenotype. These results indicate that rs12403599 is tagging the risk haplotype for NSCL/P better than rs2235371 in Han Chinese and supports investigation of the mechanisms by which the allele of rs12403599 affects IRF6 expression and tests of this association in different populations.

Point Cloud Scene Completion with Joint Color and Semantic Estimation from Single RGB-D Image

We present a deep reinforcement learning method of progressive view inpainting for colored semantic point cloud scene completion under volume guidance, achieving high-quality scene reconstruction from only a single RGB-D image with severe occlusion. Our approach is end-to-end, consisting of three modules: 3D scene volume reconstruction, 2D RGB-D and segmentation image inpainting, and multi-view selection for completion. Given a single RGB-D image, our method first predicts its semantic segmentation map and goes through the 3D volume branch to obtain a volumetric scene reconstruction as a guide to the next view inpainting step, which attempts to make up the missing information; the third step involves projecting the volume under the same view of the input, concatenating them to complete the current view RGB-D and segmentation map, and integrating all RGB-D and segmentation maps into the point cloud. Since the occluded areas are unavailable, we resort to a A3C network to glance around and pick the next best view for large hole completion progressively until a scene is adequately reconstructed while guaranteeing validity. All steps are learned jointly to achieve robust and consistent results. We perform qualitative and quantitative evaluations with extensive experiments on the 3D-FUTURE data, obtaining better results than state-of-the-arts.

Absolute and Relative Depth-Induced Network for RGB-D Salient Object Detection

Detecting salient objects in complicated scenarios is a challenging problem. Except for semantic features from the RGB image, spatial information from the depth image also provides sufficient cues about the object. Therefore, it is crucial to rationally integrate RGB and depth features for the RGB-D salient object detection task. Most existing RGB-D saliency detectors modulate RGB semantic features with absolution depth values. However, they ignore the appearance contrast and structure knowledge indicated by relative depth values between pixels. In this work, we propose a depth-induced network (DIN) for RGB-D salient object detection, to take full advantage of both absolute and relative depth information, and further, enforce the in-depth fusion of the RGB-D cross-modalities. Specifically, an absolute depth-induced module (ADIM) is proposed, to hierarchically integrate absolute depth values and RGB features, to allow the interaction between the appearance and structural information in the encoding stage. A relative depth-induced module (RDIM) is designed, to capture detailed saliency cues, by exploring contrastive and structural information from relative depth values in the decoding stage. By combining the ADIM and RDIM, we can accurately locate salient objects with clear boundaries, even from complex scenes. The proposed DIN is a lightweight network, and the model size is much smaller than that of state-of-the-art algorithms. Extensive experiments on six challenging benchmarks, show that our method outperforms most existing RGB-D salient object detection models.

Logarithmic Schatten- p Norm Minimization for Tensorial Multi-View Subspace Clustering

The low-rank tensor could characterize inner structure and explore high-order correlation among multi-view representations, which has been widely used in multi-view clustering. Existing approaches adopt the tensor nuclear norm (TNN) as a convex approximation of non-convex tensor rank function. However, TNN treats the different singular values equally and over-penalizes the main rank components, leading to sub-optimal tensor representation. In this paper, we devise a better surrogate of tensor rank, namely the tensor logarithmic Schatten- p norm ([Formula: see text]N), which fully considers the physical difference between singular values by the non-convex and non-linear penalty function. Further, a tensor logarithmic Schatten- p norm minimization ([Formula: see text]NM)-based multi-view subspace clustering ([Formula: see text]NM-MSC) model is proposed. Specially, the proposed [Formula: see text]NM can not only protect the larger singular values encoded with useful structural information, but also remove the smaller ones encoded with redundant information. Thus, the learned tensor representation with compact low-rank structure will well explore the complementary information and accurately characterize the high-order correlation among multi-views. The alternating direction method of multipliers (ADMM) is used to solve the non-convex multi-block [Formula: see text]NM-MSC model where the challenging [Formula: see text]NM problem is carefully handled. Importantly, the algorithm convergence analysis is mathematically established by showing that the sequence generated by the algorithm is of Cauchy and converges to a Karush-Kuhn-Tucker (KKT) point. Experimental results on nine benchmark databases reveal the superiority of the [Formula: see text]NM-MSC model.

Inferring student social link from spatiotemporal behavior data via entropy-based analyzing model

Social link is an important index to understand master students’ mental health and social ability in educational management. Extracting hidden social strength from students’ rich daily life behaviors has also become an attractive research hotspot. Devices with positioning functions record many students’ spatiotemporal behavior data, which can infer students’ social links. However, under the guidance of school regulations, students’ daily activities have a certain regularity and periodicity. Traditional methods usually compare the co-occurrence frequency of two users to infer social association but do not consider the location-intensive and time-sensitive in campus scenes. Aiming at the campus environment, a Spatiotemporal Entropy-Based Analyzing (S-EBA) model for inferring students’ social strength is proposed. The model is based on students’ multi-source heterogeneous behavioral data to calculate the frequency of co-occurrence under the influence of time intervals. Then, the three features of diversity, spatiotemporal hotspot and behavior similarity are introduced to calculate social strength. Experiments show that our method is superior to the traditional methods under many evaluating criteria. The inferred social strength is used as the weight of the edge to construct a social network further to analyze its important impact on students’ education management.

Multi-graph Fusion Graph Convolutional Networks with pseudo-label supervision

Graph convolutional networks (GCNs) have become a popular tool for learning unstructured graph data due to their powerful learning ability. Many researchers have been interested in fusing topological structures and node features to extract the correlation information for classification tasks. However, it is inadequate to integrate the embedding from topology and feature spaces to gain the most correlated information. At the same time, most GCN-based methods assume that the topology graph or feature graph is compatible with the properties of GCNs, but this is usually not satisfied since meaningless, missing, or even unreal edges are very common in actual graphs. To obtain a more robust and accurate graph structure, we intend to construct an adaptive graph with topology and feature graphs. We propose Multi-graph Fusion Graph Convolutional Networks with pseudo-label supervision (MFGCN), which learn a connected embedding by fusing the multi-graphs and node features. We can obtain the final node embedding for semi-supervised node classification by propagating node features over multi-graphs. Furthermore, to alleviate the problem of labels missing in semi-supervised classification, a pseudo-label generation mechanism is proposed to generate more reliable pseudo-labels based on the similarity of node features. Extensive experiments on six benchmark datasets demonstrate the superiority of MFGCN over state-of-the-art classification methods.

Multi-Attribute Subspace Clustering via Auto-Weighted Tensor Nuclear Norm Minimization

Self-expressiveness based subspace clustering methods have received wide attention for unsupervised learning tasks. However, most existing subspace clustering methods consider data features as a whole and then focus only on one single self-representation. These approaches ignore the intrinsic multi-attribute information embedded in the original data feature and result in one-attribute self-representation. This paper proposes a novel multi-attribute subspace clustering (MASC) model that understands data from multiple attributes. MASC simultaneously learns multiple subspace representations corresponding to each specific attribute by exploiting the intrinsic multi-attribute features drawn from original data. In order to better capture the high-order correlation among multi-attribute representations, we represent them as a tensor in low-rank structure and propose the auto-weighted tensor nuclear norm (AWTNN) as a superior low-rank tensor approximation. Especially, the non-convex AWTNN fully considers the difference between singular values through the implicit and adaptive weights splitting during the AWTNN optimization procedure. We further develop an efficient algorithm to optimize the non-convex and multi-block MASC model and establish the convergence guarantees. A more comprehensive subspace representation can be obtained via aggregating these multi-attribute representations, which can be used to construct a clustering-friendly affinity matrix. Extensive experiments on eight real-world databases reveal that the proposed MASC exhibits superior performance over other subspace clustering methods.

A Decoder-Free Variational Deep Embedding for Unsupervised Clustering

In deep clustering frameworks, autoencoder (AE)- or variational AE-based clustering approaches are the most popular and competitive ones that encourage the model to obtain suitable representations and avoid the tendency for degenerate solutions simultaneously. However, for the clustering task, the decoder for reconstructing the original input is usually useless when the model is finished training. The encoder-decoder architecture limits the depth of the encoder so that the learning capacity is reduced severely. In this article, we propose a decoder-free variational deep embedding for unsupervised clustering (DFVC). It is well known that minimizing reconstruction error amounts to maximizing a lower bound on the mutual information (MI) between the input and its representation. That provides a theoretical guarantee for us to discard the bloated decoder. Inspired by contrastive self-supervised learning, we can directly calculate or estimate the MI of the continuous variables. Specifically, we investigate unsupervised representation learning by simultaneously considering the MI estimation of continuous representations and the MI computation of categorical representations. By introducing the data augmentation technique, we incorporate the original input, the augmented input, and their high-level representations into the MI estimation framework to learn more discriminative representations. Instead of matching to a simple standard normal distribution adversarially, we use end-to-end learning to constrain the latent space to be cluster-friendly by applying the Gaussian mixture distribution as the prior. Extensive experiments on challenging data sets show that our model achieves higher performance over a wide range of state-of-the-art clustering approaches.

Rank Consistency Induced Multiview Subspace Clustering via Low-Rank Matrix Factorization

Multiview subspace clustering has been demonstrated to achieve excellent performance in practice by exploiting multiview complementary information. One of the strategies used in most existing methods is to learn a shared self-expressiveness coefficient matrix for all the view data. Different from such a strategy, this article proposes a rank consistency induced multiview subspace clustering model to pursue a consistent low-rank structure among view-specific self-expressiveness coefficient matrices. To facilitate a practical model, we parameterize the low-rank structure on all self-expressiveness coefficient matrices through the tri-factorization along with orthogonal constraints. This specification ensures that self-expressiveness coefficient matrices of different views have the same rank to effectively promote structural consistency across multiviews. Such a model can learn a consistent subspace structure and fully exploit the complementary information from the view-specific self-expressiveness coefficient matrices, simultaneously. The proposed model is formulated as a nonconvex optimization problem. An efficient optimization algorithm with guaranteed convergence under mild conditions is proposed. Extensive experiments on several benchmark databases demonstrate the advantage of the proposed model over the state-of-the-art multiview clustering approaches.

Improving object detection quality with structural constraints

Recent researches revealed object detection networks using the simple “classification loss + localization loss” training objective are not effectively optimized in many cases, while providing additional constraints on network features could effectively improve object detection quality. Specifically, some works used constraints on training sample relations to successfully learn discriminative network features. Based on these observations, we propose Structural Constraint for improving object detection quality. Structural constraint supervises feature learning in classification and localization network branches with Fisher Loss and Equi-proportion Loss respectively, by requiring feature similarities of training sample pairs to be consistent with corresponding ground truth label similarities. Structural constraint could be applied to all object detection network architectures with the assist of our Proxy Feature design. Our experiment results showed that structural constraint mechanism is able to optimize object class instances’ distribution in network feature space, and consequently detection results. Evaluations on MSCOCO2017 and KITTI datasets showed that our structural constraint mechanism is able to assist baseline networks to outperform modern counterpart detectors in terms of object detection quality.

Tracking the sliding of grain boundaries at the atomic scale

Grain boundaries (GBs) play an important role in the mechanical behavior of polycrystalline materials. Despite decades of investigation, the atomic-scale dynamic processes of GB deformation remain elusive, particularly for the GBs in polycrystals, which are commonly of the asymmetric and general type. We conducted an in situ atomic-resolution study to reveal how sliding-dominant deformation is accomplished at general tilt GBs in platinum bicrystals. We observed either direct atomic-scale sliding along the GB or sliding with atom transfer across the boundary plane. The latter sliding process was mediated by movements of disconnections that enabled the transport of GB atoms, leading to a previously unrecognized mode of coupled GB sliding and atomic plane transfer. These results enable an atomic-scale understanding of how general GBs slide in polycrystalline materials.

Probabilistic Linear Discriminant Analysis Based on L1-Norm and Its Bayesian Variational Inference

Probabilistic linear discriminant analysis (PLDA) is a very effective feature extraction approach and has obtained extensive and successful applications in supervised learning tasks. It employs the squared L₂ -norm to measure the model errors, which assumes a Gaussian noise distribution implicitly. However, the noise in real-life applications may not follow a Gaussian distribution. Particularly, the squared L₂ -norm could extremely exaggerate data outliers. To address this issue, this article proposes a robust PLDA model under the assumption of a Laplacian noise distribution, called L1-PLDA. The learning process employs the approach by expressing the Laplacian density function as a superposition of an infinite number of Gaussian distributions via introducing a new latent variable and then adopts the variational expectation-maximization (EM) algorithm to learn parameters. The most significant advantage of the new model is that the introduced latent variable can be used to detect data outliers. The experiments on several public databases show the superiority of the proposed L1-PLDA model in terms of classification and outlier detection.

Aetiology of tinea capitis in China: A multicentre prospective study

BACKGROUND

Tinea capitis is still common in developing countries, such as China. Its pathogen spectrum varies across regions and changes over time.

OBJECTIVES

This study aimed to clarify the current epidemiological characteristics and pathogen spectrum of tinea capitis in China.

METHODS

A multicentre, prospective descriptive study involving 29 tertiary hospitals in China was conducted. From August 2019 to July 2020, 611 patients with tinea capitis were enrolled. Data concerning demography, risk factors and fungal tests were collected. The pathogens were further identified by morphology or molecular sequencing when necessary in the central laboratory.

RESULTS

Among all enrolled patients, 74.1% of the cases were 2- to 8-year-olds. The children with tinea capitis were mainly boys (56.2%) and more likely to have an animal contact history (57.4% vs. 35.3%, P = 0.012) and zoophilic dermatophyte infection (73.5%). The adults were mainly females (83.3%) and more likely to have anthropophilic agent infection (53.5%). The most common pathogen was zoophilic Microsporum canis (354, 65.2%), followed by anthropophilic Trichophyton violaceum (74, 13.6%). In contrast to the eastern, western and northeastern regions where zoophilic M. canis predominated, anthropophilic T. violaceum predominated in central China (69.2%, P < 0.0001), where the patients had the most tinea at other sites (20.3%) and dermatophytosis contact (25.9%) with the least animal contact (38.8%). Microsporum ferrugineum was the most common anthropophilic agent in the western area, especially in Xinjiang Province.

CONCLUSIONS

Boys aged approximately 5 years were mainly affected. Dermatologists are advised to pay more attention to the different transmission routes and pathogen spectra in different age groups from different regions.

Intensity-Aware Single-Image Deraining With Semantic and Color Regularization

Rain degrades image visual quality and disrupts object structures, obscuring their details and erasing their colors. Existing deraining methods are primarily based on modeling either visual appearances of rain or its physical characteristics (e.g., rain direction and density), and thus suffer from two common problems. First, due to the stochastic nature of rain, they tend to fail in recognizing rain streaks correctly, and wrongly remove image structures and details. Second, they fail to recover the image colors erased by heavy rain. In this paper, we address these two problems with the following three contributions. First, we propose a novel PHP block to aggregate comprehensive spatial and hierarchical information for removing rain streaks of different sizes. Second, we propose a novel network to first remove rain streaks, then recover objects structures/colors, and finally enhance details. Third, to train the network, we prepare a new dataset, and propose a novel loss function to introduce semantic and color regularization for deraining. Extensive experiments demonstrate the superiority of the proposed method over state-of-the-art deraining methods on both synthesized and real-world data, in terms of visual quality, quantitative accuracy, and running speed.

The human-AI scoring system: A new method for CT-based assessment of COVID-19 severity

BACKGROUND

Chest computed tomography (CT) plays an important role in the diagnosis and assessment of coronavirus disease 2019 (COVID-19).

OBJECTIVE

To evaluate the value of an artificial intelligence (AI) scoring system for radiologically assessing the severity of COVID-19.

MATERIALS AND METHODS

Chest CT images of 81 patients (61 of normal type and 20 of severe type) with confirmed COVID-19 were used. The test data were anonymized. The scores achieved by four methods (junior radiologists; AI scoring system; human-AI segmentation system; human-AI scoring system) were compared with that by two experienced radiologists (reference score). The mean absolute errors (MAEs) between the four methods and experienced radiologists were calculated separately. The Wilcoxon test is used to predict the significance of the severity of COVID-19. Then use Spearman correlation analysis ROC analysis was used to evaluate the performance of different scores.

RESULTS

The AI score had a relatively low MAE (1.67-2.21). Score of human-AI scoring system had the lowest MAE (1.67), a diagnostic value almost equal to reference score (r= 0.97), and a strongest correlation with clinical severity (r= 0.59, p< 0.001). The AUCs of reference score, score of junior radiologists, AI score, score of human-AI segmentation system, and score of human-AI scoring system were 0.874, 0.841, 0.852, 0.857 and 0.865, respectively.

CONCLUSION

The human-AI scoring system can help radiologists to improve the accuracy of COVID-19 severity assessment.

Real-Time Human Action Recognition Using Locally Aggregated Kinematic-Guided Skeletonlet and Supervised Hashing-by-Analysis Model

3-D action recognition is referred to as the classification of action sequences which consist of 3-D skeleton joints. While many research works are devoted to 3-D action recognition, it mainly suffers from three problems: 1) highly complicated articulation; 2) a great amount of noise; and 3) low implementation efficiency. To tackle all these problems, we propose a real-time 3-D action-recognition framework by integrating the locally aggregated kinematic-guided skeletonlet (LAKS) with a supervised hashing-by-analysis (SHA) model. We first define the skeletonlet as a few combinations of joint offsets grouped in terms of the kinematic principle and then represent an action sequence using LAKS, which consists of a denoising phase and a locally aggregating phase. The denoising phase detects the noisy action data and adjusts it by replacing all the features within it with the features of the corresponding previous frame, while the locally aggregating phase sums the difference between an offset feature of the skeletonlet and its cluster center together over all the offset features of the sequence. Finally, the SHA model combines sparse representation with a hashing model, aiming at promoting the recognition accuracy while maintaining high efficiency. Experimental results on MSRAction3D, UTKinectAction3D, and Florence3DAction datasets demonstrate that the proposed method outperforms state-of-the-art methods in both recognition accuracy and implementation efficiency.

Adaptive Fusion of Heterogeneous Manifolds for Subspace Clustering

Multiview clustering (MVC) has recently received great interest due to its pleasing efficacy in combining the abundant and complementary information to improve clustering performance, which overcomes the drawbacks of view limitation existed in the standard single-view clustering. However, the existing MVC methods are mostly designed for vectorial data from linear spaces and, thus, are not suitable for multiple dimensional data with intrinsic nonlinear manifold structures, e.g., videos or image sets. Some works have introduced manifolds' representation methods of data into MVC and obtained considerable improvements, but how to fuse multiple manifolds efficiently for clustering is still a challenging problem. Particularly, for heterogeneous manifolds, it is an entirely new problem. In this article, we propose to represent the complicated multiviews' data as heterogeneous manifolds and a fusion framework of heterogeneous manifolds for clustering. Different from the empirical weighting methods, an adaptive fusion strategy is designed to weight the importance of different manifolds in a data-driven manner. In addition, the low-rank representation is generalized onto the fused heterogeneous manifolds to explore the low-dimensional subspace structures embedded in data for clustering. We assessed the proposed method on several public data sets, including human action video, facial image, and traffic scenario video. The experimental results show that our method obviously outperforms a number of state-of-the-art clustering methods.

SMSIR: Spherical Measure Based Spherical Image Representation

This paper presents a spherical measure based spherical image representation(SMSIR) and sphere-based resampling methods for generating our representation. On this basis, a spherical wavelet transform is also proposed. We first propose a formal recursive definition of the spherical triangle elements of SMSIR and a dyadic index scheme. The index scheme, which supports global random access and needs not to be pre-computed and stored, can efficiently index the elements of SMSIR like planar images. Two resampling methods to generate SMSIR from the most commonly used ERP(Equirectangular Projection) representation are presented. Notably, the spherical measure based resampling, which exploits the mapping between the spherical and the parameter domain, achieves higher computational efficiency than the spherical RBF(Radial Basis Function) based resampling. Finally, we design high-pass and low-pass filters with lifting schemes based on the dyadic index to further verify the efficiency of our index and deal with the spherical isotropy. It provides novel Multi-Resolution Analysis(MRA) for spherical images. Experiments on continuous synthetic spherical images indicate that our representation can recover the original image signals with higher accuracy than the ERP and CMP(Cubemap) representations at the same sampling rate. Besides, the resampling experiments on natural spherical images show that our resampling methods outperform the bilinear and bicubic interpolations concerning the subjective and objective quality. Particularly, as high as 2dB gain in terms of S-PSNR is achieved. Experiments also show that our spherical image transform can capture more geometric features of spherical images than traditional wavelet transform.