A gentle introduction to deep learning for graphs

A comprehensive review and benchmark of differential analysis tools for Hi-C data

MOTIVATION

The 3D organization of the genome plays a crucial role in various biological processes. Hi-C technology is widely used to investigate chromosome structures by quantifying 3D proximity between genomic regions. While numerous computational tools exist for detecting differences in Hi-C data between conditions, a comprehensive review and benchmark comparing their effectiveness is lacking.

RESULTS

This study offers a comprehensive review and benchmark of 10 generic tools for differential analysis of Hi-C matrices at the interaction count level. The benchmark assesses the statistical methods, usability, and performance (in terms of precision and power) of these tools, using both real and simulated Hi-C data. Results reveal a striking variability in performance among the tools, highlighting the substantial impact of preprocessing filters and the difficulty all tools encounter in effectively controlling the false discovery rate across varying resolutions and chromosome sizes.

AVAILABILITY

The complete benchmark is available at https://forgemia.inra.fr/scales/replication-chrocodiff using processed data deposited at https://doi.org/10.57745/LR0W9R.

CONTACT

nathalie.vialaneix@inrae.fr.

Position-Sensing Graph Neural Networks: Proactively Learning Nodes Relative Positions

Most existing graph neural networks (GNNs) learn node embeddings using the framework of message passing and aggregation. Such GNNs are incapable of learning relative positions between graph nodes within a graph. To empower GNNs with the awareness of node positions, some nodes are set as anchors. Then, using the distances from a node to the anchors, GNNs can infer relative positions between nodes. However, position-aware GNNs (P-GNNs) arbitrarily select anchors, leading to compromising position awareness and feature extraction. To eliminate this compromise, we demonstrate that selecting evenly distributed and asymmetric anchors is essential. On the other hand, we show that choosing anchors that can aggregate embeddings of all the nodes within a graph is NP-complete. Therefore, devising efficient optimal algorithms in a deterministic approach is practically not feasible. To ensure position awareness and bypass NP-completeness, we propose position-sensing GNNs (PSGNNs), learning how to choose anchors in a backpropagatable fashion. Experiments verify the effectiveness of PSGNNs against state-of-the-art GNNs, substantially improving performance on various synthetic and real-world graph datasets while enjoying stable scalability. Specifically, PSGNNs on average boost area under the curve (AUC) more than 14% for pairwise node classification and 18% for link prediction over the existing state-of-the-art position-aware methods. Our source code is publicly available at: https://github.com/ZhenyueQin/PSGNN.

Artificial intelligence-enabled discovery of a RIPK3 inhibitor with neuroprotective effects in an acute glaucoma mouse model

BACKGROUND

Retinal ganglion cell (RGC) death caused by acute ocular hypertension is an important characteristic of acute glaucoma. Receptor-interacting protein kinase 3 (RIPK3) that mediates necroptosis is a potential therapeutic target for RGC death. However, the current understanding of the targeting agents and mechanisms of RIPK3 in the treatment of glaucoma remains limited. Notably, artificial intelligence (AI) technologies have significantly advanced drug discovery. This study aimed to discover RIPK3 inhibitor with AI assistance.

METHODS

An acute ocular hypertension model was used to simulate pathological ocular hypertension in vivo . We employed a series of AI methods, including large language and graph neural network models, to identify the target compounds of RIPK3. Subsequently, these target candidates were validated using molecular simulations (molecular docking, absorption, distribution, metabolism, excretion, and toxicity [ADMET] prediction, and molecular dynamics simulations) and biological experiments (Western blotting and fluorescence staining) in vitro and in vivo .

RESULTS

AI-driven drug screening techniques have the potential to greatly accelerate drug development. A compound called HG9-91-01, identified using AI methods, exerted neuroprotective effects in acute glaucoma. Our research indicates that all five candidates recommended by AI were able to protect the morphological integrity of RGC cells when exposed to hypoxia and glucose deficiency, and HG9-91-01 showed a higher cell survival rate compared to the other candidates. Furthermore, HG9-91-01 was found to protect the retinal structure and reduce the loss of retinal layers in an acute glaucoma model. It was also observed that the neuroprotective effects of HG9-91-01 were highly correlated with the inhibition of PANoptosis (apoptosis, pyroptosis, and necroptosis). Finally, we found that HG9-91-01 can regulate key proteins related to PANoptosis, indicating that this compound exerts neuroprotective effects in the retina by inhibiting the expression of proteins related to apoptosis, pyroptosis, and necroptosis.

CONCLUSION

AI-enabled drug discovery revealed that HG9-91-01 could serve as a potential treatment for acute glaucoma.

Identification of mild cognitive impairment using multimodal 3D imaging data and graph convolutional networks

Objective.Mild cognitive impairment (MCI) is a precursor stage of dementia characterized by mild cognitive decline in one or more cognitive domains, without meeting the criteria for dementia. MCI is considered a prodromal form of Alzheimer's disease (AD). Early identification of MCI is crucial for both intervention and prevention of AD. To accurately identify MCI, a novel multimodal 3D imaging data integration graph convolutional network (GCN) model is designed in this paper.Approach.The proposed model utilizes 3D-VGGNet to extract three-dimensional features from multimodal imaging data (such as structural magnetic resonance imaging and fluorodeoxyglucose positron emission tomography), which are then fused into feature vectors as the node features of a population graph. Non-imaging features of participants are combined with the multimodal imaging data to construct a population sparse graph. Additionally, in order to optimize the connectivity of the graph, we employed the pairwise attribute estimation （PAE） method to compute the edge weights based on non-imaging data, thereby enhancing the effectiveness of the graph structure. Subsequently, a population-based GCN integrates the structural and functional features of different modal images into the features of each participant for MCI classification.Main results.Experiments on the AD Neuroimaging Initiative demonstrated accuracies of 98.57%, 96.03%, and 96.83% for the normal controls (NC)-early MCI (EMCI), NC-late MCI (LMCI), and EMCI-LMCI classification tasks, respectively. The AUC, specificity, sensitivity, and F1-score are also superior to state-of-the-art models, demonstrating the effectiveness of the proposed model. Furthermore, the proposed model is applied to the ABIDE dataset for autism diagnosis, achieving an accuracy of 91.43% and outperforming the state-of-the-art models, indicating excellent generalization capabilities of the proposed model.Significance.This study demonstratesthe proposed model's ability to integrate multimodal imaging data and its excellent ability to recognize MCI. This will help achieve early warning for AD and intelligent diagnosis of other brain neurodegenerative diseases.

Accurately Predicting Barrier Heights for Radical Reactions in Solution Using Deep Graph Networks

Quantitative estimates of reaction barriers and solvent effects are essential for developing kinetic mechanisms and predicting reaction outcomes. Here, we create a new data set of 5,600 unique elementary radical reactions calculated using the M06-2X/def2-QZVP//B3LYP-D3(BJ)/def2-TZVP level of theory. A conformer search is done for each species using TPSS/def2-TZVP. Gibbs free energies of activation and of reaction for these radical reactions in 40 common solvents are obtained using COSMO-RS for solvation effects. These balanced reactions involve the elements H, C, N, O, and S, contain up to 19 heavy atoms, and have atom-mapped SMILES. All transition states are verified by an intrinsic reaction coordinate calculation. We next train a deep graph network to directly estimate the Gibbs free energy of activation and of reaction in both gas and solution phases using only the atom-mapped SMILES of the reactant and product and the SMILES of the solvent. This simple input representation avoids computationally expensive optimizations for the reactant, transition state, and product structures during inference, making our model well-suited for high-throughput predictive chemistry and quickly providing information for (retro-)synthesis planning tools. To properly measure model performance, we report results on both interpolative and extrapolative data splits and also compare to several baseline models. During training and testing, the data set is augmented by including the reverse direction of each reaction and variants with different resonance structures. After data augmentation, we have around 2 million entries to train the model, which achieves a testing set mean absolute error of 1.16 kcal mol-1 for the Gibbs free energy of activation in solution. We anticipate this model will accelerate predictions for high-throughput screening to quickly identify relevant reactions in solution, and our data set will serve as a benchmark for future studies.

MultiplexSAGE: A Multiplex Embedding Algorithm for Inter-Layer Link Prediction

Research on graph representation learning has received great attention in recent years. However, most of the studies so far have focused on the embedding of single-layer graphs. The few studies dealing with the problem of representation learning of multilayer structures rely on the strong hypothesis that the inter-layer links are known, and this limits the range of possible applications. Here we propose MultiplexSAGE, a generalization of the GraphSAGE algorithm that allows embedding multiplex networks. We show that MultiplexSAGE is capable to reconstruct both the intra-layer and the inter-layer connectivity, outperforming competing methods. Next, through a comprehensive experimental analysis, we shed light also on the performance of the embedding, both in simple and multiplex networks, showing that both the density of the graph and the randomness of the links strongly influences the quality of the embedding.

Graph Aggregating-Repelling Network: Do Not Trust All Neighbors in Heterophilic Graphs

Graph neural networks (GNNs) have demonstrated exceptional performance in processing various types of graph data, such as citation networks and social networks, etc. Although many of these GNNs prove their superiority in handling homophilic graphs, they often overlook the other kind of widespread heterophilic graphs, in which adjacent nodes tend to have different classes or dissimilar features. Recent methods attempt to address heterophilic graphs from the graph spatial domain, which try to aggregate more similar nodes or prevent dissimilar nodes with negative weights. However, they may neglect valuable heterophilic information or extract heterophilic information ineffectively, which could cause poor performance of downstream tasks on heterophilic graphs, including node classification and graph classification, etc. Hence, a novel framework named GARN is proposed to effectively extract both homophilic and heterophilic information. First, we analyze the shortcomings of most GNNs in tackling heterophilic graphs from the perspective of graph spectral and spatial theory. Then, motivated by these analyses, a Graph Aggregating-Repelling Convolution (GARC) mechanism is designed with the objective of fusing both low-pass and high-pass graph filters. Technically, it learns positive attention weights as a low-pass filter to aggregate similar adjacent nodes, and learns negative attention weights as a high-pass filter to repel dissimilar adjacent nodes. A learnable integration weight is used to adaptively fuse these two filters and balance the proportion of the learned positive and negative weights, which could control our GARC to evolve into different types of graph filters and prevent it from over-relying on high intra-class similarity. Finally, a framework named GARN is established by simply stacking several layers of GARC to evaluate its graph representation learning ability on both the node classification and image-converted graph classification tasks. Extensive experiments conducted on multiple homophilic and heterophilic graphs and complex real-world image-converted graphs indicate the effectiveness of our proposed framework and mechanism over several representative GNN baselines.

Exploiting Neighbor Effect: Conv-Agnostic GNN Framework for Graphs With Heterophily

Due to the homophily assumption in graph convolution networks (GCNs), a common consensus in the graph node classification task is that graph neural networks (GNNs) perform well on homophilic graphs but may fail on heterophilic graphs with many interclass edges. However, the previous interclass edges' perspective and related homo-ratio metrics cannot well explain the GNNs' performance under some heterophilic datasets, which implies that not all the interclass edges are harmful to GNNs. In this work, we propose a new metric based on the von Neumann entropy to reexamine the heterophily problem of GNNs and investigate the feature aggregation of interclass edges from an entire neighbor identifiable perspective. Moreover, we propose a simple yet effective Conv-Agnostic GNN framework (CAGNNs) to enhance the performance of most GNNs on the heterophily datasets by learning the neighbor effect for each node. Specifically, we first decouple the feature of each node into the discriminative feature for downstream tasks and the aggregation feature for graph convolution (GC). Then, we propose a shared mixer module to adaptively evaluate the neighbor effect of each node to incorporate the neighbor information. The proposed framework can be regarded as a plug-in component and is compatible with most GNNs. The experimental results over nine well-known benchmark datasets indicate that our framework can significantly improve performance, especially for the heterophily graphs. The average performance gain is 9.81%, 25.81%, and 20.61% compared with graph isomorphism network (GIN), graph attention network (GAT), and GCN, respectively. Extensive ablation studies and robustness analysis further verify the effectiveness, robustness, and interpretability of our framework. Code is available at https://github.com/JC-202/CAGNN.

Deep Learning for Dynamic Graphs: Models and Benchmarks

Recent progress in research on deep graph networks (DGNs) has led to a maturation of the domain of learning on graphs. Despite the growth of this research field, there are still important challenges that are yet unsolved. Specifically, there is an urge of making DGNs suitable for predictive tasks on real-world systems of interconnected entities, which evolve over time. With the aim of fostering research in the domain of dynamic graphs, first, we survey recent advantages in learning both temporal and spatial information, providing a comprehensive overview of the current state-of-the-art in the domain of representation learning for dynamic graphs. Second, we conduct a fair performance comparison among the most popular proposed approaches on node- and edge-level tasks, leveraging rigorous model selection and assessment for all the methods, thus establishing a sound baseline for evaluating new architectures and approaches.

When Do Quantum Mechanical Descriptors Help Graph Neural Networks to Predict Chemical Properties?

Deep graph neural networks are extensively utilized to predict chemical reactivity and molecular properties. However, because of the complexity of chemical space, such models often have difficulty extrapolating beyond the chemistry contained in the training set. Augmenting the model with quantum mechanical (QM) descriptors is anticipated to improve its generalizability. However, obtaining QM descriptors often requires CPU-intensive computational chemistry calculations. To identify when QM descriptors help graph neural networks predict chemical properties, we conduct a systematic investigation of the impact of atom, bond, and molecular QM descriptors on the performance of directed message passing neural networks (D-MPNNs) for predicting 16 molecular properties. The analysis surveys computational and experimental targets, as well as classification and regression tasks, and varied data set sizes from several hundred to hundreds of thousands of data points. Our results indicate that QM descriptors are mostly beneficial for D-MPNN performance on small data sets, provided that the descriptors correlate well with the targets and can be readily computed with high accuracy. Otherwise, using QM descriptors can add cost without benefit or even introduce unwanted noise that can degrade model performance. Strategic integration of QM descriptors with D-MPNN unlocks potential for physics-informed, data-efficient modeling with some interpretability that can streamline de novo drug and material designs. To facilitate the use of QM descriptors in machine learning workflows for chemistry, we provide a set of guidelines regarding when and how to best leverage QM descriptors, a high-throughput workflow to compute them, and an enhancement to Chemprop, a widely adopted open-source D-MPNN implementation for chemical property prediction.

GNN-based structural information to improve DNN-based basal ganglia segmentation in children following early brain lesion

Analyzing the basal ganglia following an early brain lesion is crucial due to their noteworthy role in sensory-motor functions. However, the segmentation of these subcortical structures on MRI is challenging in children and is further complicated by the presence of a lesion. Although current deep neural networks (DNN) perform well in segmenting subcortical brain structures in healthy brains, they lack robustness when faced with lesion variability, leading to structural inconsistencies. Given the established spatial organization of the basal ganglia, we propose enhancing the DNN-based segmentation through post-processing with a graph neural network (GNN). The GNN conducts node classification on graphs encoding both class probabilities and spatial information regarding the regions segmented by the DNN. In this study, we focus on neonatal arterial ischemic stroke (NAIS) in children. The approach is evaluated on both healthy children and children after NAIS using three DNN backbones: U-Net, UNETr, and MSGSE-Net. The results show an improvement in segmentation performance, with an increase in the median Dice score by up to 4% and a reduction in the median Hausdorff distance (HD) by up to 93% for healthy children (from 36.45 to 2.57) and up to 91% for children suffering from NAIS (from 40.64 to 3.50). The performance of the method is compared with atlas-based methods. Severe cases of neonatal stroke result in a decline in performance in the injured hemisphere, without negatively affecting the segmentation of the contra-injured hemisphere. Furthermore, the approach demonstrates resilience to small training datasets, a widespread challenge in the medical field, particularly in pediatrics and for rare pathologies.

PredCoffee: A binary classification approach specifically for coffee odor

Compared to traditional methods, using machine learning to assess or predict the odor of molecules can save costs in various aspects. Our research aims to collect molecules with coffee odor and summarize the regularity of these molecules, ultimately creating a binary classifier that can determine whether a molecule has a coffee odor. In this study, a total of 371 coffee-odor molecules and 9,700 non-coffee-odor molecules were collected. The Knowledge-guided Pre-training of Graph Transformer (KPGT), support vector machine (SVM), random forest (RF), multi-layer perceptron (MLP), and message-passing neural networks (MPNN) were used to train the data. The model with the best performance was selected as the basis of the predictor. The prediction accuracy value of the KPGT model exceeded 0.84 and the predictor has been deployed as a webserver PredCoffee.

Ethereum Phishing Scam Detection Based on Data Augmentation Method and Hybrid Graph Neural Network Model

The rapid advancement of blockchain technology has fueled the prosperity of the cryptocurrency market. Unfortunately, it has also facilitated certain criminal activities, particularly the increasing issue of phishing scams on blockchain platforms such as Ethereum. Consequently, developing an efficient phishing detection system is critical for ensuring the security and reliability of cryptocurrency transactions. However, existing methods have shortcomings in dealing with sample imbalance and effective feature extraction. To address these issues, this study proposes an Ethereum phishing scam detection method based on DA-HGNN (Data Augmentation Method and Hybrid Graph Neural Network Model), validated by real Ethereum datasets to prove its effectiveness. Initially, basic node features consisting of 11 attributes were designed. This study applied a sliding window sampling method based on node transactions for data augmentation. Since phishing nodes often initiate numerous transactions, the augmented samples tended to balance. Subsequently, the Temporal Features Extraction Module employed Conv1D (One-Dimensional Convolutional neural network) and GRU-MHA (GRU-Multi-Head Attention) models to uncover intrinsic relationships between features from the time sequences and to mine adequate local features, culminating in the extraction of temporal features. The GAE (Graph Autoencoder) concept was then leveraged, with SAGEConv (Graph SAGE Convolution) as the encoder. In the SAGEConv reconstruction module, by reconstructing the relationships between transaction graph nodes, the structural features of the nodes were learned, obtaining reconstructed node embedding representations. Ultimately, phishing fraud nodes were further identified by integrating temporal features, basic features, and embedding representations. A real Ethereum dataset was collected for evaluation, and the DA-HGNN model achieved an AUC-ROC (Area Under the Receiver Operating Characteristic Curve) of 0.994, a Recall of 0.995, and an F1-score of 0.994, outperforming existing methods and baseline models.

Antibody design using deep learning: from sequence and structure design to affinity maturation

Deep learning has achieved impressive results in various fields such as computer vision and natural language processing, making it a powerful tool in biology. Its applications now encompass cellular image classification, genomic studies and drug discovery. While drug development traditionally focused deep learning applications on small molecules, recent innovations have incorporated it in the discovery and development of biological molecules, particularly antibodies. Researchers have devised novel techniques to streamline antibody development, combining in vitro and in silico methods. In particular, computational power expedites lead candidate generation, scaling and potential antibody development against complex antigens. This survey highlights significant advancements in protein design and optimization, specifically focusing on antibodies. This includes various aspects such as design, folding, antibody-antigen interactions docking and affinity maturation.

A novel machine learning-based prediction method for patients at risk of developing depressive symptoms using a small data

The prediction of depression is a crucial area of research which makes it one of the top priorities in mental health research as it enables early intervention and can lead to higher success rates in treatment. Self-reported feelings by patients represent a valuable biomarker for predicting depression as they can be expressed in a lower-dimensional network form, offering an advantage in visualizing the interactive characteristics of depression-related feelings. Furthermore, the network form of data expresses high-dimensional data in a compact form, making the data easy to use as input for the machine learning processes. In this study, we applied the graph convolutional network (GCN) algorithm, an effective machine learning tool for handling network data, to predict depression-prone patients using the network form of self-reported log data as the input. We took a data augmentation step to expand the initially small dataset and fed the resulting data into the GCN algorithm, which achieved a high level of accuracy from 86-97% and an F1 (harmonic mean of precision and recall) score of 0.83-0.94 through three experimental cases. In these cases, the ratio of depressive cases varied, and high accuracy and F1 scores were observed in all three cases. This study not only demonstrates the potential for predicting depression-prone patients using self-reported logs as a biomarker in advance, but also shows promise in handling small data sets in the prediction, which is critical given the challenge of obtaining large datasets for biomarker research. The combination of self-reported logs and the GCN algorithm is a promising approach for predicting depression and warrants further investigation.

A Comprehensive Survey on Deep Graph Representation Learning

Graph representation learning aims to effectively encode high-dimensional sparse graph-structured data into low-dimensional dense vectors, which is a fundamental task that has been widely studied in a range of fields, including machine learning and data mining. Classic graph embedding methods follow the basic idea that the embedding vectors of interconnected nodes in the graph can still maintain a relatively close distance, thereby preserving the structural information between the nodes in the graph. However, this is sub-optimal due to: (i) traditional methods have limited model capacity which limits the learning performance; (ii) existing techniques typically rely on unsupervised learning strategies and fail to couple with the latest learning paradigms; (iii) representation learning and downstream tasks are dependent on each other which should be jointly enhanced. With the remarkable success of deep learning, deep graph representation learning has shown great potential and advantages over shallow (traditional) methods, there exist a large number of deep graph representation learning techniques have been proposed in the past decade, especially graph neural networks. In this survey, we conduct a comprehensive survey on current deep graph representation learning algorithms by proposing a new taxonomy of existing state-of-the-art literature. Specifically, we systematically summarize the essential components of graph representation learning and categorize existing approaches by the ways of graph neural network architectures and the most recent advanced learning paradigms. Moreover, this survey also provides the practical and promising applications of deep graph representation learning. Last but not least, we state new perspectives and suggest challenging directions which deserve further investigations in the future.

A Meta-Learning Approach for Training Explainable Graph Neural Networks

In this article, we investigate the degree of explainability of graph neural networks (GNNs). The existing explainers work by finding global/local subgraphs to explain a prediction, but they are applied after a GNN has already been trained. Here, we propose a meta-explainer for improving the level of explainability of a GNN directly at training time, by steering the optimization procedure toward minima that allow post hoc explainers to achieve better results, without sacrificing the overall accuracy of GNN. Our framework (called MATE, MetA-Train to Explain) jointly trains a model to solve the original task, e.g., node classification, and to provide easily processable outputs for downstream algorithms that explain the model's decisions in a human-friendly way. In particular, we meta-train the model's parameters to quickly minimize the error of an instance-level GNNExplainer trained on-the-fly on randomly sampled nodes. The final internal representation relies on a set of features that can be "better" understood by an explanation algorithm, e.g., another instance of GNNExplainer. Our model-agnostic approach can improve the explanations produced for different GNN architectures and use any instance-based explainer to drive this process. Experiments on synthetic and real-world datasets for node and graph classification show that we can produce models that are consistently easier to explain by different algorithms. Furthermore, this increase in explainability comes at no cost to the accuracy of the model.

Graph Neural Networks for Graph Drawing

Graph drawing techniques have been developed in the last few years with the purpose of producing esthetically pleasing node-link layouts. Recently, the employment of differentiable loss functions has paved the road to the massive usage of gradient descent and related optimization algorithms. In this article, we propose a novel framework for the development of Graph Neural Drawers (GNDs), machines that rely on neural computation for constructing efficient and complex maps. GND is Graph Neural Networks (GNNs) whose learning process can be driven by any provided loss function, such as the ones commonly employed in Graph Drawing. Moreover, we prove that this mechanism can be guided by loss functions computed by means of feedforward neural networks, on the basis of supervision hints that express beauty properties, like the minimization of crossing edges. In this context, we show that GNNs can nicely be enriched by positional features to deal also with unlabeled vertexes. We provide a proof-of-concept by constructing a loss function for the edge crossing and provide quantitative and qualitative comparisons among different GNN models working under the proposed framework.

Towards generalizable Graph Contrastive Learning: An information theory perspective

Graph Contrastive Learning (GCL) is increasingly employed in graph representation learning with the primary aim of learning node/graph representations from a predefined pretext task that can generalize to various downstream tasks. Meanwhile, the transition from a specific pretext task to diverse and unpredictable downstream tasks poses a significant challenge for GCL's generalization ability. Most existing GCL approaches maximize mutual information between two views derived from the original graph, either randomly or heuristically. However, the generalization ability of GCL and its theoretical principles are still less studied. In this paper, we introduce a novel metric GCL-GE, to quantify the generalization gap between predefined pretext and agnostic downstream tasks. Given the inherent intractability of GCL-GE, we leverage concepts from information theory to derive a mutual information upper bound that is independent of the downstream tasks, thus enabling the metric's optimization despite the variability in downstream tasks. Based on the theoretical insight, we propose InfoAdv, a GCL framework to directly enhance generalization by jointly optimizing GCL-GE and InfoMax. Extensive experiments validate the capability of InfoAdv to enhance performance across a wide variety of downstream tasks, demonstrating its effectiveness in improving the generalizability of GCL.

Transition Propagation Graph Neural Networks for Temporal Networks

Researchers of temporal networks (e.g., social networks and transaction networks) have been interested in mining dynamic patterns of nodes from their diverse interactions. Inspired by recently powerful graph mining methods like skip-gram models and graph neural networks (GNNs), existing approaches focus on generating temporal node embeddings sequentially with nodes' sequential interactions. However, the sequential modeling of previous approaches cannot handles the transition structure between nodes' neighbors with limited memorization capacity. In detail, an effective method for the transition structures is required to both model nodes' personalized patterns adaptively and capture node dynamics accordingly. In this article, we propose a method, namely t ransition p ropagation g raph n eural n etworks (TIP-GNN), to tackle the challenges of encoding nodes' transition structures. The proposed TIP-GNN focuses on the bilevel graph structure in temporal networks: besides the explicit interaction graph, a node's sequential interactions can also be constructed as a transition graph. Based on the bilevel graph, TIP-GNN further encodes transition structures by multistep transition propagation and distills information from neighborhoods by a bilevel graph convolution. Experimental results over various temporal networks reveal the efficiency of our TIP-GNN, with at most 7.2% improvements of accuracy on temporal link prediction. Extensive ablation studies further verify the effectiveness and limitations of the transition propagation module. Our code is available at https://github.com/doujiang-zheng/TIP-GNN.

Should we really use graph neural networks for transcriptomic prediction?

The recent development of deep learning methods have undoubtedly led to great improvement in various machine learning tasks, especially in prediction tasks. This type of methods have also been adapted to answer various problems in bioinformatics, including automatic genome annotation, artificial genome generation or phenotype prediction. In particular, a specific type of deep learning method, called graph neural network (GNN) has repeatedly been reported as a good candidate to predict phenotypes from gene expression because its ability to embed information on gene regulation or co-expression through the use of a gene network. However, up to date, no complete and reproducible benchmark has ever been performed to analyze the trade-off between cost and benefit of this approach compared to more standard (and simpler) machine learning methods. In this article, we provide such a benchmark, based on clear and comparable policies to evaluate the different methods on several datasets. Our conclusion is that GNN rarely provides a real improvement in prediction performance, especially when compared to the computation effort required by the methods. Our findings on a limited but controlled simulated dataset shows that this could be explained by the limited quality or predictive power of the input biological gene network itself.

Hand function after neonatal stroke: A graph model based on basal ganglia and thalami structure

INTRODUCTION

Neonatal arterial ischemic stroke (NAIS) is a common model to study the impact of a unilateral early brain insult on developmental brain plasticity and the appearance of long-term outcomes. Motor difficulties that may arise are typically related to poor function of the affected (contra-lesioned) hand, but surprisingly also of the ipsilesional hand. Although many longitudinal studies after NAIS have shown that predicting the occurrence of gross motor difficulties is easier, accurately predicting hand motor function (for both hands) from morphometric MRI remains complicated. The hypothesis of an association between the structural organization of the basal ganglia (BG) and thalamus with hand motor function seems intuitive given their key role in sensorimotor function. Neuroimaging studies have frequently investigated these structures to evaluate the correlation between their volumes and motor function following early brain injury. However, the results have been controversial. We hypothesize the involvement of other structural parameters.

METHOD

The study involves 35 children (mean age 7.3 years, SD 0.4) with middle cerebral artery NAIS who underwent a structural T1-weighted 3D MRI and clinical examination to assess manual dexterity using the Box and Blocks Test (BBT). Graphs are used to represent high-level structural information of the BG and thalami (volumes, elongations, distances) measured from the MRI. A graph neural network (GNN) is proposed to predict children's hand motor function through a graph regression. To reduce the impact of external factors on motor function (such as behavior and cognition), we calculate a BBT score ratio for each child and hand.

RESULTS

The results indicate a significant correlation between the score ratios predicted by our method and the actual score ratios of both hands (p < 0.05), together with a relatively high accuracy of prediction (mean L1 distance < 0.03). The structural information seems to have a different influence on each hand's motor function. The affected hand's motor function is more correlated with the volume, while the 'unaffected' hand function is more correlated with the elongation of the structures. Experiments emphasize the importance of considering the whole macrostructural organization of the basal ganglia and thalami networks, rather than the volume alone, to predict hand motor function.

CONCLUSION

There is a significant correlation between the structural characteristics of the basal ganglia/thalami and motor function in both hands. These results support the use of MRI macrostructural features of the basal ganglia and thalamus as an early biomarker for predicting motor function in both hands after early brain injury.

Enhancing Enterprise Credit Risk Assessment with Cascaded Multi-level Graph Representation Learning

The assessment of Enterprise Credit Risk (ECR) is a critical technique for investment decisions and financial regulation. Previous methods usually construct enterprise representations by credit-related indicators, such as liquidity and staff quality. However, indicators of many enterprises are not accessible, especially for the small- and medium-sized enterprises. To alleviate the indicator deficiency, graph learning based methods are proposed to enhance enterprise representation learning by the neighbor structure of enterprise graphs. However, existing methods usually only focus on pairwise relationships, and overlook the ubiquitous high-order relationships among enterprises, e.g., supply chain connecting multiple enterprises. To resolve this issue, we propose a Multi-Structure Cascaded Graph Neural Network framework (MS-CGNN) for ECR assessment. It enhances enterprise representation learning based on enterprise graph structures of different granularity, including knowledge graphs of pairwise relationships, homogeneous and heterogeneous hypergraphs of high-order relationships. To distinguish influences of different types of hyperedges, MS-CGNN redefine new type-dependent hyperedge weight matrices for heterogeneous hypergraph convolutions. Experimental results show that MS-CGNN achieves state-of-the-art performance on real-world ECR datasets.

Self-tuning Hamiltonian Monte Carlo for accelerated sampling

The performance of Hamiltonian Monte Carlo simulations crucially depends on both the integration timestep and the number of integration steps. We present an adaptive general-purpose framework to automatically tune such parameters based on a local loss function that promotes the fast exploration of phase space. We show that a good correspondence between loss and autocorrelation time can be established, allowing for gradient-based optimization using a fully differentiable set-up. The loss is constructed in such a way that it also allows for gradient-driven learning of a distribution over the number of integration steps. Our approach is demonstrated for the one-dimensional harmonic oscillator and alanine dipeptide, a small protein commonly used as a test case for simulation methods. Through the application to the harmonic oscillator, we highlight the importance of not using a fixed timestep to avoid a rugged loss surface with many local minima, otherwise trapping the optimization. In the case of alanine dipeptide, by tuning the only free parameter of our loss definition, we find a good correspondence between it and the autocorrelation times, resulting in a >100 fold speedup in the optimization of simulation parameters compared to a grid search. For this system, we also extend the integrator to allow for atom-dependent timesteps, providing a further reduction of 25% in autocorrelation times.

Explaining Deep Graph Networks via Input Perturbation

Deep graph networks (DGNs) are a family of machine learning models for structured data which are finding heavy application in life sciences (drug repurposing, molecular property predictions) and on social network data (recommendation systems). The privacy and safety-critical nature of such domains motivates the need for developing effective explainability methods for this family of models. So far, progress in this field has been challenged by the combinatorial nature and complexity of graph structures. In this respect, we present a novel local explanation framework specifically tailored to graph data and DGNs. Our approach leverages reinforcement learning to generate meaningful local perturbations of the input graph, whose prediction we seek an interpretation for. These perturbed data points are obtained by optimizing a multiobjective score taking into account similarities both at a structural level as well as at the level of the deep model outputs. By this means, we are able to populate a set of informative neighboring samples for the query graph, which is then used to fit an interpretable model for the predictive behavior of the deep network locally to the query graph prediction. We show the effectiveness of the proposed explainer by a qualitative analysis on two chemistry datasets, TOX21 and Estimated SOLubility (ESOL) and by quantitative results on a benchmark dataset for explanations, CYCLIQ.

Exploiting the structure of biochemical pathways to investigate dynamical properties with neural networks for graphs

MOTIVATION

Dynamical properties of biochemical pathways (BPs) help in understanding the functioning of living cells. Their in silico assessment requires simulating a dynamical system with a large number of parameters such as kinetic constants and species concentrations. Such simulations are based on numerical methods that can be time-expensive for large BPs. Moreover, parameters are often unknown and need to be estimated.

RESULTS

We developed a framework for the prediction of dynamical properties of BPs directly from the structure of their graph representation. We represent BPs as Petri nets, which can be automatically generated, for instance, from standard SBML representations. The core of the framework is a neural network for graphs that extracts relevant information directly from the Petri net structure and exploits them to learn the association with the desired dynamical property. We show experimentally that the proposed approach reliably predicts a range of diverse dynamical properties (robustness, monotonicity, and sensitivity) while being faster than numerical methods at prediction time. In synergy with the neural network models, we propose a methodology based on Petri nets arc knock-out that allows the role of each molecule in the occurrence of a certain dynamical property to be better elucidated. The methodology also provides insights useful for interpreting the predictions made by the model. The results support the conjecture often considered in the context of systems biology that the BP structure plays a primary role in the assessment of its dynamical properties.

AVAILABILITY AND IMPLEMENTATION

https://github.com/marcopodda/petri-bio (code), https://zenodo.org/record/7610382 (data).

Drop edges and adapt: A fairness enforcing fine-tuning for graph neural networks

The rise of graph representation learning as the primary solution for many different network science tasks led to a surge of interest in the fairness of this family of methods. Link prediction, in particular, has a substantial social impact. However, link prediction algorithms tend to increase the segregation in social networks by disfavouring the links between individuals in specific demographic groups. This paper proposes a novel way to enforce fairness on graph neural networks with a fine-tuning strategy. We Drop the unfair Edges and, simultaneously, we Adapt the model's parameters to those modifications, DEA in short. We introduce two covariance-based constraints designed explicitly for the link prediction task. We use these constraints to guide the optimization process responsible for learning the new 'fair' adjacency matrix. One novelty of DEA is that we can use a discrete yet learnable adjacency matrix in our fine-tuning. We demonstrate the effectiveness of our approach on five real-world datasets and show that we can improve both the accuracy and the fairness of the link prediction tasks. In addition, we present an in-depth ablation study demonstrating that our training algorithm for the adjacency matrix can be used to improve link prediction performances during training. Finally, we compute the relevance of each component of our framework to show that the combination of both the constraints and the training of the adjacency matrix leads to optimal performances.

DyVGRNN: DYnamic mixture Variational Graph Recurrent Neural Networks

Although graph representation learning has been studied extensively in static graph settings, dynamic graphs are less investigated in this context. This paper proposes a novel integrated variational framework called DYnamic mixture Variational Graph Recurrent Neural Networks (DyVGRNN), which consists of extra latent random variables in structural and temporal modelling. Our proposed framework comprises an integration of Variational Graph Auto-Encoder (VGAE) and Graph Recurrent Neural Network (GRNN) by exploiting a novel attention mechanism. The Gaussian Mixture Model (GMM) and the VGAE framework are combined in DyVGRNN to model the multimodal nature of data, which enhances performance. To consider the significance of time steps, our proposed method incorporates an attention-based module. The experimental results demonstrate that our method greatly outperforms state-of-the-art dynamic graph representation learning methods in terms of link prediction and clustering.2.

Generating post-hoc explanations for Skip-gram-based node embeddings by identifying important nodes with bridgeness

Node representation learning in a network is an important machine learning technique for encoding relational information in a continuous vector space while preserving the inherent properties and structures of the network. Recently, unsupervised node embedding methods such as DeepWalk (Perozzi et al., 2014), LINE (Tang et al., 2015), struc2vec (Ribeiro et al., 2017), PTE (Tang et al., 2015), UserItem2vec (Wu et al., 2020), and RWJBG (Li et al., 2021) have emerged from the Skip-gram model (Mikolov et al., 2013) and perform better performance in several downstream tasks such as node classification and link prediction than the existing relational models. However, providing post-hoc explanations of unsupervised embeddings remains a challenging problem because of the lack of explanation methods and theoretical studies applicable for embeddings. In this paper, we first show that global explanations to the Skip-gram-based embeddings can be found by computing bridgeness under a spectral cluster-aware local perturbation. Moreover, a novel gradient-based explanation method, which we call GRAPH-wGD, is proposed that allows the top-q global explanations about learned graph embedding vectors more efficiently. Experiments show that the ranking of nodes by scores using GRAPH-wGD is highly correlated with true bridgeness scores. We also observe that the top-q node-level explanations selected by GRAPH-wGD have higher importance scores and produce more changes in class label prediction when perturbed, compared with the nodes selected by recent alternatives, using five real-world graphs.

AttentionDTA: Drug-Target Binding Affinity Prediction by Sequence-Based Deep Learning With Attention Mechanism

The identification of drug-target relations (DTRs) is substantial in drug development. A large number of methods treat DTRs as drug-target interactions (DTIs), a binary classification problem. The main drawback of these methods are the lack of reliable negative samples and the absence of many important aspects of DTR, including their dose dependence and quantitative affinities. With increasing number of publications of drug-protein binding affinity data recently, DTRs prediction can be viewed as a regression problem of drug-target affinities (DTAs) which reflects how tightly the drug binds to the target and can present more detailed and specific information than DTIs. The growth of affinity data enables the use of deep learning architectures, which have been shown to be among the state-of-the-art methods in binding affinity prediction. Although relatively effective, due to the black-box nature of deep learning, these models are less biologically interpretable. In this study, we proposed a deep learning-based model, named AttentionDTA, which uses attention mechanism to predict DTAs. Different from the models using 3D structures of drug-target complexes or graph representation of drugs and proteins, the novelty of our work is to use attention mechanism to focus on key subsequences which are important in drug and protein sequences when predicting its affinity. We use two separate one-dimensional Convolution Neural Networks (1D-CNNs) to extract the semantic information of drug's SMILES string and protein's amino acid sequence. Furthermore, a two-side multi-head attention mechanism is developed and embedded to our model to explore the relationship between drug features and protein features. We evaluate our model on three established DTA benchmark datasets, Davis, Metz, and KIBA. AttentionDTA outperforms the state-of-the-art deep learning methods under different evaluation metrics. The results show that the attention-based model can effectively extract protein features related to drug information and drug features related to protein information to better predict drug target affinities. It is worth mentioning that we test our model on IC50 dataset, which provides the binding sites between drugs and proteins, to evaluate the ability of our model to locate binding sites. Finally, we visualize the attention weight to demonstrate the biological significance of the model. The source code of AttentionDTA can be downloaded from https://github.com/zhaoqichang/AttentionDTA_TCBB.

Echo state graph neural networks with analogue random resistive memory arrays

Recent years have witnessed a surge of interest in learning representations of graph-structured data, with applications from social networks to drug discovery. However, graph neural networks, the machine learning models for handling graph-structured data, face significant challenges when running on conventional digital hardware, including the slowdown of Moore’s law due to transistor scaling limits and the von Neumann bottleneck incurred by physically separated memory and processing units, as well as a high training cost. Here we present a hardware–software co-design to address these challenges, by designing an echo state graph neural network based on random resistive memory arrays, which are built from low-cost, nanoscale and stackable resistors for efficient in-memory computing. This approach leverages the intrinsic stochasticity of dielectric breakdown in resistive switching to implement random projections in hardware for an echo state network that effectively minimizes the training complexity thanks to its fixed and random weights. The system demonstrates state-of-the-art performance on both graph classification using the MUTAG and COLLAB datasets and node classification using the CORA dataset, achieving 2.16×, 35.42× and 40.37× improvements in energy efficiency for a projected random resistive memory-based hybrid analogue–digital system over a state-of-the-art graphics processing unit and 99.35%, 99.99% and 91.40% reductions of backward pass complexity compared with conventional graph learning. The results point to a promising direction for next-generation artificial intelligence systems for graph learning.

Graph metric learning quantifies morphological differences between two genotypes of shoot apical meristem cells in Arabidopsis

We present a method for learning 'spectrally descriptive' edge weights for graphs. We generalize a previously known distance measure on graphs (graph diffusion distance [GDD]), thereby allowing it to be tuned to minimize an arbitrary loss function. Because all steps involved in calculating this modified GDD are differentiable, we demonstrate that it is possible for a small neural network model to learn edge weights which minimize loss. We apply this method to discriminate between graphs constructed from shoot apical meristem images of two genotypes of Arabidopsis thaliana specimens: wild-type and trm678 triple mutants with cell division phenotype. Training edge weights and kernel parameters with contrastive loss produce a learned distance metric with large margins between these graph categories. We demonstrate this by showing improved performance of a simple k -nearest-neighbour classifier on the learned distance matrix. We also demonstrate a further application of this method to biological image analysis. Once trained, we use our model to compute the distance between the biological graphs and a set of graphs output by a cell division simulator. Comparing simulated cell division graphs to biological ones allows us to identify simulation parameter regimes which characterize mutant versus wild-type Arabidopsis cells. We find that trm678 mutant cells are characterized by increased randomness of division planes and decreased ability to avoid previous vertices between cell walls.

Transition1x - a dataset for building generalizable reactive machine learning potentials

Machine Learning (ML) models have, in contrast to their usefulness in molecular dynamics studies, had limited success as surrogate potentials for reaction barrier search. This is primarily because available datasets for training ML models on small molecular systems almost exclusively contain configurations at or near equilibrium. In this work, we present the dataset Transition1x containing 9.6 million Density Functional Theory (DFT) calculations of forces and energies of molecular configurations on and around reaction pathways at the ωB97x/6-31 G(d) level of theory. The data was generated by running Nudged Elastic Band (NEB) with DFT on 10k organic reactions of various types while saving intermediate calculations. We train equivariant graph message-passing neural network models on Transition1x and cross-validate on the popular ANI1x and QM9 datasets. We show that ML models cannot learn features in transition state regions solely by training on hitherto popular benchmark datasets. Transition1x is a new challenging benchmark that will provide an important step towards developing next-generation ML force fields that also work far away from equilibrium configurations and reactive systems.

Graph neural network-based cell switching for energy optimization in ultra-dense heterogeneous networks

The development of ultra-dense heterogeneous networks (HetNets) will cause a significant rise in energy consumption with large-scale base station (BS) deployments, requiring cellular networks to be more energy efficient to reduce operational expense and promote sustainability. Cell switching is an effective method to achieve the energy efficiency goals, but traditional heuristic cell switching algorithms are computationally demanding with limited generalization abilities for ultra-dense HetNet applications, motivating the usage of machine learning techniques for adaptive cell switching. Graph neural networks (GNNs) are powerful deep learning models with strong generalization abilities but receive little attention for cell switching. This paper proposes a GNN-based cell switching solution (GBCSS) that has a smaller computational complexity than existing heuristic algorithms. The presented performance evaluation uses the Milan telecommunication dataset based on real-world call detail records, comparing GBCSS with a traditional exhaustive search (ES) algorithm, a state-of-the-art learning-based algorithm, and the baseline without cell switching. Results indicate that GBCSS achieves a 10.41% energy efficiency gain when compared with the baseline and achieves 75.76% of the optimal performance obtained with ES algorithm. The results also demonstrate GBCSS' significant scalability and generalization abilities to differing load conditions and the number of BSs, suggesting this approach is well-suited to ultra-dense HetNet deployment.

Eleven quick tips for data cleaning and feature engineering

Applying computational statistics or machine learning methods to data is a key component of many scientific studies, in any field, but alone might not be sufficient to generate robust and reliable outcomes and results. Before applying any discovery method, preprocessing steps are necessary to prepare the data to the computational analysis. In this framework, data cleaning and feature engineering are key pillars of any scientific study involving data analysis and that should be adequately designed and performed since the first phases of the project. We call "feature" a variable describing a particular trait of a person or an observation, recorded usually as a column in a dataset. Even if pivotal, these data cleaning and feature engineering steps sometimes are done poorly or inefficiently, especially by beginners and unexperienced researchers. For this reason, we propose here our quick tips for data cleaning and feature engineering on how to carry out these important preprocessing steps correctly avoiding common mistakes and pitfalls. Although we designed these guidelines with bioinformatics and health informatics scenarios in mind, we believe they can more in general be applied to any scientific area. We therefore target these guidelines to any researcher or practitioners wanting to perform data cleaning or feature engineering. We believe our simple recommendations can help researchers and scholars perform better computational analyses that can lead, in turn, to more solid outcomes and more reliable discoveries.

On the application of population-based structural health monitoring in aerospace engineering

One of the major obstacles to the widespread uptake of data-based Structural Health Monitoring so far, has been the lack of damage-state data for the (mostly high-value) structures of interest. To address this issue, a methodology for sharing data and models between structures has been developed-Population-Based Structural Health Monitoring (PBSHM). PBSHM works on the principle that, if populations of structures are sufficiently similar, or share sections which can be considered similar, then data and models can be shared between them for use in diagnostic inference. The PBSHM methodology therefore relies on two key components: firstly, identifying whether structures are sufficiently similar for successful transfer of diagnostics; this is achieved by the use of an abstract representation of structures. Secondly, machine learning techniques are exploited to effectively transfer information between the structures in a way that improves damage detection and classification across the whole population. Although PBSHM has been conceived to deal with large and general classes of structures, much of the detailed developments presented so far have concerned bridges; the aim of this paper is to provide similarly detailed discussions in the aerospace context. The overview here will examine data transfer between aircraft components, as well as illustrating how one might construct an abstract representation of a full aircraft.

GNN-SubNet: disease subnetwork detection with explainable graph neural networks

MOTIVATION

The tremendous success of graphical neural networks (GNNs) already had a major impact on systems biology research. For example, GNNs are currently being used for drug target recognition in protein-drug interaction networks, as well as for cancer gene discovery and more. Important aspects whose practical relevance is often underestimated are comprehensibility, interpretability and explainability.

RESULTS

In this work, we present a novel graph-based deep learning framework for disease subnetwork detection via explainable GNNs. Each patient is represented by the topology of a protein-protein interaction (PPI) network, and the nodes are enriched with multi-omics features from gene expression and DNA methylation. In addition, we propose a modification of the GNNexplainer that provides model-wide explanations for improved disease subnetwork detection.

AVAILABILITY AND IMPLEMENTATION

The proposed methods and tools are implemented in the GNN-SubNet Python package, which we have made available on our GitHub for the international research community (https://github.com/pievos101/GNN-SubNet).

SUPPLEMENTARY INFORMATION

Supplementary data are available at Bioinformatics online.

An Early Warning System for Earthquake Prediction from Seismic Data Using Batch Normalized Graph Convolutional Neural Network with Attention Mechanism (BNGCNNATT)

Earthquakes threaten people, homes, and infrastructure. Early warning systems provide prior warning of oncoming significant shaking to decrease seismic risk by providing location, magnitude, and depth information of the event. Their usefulness depends on how soon a strong shake begins after the warning. In this article, the authors implement a deep learning model for predicting earthquakes. This model is based on a graph convolutional neural network with batch normalization and attention mechanism techniques that can successfully predict the depth and magnitude of an earthquake event at any number of seismic stations in any number of locations. After preprocessing the waveform data, CNN extracts the feature map. Attention mechanism is used to focus on important features. The batch normalization technique takes place in batches for stable and faster training of the model by adding an extra layer. GNN with extracted features and event location information predicts the event information accurately. We test the proposed model on two datasets from Japan and Alaska, which have different seismic dynamics. The proposed model achieves 2.8 and 4.0 RMSE values in Alaska and Japan for magnitude prediction, and 2.87 and 2.66 RMSE values for depth prediction. Low RMSE values show that the proposed model significantly outperforms the three baseline models on both datasets to provide an accurate estimation of the depth and magnitude of small, medium, and large-magnitude events.

Location-aware convolutional neural networks for graph classification

Graph patterns play a critical role in various graph classification tasks, e.g., chemical patterns often determine the properties of molecular graphs. Researchers devote themselves to adapting Convolutional Neural Networks (CNNs) to graph classification due to their powerful capability in pattern learning. The varying numbers of neighbor nodes and the lack of canonical order of nodes on graphs pose challenges in constructing receptive fields for CNNs. Existing methods generally follow a heuristic ranking-based framework, which constructs receptive fields by selecting a fixed number of nodes and dropping the others according to predetermined rules. However, such methods may lose important structure information through dropping nodes, and they also cannot learn task-oriented graph patterns. In this paper, we propose a Location learning-based Convolutional Neural Networks (LCNN) for graph classification. LCNN constructs receptive fields by learning the location of each node according to its embedding that contains structures and features information, then standard CNNs are applied to capture graph patterns. Such a location learning mechanism not only retains the information of all nodes, but also provides the ability for task-oriented pattern learning. Experimental results show the effectiveness of the proposed LCNN, and visualization results further illustrate the valid pattern learning ability of our method for graph classification.

Early Earthquake Detection Using Batch Normalization Graph Convolutional Neural Network (BNGCNN)

Earthquake is a major hazard to humans, buildings, and infrastructure. Early warning systems should detect an earthquake and issue a warning with earthquake information such as location, magnitude, and depth. Earthquake detection from raw waveform data using deep learning models such as graph neural networks (GNN) is becoming an important research area. The multilayered structure of the GNN with a number of epochs takes more training time. It is also hard to train the model with saturating nonlinearities. The batch normalization technique is applied to each mini-batch to reduce epochs in training and obtain a steady distribution of activation values. It improves model training and prediction accuracy. This study proposes a deep learning model batch normalization graph convolutional neural network (BNGCNN) for early earthquake detection. It consists of two main components: CNN and GNN. Input to the CNN model is multi-station and three-component waveform data with magnitude ≥3.0 were collected from January 2000 to January 2015 for Southern California. The extracted features of CNN are appended with location information and input to GNN model for earthquake detection. After hyperparameter tuning of the BNGCNN, when testing and evaluating the model on the Southern California dataset, our method shows promising results to the baseline model GNN by obtaining a low error rate to predict the magnitude, depth, and location of an earthquake.

Fast Predictions of Reaction Barrier Heights: Toward Coupled-Cluster Accuracy

Quantitative estimates of reaction barriers are essential for developing kinetic mechanisms and predicting reaction outcomes. However, the lack of experimental data and the steep scaling of accurate quantum calculations often hinder the ability to obtain reliable kinetic values. Here, we train a directed message passing neural network on nearly 24,000 diverse gas-phase reactions calculated at CCSD(T)-F12a/cc-pVDZ-F12//ωB97X-D3/def2-TZVP. Our model uses 75% fewer parameters than previous studies, an improved reaction representation, and proper data splits to accurately estimate performance on unseen reactions. Using information from only the reactant and product, our model quickly predicts barrier heights with a testing MAE of 2.6 kcal mol^-1 relative to the coupled-cluster data, making it more accurate than a good density functional theory calculation. Furthermore, our results show that future modeling efforts to estimate reaction properties would significantly benefit from fine-tuning calibration using a transfer learning technique. We anticipate this model will accelerate and improve kinetic predictions for small molecule chemistry.

Study-GNN: A Novel Pipeline for Student Performance Prediction Based on Multi-Topology Graph Neural Networks

Student performance prediction has attracted increasing attention in the field of educational data mining, or more broadly, intelligent education or “AI + education”. Accurate performance prediction plays a significant role in solving the problem of a student dropping out, promoting personalized learning and improving teaching efficiency, etc. Traditional student performance prediction methods usually ignore the potential (underlying) relationship among students. In this paper, we use graph structure to reflect the students’ relationships and propose a novel pipeline for student performance prediction based on newly-developed multi-topology graph neural networks (termed MTGNN). In particular, we propose various ways for graph construction based on similarity learning using different distance metrics. Based on the multiple graphs of different topologies, we design an MTGNN module, as a key module in the pipeline, to deal with the semi-supervised node classification problem where each node represents a student (and the node label is the student’s performance, e.g., Pass/Fail/Withdrawal). An attention-based method is developed to produce the unified graph representation in MTGNN. The effectiveness of the proposed pipeline is verified in a case study, where a real-world educational dataset and several existing approaches are used for performance comparison. The experiment results show that, compared with some traditional machine learning methods and the vanilla graph convolutional network with only a single graph topology, our proposed pipeline works effectively and favorably in student performance prediction.

Graph Neural Network for Context-Aware Recommendation

Recommendation problems are naturally tackled as a link prediction task in a bipartite graph between user and item nodes, labelled with rating information on edges. To provide personal recommendations and improve the performance of the recommender system, it is necessary to integrate side information along with user-item interactions. The integration of context is a key success factor in recommendation systems because it allows catering for user preferences and opinions, especially when this pertains to the circumstances surrounding the interaction between users and items. In this paper, we propose a context-aware Graph Convolutional Matrix Completion which captures structural information and integrates the user’s opinion on items along with the surrounding context on edges and static features of user and item nodes. Our graph encoder produces user and item representations with respect to context, features and opinion. The decoder takes the aggregated embeddings to predict the user-item score considering the surrounding context. We have evaluated the performance of our model on 14 five publicly available datasets and compared it with state-of-the-art algorithms. Throughout this we show how it can effectively integrate user opinion along with surrounding context to produce a final node representation which is aware of the favourite circumstances of the particular node.

Controlling astrocyte-mediated synaptic pruning signals for schizophrenia drug repurposing with deep graph networks

Schizophrenia is a debilitating psychiatric disorder, leading to both physical and social morbidity. Worldwide 1% of the population is struggling with the disease, with 100,000 new cases annually only in the United States. Despite its importance, the goal of finding effective treatments for schizophrenia remains a challenging task, and previous work conducted expensive large-scale phenotypic screens. This work investigates the benefits of Machine Learning for graphs to optimize drug phenotypic screens and predict compounds that mitigate abnormal brain reduction induced by excessive glial phagocytic activity in schizophrenia subjects. Given a compound and its concentration as input, we propose a method that predicts a score associated with three possible compound effects, i.e., reduce, increase, or not influence phagocytosis. We leverage a high-throughput screening to prove experimentally that our method achieves good generalization capabilities. The screening involves 2218 compounds at five different concentrations. Then, we analyze the usability of our approach in a practical setting, i.e., prioritizing the selection of compounds in the SWEETLEAD library. We provide a list of 64 compounds from the library that have the most potential clinical utility for glial phagocytosis mitigation. Lastly, we propose a novel approach to computationally validate their utility as possible therapies for schizophrenia.