Efficiently inferring community structure in bipartite networks

Complements and competitors: Examining technological co-diffusion and relatedness on a collaborative coding platform

Diffusive and contagious processes spread in the context of one another in connected populations. Diffusions may be more likely to pass through portions of a network where compatible diffusions are already present. We examine this by incorporating the concept of "relatedness" from the economic complexity literature into a network co-diffusion model. Building on the "product space" concept used in this work, we consider technologies themselves as nodes in "product networks," where edges define relationships between products. Specifically, coding languages on GitHub, an online platform for collaborative coding, are considered. From rates of language co-occurrence in coding projects, we calculate rates of functional cohesion and functional equivalence for each pair of languages. From rates of how individuals adopt and abandon coding languages over time, we calculate measures of complementary diffusion and substitutive diffusion for each pair of languages relative to one another. Consistent with the principle of relatedness, network regression techniques (MR-QAP) reveal strong evidence that functional cohesion positively predicts complementary diffusion. We also find limited evidence that functional equivalence predicts substitutive (competitive) diffusion. Results support the broader finding that functional dependencies between diffusive processes will dictate how said processes spread relative to one another across a population of potential adopters.

Community detection in large hypergraphs

Hypergraphs, describing networks where interactions take place among any number of units, are a natural tool to model many real-world social and biological systems. Here, we propose a principled framework to model the organization of higher-order data. Our approach recovers community structure with accuracy exceeding that of currently available state-of-the-art algorithms, as tested in synthetic benchmarks with both hard and overlapping ground-truth partitions. Our model is flexible and allows capturing both assortative and disassortative community structures. Moreover, our method scales orders of magnitude faster than competing algorithms, making it suitable for the analysis of very large hypergraphs, containing millions of nodes and interactions among thousands of nodes. Our work constitutes a practical and general tool for hypergraph analysis, broadening our understanding of the organization of real-world higher-order systems.

The making of the oral microbiome in Agta hunter-gatherers

Ecological and genetic factors have influenced the composition of the human microbiome during our evolutionary history. We analysed the oral microbiota of the Agta, a hunter-gatherer population where some members have adopted an agricultural diet. We show that age is the strongest factor modulating the microbiome, probably through immunosenescence since we identified an increase in the number of species classified as pathogens with age. We also characterised biological and cultural processes generating sexual dimorphism in the oral microbiome. A small subset of oral bacteria is influenced by the host genome, linking host collagen genes to bacterial biofilm formation. Our data also suggest that shifting from a fish/meat diet to a rice-rich diet transforms their microbiome, mirroring the Neolithic transition. All of these factors have implications in the epidemiology of oral diseases. Thus, the human oral microbiome is multifactorial and shaped by various ecological and social factors that modify the oral environment.

Software Components and Product Variety in a Platform Ecosystem: A Dynamic Network Analysis of WordPress

Software components such as application programming interfaces (APIs) provided by external developers are vital to online digital platforms. Although APIs generally increase the variety of products according to anecdote, the precise relationship between the categories of APIs and product variety is not yet known. We find that APIs, regarding their use frequency, are categorized into three groups. The core is a group of frequently used APIs, whereas the periphery is a group of sparsely used APIs. In a large and mature platform ecosystem, an additional group of APIs, the regular core, mainly provided by third-party developers, emerges. APIs in the regular core are the main driver of product variety. However, we also find that the strength of this effect diminishes in a newly created product category when most of the new products are built by duplicating the usage of APIs from other products. A platform owner can stimulate developers’ creativity by acting as a bridge between digital product providers and third-party developers. It can collect functional needs from third-party developers and then share them with product providers. Therefore, the latter can build APIs that developers need.

The collective vs individual nature of mountaineering: a network and simplicial approach

Mountaineering is a sport of contrary forces: teamwork plays a large role in mental fortitude and skills, but the actual act of climbing, and indeed survival, is largely individualistic. This work studies the effects of the structure and topology of relationships within climbers on the level of cooperation and success. It does so using simplicial complexes, where relationships between climbers are captured through simplices that correspond to joint previous expeditions with dimension given by the number of climbers minus one and weight given by the number of occurrences of the simplex. First, this analysis establishes the importance of relationships in mountaineering and shows that chances of failure to summit reduce drastically when climbing with repeated partners. From a climber-centric perspective, it finds that climbers that belong to simplices with large dimension were more likely to be successful, across all experience levels. Then, the distribution of relationships within a group is explored to categorize collective human behavior in expeditions, on a spectrum from polarized to cooperative. Expeditions containing simplices with large dimension, and usually low weight (weak relationships), implying that a large number of people participated in a small number of joint expeditions, tended to be more cooperative, improving chances of success of all members of the group, not just those that were part of the simplex. On the other hand, the existence of small, usually high weight (i.e., strong relationships) simplices, subgroups lead to a polarized style where climbers that were not a part of the subgroup were less likely to succeed. Lastly, this work examines the effects of individual features (such as age, gender, climber experience etc.) and expedition-wide factors (number of camps, total number of days etc.) that are more important determiners of success in individualistic and cooperative expeditions respectively. Centrality indicates that individual features of youth and oxygen use while ascending are the most important predictors of success. Of expedition-wide factors, the expedition size and number of expedition days are found to be strongly correlated with success rate.

An empirical Bayes approach to stochastic blockmodels and graphons: shrinkage estimation and model selection

The graphon (W-graph), including the stochastic block model as a special case, has been widely used in modeling and analyzing network data. Estimation of the graphon function has gained a lot of recent research interests. Most existing works focus on inference in the latent space of the model, while adopting simple maximum likelihood or Bayesian estimates for the graphon or connectivity parameters given the identified latent variables. In this work, we propose a hierarchical model and develop a novel empirical Bayes estimate of the connectivity matrix of a stochastic block model to approximate the graphon function. Based on our hierarchical model, we further introduce a new model selection criterion for choosing the number of communities. Numerical results on extensive simulations and two well-annotated social networks demonstrate the superiority of our approach in terms of parameter estimation and model selection.

Comodularity and detection of co-communities

This paper introduces the notion of comodularity, to cocluster observations of bipartite networks into co-communities. The task of coclustering is to group together nodes of one type with nodes of another type, according to the interactions that are the most similar. The measure of comodularity is introduced to assess the strength of co-communities, as well as to arrange the representation of nodes and clusters for visualization, and to define an objective function for optimization. We demonstrate the usefulness of our proposed methodology on simulated data, and with examples from genomics and consumer-product reviews.

Graph Matching between Bipartite and Unipartite Networks: to Collapse, or not to Collapse, that is the Question

Graph matching consists of aligning the vertices of two unlabeled graphs in order to maximize the shared structure across networks; when the graphs are unipartite, this is commonly formulated as minimizing their edge disagreements. In this paper we address the common setting in which one of the graphs to match is a bipartite network and one is unipartite. Commonly, the bipartite networks are collapsed or projected into a unipartite graph, and graph matching proceeds as in the classical setting. This potentially leads to noisy edge estimates and loss of information. We formulate the graph matching problem between a bipartite and a unipartite graph using an undirected graphical model, and introduce methods to find the alignment with this model without collapsing. We theoretically demonstrate that our methodology is consistent, and provide non-asymptotic conditions that ensure exact recovery of the matching solution. In simulations and real data examples, we show how our methods can result in a more accurate matching than the naive approach of transforming the bipartite networks into unipartite, and we demonstrate the performance gains achieved by our method in simulated and real data networks, including a co-authorship-citation network pair, and brain structural and functional data.

Graph-based open-ended survey on concerns related to COVID-19

The COVID-19 pandemic is an unprecedented public health crisis with broad social and economic consequences. We conducted four surveys between April and August 2020 using the graph-based open-ended survey (GOS) framework, and investigated the most pressing concerns and issues for the general public in Japan. The GOS framework is a hybrid of the two traditional survey frameworks that allows respondents to post their opinions in a free-format style, which can subsequently serve as one of the choice items for other respondents, just as in a multiple-choice survey. As a result, this framework generates an opinion graph that relates opinions and respondents. We can also construct annotated opinion graphs to achieve a higher resolution. By clustering the annotated opinion graphs, we revealed the characteristic evolution of the response patterns as well as the interconnectedness and multi-faceted nature of opinions. Substantively, our notable finding is that “social pressure,” not “infection risk,” was one of the major concerns of our respondents. Social pressure refers to criticism and discrimination that they anticipate receiving from others should they contract COVID-19. It is possible that the collectivist nature of Japanese culture coupled with the government’s policy of relying on personal responsibility to combat COVID-19 explains some of the above findings, as the latter has led to the emergence of vigilantes. The presence of mutual surveillance can contribute to growing skepticism toward others as well as fear of ostracism, which may have negative consequences at both the societal and individual levels.

Examining the sources of evidence in e-cigarette policy recommendations: A citation network analysis of international public health recommendations

BACKGROUND

Public health policies and recommendations aim to be informed by the best available evidence. Evidence underpinning e-cigarettes policy recommendations has been necessarily limited due to the novelty of the technology and the lack of long-term epidemiological studies and trials. Some public health bodies have actively encouraged e-cigarette use whilst others have raised concerns over introducing new health risks and renormalising tobacco smoking. Using citation network analysis we investigated the author conflicts of interest and study funding statements within sources of evidence used by public health bodies when making recommendations about e-cigarette policy.

METHODS

We conducted citation network analysis of public health recommendation documents across four purposively selected diverse jurisdictions: WHO, UK, Australia, and USA. We extracted all citations from 15 public health recommendation documents, with more detailed data collected for influential citations (used in 3+ recommendation documents). We analysed the relationships between the sources of evidence used across jurisdictions using block modelling to determine if similar groups of documents were used across different jurisdictions. We assessed the frequency and nature of conflicts of interest.

RESULTS

1700 unique citations were included across the 15 public health recommendation documents, with zero to 923 citations per document (median = 63, IQR = 7.5-132). The evidence base underpinning public health recommendations did not systematically differ across jurisdictions. Of the 1700 citations included, the majority were journal articles (n = 1179). Across 1081 journal articles published between 1998-2018, 200 declared a conflict of interest, 288 contained no mention of conflicts of interest, and 593 declared none. Conflicts of interest were reported with tobacco (3%; n = 37 journal articles of 1081), e-cigarette (7%; n = 72), and pharmaceutical companies (12%; n = 127), with such conflicts present even in the most recent years. There were 53 influential citations, the most common study type was basic science research without human subjects (e.g. examination of aerosols and e-liquids) (n = 18) followed by systematic review (n = 10); with randomised control trial being least common (n = 4). Network analysis identified clusters of highly-cited articles with a higher prevalence of conflicts of interest.

CONCLUSION

Public health bodies across different jurisdictions drew upon similar sources of evidence, despite articulating different policy approaches to e-cigarettes. The evidence drawn upon, including the most influential evidence, contained substantial conflicts of interest (including relationships with e-cigarette and tobacco industries). Processes to explicitly manage conflicts of interest arising from the underlying evidence base may be required when developing public health recommendations.

Generative hypergraph clustering: From blockmodels to modularity

Hypergraphs are a natural modeling paradigm for networked systems with multiway interactions. A standard task in network analysis is the identification of closely related or densely interconnected nodes. We propose a probabilistic generative model of clustered hypergraphs with heterogeneous node degrees and edge sizes. Approximate maximum likelihood inference in this model leads to a clustering objective that generalizes the popular modularity objective for graphs. From this, we derive an inference algorithm that generalizes the Louvain graph community detection method, and a faster, specialized variant in which edges are expected to lie fully within clusters. Using synthetic and empirical data, we demonstrate that the specialized method is highly scalable and can detect clusters where graph-based methods fail. We also use our model to find interpretable higher-order structure in school contact networks, U.S. congressional bill cosponsorship and committees, product categories in copurchasing behavior, and hotel locations from web browsing sessions.

Bipartite tight spectral clustering (BiTSC) algorithm for identifying conserved gene co-clusters in two species

MOTIVATION

Gene clustering is a widely used technique that has enabled computational prediction of unknown gene functions within a species. However, it remains a challenge to refine gene function prediction by leveraging evolutionarily conserved genes in another species. This challenge calls for a new computational algorithm to identify gene co-clusters in two species, so that genes in each co-cluster exhibit similar expression levels in each species and strong conservation between the species.

RESULTS

Here, we develop the bipartite tight spectral clustering (BiTSC) algorithm, which identifies gene co-clusters in two species based on gene orthology information and gene expression data. BiTSC novelly implements a formulation that encodes gene orthology as a bipartite network and gene expression data as node covariates. This formulation allows BiTSC to adopt and combine the advantages of multiple unsupervised learning techniques: kernel enhancement, bipartite spectral clustering, consensus clustering, tight clustering and hierarchical clustering. As a result, BiTSC is a flexible and robust algorithm capable of identifying informative gene co-clusters without forcing all genes into co-clusters. Another advantage of BiTSC is that it does not rely on any distributional assumptions. Beyond cross-species gene co-clustering, BiTSC also has wide applications as a general algorithm for identifying tight node co-clusters in any bipartite network with node covariates. We demonstrate the accuracy and robustness of BiTSC through comprehensive simulation studies. In a real data example, we use BiTSC to identify conserved gene co-clusters of Drosophila melanogaster and Caenorhabditis elegans, and we perform a series of downstream analysis to both validate BiTSC and verify the biological significance of the identified co-clusters.

AVAILABILITY AND IMPLEMENTATION

The Python package BiTSC is open-access and available at https://github.com/edensunyidan/BiTSC.

SUPPLEMENTARY INFORMATION

Supplementary data are available at Bioinformatics online.

A clarified typology of core-periphery structure in networks

Core-periphery structure, the arrangement of a network into a dense core and sparse periphery, is a versatile descriptor of various social, biological, and technological networks. In practice, different core-periphery algorithms are often applied interchangeably despite the fact that they can yield inconsistent descriptions of core-periphery structure. For example, two of the most widely used algorithms, the k-cores decomposition and the classic two-block model of Borgatti and Everett, extract fundamentally different structures: The latter partitions a network into a binary hub-and-spoke layout, while the former divides it into a layered hierarchy. We introduce a core-periphery typology to clarify these differences, along with Bayesian stochastic block modeling techniques to classify networks in accordance with this typology. Empirically, we find a rich diversity of core-periphery structure among networks. Through a detailed case study, we demonstrate the importance of acknowledging this diversity and situating networks within the core-periphery typology when conducting domain-specific analyses.

BiNeTClus

We investigate the problem of community detection in bipartite networks that are characterized by the presence of two types of nodes such that connections exist only between nodes of different types. While some approaches have been proposed to identify community structures in bipartite networks, there are a number of problems still to solve. In fact, the majority of the proposed approaches suffer from one or even more of the following limitations: (1) difficulty in detecting communities in the presence of many non-discriminating nodes with atypical connections that hide the community structures, (2) loss of relevant topological information due to the transformation of the bipartite network to standard plain graphs, and (3) manually specifying several input parameters, including the number of communities to be identified. To alleviate these problems, we propose BiNeTClus, a parameter-free community detection algorithm in bipartite networks that operates in two phases. The first phase focuses on identifying an initial grouping of nodes through a transactional data model capable of dealing with the situation that involves networks with many atypical connections, that is, sparsely connected nodes and nodes of one type that massively connect to all other nodes of the second type. The second phase aims to refine the clustering results of the first phase via an optimization strategy of the bipartite modularity to identify the final community structures. Our experiments on both synthetic and real networks illustrate the suitability of the proposed approach.

Hierarchical clustering of bipartite data sets based on the statistical significance of coincidences

When some 'entities' are related by the 'features' they share they are amenable to a bipartite network representation. Plant-pollinator ecological communities, co-authorship of scientific papers, customers and purchases, or answers in a poll, are but a few examples. Analyzing clustering of such entities in the network is a useful tool with applications in many fields, like internet technology, recommender systems, or detection of diseases. The algorithms most widely applied to find clusters in bipartite networks are variants of modularity optimization. Here, we provide a hierarchical clustering algorithm based on a dissimilarity between entities that quantifies the probability that the features shared by two entities are due to mere chance. The algorithm performance is O(n^{2}) when applied to a set of n entities, and its outcome is a dendrogram exhibiting the connections of those entities. Through the introduction of a 'susceptibility' measure we can provide an 'optimal' choice for the clustering as well as quantify its quality. The dendrogram reveals further useful structural information though-like the existence of subclusters within clusters or of nodes that do not fit in any cluster. We illustrate the algorithm by applying it first to a set of synthetic networks, and then to a selection of examples. We also illustrate how to transform our algorithm into a valid alternative for one-mode networks as well, and show that it performs at least as well as the standard, modularity-based algorithms-with a higher numerical performance. We provide an implementation of the algorithm in python freely accessible from GitHub.

Mapping flows on bipartite networks

Mapping network flows provides insight into the organization of networks, but even though many real networks are bipartite, no method for mapping flows takes advantage of the bipartite structure. What do we miss by discarding this information and how can we use it to understand the structure of bipartite networks better? The map equation models network flows with a random walk and exploits the information-theoretic duality between compression and finding regularities to detect communities in networks. However, it does not use the fact that random walks in bipartite networks alternate between node types, information worth 1 bit. To make some or all of this information available to the map equation, we developed a coding scheme that remembers node types at different rates. We explored the community landscape of bipartite real-world networks from no node-type information to full node-type information and found that using node types at a higher rate generally leads to deeper community hierarchies and a higher resolution. The corresponding compression of network flows exceeds the amount of extra information provided. Consequently, taking advantage of the bipartite structure increases the resolution and reveals more network regularities.

Community detection in bipartite networks with stochastic block models

In bipartite networks, community structures are restricted to being disassortative, in that nodes of one type are grouped according to common patterns of connection with nodes of the other type. This makes the stochastic block model (SBM), a highly flexible generative model for networks with block structure, an intuitive choice for bipartite community detection. However, typical formulations of the SBM do not make use of the special structure of bipartite networks. Here we introduce a Bayesian nonparametric formulation of the SBM and a corresponding algorithm to efficiently find communities in bipartite networks which parsimoniously chooses the number of communities. The biSBM improves community detection results over general SBMs when data are noisy, improves the model resolution limit by a factor of sqrt[2], and expands our understanding of the complicated optimization landscape associated with community detection tasks. A direct comparison of certain terms of the prior distributions in the biSBM and a related high-resolution hierarchical SBM also reveals a counterintuitive regime of community detection problems, populated by smaller and sparser networks, where nonhierarchical models outperform their more flexible counterpart.

Public Strategy and Eco-Social Engagement in Latin American States: An Analysis of Complex Networks Arising from Their Constitutions

Over the past four decades, Latin American states have drafted relatively new constitutions in comparison with other regions of the world. These transformations, in some cases, have helped governments leave behind the former authoritarian regimes, or in others, have simply established a more democratic system incorporating a forward-looking approach to rights. For example, stronger individual and collective rights have been forged, together with new avenues for citizen participation. Certainly, many of the new constitutions grant a much broader base of rights, including collective political and territorial rights for indigenous communities, protections against ethnic, racial, and gender discrimination, and greater guarantees of privacy and control over information. Consequently, some Latin American constitutions are held up as among the best in the world. For this study, the constitutional texts of 22 Latin American countries were analyzed with the aim of understanding their regulatory changes and impacts, pointing out the existing inequalities they address, as well as the clear positive trend established in terms of the generation of greater social engagement.

Identifying tumor clones in sparse single-cell mutation data

MOTIVATION

Recent single-cell DNA sequencing technologies enable whole-genome sequencing of hundreds to thousands of individual cells. However, these technologies have ultra-low sequencing coverage (<0.5× per cell) which has limited their use to the analysis of large copy-number aberrations (CNAs) in individual cells. While CNAs are useful markers in cancer studies, single-nucleotide mutations are equally important, both in cancer studies and in other applications. However, ultra-low coverage sequencing yields single-nucleotide mutation data that are too sparse for current single-cell analysis methods.

RESULTS

We introduce SBMClone, a method to infer clusters of cells, or clones, that share groups of somatic single-nucleotide mutations. SBMClone uses a stochastic block model to overcome sparsity in ultra-low coverage single-cell sequencing data, and we show that SBMClone accurately infers the true clonal composition on simulated datasets with coverage at low as 0.2×. We applied SBMClone to single-cell whole-genome sequencing data from two breast cancer patients obtained using two different sequencing technologies. On the first patient, sequenced using the 10X Genomics CNV solution with sequencing coverage ≈0.03×, SBMClone recovers the major clonal composition when incorporating a small amount of additional information. On the second patient, where pre- and post-treatment tumor samples were sequenced using DOP-PCR with sequencing coverage ≈0.5×, SBMClone shows that tumor cells are present in the post-treatment sample, contrary to published analysis of this dataset.

AVAILABILITY AND IMPLEMENTATION

SBMClone is available on the GitHub repository https://github.com/raphael-group/SBMClone.

SUPPLEMENTARY INFORMATION

Supplementary data are available at Bioinformatics online.

Transitivity and degree assortativity explained: The bipartite structure of social networks

Dynamical processes, such as the diffusion of knowledge, opinions, pathogens, "fake news," innovation, and others, are highly dependent on the structure of the social network in which they occur. However, questions on why most social networks present some particular structural features, namely, high levels of transitivity and degree assortativity, when compared to other types of networks remain open. First, we argue that every one-mode network can be regarded as a projection of a bipartite network, and we show that this is the case using two simple examples solved with the generating functions formalism. Second, using synthetic and empirical data, we reveal how the combination of the degree distribution of both sets of nodes of the bipartite network-together with the presence of cycles of lengths four and six-explain the observed values of transitivity and degree assortativity coefficients in the one-mode projected network. Bipartite networks with top node degrees that display a more right-skewed distribution than the bottom nodes result in highly transitive and degree assortative projections, especially if a large number of small cycles are present in the bipartite structure.

Finding missing links in interaction networks

Documenting which species interact within ecological communities is challenging and labor intensive. As a result, many interactions remain unrecorded, potentially distorting our understanding of network structure and dynamics. We test the utility of four structural models and a new coverage-deficit model for predicting missing links in both simulated and empirical bipartite networks. We find they can perform well, although the predictive power of structural models varies with the underlying network structure. The accuracy of predictions can be improved by ensembling multiple models. Augmenting observed networks with most-likely missing links improves estimates of qualitative network metrics. Tools to identify likely missing links can be simple to implement, allowing the prioritization of research effort and more robust assessment of network properties.

Optimizing Variational Graph Autoencoder for Community Detection with Dual Optimization

Variational Graph Autoencoder (VGAE) has recently gained traction for learning representations on graphs. Its inception has allowed models to achieve state-of-the-art performance for challenging tasks such as link prediction, rating prediction, and node clustering. However, a fundamental flaw exists in Variational Autoencoder (VAE)-based approaches. Specifically, merely minimizing the loss of VAE increases the deviation from its primary objective. Focusing on Variational Graph Autoencoder for Community Detection (VGAECD) we found that optimizing the loss using the stochastic gradient descent often leads to sub-optimal community structure especially when initialized poorly. We address this shortcoming by introducing a dual optimization procedure. This procedure aims to guide the optimization process and encourage learning of the primary objective. Additionally, we linearize the encoder to reduce the number of learning parameters. The outcome is a robust algorithm that outperforms its predecessor.

Detectability of macroscopic structures in directed asymmetric stochastic block model

We study the problem of identifying macroscopic structures in networks, characterizing the impact of introducing link directions on the detectability phase transition. To this end, building on the stochastic block model, we construct a class of nontrivially detectable directed networks. We find closed-form solutions by using the belief propagation method, showing how the transition line depends on the assortativity and the asymmetry of the network. Finally, we numerically identify the existence of a hard phase for detection close to the transition point.

Entropy-based randomization of rating networks

In recent years, due to the great diffusion of e-commerce, online rating platforms quickly became a common tool for purchase recommendations. However, instruments for their analysis did not evolve at the same speed. Indeed, interesting information about users' habits and tastes can be recovered just considering the bipartite network of users and products, in which links represent products' purchases and have different weights due to the score assigned to the item in users' reviews. With respect to other weighted bipartite networks, in these systems we observe a maximum possible weight per link, that limits the variability of the outcomes. In the present article we propose an entropy-based randomization method for this type of networks (i.e., bipartite rating networks) by extending the configuration model framework: the randomized network satisfies the constraints of the degree per rating, i.e., the number of given ratings received by the specified product or assigned by the single user. We first show that such a null model is able to reproduce several nontrivial features of the real network better than other null models. Then, using our model as benchmark, we project the information contained in the real system on one of the layers: To provide an interpretation of the projection obtained, we run the Louvain community detection on the obtained network and discuss the observed division in clusters. We are able to detect groups of music albums due to the consumers' taste or communities of movies due to their audience. Finally, we show that our method is also able to handle the special case of categorical bipartite networks: we consider the bipartite categorical network of scientific journals recognized for the scientific qualification in economics and statistics. In the end, from the outcome of our method, the probability that each user appreciate every product can be easily recovered. Therefore, this information may be employed in future applications to implement a more detailed recommendation system that also takes into account information regarding the topology of the observed network.

Efficient Detection of Communities in Biological Bipartite Networks

Methods to efficiently uncover and extract community structures are required in a number of biological applications where networked data and their interactions can be modeled as graphs, and observing tightly-knit groups of vertices ("communities") can offer insights into the structural and functional building blocks of the underlying network. Classical applications of community detection have largely focused on unipartite networks - i.e., graphs built out of a single type of objects. However, due to increased availability of biological data from various sources, there is now an increasing need for handling heterogeneous networks which are built out of multiple types of objects. In this paper, we address the problem of identifying communities from biological bipartite networks - i.e., networks where interactions are observed between two different types of objects (e.g., genes and diseases, drugs and protein complexes, plants and pollinators, and hosts and pathogens). Toward detecting communities in such bipartite networks, we make the following contributions: i) (metric) we propose a variant of bipartite modularity; ii) (algorithms) we present an efficient algorithm called biLouvain that implements a set of heuristics toward fast and precise community detection in bipartite networks (https://github.com/paolapesantez/biLouvain); and iii) (experiments) we present a thorough experimental evaluation of our algorithm including comparison to other state-of-the-art methods to identify communities in bipartite networks. Experimental results show that our biLouvain algorithm identifies communities that have a comparable or better quality (as measured by bipartite modularity) than existing methods, while significantly reducing the time-to-solution between one and four orders of magnitude.

Degree distributions of bipartite networks and their projections

Bipartite (two-mode) networks are important in the analysis of social and economic systems as they explicitly show conceptual links between different types of entities. However, applications of such networks often work with a projected (one-mode) version of the original bipartite network. The topology of the projected network, and the dynamics that take place on it, are highly dependent on the degree distributions of the two different node types from the original bipartite structure. To date, the interaction between the degree distributions of bipartite networks and their one-mode projections is well understood for only a few cases, or for networks that satisfy a restrictive set of assumptions. Here we show a broader analysis in order to fill the gap left by previous studies. We use the formalism of generating functions to prove that the degree distributions of both node types in the original bipartite network affect the degree distribution in the projected version. To support our analysis, we simulate several types of synthetic bipartite networks using a configuration model where node degrees are assigned from specific probability distributions, ranging from peaked to heavy-tailed distributions. Our findings show that when projecting a bipartite network onto a particular set of nodes, the degree distribution for the resulting one-mode network follows the distribution of the nodes being projected on to, but only so long as the degree distribution for the opposite set of nodes does not have a heavier tail. Furthermore, we show that bipartite degree distributions are not the only feature driving topology formation of projected networks, in contrast to what is commonly described in the literature.

Cluster-based network proximities for arbitrary nodal subsets

The concept of a cluster or community in a network context has been of considerable interest in a variety of settings in recent years. In this paper, employing random walks and geodesic distance, we introduce a unified measure of cluster-based proximity between nodes, relative to a given subset of interest. The inherent simplicity and informativeness of the approach could make it of value to researchers in a variety of scientific fields. Applicability is demonstrated via application to clustering for a number of existent data sets (including multipartite networks). We view community detection (i.e. when the full set of network nodes is considered) as simply the limiting instance of clustering (for arbitrary subsets). This perspective should add to the dialogue on what constitutes a cluster or community within a network. In regards to health-relevant attributes in social networks, identification of clusters of individuals with similar attributes can support targeting of collective interventions. The method performs well in comparisons with other approaches, based on comparative measures such as NMI and ARI.

Bipartite graphs in systems biology and medicine: a survey of methods and applications

The latest advances in high-throughput techniques during the past decade allowed the systems biology field to expand significantly. Today, the focus of biologists has shifted from the study of individual biological components to the study of complex biological systems and their dynamics at a larger scale. Through the discovery of novel bioentity relationships, researchers reveal new information about biological functions and processes. Graphs are widely used to represent bioentities such as proteins, genes, small molecules, ligands, and others such as nodes and their connections as edges within a network. In this review, special focus is given to the usability of bipartite graphs and their impact on the field of network biology and medicine. Furthermore, their topological properties and how these can be applied to certain biological case studies are discussed. Finally, available methodologies and software are presented, and useful insights on how bipartite graphs can shape the path toward the solution of challenging biological problems are provided.

Algorithmic detectability threshold of the stochastic block model

The assumption that the values of model parameters are known or correctly learned, i.e., the Nishimori condition, is one of the requirements for the detectability analysis of the stochastic block model in statistical inference. In practice, however, there is no example demonstrating that we can know the model parameters beforehand, and there is no guarantee that the model parameters can be learned accurately. In this study, we consider the expectation-maximization (EM) algorithm with belief propagation (BP) and derive its algorithmic detectability threshold. Our analysis is not restricted to the community structure but includes general modular structures. Because the algorithm cannot always learn the planted model parameters correctly, the algorithmic detectability threshold is qualitatively different from the one with the Nishimori condition.

Statistics of the network of organic chemistry

A comprehensive analysis of statistical properties of a network of organic reactions reveals several generic traits. This knowledge can be used in the development of optimal reaction sequences.

Similarity-based Regularized Latent Feature Model for Link Prediction in Bipartite Networks

Link prediction is an attractive research topic in the field of data mining and has significant applications in improving performance of recommendation system and exploring evolving mechanisms of the complex networks. A variety of complex systems in real world should be abstractly represented as bipartite networks, in which there are two types of nodes and no links connect nodes of the same type. In this paper, we propose a framework for link prediction in bipartite networks by combining the similarity based structure and the latent feature model from a new perspective. The framework is called Similarity Regularized Nonnegative Matrix Factorization (SRNMF), which explicitly takes the local characteristics into consideration and encodes the geometrical information of the networks by constructing a similarity based matrix. We also develop an iterative scheme to solve the objective function based on gradient descent. Extensive experiments on a variety of real world bipartite networks show that the proposed framework of link prediction has a more competitive, preferable and stable performance in comparison with the state-of-art methods.

Generating Bipartite Networks with a Prescribed Joint Degree Distribution

We describe a class of new algorithms to construct bipartite networks that preserves a prescribed degree and joint-degree (degree-degree) distribution of the nodes. Bipartite networks are graphs that can represent real-world interactions between two disjoint sets, such as actor-movie networks, author-article networks, co-occurrence networks, and heterosexual partnership networks. Often there is a strong correlation between the degree of a node and the degrees of the neighbors of that node that must be preserved when generating a network that reflects the structure of the underling system. Our bipartite 2K (B2K) algorithms generate an ensemble of networks that preserve prescribed degree sequences for the two disjoint set of nodes in the bipartite network, and the joint-degree distribution that is the distribution of the degrees of all neighbors of nodes with the same degree. We illustrate the effectiveness of the algorithms on a romance network using the NetworkX software environment to compare other properties of a target network that are not directly enforced by the B2K algorithms. We observe that when average degree of nodes is low, as is the case for romance and heterosexual partnership networks, then the B2K networks tend to preserve additional properties, such as the cluster coefficients, than algorithms that do not preserve the joint-degree distribution of the original network.

Core of communities in bipartite networks

We use the information present in a bipartite network to detect cores of communities of each set of the bipartite system. Cores of communities are found by investigating statistically validated projected networks obtained using information present in the bipartite network. Cores of communities are highly informative and robust with respect to the presence of errors or missing entries in the bipartite network. We assess the statistical robustness of cores by investigating an artificial benchmark network, the coauthorship network, and the actor-movie network. The accuracy and precision of the partition obtained with respect to the reference partition are measured in terms of the adjusted Rand index and the adjusted Wallace index, respectively. The detection of cores is highly precise, although the accuracy of the methodology can be limited in some cases.

Using null models to infer microbial co-occurrence networks

Although microbial communities are ubiquitous in nature, relatively little is known about the structural and functional roles of their constituent organisms' underlying interactions. A common approach to study such questions begins with extracting a network of statistically significant pairwise co-occurrences from a matrix of observed operational taxonomic unit (OTU) abundances across sites. The structure of this network is assumed to encode information about ecological interactions and processes, resistance to perturbation, and the identity of keystone species. However, common methods for identifying these pairwise interactions can contaminate the network with spurious patterns that obscure true ecological signals. Here, we describe this problem in detail and develop a solution that incorporates null models to distinguish ecological signals from statistical noise. We apply these methods to the initial OTU abundance matrix and to the extracted network. We demonstrate this approach by applying it to a large soil microbiome data set and show that many previously reported patterns for these data are statistical artifacts. In contrast, we find the frequency of three-way interactions among microbial OTUs to be highly statistically significant. These results demonstrate the importance of using appropriate null models when studying observational microbiome data, and suggest that extracting and characterizing three-way interactions among OTUs is a promising direction for unraveling the structure and function of microbial ecosystems.

The ground truth about metadata and community detection in networks

Across many scientific domains, there is a common need to automatically extract a simplified view or coarse-graining of how a complex system's components interact. This general task is called community detection in networks and is analogous to searching for clusters in independent vector data. It is common to evaluate the performance of community detection algorithms by their ability to find so-called ground truth communities. This works well in synthetic networks with planted communities because these networks' links are formed explicitly based on those known communities. However, there are no planted communities in real-world networks. Instead, it is standard practice to treat some observed discrete-valued node attributes, or metadata, as ground truth. We show that metadata are not the same as ground truth and that treating them as such induces severe theoretical and practical problems. We prove that no algorithm can uniquely solve community detection, and we prove a general No Free Lunch theorem for community detection, which implies that there can be no algorithm that is optimal for all possible community detection tasks. However, community detection remains a powerful tool and node metadata still have value, so a careful exploration of their relationship with network structure can yield insights of genuine worth. We illustrate this point by introducing two statistical techniques that can quantify the relationship between metadata and community structure for a broad class of models. We demonstrate these techniques using both synthetic and real-world networks, and for multiple types of metadata and community structures.

Latent geometry of bipartite networks

Despite the abundance of bipartite networked systems, their organizing principles are less studied compared to unipartite networks. Bipartite networks are often analyzed after projecting them onto one of the two sets of nodes. As a result of the projection, nodes of the same set are linked together if they have at least one neighbor in common in the bipartite network. Even though these projections allow one to study bipartite networks using tools developed for unipartite networks, one-mode projections lead to significant loss of information and artificial inflation of the projected network with fully connected subgraphs. Here we pursue a different approach for analyzing bipartite systems that is based on the observation that such systems have a latent metric structure: network nodes are points in a latent metric space, while connections are more likely to form between nodes separated by shorter distances. This approach has been developed for unipartite networks, and relatively little is known about its applicability to bipartite systems. Here, we fully analyze a simple latent-geometric model of bipartite networks and show that this model explains the peculiar structural properties of many real bipartite systems, including the distributions of common neighbors and bipartite clustering. We also analyze the geometric information loss in one-mode projections in this model and propose an efficient method to infer the latent pairwise distances between nodes. Uncovering the latent geometry underlying real bipartite networks can find applications in diverse domains, ranging from constructing efficient recommender systems to understanding cell metabolism.

A seed-expanding method based on random walks for community detection in networks with ambiguous community structures

Community detection has received a great deal of attention, since it could help to reveal the useful information hidden in complex networks. Although most previous modularity-based and local modularity-based community detection algorithms could detect strong communities, they may fail to exactly detect several weak communities. In this work, we define a network with clear or ambiguous community structures based on the types of its communities. A seed-expanding method based on random walks is proposed to detect communities for networks, especially for the networks with ambiguous community structures. We identify local maximum degree nodes, and detect seed communities in a network. Then, the probability of a node belonging to each community is calculated based on the total probability model and random walks, and each community is expanded by repeatedly adding the node which is most likely to belong to it. Finally, we use the community optimization method to ensure that each node is in a community. Experimental results on both computer-generated and real-world networks demonstrate that the quality of the communities detected by the proposed algorithm is superior to the- state-of-the-art algorithms in the networks with ambiguous community structures.