Community detection algorithms: a comparative analysis

Decoding communities in networks

According to a recent information-theoretical proposal, the problem of defining and identifying communities in networks can be interpreted as a classical communication task over a noisy channel: memberships of nodes are information bits erased by the channel, edges and nonedges in the network are parity bits introduced by the encoder but degraded through the channel, and a community identification algorithm is a decoder. The interpretation is perfectly equivalent to the one at the basis of well-known statistical inference algorithms for community detection. The only difference in the interpretation is that a noisy channel replaces a stochastic network model. However, the different perspective gives the opportunity to take advantage of the rich set of tools of coding theory to generate novel insights on the problem of community detection. In this paper, we illustrate two main applications of standard coding-theoretical methods to community detection. First, we leverage a state-of-the-art decoding technique to generate a family of quasioptimal community detection algorithms. Second and more important, we show that the Shannon's noisy-channel coding theorem can be invoked to establish a lower bound, here named as decodability bound, for the maximum amount of noise tolerable by an ideal decoder to achieve perfect detection of communities. When computed for well-established synthetic benchmarks, the decodability bound explains accurately the performance achieved by the best community detection algorithms existing on the market, telling us that only little room for their improvement is still potentially left.

Association between expression of random gene sets and survival is evident in multiple cancer types and may be explained by sub-classification

One of the goals of cancer research is to identify a set of genes that cause or control disease progression. However, although multiple such gene sets were published, these are usually in very poor agreement with each other, and very few of the genes proved to be functional therapeutic targets. Furthermore, recent findings from a breast cancer gene-expression cohort showed that sets of genes selected randomly can be used to predict survival with a much higher probability than expected. These results imply that many of the genes identified in breast cancer gene expression analysis may not be causal of cancer progression, even though they can still be highly predictive of prognosis. We performed a similar analysis on all the cancer types available in the cancer genome atlas (TCGA), namely, estimating the predictive power of random gene sets for survival. Our work shows that most cancer types exhibit the property that random selections of genes are more predictive of survival than expected. In contrast to previous work, this property is not removed by using a proliferation signature, which implies that proliferation may not always be the confounder that drives this property. We suggest one possible solution in the form of data-driven sub-classification to reduce this property significantly. Our results suggest that the predictive power of random gene sets may be used to identify the existence of sub-classes in the data, and thus may allow better understanding of patient stratification. Furthermore, by reducing the observed bias this may allow more direct identification of biologically relevant, and potentially causal, genes.

Multiple gene sets have been published as predictive of cancer progression and metastasis in several cancer types. Although many of these sets proved to be highly predictive of survival, even gene sets for the same cancer (but from different data-sets or different analyses) exhibit very little overlap and to date did not provide functional therapeutic targets. Recent studies found that in breast cancer, even random gene sets can predict survival much better than would be expected, and on average are better than many published gene sets. Together, these results undermine the causal role of the published gene sets and their potential clinical implications. We show that random gene sets predict survival in many cancer types, and that this property no longer exists after splitting the data into subclasses based on data-driven clusters. This suggests that such sub-classification could increase the likelihood to identify causal genes that are potential therapeutic targets, and that this property can be used as an indication that there may be subclasses within the dataset.

Infomap Bioregions: Interactive Mapping of Biogeographical Regions from Species Distributions

Biogeographical regions (bioregions) reveal how different sets of species are spatially grouped and therefore are important units for conservation, historical biogeography, ecology, and evolution. Several methods have been developed to identify bioregions based on species distribution data rather than expert opinion. One approach successfully applies network theory to simplify and highlight the underlying structure in species distributions. However, this method lacks tools for simple and efficient analysis. Here, we present Infomap Bioregions, an interactive web application that inputs species distribution data and generates bioregion maps. Species distributions may be provided as georeferenced point occurrences or range maps, and can be of local, regional, or global scale. The application uses a novel adaptive resolution method to make best use of often incomplete species distribution data. The results can be downloaded as vector graphics, shapefiles, or in table format. We validate the tool by processing large data sets of publicly available species distribution data of the world’s amphibians using species ranges, and mammals using point occurrences. We then calculate the fit between the inferred bioregions and WWF ecoregions. As examples of applications, researchers can reconstruct ancestral ranges in historical biogeography or identify indicator species for targeted conservation.

Brain state flexibility accompanies motor-skill acquisition

Learning requires the traversal of inherently distinct cognitive states to produce behavioral adaptation. Yet, tools to explicitly measure these states with non-invasive imaging – and to assess their dynamics during learning – remain limited. Here, we describe an approach based on a distinct application of graph theory in which points in time are represented by network nodes, and similarities in brain states between two different time points are represented as network edges. We use a graph-based clustering technique to identify clusters of time points representing canonical brain states, and to assess the manner in which the brain moves from one state to another as learning progresses. We observe the presence of two primary states characterized by either high activation in sensorimotor cortex or high activation in a frontal-subcortical system. Flexible switching among these primary states and other less common states becomes more frequent as learning progresses, and is inversely correlated with individual differences in learning rate. These results are consistent with the notion that the development of automaticity is associated with a greater freedom to use cognitive resources for other processes. Taken together, our work offers new insights into the constrained, low dimensional nature of brain dynamics characteristic of early learning, which give way to less constrained, high-dimensional dynamics in later learning.

A complex network approach reveals a pivotal substructure of genes linked to schizophrenia

Research on brain disorders with a strong genetic component and complex heritability, such as schizophrenia, has led to the development of brain transcriptomics. This field seeks to gain a deeper understanding of gene expression, a key factor in exploring further research issues. Our study focused on how genes are associated amongst each other. In this perspective, we have developed a novel data-driven strategy for characterizing genetic modules, i.e., clusters of strongly interacting genes. The aim was to uncover a pivotal community of genes linked to a target gene for schizophrenia. Our approach combined network topological properties with information theory to highlight the presence of a pivotal community, for a specific gene, and to simultaneously assess the information content of partitions with the Shannon’s entropy based on betweenness. We analyzed the publicly available BrainCloud dataset containing post-mortem gene expression data and focused on the Dopamine D2 receptor, encoded by the DRD2 gene. We used four different community detection algorithms to evaluate the consistence of our approach. A pivotal DRD2 community emerged for all the procedures applied, with a considerable reduction in size, compared to the initial network. The stability of the results was confirmed by a Dice index ≥80% within a range of tested parameters. The detected community was also the most informative, as it represented an optimization of the Shannon entropy. Lastly, we verified the strength of connection of the DRD2 community, which was stronger than any other randomly selected community and even more so than the Weighted Gene Co-expression Network Analysis module, commonly considered the standard approach for such studies. This finding substantiates the conclusion that the detected community represents a more connected and informative cluster of genes for the DRD2 community, and therefore better elucidates the behavior of this module of strongly related DRD2 genes. Because this gene plays a relevant role in Schizophrenia, this finding of a more specific DRD2 community will improve the understanding of the genetic factors related with this disorder.

Analyzing hidden populations online: topic, emotion, and social network of HIV-related users in the largest Chinese online community

BACKGROUND

Traditional survey methods are limited in the study of hidden populations due to the hard to access properties, including lack of a sampling frame, sensitivity issue, reporting error, small sample size, etc. The rapid increase of online communities, of which members interact with others via the Internet, have generated large amounts of data, offering new opportunities for understanding hidden populations with unprecedented sample sizes and richness of information. In this study, we try to understand the multidimensional characteristics of a hidden population by analyzing the massive data generated in the online community.

METHODS

By elaborately designing crawlers, we retrieved a complete dataset from the "HIV bar," the largest bar related to HIV on the Baidu Tieba platform, for all records from January 2005 to August 2016. Through natural language processing and social network analysis, we explored the psychology, behavior and demand of online HIV population and examined the network community structure.

RESULTS

In HIV communities, the average topic similarity among members is positively correlated to network efficiency (r = 0.70, p < 0.001), indicating that the closer the social distance between members of the community, the more similar their topics. The proportion of negative users in each community is around 60%, weakly correlated with community size (r = 0.25, p = 0.002). It is found that users suspecting initial HIV infection or first in contact with high-risk behaviors tend to seek help and advice on the social networking platform, rather than immediately going to a hospital for blood tests.

CONCLUSIONS

Online communities have generated copious amounts of data offering new opportunities for understanding hidden populations with unprecedented sample sizes and richness of information. It is recommended that support through online services for HIV/AIDS consultation and diagnosis be improved to avoid privacy concerns and social discrimination in China.

A unified method of detecting core-periphery structure and community structure in networks

The core-periphery structure and the community structure are two typical meso-scale structures in complex networks. Although community detection has been extensively investigated from different perspectives, the definition and the detection of the core-periphery structure have not received much attention. Furthermore, the detection problems of the core-periphery and community structure were separately investigated. In this paper, we develop a unified framework to simultaneously detect the core-periphery structure and community structure in complex networks. Moreover, there are several extra advantages of our algorithm: our method can detect not only single but also multiple pairs of core-periphery structures; the overlapping nodes belonging to different communities can be identified; different scales of core-periphery structures can be detected by adjusting the size of the core. The good performance of the method has been validated on synthetic and real complex networks. So, we provide a basic framework to detect the two typical meso-scale structures: the core-periphery structure and the community structure.

Network size, structure, and pathogen transmission: a simulation study comparing different community detection algorithms

Social substructure can influence pathogen transmission. Modularity measures the degree of social contact within versus between “communities” in a network, with increasing modularity expected to reduce transmission opportunities. We investigated how social substructure scales with network size and disease transmission. Using small-scale primate social networks, we applied seven community detection algorithms to calculate modularity and subgroup cohesion, defined as individuals’ interactions within subgroups proportional to the network. We found larger networks were more modular with higher subgroup cohesion, but the association’s strength varied by community detection algorithm and substructure measure. These findings highlight the importance of choosing an appropriate community detection algorithm for the question of interest, and if not possible, using multiple algorithms. Disease transmission simulations revealed higher modularity and subgroup cohesion resulted in fewer infections, confirming that social substructure has epidemiological consequences. Increased subdivision in larger networks could reflect constrained time budgets or evolved defences against disease risk.

Community Detection Using Dual-Representation Chemical Reaction Optimization

Many complex networks have been shown to have community structures. Detecting those structures is very important for understanding the organization and function of networks. Because this problem is NP-hard, it is appropriate to resort to evolutionary algorithms. Chemical reaction optimization (CRO) is a novel evolutionary algorithm inspired by the interactions among molecules during chemical reactions. In this paper, we propose a CRO variant named dual-representation CRO (DCRO) to address the community detection problem. DCRO encodes a solution in two representations: one is locus-based and the other is vector-based. The former representation can ensure the validity of a solution and fits for diversification search, and the latter is convenient for intensification search. We thus design two operators for CRO based on these two representations. Their cooperation enables DCRO to achieve a good balance between exploration and exploitation. Experimental results on synthetic and real-life networks show that DCRO can find community structures close to the actual ones and is capable of achieving solutions comparable to several state-of-the-art methods.

Intelligence is associated with the modular structure of intrinsic brain networks

General intelligence is a psychological construct that captures in a single metric the overall level of behavioural and cognitive performance in an individual. While previous research has attempted to localise intelligence in circumscribed brain regions, more recent work focuses on functional interactions between regions. However, even though brain networks are characterised by substantial modularity, it is unclear whether and how the brain's modular organisation is associated with general intelligence. Modelling subject-specific brain network graphs from functional MRI resting-state data (N = 309), we found that intelligence was not associated with global modularity features (e.g., number or size of modules) or the whole-brain proportions of different node types (e.g., connector hubs or provincial hubs). In contrast, we observed characteristic associations between intelligence and node-specific measures of within- and between-module connectivity, particularly in frontal and parietal brain regions that have previously been linked to intelligence. We propose that the connectivity profile of these regions may shape intelligence-relevant aspects of information processing. Our data demonstrate that not only region-specific differences in brain structure and function, but also the network-topological embedding of fronto-parietal as well as other cortical and subcortical brain regions is related to individual differences in higher cognitive abilities, i.e., intelligence.

Network-based analysis of diagnosis progression patterns using claims data

In recent years, several network models have been introduced to elucidate the relationships between diseases. However, important risk factors that contribute to many human diseases, such as age, gender and prior diagnoses, have not been considered in most networks. Here, we construct a diagnosis progression network of human diseases using large-scale claims data and analyze the associations between diagnoses. Our network is a scale-free network, which means that a small number of diagnoses share a large number of links, while most diagnoses show limited associations. Moreover, we provide strong evidence that gender, age and disease class are major factors in determining the structure of the disease network. Practically, our network represents a methodology not only for identifying new connectivity that is not found in genome-based disease networks but also for estimating directionality, strength, and progression time to transition between diseases considering gender, age and incidence. Thus, our network provides a guide for investigators for future research and contributes to achieving precision medicine.

A community detection algorithm using network topologies and rule-based hierarchical arc-merging strategies

The authors use four criteria to examine a novel community detection algorithm: (a) effectiveness in terms of producing high values of normalized mutual information (NMI) and modularity, using well-known social networks for testing; (b) examination, meaning the ability to examine mitigating resolution limit problems using NMI values and synthetic networks; (c) correctness, meaning the ability to identify useful community structure results in terms of NMI values and Lancichinetti-Fortunato-Radicchi (LFR) benchmark networks; and (d) scalability, or the ability to produce comparable modularity values with fast execution times when working with large-scale real-world networks. In addition to describing a simple hierarchical arc-merging (HAM) algorithm that uses network topology information, we introduce rule-based arc-merging strategies for identifying community structures. Five well-studied social network datasets and eight sets of LFR benchmark networks were employed to validate the correctness of a ground-truth community, eight large-scale real-world complex networks were used to measure its efficiency, and two synthetic networks were used to determine its susceptibility to two resolution limit problems. Our experimental results indicate that the proposed HAM algorithm exhibited satisfactory performance efficiency, and that HAM-identified and ground-truth communities were comparable in terms of social and LFR benchmark networks, while mitigating resolution limit problems.

Associations between dynamic functional connectivity and age, metabolic risk, and cognitive performance

Advanced age is associated with reduced within-network functional connectivity, particularly within the default mode network. Most studies to date have examined age differences in functional connectivity via static indices that are computed over the entire blood-oxygen-level dependent time series. Little is known about the effects of age on short-term temporal dynamics of functional connectivity. Here, we examined age differences in dynamic connectivity as well as associations between connectivity, metabolic risk, and cognitive performance in healthy adults (N = 168; age, 18-83 years). A sliding-window k-means clustering approach was used to assess dynamic connectivity from resting-state functional magnetic resonance imaging data. Three out of 8 dynamic connectivity profiles were associated with age. Furthermore, metabolic risk was associated with the relative amount of time allocated to 2 of these profiles. Finally, the relative amount of time allocated to a dynamic connectivity profile marked by heightened connectivity between default mode and medial temporal regions was positively associated with executive functions. Thus, dynamic connectivity analyses can enrich understanding of age-related differences beyond what is revealed by static analyses.

Pig movements in France: Designing network models fitting the transmission route of pathogens

Pathogen spread between farms results from interaction between the epidemiological characteristics of infectious agents, such as transmission route, and the contact structure between holdings. The objective of our study was to design network models of pig movements matching with epidemiological features of pathogens. Our first model represents the transmission of infectious diseases between farms only through the introduction of animals to holdings (Animal Introduction Model AIM), whereas the second one also accounts for pathogen spread through intermediate transit of trucks through farms even without any animal unloading (i.e. indirect transmission–Transit Model TM). To take the pyramidal organisation of pig production into consideration, these networks were studied at three different scales: the whole network and two subnetworks containing only breeding or production farms. The two models were applied to pig movement data recorded in France from June 2012 to December 2014. For each type of model, we calculated network descriptive statistics, looked for weakly/strongly connected components (WCCs/SCCs) and communities, and analysed temporal patterns. Whatever the model, the network exhibited scale-free and small-world topologies. Differences in centrality values between the two models showed that nucleus, multiplication and post-weaning farms played a key role in the spread of diseases transmitted exclusively by the introduction of infected animals, whereas farrowing and farrow-to-finish herds appeared more vulnerable to the introduction of infectious diseases through indirect contacts. The second network was less fragmented than the first one, a giant SCC being detected. The topology of network communities also varied with modelling assumptions: in the first approach, a huge geographically dispersed community was found, whereas the second model highlighted several small geographically clustered communities. These results underline the relevance of developing network models corresponding to pathogen features (e.g. their transmission route), and the need to target specific types of holdings/areas for surveillance depending on the epidemiological context.

The significance of negative correlations in brain connectivity

Understanding the modularity of functional magnetic resonance imaging (fMRI)-derived brain networks or "connectomes" can inform the study of brain function organization. However, fMRI connectomes additionally involve negative edges, which may not be optimally accounted for by existing approaches to modularity that variably threshold, binarize, or arbitrarily weight these connections. Consequently, many existing Q maximization-based modularity algorithms yield variable modular structures. Here, we present an alternative complementary approach that exploits how frequent the blood-oxygen-level-dependent (BOLD) signal correlation between two nodes is negative. We validated this novel probability-based modularity approach on two independent publicly-available resting-state connectome data sets (the Human Connectome Project [HCP] and the 1,000 functional connectomes) and demonstrated that negative correlations alone are sufficient in understanding resting-state modularity. In fact, this approach (a) permits a dual formulation, leading to equivalent solutions regardless of whether one considers positive or negative edges; (b) is theoretically linked to the Ising model defined on the connectome, thus yielding modularity result that maximizes data likelihood. Additionally, we were able to detect novel and consistent sex differences in modularity in both data sets. As data sets like HCP become widely available for analysis by the neuroscience community at large, alternative and perhaps more advantageous computational tools to understand the neurobiological information of negative edges in fMRI connectomes are increasingly important.

CONE: Community Oriented Network Estimation Is a Versatile Framework for Inferring Population Structure in Large-Scale Sequencing Data

Estimation of genetic population structure based on molecular markers is a common task in population genetics and ecology. We apply a generalized linear model with LASSO regularization to infer relationships between individuals and populations from molecular marker data. Specifically, we apply a neighborhood selection algorithm to infer population genetic structure and gene flow between populations. The resulting relationships are used to construct an individual-level population graph. Different network substructures known as communities are then dissociated from each other using a community detection algorithm. Inference of population structure using networks combines the good properties of: (i) network theory (broad collection of tools, including aesthetically pleasing visualization), (ii) principal component analysis (dimension reduction together with simple visual inspection), and (iii) model-based methods (e.g., ancestry coefficient estimates). We have named our process CONE (for community oriented network estimation). CONE has fewer restrictions than conventional assignment methods in that properties such as the number of subpopulations need not be fixed before the analysis and the sample may include close relatives or involve uneven sampling. Applying CONE on simulated data sets resulted in more accurate estimates of the true number of subpopulations than model-based methods, and provided comparable ancestry coefficient estimates. Inference of empirical data sets of teosinte single nucleotide polymorphism, bacterial disease outbreak, and the human genome diversity panel illustrate that population structures estimated with CONE are consistent with the earlier findings

Efficient method for estimating the number of communities in a network

While there exist a wide range of effective methods for community detection in networks, most of them require one to know in advance how many communities one is looking for. Here we present a method for estimating the number of communities in a network using a combination of Bayesian inference with a novel prior and an efficient Monte Carlo sampling scheme. We test the method extensively on both real and computer-generated networks, showing that it performs accurately and consistently, even in cases where groups are widely varying in size or structure.

Human orbital and anterior medial prefrontal cortex: Intrinsic connectivity parcellation and functional organization

The orbital and medial prefrontal cortex (OMPFC) has been implicated in decision-making, reward and emotion processing, and psychopathology, such as depression and obsessive-compulsive disorder. Human and monkey anatomical studies indicate the presence of various cortical subdivisions and suggest that these are organized in two extended networks, a medial and an orbital one. Attempts have been made to replicate these neuroanatomical findings in vivo using MRI techniques for imaging connectivity. These revealed several consistencies, but also many inconsistencies between reported results. Here, we use fMRI resting-state functional connectivity (FC) and data-driven modularity optimization to parcellate the OMPFC to investigate replicability of in vivo parcellation more systematically. By collecting two resting-state data sets per participant, we were able to quantify the reliability of the observed modules and their boundaries. Results show that there was significantly more than chance overlap in modules and their boundaries at the level of individual data sets. Moreover, some of these consistent boundaries significantly co-localized across participants. Hierarchical clustering showed that the whole-brain FC profiles of the OMPFC subregions separate them in two networks, a medial and orbital one, which overlap with the organization proposed by Barbas and Pandya (J Comp Neurol 286:353-375, 1989) and Ongür and Price (Cereb Cortex 10:206-219, 2000). We conclude that in vivo resting-state FC can delineate reliable and neuroanatomically plausible subdivisions that agree with established cytoarchitectonic trends and connectivity patterns, while other subdivisions do not show the same consistency across data sets and studies.

Brain Modularity Mediates the Relation between Task Complexity and Performance

Recent work in cognitive neuroscience has focused on analyzing the brain as a network, rather than as a collection of independent regions. Prior studies taking this approach have found that individual differences in the degree of modularity of the brain network relate to performance on cognitive tasks. However, inconsistent results concerning the direction of this relationship have been obtained, with some tasks showing better performance as modularity increases and other tasks showing worse performance. A recent theoretical model [Chen, M., & Deem, M. W. 2015. Development of modularity in the neural activity of children's brains. Physical Biology, 12, 016009] suggests that these inconsistencies may be explained on the grounds that high-modularity networks favor performance on simple tasks whereas low-modularity networks favor performance on more complex tasks. The current study tests these predictions by relating modularity from resting-state fMRI to performance on a set of simple and complex behavioral tasks. Complex and simple tasks were defined on the basis of whether they did or did not draw on executive attention. Consistent with predictions, we found a negative correlation between individuals' modularity and their performance on a composite measure combining scores from the complex tasks but a positive correlation with performance on a composite measure combining scores from the simple tasks. These results and theory presented here provide a framework for linking measures of whole-brain organization from network neuroscience to cognitive processing.

Detection of communities with Naming Game-based methods

Complex networks are often organized in groups or communities of agents that share the same features and/or functions, and this structural organization is built naturally with the formation of the system. In social networks, we argue that the dynamic of linguistic interactions of agreement among people can be a crucial factor in generating this community structure, given that sharing opinions with another person bounds them together, and disagreeing constantly would probably weaken the relationship. We present here a computational model of opinion exchange that uncovers the community structure of a network. Our aim is not to present a new community detection method proper, but to show how a model of social communication dynamics can reveal the (simple and overlapping) community structure in an emergent way. Our model is based on a standard Naming Game, but takes into consideration three social features: trust, uncertainty and opinion preference, that are built over time as agents communicate among themselves. We show that the separate addition of each social feature in the Naming Game results in gradual improvements with respect to community detection. In addition, the resulting uncertainty and trust values classify nodes and edges according to role and position in the network. Also, our model has shown a degree of accuracy both for non-overlapping and overlapping communities that are comparable with most algorithms specifically designed for topological community detection.

Identifying cell populations with scRNASeq

Single-cell RNASeq (scRNASeq) has emerged as a powerful method for quantifying the transcriptome of individual cells. However, the data from scRNASeq experiments is often both noisy and high dimensional, making the computational analysis non-trivial. Here we provide an overview of different experimental protocols and the most popular methods for facilitating the computational analysis. We focus on approaches for identifying biologically important genes, projecting data into lower dimensions and clustering data into putative cell-populations. Finally we discuss approaches to validation and biological interpretation of the identified cell-types or cell-states.

Stance and influence of Twitter users regarding the Brexit referendum

Social media are an important source of information about the political issues, reflecting, as well as influencing, public mood. We present an analysis of Twitter data, collected over 6 weeks before the Brexit referendum, held in the UK in June 2016. We address two questions: what is the relation between the Twitter mood and the referendum outcome, and who were the most influential Twitter users in the pro- and contra-Brexit camps? First, we construct a stance classification model by machine learning methods, and are then able to predict the stance of about one million UK-based Twitter users. The demography of Twitter users is, however, very different from the demography of the voters. By applying a simple age-adjusted mapping to the overall Twitter stance, the results show the prevalence of the pro-Brexit voters, something unexpected by most of the opinion polls. Second, we apply the Hirsch index to estimate the influence, and rank the Twitter users from both camps. We find that the most productive Twitter users are not the most influential, that the pro-Brexit camp was four times more influential, and had considerably larger impact on the campaign than the opponents. Third, we find that the top pro-Brexit communities are considerably more polarized than the contra-Brexit camp. These results show that social media provide a rich resource of data to be exploited, but accumulated knowledge and lessons learned from the opinion polls have to be adapted to the new data sources.

A theorem proving approach for automatically synthesizing visualizations of flow cytometry data

Background

Polychromatic flow cytometry is a popular technique that has wide usage in the medical sciences, especially for studying phenotypic properties of cells. The high-dimensionality of data generated by flow cytometry usually makes it difficult to visualize. The naive solution of simply plotting two-dimensional graphs for every combination of observables becomes impractical as the number of dimensions increases. A natural solution is to project the data from the original high dimensional space to a lower dimensional space while approximately preserving the overall relationship between the data points. The expert can then easily visualize and analyze this low-dimensional embedding of the original dataset.

Results

This paper describes a new method, SANJAY, for visualizing high-dimensional flow cytometry datasets. This technique uses a decision procedure to automatically synthesize two-dimensional and three-dimensional projections of the original high-dimensional data while trying to minimize distortion. We compare SANJAY to the popular multidimensional scaling (MDS) approach for visualization of small data sets drawn from a representative set of benchmarks, and our experiments show that SANJAY produces distortions that are 1.44 to 4.15 times smaller than those caused due to MDS. Our experimental results show that SANJAY also outperforms the Random Projections technique in terms of the distortions in the projections.

Conclusions

We describe a new algorithmic technique that uses a symbolic decision procedure to automatically synthesize low-dimensional projections of flow cytometry data that typically have a high number of dimensions. Our algorithm is the first application, to our knowledge, of using automated theorem proving for automatically generating highly-accurate, low-dimensional visualizations of high-dimensional data.

DMclust, a Density-based Modularity Method for Accurate OTU Picking of 16S rRNA Sequences

Clustering 16S rRNA sequences into operational taxonomic units (OTUs) is a crucial step in analyzing metagenomic data. Although many methods have been developed, how to obtain an appropriate balance between clustering accuracy and computational efficiency is still a major challenge. A novel density-based modularity clustering method, called DMclust, is proposed in this paper to bin 16S rRNA sequences into OTUs with high clustering accuracy. The DMclust algorithm consists of four main phases. It first searches for the sequence dense group defined as n-sequence community, in which the distance between any two sequences is less than a threshold. Then these dense groups are used to construct a weighted network, where dense groups are viewed as nodes, each pair of dense groups is connected by an edge, and the distance of pairwise groups represents the weight of the edge. Then, a modularity-based community detection method is employed to generate the preclusters. Finally, the remaining sequences are assigned to their nearest preclusters to form OTUs. Compared with existing widely used methods, the experimental results on several metagenomic datasets show that DMclust has higher accurate clustering performance with acceptable memory usage.

Initial Validation for the Estimation of Resting-State fMRI Effective Connectivity by a Generalization of the Correlation Approach

Resting-state functional MRI (rs-fMRI) is widely used to noninvasively study human brain networks. Network functional connectivity is often estimated by calculating the timeseries correlation between blood-oxygen-level dependent (BOLD) signal from different regions of interest (ROIs). However, standard correlation cannot characterize the direction of information flow between regions. In this paper, we introduce and test a new concept, prediction correlation, to estimate effective connectivity in functional brain networks from rs-fMRI. In this approach, the correlation between two BOLD signals is replaced by a correlation between one BOLD signal and a prediction of this signal via a causal system driven by another BOLD signal. Three validations are described: (1) Prediction correlation performed well on simulated data where the ground truth was known, and outperformed four other methods. (2) On simulated data designed to display the "common driver" problem, prediction correlation did not introduce false connections between non-interacting driven ROIs. (3) On experimental data, prediction correlation recovered the previously identified network organization of human brain. Prediction correlation scales well to work with hundreds of ROIs, enabling it to assess whole brain interregional connectivity at the single subject level. These results provide an initial validation that prediction correlation can capture the direction of information flow and estimate the duration of extended temporal delays in information flow between regions of interest ROIs based on BOLD signal. This approach not only maintains the high sensitivity to network connectivity provided by the correlation analysis, but also performs well in the estimation of causal information flow in the brain.

The ground truth about metadata and community detection in networks

Across many scientific domains, there is a common need to automatically extract a simplified view or coarse-graining of how a complex system's components interact. This general task is called community detection in networks and is analogous to searching for clusters in independent vector data. It is common to evaluate the performance of community detection algorithms by their ability to find so-called ground truth communities. This works well in synthetic networks with planted communities because these networks' links are formed explicitly based on those known communities. However, there are no planted communities in real-world networks. Instead, it is standard practice to treat some observed discrete-valued node attributes, or metadata, as ground truth. We show that metadata are not the same as ground truth and that treating them as such induces severe theoretical and practical problems. We prove that no algorithm can uniquely solve community detection, and we prove a general No Free Lunch theorem for community detection, which implies that there can be no algorithm that is optimal for all possible community detection tasks. However, community detection remains a powerful tool and node metadata still have value, so a careful exploration of their relationship with network structure can yield insights of genuine worth. We illustrate this point by introducing two statistical techniques that can quantify the relationship between metadata and community structure for a broad class of models. We demonstrate these techniques using both synthetic and real-world networks, and for multiple types of metadata and community structures.

Implications of the cattle trade network in Cameroon for regional disease prevention and control

Movement of live animals is a major risk factor for the spread of livestock diseases and zoonotic infections. Understanding contact patterns is key to informing cost-effective surveillance and control strategies. In West and Central Africa some of the most rapid urbanization globally is expected to increase the demand for animal-source foods and the need for safer and more efficient animal production. Livestock trading points represent a strategic contact node in the dissemination of multiple pathogens. From October 2014 to May 2015 official transaction records were collected and a questionnaire-based survey was carried out in cattle markets throughout Western and Central-Northern Cameroon. The data were used to analyse the cattle trade network including a total of 127 livestock markets within Cameroon and five neighboring countries. This study explores for the first time the influence of animal trade on infectious disease spread in the region. The investigations showed that national borders do not present a barrier against pathogen dissemination and that non-neighbouring countries are epidemiologically connected, highlighting the importance of a regional approach to disease surveillance, prevention and control. Furthermore, these findings provide evidence for the benefit of strategic risk-based approaches for disease monitoring, surveillance and control, as well as for communication and training purposes through targeting key regions, highly connected livestock markets and central trading links.

The modular organization of human anatomical brain networks: Accounting for the cost of wiring

Brain networks are expected to be modular. However, existing techniques for estimating a network's modules make it difficult to assess the influence of organizational principles such as wiring cost reduction on the detected modules. Here we present a modification of an existing module detection algorithm that allowed us to focus on connections that are unexpected under a cost-reduction wiring rule and to identify modules from among these connections. We applied this technique to anatomical brain networks and showed that the modules we detected differ from those detected using the standard technique. We demonstrated that these novel modules are spatially distributed, exhibit unique functional fingerprints, and overlap considerably with rich clubs, giving rise to an alternative and complementary interpretation of the functional roles of specific brain regions. Finally, we demonstrated that, using the modified module detection approach, we can detect modules in a developmental dataset that track normative patterns of maturation. Collectively, these findings support the hypothesis that brain networks are composed of modules and provide additional insight into the function of those modules.

Differential Flattening

A multi-layer graph consists of multiple layers of weighted graphs, where the multiple layers represent the different aspects of relationships. Considering multiple aspects (i.e., layers) together is essential to achieve a comprehensive and consolidated view. In this article, we propose a novel framework of differential flattening , which facilitates the analysis of multi-layer graphs, and apply this framework to community detection. Differential flattening merges multiple graphs into a single graph such that the graph structure with the maximum clustering coefficient is obtained from the single graph. It has two distinct features compared with existing approaches. First, dealing with multiple layers is done independently of a specific community detection algorithm, whereas previous approaches rely on a specific algorithm. Thus, any algorithm for a single graph becomes applicable to multi-layer graphs. Second, the contribution of each layer to the single graph is determined automatically for the maximum clustering coefficient. Since differential flattening is formulated by an optimization problem, the optimal solution is easily obtained by well-known algorithms such as interior point methods. Extensive experiments were conducted using the Lancichinetti-Fortunato-Radicchi (LFR) benchmark networks as well as the DBLP, 20 Newsgroups, and MIT Reality Mining networks. The results show that our approach of differential flattening leads to discovery of higher-quality communities than baseline approaches and the state-of-the-art algorithms.

Locating Structural Centers: A Density-Based Clustering Method for Community Detection

Uncovering underlying community structures in complex networks has received considerable attention because of its importance in understanding structural attributes and group characteristics of networks. The algorithmic identification of such structures is a significant challenge. Local expanding methods have proven to be efficient and effective in community detection, but most methods are sensitive to initial seeds and built-in parameters. In this paper, we present a local expansion method by density-based clustering, which aims to uncover the intrinsic network communities by locating the structural centers of communities based on a proposed structural centrality. The structural centrality takes into account local density of nodes and relative distance between nodes. The proposed algorithm expands a community from the structural center to the border with a single local search procedure. The local expanding procedure follows a heuristic strategy as allowing it to find complete community structures. Moreover, it can identify different node roles (cores and outliers) in communities by defining a border region. The experiments involve both on real-world and artificial networks, and give a comparison view to evaluate the proposed method. The result of these experiments shows that the proposed method performs more efficiently with a comparative clustering performance than current state of the art methods.