Symbolic transfer entropy

Reconstruction of recurrent synaptic connectivity of thousands of neurons from simulated spiking activity

Dynamics and function of neuronal networks are determined by their synaptic connectivity. Current experimental methods to analyze synaptic network structure on the cellular level, however, cover only small fractions of functional neuronal circuits, typically without a simultaneous record of neuronal spiking activity. Here we present a method for the reconstruction of large recurrent neuronal networks from thousands of parallel spike train recordings. We employ maximum likelihood estimation of a generalized linear model of the spiking activity in continuous time. For this model the point process likelihood is concave, such that a global optimum of the parameters can be obtained by gradient ascent. Previous methods, including those of the same class, did not allow recurrent networks of that order of magnitude to be reconstructed due to prohibitive computational cost and numerical instabilities. We describe a minimal model that is optimized for large networks and an efficient scheme for its parallelized numerical optimization on generic computing clusters. For a simulated balanced random network of 1000 neurons, synaptic connectivity is recovered with a misclassification error rate of less than 1 % under ideal conditions. We show that the error rate remains low in a series of example cases under progressively less ideal conditions. Finally, we successfully reconstruct the connectivity of a hidden synfire chain that is embedded in a random network, which requires clustering of the network connectivity to reveal the synfire groups. Our results demonstrate how synaptic connectivity could potentially be inferred from large-scale parallel spike train recordings.

Measuring information transfer in a soft robotic arm

Soft robots can exhibit diverse behaviors with simple types of actuation by partially outsourcing control to the morphological and material properties of their soft bodies, which is made possible by the tight coupling between control, body, and environment. In this paper, we present a method that will quantitatively characterize these diverse spatiotemporal dynamics of a soft body based on the information-theoretic approach. In particular, soft bodies have the ability to propagate the effect of actuation through the entire body, with a certain time delay, due to their elasticity. Our goal is to capture this delayed interaction in a quantitative manner based on a measure called momentary information transfer. We extend this measure to soft robotic applications and demonstrate its power using a physical soft robotic platform inspired by the octopus. Our approach is illustrated in two ways. First, we statistically characterize the delayed actuation propagation through the body as a strength of information transfer. Second, we capture this information propagation directly as local information dynamics. As a result, we show that our approach can successfully characterize the spatiotemporal dynamics of the soft robotic platform, explicitly visualizing how information transfers through the entire body with delays. Further extension scenarios of our approach are discussed for soft robotic applications in general.

General relationship of global topology, local dynamics, and directionality in large-scale brain networks

The balance of global integration and functional specialization is a critical feature of efficient brain networks, but the relationship of global topology, local node dynamics and information flow across networks has yet to be identified. One critical step in elucidating this relationship is the identification of governing principles underlying the directionality of interactions between nodes. Here, we demonstrate such principles through analytical solutions based on the phase lead/lag relationships of general oscillator models in networks. We confirm analytical results with computational simulations using general model networks and anatomical brain networks, as well as high-density electroencephalography collected from humans in the conscious and anesthetized states. Analytical, computational, and empirical results demonstrate that network nodes with more connections (i.e., higher degrees) have larger amplitudes and are directional targets (phase lag) rather than sources (phase lead). The relationship of node degree and directionality therefore appears to be a fundamental property of networks, with direct applicability to brain function. These results provide a foundation for a principled understanding of information transfer across networks and also demonstrate that changes in directionality patterns across states of human consciousness are driven by alterations of brain network topology.

Current brain connectome projects are attempting to construct a map of the structural and functional network connections in the brain. One goal of these projects is to understand how network organization determines local functions and information transfer patterns, which is essential to achieve higher cognitive brain functions. Because of the limitation of constructing all brain maps for all cognitive states, finding a general relationship of global topology, local dynamics and the directionality of information transfer in a network is crucial. In this study, we show that inter-node directionality arises naturally from the topology of the network. Analytical, computational, and empirical results all demonstrate that network nodes with more connections (i.e., higher degree) lag in phase, while lower-degree nodes lead. Our mathematical analysis allowed us to predict the directionality patterns in general model networks as well as human brain networks across different states of consciousness. These findings may provide more straightforward approaches to dissecting how directionality between interacting nodes is shaped in complex brain networks, providing a foundation for understanding principles of information transfer. Furthermore, the underlying mathematical relationship between node connections and directionality patterns has the potential to advance network science across numerous disciplines.

Causation entropy from symbolic representations of dynamical systems

Identification of causal structures and quantification of direct information flows in complex systems is a challenging yet important task, with practical applications in many fields. Data generated by dynamical processes or large-scale systems are often symbolized, either because of the finite resolution of the measurement apparatus, or because of the need of statistical estimation. By algorithmic application of causation entropy, we investigated the effects of symbolization on important concepts such as Markov order and causal structure of the tent map. We uncovered that these quantities depend nonmonotonically and, most of all, sensitively on the choice of symbolization. Indeed, we show that Markov order and causal structure do not necessarily converge to their original analog counterparts as the resolution of the partitioning becomes finer.

Ordinal symbolic analysis and its application to biomedical recordings

Ordinal symbolic analysis opens an interesting and powerful perspective on time-series analysis. Here, we review this relatively new approach and highlight its relation to symbolic dynamics and representations. Our exposition reaches from the general ideas up to recent developments, with special emphasis on its applications to biomedical recordings. The latter will be illustrated with epilepsy data.

Assessing directionality and strength of coupling through symbolic analysis: an application to epilepsy patients

Inferring strength and direction of interactions from electroencephalographic (EEG) recordings is of crucial importance to improve our understanding of dynamical interdependencies underlying various physiological and pathophysiological conditions in the human epileptic brain. We here use approaches from symbolic analysis to investigate--in a time-resolved manner--weighted and directed, short- to long-ranged interactions between various brain regions constituting the epileptic network. Our observations point to complex spatial-temporal interdependencies underlying the epileptic process and their role in the generation of epileptic seizures, despite the massive reduction of the complex information content of multi-day, multi-channel EEG recordings through symbolization. We discuss limitations and potential future improvements of this approach.

Assessing levels of consciousness with symbolic analysis

'Covert consciousness' is a state in which consciousness is present without the capacity for behavioural response, and it can occur in patients with intraoperative awareness or unresponsive wakefulness syndrome. To detect and prevent this undesirable state, it is critical to develop a reliable neurobiological assessment of an individual's level of consciousness that is independent of behaviour. One such approach that shows potential is measuring surrogates of cortical communication in the brain using electroencephalography (EEG). EEG is practicable in clinical application, but involves many fundamental signal processing problems, including signal-to-noise ratio and high dimensional complexity. Symbolic analysis of EEG can mitigate these problems, improving the measurement of brain connectivity and the ability to successfully assess levels of consciousness. In this article, we review the problem of covert consciousness, basic neurobiological principles of consciousness, current methods of measuring brain connectivity and the advantages of symbolic processing, with a focus on symbolic transfer entropy (STE). Finally, we discuss recent advances and clinical applications of STE and other symbolic analyses to assess levels of consciousness.

Quantification of causal couplings via dynamical effects: a unifying perspective

Quantitative characterization of causal couplings from time series is crucial in studies of complex systems of different origin. Various statistical tools for that exist and new ones are still being developed with a tendency to creating a single, universal, model-free quantifier of coupling strength. However, a clear and generally applicable way of interpreting such universal characteristics is lacking. This work suggests a general conceptual framework for causal coupling quantification, which is based on state space models and extends the concepts of virtual interventions and dynamical causal effects. Namely, two basic kinds of interventions (state space and parametric) and effects (orbital or transient and stationary or limit) are introduced, giving four families of coupling characteristics. The framework provides a unifying view of apparently different well-established measures and allows us to introduce new characteristics, always with a definite "intervention-effect" interpretation. It is shown that diverse characteristics cannot be reduced to any single coupling strength quantifier and their interpretation is inevitably model based. The proposed set of dynamical causal effect measures quantifies different aspects of "how the coupling manifests itself in the dynamics," reformulating the very question about the "causal coupling strength."

Identifying delayed directional couplings with symbolic transfer entropy

We propose a straightforward extension of symbolic transfer entropy to enable the investigation of delayed directional relationships between coupled dynamical systems from time series. Analyzing time series from chaotic model systems, we demonstrate the applicability and limitations of our approach. Our findings obtained from applying our method to infer delayed directed interactions in the human epileptic brain underline the importance of our approach for improving the construction of functional network structures from data.

Dynamics on networks: the role of local dynamics and global networks on the emergence of hypersynchronous neural activity

Graph theory has evolved into a useful tool for studying complex brain networks inferred from a variety of measures of neural activity, including fMRI, DTI, MEG and EEG. In the study of neurological disorders, recent work has discovered differences in the structure of graphs inferred from patient and control cohorts. However, most of these studies pursue a purely observational approach; identifying correlations between properties of graphs and the cohort which they describe, without consideration of the underlying mechanisms. To move beyond this necessitates the development of computational modeling approaches to appropriately interpret network interactions and the alterations in brain dynamics they permit, which in the field of complexity sciences is known as dynamics on networks. In this study we describe the development and application of this framework using modular networks of Kuramoto oscillators. We use this framework to understand functional networks inferred from resting state EEG recordings of a cohort of 35 adults with heterogeneous idiopathic generalized epilepsies and 40 healthy adult controls. Taking emergent synchrony across the global network as a proxy for seizures, our study finds that the critical strength of coupling required to synchronize the global network is significantly decreased for the epilepsy cohort for functional networks inferred from both theta (3-6 Hz) and low-alpha (6-9 Hz) bands. We further identify left frontal regions as a potential driver of seizure activity within these networks. We also explore the ability of our method to identify individuals with epilepsy, observing up to 80% predictive power through use of receiver operating characteristic analysis. Collectively these findings demonstrate that a computer model based analysis of routine clinical EEG provides significant additional information beyond standard clinical interpretation, which should ultimately enable a more appropriate mechanistic stratification of people with epilepsy leading to improved diagnostics and therapeutics.

Functional brain connectivity from EEG in epilepsy: seizure prediction and epileptogenic focus localization

Today, neuroimaging techniques are frequently used to investigate the integration of functionally specialized brain regions in a network. Functional connectivity, which quantifies the statistical dependencies among the dynamics of simultaneously recorded signals, allows to infer the dynamical interactions of segregated brain regions. In this review we discuss how the functional connectivity patterns obtained from intracranial and scalp electroencephalographic (EEG) recordings reveal information about the dynamics of the epileptic brain and can be used to predict upcoming seizures and to localize the seizure onset zone. The added value of extracting information that is not visibly identifiable in the EEG data using functional connectivity analysis is stressed. Despite the fact that many studies have showed promising results, we must conclude that functional connectivity analysis has not made its way into clinical practice yet.

Dynamical inference: where phase synchronization and generalized synchronization meet

Synchronization is a widespread phenomenon that occurs among interacting oscillatory systems. It facilitates their temporal coordination and can lead to the emergence of spontaneous order. The detection of synchronization from the time series of such systems is of great importance for the understanding and prediction of their dynamics, and several methods for doing so have been introduced. However, the common case where the interacting systems have time-variable characteristic frequencies and coupling parameters, and may also be subject to continuous external perturbation and noise, still presents a major challenge. Here we apply recent developments in dynamical Bayesian inference to tackle these problems. In particular, we discuss how to detect phase slips and the existence of deterministic coupling from measured data, and we unify the concepts of phase synchronization and general synchronization. Starting from phase or state observables, we present methods for the detection of both phase and generalized synchronization. The consistency and equivalence of phase and generalized synchronization are further demonstrated, by the analysis of time series from analog electronic simulations of coupled nonautonomous van der Pol oscillators. We demonstrate that the detection methods work equally well on numerically simulated chaotic systems. In all the cases considered, we show that dynamical Bayesian inference can clearly identify noise-induced phase slips and distinguish coherence from intrinsic coupling-induced synchronization.

Detecting functional hubs of ictogenic networks

Quantitative EEG (qEEG) has modified our understanding of epileptic seizures, shifting our view from the traditionally accepted hyper-synchrony paradigm toward more complex models based on re-organization of functional networks. However, qEEG measurements are so far rarely considered during the clinical decision-making process. To better understand the dynamics of intracranial EEG signals, we examine a functional network derived from the quantification of information flow between intracranial EEG signals. Using transfer entropy, we analyzed 198 seizures from 27 patients undergoing pre-surgical evaluation for pharmaco-resistant epilepsy. During each seizure we considered for each network the in-, out- and total "hubs", defined respectively as the time and the EEG channels with the maximal incoming, outgoing or total (bidirectional) information flow. In the majority of cases we found that the hubs occur around the middle of seizures, and interestingly not at the beginning or end, where the most dramatic EEG signal changes are found by visual inspection. For the patients who then underwent surgery, good postoperative clinical outcome was on average associated with a higher percentage of out- or total-hubs located in the resected area (for out-hubs p = 0.01, for total-hubs p = 0.04). The location of in-hubs showed no clear predictive value. We conclude that the study of functional networks based on qEEG measurements may help to identify brain areas that are critical for seizure generation and are thus potential targets for focused therapeutic interventions.

Information transfer and criticality in the Ising model on the human connectome

We implement the Ising model on a structural connectivity matrix describing the brain at two different resolutions. Tuning the model temperature to its critical value, i.e. at the susceptibility peak, we find a maximal amount of total information transfer between the spin variables. At this point the amount of information that can be redistributed by some nodes reaches a limit and the net dynamics exhibits signature of the law of diminishing marginal returns, a fundamental principle connected to saturated levels of production. Our results extend the recent analysis of dynamical oscillators models on the connectome structure, taking into account lagged and directional influences, focusing only on the nodes that are more prone to became bottlenecks of information. The ratio between the outgoing and the incoming information at each node is related to the the sum of the weights to that node and to the average time between consecutive time flips of spins. The results for the connectome of 66 nodes and for that of 998 nodes are similar, thus suggesting that these properties are scale-independent. Finally, we also find that the brain dynamics at criticality is organized maximally to a rich-club w.r.t. the network of information flows.

Symbolic transfer entropy analysis of the dust interaction in the presence of wakefields in dusty plasmas

The method of symbolic transfer entropy has been applied to analyze the behavior of charged-particle systems under the influence of an ion focus (wakefield) in a dusty plasma. Using long-run experiments under various plasma and trapping conditions, it is revealed from the transfer entropy that information is transported from the upper particle in an ion flow to the lower. The information transfer increases with smaller interparticle distance and with reduced height in the sheath. This can be consistently explained by the formation of the ion focus by an ion flow in the sheath. From the analysis of two-particle and many-particle systems, the symbolic entropy transfer can be judged as a reliable measure for information asymmetry, and hence interaction asymmetry, in dusty plasma systems.

Fronto-parietal connectivity is a non-static phenomenon with characteristic changes during unconsciousness

Background

It has been previously shown that loss of consciousness is associated with a breakdown of dominating fronto-parietal feedback connectivity as assessed by electroencephalogram (EEG) recordings. Structure and strength of network connectivity may change over time. Aim of the current study is to investigate cortico-cortical connectivity at different time intervals during consciousness and unconsciousness. For this purpose, EEG symbolic transfer entropy (STEn) was calculated to indicate cortico-cortical information transfer at different transfer times.

Methods

The study was performed in 15 male volunteers. 29-channel EEG was recorded during consciousness and propofol-induced unconsciousness. EEG data were analyzed by STEn, which quantifies intensity and directionality of the mutual information flow between two EEG channels. STEn was computed over fronto-parietal channel pair combinations (10 s length, 0.5–45 Hz total bandwidth) to analyze changes of intercortical directional connectivity. Feedback (fronto → parietal) and feedforward (parieto → frontal) connectivity was calculated for transfer times from 25 ms to 250 ms in 5 ms steps. Transfer times leading to maximum directed interaction were identified to detect changes of cortical information transfer (directional connectivity) induced by unconsciousness (p<0.05).

Results

The current analyses show that fronto-parietal connectivity is a non-static phenomenon. Maximum detected interaction occurs at decreased transfer times during propofol-induced unconsciousness (feedback interaction: 60 ms to 40 ms, p = 0.002; feedforward interaction: 65 ms to 45 ms, p = 0.001). Strength of maximum feedback interaction decreases during unconsciousness (p = 0.026), while no effect of propofol was observed on feedforward interaction. During both consciousness and unconsciousness, intensity of fronto-parietal interaction fluctuates with increasing transfer times.

Conclusion

Non-stationarity of directional connectivity may play a functional role for cortical network communication as it shows characteristic changes during propofol-induced unconsciousness.

Simultaneous electroencephalographic and functional magnetic resonance imaging indicate impaired cortical top-down processing in association with anesthetic-induced unconsciousness

BACKGROUND

In imaging functional connectivity (FC) analyses of the resting brain, alterations of FC during unconsciousness have been reported. These results are in accordance with recent electroencephalographic studies observing impaired top-down processing during anesthesia. In this study, simultaneous records of functional magnetic resonance imaging (fMRI) and electroencephalogram were performed to investigate the causality of neural mechanisms during propofol-induced loss of consciousness by correlating FC in fMRI and directional connectivity (DC) in electroencephalogram.

METHODS

Resting-state 63-channel electroencephalogram and blood oxygen level-dependent 3-Tesla fMRI of 15 healthy subjects were simultaneously registered during consciousness and propofol-induced loss of consciousness. To indicate DC, electroencephalographic symbolic transfer entropy was applied as a nonlinear measure of mutual interdependencies between underlying physiological processes. The relationship between FC of resting-state networks of the brain (z values) and DC was analyzed by a partial correlation.

RESULTS

Independent component analyses of resting-state fMRI showed decreased FC in frontoparietal default networks during unconsciousness, whereas FC in primary sensory networks increased. DC indicated a decline in frontal-parietal (area under the receiver characteristic curve, 0.92; 95% CI, 0.68-1.00) and frontooccipital (0.82; 0.53-1.00) feedback DC (P<0.05 corrected). The changes of FC in the anterior default network correlated with the changes of DC in frontal-parietal (rpartial=+0.62; P=0.030) and frontal-occipital (+0.63; 0.048) electroencephalographic electrodes (P<0.05 corrected).

CONCLUSION

The simultaneous propofol-induced suppression of frontal feedback connectivity in the electroencephalogram and of frontoparietal FC in the fMRI indicates a fundamental role of top-down processing for consciousness.

Sleep Electroencephalogram Analysis Based on Symbolic Transfer Entropy

The quality of sleep has a great relationship with health. The result of sleep stage classification is an important indicator to measure the quality of sleep. It was found that the symbolic transfer entropy about wake and the first stage of non-rapid eye movement sleep reflect on the changes of sleep stage. And it was confirmed by T test and multi-samples experiments. The symbolic transfer entropy can apply into automatic sleep stage classification. By Multi-parameter analysis it could achieve a higher accuracy of sleep stage classification.

Cardiovascular and cardiorespiratory coupling analyses: a review

Recently, methods have been developed to analyse couplings in dynamic systems. In the field of medical analysis of complex cardiovascular and cardiorespiratory systems, there is growing interest in how insights may be gained into the interaction between regulatory mechanisms in healthy and diseased persons. The couplings within and between these systems can be linear or nonlinear. However, the complex mechanisms involved in cardiovascular and cardiorespiratory regulation very likely interact with each other in a nonlinear way. Recent advances in nonlinear dynamics and information theory have allowed the multivariate study of information transfer between time series. They therefore might be able to provide additional diagnostic and prognostic information in medicine and might, in particular, be able to complement traditional linear coupling analysis techniques. In this review, we describe the approaches (Granger causality, nonlinear prediction, entropy, symbolization, phase synchronization) most commonly applied to detect direct and indirect couplings between time series, especially focusing on nonlinear approaches. We will discuss their capacity to quantify direct and indirect couplings and the direction (driver-response relationship) of the considered interaction between different biological time series. We also give their basic theoretical background, their basic requirements for application, their main features and demonstrate their usefulness in different applications in the field of cardiovascular and cardiorespiratory coupling analyses.

Estimating temporal causal interaction between spike trains with permutation and transfer entropy

Estimating the causal interaction between neurons is very important for better understanding the functional connectivity in neuronal networks. We propose a method called normalized permutation transfer entropy (NPTE) to evaluate the temporal causal interaction between spike trains, which quantifies the fraction of ordinal information in a neuron that has presented in another one. The performance of this method is evaluated with the spike trains generated by an Izhikevich’s neuronal model. Results show that the NPTE method can effectively estimate the causal interaction between two neurons without influence of data length. Considering both the precision of time delay estimated and the robustness of information flow estimated against neuronal firing rate, the NPTE method is superior to other information theoretic method including normalized transfer entropy, symbolic transfer entropy and permutation conditional mutual information. To test the performance of NPTE on analyzing simulated biophysically realistic synapses, an Izhikevich’s cortical network that based on the neuronal model is employed. It is found that the NPTE method is able to characterize mutual interactions and identify spurious causality in a network of three neurons exactly. We conclude that the proposed method can obtain more reliable comparison of interactions between different pairs of neurons and is a promising tool to uncover more details on the neural coding.

Information directionality in coupled time series using transcripts

In ordinal symbolic dynamics, transcripts describe the algebraic relationship between ordinal patterns. Using the concept of transcript, we exploit the mathematical structure of the group of permutations to derive properties and relations among information measures of the symbolic representations of time series. These theoretical results are then applied for the assessment of coupling directionality in dynamical systems, where suitable coupling directionality measures are introduced depending only on transcripts. These measures improve the reliability of the information flow estimates and reduce to well-established coupling directionality quantifiers when some general conditions are satisfied. Furthermore, by generalizing the definition of transcript to ordinal patterns of different lengths, several of the commonly used information directionality measures can be encompassed within the same framework.

Disruption of frontal-parietal communication by ketamine, propofol, and sevoflurane

INTRODUCTION

Directional connectivity from anterior to posterior brain regions (or "feedback" connectivity) has been shown to be inhibited by propofol and sevoflurane. In this study the authors tested the hypothesis that ketamine would also inhibit cortical feedback connectivity in frontoparietal networks.

METHODS

Surgical patients (n = 30) were recruited for induction of anesthesia with intravenous ketamine (2 mg/kg); electroencephalography of the frontal and parietal regions was acquired. The authors used normalized symbolic transfer entropy, a computational method based on information theory, to measure directional connectivity across frontal and parietal regions. Statistical analysis of transfer entropy measures was performed with the permutation test and the time-shift test to exclude false-positive connectivity. For comparison, the authors used normalized symbolic transfer entropy to reanalyze electroencephalographic data gathered from surgical patients receiving either propofol (n = 9) or sevoflurane (n = 9) for anesthetic induction.

RESULTS

Ketamine reduced alpha power and increased gamma power, in contrast to both propofol and sevoflurane. During administration of ketamine, feedback connectivity gradually diminished and was significantly inhibited after loss of consciousness (mean ± SD of baseline and anesthesia: 0.0074 ± 0.003 and 0.0055 ± 0.0027; F(5, 179) = 7.785, P < 0.0001). By contrast, feedforward connectivity was preserved during exposure to ketamine (mean ± SD of baseline and anesthesia: 0.0041 ± 0.0015 and 0.0046 ± 0.0018; F(5, 179) = 2.07; P = 0.072). Like ketamine, propofol and sevoflurane selectively inhibited feedback connectivity after anesthetic induction.

CONCLUSIONS

Diverse anesthetics disrupt frontal-parietal communication, despite molecular and neurophysiologic differences. Analysis of directional connectivity in frontal-parietal networks could provide a common metric of general anesthesia and insight into the cognitive neuroscience of anesthetic-induced unconsciousness.

Quantification of Drive-Response Relationships Between Residues During Protein Folding

Mutual correlation and cooperativity are commonly used to describe residue-residue interactions in protein folding/function. However, these metrics do not provide any information on the causality relationships between residues. Such drive-response relationships are poorly studied in protein folding/function and difficult to measure experimentally due to technical limitations. In this study, using the information theory transfer entropy (TE) that provides a direct measurement of causality between two times series, we have quantified the drive-response relationships between residues in the folding/unfolding processes of four small proteins generated by molecular dynamics simulations. Instead of using a time-averaged single TE value, the time-dependent TE is measured with the Q-scores based on residue-residue contacts and with the statistical significance analysis along the folding/unfolding processes. The TE analysis is able to identify the driving and responding residues that are different from the highly correlated residues revealed by the mutual information analysis. In general, the driving residues have more regular secondary structures, are more buried, and show greater effects on the protein stability as well as folding and unfolding rates. In addition, the dominant driving and responding residues from the TE analysis on the whole trajectory agree with those on a single folding event, demonstrating that the drive-response relationships are preserved in the non-equilibrium process. Our study provides detailed insights into the protein folding process and has potential applications in protein engineering and interpretation of time-dependent residue-based experimental observables for protein function.

Exploring default mode and information flow on the web

Social networking services (e.g., Twitter, Facebook) are now major sources of World Wide Web (called “Web”) dynamics, together with Web search services (e.g., Google). These two types of Web services mutually influence each other but generate different dynamics. In this paper, we distinguish two modes of Web dynamics: the reactive mode and the default mode. It is assumed that Twitter messages (called “tweets”) and Google search queries react to significant social movements and events, but they also demonstrate signs of becoming self-activated, thereby forming a baseline Web activity. We define the former as the reactive mode and the latter as the default mode of the Web. In this paper, we investigate these reactive and default modes of the Web's dynamics using transfer entropy (TE). The amount of information transferred between a time series of 1,000 frequent keywords in Twitter and the same keywords in Google queries is investigated across an 11-month time period. Study of the information flow on Google and Twitter revealed that information is generally transferred from Twitter to Google, indicating that Twitter time series have some preceding information about Google time series. We also studied the information flow among different Twitter keywords time series by taking keywords as nodes and flow directions as edges of a network. An analysis of this network revealed that frequent keywords tend to become an information source and infrequent keywords tend to become sink for other keywords. Based on these findings, we hypothesize that frequent keywords form the Web's default mode, which becomes an information source for infrequent keywords that generally form the Web's reactive mode. We also found that the Web consists of different time resolutions with respect to TE among Twitter keywords, which will be another focal point of this paper.

Altered cortical communication in amyotrophic lateral sclerosis

Amyotrophic lateral sclerosis (ALS) is a disorder associated primarily with the degeneration of the motor system. More recently, functional connectivity studies have demonstrated potentially adaptive changes in ALS brain organization, but disease-related changes in cortical communication remain unknown. We recruited individuals with ALS and age-matched controls to operate a brain-computer interface while electroencephalography was recorded over three sessions. Using normalized symbolic transfer entropy, we measured directed functional connectivity from frontal to parietal (feedback connectivity) and parietal to frontal (feedforward connectivity) regions. Feedback connectivity was not significantly different between groups, but feedforward connectivity was significantly higher in individuals with ALS. This result was consistent across a broad electroencephalographic spectrum (4-35 Hz), and in theta, alpha and beta frequency bands. Feedback connectivity has been associated with conscious state and was found to be independent of ALS symptom severity in this study, which may have significant implications for the detection of consciousness in individuals with advanced ALS. We suggest that increases in feedforward connectivity represent a compensatory response to the ALS-related loss of input such that sensory stimuli have sufficient strength to cross the threshold necessary for conscious processing in the global neuronal workspace.

Spurious causalities with transfer entropy

Transfer entropy (TE) seems currently to be the most widely used tool to characterize causal influences in ensembles of complex systems from observed time series. In particular, in an elemental case of two systems, nonzero TEs in both directions are usually interpreted as a sign of a bidirectional coupling. However, one often overlooks that both positive TEs may well be encountered for unidirectionally coupled systems so that a false detection of a causal influence on the basis of a nonzero TE is rather possible. This work highlights typical factors leading to such "spurious couplings": (i) unobserved state variables of the driving system, (ii) low temporal resolution, and (iii) observation errors. All are shown to be particular cases of a general problem: imperfect observations of states of the driving system. Importantly, exact values of TEs, rather than their statistical estimates, are computed here for selected benchmark systems. Conditions for a "spurious" TE to be large and even strongly exceed a "correct" TE are presented and discussed.

K-nearest-neighbor conditional entropy approach for the assessment of the short-term complexity of cardiovascular control

Complexity analysis of short-term cardiovascular control is traditionally performed using entropy-based approaches including corrective terms or strategies to cope with the loss of reliability of conditional distributions with pattern length. This study proposes a new approach aiming at the estimation of conditional entropy (CE) from short data segments (about 250 samples) based on the k-nearest-neighbor technique. The main advantages are: (i) the control of the loss of reliability of the conditional distributions with the pattern length without introducing a priori information; (ii) the assessment of complexity indexes without fixing the pattern length to an arbitrary low value. The approach, referred to as k-nearest-neighbor conditional entropy (KNNCE), was contrasted with corrected approximate entropy (CApEn), sample entropy (SampEn) and corrected CE (CCE), being the most frequently exploited approaches for entropy-based complexity analysis of short cardiovascular series. Complexity indexes were evaluated during the selective pharmacological blockade of the vagal and/or sympathetic branches of the autonomic nervous system. We found that KNNCE was more powerful than CCE in detecting the decrease of complexity of heart period variability imposed by double autonomic blockade. In addition, KNNCE provides indexes indistinguishable from those derived from CApEn and SampEn. Since this result was obtained without using strategies to correct the CE estimate and without fixing the embedding dimension to an arbitrary low value, KNNCE is potentially more valuable than CCE, CApEn and SampEn when the number of past samples most useful to reduce the uncertainty of future behaviors is high and/or variable among conditions and/or groups.

Expanding the transfer entropy to identify information circuits in complex systems

We propose a formal expansion of the transfer entropy to put in evidence irreducible sets of variables which provide information for the future state of each assigned target. Multiplets characterized by a large contribution to the expansion are associated to the informational circuits present in the system, with an informational character which can be associated to the sign of the contribution. For the sake of computational complexity, we adopt the assumption of Gaussianity and use the corresponding exact formula for the conditional mutual information. We report the application of the proposed methodology on two electroencephalography (EEG) data sets.

Dynamical Bayesian inference of time-evolving interactions: from a pair of coupled oscillators to networks of oscillators

Living systems have time-evolving interactions that, until recently, could not be identified accurately from recorded time series in the presence of noise. Stankovski et al. [Phys. Rev. Lett. 109, 024101 (2012)] introduced a method based on dynamical Bayesian inference that facilitates the simultaneous detection of time-varying synchronization, directionality of influence, and coupling functions. It can distinguish unsynchronized dynamics from noise-induced phase slips. The method is based on phase dynamics, with Bayesian inference of the time-evolving parameters being achieved by shaping the prior densities to incorporate knowledge of previous samples. We now present the method in detail using numerically generated data, data from an analog electronic circuit, and cardiorespiratory data. We also generalize the method to encompass networks of interacting oscillators and thus demonstrate its applicability to small-scale networks.

Order Patterns Networks (ORPAN)-a method to estimate time-evolving functional connectivity from multivariate time series

Complex networks provide an excellent framework for studying the function of the human brain activity. Yet estimating functional networks from measured signals is not trivial, especially if the data is non-stationary and noisy as it is often the case with physiological recordings. In this article we propose a method that uses the local rank structure of the data to define functional links in terms of identical rank structures. The method yields temporal sequences of networks which permits to trace the evolution of the functional connectivity during the time course of the observation. We demonstrate the potentials of this approach with model data as well as with experimental data from an electrophysiological study on language processing.

Transfer entropy as a log-likelihood ratio

Transfer entropy, an information-theoretic measure of time-directed information transfer between joint processes, has steadily gained popularity in the analysis of complex stochastic dynamics in diverse fields, including the neurosciences, ecology, climatology, and econometrics. We show that for a broad class of predictive models, the log-likelihood ratio test statistic for the null hypothesis of zero transfer entropy is a consistent estimator for the transfer entropy itself. For finite Markov chains, furthermore, no explicit model is required. In the general case, an asymptotic χ2 distribution is established for the transfer entropy estimator. The result generalizes the equivalence in the Gaussian case of transfer entropy and Granger causality, a statistical notion of causal influence based on prediction via vector autoregression, and establishes a fundamental connection between directed information transfer and causality in the Wiener-Granger sense.

Information flow in networks and the law of diminishing marginal returns: evidence from modeling and human electroencephalographic recordings

We analyze simple dynamical network models which describe the limited capacity of nodes to process the input information. For a proper range of their parameters, the information flow pattern in these models is characterized by exponential distribution of the incoming information and a fat-tailed distribution of the outgoing information, as a signature of the law of diminishing marginal returns. We apply this analysis to effective connectivity networks from human EEG signals, obtained by Granger Causality, which has recently been given an interpretation in the framework of information theory. From the distributions of the incoming versus the outgoing values of the information flow it is evident that the incoming information is exponentially distributed whilst the outgoing information shows a fat tail. This suggests that overall brain effective connectivity networks may also be considered in the light of the law of diminishing marginal returns. Interestingly, this pattern is reproduced locally but with a clear modulation: a topographic analysis has also been made considering the distribution of incoming and outgoing values at each electrode, suggesting a functional role for this phenomenon.

Synchronisation effects on the behavioural performance and information dynamics of a simulated minimally cognitive robotic agent

Oscillatory activity is ubiquitous in nervous systems, with solid evidence that synchronisation mechanisms underpin cognitive processes. Nevertheless, its informational content and relationship with behaviour are still to be fully understood. In addition, cognitive systems cannot be properly appreciated without taking into account brain-body- environment interactions. In this paper, we developed a model based on the Kuramoto Model of coupled phase oscillators to explore the role of neural synchronisation in the performance of a simulated robotic agent in two different minimally cognitive tasks. We show that there is a statistically significant difference in performance and evolvability depending on the synchronisation regime of the network. In both tasks, a combination of information flow and dynamical analyses show that networks with a definite, but not too strong, propensity for synchronisation are more able to reconfigure, to organise themselves functionally and to adapt to different behavioural conditions. The results highlight the asymmetry of information flow and its behavioural correspondence. Importantly, it also shows that neural synchronisation dynamics, when suitably flexible and reconfigurable, can generate minimally cognitive embodied behaviour.

Inference of time-evolving coupled dynamical systems in the presence of noise

A new method is introduced for analysis of interactions between time-dependent coupled oscillators, based on the signals they generate. It distinguishes unsynchronized dynamics from noise-induced phase slips and enables the evolution of the coupling functions and other parameters to be followed. It is based on phase dynamics, with Bayesian inference of the time-evolving parameters achieved by shaping the prior densities to incorporate knowledge of previous samples. The method is tested numerically and applied to reveal and quantify the time-varying nature of cardiorespiratory interactions.

Towards the prediction of order parameters from molecular dynamics simulations in proteins

A molecular understanding of how protein function is related to protein structure requires an ability to understand large conformational changes between multiple states. Unfortunately these states are often separated by high free energy barriers and within a complex energy landscape. This makes it very difficult to reliably connect, for example by all-atom molecular dynamics calculations, the states, their energies, and the pathways between them. A major issue needed to improve sampling on the intermediate states is an order parameter--a reduced descriptor for the major subset of degrees of freedom--that can be used to aid sampling for the large conformational change. We present a method to combine information from molecular dynamics using non-linear time series and dimensionality reduction, in order to quantitatively determine an order parameter connecting two large-scale conformationally distinct protein states. This new method suggests an implementation for molecular dynamics calculations that may be used to enhance sampling of intermediate states.