Symbolic transfer entropy

Cross-Frequency Transfer Entropy Characterize Coupling of Interacting Nonlinear Oscillators in Complex Systems

The purpose of this study is to introduce a method in quantifying cross-frequency information transfer to characterize directional couplers between irregular oscillations in complex systems. Importantly, the method should be able to reflect the intrinsic mechanism of interacting oscillations faithfully. Six types of interacting oscillators, including phase-amplitude, amplitude-amplitude, and component-amplitude cross-frequency transfer entropy as well as their inverse transfer entropies, are within our scope in untangling the brain connectivity. Challenges with nonlinear and nonstationary patterns are designed to validate the robustness of the proposed method. We suggest this approach could be effective in identifying driving and responding elements of interacting oscillators across different time scales. Meanwhile, an atlas of interacting oscillators in sleep is constructed. High-frequency amplitude can inversely drive low-frequency phase stronger than the standard phase-amplitude coupling, and the low-frequency amplitude can be the driving force to the high-frequency amplitude in addition to the low-frequency phase. Unlike the standard phase-amplitude coupling, the proposed cross-frequency transfer entropy is applicable to quantify the interactions across phases, amplitudes, or even the components without methodological adjustments. Meanwhile, the exploration of causal relationship enables the identification of the driving force of information flow.

Information-Theoretic Performance Analysis of Sensor Networks via Markov Modeling of Time Series Data

This paper presents information-theoretic performance analysis of passive sensor networks for detection of moving targets. The proposed method falls largely under the category of data-level information fusion in sensor networks. To this end, a measure of information contribution for sensors is formulated in a symbolic dynamics framework. The network information state is approximately represented as the largest principal component of the time series collected across the network. To quantify each sensor's contribution for generation of the information content, Markov machine models as well as x-Markov (pronounced as cross-Markov) machine models, conditioned on the network information state, are constructed; the difference between the conditional entropies of these machines is then treated as an approximate measure of information contribution by the respective sensors. The x-Markov models represent the conditional temporal statistics given the network information state. The proposed method has been validated on experimental data collected from a local area network of passive sensors for target detection, where the statistical characteristics of environmental disturbances are similar to those of the target signal in the sense of time scale and texture. A distinctive feature of the proposed algorithm is that the network decisions are independent of the behavior and identity of the individual sensors, which is desirable from computational perspectives. Results are presented to demonstrate the proposed method's efficacy to correctly identify the presence of a target with very low false-alarm rates. The performance of the underlying algorithm is compared with that of a recent data-driven, feature-level information fusion algorithm. It is shown that the proposed algorithm outperforms the other algorithm.

A Tutorial for Information Theory in Neuroscience

Understanding how neural systems integrate, encode, and compute information is central to understanding brain function. Frequently, data from neuroscience experiments are multivariate, the interactions between the variables are nonlinear, and the landscape of hypothesized or possible interactions between variables is extremely broad. Information theory is well suited to address these types of data, as it possesses multivariate analysis tools, it can be applied to many different types of data, it can capture nonlinear interactions, and it does not require assumptions about the structure of the underlying data (i.e., it is model independent). In this article, we walk through the mathematics of information theory along with common logistical problems associated with data type, data binning, data quantity requirements, bias, and significance testing. Next, we analyze models inspired by canonical neuroscience experiments to improve understanding and demonstrate the strengths of information theory analyses. To facilitate the use of information theory analyses, and an understanding of how these analyses are implemented, we also provide a free MATLAB software package that can be applied to a wide range of data from neuroscience experiments, as well as from other fields of study.

Brief Introduction to Electroencephalography

Electroencephalography (EEG) has a long history in neuroscience starting with its original description in humans by Hans Berger in 1929 (Berger, 1932). Investigations of EEG under anesthesia started soon after in the mid-1930s (Gibbs, 1937). No single methodology paper can credibly cover all of the issues relating to this rich field. The purpose of this chapter is to introduce some caveats that complicate and inform analysis of the EEG. Special emphasis will be given to common issues such as choice of reference electrode, filtering, artifact rejection, and spectral analysis. We will specifically emphasize high-density EEG recordings that have become the norm due to technological improvement in electrode and data acquisition design methods. In the last section we will discuss some applications of EEG analysis techniques to the study of the effects of anesthetics on the nervous system.

Sleep in patients with disorders of consciousness characterized by means of machine learning

Sleep has been proposed to indicate preserved residual brain functioning in patients suffering from disorders of consciousness (DOC) after awakening from coma. However, a reliable characterization of sleep patterns in this clinical population continues to be challenging given severely altered brain oscillations, frequent and extended artifacts in clinical recordings and the absence of established staging criteria. In the present study, we try to address these issues and investigate the usefulness of a multivariate machine learning technique based on permutation entropy, a complexity measure. Specifically, we used long-term polysomnography (PSG), along with video recordings in day and night periods in a sample of 23 DOC; 12 patients were diagnosed as Unresponsive Wakefulness Syndrome (UWS) and 11 were diagnosed as Minimally Conscious State (MCS). Eight hour PSG recordings of healthy sleepers (N = 26) were additionally used for training and setting parameters of supervised and unsupervised model, respectively. In DOC, the supervised classification (wake, N1, N2, N3 or REM) was validated using simultaneous videos which identified periods with prolonged eye opening or eye closure.The supervised classification revealed that out of the 23 subjects, 11 patients (5 MCS and 6 UWS) yielded highly accurate classification with an average F1-score of 0.87 representing high overlap between the classifier predicting sleep (i.e. one of the 4 sleep stages) and closed eyes. Furthermore, the unsupervised approach revealed a more complex pattern of sleep-wake stages during the night period in the MCS group, as evidenced by the presence of several distinct clusters. In contrast, in UWS patients no such clustering was found. Altogether, we present a novel data-driven method, based on machine learning that can be used to gain new and unambiguous insights into sleep organization and residual brain functioning of patients with DOC.

The influence of filtering and downsampling on the estimation of transfer entropy

Transfer entropy (TE) provides a generalized and model-free framework to study Wiener-Granger causality between brain regions. Because of its nonparametric character, TE can infer directed information flow also from nonlinear systems. Despite its increasing number of applications in neuroscience, not much is known regarding the influence of common electrophysiological preprocessing on its estimation. We test the influence of filtering and downsampling on a recently proposed nearest neighborhood based TE estimator. Different filter settings and downsampling factors were tested in a simulation framework using a model with a linear coupling function and two nonlinear models with sigmoid and logistic coupling functions. For nonlinear coupling and progressively lower low-pass filter cut-off frequencies up to 72% false negative direct connections and up to 26% false positive connections were identified. In contrast, for the linear model, a monotonic increase was only observed for missed indirect connections (up to 86%). High-pass filtering (1 Hz, 2 Hz) had no impact on TE estimation. After low-pass filtering interaction delays were significantly underestimated. Downsampling the data by a factor greater than the assumed interaction delay erased most of the transmitted information and thus led to a very high percentage (67–100%) of false negative direct connections. Low-pass filtering increases the number of missed connections depending on the filters cut-off frequency. Downsampling should only be done if the sampling factor is smaller than the smallest assumed interaction delay of the analyzed network.

Symbolic dynamics of animal interaction

Since its introduction nearly two decades ago, transfer entropy has contributed to an improved understanding of cause-and-effect relationships in coupled dynamical systems from raw time series. In the context of animal behavior, transfer entropy might help explain the determinants of leadership in social groups and elucidate escape response to predator attacks. Despite its promise, the potential of transfer entropy in animal behavior is yet to be fully tested, and a number of technical challenges in information theory and statistics remain open. Here, we examine an alternative approach to the computation of transfer entropy based on symbolic dynamics. In this context, a symbol is associated with a specific locomotory bout across two or more consecutive time instants, such as reversing the swimming direction. Symbols encapsulate salient locomotory patterns and the associated permutation transfer entropy quantifies the ability to predict the patterns of an individual given those of another individual. We demonstrate this framework on an existing dataset on fish, for which we have knowledge of the underlying cause-and-effect relationship between the focal subject and the stimulus. Symbolic dynamics offers an intuitive and robust approach to study animal behavior, which could enable the inference of causal relationship from noisy experimental observations of limited duration.

Study of the cross-market effects of Brexit based on the improved symbolic transfer entropy GARCH model-An empirical analysis of stock-bond correlations

In this paper, we study the cross-market effects of Brexit on the stock and bond markets of nine major countries in the world. By incorporating information theory, we introduce the time-varying impact weights based on symbolic transfer entropy to improve the traditional GARCH model. The empirical results show that under the influence of Brexit, flight-to-quality not only commonly occurs between the stocks and bonds of each country but also simultaneously occurs among different countries. We also find that the accuracy of the time-varying symbolic transfer entropy GARCH model proposed in this paper has been improved compared to the traditional GARCH model, which indicates that it has a certain practical application value.

Network Inference and Maximum Entropy Estimation on Information Diagrams

Maximum entropy estimation is of broad interest for inferring properties of systems across many disciplines. Using a recently introduced technique for estimating the maximum entropy of a set of random discrete variables when conditioning on bivariate mutual informations and univariate entropies, we show how this can be used to estimate the direct network connectivity between interacting units from observed activity. As a generic example, we consider phase oscillators and show that our approach is typically superior to simply using the mutual information. In addition, we propose a nonparametric formulation of connected informations, used to test the explanatory power of a network description in general. We give an illustrative example showing how this agrees with the existing parametric formulation, and demonstrate its applicability and advantages for resting-state human brain networks, for which we also discuss its direct effective connectivity. Finally, we generalize to continuous random variables and vastly expand the types of information-theoretic quantities one can condition on. This allows us to establish significant advantages of this approach over existing ones. Not only does our method perform favorably in the undersampled regime, where existing methods fail, but it also can be dramatically less computationally expensive as the cardinality of the variables increases.

Assessment of resampling methods for causality testing: A note on the US inflation behavior

Different resampling methods for the null hypothesis of no Granger causality are assessed in the setting of multivariate time series, taking into account that the driving-response coupling is conditioned on the other observed variables. As appropriate test statistic for this setting, the partial transfer entropy (PTE), an information and model-free measure, is used. Two resampling techniques, time-shifted surrogates and the stationary bootstrap, are combined with three independence settings (giving a total of six resampling methods), all approximating the null hypothesis of no Granger causality. In these three settings, the level of dependence is changed, while the conditioning variables remain intact. The empirical null distribution of the PTE, as the surrogate and bootstrapped time series become more independent, is examined along with the size and power of the respective tests. Additionally, we consider a seventh resampling method by contemporaneously resampling the driving and the response time series using the stationary bootstrap. Although this case does not comply with the no causality hypothesis, one can obtain an accurate sampling distribution for the mean of the test statistic since its value is zero under H0. Results indicate that as the resampling setting gets more independent, the test becomes more conservative. Finally, we conclude with a real application. More specifically, we investigate the causal links among the growth rates for the US CPI, money supply and crude oil. Based on the PTE and the seven resampling methods, we consistently find that changes in crude oil cause inflation conditioning on money supply in the post-1986 period. However this relationship cannot be explained on the basis of traditional cost-push mechanisms.

Reconstructing networks of pulse-coupled oscillators from spike trains

We present an approach for reconstructing networks of pulse-coupled neuronlike oscillators from passive observation of pulse trains of all nodes. It is assumed that units are described by their phase response curves and that their phases are instantaneously reset by incoming pulses. Using an iterative procedure, we recover the properties of all nodes, namely their phase response curves and natural frequencies, as well as strengths of all directed connections.

Directional information flow in patients with Alzheimer's disease. A source-space resting-state MEG study

In a recent magnetoencephalography (MEG) study, we found posterior-to-anterior information flow over the cortex in higher frequency bands in healthy subjects, with a reversed pattern in the theta band. A disruption of information flow may underlie clinical symptoms in Alzheimer's disease (AD). In AD, highly connected regions (hubs) in posterior areas are mostly disrupted. We therefore hypothesized that in AD the information flow from these hub regions would be disturbed. We used resting-state MEG recordings from 27 early-onset AD patients and 26 healthy controls. Using beamformer-based virtual electrodes, we estimated neuronal oscillatory activity for 78 cortical regions of interest (ROIs) and 12 subcortical ROIs of the AAL atlas, and calculated the directed phase transfer entropy (dPTE) as a measure of information flow between these ROIs. Group differences were evaluated using permutation tests and, for the AD group, associations between dPTE and general cognition or CSF biomarkers were determined using Spearman correlation coefficients. We confirmed the previously reported posterior-to-anterior information flow in the higher frequency bands in the healthy controls, and found it to be disturbed in the beta band in AD. Most prominently, the information flow from the precuneus and the visual cortex, towards frontal and subcortical structures, was decreased in AD. These disruptions did not correlate with cognitive impairment or CSF biomarkers. We conclude that AD pathology may affect the flow of information between brain regions, particularly from posterior hub regions, and that changes in the information flow in the beta band indicate an aspect of the pathophysiological process in AD.

Neural Correlates of Sevoflurane-induced Unconsciousness Identified by Simultaneous Functional Magnetic Resonance Imaging and Electroencephalography

BACKGROUND

The neural correlates of anesthetic-induced unconsciousness have yet to be fully elucidated. Sedative and anesthetic states induced by propofol have been studied extensively, consistently revealing a decrease of frontoparietal and thalamocortical connectivity. There is, however, less understanding of the effects of halogenated ethers on functional brain networks.

METHODS

The authors recorded simultaneous resting-state functional magnetic resonance imaging and electroencephalography in 16 artificially ventilated volunteers during sevoflurane anesthesia at burst suppression and 3 and 2 vol% steady-state concentrations for 700 s each to assess functional connectivity changes compared to wakefulness. Electroencephalographic data were analyzed using symbolic transfer entropy (surrogate of information transfer) and permutation entropy (surrogate of cortical information processing). Functional magnetic resonance imaging data were analyzed by an independent component analysis and a region-of-interest-based analysis.

RESULTS

Electroencephalographic analysis showed a significant reduction of anterior-to-posterior symbolic transfer entropy and global permutation entropy. At 2 vol% sevoflurane concentrations, frontal and thalamic networks identified by independent component analysis showed significantly reduced within-network connectivity. Primary sensory networks did not show a significant change. At burst suppression, all cortical networks showed significantly reduced functional connectivity. Region-of-interest-based thalamic connectivity at 2 vol% was significantly reduced to frontoparietal and posterior cingulate cortices but not to sensory areas.

CONCLUSIONS

Sevoflurane decreased frontal and thalamocortical connectivity. The changes in blood oxygenation level dependent connectivity were consistent with reduced anterior-to-posterior directed connectivity and reduced cortical information processing. These data advance the understanding of sevoflurane-induced unconsciousness and contribute to a neural basis of electroencephalographic measures that hold promise for intraoperative anesthesia monitoring.

Breakdown of local information processing may underlie isoflurane anesthesia effects

The disruption of coupling between brain areas has been suggested as the mechanism underlying loss of consciousness in anesthesia. This hypothesis has been tested previously by measuring the information transfer between brain areas, and by taking reduced information transfer as a proxy for decoupling. Yet, information transfer is a function of the amount of information available in the information source—such that transfer decreases even for unchanged coupling when less source information is available. Therefore, we reconsidered past interpretations of reduced information transfer as a sign of decoupling, and asked whether impaired local information processing leads to a loss of information transfer. An important prediction of this alternative hypothesis is that changes in locally available information (signal entropy) should be at least as pronounced as changes in information transfer. We tested this prediction by recording local field potentials in two ferrets after administration of isoflurane in concentrations of 0.0%, 0.5%, and 1.0%. We found strong decreases in the source entropy under isoflurane in area V1 and the prefrontal cortex (PFC)—as predicted by our alternative hypothesis. The decrease in source entropy was stronger in PFC compared to V1. Information transfer between V1 and PFC was reduced bidirectionally, but with a stronger decrease from PFC to V1. This links the stronger decrease in information transfer to the stronger decrease in source entropy—suggesting reduced source entropy reduces information transfer. This conclusion fits the observation that the synaptic targets of isoflurane are located in local cortical circuits rather than on the synapses formed by interareal axonal projections. Thus, changes in information transfer under isoflurane seem to be a consequence of changes in local processing more than of decoupling between brain areas. We suggest that source entropy changes must be considered whenever interpreting changes in information transfer as decoupling.

Currently we do not understand how anesthesia leads to loss of consciousness (LOC). One popular idea is that we loose consciousness when brain areas lose their ability to communicate with each other–as anesthetics might interrupt transmission on nerve fibers coupling them. This idea has been tested by measuring the amount of information transferred between brain areas, and taking this transfer to reflect the coupling itself. Yet, information that isn’t available in the source area can’t be transferred to a target. Hence, the decreases in information transfer could be related to less information being available in the source, rather than to a decoupling. We tested this possibility measuring the information available in source brain areas and found that it decreased under isoflurane anesthesia. In addition, a stronger decrease in source information lead to a stronger decrease of the information transfered. Thus, the input to the connection between brain areas determined the communicated information, not the strength of the coupling (which would result in a stronger decrease in the target). We suggest that interrupted information processing within brain areas has an important contribution to LOC, and should be focused on more in attempts to understand loss of consciousness under anesthesia.

Emerging interdependence between stock values during financial crashes

To identify emerging interdependencies between traded stocks we investigate the behavior of the stocks of FTSE 100 companies in the period 2000-2015, by looking at daily stock values. Exploiting the power of information theoretical measures to extract direct influences between multiple time series, we compute the information flow across stock values to identify several different regimes. While small information flows is detected in most of the period, a dramatically different situation occurs in the proximity of global financial crises, where stock values exhibit strong and substantial interdependence for a prolonged period. This behavior is consistent with what one would generally expect from a complex system near criticality in physical systems, showing the long lasting effects of crashes on stock markets.

Horizontal visibility graph transfer entropy (HVG-TE): A novel metric to characterize directed connectivity in large-scale brain networks

We propose a new measure, horizontal visibility graph transfer entropy (HVG-TE), to estimate the direction of information flow between pairs of time series. HVG-TE quantifies the transfer entropy between the degree sequences of horizontal visibility graphs derived from original time series. Twenty-one Rössler attractors unidirectionally coupled in the posterior-to-anterior direction were used to simulate 21-channel Electroencephalography (EEG) brain networks and validate the performance of the HVG-TE. We showed that the HVG-TE is robust to different levels of coupling strengths between the coupled Rössler attractors, a wide range of time delays, different sample sizes, the effects of noise and linear mixing, and the choice of reference for EEG data. We also applied HVG-TE to EEG data in 20 healthy controls and compared its performance to a recently introduces phase-based TE measure (PTE). We found that compared with PTE, HVG-TE consistently detected stronger posterior-to-anterior information flow patterns in the alpha-band (8-13Hz) EEG brain networks for three different references. Moreover, in contrast to PTE, HVG-TE does not require an assumption on the periodicity of input signals, therefore it can be more widely applicable, even for non-periodic signals. This study shows that the HVG-TE is a directed connectivity measure to characterise the direction of information flow in large-scale brain networks.

Structure Shapes Dynamics and Directionality in Diverse Brain Networks: Mathematical Principles and Empirical Confirmation in Three Species

Identifying how spatially distributed information becomes integrated in the brain is essential to understanding higher cognitive functions. Previous computational and empirical studies suggest a significant influence of brain network structure on brain network function. However, there have been few analytical approaches to explain the role of network structure in shaping regional activities and directionality patterns. In this study, analytical methods are applied to a coupled oscillator model implemented in inhomogeneous networks. We first derive a mathematical principle that explains the emergence of directionality from the underlying brain network structure. We then apply the analytical methods to the anatomical brain networks of human, macaque, and mouse, successfully predicting simulation and empirical electroencephalographic data. The results demonstrate that the global directionality patterns in resting state brain networks can be predicted solely by their unique network structures. This study forms a foundation for a more comprehensive understanding of how neural information is directed and integrated in complex brain networks.

Reconstructing the structure of directed and weighted networks of nonlinear oscillators

The formalism of complex networks is extensively employed to describe the dynamics of interacting agents in several applications. The features of the connections among the nodes in a network are not always provided beforehand, hence the problem of appropriately inferring them often arises. Here, we present a method to reconstruct directed and weighted topologies of networks of heterogeneous nonlinear oscillators. We illustrate the theory on a set of representative examples.

Accelerated Recovery of Consciousness after General Anesthesia Is Associated with Increased Functional Brain Connectivity in the High-Gamma Bandwidth

Recent data from our laboratory demonstrate that high-frequency gamma connectivity across the cortex is present during consciousness and depressed during unconsciousness. However, these data were derived from static and well-defined states of arousal rather than during transitions that would suggest functional relevance. We also recently found that subanesthetic ketamine administered during isoflurane anesthesia accelerates recovery upon discontinuation of the primary anesthetic and increases gamma power during emergence. In the current study we re-analyzed electroencephalogram (EEG) data to test the hypothesis that functional cortical connectivity between anterior and posterior cortical regions would be increased during accelerated recovery induced by ketamine when compared to saline-treated controls. Rodents were instrumented with intracranial EEG electrodes and general anesthesia was induced with isoflurane anesthesia. After 37.5 min of continuous isoflurane anesthesia, a subanesthetic dose of ketamine (25 mg/kg intraperitoneal) was administered, with evidence of a 44% reduction in emergence time. In this study, we analyzed gamma and theta coherence (measure of undirected functional connectivity) and normalized symbolic transfer entropy (measure of directed functional connectivity) between frontal and parietal cortices during various levels of consciousness, with a focus on emergence from isoflurane anesthesia. During accelerated emergence in the ketamine-treated group, there was increased frontal-parietal coherence {p = 0.005, 0.05-0.23 [95% confidence interval (CI)]} and normalized symbolic transfer entropy [frontal to parietal: p < 0.001, 0.010-0.026 (95% CI); parietal to frontal: p < 0.001, 0.009-0.025 (95% CI)] in high-frequency gamma bandwidth as compared with the saline-treated group. Surrogates of cortical information exchange in high-frequency gamma are increased in association with accelerated recovery from anesthesia. This finding adds evidence suggesting a functional significance of high-gamma information transfer in consciousness.

Time-varying coupling functions: Dynamical inference and cause of synchronization transitions

Interactions in nature can be described by their coupling strength, direction of coupling, and coupling function. The coupling strength and directionality are relatively well understood and studied, at least for two interacting systems; however, there can be a complexity in the interactions uniquely dependent on the coupling functions. Such a special case is studied here: synchronization transition occurs only due to the time variability of the coupling functions, while the net coupling strength is constant throughout the observation time. To motivate the investigation, an example is used to present an analysis of cross-frequency coupling functions between delta and alpha brain waves extracted from the electroencephalography recording of a healthy human subject in a free-running resting state. The results indicate that time-varying coupling functions are a reality for biological interactions. A model of phase oscillators is used to demonstrate and detect the synchronization transition caused by the varying coupling functions during an invariant coupling strength. The ability to detect this phenomenon is discussed with the method of dynamical Bayesian inference, which was able to infer the time-varying coupling functions. The form of the coupling function acts as an additional dimension for the interactions, and it should be taken into account when detecting biological or other interactions from data.

Backward transfer entropy: Informational measure for detecting hidden Markov models and its interpretations in thermodynamics, gambling and causality

The transfer entropy is a well-established measure of information flow, which quantifies directed influence between two stochastic time series and has been shown to be useful in a variety fields of science. Here we introduce the transfer entropy of the backward time series called the backward transfer entropy, and show that the backward transfer entropy quantifies how far it is from dynamics to a hidden Markov model. Furthermore, we discuss physical interpretations of the backward transfer entropy in completely different settings of thermodynamics for information processing and the gambling with side information. In both settings of thermodynamics and the gambling, the backward transfer entropy characterizes a possible loss of some benefit, where the conventional transfer entropy characterizes a possible benefit. Our result implies the deep connection between thermodynamics and the gambling in the presence of information flow, and that the backward transfer entropy would be useful as a novel measure of information flow in nonequilibrium thermodynamics, biochemical sciences, economics and statistics.

Inferring Weighted Directed Association Network from Multivariate Time Series with a Synthetic Method of Partial Symbolic Transfer Entropy Spectrum and Granger Causality

Complex network methodology is very useful for complex system explorer. However, the relationships among variables in complex system are usually not clear. Therefore, inferring association networks among variables from their observed data has been a popular research topic. We propose a synthetic method, named small-shuffle partial symbolic transfer entropy spectrum (SSPSTES), for inferring association network from multivariate time series. The method synthesizes surrogate data, partial symbolic transfer entropy (PSTE) and Granger causality. A proper threshold selection is crucial for common correlation identification methods and it is not easy for users. The proposed method can not only identify the strong correlation without selecting a threshold but also has the ability of correlation quantification, direction identification and temporal relation identification. The method can be divided into three layers, i.e. data layer, model layer and network layer. In the model layer, the method identifies all the possible pair-wise correlation. In the network layer, we introduce a filter algorithm to remove the indirect weak correlation and retain strong correlation. Finally, we build a weighted adjacency matrix, the value of each entry representing the correlation level between pair-wise variables, and then get the weighted directed association network. Two numerical simulated data from linear system and nonlinear system are illustrated to show the steps and performance of the proposed approach. The ability of the proposed method is approved by an application finally.

Information flow and causality as rigorous notions ab initio

Information flow or information transfer the widely applicable general physics notion can be rigorously derived from first principles, rather than axiomatically proposed as an ansatz. Its logical association with causality is firmly rooted in the dynamical system that lies beneath. The principle of nil causality that reads, an event is not causal to another if the evolution of the latter is independent of the former, which transfer entropy analysis and Granger causality test fail to verify in many situations, turns out to be a proven theorem here. Established in this study are the information flows among the components of time-discrete mappings and time-continuous dynamical systems, both deterministic and stochastic. They have been obtained explicitly in closed form, and put to applications with the benchmark systems such as the Kaplan-Yorke map, Rössler system, baker transformation, Hénon map, and stochastic potential flow. Besides unraveling the causal relations as expected from the respective systems, some of the applications show that the information flow structure underlying a complex trajectory pattern could be tractable. For linear systems, the resulting remarkably concise formula asserts analytically that causation implies correlation, while correlation does not imply causation, providing a mathematical basis for the long-standing philosophical debate over causation versus correlation.

Time evolution of interhemispheric coupling in a model of focal neocortical epilepsy in mice

Epilepsy is characterized by substantial network rearrangements leading to spontaneous seizures and little is known on how an epileptogenic focus impacts on neural activity in the contralateral hemisphere. Here, we used a model of unilateral epilepsy induced by injection of the synaptic blocker tetanus neurotoxin (TeNT) in the mouse primary visual cortex (V1). Local field potential (LFP) signals were simultaneously recorded from both hemispheres of each mouse in acute phase (peak of toxin action) and chronic condition (completion of TeNT effects). To characterize the neural electrical activities the corresponding LFP signals were analyzed with several methods of time series analysis. For the epileptic mice, the spectral analysis showed that TeNT determines a power redistribution among the different neurophysiological bands in both acute and chronic phases. Using linear and nonlinear interdependence measures in both time and frequency domains, it was found in the acute phase that TeNT injection promotes a reduction of the interhemispheric coupling for high frequencies (12-30 Hz) and small time lag (<20 ms), whereas an increase of the coupling is present for low frequencies (0.5-4 Hz) and long time lag (>40 ms). On the other hand, the chronic period is characterized by a partial or complete recovery of the interhemispheric interdependence level. Granger causality test and symbolic transfer entropy indicate a greater driving influence of the TeNT-injected side on activity in the contralateral hemisphere in the chronic phase. Lastly, based on experimental observations, we built a computational model of LFPs to investigate the role of the ipsilateral inhibition and exicitatory interhemispheric connections in the dampening of the interhemispheric coupling. The time evolution of the interhemispheric coupling in such a relevant model of epilepsy has been addressed here.

Two step transfer entropy - An estimator of delayed directional couplings between multivariate EEG time series

Quantifying delayed directional couplings between electroencephalographic (EEG) time series requires an efficient method of causal network inference. This is especially due to the limited knowledge about the underlying dynamics of the brain activity. Recent methods based on information theoretic measures such as Transfer Entropy (TE) made significant progress on this issue by providing a model-free framework for causality detection. However, TE estimation from observed data is not a trivial task, especially when the number of variables is large which is the case in a highly complex system like human brain. Here we propose a computationally efficient procedure for TE estimation based on using sets of the Most Informative Variables that effectively contribute to resolving the uncertainty of the destination. In the first step of this method, some conditioning sets are determined through a nonlinear state space reconstruction; then in the second step, optimal estimation of TE is done based on these sets. Validation of the proposed method using synthetic data and neurophysiological signals demonstrates computational efficiency in quantifying delayed directional couplings compared with the common TE analysis.

Weighted and directed interactions in evolving large-scale epileptic brain networks

Epilepsy can be regarded as a network phenomenon with functionally and/or structurally aberrant connections in the brain. Over the past years, concepts and methods from network theory substantially contributed to improve the characterization of structure and function of these epileptic networks and thus to advance understanding of the dynamical disease epilepsy. We extend this promising line of research and assess-with high spatial and temporal resolution and using complementary analysis approaches that capture different characteristics of the complex dynamics-both strength and direction of interactions in evolving large-scale epileptic brain networks of 35 patients that suffered from drug-resistant focal seizures with different anatomical onset locations. Despite this heterogeneity, we find that even during the seizure-free interval the seizure onset zone is a brain region that, when averaged over time, exerts strongest directed influences over other brain regions being part of a large-scale network. This crucial role, however, manifested by averaging on the population-sample level only - in more than one third of patients, strongest directed interactions can be observed between brain regions far off the seizure onset zone. This may guide new developments for individualized diagnosis, treatment and control.

On causality of extreme events

Multiple metrics have been developed to detect causality relations between data describing the elements constituting complex systems, all of them considering their evolution through time. Here we propose a metric able to detect causality within static data sets, by analysing how extreme events in one element correspond to the appearance of extreme events in a second one. The metric is able to detect non-linear causalities; to analyse both cross-sectional and longitudinal data sets; and to discriminate between real causalities and correlations caused by confounding factors. We validate the metric through synthetic data, dynamical and chaotic systems, and data representing the human brain activity in a cognitive task. We further show how the proposed metric is able to outperform classical causality metrics, provided non-linear relationships are present and large enough data sets are available.

Direction of information flow in large-scale resting-state networks is frequency-dependent

Normal brain function requires interactions between spatially separated, and functionally specialized, macroscopic regions, yet the directionality of these interactions in large-scale functional networks is unknown. Magnetoencephalography was used to determine the directionality of these interactions, where directionality was inferred from time series of beamformer-reconstructed estimates of neuronal activation, using a recently proposed measure of phase transfer entropy. We observed well-organized posterior-to-anterior patterns of information flow in the higher-frequency bands (alpha1, alpha2, and beta band), dominated by regions in the visual cortex and posterior default mode network. Opposite patterns of anterior-to-posterior flow were found in the theta band, involving mainly regions in the frontal lobe that were sending information to a more distributed network. Many strong information senders in the theta band were also frequent receivers in the alpha2 band, and vice versa. Our results provide evidence that large-scale resting-state patterns of information flow in the human brain form frequency-dependent reentry loops that are dominated by flow from parieto-occipital cortex to integrative frontal areas in the higher-frequency bands, which is mirrored by a theta band anterior-to-posterior flow.

The dynamics of information-driven coordination phenomena: A transfer entropy analysis

Data from social media provide unprecedented opportunities to investigate the processes that govern the dynamics of collective social phenomena. We consider an information theoretical approach to define and measure the temporal and structural signatures typical of collective social events as they arise and gain prominence. We use the symbolic transfer entropy analysis of microblogging time series to extract directed networks of influence among geolocalized subunits in social systems. This methodology captures the emergence of system-level dynamics close to the onset of socially relevant collective phenomena. The framework is validated against a detailed empirical analysis of five case studies. In particular, we identify a change in the characteristic time scale of the information transfer that flags the onset of information-driven collective phenomena. Furthermore, our approach identifies an order-disorder transition in the directed network of influence between social subunits. In the absence of clear exogenous driving, social collective phenomena can be represented as endogenously driven structural transitions of the information transfer network. This study provides results that can help define models and predictive algorithms for the analysis of societal events based on open source data.

From functional to structural connectivity using partial correlation in neuronal assemblies

OBJECTIVE

Our goal is to re-introduce an optimized version of the partial correlation to infer structural connections from functional-effective ones in dissociated neuronal cultures coupled to microelectrode arrays.

APPROACH

We first validate our partialization procedure on in silico networks, mimicking different experimental conditions (i.e., different connectivity degrees and number of nodes) and comparing the partial correlation's performance with two gold-standard methods: cross-correlation and transfer entropy. Afterwards, to infer the structural connections in in vitro neuronal networks where the ground truth is unknown, we propose a thresholding heuristic approach. Then, to validate whether the partialization process correctly reconstructs macroscopic features of the network structure, we extract a modularity index from segregated in silico and in vitro models. Finally, as a case study, we apply our partialization procedure to analyze connectivity and topology on spontaneous developing and electrically stimulated in vitro cultures.

MAIN RESULTS

In simulated networks, partial correlation outperforms cross-correlation and transfer entropy at low and medium connectivity degrees, not only in relatively small (60 nodes) but also in larger (120-240 nodes) assemblies. Furthermore, partial correlation correctly identifies interconnected neuronal sub-populations and allows one to derive network topology in in vitro cortical networks.

SIGNIFICANCE

Our results support the idea that partial correlation is a good method for connectivity studies and can be applied to derive topological and structural features of neuronal assemblies.

Oscillatory Diffusion and Second-Order Cyclostationarity in Alanine Tripeptide from Molecular Dynamics Simulation

Molecular dynamics (MD) simulation distinctly describes motions of biomolecules at high resolution and can potentially be used to explain allosteric mechanism in subcellular processes. Statistical methods are necessary to realize this potential because MD simulations generate a large volume of data and because the analysis is never efficient, objective, or thorough without using appropriate statistical approaches. Tracing the flow of information within a biomolecule requires not only a description of an overall mechanism but also a multiscale statistical description from atomic interactions to the overall mechanism. The foundation of this multiscale description, in general, is a measure of correlation between motions of atoms or residues, as reflected by dynamic cross-correlation, Pearson correlation, or mutual information. However, these correlations can be inadequate because they assume wide sense stationarity, which means that the instantaneous average and correlation of a particular property are time-independent. Consequently, these measures of correlation cannot account for correlation between motions of different frequencies, since frequency implies oscillation and variation over time. Here, we characterize the nonstationarity in the form of pure oscillatory instantaneous variance in the signed dihedral angular accelerations (SDAA) along the main chain of alanine tripeptide in MD simulations by power spectrum, corrected squared envelope spectrum (CSES), and cross-CSES. This oscillation has a physical interpretation of an oscillatory diffusion. The fraction of this oscillation in all motions is as high as about 40% at some frequencies. This shows that oscillatory instantaneous variance exists in the SDAA and that significant correlation may not be accounted for in current correlation analysis. This oscillation is also found to transmit between dihedral angles. These results could have implications in the understanding of the dynamics of biomolecules.

Causality networks from multivariate time series and application to epilepsy

Granger causality and variants of this concept allow the study of complex dynamical systems as networks constructed from multivariate time series. In this work, a large number of Granger causality measures used to form causality networks from multivariate time series are assessed. For this, realizations on high dimensional coupled dynamical systems are considered and the performance of the Granger causality measures is evaluated, seeking for the measures that form networks closest to the true network of the dynamical system. In particular, the comparison focuses on Granger causality measures that reduce the state space dimension when many variables are observed. Further, the linear and nonlinear Granger causality measures of dimension reduction are compared to a standard Granger causality measure on electroencephalographic (EEG) recordings containing episodes of epileptiform discharges.

The Virtual Teacher (VT) Paradigm: Learning New Patterns of Interpersonal Coordination Using the Human Dynamic Clamp

The Virtual Teacher paradigm, a version of the Human Dynamic Clamp (HDC), is introduced into studies of learning patterns of inter-personal coordination. Combining mathematical modeling and experimentation, we investigate how the HDC may be used as a Virtual Teacher (VT) to help humans co-produce and internalize new inter-personal coordination pattern(s). Human learners produced rhythmic finger movements whilst observing a computer-driven avatar, animated by dynamic equations stemming from the well-established Haken-Kelso-Bunz (1985) and Schöner-Kelso (1988) models of coordination. We demonstrate that the VT is successful in shifting the pattern co-produced by the VT-human system toward any value (Experiment 1) and that the VT can help humans learn unstable relative phasing patterns (Experiment 2). Using transfer entropy, we find that information flow from one partner to the other increases when VT-human coordination loses stability. This suggests that variable joint performance may actually facilitate interaction, and in the long run learning. VT appears to be a promising tool for exploring basic learning processes involved in social interaction, unraveling the dynamics of information flow between interacting partners, and providing possible rehabilitation opportunities.

Prognostic and diagnostic value of EEG signal coupling measures in coma

OBJECTIVE

Our aim was to assess the diagnostic and predictive value of several quantitative EEG (qEEG) analysis methods in comatose patients.

METHODS

In 79 patients, coupling between EEG signals on the left-right (inter-hemispheric) axis and on the anterior-posterior (intra-hemispheric) axis was measured with four synchronization measures: relative delta power asymmetry, cross-correlation, symbolic mutual information and transfer entropy directionality. Results were compared with etiology of coma and clinical outcome. Using cross-validation, the predictive value of measure combinations was assessed with a Bayes classifier with mixture of Gaussians.

RESULTS

Five of eight measures showed a statistically significant difference between patients grouped according to outcome; one measure revealed differences in patients grouped according to the etiology. Interestingly, a high level of synchrony between the left and right hemisphere was associated with mortality on intensive care unit, whereas higher synchrony between anterior and posterior brain regions was associated with survival. The combination with the best predictive value reached an area-under the curve of 0.875 (for patients with post anoxic encephalopathy: 0.946).

CONCLUSIONS

EEG synchronization measures can contribute to clinical assessment, and provide new approaches for understanding the pathophysiology of coma.

SIGNIFICANCE

Prognostication in coma remains a challenging task. qEEG could improve current multi-modal approaches.

Relating Granger causality to long-term causal effects

In estimation of causal couplings between observed processes, it is important to characterize coupling roles at various time scales. The widely used Granger causality reflects short-term effects: it shows how strongly perturbations of a current state of one process affect near future states of another process, and it quantifies that via prediction improvement (PI) in autoregressive models. However, it is often more important to evaluate the effects of coupling on long-term statistics, e.g., to find out how strongly the presence of coupling changes the variance of a driven process as compared to an uncoupled case. No general relationships between Granger causality and such long-term effects are known. Here, we pose the problem of relating these two types of coupling characteristics, and we solve it for a class of stochastic systems. Namely, for overdamped linear oscillators, we rigorously derive that the above long-term effect is proportional to the short-term effects, with the proportionality coefficient depending on the prediction interval and relaxation times. We reveal that this coefficient is typically considerably greater than unity so that small normalized PI values may well correspond to quite large long-term effects of coupling. The applicability of the derived relationship to wider classes of systems, its limitations, and its value for further research are discussed. To give a real-world example, we analyze couplings between large-scale climatic processes related to sea surface temperature variations in equatorial Pacific and North Atlantic regions.

EEG entropy measures indicate decrease of cortical information processing in Disorders of Consciousness

OBJECTIVE

Clinical assessments that rely on behavioral responses to differentiate Disorders of Consciousness are at times inapt because of some patients' motor disabilities. To objectify patients' conditions of reduced consciousness the present study evaluated the use of electroencephalography to measure residual brain activity.

METHODS

We analyzed entropy values of 18 scalp EEG channels of 15 severely brain-damaged patients with clinically diagnosed Minimally-Conscious-State (MCS) or Unresponsive-Wakefulness-Syndrome (UWS) and compared the results to a sample of 24 control subjects. Permutation entropy (PeEn) and symbolic transfer entropy (STEn), reflecting information processes in the EEG, were calculated for all subjects. Participants were tested on a modified active own-name paradigm to identify correlates of active instruction following.

RESULTS

PeEn showed reduced local information content in the EEG in patients, that was most pronounced in UWS. STEn analysis revealed altered directed information flow in the EEG of patients, indicating impaired feed-backward connectivity. Responses to auditory stimulation yielded differences in entropy measures, indicating reduced information processing in MCS and UWS.

CONCLUSIONS

Local EEG information content and information flow are affected in Disorders of Consciousness. This suggests local cortical information capacity and feedback information transfer as neural correlates of consciousness.

SIGNIFICANCE

The utilized EEG entropy analyses were able to relate to patient groups with different Disorders of Consciousness.

Understanding Networks of Computing Chemical Droplet Neurons Based on Information Flow

In this paper, we present general methods that can be used to explore the information processing potential of a medium composed of oscillating (self-exciting) droplets. Networks of Belousov–Zhabotinsky (BZ) droplets seem especially interesting as chemical reaction-diffusion computers because their time evolution is qualitatively similar to neural network activity. Moreover, such networks can be self-generated in microfluidic reactors. However, it is hard to track and to understand the function performed by a medium composed of droplets due to its complex dynamics. Corresponding to recurrent neural networks, the flow of excitations in a network of droplets is not limited to a single direction and spreads throughout the whole medium. In this work, we analyze the operation performed by droplet systems by monitoring the information flow. This is achieved by measuring mutual information and time delayed mutual information of the discretized time evolution of individual droplets. To link the model with reality, we use experimental results to estimate the parameters of droplet interactions. We exemplarily investigate an evolutionary generated droplet structure that operates as a NOR gate. The presented methods can be applied to networks composed of at least hundreds of droplets.