Symbolic transfer entropy

Dynamic effective connectivity of inter-areal brain circuits

Anatomic connections between brain areas affect information flow between neuronal circuits and the synchronization of neuronal activity. However, such structural connectivity does not coincide with effective connectivity (or, more precisely, causal connectivity), related to the elusive question “Which areas cause the present activity of which others?”. Effective connectivity is directed and depends flexibly on contexts and tasks. Here we show that dynamic effective connectivity can emerge from transitions in the collective organization of coherent neural activity. Integrating simulation and semi-analytic approaches, we study mesoscale network motifs of interacting cortical areas, modeled as large random networks of spiking neurons or as simple rate units. Through a causal analysis of time-series of model neural activity, we show that different dynamical states generated by a same structural connectivity motif correspond to distinct effective connectivity motifs. Such effective motifs can display a dominant directionality, due to spontaneous symmetry breaking and effective entrainment between local brain rhythms, although all connections in the considered structural motifs are reciprocal. We show then that transitions between effective connectivity configurations (like, for instance, reversal in the direction of inter-areal interactions) can be triggered reliably by brief perturbation inputs, properly timed with respect to an ongoing local oscillation, without the need for plastic synaptic changes. Finally, we analyze how the information encoded in spiking patterns of a local neuronal population is propagated across a fixed structural connectivity motif, demonstrating that changes in the active effective connectivity regulate both the efficiency and the directionality of information transfer. Previous studies stressed the role played by coherent oscillations in establishing efficient communication between distant areas. Going beyond these early proposals, we advance here that dynamic interactions between brain rhythms provide as well the basis for the self-organized control of this “communication-through-coherence”, making thus possible a fast “on-demand” reconfiguration of global information routing modalities.

The circuits of the brain must perform a daunting amount of functions. But how can “brain states” be flexibly controlled, given that anatomic inter-areal connections can be considered as fixed, on timescales relevant for behavior? We hypothesize that, thanks to the nonlinear interaction between brain rhythms, even a simple circuit involving few brain areas can originate a multitude of effective circuits, associated with alternative functions selectable “on demand”. A distinction is usually made between structural connectivity, which describes actual synaptic connections, and effective connectivity, quantifying, beyond correlation, directed inter-areal causal influences. In our study, we measure effective connectivity based on time-series of neural activity generated by model inter-areal circuits. We find that “causality follows dynamics”. We show indeed that different effective networks correspond to different dynamical states associated to a same structural network (in particular, different phase-locking patterns between local neuronal oscillations). We then find that “information follows causality” (and thus, again, dynamics). We demonstrate that different effective networks give rise to alternative modalities of information routing between brain areas wired together in a fixed structural network. In particular, we show that the self-organization of interacting “analog” rate oscillations control the flow of “digital-like” information encoded in complex spiking patterns.

Extending transfer entropy improves identification of effective connectivity in a spiking cortical network model

Transfer entropy (TE) is an information-theoretic measure which has received recent attention in neuroscience for its potential to identify effective connectivity between neurons. Calculating TE for large ensembles of spiking neurons is computationally intensive, and has caused most investigators to probe neural interactions at only a single time delay and at a message length of only a single time bin. This is problematic, as synaptic delays between cortical neurons, for example, range from one to tens of milliseconds. In addition, neurons produce bursts of spikes spanning multiple time bins. To address these issues, here we introduce a free software package that allows TE to be measured at multiple delays and message lengths. To assess performance, we applied these extensions of TE to a spiking cortical network model (Izhikevich, 2006) with known connectivity and a range of synaptic delays. For comparison, we also investigated single-delay TE, at a message length of one bin (D1TE), and cross-correlation (CC) methods. We found that D1TE could identify 36% of true connections when evaluated at a false positive rate of 1%. For extended versions of TE, this dramatically improved to 73% of true connections. In addition, the connections correctly identified by extended versions of TE accounted for 85% of the total synaptic weight in the network. Cross correlation methods generally performed more poorly than extended TE, but were useful when data length was short. A computational performance analysis demonstrated that the algorithm for extended TE, when used on currently available desktop computers, could extract effective connectivity from 1 hr recordings containing 200 neurons in ∼5 min. We conclude that extending TE to multiple delays and message lengths improves its ability to assess effective connectivity between spiking neurons. These extensions to TE soon could become practical tools for experimentalists who record hundreds of spiking neurons.

Effects of coarse-graining on the scaling behavior of long-range correlated and anti-correlated signals

We investigate how various coarse-graining (signal quantization) methods affect the scaling properties of long-range power-law correlated and anti-correlated signals, quantified by the detrended fluctuation analysis. Specifically, for coarse-graining in the magnitude of a signal, we consider (i) the Floor, (ii) the Symmetry and (iii) the Centro-Symmetry coarse-graining methods. We find that for anti-correlated signals coarse-graining in the magnitude leads to a crossover to random behavior at large scales, and that with increasing the width of the coarse-graining partition interval Δ, this crossover moves to intermediate and small scales. In contrast, the scaling of positively correlated signals is less affected by the coarse-graining, with no observable changes when Δ < 1, while for Δ > 1 a crossover appears at small scales and moves to intermediate and large scales with increasing Δ. For very rough coarse-graining (Δ > 3) based on the Floor and Symmetry methods, the position of the crossover stabilizes, in contrast to the Centro-Symmetry method where the crossover continuously moves across scales and leads to a random behavior at all scales; thus indicating a much stronger effect of the Centro-Symmetry compared to the Floor and the Symmetry method. For coarse-graining in time, where data points are averaged in non-overlapping time windows, we find that the scaling for both anti-correlated and positively correlated signals is practically preserved. The results of our simulations are useful for the correct interpretation of the correlation and scaling properties of symbolic sequences.

Assessing directed interactions from neurophysiological signals--an overview

The study of synchronization phenomena in coupled dynamical systems is an active field of research in many scientific disciplines including the neurosciences. Over the last decades, a number of time series analysis techniques have been proposed to capture both linear and nonlinear aspects of interactions. While most of these techniques allow one to quantify the strength of interactions, developments that resulted from advances in nonlinear dynamics and in information and synchronization theory aim at assessing directed interactions. Most of these techniques, however, assume the underlying systems to be at least approximately stationary and require a large number of data points to robustly assess directed interactions. Recent extensions allow assessing directed interactions from short and transient signals and are particularly suited for the analysis of evoked and event-related activity.

Preferential inhibition of frontal-to-parietal feedback connectivity is a neurophysiologic correlate of general anesthesia in surgical patients

Background

The precise mechanism and optimal measure of anesthetic-induced unconsciousness has yet to be elucidated. Preferential inhibition of feedback connectivity from frontal to parietal brain networks is one potential neurophysiologic correlate, but has only been demonstrated in animals or under limited conditions in healthy volunteers.

Methods and Findings

We recruited eighteen patients presenting for surgery under general anesthesia; electroencephalography of the frontal and parietal regions was acquired during (i) baseline consciousness, (ii) anesthetic induction with propofol or sevoflurane, (iii) general anesthesia, (iv) recovery of consciousness, and (v) post-recovery states. We used two measures of effective connectivity, evolutional map approach and symbolic transfer entropy, to analyze causal interactions of the frontal and parietal regions. The dominant feedback connectivity of the baseline conscious state was inhibited after anesthetic induction and during general anesthesia, resulting in reduced asymmetry of feedback and feedforward connections in the frontoparietal network. Dominant feedback connectivity returned when patients recovered from anesthesia. Both analytic techniques and both classes of anesthetics demonstrated similar results in this heterogeneous population of surgical patients.

Conclusions

The disruption of dominant feedback connectivity in the frontoparietal network is a common neurophysiologic correlate of general anesthesia across two anesthetic classes and two analytic measures. This study represents a key translational step from the underlying cognitive neuroscience of consciousness to more sophisticated monitoring of anesthetic effects in human surgical patients.

Distinguishing anticipation from causality: anticipatory bias in the estimation of information flow

We report that transfer entropy estimates obtained from low-resolution and/or small data sets show net information flow away from a purely anticipatory element whereas transfer entropy calculated using exact distributions show the flow towards it. This means that for real-world data sets anticipatory elements can appear to be strongly driving the network dynamics even when there is no possibility of such an influence. Furthermore, we show that in the low-resolution limit there is no statistic that can distinguish anticipatory elements from causal ones.

Identification of the direction of the neural network activation with a cellular resolution by fast two-photon imaging

Spatiotemporal activity patterns in local neural networks are fundamental to understanding how information is processed and stored in brain microcircuits. Currently, imaging techniques are able to map the directional activation of macronetworks across brain areas; however, these strategies still fail to resolve the activation direction for fine microcircuits with cellular spatial resolution. Here, we show the capability to identify the activation direction of a multicell network with a cellular resolution and millisecond precision by using fast two-photon microscopy and cross correlation procedures. As an example, we characterized a directional neuronal network in an epilepsy brain slice to provide different initiation delay among multiple neurons defined at a millisecond scale.

Causality, criticality, and reading words: distinct sources of fractal scaling in behavioral sequences

The finding of fractal scaling (FS) in behavioral sequences has raised a debate on whether FS is a pervasive property of the cognitive system or is the result of specific processes. Inferences about the origins of properties in time sequences are causal. That is, as opposed to correlational inferences reflecting instantaneous symmetrical relations, causal inferences concern asymmetric relations lagged in time. Here, I integrate Granger-causality with inferences about FS. Four simulations illustrate that causal analyses can isolate distinct FS sources, whereas correlational techniques cannot. I then analyze three simultaneous sequences of responses from a database of word-naming trials. I find that two, or perhaps three, distinct sources account for the presence of FS in these sequences, but FS is not a general property of the system. This suggests that FS arises due to the properties of a limited number of identifiable psychological and/or neural processes. Finally, I reanalyze a previously published dataset of acoustic frequency spectra using the new tools. The causality/criticality combination introduced here offers a new important perspective in the study of cognition.

Momentary information transfer as a coupling measure of time series

We propose a method to analyze couplings between two simultaneously measured time series. Our approach is based on conditional mutual sorting information. It is related to other concepts for detecting coupling directions: the old idea of Marko for directed information and the more recent concept of Schreiber's transfer entropy. By setting suitable conditions we first of all consider momentary information in both time series. This enables the detection not only of coupling directions but also delays. Sorting information refers to ordinal properties of time series, which makes the analysis robust with respect to strictly monotonous distortions and thus very useful in the analysis of proxy data in climatology. Fortunately, ordinal analysis is easy and fast to compute. We consider also the problem of reliable estimation from finite time series.

Using bivariate signal analysis to characterize the epileptic focus: the benefit of surrogates

The disease epilepsy is related to hypersynchronous activity of networks of neurons. While acute epileptic seizures are the most extreme manifestation of this hypersynchronous activity, an elevated level of interdependence of neuronal dynamics is thought to persist also during the seizure-free interval. In multichannel recordings from brain areas involved in the epileptic process, this interdependence can be reflected in an increased linear cross correlation but also in signal properties of higher order. Bivariate time series analysis comprises a variety of approaches, each with different degrees of sensitivity and specificity for interdependencies reflected in lower- or higher-order properties of pairs of simultaneously recorded signals. Here we investigate which approach is best suited to detect putatively elevated interdependence levels in signals recorded from brain areas involved in the epileptic process. For this purpose, we use the linear cross correlation that is sensitive to lower-order signatures of interdependence, a nonlinear interdependence measure that integrates both lower- and higher-order properties, and a surrogate-corrected nonlinear interdependence measure that aims to specifically characterize higher-order properties. We analyze intracranial electroencephalographic recordings of the seizure-free interval from 29 patients with an epileptic focus located in the medial temporal lobe. Our results show that all three approaches detect higher levels of interdependence for signals recorded from the brain hemisphere containing the epileptic focus as compared to signals recorded from the opposite hemisphere. For the linear cross correlation, however, these differences are not significant. For the nonlinear interdependence measure, results are significant but only of moderate accuracy with regard to the discriminative power for the focal and nonfocal hemispheres. The highest significance and accuracy is obtained for the surrogate-corrected nonlinear interdependence measure.

Reducing the bias of causality measures

Measures of the direction and strength of the interdependence between two time series are evaluated and modified to reduce the bias in the estimation of the measures, so that they give zero values when there is no causal effect. For this, point shuffling is employed as used in the frame of surrogate data. This correction is not specific to a particular measure and it is implemented here on measures based on state space reconstruction and information measures. The performance of the causality measures and their modifications is evaluated on simulated uncoupled and coupled dynamical systems and for different settings of embedding dimension, time series length, and noise level. The corrected measures, and particularly the suggested corrected transfer entropy, turn out to stabilize at the zero level in the absence of a causal effect and detect correctly the direction of information flow when it is present. The measures are also evaluated on electroencephalograms (EEG) for the detection of the information flow in the brain of an epileptic patient. The performance of the measures on EEG is interpreted in view of the results from the simulation study.

Inferring directional interactions from transient signals with symbolic transfer entropy

We extend the concept of symbolic transfer entropy to enable the time-resolved investigation of directional relationships between coupled dynamical systems from short and transient noisy time series. For our approach, we consider an observed ensemble of a sufficiently large number of time series as multiple realizations of a process. We derive an index that quantifies the preferred direction of transient interactions and assess its significance using a surrogate-based testing scheme. Analyzing time series from noisy chaotic systems, we demonstrate numerically the applicability and limitations of our approach. Our findings obtained from an analysis of event-related brain activities underline the importance of our method to improve understanding of gross neural interactions underlying cognitive processes.

Symbolic coupling traces for causality analysis of cardiovascular control

Directional coupling analysis of time series is an important subject of current research. In this paper, a method based on symbolic dynamics for the detection of time-delayed coupling in biosignals is presented. The symbolic coupling traces, defined as the symmetric and diametric traces of the bivariate word distribution, allow for a more reliable quantification of coupling and are compared with established methods like mutual information and cross recurrence analysis. The symbolic coupling traces method is applied to appropriate model systems and cardiological data which demonstrate its advantages especially for nonstationary and noisy data. Moreover, the method of symbolic coupling traces is used to analyze and quantify time-delayed coupling of cardiovascular measurements during different sleep stages. Significant different regulatory mechanisms are detected not only between the deep sleep and the other sleep stages but also between healthy subjects and patients. The proposed method may help to indicate pathological changes in cardiovascular regulation and also effects of continuous positive airway pressure therapy on the cardiovascular system.

Cardiovascular regulation during sleep quantified by symbolic coupling traces

Sleep is a complex regulated process with short periods of wakefulness and different sleep stages. These sleep stages modulate autonomous functions such as blood pressure and heart rate. The method of symbolic coupling traces (SCT) is used to analyze and quantify time-delayed coupling of these measurements during different sleep stages. The symbolic coupling traces, defined as the symmetric and diametric traces of the bivariate word distribution matrix, allow the quantification of time-delayed coupling. In this paper, the method is applied to heart rate and systolic blood pressure time series during different sleep stages for healthy controls as well as for normotensive and hypertensive patients with sleep apneas. Using the SCT, significant different cardiovascular mechanisms not only between the deep sleep and the other sleep stages but also between healthy subjects and patients can be revealed. The SCT method is applied to model systems, compared with established methods, such as cross correlation, mutual information, and cross recurrence analysis and demonstrates its advantages especially for nonstationary physiological data. As a result, SCT proves to be more specific in detecting delays of directional interactions than standard coupling analysis methods and yields additional information which cannot be measured by standard parameters of heart rate and blood pressure variability. The proposed method may help to indicate the pathological changes in cardiovascular regulation and also the effects of continuous positive airway pressure therapy on the cardiovascular system.

Permutation-information-theory approach to unveil delay dynamics from time-series analysis

In this paper an approach to identify delay phenomena from time series is developed. We show that it is possible to perform a reliable time delay identification by using quantifiers derived from information theory, more precisely, permutation entropy and permutation statistical complexity. These quantifiers show clear extrema when the embedding delay τ of the symbolic reconstruction matches the characteristic time delay τ(S) of the system. Numerical data originating from a time delay system based on the well-known Mackey-Glass equations operating in the chaotic regime were used as test beds. We show that our method is straightforward to apply and robust to additive observational and dynamical noise. Moreover, we find that the identification of the time delay is even more efficient in a noise environment. Our permutation approach is also able to recover the time delay in systems with low feedback rate or high nonlinearity.

Symbolic transfer entropy: inferring directionality in biosignals

Inferring directional interactions from biosignals is of crucial importance to improve understanding of dynamical interdependences underlying various physiological and pathophysiological conditions. We here present symbolic transfer entropy as a robust measure to infer the direction of interactions between multidimensional dynamical systems. We demonstrate its performance in quantifying driver-responder relationships in a network of coupled nonlinear oscillators and in the human epileptic brain.

Extracting the causality of correlated motions from molecular dynamics simulations

The information theory measure of transfer entropy is used to extract the causality of correlated motions from molecular dynamics simulations. For each pair of correlated residues, the method quantifies which residue drives the correlated motions, and which residue responds. The measure reveals how correlated motions are used to transmit information through the system, and helps to clarify the link between correlated motions and biological function in biomolecular systems. The method is illustrated by its application to the Ets-1 transcription factor, which partially unfolds upon binding DNA. The calculations show dramatic changes in the direction of information flow upon DNA binding, and elucidate how the presence of DNA is communicated from the DNA binding H1 and H3 helices to inhibitory helix HI-1. Helix H4 is shown to act as a relay, which is attenuated in the apo state.

From Granger causality to long-term causality: application to climatic data

Quantitative characterization of interaction between processes from time series is often required in different fields of natural science including geophysics and biophysics. Typically, one estimates "short-term" influences, e.g., the widely used Granger causality is defined via one-step-ahead predictions. Such an approach does not reveal how strongly the "long-term" behavior of one process under study is affected by the others. To overcome this problem, we introduce the concept of long-term causality, which extends the concept of Granger causality. The long-term causality is estimated from data via empirical modeling and analysis of model dynamics under different conditions. Apart from mathematical examples, we apply both approaches to find out how strongly the global surface temperature (GST) is affected by variations in carbon dioxide atmospheric content, solar activity, and volcanic activity during the last 150 years. Influences of all the three factors on GST are detected with the Granger causality. However, the long-term causality shows that the rise in GST during the last decades can be explained only if the anthropogenic factor (CO2) is taken into account in a model.

Sleep-dependent directional coupling between human neocortex and hippocampus

Complex interactions between neocortex and hippocampus are the neural basis of memory formation. Two-step theories of memory formation suggest that initial encoding of novel information depends on the induction of rapid plasticity within the hippocampus, and is followed by a second sleep-dependent step of memory consolidation. These theories predict information flow from the neocortex into the hippocampus during waking state and in the reverse direction during sleep. However, experimental evidence that interactions between hippocampus and neocortex have a predominant direction which reverses during sleep rely on cross-correlation analysis of data from animal experiments and yielded inconsistent results. Here, we investigated directional coupling in intracranial EEG data from human subjects using a phase-modeling approach which is well suited to reveal functional interdependencies in oscillatory data. In general, we observed that the anterior hippocampus predominantly drives nearby and remote brain regions. Surprisingly, however, the influence of neocortical regions on the hippocampus significantly increased during sleep as compared to waking state. These results question the standard model of hippocampal-neocortical interactions and suggest that sleep-dependent consolidation is accomplished by an active retrieval of hippocampal information by the neocortex.

Epilepsy and nonlinear dynamics

This overview summarizes findings obtained from analyzing electroencephalographic (EEG) recordings from epilepsy patients with methods from the theory of nonlinear dynamical systems. The last two decades have shown that nonlinear time series analysis techniques allow an improved characterization of epileptic brain states and help to gain deeper insights into the spatial and temporal dynamics of the epileptic process. Nonlinear EEG analyses can help to improve the evaluation of patients prior to neurosurgery, and with an unequivocal identification of precursors of seizures, they can be of great value in the development of seizure warning and prevention techniques.