Directionality of coupling from bivariate time series: how to avoid false causalities and missed connections

Detecting directional couplings from multivariate flows by the joint distance distribution

The identification of directional couplings (or drive-response relationships) in the analysis of interacting nonlinear systems is an important piece of information to understand their dynamics. This task is especially challenging when the analyst's knowledge of the systems reduces virtually to time series of observations. Spurred by the success of Granger causality in econometrics, the study of cause-effect relationships (not to be confounded with statistical correlations) was extended to other fields, thus favoring the introduction of further tools such as transfer entropy. Currently, the research on old and new causality tools along with their pitfalls and applications in ever more general situations is going through a time of much activity. In this paper, we re-examine the method of the joint distance distribution to detect directional couplings between two multivariate flows. This method is based on the forced Takens theorem, and, more specifically, it exploits the existence of a continuous mapping from the reconstructed attractor of the response system to the reconstructed attractor of the driving system, an approach that is increasingly drawing the attention of the data analysts. The numerical results with Lorenz and Rössler oscillators in three different interaction networks (including hidden common drivers) are quite satisfactory, except when phase synchronization sets in. They also show that the method of the joint distance distribution outperforms the lowest dimensional transfer entropy in the cases considered. The robustness of the results to the sampling interval, time series length, observational noise, and metric is analyzed too.

Causality, dynamical systems and the arrow of time

Using several methods for detection of causality in time series, we show in a numerical study that coupled chaotic dynamical systems violate the first principle of Granger causality that the cause precedes the effect. While such a violation can be observed in formal applications of time series analysis methods, it cannot occur in nature, due to the relation between entropy production and temporal irreversibility. The obtained knowledge, however, can help to understand the type of causal relations observed in experimental data, namely, it can help to distinguish linear transfer of time-delayed signals from nonlinear interactions. We illustrate these findings in causality detected in experimental time series from the climate system and mammalian cardio-respiratory interactions.

Comparison of six methods for the detection of causality in a bivariate time series

In this comparative study, six causality detection methods were compared, namely, the Granger vector autoregressive test, the extended Granger test, the kernel version of the Granger test, the conditional mutual information (transfer entropy), the evaluation of cross mappings between state spaces, and an assessment of predictability improvement due to the use of mixed predictions. Seven test data sets were analyzed: linear coupling of autoregressive models, a unidirectional connection of two Hénon systems, a unidirectional connection of chaotic systems of Rössler and Lorenz type and of two different Rössler systems, an example of bidirectionally connected two-species systems, a fishery model as an example of two correlated observables without a causal relationship, and an example of mediated causality. We tested not only 20000 points long clean time series but also noisy and short variants of the data. The standard and the extended Granger tests worked only for the autoregressive models. The remaining methods were more successful with the more complex test examples, although they differed considerably in their capability to reveal the presence and the direction of coupling and to distinguish causality from mere correlation.

Model-free inference of direct network interactions from nonlinear collective dynamics

The topology of interactions in network dynamical systems fundamentally underlies their function. Accelerating technological progress creates massively available data about collective nonlinear dynamics in physical, biological, and technological systems. Detecting direct interaction patterns from those dynamics still constitutes a major open problem. In particular, current nonlinear dynamics approaches mostly require to know a priori a model of the (often high dimensional) system dynamics. Here we develop a model-independent framework for inferring direct interactions solely from recording the nonlinear collective dynamics generated. Introducing an explicit dependency matrix in combination with a block-orthogonal regression algorithm, the approach works reliably across many dynamical regimes, including transient dynamics toward steady states, periodic and non-periodic dynamics, and chaos. Together with its capabilities to reveal network (two point) as well as hypernetwork (e.g., three point) interactions, this framework may thus open up nonlinear dynamics options of inferring direct interaction patterns across systems where no model is known.

Network dynamical systems can represent the interactions involved in the collective dynamics of gene regulatory networks or metabolic circuits. Here Casadiego et al. present a method for inferring these types of interactions directly from observed time series without relying on their model.

Directionality indices: Testing information transfer with surrogate correction

Directionality indices can be used as an indicator of the asymmetry in coupling between systems and have found particular application in relation to neurological systems. The directionality index between two systems is a function of measures of information transfer in both directions. Here we illustrate that before inferring the directionality of coupling it is first necessary to consider the use of appropriate tests of significance. We propose a surrogate corrected directionality index which incorporates such testing. We also highlight the differences between testing the significance of the directionality index itself versus testing the individual measures of information transfer in each direction. To validate the approach we compared two different methods of estimating coupling, both of which have previously been used to estimate directionality indices. These were the modeling-based evolution map approach and a conditional mutual information (CMI) method for calculating dynamic information rates. For the CMI-based approach we also compared two different methods for estimating the CMI, an equiquantization-based estimator and a k-nearest neighbors estimator.

Functional and effective whole brain connectivity using magnetoencephalography to identify monozygotic twin pairs

Resting-state functional connectivity patterns are highly stable over time within subjects. This suggests that such 'functional fingerprints' may have strong genetic component. We investigated whether the functional (FC) or effective (EC) connectivity patterns of one monozygotic twin could be used to identify the co-twin among a larger sample and determined the overlap in functional fingerprints within monozygotic (MZ) twin pairs using resting state magnetoencephalography (MEG). We included 32 cognitively normal MZ twin pairs from the Netherlands Twin Register who participate in the EMIF-AD preclinAD study (average age 68 years). Combining EC information across multiple frequency bands we obtained an identification rate over 75%. Since MZ twin pairs are genetically identical these results suggest a high genetic contribution to MEG-based EC patterns, leading to large similarities in brain connectivity patterns between two individuals even after 60 years of life or more.

Detection of coupling delay: A problem not yet solved

Nonparametric detection of coupling delay in unidirectionally and bidirectionally coupled nonlinear dynamical systems is examined. Both continuous and discrete-time systems are considered. Two methods of detection are assessed-the method based on conditional mutual information-the CMI method (also known as the transfer entropy method) and the method of convergent cross mapping-the CCM method. Computer simulations show that neither method is generally reliable in the detection of coupling delays. For continuous-time chaotic systems, the CMI method appears to be more sensitive and applicable in a broader range of coupling parameters than the CCM method. In the case of tested discrete-time dynamical systems, the CCM method has been found to be more sensitive, while the CMI method required much stronger coupling strength in order to bring correct results. However, when studied systems contain a strong oscillatory component in their dynamics, results of both methods become ambiguous. The presented study suggests that results of the tested algorithms should be interpreted with utmost care and the nonparametric detection of coupling delay, in general, is a problem not yet solved.

Robustness and versatility of a nonlinear interdependence method for directional coupling detection from spike trains

The detection of directional couplings between dynamics based on measured spike trains is a crucial problem in the understanding of many different systems. In particular, in neuroscience it is important to assess the connectivity between neurons. One of the approaches that can estimate directional coupling from the analysis of point processes is the nonlinear interdependence measure L. Although its efficacy has already been demonstrated, it still needs to be tested under more challenging and realistic conditions prior to an application to real data. Thus, in this paper we use the Hindmarsh-Rose model system to test the method in the presence of noise and for different spiking regimes. We also examine the influence of different parameters and spike train distances. Our results show that the measure L is versatile and robust to various types of noise, and thus suitable for application to experimental data.

Directional information flow in patients with Alzheimer's disease. A source-space resting-state MEG study

In a recent magnetoencephalography (MEG) study, we found posterior-to-anterior information flow over the cortex in higher frequency bands in healthy subjects, with a reversed pattern in the theta band. A disruption of information flow may underlie clinical symptoms in Alzheimer's disease (AD). In AD, highly connected regions (hubs) in posterior areas are mostly disrupted. We therefore hypothesized that in AD the information flow from these hub regions would be disturbed. We used resting-state MEG recordings from 27 early-onset AD patients and 26 healthy controls. Using beamformer-based virtual electrodes, we estimated neuronal oscillatory activity for 78 cortical regions of interest (ROIs) and 12 subcortical ROIs of the AAL atlas, and calculated the directed phase transfer entropy (dPTE) as a measure of information flow between these ROIs. Group differences were evaluated using permutation tests and, for the AD group, associations between dPTE and general cognition or CSF biomarkers were determined using Spearman correlation coefficients. We confirmed the previously reported posterior-to-anterior information flow in the higher frequency bands in the healthy controls, and found it to be disturbed in the beta band in AD. Most prominently, the information flow from the precuneus and the visual cortex, towards frontal and subcortical structures, was decreased in AD. These disruptions did not correlate with cognitive impairment or CSF biomarkers. We conclude that AD pathology may affect the flow of information between brain regions, particularly from posterior hub regions, and that changes in the information flow in the beta band indicate an aspect of the pathophysiological process in AD.

Horizontal visibility graph transfer entropy (HVG-TE): A novel metric to characterize directed connectivity in large-scale brain networks

We propose a new measure, horizontal visibility graph transfer entropy (HVG-TE), to estimate the direction of information flow between pairs of time series. HVG-TE quantifies the transfer entropy between the degree sequences of horizontal visibility graphs derived from original time series. Twenty-one Rössler attractors unidirectionally coupled in the posterior-to-anterior direction were used to simulate 21-channel Electroencephalography (EEG) brain networks and validate the performance of the HVG-TE. We showed that the HVG-TE is robust to different levels of coupling strengths between the coupled Rössler attractors, a wide range of time delays, different sample sizes, the effects of noise and linear mixing, and the choice of reference for EEG data. We also applied HVG-TE to EEG data in 20 healthy controls and compared its performance to a recently introduces phase-based TE measure (PTE). We found that compared with PTE, HVG-TE consistently detected stronger posterior-to-anterior information flow patterns in the alpha-band (8-13Hz) EEG brain networks for three different references. Moreover, in contrast to PTE, HVG-TE does not require an assumption on the periodicity of input signals, therefore it can be more widely applicable, even for non-periodic signals. This study shows that the HVG-TE is a directed connectivity measure to characterise the direction of information flow in large-scale brain networks.

Changes in resting-state directed connectivity in cortico-subcortical networks correlate with cognitive function in Parkinson's disease

OBJECTIVE

The pathophysiological mechanisms underlying Parkinson's disease (PD)-related cognitive decline and conversion to PD dementia are poorly understood. In the healthy human brain, stable patterns of posterior-to-anterior cortical information flow have recently been demonstrated in the higher frequency bands using magnetoencephalography (MEG). In this study we estimated PD-related changes in information flow patterns, as well as the contribution of subcortical regions.

METHODS

Resting-state MEG recordings were acquired in moderately advanced PD patients (n=34; mean Hoehn and Yahr-stage 2.5) and healthy controls (n=12). MEG signals were projected to both cortical and subcortical brain regions, following which we estimated the balance between incoming and outgoing information flow per region.

RESULTS

In PD patients, compared to controls, preferential beta band information outflow was significantly higher for the basal ganglia and frontotemporal cortical regions, and significantly lower for parieto-occipital regions. In addition, in patients, low preferential information outflow from occipital regions correlated with poor global cognitive performance.

CONCLUSION

In the PD brain, a shift in balance towards more anterior-to-posterior beta band information flow takes place and is associated with poorer cognitive performance.

SIGNIFICANCE

Our results indicate that a reversal of the physiological posterior-to-anterior information flow may be an important mechanism in PD-related cognitive decline.

Functional connectivity and network analysis during hypoactive delirium and recovery from anesthesia

OBJECTIVE

To gain insight in the underlying mechanism of reduced levels of consciousness due to hypoactive delirium versus recovery from anesthesia, we studied functional connectivity and network topology using electroencephalography (EEG).

METHODS

EEG recordings were performed in age and sex-matched patients with hypoactive delirium (n=18), patients recovering from anesthesia (n=20), and non-delirious control patients (n=20), all after cardiac surgery. Functional and directed connectivity were studied with phase lag index and directed phase transfer entropy. Network topology was characterized using the minimum spanning tree (MST). A random forest classifier was calculated based on all measures to obtain discriminative ability between the three groups.

RESULTS

Non-delirious control subjects showed a back-to-front information flow, which was lost during hypoactive delirium (p=0.01) and recovery from anesthesia (p<0.01). The recovery from anesthesia group had more integrated network in the delta band compared to non-delirious controls. In contrast, hypoactive delirium showed a less integrated network in the alpha band. High accuracy for discrimination between hypoactive delirious patients and controls (86%) and recovery from anesthesia and controls (95%) were found. Accuracy for discrimination between hypoactive delirium and recovery from anesthesia was 73%.

CONCLUSION

Loss of functional and directed connectivity were observed in both hypoactive delirium and recovery from anesthesia, which might be related to the reduced level of consciousness in both states. These states could be distinguished in topology, which was a less integrated network during hypoactive delirium.

SIGNIFICANCE

Functional and directed connectivity are similarly disturbed during a reduced level of consciousness due to hypoactive delirium and sedatives, however topology was differently affected.

Smooth information flow in temperature climate network reflects mass transport

A directed climate network is constructed by Granger causality analysis of air temperature time series from a regular grid covering the whole Earth. Using winner-takes-all network thresholding approach, a structure of a smooth information flow is revealed, hidden to previous studies. The relevance of this observation is confirmed by comparison with the air mass transfer defined by the wind field. Their close relation illustrates that although the information transferred due to the causal influence is not a physical quantity, the information transfer is tied to the transfer of mass and energy.

Coupling strength versus coupling impact in nonidentical bidirectionally coupled dynamics

The understanding of interacting dynamics is important for the characterization of real-world networks. In general, real-world networks are heterogeneous in the sense that each node of the network is a dynamics with different properties. For coupled nonidentical dynamics symmetric interactions are not straightforwardly defined from the coupling strength values. Thus, a challenging issue is whether we can define a symmetric interaction in this asymmetric setting. To address this problem we introduce the notion of the coupling impact. The coupling impact considers not only the coupling strength but also the energy of the individual dynamics, which is conveyed via the coupling. To illustrate this concept, we follow a data-driven approach by analyzing signals from pairs of coupled model dynamics using two different connectivity measures. We find that the coupling impact, but not the coupling strength, correctly detects a symmetric interaction between pairs of coupled dynamics regardless of their degree of asymmetry. Therefore, this approach allows us to reveal the real impact that one dynamics has on the other and hence to define symmetric interactions in pairs of nonidentical dynamics.

Testing for causality in reconstructed state spaces by an optimized mixed prediction method

In this study, a method of causality detection was designed to reveal coupling between dynamical systems represented by time series. The method is based on the predictions in reconstructed state spaces. The results of the proposed method were compared with outcomes of two other methods, the Granger VAR test of causality and the convergent cross-mapping. We used two types of test data. The first test example is a unidirectional connection of chaotic systems of Rössler and Lorenz type. The second one, the fishery model, is an example of two correlated observables without a causal relationship. The results showed that the proposed method of optimized mixed prediction was able to reveal the presence and the direction of coupling and distinguish causality from mere correlation as well.

Computing algebraic transfer entropy and coupling directions via transcripts

Most random processes studied in nonlinear time series analysis take values on sets endowed with a group structure, e.g., the real and rational numbers, and the integers. This fact allows to associate with each pair of group elements a third element, called their transcript, which is defined as the product of the second element in the pair times the first one. The transfer entropy of two such processes is called algebraic transfer entropy. It measures the information transferred between two coupled processes whose values belong to a group. In this paper, we show that, subject to one constraint, the algebraic transfer entropy matches the (in general, conditional) mutual information of certain transcripts with one variable less. This property has interesting practical applications, especially to the analysis of short time series. We also derive weak conditions for the 3-dimensional algebraic transfer entropy to yield the same coupling direction as the corresponding mutual information of transcripts. A related issue concerns the use of mutual information of transcripts to determine coupling directions in cases where the conditions just mentioned are not fulfilled. We checked the latter possibility in the lowest dimensional case with numerical simulations and cardiovascular data, and obtained positive results.

Weighted and directed interactions in evolving large-scale epileptic brain networks

Epilepsy can be regarded as a network phenomenon with functionally and/or structurally aberrant connections in the brain. Over the past years, concepts and methods from network theory substantially contributed to improve the characterization of structure and function of these epileptic networks and thus to advance understanding of the dynamical disease epilepsy. We extend this promising line of research and assess-with high spatial and temporal resolution and using complementary analysis approaches that capture different characteristics of the complex dynamics-both strength and direction of interactions in evolving large-scale epileptic brain networks of 35 patients that suffered from drug-resistant focal seizures with different anatomical onset locations. Despite this heterogeneity, we find that even during the seizure-free interval the seizure onset zone is a brain region that, when averaged over time, exerts strongest directed influences over other brain regions being part of a large-scale network. This crucial role, however, manifested by averaging on the population-sample level only - in more than one third of patients, strongest directed interactions can be observed between brain regions far off the seizure onset zone. This may guide new developments for individualized diagnosis, treatment and control.

Distinguishing between direct and indirect directional couplings in large oscillator networks: Partial or non-partial phase analyses?

We investigate the relative merit of phase-based methods for inferring directional couplings in complex networks of weakly interacting dynamical systems from multivariate time-series data. We compare the evolution map approach and its partialized extension to each other with respect to their ability to correctly infer the network topology in the presence of indirect directional couplings for various simulated experimental situations using coupled model systems. In addition, we investigate whether the partialized approach allows for additional or complementary indications of directional interactions in evolving epileptic brain networks using intracranial electroencephalographic recordings from an epilepsy patient. For such networks, both direct and indirect directional couplings can be expected, given the brain's connection structure and effects that may arise from limitations inherent to the recording technique. Our findings indicate that particularly in larger networks (number of nodes ≫10), the partialized approach does not provide information about directional couplings extending the information gained with the evolution map approach.

Discrimination of coupling structures using causality networks from multivariate time series

Measures of Granger causality on multivariate time series have been used to form the so-called causality networks. A causality network represents the interdependence structure of the underlying dynamical system or coupled dynamical systems, and its properties are quantified by network indices. In this work, it is investigated whether network indices on networks generated by an appropriate Granger causality measure can discriminate different coupling structures. The information based Granger causality measure of partial mutual information from mixed embedding (PMIME) is used to form causality networks, and a large number of network indices are ranked according to their ability to discriminate the different coupling structures. The evaluation of the network indices is done with a simulation study based on two dynamical systems, the coupled Mackey-Glass delay differential equations and the neural mass model, both of 25 variables, and three prototypes of coupling structures, i.e., random, small-world, and scale-free. It is concluded that the setting of PMIME combined with a network index attains high level of discrimination of the coupling structures solely on the basis of the observed multivariate time series. This approach is demonstrated to identify epileptic seizures emerging during electroencephalogram recordings.

Detecting Causality by Combined Use of Multiple Methods: Climate and Brain Examples

Identifying causal relations from time series is the first step to understanding the behavior of complex systems. Although many methods have been proposed, few papers have applied multiple methods together to detect causal relations based on time series generated from coupled nonlinear systems with some unobserved parts. Here we propose the combined use of three methods and a majority vote to infer causality under such circumstances. Two of these methods are proposed here for the first time, and all of the three methods can be applied even if the underlying dynamics is nonlinear and there are hidden common causes. We test our methods with coupled logistic maps, coupled Rössler models, and coupled Lorenz models. In addition, we show from ice core data how the causal relations among the temperature, the CH4 level, and the CO2 level in the atmosphere changed in the last 800,000 years, a conclusion also supported by irregularly sampled data analysis. Moreover, these methods show how three regions of the brain interact with each other during the visually cued, two-choice arm reaching task. Especially, we demonstrate that this is due to bottom up influences at the beginning of the task, while there exist mutual influences between the posterior medial prefrontal cortex and the presupplementary motor area. Based on our results, we conclude that identifying causality with an appropriate ensemble of multiple methods ensures the validity of the obtained results more firmly.

A Nonlinear Causality Estimator Based on Non-Parametric Multiplicative Regression

Causal prediction has become a popular tool for neuroscience applications, as it allows the study of relationships between different brain areas during rest, cognitive tasks or brain disorders. We propose a nonparametric approach for the estimation of nonlinear causal prediction for multivariate time series. In the proposed estimator, C NPMR , Autoregressive modeling is replaced by Nonparametric Multiplicative Regression (NPMR). NPMR quantifies interactions between a response variable (effect) and a set of predictor variables (cause); here, we modified NPMR for model prediction. We also demonstrate how a particular measure, the sensitivity Q, could be used to reveal the structure of the underlying causal relationships. We apply C NPMR on artificial data with known ground truth (5 datasets), as well as physiological data (2 datasets). C NPMR correctly identifies both linear and nonlinear causal connections that are present in the artificial data, as well as physiologically relevant connectivity in the real data, and does not seem to be affected by filtering. The Sensitivity measure also provides useful information about the latent connectivity.The proposed estimator addresses many of the limitations of linear Granger causality and other nonlinear causality estimators. C NPMR is compared with pairwise and conditional Granger causality (linear) and Kernel-Granger causality (nonlinear). The proposed estimator can be applied to pairwise or multivariate estimations without any modifications to the main method. Its nonpametric nature, its ability to capture nonlinear relationships and its robustness to filtering make it appealing for a number of applications.

Relating Granger causality to long-term causal effects

In estimation of causal couplings between observed processes, it is important to characterize coupling roles at various time scales. The widely used Granger causality reflects short-term effects: it shows how strongly perturbations of a current state of one process affect near future states of another process, and it quantifies that via prediction improvement (PI) in autoregressive models. However, it is often more important to evaluate the effects of coupling on long-term statistics, e.g., to find out how strongly the presence of coupling changes the variance of a driven process as compared to an uncoupled case. No general relationships between Granger causality and such long-term effects are known. Here, we pose the problem of relating these two types of coupling characteristics, and we solve it for a class of stochastic systems. Namely, for overdamped linear oscillators, we rigorously derive that the above long-term effect is proportional to the short-term effects, with the proportionality coefficient depending on the prediction interval and relaxation times. We reveal that this coefficient is typically considerably greater than unity so that small normalized PI values may well correspond to quite large long-term effects of coupling. The applicability of the derived relationship to wider classes of systems, its limitations, and its value for further research are discussed. To give a real-world example, we analyze couplings between large-scale climatic processes related to sea surface temperature variations in equatorial Pacific and North Atlantic regions.

Neural networks with non-uniform embedding and explicit validation phase to assess Granger causality

A challenging problem when studying a dynamical system is to find the interdependencies among its individual components. Several algorithms have been proposed to detect directed dynamical influences between time series. Two of the most used approaches are a model-free one (transfer entropy) and a model-based one (Granger causality). Several pitfalls are related to the presence or absence of assumptions in modeling the relevant features of the data. We tried to overcome those pitfalls using a neural network approach in which a model is built without any a priori assumptions. In this sense this method can be seen as a bridge between model-free and model-based approaches. The experiments performed will show that the method presented in this work can detect the correct dynamical information flows occurring in a system of time series. Additionally we adopt a non-uniform embedding framework according to which only the past states that actually help the prediction are entered into the model, improving the prediction and avoiding the risk of overfitting. This method also leads to a further improvement with respect to traditional Granger causality approaches when redundant variables (i.e. variables sharing the same information about the future of the system) are involved. Neural networks are also able to recognize dynamics in data sets completely different from the ones used during the training phase.

Quantifying the causal strength of multivariate cardiovascular couplings with momentary information transfer

This article studies a recently introduced information-theoretic approach to detect and quantify the causal couplings in a complex cardiovascular system. In the first step a causal algorithm detects the coupling delays and in the second step the causal strength of each coupling mechanism is quantified using the recently introduced momentary information transfer. As an example, the method is applied to time series of respiration, systolic and diastolic blood pressure, and heart rate of pregnant healthy women and women suffering from pre-eclampsia. A possible explanation for the influence of heart rate on systolic blood pressure is found and some differences between healthy women and patients are discussed.

Estimating the decomposition of predictive information in multivariate systems

In the study of complex systems from observed multivariate time series, insight into the evolution of one system may be under investigation, which can be explained by the information storage of the system and the information transfer from other interacting systems. We present a framework for the model-free estimation of information storage and information transfer computed as the terms composing the predictive information about the target of a multivariate dynamical process. The approach tackles the curse of dimensionality employing a nonuniform embedding scheme that selects progressively, among the past components of the multivariate process, only those that contribute most, in terms of conditional mutual information, to the present target process. Moreover, it computes all information-theoretic quantities using a nearest-neighbor technique designed to compensate the bias due to the different dimensionality of individual entropy terms. The resulting estimators of prediction entropy, storage entropy, transfer entropy, and partial transfer entropy are tested on simulations of coupled linear stochastic and nonlinear deterministic dynamic processes, demonstrating the superiority of the proposed approach over the traditional estimators based on uniform embedding. The framework is then applied to multivariate physiologic time series, resulting in physiologically well-interpretable information decompositions of cardiovascular and cardiorespiratory interactions during head-up tilt and of joint brain-heart dynamics during sleep.

Assessing directionality and strength of coupling through symbolic analysis: an application to epilepsy patients

Inferring strength and direction of interactions from electroencephalographic (EEG) recordings is of crucial importance to improve our understanding of dynamical interdependencies underlying various physiological and pathophysiological conditions in the human epileptic brain. We here use approaches from symbolic analysis to investigate--in a time-resolved manner--weighted and directed, short- to long-ranged interactions between various brain regions constituting the epileptic network. Our observations point to complex spatial-temporal interdependencies underlying the epileptic process and their role in the generation of epileptic seizures, despite the massive reduction of the complex information content of multi-day, multi-channel EEG recordings through symbolization. We discuss limitations and potential future improvements of this approach.

Quantification of causal couplings via dynamical effects: a unifying perspective

Quantitative characterization of causal couplings from time series is crucial in studies of complex systems of different origin. Various statistical tools for that exist and new ones are still being developed with a tendency to creating a single, universal, model-free quantifier of coupling strength. However, a clear and generally applicable way of interpreting such universal characteristics is lacking. This work suggests a general conceptual framework for causal coupling quantification, which is based on state space models and extends the concepts of virtual interventions and dynamical causal effects. Namely, two basic kinds of interventions (state space and parametric) and effects (orbital or transient and stationary or limit) are introduced, giving four families of coupling characteristics. The framework provides a unifying view of apparently different well-established measures and allows us to introduce new characteristics, always with a definite "intervention-effect" interpretation. It is shown that diverse characteristics cannot be reduced to any single coupling strength quantifier and their interpretation is inevitably model based. The proposed set of dynamical causal effect measures quantifies different aspects of "how the coupling manifests itself in the dynamics," reformulating the very question about the "causal coupling strength."

Mathematical modeling for evolution of heterogeneous modules in the brain

Modular architecture has been found in most cortical areas of mammalian brains, but little is known about its evolutionary origin. It has been proposed by several researchers that maximizing information transmission among subsystems can be used as a principle for understanding the development of complex brain networks. In this paper, we study how heterogeneous modules develop in coupled-map networks via a genetic algorithm, where selection is based on maximizing bidirectional information transmission. Two functionally differentiated modules evolved from two homogeneous systems with random couplings, which are associated with symmetry breaking of intrasystem and intersystem couplings. By exploring the parameter space of the network around the optimal parameter values, it was found that the optimum network exists near transition points, at which the incoherent state loses its stability and an extremely slow oscillatory motion emerges.

Inverse approach to chronotaxic systems for single-variable time series

Following the development of a new class of self-sustained oscillators with a time-varying but stable frequency, the inverse approach to these systems is now formulated. We show how observed data arranged in a single-variable time series can be used to recognize such systems. This approach makes use of time-frequency domain information using the wavelet transform as well as the recently developed method of Bayesian-based inference. In addition, a set of methods, named phase fluctuation analysis, is introduced to detect the defining properties of the new class of systems by directly analyzing the statistics of the observed perturbations.We apply these methods to numerical examples but also elaborate further on the cardiac system.

Multiscale atmospheric dynamics: cross-frequency phase-amplitude coupling in the air temperature

Interactions between dynamics on different temporal scales of about a century long record of data of the daily mean surface air temperature from various European locations have been detected using a form of the conditional mutual information, statistically tested using the Fourier-transform and multifractal surrogate data methods. An information transfer from larger to smaller time scales has been observed as the influence of the phase of slow oscillatory phenomena with the periods around 6-11 yr on the amplitudes of the variability characterized by the smaller temporal scales from a few months to 4-5 yr. The overall effect of the slow oscillations on the interannual temperature variability within the range 1-2 ° C has been observed in large areas of Europe.

Limits to detection of generalized synchronization in delay-coupled chaotic oscillators

We study how reliably generalized synchronization can be detected and characterized from time-series analysis. To that end, we analyze synchronization in a generalized sense of delay-coupled chaotic oscillators in unidirectional ring configurations. The generalized synchronization condition can be verified via the auxiliary system approach; however, in practice, this might not always be possible. Therefore, in this study, widely used indicators to directly quantify generalized and phase synchronization from noise-free time series of two oscillators are employed complementarily to the auxiliary system approach. In our analysis, none of the indices provide the consistent results of the auxiliary system approach. Our findings indicate that it is a major challenge to directly detect synchronization in a generalized sense between two oscillators that are connected via a chain of other oscillators, even if the oscillators are identical. This has major consequences for the interpretation of the dynamics of coupled systems and applications thereof.

Estimating coupling directions in the cardiorespiratory system using recurrence properties

The asymmetry of coupling between complex systems can be studied by conditional probabilities of recurrence, which can be estimated by joint recurrence plots. This approach is applied for the first time on experimental data: time series of the human cardiorespiratory system in order to investigate the couplings between heart rate, mean arterial blood pressure and respiration. We find that the respiratory system couples towards the heart rate, and the heart rate towards the mean arterial blood pressure. However, our analysis could not detect a clear coupling direction between the mean arterial blood pressure and respiration.

Respiration and heart rate complexity: effects of age and gender assessed by band-limited transfer entropy

Aging and disease are accompanied with a reduction of complex variability in the temporal patterns of heart rate. This reduction has been attributed to a break down of the underlying regulatory feedback mechanisms that maintain a homeodynamic state. Previous work has established the utility of entropy as an index of disorder, for quantification of changes in heart rate complexity. However, questions remain regarding the origin of heart rate complexity and the mechanisms involved in its reduction with aging and disease. In this work we use a newly developed technique based on the concept of band-limited transfer entropy to assess the aging-related changes in contribution of respiration and blood pressure to entropy of heart rate at different frequency bands. Noninvasive measurements of heart beat interval, respiration, and systolic blood pressure were recorded from 20 young (21-34 years) and 20 older (68-85 years) healthy adults. Band-limited transfer entropy analysis revealed a reduction in high-frequency contribution of respiration to heart rate complexity (p<0.001) with normal aging, particularly in men. These results have the potential for dissecting the relative contributions of respiration and blood pressure-related reflexes to heart rate complexity and their degeneration with normal aging.

Information flow in heterogeneously interacting systems

Motivated by studies on the dynamics of heterogeneously interacting systems in neocortical neural networks, we studied heterogeneously-coupled chaotic systems. We used information-theoretic measures to investigate directions of information flow in heterogeneously coupled Rössler systems, which we selected as a typical chaotic system. In bi-directionally coupled systems, spontaneous and irregular switchings of the phase difference between two chaotic oscillators were observed. The direction of information transmission spontaneously switched in an intermittent manner, depending on the phase difference between the two systems. When two further oscillatory inputs are added to the coupled systems, this system dynamically selects one of the two inputs by synchronizing, selection depending on the internal phase differences between the two systems. These results indicate that the effective direction of information transmission dynamically changes, induced by a switching of phase differences between the two systems.

Spurious causalities with transfer entropy

Transfer entropy (TE) seems currently to be the most widely used tool to characterize causal influences in ensembles of complex systems from observed time series. In particular, in an elemental case of two systems, nonzero TEs in both directions are usually interpreted as a sign of a bidirectional coupling. However, one often overlooks that both positive TEs may well be encountered for unidirectionally coupled systems so that a false detection of a causal influence on the basis of a nonzero TE is rather possible. This work highlights typical factors leading to such "spurious couplings": (i) unobserved state variables of the driving system, (ii) low temporal resolution, and (iii) observation errors. All are shown to be particular cases of a general problem: imperfect observations of states of the driving system. Importantly, exact values of TEs, rather than their statistical estimates, are computed here for selected benchmark systems. Conditions for a "spurious" TE to be large and even strongly exceed a "correct" TE are presented and discussed.

Confounding effects of phase delays on causality estimation

Linear and non-linear techniques for inferring causal relations between the brain signals representing the underlying neuronal systems have become a powerful tool to extract the connectivity patterns in the brain. Typically these tools employ the idea of Granger causality, which is ultimately based on the temporal precedence between the signals. At the same time, phase synchronization between coupled neural ensembles is considered a mechanism implemented in the brain to integrate relevant neuronal ensembles to perform a cognitive or perceptual task. Phase synchronization can be studied by analyzing the effects of phase-locking between the brain signals. However, we should expect that there is no one-to-one mapping between the observed phase lag and the time precedence as specified by physically interacting systems. Specifically, phase lag observed between two signals may interfere with inferring causal relations. This could be of critical importance for the coupled non-linear oscillating systems, with possible time delays in coupling, when classical linear cross-spectrum strategies for solving phase ambiguity are not efficient. To demonstrate this, we used a prototypical model of coupled non-linear systems, and compared three typical pipelines of inferring Granger causality, as established in the literature. Specifically, we compared the performance of the spectral and information-theoretic Granger pipelines as well as standard Granger causality in their relations to the observed phase differences for frequencies at which the signals become synchronized to each other. We found that an information-theoretic approach, which takes into account different time lags between the past of one signal and the future of another signal, was the most robust to phase effects.

Phanerozoic Earth system evolution and marine biodiversity

The Phanerozoic fossil record of marine animal diversity covaries with the amount of marine sedimentary rock. The extent to which this covariation reflects a geologically controlled sampling bias remains unknown. We show that Phanerozoic records of seawater chemistry and continental flooding contain information on the diversity of marine animals that is independent of sedimentary rock quantity and sampling. Interrelationships among variables suggest long-term interactions among continental flooding, sulfur and carbon cycling, and macroevolution. Thus, mutual responses to interacting Earth systems, not sampling biases, explain much of the observed covariation between Phanerozoic patterns of sedimentation and fossil biodiversity. Linkages between biodiversity and environmental records likely reflect complex biotic responses to changing ocean redox conditions and long-term sea-level fluctuations driven by plate tectonics.

Framework to study dynamic dependencies in networks of interacting processes

The analysis of dynamic dependencies in complex systems such as the brain helps to understand how emerging properties arise from interactions. Here we propose an information-theoretic framework to analyze the dynamic dependencies in multivariate time-evolving systems. This framework constitutes a fully multivariate extension and unification of previous approaches based on bivariate or conditional mutual information and Granger causality or transfer entropy. We define multi-information measures that allow us to study the global statistical structure of the system as a whole, the total dependence between subsystems, and the temporal statistical structure of each subsystem. We develop a stationary and a nonstationary formulation of the framework. We then examine different decompositions of these multi-information measures. The transfer entropy naturally appears as a term in some of these decompositions. This allows us to examine its properties not as an isolated measure of interdependence but in the context of the complete framework. More generally we use causal graphs to study the specificity and sensitivity of all the measures appearing in these decompositions to different sources of statistical dependence arising from the causal connections between the subsystems. We illustrate that there is no straightforward relation between the strength of specific connections and specific terms in the decompositions. Furthermore, causal and noncausal statistical dependencies are not separable. In particular, the transfer entropy can be nonmonotonic in dependence on the connectivity strength between subsystems and is also sensitive to internal changes of the subsystems, so it should not be interpreted as a measure of connectivity strength. Altogether, in comparison to an analysis based on single isolated measures of interdependence, this framework is more powerful to analyze emergent properties in multivariate systems and to characterize functionally relevant changes in the dynamics.

Detecting causality in complex ecosystems

Identifying causal networks is important for effective policy and management recommendations on climate, epidemiology, financial regulation, and much else. We introduce a method, based on nonlinear state space reconstruction, that can distinguish causality from correlation. It extends to nonseparable weakly connected dynamic systems (cases not covered by the current Granger causality paradigm). The approach is illustrated both by simple models (where, in contrast to the real world, we know the underlying equations/relations and so can check the validity of our method) and by application to real ecological systems, including the controversial sardine-anchovy-temperature problem.

Finding weak directional coupling in multiscale time series

We propose a method to identify the direction of interactions between the components of a coupled system from observed data. This method tries to identify the direction of coupling for various severe conditions including for nonlinear dynamics, small sample size, weak coupling strength, noisy and multiscale data, and multistructure systems. Thus, it appears to be a promising method for real-world problems encountered in a variety of disciplines, such as physics, economics, and biology.

IDENTIFICATION OF NONLINEAR OSCILLATORY ACTIVITY EMBEDDED IN BROADBAND NEURAL SIGNALS

Oscillatory phenomena in the brain activity and their synchronization are frequently studied using mathematical models and analytic tools derived from nonlinear dynamics. In many experimental situations, however, neural signals have a broadband character and if oscillatory activity is present, its dynamical origin is unknown. To cope with these problems, a framework for detecting nonlinear oscillatory activity in broadband time series is presented. First, a narrow-band oscillatory mode is extracted from a broadband background. Second, it is tested whether the extracted mode is significantly different from linearly filtered noise, modelled as a linear stochastic process possibly passed through a static nonlinear transformation. If a nonlinear oscillatory mode is positively detected, further analysis using nonlinear approaches such as the phase synchronization analysis can potentially bring new information. For linear processes, however, standard approaches such as the coherence analysis are more appropriate and provide sufficient description of underlying interactions with smaller computational effort. The method is illustrated in a numerical example and applied to analyze experimentally obtained human EEG time series from a sleeping subject.

Transfer entropy estimation and directional coupling change detection in biomedical time series

Background

The detection of change in magnitude of directional coupling between two non-linear time series is a common subject of interest in the biomedical domain, including studies involving the respiratory chemoreflex system. Although transfer entropy is a useful tool in this avenue, no study to date has investigated how different transfer entropy estimation methods perform in typical biomedical applications featuring small sample size and presence of outliers.

Methods

With respect to detection of increased coupling strength, we compared three transfer entropy estimation techniques using both simulated time series and respiratory recordings from lambs. The following estimation methods were analyzed: fixed-binning with ranking, kernel density estimation (KDE), and the Darbellay-Vajda (D-V) adaptive partitioning algorithm extended to three dimensions. In the simulated experiment, sample size was varied from 50 to 200, while coupling strength was increased. In order to introduce outliers, the heavy-tailed Laplace distribution was utilized. In the lamb experiment, the objective was to detect increased respiratory-related chemosensitivity to O₂ and CO₂ induced by a drug, domperidone. Specifically, the separate influence of end-tidal PO₂ and PCO₂ on minute ventilation (V˙E) before and after administration of domperidone was analyzed.

Results

In the simulation, KDE detected increased coupling strength at the lowest SNR among the three methods. In the lamb experiment, D-V partitioning resulted in the statistically strongest increase in transfer entropy post-domperidone for PO2→V˙E. In addition, D-V partitioning was the only method that could detect an increase in transfer entropy for PCO2→V˙E, in agreement with experimental findings.

Conclusions

Transfer entropy is capable of detecting directional coupling changes in non-linear biomedical time series analysis featuring a small number of observations and presence of outliers. The results of this study suggest that fixed-binning, even with ranking, is too primitive, and although there is no clear winner between KDE and D-V partitioning, the reader should note that KDE requires more computational time and extensive parameter selection than D-V partitioning. We hope this study provides a guideline for selection of an appropriate transfer entropy estimation method.