Directionality of coupling from bivariate time series: how to avoid false causalities and missed connections

Methamphetamine alters the circadian oscillator and its couplings on multiple scales in Per1/2/3 knockout mice

Disruptions to circadian rhythms in mammals are associated with alterations in their physiological and mental states. Circadian rhythms are currently analyzed in the time domain using approaches such as actograms, thus failing to appreciate their time-localized characteristics, time-varying nature and multiscale dynamics. In this study, we apply time-resolved analysis to investigate behavioral rhythms in Per1/2/3 knockout (KO) mice and their changes following methamphetamine administration, focusing on circadian (around 24 h), low-frequency ultradian (around 7 h), high-frequency ultradian (around 30 min), and circabidian (around 48 h) oscillations. In the absence of methamphetamine, Per1/2/3 KO mice in constant darkness exhibited a dominant, ∼7 h oscillation. We demonstrate that methamphetamine exposure restores the circadian rhythm, although the frequency of the methamphetamine sensitive circadian oscillator varied considerably compared to the highly regular wild-type circadian rhythm. Additionally, methamphetamine increased multiscale activity and induced a circabidian oscillation in the Per1/2/3 KO mice. The information transfer between oscillatory modes, with frequencies around circadian, low-frequency ultradian and high-frequency ultradian activity, due to their mutual couplings, was also investigated. For Per1/2/3 KO mice in constant darkness, the most prevalent coupling was between low and high-frequency ultradian activity. Following methamphetamine administration, the coupling between the circadian and high-frequency ultradian activity became dominant. In each case, the direction of information transfer was between the corresponding phases from the slower to faster oscillations. The time-varying nature of the circadian rhythm exhibited in the absence of Per1/2/3 genes and following methamphetamine administration may have profound implications for health and disease.

Determining interaction directionality in complex biochemical networks from stationary measurements

Revealing interactions in complex systems from observed collective dynamics constitutes a fundamental inverse problem in science. Some methods may reveal undirected network topology, e.g., using node-node correlation. Yet, the direction of the interaction, thus a causal inference, remains to be determined - especially in steady-state observations. We introduce a method to infer the directionality within this network only from a "snapshot" of the abundances of the relevant molecules. We examine the validity of the approach for different properties of the system and the data recorded, such as the molecule's level variability, the effect of sampling and measurement errors. Simulations suggest that the given approach successfully infer the reaction rates in various cases.

Linear and nonlinear causality in financial markets

Identifying and quantifying co-dependence between financial instruments is a key challenge for researchers and practitioners in the financial industry. Linear measures such as the Pearson correlation are still widely used today, although their limited explanatory power is well known. In this paper, we present a much more general framework for assessing co-dependencies by identifying linear and nonlinear causalities in the complex system of financial markets. To do so, we use two different causal inference methods, transfer entropy and convergent cross-mapping, and employ Fourier transform surrogates to separate their linear and nonlinear contributions. We find that stock indices in Germany and the U.S. exhibit a significant degree of nonlinear causality and that correlation, while a very good proxy for linear causality, disregards nonlinear effects and hence underestimates causality itself. The presented framework enables the measurement of nonlinear causality, the correlation-causality fallacy, and motivates how causality can be used for inferring market signals, pair trading, and risk management of portfolios. Our results suggest that linear and nonlinear causality can be used as early warning indicators of abnormal market behavior, allowing for better trading strategies and risk management.

Causes of extreme events revealed by Rényi information transfer

Information-theoretic generalization of Granger causality principle, based on evaluation of conditional mutual information, also known as transfer entropy (CMI/TE), is redefined in the framework of Rényi entropy (RCMI/RTE). Using numerically generated data with a defined causal structure and examples of real data from the climate system, it is demonstrated that RCMI/RTE is able to identify the cause variable responsible for the occurrence of extreme values in an effect variable. In the presented example, the Siberian High was identified as the cause responsible for the increased probability of cold extremes in the winter and spring surface air temperature in Europe, while the North Atlantic Oscillation and blocking events can induce shifts of the whole temperature probability distribution.

Robust inference of causality in high-dimensional dynamical processes from the Information Imbalance of distance ranks

We introduce an approach which allows detecting causal relationships between variables for which the time evolution is available. Causality is assessed by a variational scheme based on the Information Imbalance of distance ranks, a statistical test capable of inferring the relative information content of different distance measures. We test whether the predictability of a putative driven system Y can be improved by incorporating information from a potential driver system X, without explicitly modeling the underlying dynamics and without the need to compute probability densities of the dynamic variables. This framework makes causality detection possible even between high-dimensional systems where only few of the variables are known or measured. Benchmark tests on coupled chaotic dynamical systems demonstrate that our approach outperforms other model-free causality detection methods, successfully handling both unidirectional and bidirectional couplings. We also show that the method can be used to robustly detect causality in human electroencephalography data.

Lead/Lag directionality is not generally equivalent to causality in nonlinear systems: Comparison of phase slope index and conditional mutual information

Applications of causal techniques to neural time series have increased extensively over last decades, including a wide and diverse family of methods focusing on electroencephalogram (EEG) analysis. Besides connectivity inferred in defined frequency bands, there is a growing interest in the analysis of cross-frequency interactions, in particular phase and amplitude coupling and directionality. Some studies show contradicting results of coupling directionality from high frequency to low frequency signal components, in spite of generally considered modulation of a high-frequency amplitude by a low-frequency phase. We have compared two widely used methods to estimate the directionality in cross frequency coupling: conditional mutual information (CMI) and phase slope index (PSI). The latter, applied to infer cross-frequency phase-amplitude directionality from animal intracranial recordings, gives opposite results when comparing to CMI. Both metrics were tested in a numerically simulated example of unidirectionally coupled Rössler systems, which helped to find the explanation of the contradictory results: PSI correctly estimates the lead/lag relationship which, however, is not generally equivalent to causality in the sense of directionality of coupling in nonlinear systems, correctly inferred by using CMI with surrogate data testing.

The time-evolving epileptic brain network: concepts, definitions, accomplishments, perspectives

Epilepsy is now considered a network disease that affects the brain across multiple levels of spatial and temporal scales. The paradigm shift from an epileptic focus-a discrete cortical area from which seizures originate-to a widespread epileptic network-spanning lobes and hemispheres-considerably advanced our understanding of epilepsy and continues to influence both research and clinical treatment of this multi-faceted high-impact neurological disorder. The epileptic network, however, is not static but evolves in time which requires novel approaches for an in-depth characterization. In this review, we discuss conceptual basics of network theory and critically examine state-of-the-art recording techniques and analysis tools used to assess and characterize a time-evolving human epileptic brain network. We give an account on current shortcomings and highlight potential developments towards an improved clinical management of epilepsy.

Network-motif delay differential analysis of brain activity during seizures

Delay Differential Analysis (DDA) is a nonlinear method for analyzing time series based on principles from nonlinear dynamical systems. DDA is extended here to incorporate network aspects to improve the dynamical characterization of complex systems. To demonstrate its effectiveness, DDA with network capabilities was first applied to the well-known Rössler system under different parameter regimes and noise conditions. Network-motif DDA, based on cortical regions, was then applied to invasive intracranial electroencephalographic data from drug-resistant epilepsy patients undergoing presurgical monitoring. The directional network motifs between brain areas that emerge from this analysis change dramatically before, during, and after seizures. Neural systems provide a rich source of complex data, arising from varying internal states generated by network interactions.

Usefulness and limitations of convergent cross sorting and continuity scaling methods for their application in simulated and real-world time series

Causality detection methods are valuable tools for detecting causal links in complex systems. The efficiency of continuity scaling (CS) and the convergent cross sorting (CSS) methods to detect causality was analysed. Usefulness and limitations of both methods in their application to simulated and real-world time series was explored under different scenarios. We find that CS is more robust and efficient than the CSS method for all simulated systems, even when increasing noise levels were considered. Both methods were not able to infer causality when time series with a marked difference in their main frequencies were analysed. Minimum time-series length required for the detection of a causal link depends on intrinsic system dynamics and on the method selected to detect it. Using simulated time series, only the CS method was capable to detect bidirectional causality. Causality detection, using the CS method, should at least include: (i) causality strength convergence analysis, (ii) statistical tests of significance, (iii) time-series standardization, and (iv) causality strength ratios as a strength indicator of relative causality between systems. Causality cannot be detected by either method in simulated time series that exhibit generalized synchronization.

A unified method for assessing the observability of dynamic complex systems

PROBLEM

Systems theory applied to biology and medicine assumes that the complexity of a system can be described by quasi-generic models to predict the behavior of many other similar systems. To this end, the aim of various research works in systems theory is to develop inductive modeling (based on data-intensive analysis) or deductive modeling (based on the deduction of mechanistic principles) to discover patterns and identify plausible correlations between past and present events, or to connect different causal relationships of interacting elements at different scales and compute mathematical predictions. Mathematical principles assume that there are constant and observable universal causal principles that apply to all biological systems. Nowadays, there are no suitable tools to assess the soundness of these universal causal principles, especially considering that organisms not only respond to environmental stimuli (and inherent processes) across multiple scales but also integrate information about and within these scales. This implies an uncontrollable degree of uncertainty.

METHODOLOGY

A method has been developed to detect the stability of causal processes by evaluating the information contained in the trajectories identified in a phase space. Time series patterns are analyzed using concepts from geometric information theory and persistent homology. In essence, recognizing these patterns in different time periods and evaluating their geometrically integrated information leads to the assessment of causal relationships. With this method, and together with the evaluation of persistent entropy in trajectories in relation to different individual systems, we have developed a method called Φ-S diagram as a complexity measure to recognize when organisms follow causal pathways leading to mechanistic responses.

RESULTS

We calculated the Φ-S diagram of a deterministic dataset available in the ICU repository to test the method's interpretability. We also calculated the Φ-S diagram of time series from health data available in the same repository. This includes patients' physiological response to sport measured with wearables outside laboratory conditions. We confirmed the mechanistic nature of both datasets in both calculations. In addition, there is evidence that some individuals show a high degree of autonomous response and variability. Therefore, persistent individual variability may limit the ability to observe the cardiac response. In this study, we present the first demonstration of the concept of developing a more robust framework for representing complex biological systems.

Directional coupling detection through cross-distance vectors

Inferring the coupling direction from measured time series of complex systems is challenging. We propose a state-space-based causality measure obtained from cross-distance vectors for quantifying interaction strength. It is a model-free noise-robust approach that requires only a few parameters. The approach is applicable to bivariate time series and is resilient to artefacts and missing values. The result is two coupling indices that quantify coupling strength in each direction more accurately than the already established state-space measures. We test the proposed method on different dynamical systems and analyze numerical stability. As a result, a procedure for optimal parameter selection is proposed, circumventing the challenge of determining the optimal embedding parameters. We show it is robust to noise and reliable in shorter time series. Moreover, we show that it can detect cardiorespiratory interaction in measured data. A numerically efficient implementation is available at https://repo.ijs.si/e2pub/cd-vec.

Improving the Process of Early-Warning Detection and Identifying the Most Affected Markets: Evidence from Subprime Mortgage Crisis and COVID-19 Outbreak-Application to American Stock Markets

Stock-market-crash predictability is of particular interest in the field of financial time-series analysis. Famous examples of major stock-market crashes are the real-estate bubble in 2008 and COVID-19 in 2020. Several studies have studied the prediction process without taking into consideration which markets might be falling into a crisis. To this end, a combination analysis is utilized in this manuscript. Firstly, the auto-regressive estimation (ARE) algorithm is successfully applied to electroencephalography (EEG) brain data for detecting diseases. The ARE algorithm is employed based on state-space modelling, which applies the expectation-maximization algorithm and Kalman filter. This manuscript introduces its application, for the first time, to stock-market data. For this purpose, a time-evolving interaction surface is constructed to observe the change in the surface topology. This enables tracking of the stock market's behavior over time and differentiates between different states. This provides a deep understanding of the underlying system behavior before, during, and after a crisis. Different patterns of the stock-market movements are recognized, providing novel information regarding detecting an early-warning sign. Secondly, a Granger-causality time-domain technique, called directed partial correlation, is employed to infer the underlying interconnectivity structure among markets. This information is crucial for investors and market players, enabling them to differentiate between those markets which will fall in a catastrophic loss, and those which will not. Consequently, they can make successful decisions towards selecting less risky portfolios, which guarantees lower losses. The results showed the effectiveness of the use of this methodology in the framework of the process of early-warning detection.

Age-dependent cardiorespiratory directional coupling in wake-resting state

Objective.Cooperation in the cardiorespiratory system helps maintain internal stability. Various types of system interactions have been investigated; however, the characteristics of the interactions have mostly been studied using data collected in well-defined physiological states, such as sleep. Furthermore, most analyses provided general information about the interaction, making it difficult to quantify how the systems influenced one another.Approach.Cardiorespiratory directional coupling was investigated in different age groups (20 young and 19 elderly subjects) in a wake-resting state. The directionality index (DI) was calculated using instantaneous phases from the heartbeat interval and respiratory signal to provide information about the strength and direction of interaction between the systems. Statistical analysis was performed between the groups on the DI and independent measures of directionality (nc_r: influence from cardiac system to respiratory system, and nc_c: influence from the respiratory system to the cardiac system).Main results.The values of DI were -0.52 and -0.17 in the young and elderly groups, respectively (p< 0.001). Furthermore, the values of nc_rand nc_cwere found to be significantly different between the groups (p< 0.001), respectively.Significance.Changes in both directions between the systems influence different aspects of cardiorespiratory coupling between the groups. This observation could be linked to different levels of autonomic modulation associated with ageing. Our approach could aid in quantitatively tracking and comprehending how systems interact in response to physiological and environmental changes. It could also be used to understand how abnormal interaction characteristics influence physiological system dysfunctions and disorders.

Identifying causality drivers and deriving governing equations of nonlinear complex systems

Identifying and describing the dynamics of complex systems is a central challenge in various areas of science, such as physics, finance, or climatology. While machine learning algorithms are increasingly overtaking traditional approaches, their inner workings and, thus, the drivers of causality remain elusive. In this paper, we analyze the causal structure of chaotic systems using Fourier transform surrogates and three different inference techniques: While we confirm that Granger causality is exclusively able to detect linear causality, transfer entropy and convergent cross-mapping indicate that causality is determined to a significant extent by nonlinear properties. For the Lorenz and Halvorsen systems, we find that their contribution is independent of the strength of the nonlinear coupling. Furthermore, we show that a simple rationale and calibration algorithm are sufficient to extract the governing equations directly from the causal structure of the data. Finally, we illustrate the applicability of the framework to real-world dynamical systems using financial data before and after the COVID-19 outbreak. It turns out that the pandemic triggered a fundamental rupture in the world economy, which is reflected in the causal structure and the resulting equations.

Compression complexity with ordinal patterns for robust causal inference in irregularly sampled time series

Distinguishing cause from effect is a scientific challenge resisting solutions from mathematics, statistics, information theory and computer science. Compression-Complexity Causality (CCC) is a recently proposed interventional measure of causality, inspired by Wiener–Granger’s idea. It estimates causality based on change in dynamical compression-complexity (or compressibility) of the effect variable, given the cause variable. CCC works with minimal assumptions on given data and is robust to irregular-sampling, missing-data and finite-length effects. However, it only works for one-dimensional time series. We propose an ordinal pattern symbolization scheme to encode multidimensional patterns into one-dimensional symbolic sequences, and thus introduce the Permutation CCC (PCCC). We demonstrate that PCCC retains all advantages of the original CCC and can be applied to data from multidimensional systems with potentially unobserved variables which can be reconstructed using the embedding theorem. PCCC is tested on numerical simulations and applied to paleoclimate data characterized by irregular and uncertain sampling and limited numbers of samples.

Causal Inference in Time Series in Terms of Rényi Transfer Entropy

Uncovering causal interdependencies from observational data is one of the great challenges of a nonlinear time series analysis. In this paper, we discuss this topic with the help of an information-theoretic concept known as Rényi’s information measure. In particular, we tackle the directional information flow between bivariate time series in terms of Rényi’s transfer entropy. We show that by choosing Rényi’s parameter α, we can appropriately control information that is transferred only between selected parts of the underlying distributions. This, in turn, is a particularly potent tool for quantifying causal interdependencies in time series, where the knowledge of “black swan” events, such as spikes or sudden jumps, are of key importance. In this connection, we first prove that for Gaussian variables, Granger causality and Rényi transfer entropy are entirely equivalent. Moreover, we also partially extend these results to heavy-tailed α-Gaussian variables. These results allow establishing a connection between autoregressive and Rényi entropy-based information-theoretic approaches to data-driven causal inference. To aid our intuition, we employed the Leonenko et al. entropy estimator and analyzed Rényi’s information flow between bivariate time series generated from two unidirectionally coupled Rössler systems. Notably, we find that Rényi’s transfer entropy not only allows us to detect a threshold of synchronization but it also provides non-trivial insight into the structure of a transient regime that exists between the region of chaotic correlations and synchronization threshold. In addition, from Rényi’s transfer entropy, we could reliably infer the direction of coupling and, hence, causality, only for coupling strengths smaller than the onset value of the transient regime, i.e., when two Rössler systems are coupled but have not yet entered synchronization.

Interaction within and between cortical networks subserving multisensory learning and its reorganization due to musical expertise

Recent advancements in the field of network science allow us to quantify inter-network information exchange and model the interaction within and between task-defined states of large-scale networks. Here, we modeled the inter- and intra- network interactions related to multisensory statistical learning. To this aim, we implemented a multifeatured statistical learning paradigm and measured evoked magnetoencephalographic responses to estimate task-defined state of functional connectivity based on cortical phase interaction. Each network state represented the whole-brain network processing modality-specific (auditory, visual and audiovisual) statistical learning irregularities embedded within a multisensory stimulation stream. The way by which domain-specific expertise re-organizes the interaction between the networks was investigated by a comparison of musicians and non-musicians. Between the modality-specific network states, the estimated connectivity quantified the characteristics of a supramodal mechanism supporting the identification of statistical irregularities that are compartmentalized and applied in the identification of uni-modal irregularities embedded within multisensory stimuli. Expertise-related re-organization was expressed by an increase of intra- and a decrease of inter-network connectivity, showing increased compartmentalization.

Phase-based causality analysis with partial mutual information from mixed embedding

Instantaneous phases extracted from multivariate time series can retain information about the relationships between the underlying mechanisms that generate the series. Although phases have been widely used in the study of nondirectional coupling and connectivity, they have not found similar appeal in the study of causality. Herein, we present a new method for phase-based causality analysis, which combines ideas from the mixed embedding technique and the information-theoretic approach to causality in coupled oscillatory systems. We then use the introduced method to investigate causality in simulated datasets of bivariate, unidirectionally paired systems from combinations of Rössler, Lorenz, van der Pol, and Mackey-Glass equations. We observe that causality analysis using the phases can capture the true causal relation for coupling strength smaller than the analysis based on the amplitudes can capture. On the other hand, the causality estimation based on the phases tends to have larger variability, which is attributed more to the phase extraction process than the actual phase-based causality method. In addition, an application on real electroencephalographic data from an experiment on elicited human emotional states reinforces the usefulness of phases in causality identification.

Relationship between Schreiber's transfer entropy and Liang-Kleeman information flow from the perspective of stochastic thermodynamics

Schreiber's transfer entropy is an important index for investigating the causal relationship between random variables. The Liang-Kleeman information flow is another analysis to demonstrate the causality within dynamical systems. Horowitz's information flow is introduced through multicomponent stochastic thermodynamics. In this study, I elucidate the relationship between Schreiber's transfer entropy and the Liang-Kleeman information flow through Horowitz's information flow. I consider the case in which the system changes according to the stochastic differential equation. I find that the Liang-Kleeman and Horowitz information flows differ by a term derived from the stochastic fluctuation. I also show that Schreiber's transfer entropy is not less than Horowitz's information flow. This study helps unify various indexes that determine the causal relationship between variables.

Dynamical ergodicity DDA reveals causal structure in time series

Determining synchronization, causality, and dynamical similarity in highly complex nonlinear systems like brains is challenging. Although distinct, these measures are related by the unknown deterministic structure of the underlying dynamical system. For two systems that are not independent on each other, either because they result from a common process or they are already synchronized, causality measures typically fail. Here, we introduce dynamical ergodicity to assess dynamical similarity between time series and then combine this new measure with cross-dynamical delay differential analysis to estimate causal interactions between time series. We first tested this approach on simulated data from coupled Rössler systems where ground truth was known. We then applied it to intracranial electroencephalographic data from patients with epilepsy and found distinct dynamical states that were highly predictive of epileptic seizures.

Causality in Reversed Time Series: Reversed or Conserved?

The inference of causal relations between observable phenomena is paramount across scientific disciplines; however, the means for such enterprise without experimental manipulation are limited. A commonly applied principle is that of the cause preceding and predicting the effect, taking into account other circumstances. Intuitively, when the temporal order of events is reverted, one would expect the cause and effect to apparently switch roles. This was previously demonstrated in bivariate linear systems and used in design of improved causal inference scores, while such behaviour in linear systems has been put in contrast with nonlinear chaotic systems where the inferred causal direction appears unchanged under time reversal. The presented work explores the conditions under which the causal reversal happens—either perfectly, approximately, or not at all—using theoretical analysis, low-dimensional examples, and network simulations, focusing on the simplified yet illustrative linear vector autoregressive process of order one. We start with a theoretical analysis that demonstrates that a perfect coupling reversal under time reversal occurs only under very specific conditions, followed up by constructing low-dimensional examples where indeed the dominant causal direction is even conserved rather than reversed. Finally, simulations of random as well as realistically motivated network coupling patterns from brain and climate show that level of coupling reversal and conservation can be well predicted by asymmetry and anormality indices introduced based on the theoretical analysis of the problem. The consequences for causal inference are discussed.

Granger-Causality Inference of the Existence of Unobserved Important Components in Network Analysis

Detecting causal interrelationships in multivariate systems, in terms of the Granger-causality concept, is of major interest for applications in many fields. Analyzing all the relevant components of a system is almost impossible, which contrasts with the concept of Granger causality. Not observing some components might, in turn, lead to misleading results, particularly if the missing components are the most influential and important in the system under investigation. In networks, the importance of a node depends on the number of nodes connected to this node. The degree of centrality is the most commonly used measure to identify important nodes in networks. There are two kinds of degree centrality, which are in-degree and out-degree. This manuscrpt is concerned with finding the highest out-degree among nodes to identify the most influential nodes. Inferring the existence of unobserved important components is critical in many multivariate interacting systems. The implications of such a situation are discussed in the Granger-causality framework. To this end, two of the most recent Granger-causality techniques, renormalized partial directed coherence and directed partial correlation, were employed. They were then compared in terms of their performance according to the extent to which they can infer the existence of unobserved important components. Sub-network analysis was conducted to aid these two techniques in inferring the existence of unobserved important components, which is evidenced in the results. By comparing the results of the two conducted techniques, it can be asserted that renormalized partial coherence outperforms directed partial correlation in the inference of existing unobserved important components that have not been included in the analysis. This measure of Granger causality and sub-network analysis emphasizes their ubiquitous successful applicability in such cases of the existence of hidden unobserved important components.

Detection of coupling in Duffing oscillator systems

In complex dynamical systems, the detection of coupling and its direction from observed time series is a challenging task. We study coupling in coupled Duffing oscillator systems in regular and chaotic dynamical regimes. By observing the conditional mutual information (CMI) based on the Shannon entropy, we successfully infer the direction of coupling for different system regimes. Moreover, we show that, in the weak coupling limit, the values of CMI can be used to infer the coupling parameters by computing the derivative of the conditional mutual information with respect to the coupling strength, called the information susceptibility. The complete numerical implementation is available at https://repo.ijs.si/mbresar/duffing-cmi.

Quantification of the Direct Solar Impact on Some Components of the Hydro-Climatic System

This study addresses the causal links between external factors and the main hydro-climatic variables by using a chain of methods to unravel the complexity of the direct sun–climate link. There is a gap in the literature on the description of a complete chain in addressing the structures of direct causal links of solar activity on terrestrial variables. This is why the present study uses the extensive facilities of the application of information theory in view of recent advances in different fields. Additionally, by other methods (e.g., neural networks) we first tested the existent non-linear links of solar–terrestrial influences on the hydro-climate system. The results related to the solar impact on terrestrial phenomena are promising, which is discriminant in the space-time domain. The implications prove robust for determining the causal measure of climate variables under direct solar impact, which makes it easier to consider solar activity in climate models by appropriate parametrizations. This study found that hydro-climatic variables are sensitive to solar impact only for certain frequencies (periods) and have a coherence with the Solar Flux only for some lags of the Solar Flux (in advance).

Cross-Scale Causality and Information Transfer in Simulated Epileptic Seizures

An information-theoretic approach for detecting causality and information transfer was applied to phases and amplitudes of oscillatory components related to different time scales and obtained using the wavelet transform from a time series generated by the Epileptor model. Three main time scales and their causal interactions were identified in the simulated epileptic seizures, in agreement with the interactions of the model variables. An approach consisting of wavelet transform, conditional mutual information estimation, and surrogate data testing applied to a single time series generated by the model was demonstrated to be successful in the identification of all directional (causal) interactions between the three different time scales described in the model. Thus, the methodology was prepared for the identification of causal cross-frequency phase–phase and phase–amplitude interactions in experimental and clinical neural data.

Fast and effective pseudo transfer entropy for bivariate data-driven causal inference

Identifying, from time series analysis, reliable indicators of causal relationships is essential for many disciplines. Main challenges are distinguishing correlation from causality and discriminating between direct and indirect interactions. Over the years many methods for data-driven causal inference have been proposed; however, their success largely depends on the characteristics of the system under investigation. Often, their data requirements, computational cost or number of parameters limit their applicability. Here we propose a computationally efficient measure for causality testing, which we refer to as pseudo transfer entropy (pTE), that we derive from the standard definition of transfer entropy (TE) by using a Gaussian approximation. We demonstrate the power of the pTE measure on simulated and on real-world data. In all cases we find that pTE returns results that are very similar to those returned by Granger causality (GC). Importantly, for short time series, pTE combined with time-shifted (T-S) surrogates for significance testing strongly reduces the computational cost with respect to the widely used iterative amplitude adjusted Fourier transform (IAAFT) surrogate testing. For example, for time series of 100 data points, pTE and T-S reduce the computational time by \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$82\%$$\end{document}82% with respect to GC and IAAFT. We also show that pTE is robust against observational noise. Therefore, we argue that the causal inference approach proposed here will be extremely valuable when causality networks need to be inferred from the analysis of a large number of short time series.

Causality and Information Transfer Between the Solar Wind and the Magnetosphere-Ionosphere System

An information-theoretic approach for detecting causality and information transfer is used to identify interactions of solar activity and interplanetary medium conditions with the Earth's magnetosphere-ionosphere systems. A causal information transfer from the solar wind parameters to geomagnetic indices is detected. The vertical component of the interplanetary magnetic field (Bz) influences the auroral electrojet (AE) index with an information transfer delay of 10 min and the geomagnetic disturbances at mid-latitudes measured by the symmetric field in the H component (SYM-H) index with a delay of about 30 min. Using a properly conditioned causality measure, no causal link between AE and SYM-H, or between magnetospheric substorms and magnetic storms can be detected. The observed causal relations can be described as linear time-delayed information transfer.

Time-Reversibility, Causality and Compression-Complexity

Detection of the temporal reversibility of a given process is an interesting time series analysis scheme that enables the useful characterisation of processes and offers an insight into the underlying processes generating the time series. Reversibility detection measures have been widely employed in the study of ecological, epidemiological and physiological time series. Further, the time reversal of given data provides a promising tool for analysis of causality measures as well as studying the causal properties of processes. In this work, the recently proposed Compression-Complexity Causality (CCC) measure (by the authors) is shown to be free of the assumption that the "cause precedes the effect", making it a promising tool for causal analysis of reversible processes. CCC is a data-driven interventional measure of causality (second rung on the Ladder of Causation) that is based on Effort-to-Compress (ETC), a well-established robust method to characterize the complexity of time series for analysis and classification. For the detection of the temporal reversibility of processes, we propose a novel measure called the Compressive Potential based Asymmetry Measure. This asymmetry measure compares the probability of the occurrence of patterns at different scales between the forward-time and time-reversed process using ETC. We test the performance of the measure on a number of simulated processes and demonstrate its effectiveness in determining the asymmetry of real-world time series of sunspot numbers, digits of the transcedental number π and heart interbeat interval variability.

Characteristics of the neural coding of causality

While causality processing is an essential cognitive capacity of the neural system, a systematic understanding of the neural coding of causality is still elusive. We propose a physically fundamental analysis of this issue and demonstrate that the neural dynamics encodes the original causality between external events near homomorphically. The causality coding is memory robust for the amount of historical information and features high precision but low recall. This coding process creates a sparser representation for the external causality. Finally, we propose a statistic characterization for the neural coding mapping from the original causality to the coded causality in neural dynamics.

Transfer entropies within dynamical effects framework

Transfer entropy (TE) is widely used in time-series analysis to detect causal couplings between temporally evolving objects. As a coupling strength quantifier, the TE alone often seems insufficient, raising the question of its further interpretations. Here the TE is related to dynamical causal effects (DCEs) which quantify long-term responses of a coupling recipient to variations in a coupling source or in a coupling itself: Detailed relationships are established for a paradigmatic stochastic dynamical system of bidirectionally coupled linear overdamped oscillators, their practical applications and possible extensions are discussed. It is shown that two widely used versions of the TE (original and infinite-history) can become qualitatively distinct, diverging to different long-term DCEs.

Measuring the Coupling Direction between Neural Oscillations with Weighted Symbolic Transfer Entropy

Neural oscillations reflect rhythmic fluctuations in the synchronization of neuronal populations and play a significant role in neural processing. To further understand the dynamic interactions between different regions in the brain, it is necessary to estimate the coupling direction between neural oscillations. Here, we developed a novel method, termed weighted symbolic transfer entropy (WSTE), that combines symbolic transfer entropy (STE) and weighted probability distribution to measure the directionality between two neuronal populations. The traditional STE ignores the degree of difference between the amplitude values of a time series. In our proposed WSTE method, this information is picked up by utilizing a weighted probability distribution. The simulation analysis shows that the WSTE method can effectively estimate the coupling direction between two neural oscillations. In comparison with STE, the new method is more sensitive to the coupling strength and is more robust against noise. When applied to epileptic electrocorticography data, a significant coupling direction from the anterior nucleus of thalamus (ANT) to the seizure onset zone (SOZ) was detected during seizures. Considering the superiorities of the WSTE method, it is greatly advantageous to measure the coupling direction between neural oscillations and consequently characterize the information flow between different brain regions.

The Human Organism as an Integrated Interaction Network: Recent Conceptual and Methodological Challenges

The field of Network Physiology aims to advance our understanding of how physiological systems and sub-systems interact to generate a variety of behaviors and distinct physiological states, to optimize the organism's functioning, and to maintain health. Within this framework, which considers the human organism as an integrated network, vertices are associated with organs while edges represent time-varying interactions between vertices. Likewise, vertices may represent networks on smaller spatial scales leading to a complex mixture of interacting homogeneous and inhomogeneous networks of networks. Lacking adequate analytic tools and a theoretical framework to probe interactions within and among diverse physiological systems, current approaches focus on inferring properties of time-varying interactions-namely strength, direction, and functional form-from time-locked recordings of physiological observables. To this end, a variety of bivariate or, in general, multivariate time-series-analysis techniques, which are derived from diverse mathematical and physical concepts, are employed and the resulting time-dependent networks can then be further characterized with methods from network theory. Despite the many promising new developments, there are still problems that evade from a satisfactory solution. Here we address several important challenges that could aid in finding new perspectives and inspire the development of theoretic and analytical concepts to deal with these challenges and in studying the complex interactions between physiological systems.

Explicit Granger causality in kernel Hilbert spaces

Granger causality (GC) is undoubtedly the most widely used method to infer cause-effect relations from observational time series. Several nonlinear alternatives to GC have been proposed based on kernel methods. We generalize kernel Granger causality by considering the variables' cross-relations explicitly in Hilbert spaces. The framework is shown to generalize the linear and kernel GC methods and comes with tighter bounds of performance based on Rademacher complexity. We successfully evaluate its performance in standard dynamical systems, as well as to identify the arrow of time in coupled Rössler systems, and it is exploited to disclose the El Niño-Southern Oscillation phenomenon footprints on soil moisture globally.

Implementation of two causal methods based on predictions in reconstructed state spaces

If deterministic dynamics is dominant in the data, then methods based on predictions in reconstructed state spaces can serve to detect causal relationships between and within the systems. Here we introduce two algorithms for such causal analysis. They are designed to detect causality from two time series but are potentially also applicable in a multivariate context. The first method is based on cross-predictions, and the second one on the so-called mixed predictions. In terms of performance, the cross-prediction method is considerably faster and less prone to false negatives. The predictability improvement method is slower, but in addition to causal detection, in a multivariate scenario, it also reveals which specific observables can help the most if we want to improve prediction. The study also highlights cases where our methods and state-space approaches generally seem to lose reliability. We propose a new perspective on these situations, namely that the variables under investigation have weak observability due to the complex nonlinear information flow in the system. Thus, in such cases, the failure of causality detection cannot be attributed to the methods themselves but to the use of data that do not allow reliable reconstruction of the underlying dynamics.

Refined nonuniform embedding for coupling detection in multivariate time series

State-space reconstruction is essential to analyze the dynamics and internal interactions of complex systems. However, it is difficult to estimate high-dimensional conditional mutual information and select the optimal time delays in most existing nonuniform state-space reconstruction methods. Therefore, we propose a nonuniform embedding method framed in information theory for state-space reconstruction. We use a low-dimensional approximation of conditional mutual information criterion for time delay selection, which is effectively solved by the particle swarm optimization algorithm. The obtained embedded vector has relatively strong independence and low redundancy, which better characterizes multivariable complex systems and detects coupling within complex systems. In addition, the proposed nonuniform embedding method exhibits good performance in coupling detection of linear stochastic, nonlinear stochastic, chaotic systems. In the actual application, the importance of small airports that cause delay propagation has been demonstrated by constructing the delay propagation network.

Transfer entropy rate through Lempel-Ziv complexity

The transfer entropy and the transfer entropy rate are closely related concepts that measure information exchange between two dynamical systems. These measures allow us to study linear and nonlinear causality relations and can be estimated through the use of different methodologies. However, some of them assume a data model and/or are computationally expensive. This article depicts a methodology to estimate the transfer entropy rate between two systems through the Lempel-Ziv complexity. This methodology offers a set of advantages: It estimates the transfer entropy rate from two single discrete series of measures, it is not computationally expensive, and it does not assume any data model. The simulation results over three different unidirectional coupled dynamical systems suggest that this methodology can be used to assess the direction and strength of the information flow between systems. Moreover, it provides good estimations for short-length time series.

Measuring Information Coupling between the Solar Wind and the Magnetosphere-Ionosphere System

The interaction between the solar wind and the Earth’s magnetosphere–ionosphere system is very complex, being essentially the result of the interplay between an external driver, the solar wind, and internal processes to the magnetosphere–ionosphere system. In this framework, modelling the Earth’s magnetosphere–ionosphere response to the changes of the solar wind conditions requires a correct identification of the causality relations between the different parameters/quantities used to monitor this coupling. Nowadays, in the framework of complex dynamical systems, both linear statistical tools and Granger causality models drastically fail to detect causal relationships between time series. Conversely, information theory-based concepts can provide powerful model-free statistical quantities capable of disentangling the complex nature of the causal relationships. In this work, we discuss how to deal with the problem of measuring causal information in the solar wind–magnetosphere–ionosphere system. We show that a time delay of about 30–60 min is found between solar wind and magnetospheric and ionospheric overall dynamics as monitored by geomagnetic indices, with a great information transfer observed between the z component of the interplanetary magnetic field and geomagnetic indices, while a lower transfer is found when other solar wind parameters are considered. This suggests that the best candidate for modelling the geomagnetic response to solar wind changes is the interplanetary magnetic field component Bz. A discussion of the relevance of our results in the framework of Space Weather is also provided.

Changes in Electroencephalography Activity in Response to Power Mobility Training: A Pilot Project

Purpose: The purposes of this pilot project were to examine the impact of power mobility training on (1) electroencephalography (EEG) activity in children with severe cerebral palsy (CP) and (2) power mobility skill acquisition. Method: A single-subject A-B-A-B research design with a 5-week duration for each phase (20 wk total) was replicated across three participants with severe CP (Gross Motor Function Classification System Level V). Data related to the target behaviour, as represented by EEG activity, were collected each week. Power mobility skills were assessed using the Canadian Occupational Performance Measure (COPM), the Wheelchair Skills Checklist (WSC), and the Assessment of Learning Powered mobility use (ALP). Weekly power mobility training was provided during the intervention phases. EEG data were analyzed by means of three measurement metrics (power spectral density, mutual information, and transfer entropy). Results: All three participants demonstrated changes in activation in frontoparietal EEG recordings and clinically significant improvements in power mobility skill acquisition as measured by the COPM as well as by the ALP and WSC. Conclusions: Power mobility training appeared to affect both neuroplastic and skill acquisition. Combining the use of EEG with direct therapist-observation measurement tools may provide a more complete understanding of the impact of power mobility training on children with severe CP.

Coupling in complex systems as information transfer across time scales

Complex systems such as the human brain or the Earth's climate consist of many subsystems interacting in intricate, nonlinear ways. Moreover, variability of such systems extends over broad ranges of spatial and temporal scales and dynamical phenomena on different scales also influence each other. In order to explain how to detect cross-scale causal interactions, we review information-theoretic formulation of the Granger causality in combination with computational statistics (surrogate data method) and demonstrate how this method can be used to infer driver-response relations from amplitudes and phases of coupled nonlinear dynamical systems. Considering complex systems evolving on multiple time scales, the reviewed methodology starts with a wavelet decomposition of a multi-scale signal into quasi-oscillatory modes of a limited bandwidth, described using their instantaneous phases and amplitudes. Then statistical associations, in particular, causality relations between phases or between phases and amplitudes on different time scales are tested using the conditional mutual information. As an application, we present the analysis of cross-scale interactions and information transfer in the dynamics of the El Niño Southern Oscillation. This article is part of the theme issue 'Coupling functions: dynamical interaction mechanisms in the physical, biological and social sciences'.

Transfer Entropy between Communities in Complex Financial Networks

In this paper, we analyze information flows between communities of financial markets, represented as complex networks. Each community, typically corresponding to a business sector, represents a significant part of the financial market and the detection of interactions between communities is crucial in the analysis of risk spreading in the financial markets. We show that the transfer entropy provides a coherent description of information flows in and between communities, also capturing non-linear interactions. Particularly, we focus on information transfer of rare events—typically large drops which can spread in the network. These events can be analyzed by Rényi transfer entropy, which enables to accentuate particular types of events. We analyze transfer entropies between communities of the five largest financial markets and compare the information flows with the correlation network of each market. From the transfer entropy picture, we can also identify the non-linear interactions, which are typical in the case of extreme events. The strongest flows can be typically observed between specific types of business sectors—financial sectors is the most significant example.

Maladaptive alterations of resting state cortical network in Tinnitus: A directed functional connectivity analysis of a larger MEG data set

The present study used resting state MEG whole-head recordings to identify how chronic tonal tinnitus relates to altered functional connectivity of brain's intrinsic cortical networks. Resting state MEG activity of 40 chronic tinnitus patients and 40 matched human controls was compared identifying significant alterations in intrinsic networks of the tinnitus population. Directed functional connectivity of the resting brain, at a whole cortex level, was estimated by means of a statistical comparison of the estimated phase Transfer Entropy (pTE) between the time-series of cortical activations, as reconstructed by LORETA. As pTE identifies the direction of the information flow, a detailed analysis of the connectivity differences between tinnitus patients and controls was possible. Results indicate that the group of tinnitus patients show increased connectivity from right dorsal prefrontal to right medial temporal areas. Our results go beyond previous findings by indicating that the role of the left para-hippocampal area is dictated by a modulation from dmPFC; a region that is part of the dorsal attention network (DAN), as well as implicated in the regulation of emotional processing. Additionally, this whole cortex analysis showed a crucial role of the left inferior parietal cortex, which modulated the activity of the right superior temporal gyrus, providing new hypotheses for the role of this area within the context of current tinnitus models. Overall, these maladaptive alterations of the structure of intrinsic cortical networks show a decrease in efficiency and small worldness of the resting state network of tinnitus patients, which is correlated to tinnitus distress.

Causality detection in cortical seizure dynamics using cross-dynamical delay differential analysis

Most natural systems, including the brain, are highly nonlinear and complex, and determining information flow among the components that make up these dynamic systems is challenging. One such example is identifying abnormal causal interactions among different brain areas that give rise to epileptic activities. Here, we introduce cross-dynamical delay differential analysis, an extension of delay differential analysis, as a tool to establish causal relationships from time series signals. Our method can infer causality from short time series signals as well as in the presence of noise. Furthermore, we can determine the onset of generalized synchronization directly from time series data, without having to consult the underlying equations. We first validate our method on simulated datasets from coupled dynamical systems and apply the method to intracranial electroencephalography data obtained from epilepsy patients to better characterize large-scale information flow during epilepsy.

The road ahead in clinical network neuroscience

Clinical network neuroscience, the study of brain network topology in neurological and psychiatric diseases, has become a mainstay field within clinical neuroscience. Being a multidisciplinary group of clinical network neuroscience experts based in The Netherlands, we often discuss the current state of the art and possible avenues for future investigations. These discussions revolve around questions like "How do dynamic processes alter the underlying structural network?" and "Can we use network neuroscience for disease classification?" This opinion paper is an incomplete overview of these discussions and expands on ten questions that may potentially advance the field. By no means intended as a review of the current state of the field, it is instead meant as a conversation starter and source of inspiration to others.

Correlation Dimension Detects Causal Links in Coupled Dynamical Systems

It is becoming increasingly clear that causal analysis of dynamical systems requires different approaches than, for example, causal analysis of interconnected autoregressive processes. In this study, a correlation dimension estimated in reconstructed state spaces is used to detect causality. If deterministic dynamics plays a dominant role in data then the method based on the correlation dimension can serve as a fast and reliable way to reveal causal relationships between and within the systems. This study demonstrates that the method, unlike most other causal approaches, detects causality well, even for very weak links. It can also identify cases of uncoupled systems that are causally affected by a hidden common driver.

Dominant Patterns of Information Flow in the Propagation of the Neuromagnetic Somatosensory Steady-State Response

Methods of functional connectivity are applied ubiquitously in studies involving non-invasive whole-brain signals, but may be not optimal for exploring the propagation of the steady-state responses, which are strong oscillatory patterns of neurodynamics evoked by periodic stimulation. In our study, we explore a functional network underlying the somatosensory steady-state response using methods of effective connectivity. Human magnetoencephalographic (MEG) data were collected in 10 young healthy adults during 23-Hz vibro-tactile stimulation of the right hand index finger. The whole-brain dynamics of MEG source activity was reconstructed with a linearly-constrained minimum variance beamformer. We applied information-theoretic tools to quantify asymmetries in information flows between primary somatosensory area SI and the rest of the brain. Our analysis identified a pattern of coupling, leading from area SI to a source in the secondary somato-sensory area SII, thalamus, and motor cortex all contralateral to stimuli as well as to a source in the cerebellum ipsilateral to the stimuli. Our results support previously reported empirical evidence collected both in in vitro and in vivo, indicating critical areas of activation of the somatosensory system at the level of systems neuroscience.

Inferring symmetric and asymmetric interactions between animals and groups from positional data

Interactions between domestic and wild species has become a global problem of growing interest. Global Position Systems (GPS) allow collection of vast records of time series of animal spatial movement, but there is need for developing analytical methods to efficiently use this information to unravel species interactions. This study assesses different methods to infer interactions and their symmetry between individual animals, social groups or species. We used two data sets, (i) a simulated one of the movement of two grazing species under different interaction scenarios by-species and by-individual, and (ii) a real time series of GPS data on the movements of sheep and deer grazing a large moorland plot. Different time series transformations were applied to capture the behaviour of the data (convex hull area, k^th nearest neighbour distance, distance to centre of mass, Voronoi tessellation area, distance to past position) to assess their efficiency in inferring the interactions using different techniques (cross correlation, Granger causality, network properties). The results indicate that the methods are more efficient assessing by-group interaction than by-individual interaction, and different transformations produce different outputs of the nature of the interaction. Both species maintained a consistent by-species grouping structure. The results do not provide clear evidence of inter-species interaction based on the traditional framework of niche partitioning in the guild of large herbivores. In view of the transformation-dependent results, it seems that in our experimental framework both species co-exist showing complex interactions. We provide guidelines for the use of the different transformations with respect to study aims and data quality. The study attempts to provide behavioural ecologists with tools to infer animal interactions and their symmetry based on positional data recorded by visual observation, conventional telemetry or GPS technology.

Symbolic time series analysis of fNIRS signals in brain development assessment

OBJECTIVE

Assessing an infant's brain development remains a challenge for neuroscientists and pediatricians despite great technological advances. As a non-invasive neuroimaging tool, functional near-infrared spectroscopy (fNIRS) has great advantages in monitoring an infant's brain activity. To explore the dynamic features of hemodynamic changes in infants, in-pattern exponent (IPE), anti-pattern exponent (APE), as well as permutation cross-mutual information (PCMI) based on symbolic dynamics are proposed to measure the phase differences and coupling strength in oxyhemoglobin (HbO) and deoxyhemoglobin (Hb) signals from fNIRS.

APPROACH

First, simulated sinusoidal oscillation signals and four coupled nonlinear systems were employed for performance assessments. Hilbert transform based measurements of hemoglobin phase oxygenation and deoxygenation (hPod) and phase-locking index of hPod (hPodL) were calculated for comparison. Then, the IPE, APE and PCMI indices from resting state fNIRS data of preterm, term infants and adults were calculated to estimate the phase difference and coupling of HbO and Hb. All indices' performance was assessed by the degree of monotonicity (DoM). The box plots and coefficients of variation (CV) were employed to assess the measurements and robustness in the results.

MAIN RESULTS

In the simulation analysis, IPE and APE can distinguish the phase difference of two sinusoidal oscillation signals. Both hPodL and PCMI can track the strength of two coupled nonlinear systems. Compared to hPodL, the PCMI had higher DoM indices in measuring the coupling of two nonlinear systems. In the fNIRS data analysis, similar to hPod, the IPE and APE can distinguish preterm, term infants, and adults in 0.01-0.05 Hz, 0.05-0.1 Hz, and 0.01-0.1 Hz frequency bands, respectively. PCMI more effectively distinguished the term and preterm infants than hPodL in the 0.05-0.1 Hz frequency band. As symbolic time series measures, the IPE and APE were able to detect the brain developmental changes in subjects of different ages. PCMI can assess the resting-state HbO and Hb coupling changes across different developmental ages, which may reflect the metabolic and neurovascular development.

SIGNIFICANCE

The symbolic-based methodologies are promising measures for fNIRS in estimating the brain development, especially in assessing newborns' brain developmental status.