Reliable detection of directional couplings using rank statistics

Reliable detection of directional couplings using cross-vector measures

Detecting directional couplings from time series is crucial in understanding complex dynamical systems. Various approaches based on reconstructed state-spaces have been developed for this purpose, including a cross-distance vector measure, which we introduced in our recent work. Here, we devise two new cross-vector measures that utilize ranks and time series estimates instead of distances. We analyze various deterministic and stochastic dynamics to compare our cross-vector approach against some established state-space-based approaches. We demonstrate that all three cross-vector measures can identify the correct coupling direction for a broader range of couplings for all considered dynamics. Among the three cross-vector measures, the rank-based variant performs the best. Comparing this novel measure to an established rank-based measure confirms that it is more noise-robust and less affected by linear cross-correlation. To extend this comparison to real-world signals, we combine both measures with the method of surrogates and analyze a database of electroencephalographic (EEG) recordings from epilepsy patients. This database contains signals from brain areas where the patients' seizures were detected first and signals from brain areas that were not involved in the seizure onset. A better discrimination between these signal classes is obtained by the cross-rank vector measure. Additionally, this measure proves to be robust to non-stationarity, as its results remain nearly unchanged when the analysis is repeated for the subset of EEG signals that were identified as stationary in previous work. These findings suggest that the cross-vector approach can serve as a valuable tool for researchers analyzing complex time series and for clinical applications.

Fractal Conditional Correlation Dimension Infers Complex Causal Networks

Determining causal inference has become popular in physical and engineering applications. While the problem has immense challenges, it provides a way to model the complex networks by observing the time series. In this paper, we present the optimal conditional correlation dimensional geometric information flow principle (oGeoC) that can reveal direct and indirect causal relations in a network through geometric interpretations. We introduce two algorithms that utilize the oGeoC principle to discover the direct links and then remove indirect links. The algorithms are evaluated using coupled logistic networks. The results indicate that when the number of observations is sufficient, the proposed algorithms are highly accurate in identifying direct causal links and have a low false positive rate.

Robust inference of causality in high-dimensional dynamical processes from the Information Imbalance of distance ranks

We introduce an approach which allows detecting causal relationships between variables for which the time evolution is available. Causality is assessed by a variational scheme based on the Information Imbalance of distance ranks, a statistical test capable of inferring the relative information content of different distance measures. We test whether the predictability of a putative driven system Y can be improved by incorporating information from a potential driver system X, without explicitly modeling the underlying dynamics and without the need to compute probability densities of the dynamic variables. This framework makes causality detection possible even between high-dimensional systems where only few of the variables are known or measured. Benchmark tests on coupled chaotic dynamical systems demonstrate that our approach outperforms other model-free causality detection methods, successfully handling both unidirectional and bidirectional couplings. We also show that the method can be used to robustly detect causality in human electroencephalography data.

The causality measure of partial mutual information from mixed embedding (PMIME) revisited

The measure of partial mutual information from mixed embedding (PMIME) is an information theory-based measure to accurately identify the direct and directional coupling, termed Granger causality or simply causality, between the observed variables or subsystems of a high-dimensional dynamical and complex system, without any a priori assumptions about the nature of the coupling relationship. In its core, it is a forward selection procedure that aims to iteratively identify the lag-dependence structure of a given observed variable (response) to all the other observed variables (candidate drivers). This model-free approach is capable of detecting nonlinear interactions, abundantly present in real-world complex systems, and it was shown to perform well on multivariate time series of moderately high dimension. However, the PMIME presents some inefficiencies in its performance mainly when applied on strongly stochastic (linear or nonlinear) systems as it may falsely detect non-existent relationships. Moreover, and by construction, the measure cannot extract purely synergetic relationships present in a system. In the current work, the issue of false detections is addressed by introducing an improved resampling significance test and a procedure of rechecking the identified drivers (backward revision). Regarding the inability to detect synergetic relationships, the PMIME is further enhanced by checking pairs as candidate drivers for the response variable after having considered all drivers individually. The effects of these modifications are investigated in a systematic simulation study on properly designed systems involving strong stochasticity, regressor terms with synergetic effects, and a system dimension ranging from 3 to 30. The overall results of the simulations indicate that these modifications indeed improve the performance of PMIME and alleviate to a significant degree the issues of the original algorithm. Guidelines for balancing between accuracy and computational efficiency are also given, particularly relevant for real-world applications. Finally, the measure performance is investigated in the study of futures of various government bonds and stock market indices in the period around COVID-19 pandemic.

Assessing cardiorespiratory interactions via lagged joint symbolic dynamics during spontaneous and controlled breathing

Introduction: Joint symbolic analysis (JSA) can be utilized to describe interactions between time series while accounting for time scales and nonlinear features. JSA is based on the computation of the rate of occurrence of joint patterns built after symbolization. Lagged JSA (LJSA) is obtained from the more classical JSA by introducing a delay/lead between patterns built over the two series and combined to form the joint scheme, thus monitoring coordinated patterns at different lags. Methods: In the present study, we applied LJSA for the assessment of cardiorespiratory coupling (CRC) from heart period (HP) variability and respiratory activity (R) in 19 healthy subjects (age: 27-35 years; 8 males, 11 females) during spontaneous breathing (SB) and controlled breathing (CB). The R rate of CB was selected to be indistinguishable from that of SB, namely, 15 breaths·minute-1 (CB15), or slower than SB, namely, 10 breaths·minute-1 (CB10), but in both cases, very rapid interactions between heart rate and R were known to be present. The ability of the LJSA approach to follow variations of the coupling strength was tested over a unidirectionally or bidirectionally coupled stochastic process and using surrogate data to test the null hypothesis of uncoupling. Results: We found that: i) the analysis of surrogate data proved that HP and R were significantly coupled in any experimental condition, and coupling was not more likely to occur at a specific time lag; ii) CB10 reduced CRC strength at the fastest time scales while increasing that at intermediate time scales, thus leaving the overall CRC strength unvaried; iii) despite exhibiting similar R rates and respiratory sinus arrhythmia, SB and CB15 induced different cardiorespiratory interactions; iv) no dominant temporal scheme was observed with relevant contributions of HP patterns either leading or lagging R. Discussion: LJSA is a useful methodology to explore HP-R dynamic interactions while accounting for time shifts and scales.

Usefulness and limitations of convergent cross sorting and continuity scaling methods for their application in simulated and real-world time series

Causality detection methods are valuable tools for detecting causal links in complex systems. The efficiency of continuity scaling (CS) and the convergent cross sorting (CSS) methods to detect causality was analysed. Usefulness and limitations of both methods in their application to simulated and real-world time series was explored under different scenarios. We find that CS is more robust and efficient than the CSS method for all simulated systems, even when increasing noise levels were considered. Both methods were not able to infer causality when time series with a marked difference in their main frequencies were analysed. Minimum time-series length required for the detection of a causal link depends on intrinsic system dynamics and on the method selected to detect it. Using simulated time series, only the CS method was capable to detect bidirectional causality. Causality detection, using the CS method, should at least include: (i) causality strength convergence analysis, (ii) statistical tests of significance, (iii) time-series standardization, and (iv) causality strength ratios as a strength indicator of relative causality between systems. Causality cannot be detected by either method in simulated time series that exhibit generalized synchronization.

Conditional cross-map-based technique: From pairwise dynamical causality to causal network reconstruction

Causality detection methods based on mutual cross mapping have been fruitfully developed and applied to data originating from nonlinear dynamical systems, where the causes and effects are non-separable. However, these pairwise methods still have shortcomings in discriminating typical network structures, including common drivers, indirect dependencies, and facing the curse of dimensionality, when they are stepping to causal network reconstruction. A few endeavors have been devoted to conquer these shortcomings. Here, we propose a novel method that could be regarded as one of these endeavors. Our method, named conditional cross-map-based technique, can eliminate third-party information and successfully detect direct dynamical causality, where the detection results can exactly be categorized into four standard normal forms by the designed criterion. To demonstrate the practical usefulness of our model-free, data-driven method, data generated from different representative models covering all kinds of network motifs and measured from real-world systems are investigated. Because correct identification of the direct causal links is essential to successful modeling, predicting, and controlling the underlying complex systems, our method does shed light on uncovering the inner working mechanisms of real-world systems only using the data experimentally obtained in a variety of disciplines.

A review of symbolic dynamics and symbolic reconstruction of dynamical systems

Discretizing a nonlinear time series enables us to calculate its statistics fast and rigorously. Before the turn of the century, the approach using partitions was dominant. In the last two decades, discretization via permutations has been developed to a powerful methodology, while recurrence plots have recently begun to be recognized as a method of discretization. In the meantime, horizontal visibility graphs have also been proposed to discretize time series. In this review, we summarize these methods and compare them from the viewpoint of symbolic dynamics, which is the right framework to study the symbolic representation of nonlinear time series and the inverse process: the symbolic reconstruction of dynamical systems. As we will show, symbolic dynamics is currently a very active research field with interesting applications.

Directional coupling detection through cross-distance vectors

Inferring the coupling direction from measured time series of complex systems is challenging. We propose a state-space-based causality measure obtained from cross-distance vectors for quantifying interaction strength. It is a model-free noise-robust approach that requires only a few parameters. The approach is applicable to bivariate time series and is resilient to artefacts and missing values. The result is two coupling indices that quantify coupling strength in each direction more accurately than the already established state-space measures. We test the proposed method on different dynamical systems and analyze numerical stability. As a result, a procedure for optimal parameter selection is proposed, circumventing the challenge of determining the optimal embedding parameters. We show it is robust to noise and reliable in shorter time series. Moreover, we show that it can detect cardiorespiratory interaction in measured data. A numerically efficient implementation is available at https://repo.ijs.si/e2pub/cd-vec.

Reliable detection of causal asymmetries in dynamical systems

Knowledge about existence, strength, and dominant direction of causal influences is of paramount importance for understanding complex systems. Current methods deduce ambiguous causal links among different observables from (complex) dynamical systems, if a limited amount of realistic data is used. It is particularly difficult to infer the dominant direction of causal influence for synchronizing systems. Missing is a statistically well defined approach that avoids false positive detection while being sensitive for weak interactions. The proposed method exploits the local inflation of manifolds to estimate upper bounds on the information loss among state reconstructions and tests for the absence of causal influences. Simulated data demonstrates that it is robust to intrinsic noise, copes with synchronization, and tolerates moderate amounts of measurement noise.

Improving the Process of Early-Warning Detection and Identifying the Most Affected Markets: Evidence from Subprime Mortgage Crisis and COVID-19 Outbreak-Application to American Stock Markets

Stock-market-crash predictability is of particular interest in the field of financial time-series analysis. Famous examples of major stock-market crashes are the real-estate bubble in 2008 and COVID-19 in 2020. Several studies have studied the prediction process without taking into consideration which markets might be falling into a crisis. To this end, a combination analysis is utilized in this manuscript. Firstly, the auto-regressive estimation (ARE) algorithm is successfully applied to electroencephalography (EEG) brain data for detecting diseases. The ARE algorithm is employed based on state-space modelling, which applies the expectation-maximization algorithm and Kalman filter. This manuscript introduces its application, for the first time, to stock-market data. For this purpose, a time-evolving interaction surface is constructed to observe the change in the surface topology. This enables tracking of the stock market's behavior over time and differentiates between different states. This provides a deep understanding of the underlying system behavior before, during, and after a crisis. Different patterns of the stock-market movements are recognized, providing novel information regarding detecting an early-warning sign. Secondly, a Granger-causality time-domain technique, called directed partial correlation, is employed to infer the underlying interconnectivity structure among markets. This information is crucial for investors and market players, enabling them to differentiate between those markets which will fall in a catastrophic loss, and those which will not. Consequently, they can make successful decisions towards selecting less risky portfolios, which guarantees lower losses. The results showed the effectiveness of the use of this methodology in the framework of the process of early-warning detection.

Contributions to surface air temperature trends estimated from climate time series: Medium-term causalities

Contributions of various natural and anthropogenic factors to trends of surface air temperatures at different latitudes of the Northern and Southern hemispheres on various temporal horizons are estimated from climate data since the 19th century in empirical autoregressive models. Along with anthropogenic forcing, we assess the impact of several natural climate modes including Atlantic Multidecadal Oscillation, El-Nino/Southern Oscillation, Interdecadal Pacific Oscillation, Pacific Decadal Oscillation, and Antarctic Oscillation. On relatively short intervals of the length of two or three decades, contributions of climate variability modes are considerable and comparable to the contributions of greenhouse gases and even exceed the latter. On longer intervals of about half a century and greater, the contributions of greenhouse gases dominate at all latitudinal belts including polar, middle, and tropical ones.

Phase-based causality analysis with partial mutual information from mixed embedding

Instantaneous phases extracted from multivariate time series can retain information about the relationships between the underlying mechanisms that generate the series. Although phases have been widely used in the study of nondirectional coupling and connectivity, they have not found similar appeal in the study of causality. Herein, we present a new method for phase-based causality analysis, which combines ideas from the mixed embedding technique and the information-theoretic approach to causality in coupled oscillatory systems. We then use the introduced method to investigate causality in simulated datasets of bivariate, unidirectionally paired systems from combinations of Rössler, Lorenz, van der Pol, and Mackey-Glass equations. We observe that causality analysis using the phases can capture the true causal relation for coupling strength smaller than the analysis based on the amplitudes can capture. On the other hand, the causality estimation based on the phases tends to have larger variability, which is attributed more to the phase extraction process than the actual phase-based causality method. In addition, an application on real electroencephalographic data from an experiment on elicited human emotional states reinforces the usefulness of phases in causality identification.

Generative formalism of causality quantifiers for processes

The concept of dynamical causal effect (DCE) is generalized and equipped with a formalism which allows one to formulate in a unified manner and interrelate a variety of causality quantifiers used in time series analysis. An elementary DCE from a subsystem Y to a subsystem X is defined within the stochastic dynamical systems framework as a response of a future X state to an appropriate variation of an initial (X,Y)-state distribution or a certain parameter of Y or of the coupling element Y→X; this response is quantified in a probabilistic sense via a certain distinction functional; elementary DCEs are assembled over a set of initial variations via an assemblage functional. To include all those aspects, a "triple brackets formula" for the general DCE is suggested and serves as a first principle to produce specific causality quantifiers as realizations of the general DCE. As an application, transfer entropy and Liang-Kleeman information flow are related surprisingly as opposite limit cases in a family of DCEs; it is shown that their "nats per time unit" may differ drastically. The suggested DCE viewpoint links any formal causality quantifier to "intervention-effect" experiments, i.e., future responses to initial variations, and so provides its dynamical interpretation, opening a way to its further physical interpretations in studies of physical systems.

Convergent cross sorting for estimating dynamic coupling

Natural systems exhibit diverse behavior generated by complex interactions between their constituent parts. To characterize these interactions, we introduce Convergent Cross Sorting (CCS), a novel algorithm based on convergent cross mapping (CCM) for estimating dynamic coupling from time series data. CCS extends CCM by using the relative ranking of distances within state-space reconstructions to improve the prior methods' performance at identifying the existence, relative strength, and directionality of coupling across a wide range of signal and noise characteristics. In particular, relative to CCM, CCS has a large performance advantage when analyzing very short time series data and data from continuous dynamical systems with synchronous behavior. This advantage allows CCS to better uncover the temporal and directional relationships within systems that undergo frequent and short-lived switches in dynamics, such as neural systems. In this paper, we validate CCS on simulated data and demonstrate its applicability to electrophysiological recordings from interacting brain regions.

Granger-Causality Inference of the Existence of Unobserved Important Components in Network Analysis

Detecting causal interrelationships in multivariate systems, in terms of the Granger-causality concept, is of major interest for applications in many fields. Analyzing all the relevant components of a system is almost impossible, which contrasts with the concept of Granger causality. Not observing some components might, in turn, lead to misleading results, particularly if the missing components are the most influential and important in the system under investigation. In networks, the importance of a node depends on the number of nodes connected to this node. The degree of centrality is the most commonly used measure to identify important nodes in networks. There are two kinds of degree centrality, which are in-degree and out-degree. This manuscrpt is concerned with finding the highest out-degree among nodes to identify the most influential nodes. Inferring the existence of unobserved important components is critical in many multivariate interacting systems. The implications of such a situation are discussed in the Granger-causality framework. To this end, two of the most recent Granger-causality techniques, renormalized partial directed coherence and directed partial correlation, were employed. They were then compared in terms of their performance according to the extent to which they can infer the existence of unobserved important components. Sub-network analysis was conducted to aid these two techniques in inferring the existence of unobserved important components, which is evidenced in the results. By comparing the results of the two conducted techniques, it can be asserted that renormalized partial coherence outperforms directed partial correlation in the inference of existing unobserved important components that have not been included in the analysis. This measure of Granger causality and sub-network analysis emphasizes their ubiquitous successful applicability in such cases of the existence of hidden unobserved important components.

Transfer entropies within dynamical effects framework

Transfer entropy (TE) is widely used in time-series analysis to detect causal couplings between temporally evolving objects. As a coupling strength quantifier, the TE alone often seems insufficient, raising the question of its further interpretations. Here the TE is related to dynamical causal effects (DCEs) which quantify long-term responses of a coupling recipient to variations in a coupling source or in a coupling itself: Detailed relationships are established for a paradigmatic stochastic dynamical system of bidirectionally coupled linear overdamped oscillators, their practical applications and possible extensions are discussed. It is shown that two widely used versions of the TE (original and infinite-history) can become qualitatively distinct, diverging to different long-term DCEs.

Measuring the Coupling Direction between Neural Oscillations with Weighted Symbolic Transfer Entropy

Neural oscillations reflect rhythmic fluctuations in the synchronization of neuronal populations and play a significant role in neural processing. To further understand the dynamic interactions between different regions in the brain, it is necessary to estimate the coupling direction between neural oscillations. Here, we developed a novel method, termed weighted symbolic transfer entropy (WSTE), that combines symbolic transfer entropy (STE) and weighted probability distribution to measure the directionality between two neuronal populations. The traditional STE ignores the degree of difference between the amplitude values of a time series. In our proposed WSTE method, this information is picked up by utilizing a weighted probability distribution. The simulation analysis shows that the WSTE method can effectively estimate the coupling direction between two neural oscillations. In comparison with STE, the new method is more sensitive to the coupling strength and is more robust against noise. When applied to epileptic electrocorticography data, a significant coupling direction from the anterior nucleus of thalamus (ANT) to the seizure onset zone (SOZ) was detected during seizures. Considering the superiorities of the WSTE method, it is greatly advantageous to measure the coupling direction between neural oscillations and consequently characterize the information flow between different brain regions.

The Human Organism as an Integrated Interaction Network: Recent Conceptual and Methodological Challenges

The field of Network Physiology aims to advance our understanding of how physiological systems and sub-systems interact to generate a variety of behaviors and distinct physiological states, to optimize the organism's functioning, and to maintain health. Within this framework, which considers the human organism as an integrated network, vertices are associated with organs while edges represent time-varying interactions between vertices. Likewise, vertices may represent networks on smaller spatial scales leading to a complex mixture of interacting homogeneous and inhomogeneous networks of networks. Lacking adequate analytic tools and a theoretical framework to probe interactions within and among diverse physiological systems, current approaches focus on inferring properties of time-varying interactions-namely strength, direction, and functional form-from time-locked recordings of physiological observables. To this end, a variety of bivariate or, in general, multivariate time-series-analysis techniques, which are derived from diverse mathematical and physical concepts, are employed and the resulting time-dependent networks can then be further characterized with methods from network theory. Despite the many promising new developments, there are still problems that evade from a satisfactory solution. Here we address several important challenges that could aid in finding new perspectives and inspire the development of theoretic and analytical concepts to deal with these challenges and in studying the complex interactions between physiological systems.

Implementation of two causal methods based on predictions in reconstructed state spaces

If deterministic dynamics is dominant in the data, then methods based on predictions in reconstructed state spaces can serve to detect causal relationships between and within the systems. Here we introduce two algorithms for such causal analysis. They are designed to detect causality from two time series but are potentially also applicable in a multivariate context. The first method is based on cross-predictions, and the second one on the so-called mixed predictions. In terms of performance, the cross-prediction method is considerably faster and less prone to false negatives. The predictability improvement method is slower, but in addition to causal detection, in a multivariate scenario, it also reveals which specific observables can help the most if we want to improve prediction. The study also highlights cases where our methods and state-space approaches generally seem to lose reliability. We propose a new perspective on these situations, namely that the variables under investigation have weak observability due to the complex nonlinear information flow in the system. Thus, in such cases, the failure of causality detection cannot be attributed to the methods themselves but to the use of data that do not allow reliable reconstruction of the underlying dynamics.

Causal network discovery by iterative conditioning: Comparison of algorithms

Estimating causal interactions in complex dynamical systems is an important problem encountered in many fields of current science. While a theoretical solution for detecting the causal interactions has been previously formulated in the framework of prediction improvement, it generally requires the computation of high-dimensional information functionals-a situation invoking the curse of dimensionality with increasing network size. Recently, several methods have been proposed to alleviate this problem, based on iterative procedures for the assessment of conditional (in)dependences. In the current work, we bring a comparison of several such prominent approaches. This is done both by theoretical comparison of the algorithms using a formulation in a common framework and by numerical simulations including realistic complex coupling patterns. The theoretical analysis highlights the key similarities and differences between the algorithms, hinting on their comparative strengths and weaknesses. The method assumptions and specific properties such as false positive control and order-dependence are discussed. Numerical simulations suggest that while the accuracy of most of the algorithms is almost indistinguishable, there are substantial differences in their computational demands, ranging theoretically from polynomial to exponential complexity and leading to substantial differences in computation time in realistic scenarios depending on the density and size of networks. Based on the analysis of the algorithms and numerical simulations, we propose a hybrid approach providing competitive accuracy with improved computational efficiency.

Evaluation of Granger Causality Measures for Constructing Networks from Multivariate Time Series

Granger causality and variants of this concept allow the study of complex dynamical systems as networks constructed from multivariate time series. In this work, a large number of Granger causality measures used to form causality networks from multivariate time series are assessed. These measures are in the time domain, such as model-based and information measures, the frequency domain, and the phase domain. The study aims also to compare bivariate and multivariate measures, linear and nonlinear measures, as well as the use of dimension reduction in linear model-based measures and information measures. The latter is particular relevant in the study of high-dimensional time series. For the performance of the multivariate causality measures, low and high dimensional coupled dynamical systems are considered in discrete and continuous time, as well as deterministic and stochastic. The measures are evaluated and ranked according to their ability to provide causality networks that match the original coupling structure. The simulation study concludes that the Granger causality measures using dimension reduction are superior and should be preferred particularly in studies involving many observed variables, such as multi-channel electroencephalograms and financial markets.

Inferring directed networks using a rank-based connectivity measure

Inferring the topology of a network using the knowledge of the signals of each of the interacting units is key to understanding real-world systems. One way to address this problem is using data-driven methods like cross-correlation or mutual information. However, these measures lack the ability to distinguish the direction of coupling. Here, we use a rank-based nonlinear interdependence measure originally developed for pairs of signals. This measure not only allows one to measure the strength but also the direction of the coupling. Our results for a system of coupled Lorenz dynamics show that we are able to consistently infer the underlying network for a subrange of the coupling strength and link density. Furthermore, we report that the addition of dynamical noise can benefit the reconstruction. Finally, we show an application to multichannel electroencephalographic recordings from an epilepsy patient.

Warped phase coherence: An empirical synchronization measure combining phase and amplitude information

The entrainment between weakly coupled nonlinear oscillators, as well as between complex signals such as those representing physiological activity, is frequently assessed in terms of whether a stable relationship is detectable between the instantaneous phases extracted from the measured or simulated time-series via the analytic signal. Here, we demonstrate that adding a possibly complex constant value to this normally null-mean signal has a non-trivial warping effect. Among other consequences, this introduces a level of sensitivity to the amplitude fluctuations and average relative phase. By means of simulations of Rössler systems and experiments on single-transistor oscillator networks, it is shown that the resulting coherence measure may have an empirical value in improving the inference of the structural couplings from the dynamics. When tentatively applied to the electroencephalogram recorded while performing imaginary and real movements, this straightforward modification of the phase locking value substantially improved the classification accuracy. Hence, its possible practical relevance in brain-computer and brain-machine interfaces deserves consideration.

Mean field phase synchronization between chimera states

We study two-layer networks of identical phase oscillators. Each individual layer is a ring network for which a non-local intra-layer coupling leads to the formation of a chimera state. The number of oscillators and their natural frequencies is in general different across the layers. We couple the phases of individual oscillators in one layer to the phase of the mean field of the other layer. This coupling from the mean field to individual oscillators is done in both directions. For a sufficient strength of this inter-layer coupling, the phases of the mean fields lock across the two layers. In contrast, both layers continue to exhibit chimera states with no locking between the phases of individual oscillators across layers, and the two mean field amplitudes remain uncorrelated. Hence, the networks' mean fields show phase synchronization which is analogous to the one between low-dimensional chaotic oscillators. The required coupling strength to achieve this mean field phase synchronization increases with the mismatches in the network sizes and the oscillators' natural frequencies.

Causality, dynamical systems and the arrow of time

Using several methods for detection of causality in time series, we show in a numerical study that coupled chaotic dynamical systems violate the first principle of Granger causality that the cause precedes the effect. While such a violation can be observed in formal applications of time series analysis methods, it cannot occur in nature, due to the relation between entropy production and temporal irreversibility. The obtained knowledge, however, can help to understand the type of causal relations observed in experimental data, namely, it can help to distinguish linear transfer of time-delayed signals from nonlinear interactions. We illustrate these findings in causality detected in experimental time series from the climate system and mammalian cardio-respiratory interactions.

Apparent remote synchronization of amplitudes: A demodulation and interference effect

A form of "remote synchronization" was recently described, wherein amplitude fluctuations across a ring of non-identical, non-linear electronic oscillators become entrained into spatially-structured patterns. According to linear models and mutual information, synchronization and causality dip at a certain distance, then recover before eventually fading. Here, the underlying mechanism is finally elucidated through novel experiments and simulations. The system non-linearity is found to have a dual role: it supports chaotic dynamics, and it enables the energy exchange between the lower and higher sidebands of a predominant frequency. This frequency acts as carrier signal in an arrangement resembling standard amplitude modulation, wherein the lower sideband and the demodulated baseband signals spectrally overlap. Due to a spatially-dependent phase relationship, at a certain distance near-complete destructive interference occurs between them, causing the observed dip. Methods suitable for detecting non-trivial entrainment, such as transfer entropy and the auxiliary system approach, nevertheless, reveal that synchronization and causality actually decrease with distance monotonically. Remoteness is, therefore, arguably only apparent, as also reflected in the propagation of external perturbations. These results demonstrate a complex mechanism of dynamical interdependence, and exemplify how it can lead to incorrectly inferring synchronization and causality.

Comparison of six methods for the detection of causality in a bivariate time series

In this comparative study, six causality detection methods were compared, namely, the Granger vector autoregressive test, the extended Granger test, the kernel version of the Granger test, the conditional mutual information (transfer entropy), the evaluation of cross mappings between state spaces, and an assessment of predictability improvement due to the use of mixed predictions. Seven test data sets were analyzed: linear coupling of autoregressive models, a unidirectional connection of two Hénon systems, a unidirectional connection of chaotic systems of Rössler and Lorenz type and of two different Rössler systems, an example of bidirectionally connected two-species systems, a fishery model as an example of two correlated observables without a causal relationship, and an example of mediated causality. We tested not only 20000 points long clean time series but also noisy and short variants of the data. The standard and the extended Granger tests worked only for the autoregressive models. The remaining methods were more successful with the more complex test examples, although they differed considerably in their capability to reveal the presence and the direction of coupling and to distinguish causality from mere correlation.

Topological Causality in Dynamical Systems

Determination of causal relations among observables is of fundamental interest in many fields dealing with complex systems. Since nonlinear systems generically behave as wholes, classical notions of causality assuming separability of subsystems often turn out inadequate. Still lacking is a mathematically transparent measure of the magnitude of effective causal influences in cyclic systems. For deterministic systems we found that the expansions of mappings among time-delay state space reconstructions from different observables not only reflect the directed coupling strengths, but also the dependency of effective influences on the system's temporally varying state. Estimation of the expansions from pairs of time series is straightforward and used to define novel causality indices. Mathematical and numerical analysis demonstrate that they reveal the asymmetry of causal influences including their time dependence, as well as provide measures for the effective strengths of causal links in complex systems.

Detection of coupling delay: A problem not yet solved

Nonparametric detection of coupling delay in unidirectionally and bidirectionally coupled nonlinear dynamical systems is examined. Both continuous and discrete-time systems are considered. Two methods of detection are assessed-the method based on conditional mutual information-the CMI method (also known as the transfer entropy method) and the method of convergent cross mapping-the CCM method. Computer simulations show that neither method is generally reliable in the detection of coupling delays. For continuous-time chaotic systems, the CMI method appears to be more sensitive and applicable in a broader range of coupling parameters than the CCM method. In the case of tested discrete-time dynamical systems, the CCM method has been found to be more sensitive, while the CMI method required much stronger coupling strength in order to bring correct results. However, when studied systems contain a strong oscillatory component in their dynamics, results of both methods become ambiguous. The presented study suggests that results of the tested algorithms should be interpreted with utmost care and the nonparametric detection of coupling delay, in general, is a problem not yet solved.

Robustness and versatility of a nonlinear interdependence method for directional coupling detection from spike trains

The detection of directional couplings between dynamics based on measured spike trains is a crucial problem in the understanding of many different systems. In particular, in neuroscience it is important to assess the connectivity between neurons. One of the approaches that can estimate directional coupling from the analysis of point processes is the nonlinear interdependence measure L. Although its efficacy has already been demonstrated, it still needs to be tested under more challenging and realistic conditions prior to an application to real data. Thus, in this paper we use the Hindmarsh-Rose model system to test the method in the presence of noise and for different spiking regimes. We also examine the influence of different parameters and spike train distances. Our results show that the measure L is versatile and robust to various types of noise, and thus suitable for application to experimental data.

Reconstructing networks of pulse-coupled oscillators from spike trains

We present an approach for reconstructing networks of pulse-coupled neuronlike oscillators from passive observation of pulse trains of all nodes. It is assumed that units are described by their phase response curves and that their phases are instantaneously reset by incoming pulses. Using an iterative procedure, we recover the properties of all nodes, namely their phase response curves and natural frequencies, as well as strengths of all directed connections.

Atypical transistor-based chaotic oscillators: Design, realization, and diversity

In this paper, we show that novel autonomous chaotic oscillators based on one or two bipolar junction transistors and a limited number of passive components can be obtained via random search with suitable heuristics. Chaos is a pervasive occurrence in these circuits, particularly after manual adjustment of a variable resistor placed in series with the supply voltage source. Following this approach, 49 unique circuits generating chaotic signals when physically realized were designed, representing the largest collection of circuits of this kind to date. These circuits are atypical as they do not trivially map onto known topologies or variations thereof. They feature diverse spectra and predominantly anti-persistent monofractal dynamics. Notably, we recurrently found a circuit comprising one resistor, one transistor, two inductors, and one capacitor, which generates a range of attractors depending on the parameter values. We also found a circuit yielding an irregular quantized spike-train resembling some aspects of neural discharge and another one generating a double-scroll attractor, which represent the smallest known transistor-based embodiments of these behaviors. Through three representative examples, we additionally show that diffusive coupling of heterogeneous oscillators of this kind may give rise to complex entrainment, such as lag synchronization with directed information transfer and generalized synchronization. The replicability and reproducibility of the experimental findings are good.

Efficient Computation of Functional Brain Networks: toward Real-Time Functional Connectivity

Functional Connectivity has demonstrated to be a key concept for unraveling how the brain balances functional segregation and integration properties while processing information. This work presents a set of open-source tools that significantly increase computational efficiency of some well-known connectivity indices and Graph-Theory measures. PLV, PLI, ImC, and wPLI as Phase Synchronization measures, Mutual Information as an information theory based measure, and Generalized Synchronization indices are computed much more efficiently than prior open-source available implementations. Furthermore, network theory related measures like Strength, Shortest Path Length, Clustering Coefficient, and Betweenness Centrality are also implemented showing computational times up to thousands of times faster than most well-known implementations. Altogether, this work significantly expands what can be computed in feasible times, even enabling whole-head real-time network analysis of brain function.

Coupling strength versus coupling impact in nonidentical bidirectionally coupled dynamics

The understanding of interacting dynamics is important for the characterization of real-world networks. In general, real-world networks are heterogeneous in the sense that each node of the network is a dynamics with different properties. For coupled nonidentical dynamics symmetric interactions are not straightforwardly defined from the coupling strength values. Thus, a challenging issue is whether we can define a symmetric interaction in this asymmetric setting. To address this problem we introduce the notion of the coupling impact. The coupling impact considers not only the coupling strength but also the energy of the individual dynamics, which is conveyed via the coupling. To illustrate this concept, we follow a data-driven approach by analyzing signals from pairs of coupled model dynamics using two different connectivity measures. We find that the coupling impact, but not the coupling strength, correctly detects a symmetric interaction between pairs of coupled dynamics regardless of their degree of asymmetry. Therefore, this approach allows us to reveal the real impact that one dynamics has on the other and hence to define symmetric interactions in pairs of nonidentical dynamics.

Measuring social interaction in music ensembles

Music ensembles are an ideal test-bed for quantitative analysis of social interaction. Music is an inherently social activity, and music ensembles offer a broad variety of scenarios which are particularly suitable for investigation. Small ensembles, such as string quartets, are deemed a significant example of self-managed teams, where all musicians contribute equally to a task. In bigger ensembles, such as orchestras, the relationship between a leader (the conductor) and a group of followers (the musicians) clearly emerges. This paper presents an overview of recent research on social interaction in music ensembles with a particular focus on (i) studies from cognitive neuroscience; and (ii) studies adopting a computational approach for carrying out automatic quantitative analysis of ensemble music performances.

Testing for causality in reconstructed state spaces by an optimized mixed prediction method

In this study, a method of causality detection was designed to reveal coupling between dynamical systems represented by time series. The method is based on the predictions in reconstructed state spaces. The results of the proposed method were compared with outcomes of two other methods, the Granger VAR test of causality and the convergent cross-mapping. We used two types of test data. The first test example is a unidirectional connection of chaotic systems of Rössler and Lorenz type. The second one, the fishery model, is an example of two correlated observables without a causal relationship. The results showed that the proposed method of optimized mixed prediction was able to reveal the presence and the direction of coupling and distinguish causality from mere correlation as well.

Discrimination of coupling structures using causality networks from multivariate time series

Measures of Granger causality on multivariate time series have been used to form the so-called causality networks. A causality network represents the interdependence structure of the underlying dynamical system or coupled dynamical systems, and its properties are quantified by network indices. In this work, it is investigated whether network indices on networks generated by an appropriate Granger causality measure can discriminate different coupling structures. The information based Granger causality measure of partial mutual information from mixed embedding (PMIME) is used to form causality networks, and a large number of network indices are ranked according to their ability to discriminate the different coupling structures. The evaluation of the network indices is done with a simulation study based on two dynamical systems, the coupled Mackey-Glass delay differential equations and the neural mass model, both of 25 variables, and three prototypes of coupling structures, i.e., random, small-world, and scale-free. It is concluded that the setting of PMIME combined with a network index attains high level of discrimination of the coupling structures solely on the basis of the observed multivariate time series. This approach is demonstrated to identify epileptic seizures emerging during electroencephalogram recordings.

Epileptogenic zone localization and seizure control in coupled neural mass models

Exact localization of the epileptogenic zone (EZ) is the first priority for ensuring epilepsy treatments and reducing side effects. The results of traditional visual methods for localizing the origin of seizures are far from satisfactory in some cases. Signal processing methods could extract substantial information that may complement visual inspection of EEG signals. In this study, EZ localization is changed into a driver identification problem, and a nonlinear interdependence measure, the weighted rank interdependence, is proposed and used as a driver indicator because it can detect coupling information, especially directionality, from EEG signals. A proportional integral derivative (PID) controller is then explored, using simulations, to establish its suitability for seizure control. The seizure control we propose rests on identifying the EZ using nonlinear interdependence measures of directed functional connectivity. Two directionally coupled neural mass models are employed for simulation investigation. Two parameters can adjust the sensitivity and completeness of the weighted rank interdependence for different applications, and their effect is discussed in the context of neural mass models. Simulation results demonstrate that use of the weighted rank interdependence for EZ identification can be applied to different EZ types, and the approach achieves an overall identification rate of 98.84 % for several EZ types. Simulations also indicate that PID control can effectively regulate synchronization between neural masses.

Remote synchronization of amplitudes across an experimental ring of non-linear oscillators

In this paper, the emergence of remote synchronization in a ring of 32 unidirectionally coupled non-linear oscillators is reported. Each oscillator consists of 3 negative voltage gain stages connected in a loop to which two integrators are superimposed and receives input from its preceding neighbour via a "mixing" stage whose gains form the main system control parameters. Collective behaviour of the network is investigated numerically and experimentally, based on a custom-designed circuit board featuring 32 field-programmable analog arrays. A diverse set of synchronization patterns is observed depending on the control parameters. While phase synchronization ensues globally, albeit imperfectly, for certain control parameter values, amplitudes delineate subsets of non-adjacent but preferentially synchronized nodes; this cannot be trivially explained by synchronization paths along sequences of structurally connected nodes and is therefore interpreted as representing a form of remote synchronization. Complex topology of functional synchronization thus emerges from underlying elementary structural connectivity. In addition to the Kuramoto order parameter and cross-correlation coefficient, other synchronization measures are considered, and preliminary findings suggest that generalized synchronization may identify functional relationships across nodes otherwise not visible. Further work elucidating the mechanism underlying this observation of remote synchronization is necessary, to support which experimental data and board design materials have been made freely downloadable.

Electroencephalography signatures of attention-deficit/hyperactivity disorder: clinical utility

The techniques and the most important results on the use of electroencephalography (EEG) to extract different measures are reviewed in this work, which can be clinically useful to study subjects with attention-deficit/hyperactivity disorder (ADHD). First, we discuss briefly and in simple terms the EEG analysis and processing techniques most used in the context of ADHD. We review techniques that both analyze individual EEG channels (univariate measures) and study the statistical interdependence between different EEG channels (multivariate measures), the so-called functional brain connectivity. Among the former ones, we review the classical indices of absolute and relative spectral power and estimations of the complexity of the channels, such as the approximate entropy and the Lempel-Ziv complexity. Among the latter ones, we focus on the magnitude square coherence and on different measures based on the concept of generalized synchronization and its estimation in the state space. Second, from a historical point of view, we present the most important results achieved with these techniques and their clinical utility (sensitivity, specificity, and accuracy) to diagnose ADHD. Finally, we propose future research lines based on these results.

Sleep electroencephalography and heart rate variability interdependence amongst healthy subjects and insomnia/schizophrenia patients

The quantification of interdependencies within autonomic nervous system has gained increasing importance to characterise healthy and psychiatric disordered subjects. The present work introduces a biosignal processing approach, suggesting a computational resource to estimate coherent or synchronised interactions as an eventual supportive aid in the diagnosis of primary insomnia and schizophrenia pathologies. By deploying linear, nonlinear and statistical methods upon 25 electroencephalographic and electrocardiographic overnight sleep recordings, the assessment of cross-correlation, wavelet coherence and [Formula: see text]:[Formula: see text] phase synchronisation is focused on tracking discerning features amongst the clinical cohorts. Our results indicate that certain neuronal oscillations interact with cardiac power bands in distinctive ways responding to standardised sleep stages and patient groups, which promotes the hypothesis of subtle functional dynamics between neuronal assembles and (para)sympathetic activity subject to pathophysiological conditions.

Detecting causality from nonlinear dynamics with short-term time series

Quantifying causality between variables from observed time series data is of great importance in various disciplines but also a challenging task, especially when the observed data are short. Unlike the conventional methods, we find it possible to detect causality only with very short time series data, based on embedding theory of an attractor for nonlinear dynamics. Specifically, we first show that measuring the smoothness of a cross map between two observed variables can be used to detect a causal relation. Then, we provide a very effective algorithm to computationally evaluate the smoothness of the cross map, or "Cross Map Smoothness" (CMS), and thus to infer the causality, which can achieve high accuracy even with very short time series data. Analysis of both mathematical models from various benchmarks and real data from biological systems validates our method.

Detecting determinism from point processes

The detection of a nonrandom structure from experimental data can be crucial for the classification, understanding, and interpretation of the generating process. We here introduce a rank-based nonlinear predictability score to detect determinism from point process data. Thanks to its modular nature, this approach can be adapted to whatever signature in the data one considers indicative of deterministic structure. After validating our approach using point process signals from deterministic and stochastic model dynamics, we show an application to neuronal spike trains recorded in the brain of an epilepsy patient. While we illustrate our approach in the context of temporal point processes, it can be readily applied to spatial point processes as well.

Identifying delayed directional couplings with symbolic transfer entropy

We propose a straightforward extension of symbolic transfer entropy to enable the investigation of delayed directional relationships between coupled dynamical systems from time series. Analyzing time series from chaotic model systems, we demonstrate the applicability and limitations of our approach. Our findings obtained from applying our method to infer delayed directed interactions in the human epileptic brain underline the importance of our approach for improving the construction of functional network structures from data.

Effect of measurement noise on Granger causality

Most of the signals recorded in experiments are inevitably contaminated by measurement noise. Hence, it is important to understand the effect of such noise on estimating causal relations between such signals. A primary tool for estimating causality is Granger causality. Granger causality can be computed by modeling the signal using a bivariate autoregressive (AR) process. In this paper, we greatly extend the previous analysis of the effect of noise by considering a bivariate AR process of general order p. From this analysis, we analytically obtain the dependence of Granger causality on various noise-dependent system parameters. In particular, we show that measurement noise can lead to spurious Granger causality and can suppress true Granger causality. These results are verified numerically. Finally, we show how true causality can be recovered numerically using the Kalman expectation maximization algorithm.