Estimation of the direction of the coupling by conditional probabilities of recurrence

Auxiliary-cavity-enhanced quantum estimation of optorotational-coupling strength

A scheme is proposed to achieve significantly enhanced quantum estimation of optorotational-coupling (ORC) strength by coupling a driven auxiliary cavity to a Laguerre-Gaussian (L-G) rotational cavity, where the ORC originates from the exchange of orbital angular momentum between a L-G light and rotational mirror. The results indicate that, by appropriately designing the auxiliary-cavity mechanism, the estimation error of the ORC parameter is significantly reduced, and revealing the estimation precision has a much stronger thermal noise and dissipation robustness in comparison with the unassisted case. Our study paves the way toward achieving high-precision quantum sensors.

The causality measure of partial mutual information from mixed embedding (PMIME) revisited

The measure of partial mutual information from mixed embedding (PMIME) is an information theory-based measure to accurately identify the direct and directional coupling, termed Granger causality or simply causality, between the observed variables or subsystems of a high-dimensional dynamical and complex system, without any a priori assumptions about the nature of the coupling relationship. In its core, it is a forward selection procedure that aims to iteratively identify the lag-dependence structure of a given observed variable (response) to all the other observed variables (candidate drivers). This model-free approach is capable of detecting nonlinear interactions, abundantly present in real-world complex systems, and it was shown to perform well on multivariate time series of moderately high dimension. However, the PMIME presents some inefficiencies in its performance mainly when applied on strongly stochastic (linear or nonlinear) systems as it may falsely detect non-existent relationships. Moreover, and by construction, the measure cannot extract purely synergetic relationships present in a system. In the current work, the issue of false detections is addressed by introducing an improved resampling significance test and a procedure of rechecking the identified drivers (backward revision). Regarding the inability to detect synergetic relationships, the PMIME is further enhanced by checking pairs as candidate drivers for the response variable after having considered all drivers individually. The effects of these modifications are investigated in a systematic simulation study on properly designed systems involving strong stochasticity, regressor terms with synergetic effects, and a system dimension ranging from 3 to 30. The overall results of the simulations indicate that these modifications indeed improve the performance of PMIME and alleviate to a significant degree the issues of the original algorithm. Guidelines for balancing between accuracy and computational efficiency are also given, particularly relevant for real-world applications. Finally, the measure performance is investigated in the study of futures of various government bonds and stock market indices in the period around COVID-19 pandemic.

Information transfers and flows in Markov chains as dynamical causal effects

A logical sequence of information-theoretic quantifiers of directional (causal) couplings in Markov chains is generated within the framework of dynamical causal effects (DCEs), starting from the simplest DCEs (in terms of localization of their functional elements) and proceeding step-by-step to more complex ones. Thereby, a system of 11 quantifiers is readily obtained, some of them coinciding with previously known causality measures widely used in time series analysis and often called "information transfers" or "flows" (transfer entropy, Ay-Polani information flow, Liang-Kleeman information flow, information response, etc.,) By construction, this step-by-step generation reveals logical relationships between all these quantifiers as specific DCEs. As a further concretization, diverse quantitative relationships between the transfer entropy and the Liang-Kleeman information flow are found both rigorously and numerically for coupled two-state Markov chains.

Improving the Process of Early-Warning Detection and Identifying the Most Affected Markets: Evidence from Subprime Mortgage Crisis and COVID-19 Outbreak-Application to American Stock Markets

Stock-market-crash predictability is of particular interest in the field of financial time-series analysis. Famous examples of major stock-market crashes are the real-estate bubble in 2008 and COVID-19 in 2020. Several studies have studied the prediction process without taking into consideration which markets might be falling into a crisis. To this end, a combination analysis is utilized in this manuscript. Firstly, the auto-regressive estimation (ARE) algorithm is successfully applied to electroencephalography (EEG) brain data for detecting diseases. The ARE algorithm is employed based on state-space modelling, which applies the expectation-maximization algorithm and Kalman filter. This manuscript introduces its application, for the first time, to stock-market data. For this purpose, a time-evolving interaction surface is constructed to observe the change in the surface topology. This enables tracking of the stock market's behavior over time and differentiates between different states. This provides a deep understanding of the underlying system behavior before, during, and after a crisis. Different patterns of the stock-market movements are recognized, providing novel information regarding detecting an early-warning sign. Secondly, a Granger-causality time-domain technique, called directed partial correlation, is employed to infer the underlying interconnectivity structure among markets. This information is crucial for investors and market players, enabling them to differentiate between those markets which will fall in a catastrophic loss, and those which will not. Consequently, they can make successful decisions towards selecting less risky portfolios, which guarantees lower losses. The results showed the effectiveness of the use of this methodology in the framework of the process of early-warning detection.

Causal Inference in Time Series in Terms of Rényi Transfer Entropy

Uncovering causal interdependencies from observational data is one of the great challenges of a nonlinear time series analysis. In this paper, we discuss this topic with the help of an information-theoretic concept known as Rényi’s information measure. In particular, we tackle the directional information flow between bivariate time series in terms of Rényi’s transfer entropy. We show that by choosing Rényi’s parameter α, we can appropriately control information that is transferred only between selected parts of the underlying distributions. This, in turn, is a particularly potent tool for quantifying causal interdependencies in time series, where the knowledge of “black swan” events, such as spikes or sudden jumps, are of key importance. In this connection, we first prove that for Gaussian variables, Granger causality and Rényi transfer entropy are entirely equivalent. Moreover, we also partially extend these results to heavy-tailed α-Gaussian variables. These results allow establishing a connection between autoregressive and Rényi entropy-based information-theoretic approaches to data-driven causal inference. To aid our intuition, we employed the Leonenko et al. entropy estimator and analyzed Rényi’s information flow between bivariate time series generated from two unidirectionally coupled Rössler systems. Notably, we find that Rényi’s transfer entropy not only allows us to detect a threshold of synchronization but it also provides non-trivial insight into the structure of a transient regime that exists between the region of chaotic correlations and synchronization threshold. In addition, from Rényi’s transfer entropy, we could reliably infer the direction of coupling and, hence, causality, only for coupling strengths smaller than the onset value of the transient regime, i.e., when two Rössler systems are coupled but have not yet entered synchronization.

Contributions to surface air temperature trends estimated from climate time series: Medium-term causalities

Contributions of various natural and anthropogenic factors to trends of surface air temperatures at different latitudes of the Northern and Southern hemispheres on various temporal horizons are estimated from climate data since the 19th century in empirical autoregressive models. Along with anthropogenic forcing, we assess the impact of several natural climate modes including Atlantic Multidecadal Oscillation, El-Nino/Southern Oscillation, Interdecadal Pacific Oscillation, Pacific Decadal Oscillation, and Antarctic Oscillation. On relatively short intervals of the length of two or three decades, contributions of climate variability modes are considerable and comparable to the contributions of greenhouse gases and even exceed the latter. On longer intervals of about half a century and greater, the contributions of greenhouse gases dominate at all latitudinal belts including polar, middle, and tropical ones.

A complex network approach to study the extreme precipitation patterns in a river basin

The quantification of spatial propagation of extreme precipitation events is vital in water resources planning and disaster mitigation. However, quantifying these extreme events has always been challenging as many traditional methods are insufficient to capture the nonlinear interrelationships between extreme event time series. Therefore, it is crucial to develop suitable methods for analyzing the dynamics of extreme events over a river basin with a diverse climate and complicated topography. Over the last decade, complex network analysis emerged as a powerful tool to study the intricate spatiotemporal relationship between many variables in a compact way. In this study, we employ two nonlinear concepts of event synchronization and edit distance to investigate the extreme precipitation pattern in the Ganga river basin. We use the network degree to understand the spatial synchronization pattern of extreme rainfall and identify essential sites in the river basin with respect to potential prediction skills. The study also attempts to quantify the influence of precipitation seasonality and topography on extreme events. The findings of the study reveal that (1) the network degree is decreased in the southwest to northwest direction, (2) the timing of 50th percentile precipitation within a year influences the spatial distribution of degree, (3) the timing is inversely related to elevation, and (4) the lower elevation greatly influences connectivity of the sites. The study highlights that edit distance could be a promising alternative to analyze event-like data by incorporating event time and amplitude and constructing complex networks of climate extremes.

Granger-Causality Inference of the Existence of Unobserved Important Components in Network Analysis

Detecting causal interrelationships in multivariate systems, in terms of the Granger-causality concept, is of major interest for applications in many fields. Analyzing all the relevant components of a system is almost impossible, which contrasts with the concept of Granger causality. Not observing some components might, in turn, lead to misleading results, particularly if the missing components are the most influential and important in the system under investigation. In networks, the importance of a node depends on the number of nodes connected to this node. The degree of centrality is the most commonly used measure to identify important nodes in networks. There are two kinds of degree centrality, which are in-degree and out-degree. This manuscrpt is concerned with finding the highest out-degree among nodes to identify the most influential nodes. Inferring the existence of unobserved important components is critical in many multivariate interacting systems. The implications of such a situation are discussed in the Granger-causality framework. To this end, two of the most recent Granger-causality techniques, renormalized partial directed coherence and directed partial correlation, were employed. They were then compared in terms of their performance according to the extent to which they can infer the existence of unobserved important components. Sub-network analysis was conducted to aid these two techniques in inferring the existence of unobserved important components, which is evidenced in the results. By comparing the results of the two conducted techniques, it can be asserted that renormalized partial coherence outperforms directed partial correlation in the inference of existing unobserved important components that have not been included in the analysis. This measure of Granger causality and sub-network analysis emphasizes their ubiquitous successful applicability in such cases of the existence of hidden unobserved important components.

Transfer entropies within dynamical effects framework

Transfer entropy (TE) is widely used in time-series analysis to detect causal couplings between temporally evolving objects. As a coupling strength quantifier, the TE alone often seems insufficient, raising the question of its further interpretations. Here the TE is related to dynamical causal effects (DCEs) which quantify long-term responses of a coupling recipient to variations in a coupling source or in a coupling itself: Detailed relationships are established for a paradigmatic stochastic dynamical system of bidirectionally coupled linear overdamped oscillators, their practical applications and possible extensions are discussed. It is shown that two widely used versions of the TE (original and infinite-history) can become qualitatively distinct, diverging to different long-term DCEs.

Youth With Concussion Have Less Adaptable Gait Patterns Than Their Uninjured Peers: Implications for Concussion Management

OBJECTIVE

To compare cross-recurrence quantification analysis measurements obtained during gait between adolescents who sustained a diagnosed concussion within 14 days of assessment and healthy adolescents.

DESIGN

Cross-sectional study.

METHODS

Youth athletes with concussion (n = 43; mean ± SD age, 14.4 ± 2.3 years; 56% female; tested median, 7 days post concussion) and healthy controls (n = 38; age, 14.9 ± 2.0 years; 55% female) completed a single-task and dual-task gait protocol while wearing a set of inertial sensors. We used cross-recurrence quantification analysis techniques to quantify the similarity between accelerations obtained from the sensor on the dorsum of each foot. Four outcome variables were compared between groups: percent determinism, average diagonal-line length, laminarity, and trapping time.

RESULTS

Athletes with concussion had significantly higher percent determinism, laminarity, and trapping time than the control group in single-task and dual-task conditions (P<.05). Gait patterns, when simultaneously completing a secondary cognitive task (dual task), were no different from gait patterns under a single-task condition.

CONCLUSION

Higher percent determinism, laminarity, and trapping time among athletes with concussion suggest that concussion may be associated with a more stuck and predictable gait pattern. These altered movement patterns may be one reason for underlying slower gait speeds that have been observed following concussion. J Orthop Sports Phys Ther 2020;50(8):438-446. Epub 22 May 2020. doi:10.2519/jospt.2020.9133.

A practical method for estimating coupling functions in complex dynamical systems

A foremost challenge in modern network science is the inverse problem of reconstruction (inference) of coupling equations and network topology from the measurements of the network dynamics. Of particular interest are the methods that can operate on real (empirical) data without interfering with the system. One such earlier attempt (Tokuda et al. 2007 Phys. Rev. Lett. 99, 064101. (doi:10.1103/PhysRevLett.99.064101)) was a method suited for general limit-cycle oscillators, yielding both oscillators' natural frequencies and coupling functions between them (phase equations) from empirically measured time series. The present paper reviews the above method in a way comprehensive to domain-scientists other than physics. It also presents applications of the method to (i) detection of the network connectivity, (ii) inference of the phase sensitivity function, (iii) approximation of the interaction among phase-coherent chaotic oscillators, and (iv) experimental data from a forced Van der Pol electric circuit. This reaffirms the range of applicability of the method for reconstructing coupling functions and makes it accessible to a much wider scientific community. This article is part of the theme issue 'Coupling functions: dynamical interaction mechanisms in the physical, biological and social sciences'.

A Brief Introduction to Nonlinear Time Series Analysis and Recurrence Plots

Nonlinear time series analysis gained prominence from the late 1980s on, primarily because of its ability to characterize, analyze, and predict nontrivial features in data sets that stem from a wide range of fields such as finance, music, human physiology, cognitive science, astrophysics, climate, and engineering. More recently, recurrence plots, initially proposed as a visual tool for the analysis of complex systems, have proven to be a powerful framework to quantify and reveal nontrivial dynamical features in time series data. This tutorial review provides a brief introduction to the fundamentals of nonlinear time series analysis, before discussing in greater detail a few (out of the many existing) approaches of recurrence plot-based analysis of time series. In particular, it focusses on recurrence plot-based measures which characterize dynamical features such as determinism, synchronization, and regime changes. The concept of surrogate-based hypothesis testing, which is crucial to drawing any inference from data analyses, is also discussed. Finally, the presented recurrence plot approaches are applied to two climatic indices related to the equatorial and North Pacific regions, and their dynamical behavior and their interrelations are investigated.

Evaluation of Granger Causality Measures for Constructing Networks from Multivariate Time Series

Granger causality and variants of this concept allow the study of complex dynamical systems as networks constructed from multivariate time series. In this work, a large number of Granger causality measures used to form causality networks from multivariate time series are assessed. These measures are in the time domain, such as model-based and information measures, the frequency domain, and the phase domain. The study aims also to compare bivariate and multivariate measures, linear and nonlinear measures, as well as the use of dimension reduction in linear model-based measures and information measures. The latter is particular relevant in the study of high-dimensional time series. For the performance of the multivariate causality measures, low and high dimensional coupled dynamical systems are considered in discrete and continuous time, as well as deterministic and stochastic. The measures are evaluated and ranked according to their ability to provide causality networks that match the original coupling structure. The simulation study concludes that the Granger causality measures using dimension reduction are superior and should be preferred particularly in studies involving many observed variables, such as multi-channel electroencephalograms and financial markets.

Application of symbolic recurrence to experimental data, from firearm prevalence to fish swimming

Recurrence plots and recurrence quantification analysis are powerful tools to study the behavior of dynamical systems. What we learn through these tools is typically determined by the choice of a distance threshold in the phase space, which introduces arbitrariness in the definition of recurrence. Not only does symbolic recurrence overcome this difficulty, but also it offers a richer representation that book-keeps the recurrent portions of the phase space. Using symbolic recurrences, we can construct recurrence plots, perform quantification analysis, and examine causal links between dynamical systems from their time-series. Although previous efforts have demonstrated the feasibility of such a symbolic framework on synthetic data, the study of real time-series remains elusive. Here, we seek to bridge this gap by systematically examining a wide range of experimental datasets, from firearm prevalence and media coverage in the United States to the effect of sex on the interaction of swimming fish. This work offers a compelling demonstration of the potential of symbolic recurrence in the study of real-world applications across different research fields while providing a computer code for researchers to perform their own time-series explorations.

Causality Detection Methods Applied to the Investigation of Malaria Epidemics

Malaria, a disease with major health and socio-economic impacts, is driven by multiple factors, including a complex interaction with various climatic variables. In this paper, five methods developed for inferring causal relations between dynamic processes based on the information encapsulated in time series are applied on cases previously studied in literature by means of statistical methods. The causality detection techniques investigated in the paper are: a version of the kernel Granger causality, transfer entropy, recurrence plot, causal decomposition and complex networks. The methods provide coherent results giving a quite good confidence in the conclusions.

Transfer entropy on symbolic recurrences

Recurrence quantification analysis offers a powerful framework to investigate complexity in dynamical systems. While several studies have demonstrated the possibility of multivariate recurrence quantification analysis, information-theoretic tools for the discovery of causal links remain elusive. Particularly enticing is to formulate information-theoretic tools on symbolic recurrence plots, which alleviate some of the methodological challenges of traditional recurrence plots and offer a richer representation of recurrences. Toward this aim, we establish a probability space in which we ground a theory of information that encodes information in the recurrences of the symbols. We introduce transfer entropy on symbolic recurrences as a tool to guide the inference of the strength and direction of the interaction between dynamical systems. We demonstrate statistically reliable discovery of causal links on synthetic and experimental time series, from only two time series or a larger dataset with multiple realizations. The proposed approach brings together recurrence plots, information theory, and symbolic dynamics to empower researchers and practitioners with effective means to visualize and quantify interactions in dynamical systems.

Ordinal partition transition network based complexity measures for inferring coupling direction and delay from time series

It has been demonstrated that the construction of ordinal partition transition networks (OPTNs) from time series provides a prospective approach to improve our understanding of the underlying dynamical system. In this work, we introduce a suite of OPTN based complexity measures to infer the coupling direction between two dynamical systems from pairs of time series. For several examples of coupled stochastic processes, we demonstrate that our approach is able to successfully identify interaction delays of both unidirectional and bidirectional coupling configurations. Moreover, we show that the causal interaction between two coupled chaotic Hénon maps can be captured by the OPTN based complexity measures for a broad range of coupling strengths before the onset of synchronization. Finally, we apply our method to two real-world observational climate time series, disclosing the interaction delays underlying the temperature records from two distinct stations in Oxford and Vienna. Our results suggest that ordinal partition transition networks can be used as complementary tools for causal inference tasks and provide insights into the potentials and theoretical foundations of time series networks.

Recurrence-based analysis of barrier breakup in the standard nontwist map

We study the standard nontwist map that describes the dynamic behaviour of magnetic field lines near a local minimum or maximum of frequency. The standard nontwist map has a shearless invariant curve that acts like a barrier in phase space. Critical parameters for the breakup of the shearless curve have been determined by procedures based on the indicator points and bifurcations of periodical orbits, a methodology that demands high computational cost. To determine the breakup critical parameters, we propose a new simpler and general procedure based on the determinism analysis performed on the recurrence plot of orbits near the critical transition. We also show that the coexistence of islands and chaotic sea in phase space can be analysed by using the recurrence plot. In particular, the measurement of determinism from the recurrence plot provides us with a simple procedure to distinguish periodic from chaotic structures in the parameter space. We identify an invariant shearless breakup scenario, and we also show that recurrence plots are useful tools to determine the presence of periodic orbit collisions and bifurcation curves.

Detecting directional couplings from multivariate flows by the joint distance distribution

The identification of directional couplings (or drive-response relationships) in the analysis of interacting nonlinear systems is an important piece of information to understand their dynamics. This task is especially challenging when the analyst's knowledge of the systems reduces virtually to time series of observations. Spurred by the success of Granger causality in econometrics, the study of cause-effect relationships (not to be confounded with statistical correlations) was extended to other fields, thus favoring the introduction of further tools such as transfer entropy. Currently, the research on old and new causality tools along with their pitfalls and applications in ever more general situations is going through a time of much activity. In this paper, we re-examine the method of the joint distance distribution to detect directional couplings between two multivariate flows. This method is based on the forced Takens theorem, and, more specifically, it exploits the existence of a continuous mapping from the reconstructed attractor of the response system to the reconstructed attractor of the driving system, an approach that is increasingly drawing the attention of the data analysts. The numerical results with Lorenz and Rössler oscillators in three different interaction networks (including hidden common drivers) are quite satisfactory, except when phase synchronization sets in. They also show that the method of the joint distance distribution outperforms the lowest dimensional transfer entropy in the cases considered. The robustness of the results to the sampling interval, time series length, observational noise, and metric is analyzed too.

Transient and equilibrium causal effects in coupled oscillators

Two quite different types of causal effects are given by (i) changes in near future states of a driven system under changes in a current state of a driving system and (ii) changes in statistical characteristics of a driven system dynamics under changes in coupling parameters, e.g., under switching the coupling off. The former can be called transient causal effects and can be estimated from a time series within the well established framework of the Wiener-Granger causality, while the latter represent equilibrium (or stationary) causal effects which are often most interesting but generally inaccessible to estimation from an observed time series recorded at fixed coupling parameters. In this work, relationships between the two kinds of causal effects are found for unidirectionally coupled stochastic linear oscillators depending on their frequencies and damping factors. Approximate closed-form expressions for these relationships are derived. Their limitations and possible extensions are discussed, and their practical applicability to extracting equilibrium causal effects from time series is argued.

Coupling relations underlying the production of speech articulator movements and their invariance to speech rate

Since the seminal works of Bernstein (The coordination and regulation of movements. Pergamon Press, Oxford, 1967) several authors have supported the idea that, to produce a goal-oriented movement in general, and a movement of the organs responsible for the production of speech sounds in particular, individuals activate a set of coupling relations that coordinate the behavior of the elements of the motor system involved in the production of the target movement or sound. In order to characterize the configurations of the coupling relations underlying speech production articulator movements, we introduce an original method based on recurrence analysis. The method is validated through the analysis of simulated dynamical systems adapted to reproduce the features of speech gesture kinematics and it is applied to the analysis of speech articulator movements recorded in five German speakers during the production of labial and coronal plosive and fricative consonants at variable speech rates. We were able to show that the underlying coupling relations change systematically between labial and coronal consonants, but are not affected by speech rate, despite the presence of qualitative changes observed in the trajectory of the jaw at fast speech rate.

Assessing Catastrophes-Dragon-Kings, Black, and Gray Swans-for Science-Policy

The threat of catastrophic incidents-from nonroutine events to extreme ones, such as Dragon-Kings (DK), Black Swans (BS), and Gray Swans-induces precautionary initiatives that, before the fact, may encounter public resistance or after the fact recriminations. This study develops three aspects of these events: (1) generating mechanisms, (2) the statistical distributions of near and far-term consequences, and (3) the aggregation of expert opinions about assumptions, mechanisms, and consequences that informs science-policy. This study shows how causal analysis should account for the: (1) nonlinear catastrophic behaviors that generate predictions, (2) common and power-law distributions of the consequences, (3) self-organizing criticality and self-similarity, and (4) feedbacks and couplings between mechanisms that produce snaps, crackles, and pops as precursor, warning signals. The distribution of the consequences associated with catastrophic incidents has longer and fatter right tails than those expected from failure analysis based on known nonroutine events. DK are extreme events that deviate from these fat tail distributions, have a much higher frequency than expected, and can be predicted unlike BS. This shows how to combine divergent expert individual beliefs over assumptions, causation, and results, and a paradox that affects agreements obtained by majority rule.

Recurrence measure of conditional dependence and applications

Identifying causal relations from observational data sets has posed great challenges in data-driven causality inference studies. One of the successful approaches to detect direct coupling in the information theory framework is transfer entropy. However, the core of entropy-based tools lies on the probability estimation of the underlying variables. Here we propose a data-driven approach for causality inference that incorporates recurrence plot features into the framework of information theory. We define it as the recurrence measure of conditional dependence (RMCD), and we present some applications. The RMCD quantifies the causal dependence between two processes based on joint recurrence patterns between the past of the possible driver and present of the potentially driven, excepting the contribution of the contemporaneous past of the driven variable. Finally, it can unveil the time scale of the influence of the sea-surface temperature of the Pacific Ocean on the precipitation in the Amazonia during recent major droughts.

Computing algebraic transfer entropy and coupling directions via transcripts

Most random processes studied in nonlinear time series analysis take values on sets endowed with a group structure, e.g., the real and rational numbers, and the integers. This fact allows to associate with each pair of group elements a third element, called their transcript, which is defined as the product of the second element in the pair times the first one. The transfer entropy of two such processes is called algebraic transfer entropy. It measures the information transferred between two coupled processes whose values belong to a group. In this paper, we show that, subject to one constraint, the algebraic transfer entropy matches the (in general, conditional) mutual information of certain transcripts with one variable less. This property has interesting practical applications, especially to the analysis of short time series. We also derive weak conditions for the 3-dimensional algebraic transfer entropy to yield the same coupling direction as the corresponding mutual information of transcripts. A related issue concerns the use of mutual information of transcripts to determine coupling directions in cases where the conditions just mentioned are not fulfilled. We checked the latter possibility in the lowest dimensional case with numerical simulations and cardiovascular data, and obtained positive results.

Discrimination of coupling structures using causality networks from multivariate time series

Measures of Granger causality on multivariate time series have been used to form the so-called causality networks. A causality network represents the interdependence structure of the underlying dynamical system or coupled dynamical systems, and its properties are quantified by network indices. In this work, it is investigated whether network indices on networks generated by an appropriate Granger causality measure can discriminate different coupling structures. The information based Granger causality measure of partial mutual information from mixed embedding (PMIME) is used to form causality networks, and a large number of network indices are ranked according to their ability to discriminate the different coupling structures. The evaluation of the network indices is done with a simulation study based on two dynamical systems, the coupled Mackey-Glass delay differential equations and the neural mass model, both of 25 variables, and three prototypes of coupling structures, i.e., random, small-world, and scale-free. It is concluded that the setting of PMIME combined with a network index attains high level of discrimination of the coupling structures solely on the basis of the observed multivariate time series. This approach is demonstrated to identify epileptic seizures emerging during electroencephalogram recordings.

Detecting Causality by Combined Use of Multiple Methods: Climate and Brain Examples

Identifying causal relations from time series is the first step to understanding the behavior of complex systems. Although many methods have been proposed, few papers have applied multiple methods together to detect causal relations based on time series generated from coupled nonlinear systems with some unobserved parts. Here we propose the combined use of three methods and a majority vote to infer causality under such circumstances. Two of these methods are proposed here for the first time, and all of the three methods can be applied even if the underlying dynamics is nonlinear and there are hidden common causes. We test our methods with coupled logistic maps, coupled Rössler models, and coupled Lorenz models. In addition, we show from ice core data how the causal relations among the temperature, the CH4 level, and the CO2 level in the atmosphere changed in the last 800,000 years, a conclusion also supported by irregularly sampled data analysis. Moreover, these methods show how three regions of the brain interact with each other during the visually cued, two-choice arm reaching task. Especially, we demonstrate that this is due to bottom up influences at the beginning of the task, while there exist mutual influences between the posterior medial prefrontal cortex and the presupplementary motor area. Based on our results, we conclude that identifying causality with an appropriate ensemble of multiple methods ensures the validity of the obtained results more firmly.

Causality networks from multivariate time series and application to epilepsy

Granger causality and variants of this concept allow the study of complex dynamical systems as networks constructed from multivariate time series. In this work, a large number of Granger causality measures used to form causality networks from multivariate time series are assessed. For this, realizations on high dimensional coupled dynamical systems are considered and the performance of the Granger causality measures is evaluated, seeking for the measures that form networks closest to the true network of the dynamical system. In particular, the comparison focuses on Granger causality measures that reduce the state space dimension when many variables are observed. Further, the linear and nonlinear Granger causality measures of dimension reduction are compared to a standard Granger causality measure on electroencephalographic (EEG) recordings containing episodes of epileptiform discharges.

Edit distance for marked point processes revisited: An implementation by binary integer programming

We implement the edit distance for marked point processes [Suzuki et al., Int. J. Bifurcation Chaos 20, 3699-3708 (2010)] as a binary integer program. Compared with the previous implementation using minimum cost perfect matching, the proposed implementation has two advantages: first, by using the proposed implementation, we can apply a wide variety of software and hardware, even spin glasses and coherent ising machines, to calculate the edit distance for marked point processes; second, the proposed implementation runs faster than the previous implementation when the difference between the numbers of events in two time windows for a marked point process is large.

Unified functional network and nonlinear time series analysis for complex systems science: The pyunicorn package

We introduce the pyunicorn (Pythonic unified complex network and recurrence analysis toolbox) open source software package for applying and combining modern methods of data analysis and modeling from complex network theory and nonlinear time series analysis. pyunicorn is a fully object-oriented and easily parallelizable package written in the language Python. It allows for the construction of functional networks such as climate networks in climatology or functional brain networks in neuroscience representing the structure of statistical interrelationships in large data sets of time series and, subsequently, investigating this structure using advanced methods of complex network theory such as measures and models for spatial networks, networks of interacting networks, node-weighted statistics, or network surrogates. Additionally, pyunicorn provides insights into the nonlinear dynamics of complex systems as recorded in uni- and multivariate time series from a non-traditional perspective by means of recurrence quantification analysis, recurrence networks, visibility graphs, and construction of surrogate time series. The range of possible applications of the library is outlined, drawing on several examples mainly from the field of climatology.

Recovering network topologies via Taylor expansion and compressive sensing

Gaining knowledge of the intrinsic topology of a complex dynamical network is the precondition to understand its evolutionary mechanisms and to control its dynamical and functional behaviors. In this article, a general framework is developed to recover topologies of complex networks with completely unknown node dynamics based on Taylor expansion and compressive sensing. Numerical simulations illustrate the feasibility and effectiveness of the proposed method. Moreover, this method is found to have good robustness to weak stochastic perturbations. Finally, the impact of two major factors on the topology identification performance is evaluated. This method provides a natural and direct point to reconstruct network topologies from measurable data, which is likely to have potential applicability in a wide range of fields.

Estimating coupling directions in the cardiorespiratory system using recurrence properties

The asymmetry of coupling between complex systems can be studied by conditional probabilities of recurrence, which can be estimated by joint recurrence plots. This approach is applied for the first time on experimental data: time series of the human cardiorespiratory system in order to investigate the couplings between heart rate, mean arterial blood pressure and respiration. We find that the respiratory system couples towards the heart rate, and the heart rate towards the mean arterial blood pressure. However, our analysis could not detect a clear coupling direction between the mean arterial blood pressure and respiration.

Inferring topologies of complex networks with hidden variables

Network topology plays a crucial role in determining a network's intrinsic dynamics and function, thus understanding and modeling the topology of a complex network will lead to greater knowledge of its evolutionary mechanisms and to a better understanding of its behaviors. In the past few years, topology identification of complex networks has received increasing interest and wide attention. Many approaches have been developed for this purpose, including synchronization-based identification, information-theoretic methods, and intelligent optimization algorithms. However, inferring interaction patterns from observed dynamical time series is still challenging, especially in the absence of knowledge of nodal dynamics and in the presence of system noise. The purpose of this work is to present a simple and efficient approach to inferring the topologies of such complex networks. The proposed approach is called "piecewise partial Granger causality." It measures the cause-effect connections of nonlinear time series influenced by hidden variables. One commonly used testing network, two regular networks with a few additional links, and small-world networks are used to evaluate the performance and illustrate the influence of network parameters on the proposed approach. Application to experimental data further demonstrates the validity and robustness of our method.

Nonrandomness, nonlinear dependence, and nonstationarity of electroencephalographic recordings from epilepsy patients

To derive tests for randomness, nonlinear-independence, and stationarity, we combine surrogates with a nonlinear prediction error, a nonlinear interdependence measure, and linear variability measures, respectively. We apply these tests to intracranial electroencephalographic recordings (EEG) from patients suffering from pharmacoresistant focal-onset epilepsy. These recordings had been performed prior to and independent from our study as part of the epilepsy diagnostics. The clinical purpose of these recordings was to delineate the brain areas to be surgically removed in each individual patient in order to achieve seizure control. This allowed us to define two distinct sets of signals: One set of signals recorded from brain areas where the first ictal EEG signal changes were detected as judged by expert visual inspection ("focal signals") and one set of signals recorded from brain areas that were not involved at seizure onset ("nonfocal signals"). We find more rejections for both the randomness and the nonlinear-independence test for focal versus nonfocal signals. In contrast more rejections of the stationarity test are found for nonfocal signals. Furthermore, while for nonfocal signals the rejection of the stationarity test increases the rejection probability of the randomness and nonlinear-independence test substantially, we find a much weaker influence for the focal signals. In consequence, the contrast between the focal and nonfocal signals obtained from the randomness and nonlinear-independence test is further enhanced when we exclude signals for which the stationarity test is rejected. To study the dependence between the randomness and nonlinear-independence test we include only focal signals for which the stationarity test is not rejected. We show that the rejection of these two tests correlates across signals. The rejection of either test is, however, neither necessary nor sufficient for the rejection of the other test. Thus, our results suggest that EEG signals from epileptogenic brain areas are less random, more nonlinear-dependent, and more stationary compared to signals recorded from nonepileptogenic brain areas. We provide the data, source code, and detailed results in the public domain.

Estimating network topology by the mean first-passage time

In this work, we employed the concept of the first-passage time in stochastic processes to estimate node degrees and the degree distribution of a network. A statistical exploration of the coupling reveals the relation between the node degree and the coupling term. In practical terms, an effective way to reveal the statistical property is to investigate the differences between coupled oscillators in a network and uncoupled ones with the same initial states. We discovered a monotonically decreasing relation between the node degree and the mean first-passage time (MFPT) for the evolution of the coupled node deviating from the uncoupled one. Moreover, this relation can be understood as the competition of different relaxational time scales. The MFPT method is independent of both the dynamics of the nodes and the topological properties of the network. This might be advantageous in our efforts to build a bridge between the topological property and the dynamics of a network.

Detecting the topologies of complex networks with stochastic perturbations

How to recover the underlying connection topology of a complex network from observed time series of a component variable of each node subject to random perturbations is studied. A new technique termed Piecewise Granger Causality is proposed. The validity of the new approach is illustrated with two FitzHugh-Nagumo neurobiological networks by only observing the membrane potential of each neuron, where the neurons are coupled linearly and nonlinearly, respectively. Comparison with the traditional Granger causality test is performed, and it is found that the new approach outperforms the traditional one. The impact of the network coupling strength and the noise intensity, as well as the data length of each partition of the time series, is further analyzed in detail. Finally, an application to a network composed of coupled chaotic Rössler systems is provided for further validation of the new method.

Assessing directed interactions from neurophysiological signals--an overview

The study of synchronization phenomena in coupled dynamical systems is an active field of research in many scientific disciplines including the neurosciences. Over the last decades, a number of time series analysis techniques have been proposed to capture both linear and nonlinear aspects of interactions. While most of these techniques allow one to quantify the strength of interactions, developments that resulted from advances in nonlinear dynamics and in information and synchronization theory aim at assessing directed interactions. Most of these techniques, however, assume the underlying systems to be at least approximately stationary and require a large number of data points to robustly assess directed interactions. Recent extensions allow assessing directed interactions from short and transient signals and are particularly suited for the analysis of evoked and event-related activity.

Information-based detection of nonlinear Granger causality in multivariate processes via a nonuniform embedding technique

We present an approach, framed in information theory, to assess nonlinear causality between the subsystems of a whole stochastic or deterministic dynamical system. The approach follows a sequential procedure for nonuniform embedding of multivariate time series, whereby embedding vectors are built progressively on the basis of a minimization criterion applied to the entropy of the present state of the system conditioned to its past states. A corrected conditional entropy estimator compensating for the biasing effect of single points in the quantized hyperspace is used to guarantee the existence of a minimum entropy rate at which to terminate the procedure. The causal coupling is detected according to the Granger notion of predictability improvement, and is quantified in terms of information transfer. We apply the approach to simulations of deterministic and stochastic systems, showing its superiority over standard uniform embedding. Effects of quantization, data length, and noise contamination are investigated. As practical applications, we consider the assessment of cardiovascular regulatory mechanisms from the analysis of heart period, arterial pressure, and respiration time series, and the investigation of the information flow across brain areas from multichannel scalp electroencephalographic recordings.

Using bivariate signal analysis to characterize the epileptic focus: the benefit of surrogates

The disease epilepsy is related to hypersynchronous activity of networks of neurons. While acute epileptic seizures are the most extreme manifestation of this hypersynchronous activity, an elevated level of interdependence of neuronal dynamics is thought to persist also during the seizure-free interval. In multichannel recordings from brain areas involved in the epileptic process, this interdependence can be reflected in an increased linear cross correlation but also in signal properties of higher order. Bivariate time series analysis comprises a variety of approaches, each with different degrees of sensitivity and specificity for interdependencies reflected in lower- or higher-order properties of pairs of simultaneously recorded signals. Here we investigate which approach is best suited to detect putatively elevated interdependence levels in signals recorded from brain areas involved in the epileptic process. For this purpose, we use the linear cross correlation that is sensitive to lower-order signatures of interdependence, a nonlinear interdependence measure that integrates both lower- and higher-order properties, and a surrogate-corrected nonlinear interdependence measure that aims to specifically characterize higher-order properties. We analyze intracranial electroencephalographic recordings of the seizure-free interval from 29 patients with an epileptic focus located in the medial temporal lobe. Our results show that all three approaches detect higher levels of interdependence for signals recorded from the brain hemisphere containing the epileptic focus as compared to signals recorded from the opposite hemisphere. For the linear cross correlation, however, these differences are not significant. For the nonlinear interdependence measure, results are significant but only of moderate accuracy with regard to the discriminative power for the focal and nonfocal hemispheres. The highest significance and accuracy is obtained for the surrogate-corrected nonlinear interdependence measure.

Reducing the bias of causality measures

Measures of the direction and strength of the interdependence between two time series are evaluated and modified to reduce the bias in the estimation of the measures, so that they give zero values when there is no causal effect. For this, point shuffling is employed as used in the frame of surrogate data. This correction is not specific to a particular measure and it is implemented here on measures based on state space reconstruction and information measures. The performance of the causality measures and their modifications is evaluated on simulated uncoupled and coupled dynamical systems and for different settings of embedding dimension, time series length, and noise level. The corrected measures, and particularly the suggested corrected transfer entropy, turn out to stabilize at the zero level in the absence of a causal effect and detect correctly the direction of information flow when it is present. The measures are also evaluated on electroencephalograms (EEG) for the detection of the information flow in the brain of an epileptic patient. The performance of the measures on EEG is interpreted in view of the results from the simulation study.

Inferring directional interactions from transient signals with symbolic transfer entropy

We extend the concept of symbolic transfer entropy to enable the time-resolved investigation of directional relationships between coupled dynamical systems from short and transient noisy time series. For our approach, we consider an observed ensemble of a sufficiently large number of time series as multiple realizations of a process. We derive an index that quantifies the preferred direction of transient interactions and assess its significance using a surrogate-based testing scheme. Analyzing time series from noisy chaotic systems, we demonstrate numerically the applicability and limitations of our approach. Our findings obtained from an analysis of event-related brain activities underline the importance of our method to improve understanding of gross neural interactions underlying cognitive processes.

Nonuniform state-space reconstruction and coupling detection

We investigate the state space reconstruction from multiple time series derived from continuous and discrete systems and propose a method for building embedding vectors progressively using information measure criteria regarding past, current, and future states. The embedding scheme can be adapted for different purposes, such as mixed modeling, cross-prediction and Granger causality. In particular, we apply this method in order to detect and evaluate information transfer in coupled systems. As a practical application, we investigate in records of scalp epileptic EEG the information flow across brain areas.