Identifying hidden common causes from bivariate time series: a method using recurrence plots

Learning Robust Predictive Control: A Spatial-Temporal Game Theoretic Approach

This article investigates robust predictive control problem for unknown dynamical systems. Since the dynamics unavailability restricts feasibility of model-driven methods, learning robust predictive control (LRPC) framework is developed from the aspect of time consistency. Under feedback-like control causality, the robust predictive control is then reconstructed as spatial-temporal games, and we guarantee stability through time-consistent Nash equilibrium. For gradation clarity, our framework is specified as four-follow contents. First, multistep feedback-like control causality is drawn from time series analysis, and Takens' theorem provides theoretical support from steady-state property. Second, control problem is reconstructed as games, while performance and robustness partition the game into temporal nonzero-sum subgames and spatial zero-sum ones, respectively. Next, multistep reinforcement learning (RL) is designed to solve robust predictive control without system model. Convergence is proven through bounds analysis of oscillatory value functions, and properties of receding horizon are derived from time consistency. Finally, data-driven implementation is given with function approximation, and neural networks are chosen to approximate value functions and feedback-like causality. Weights are estimated with least squares errors. Numerical results verify the effectiveness.

Data-Efficient Inference of Nonlinear Oscillator Networks

Decoding the connectivity structure of a network of nonlinear oscillators from measurement data is a difficult yet essential task for understanding and controlling network functionality. Several data-driven network inference algorithms have been presented, but the commonly considered premise of ample measurement data is often difficult to satisfy in practice. In this paper, we propose a data-efficient network inference technique by combining correlation statistics with the model-fitting procedure. The proposed approach can identify the network structure reliably in the case of limited measurement data. We compare the proposed method with existing techniques on a network of Stuart-Landau oscillators, oscillators describing circadian gene expression, and noisy experimental data obtained from Rössler Electronic Oscillator network.

Conditional cross-map-based technique: From pairwise dynamical causality to causal network reconstruction

Causality detection methods based on mutual cross mapping have been fruitfully developed and applied to data originating from nonlinear dynamical systems, where the causes and effects are non-separable. However, these pairwise methods still have shortcomings in discriminating typical network structures, including common drivers, indirect dependencies, and facing the curse of dimensionality, when they are stepping to causal network reconstruction. A few endeavors have been devoted to conquer these shortcomings. Here, we propose a novel method that could be regarded as one of these endeavors. Our method, named conditional cross-map-based technique, can eliminate third-party information and successfully detect direct dynamical causality, where the detection results can exactly be categorized into four standard normal forms by the designed criterion. To demonstrate the practical usefulness of our model-free, data-driven method, data generated from different representative models covering all kinds of network motifs and measured from real-world systems are investigated. Because correct identification of the direct causal links is essential to successful modeling, predicting, and controlling the underlying complex systems, our method does shed light on uncovering the inner working mechanisms of real-world systems only using the data experimentally obtained in a variety of disciplines.

Solar activity facilitates daily forecasts of large earthquakes

Despite the extensive literature related to earthquakes, an effective method to forecast and avoid occasional seismic hazards that cause substantial damage is lacking. The Sun has recently been identified as a potential precursor to earthquakes, although no causal relationship between its activity and the Earth's seismicity has been established. This study was aimed at investigating whether such a relationship exists and whether it can be used to improve earthquake forecasting. The edit distances between earthquake point processes were combined with delay-coordinate distances for sunspot numbers. The comparison of these two indicated the existence of unidirectional causal coupling from solar activity to seismicity on Earth, and a radial basis function regressor showed accuracy improvements in the largest magnitude prediction of next days by 2.6%-17.9% in the odds ratio when sunspot distances were included.

Continuity Scaling: A Rigorous Framework for Detecting and Quantifying Causality Accurately

Data-based detection and quantification of causation in complex, nonlinear dynamical systems is of paramount importance to science, engineering, and beyond. Inspired by the widely used methodology in recent years, the cross-map-based techniques, we develop a general framework to advance towards a comprehensive understanding of dynamical causal mechanisms, which is consistent with the natural interpretation of causality. In particular, instead of measuring the smoothness of the cross-map as conventionally implemented, we define causation through measuring the scaling law for the continuity of the investigated dynamical system directly. The uncovered scaling law enables accurate, reliable, and efficient detection of causation and assessment of its strength in general complex dynamical systems, outperforming those existing representative methods. The continuity scaling-based framework is rigorously established and demonstrated using datasets from model complex systems and the real world.

Embedding entropy: a nonlinear measure of dynamical causality

Research on concepts and computational methods of causality has a long history, and there are various concepts of causality as well as corresponding computing algorithms based on measured data. Here, by considering causes and effects from a dynamical perspective, we present a unified mathematical framework for the so-called dynamical causality (DC), which can detect causal interactions over time; notably, this framework covers Granger causality, transfer entropy, embedding causality and their conditional versions. Based on this framework, we further propose a causality criterion called embedding entropy (EE) to measure the DC between two variables. Moreover, its conditional version, conditional embedding entropy (cEE), is also derived for detecting conditional/direct causality. The significant advantages of EE and cEE are that they can be employed for solving not only nonlinear causal inference but also the non-separability problem, and they can reduce the scale bias in numerical calculation. The performance and robustness of EE and cEE were demonstrated through numerical simulations, and causal inference on various real-world datasets validated their effectiveness.

Recurrence-Based Synchronization Analysis of Weakly Coupled Bursting Neurons Under External ELF Fields

We investigate the response characteristics of a two-dimensional neuron model exposed to an externally applied extremely low frequency (ELF) sinusoidal electric field and the synchronization of neurons weakly coupled with gap junction. We find, by numerical simulations, that neurons can exhibit different spiking patterns, which are well observed in the structure of the recurrence plot (RP). We further study the synchronization between weakly coupled neurons in chaotic regimes under the influence of a weak ELF electric field. In general, detecting the phases of chaotic spiky signals is not easy by using standard methods. Recurrence analysis provides a reliable tool for defining phases even for noncoherent regimes or spiky signals. Recurrence-based synchronization analysis reveals that, even in the range of weak coupling, phase synchronization of the coupled neurons occurs and, by adding an ELF electric field, this synchronization increases depending on the amplitude of the externally applied ELF electric field. We further suggest a novel measure for RP-based phase synchronization analysis, which better takes into account the probabilities of recurrences.

Studying the functional connectivity of the primary motor cortex with the binarized cross recurrence plot: The influence of Parkinson's disease

Two new recurrence plot methods (the binary recurrence plot and binary cross recurrence plot) were introduced here to study the long-term dynamic of the primary motor cortex and its interaction with the primary somatosensory cortex, the anterior motor thalamus of the basal ganglia motor loop and the precuneous nucleus of the default mode network. These recurrence plot methods: 1. identify short-term transient interactions; 2. identify long-lasting delayed interactions that are common in complex systems; 3. work with non-stationary blood oxygen level dependent (BOLD) data; 4. may study the relationship of centers with non-linear functional interactions; 5 may compare different experimental groups performing different tasks. These methods were applied to BOLD time-series obtained in 20 control subjects and 20 Parkinson´s patients during the execution of motor activity and body posture tasks (task-block design). The binary recurrence plot showed the task-block BOLD response normally observed in the primary motor cortex with functional magnetic resonance imaging methods, but also shorter and longer BOLD-fluctuations than the task-block and which provided information about the long-term dynamic of this center. The binary cross recurrence plot showed short-lasting and long-lasting functional interactions between the primary motor cortex and the primary somatosensory cortex, anterior motor thalamus and precuneous nucleus, interactions which changed with the resting and motor tasks. Most of the interactions found in healthy controls were disrupted in Parkinson's patients, and may be at the basis of some of the motor disorders and side-effects of dopaminergic drugs commonly observed in these patients.

A Brief Introduction to Nonlinear Time Series Analysis and Recurrence Plots

Nonlinear time series analysis gained prominence from the late 1980s on, primarily because of its ability to characterize, analyze, and predict nontrivial features in data sets that stem from a wide range of fields such as finance, music, human physiology, cognitive science, astrophysics, climate, and engineering. More recently, recurrence plots, initially proposed as a visual tool for the analysis of complex systems, have proven to be a powerful framework to quantify and reveal nontrivial dynamical features in time series data. This tutorial review provides a brief introduction to the fundamentals of nonlinear time series analysis, before discussing in greater detail a few (out of the many existing) approaches of recurrence plot-based analysis of time series. In particular, it focusses on recurrence plot-based measures which characterize dynamical features such as determinism, synchronization, and regime changes. The concept of surrogate-based hypothesis testing, which is crucial to drawing any inference from data analyses, is also discussed. Finally, the presented recurrence plot approaches are applied to two climatic indices related to the equatorial and North Pacific regions, and their dynamical behavior and their interrelations are investigated.

Causality detection in cortical seizure dynamics using cross-dynamical delay differential analysis

Most natural systems, including the brain, are highly nonlinear and complex, and determining information flow among the components that make up these dynamic systems is challenging. One such example is identifying abnormal causal interactions among different brain areas that give rise to epileptic activities. Here, we introduce cross-dynamical delay differential analysis, an extension of delay differential analysis, as a tool to establish causal relationships from time series signals. Our method can infer causality from short time series signals as well as in the presence of noise. Furthermore, we can determine the onset of generalized synchronization directly from time series data, without having to consult the underlying equations. We first validate our method on simulated datasets from coupled dynamical systems and apply the method to intracranial electroencephalography data obtained from epilepsy patients to better characterize large-scale information flow during epilepsy.

Identification of Hidden Sources by Estimating Instantaneous Causality in High-Dimensional Biomedical Time Series

The study of connectivity patterns of a system’s variables, such as multi-channel electroencephalograms (EEG), is of utmost importance towards a better understanding of its internal evolutionary mechanisms. Here, the problem of estimating the connectivity network from multivariate time series in the presence of prominent unobserved variables is addressed. The causality measure of partial mutual information from mixed embedding (PMIME), designed to estimate direct lag-causal effects in the presence of many observed variables, is adapted to estimate also zero-lag effects, the so-called instantaneous causality. We term the proposed advanced method, PMIME0. The estimation of instantaneous causality by PMIME0 is a signature of the presence of hidden source in the observed system, as demonstrated analytically in a toy model. It is further demonstrated that the PMIME0 identifies the true instantaneous with great accuracy in a variety of high-dimensional dynamical systems. The method is applied to EEG data with epileptiform discharges (EDs), and the results imply a strong impact of unobserved confounders during the EDs. This finding comes as a possible explanation for the increased levels of causality during epileptic seizures estimated by some measures affected by the presence of a common source.

Detecting directional couplings from multivariate flows by the joint distance distribution

The identification of directional couplings (or drive-response relationships) in the analysis of interacting nonlinear systems is an important piece of information to understand their dynamics. This task is especially challenging when the analyst's knowledge of the systems reduces virtually to time series of observations. Spurred by the success of Granger causality in econometrics, the study of cause-effect relationships (not to be confounded with statistical correlations) was extended to other fields, thus favoring the introduction of further tools such as transfer entropy. Currently, the research on old and new causality tools along with their pitfalls and applications in ever more general situations is going through a time of much activity. In this paper, we re-examine the method of the joint distance distribution to detect directional couplings between two multivariate flows. This method is based on the forced Takens theorem, and, more specifically, it exploits the existence of a continuous mapping from the reconstructed attractor of the response system to the reconstructed attractor of the driving system, an approach that is increasingly drawing the attention of the data analysts. The numerical results with Lorenz and Rössler oscillators in three different interaction networks (including hidden common drivers) are quite satisfactory, except when phase synchronization sets in. They also show that the method of the joint distance distribution outperforms the lowest dimensional transfer entropy in the cases considered. The robustness of the results to the sampling interval, time series length, observational noise, and metric is analyzed too.

Space-time nature of causality

In a causal world the direction of the time arrow dictates how past causal events in a variable X produce future effects in Y. X is said to cause an effect in Y, if the predictability (uncertainty) about the future states of Y increases (decreases) as its own past and the past of X are taken into consideration. Causality is thus intrinsic dependent on the observation of the past events of both variables involved, to the prediction (or uncertainty reduction) of future event of the other variable. We will show that this temporal notion of causality leads to another natural spatiotemporal definition for it, and that can be exploited to detect the arrow of influence from X to Y, either by considering shorter time-series of X and longer time-series of Y (an approach that explores the time nature of causality) or lower precision measured time-series in X and higher precision measured time-series in Y (an approach that explores the spatial nature of causality). Causality has thus space and time signatures, causing a break of symmetry in the topology of the probabilistic space, or causing a break of symmetry in the length of the measured time-series, a consequence of the fact that information flows from X to Y.

Detecting causality using symmetry transformations

Detecting causality between variables in a time series is a challenge, particularly when the relationship is nonlinear and the dataset is noisy. Here, we present a novel tool for detecting causality that leverages the properties of symmetry transformations. The aim is to develop an algorithm with the potential to detect both unidirectional and bidirectional coupling for nonlinear systems in the presence of significant sampling noise. Most of the existing tools for detecting causality can make determinations of directionality, but those determinations are relatively fragile in the presence of noise. The novel algorithm developed in the present study is robust and very conservative in that it reliably detects causal structure with a very low rate of error even in the presence of high sampling noise. We demonstrate the performance of our algorithm and compare it with two popular model-free methods, namely transfer entropy and convergent cross map. This first implementation of the method of symmetry transformations is limited in that it applies only to first-order autonomous systems.

Introduction to Focus Issue: Causation inference and information flow in dynamical systems: Theory and applications

Questions of causation are foundational across science and often relate further to problems of control, policy decisions, and forecasts. In nonlinear dynamics and complex systems science, causation inference and information flow are closely related concepts, whereby "information" or knowledge of certain states can be thought of as coupling influence onto the future states of other processes in a complex system. While causation inference and information flow are by now classical topics, incorporating methods from statistics and time series analysis, information theory, dynamical systems, and statistical mechanics, to name a few, there remain important advancements in continuing to strengthen the theory, and pushing the context of applications, especially with the ever-increasing abundance of data collected across many fields and systems. This Focus Issue considers different aspects of these questions, both in terms of founding theory and several topical applications.

Model-free inference of direct network interactions from nonlinear collective dynamics

The topology of interactions in network dynamical systems fundamentally underlies their function. Accelerating technological progress creates massively available data about collective nonlinear dynamics in physical, biological, and technological systems. Detecting direct interaction patterns from those dynamics still constitutes a major open problem. In particular, current nonlinear dynamics approaches mostly require to know a priori a model of the (often high dimensional) system dynamics. Here we develop a model-independent framework for inferring direct interactions solely from recording the nonlinear collective dynamics generated. Introducing an explicit dependency matrix in combination with a block-orthogonal regression algorithm, the approach works reliably across many dynamical regimes, including transient dynamics toward steady states, periodic and non-periodic dynamics, and chaos. Together with its capabilities to reveal network (two point) as well as hypernetwork (e.g., three point) interactions, this framework may thus open up nonlinear dynamics options of inferring direct interaction patterns across systems where no model is known.

Network dynamical systems can represent the interactions involved in the collective dynamics of gene regulatory networks or metabolic circuits. Here Casadiego et al. present a method for inferring these types of interactions directly from observed time series without relying on their model.

Dimensionless embedding for nonlinear time series analysis

Recently, infinite-dimensional delay coordinates (InDDeCs) have been proposed for predicting high-dimensional dynamics instead of conventional delay coordinates. Although InDDeCs can realize faster computation and more accurate short-term prediction, it is still not well-known whether InDDeCs can be used in other applications of nonlinear time series analysis in which reconstruction is needed for the underlying dynamics from a scalar time series generated from a dynamical system. Here, we give theoretical support for justifying the use of InDDeCs and provide numerical examples to show that InDDeCs can be used for various applications for obtaining the recurrence plots, correlation dimensions, and maximal Lyapunov exponents, as well as testing directional couplings and extracting slow-driving forces. We demonstrate performance of the InDDeCs using the weather data. Thus, InDDeCs can eventually realize "dimensionless embedding" while we enjoy faster and more reliable computations.

Detection of time delays and directional interactions based on time series from complex dynamical systems

Data-based and model-free accurate identification of intrinsic time delays and directional interactions is an extremely challenging problem in complex dynamical systems and their networks reconstruction. A model-free method with new scores is proposed to be generally capable of detecting single, multiple, and distributed time delays. The method is applicable not only to mutually interacting dynamical variables but also to self-interacting variables in a time-delayed feedback loop. Validation of the method is carried out using physical, biological, and ecological models and real data sets. Especially, applying the method to air pollution data and hospital admission records of cardiovascular diseases in Hong Kong reveals the major air pollutants as a cause of the diseases and, more importantly, it uncovers a hidden time delay (about 30-40 days) in the causal influence that previous studies failed to detect. The proposed method is expected to be universally applicable to ascertaining and quantifying subtle interactions (e.g., causation) in complex systems arising from a broad range of disciplines.

Computing algebraic transfer entropy and coupling directions via transcripts

Most random processes studied in nonlinear time series analysis take values on sets endowed with a group structure, e.g., the real and rational numbers, and the integers. This fact allows to associate with each pair of group elements a third element, called their transcript, which is defined as the product of the second element in the pair times the first one. The transfer entropy of two such processes is called algebraic transfer entropy. It measures the information transferred between two coupled processes whose values belong to a group. In this paper, we show that, subject to one constraint, the algebraic transfer entropy matches the (in general, conditional) mutual information of certain transcripts with one variable less. This property has interesting practical applications, especially to the analysis of short time series. We also derive weak conditions for the 3-dimensional algebraic transfer entropy to yield the same coupling direction as the corresponding mutual information of transcripts. A related issue concerns the use of mutual information of transcripts to determine coupling directions in cases where the conditions just mentioned are not fulfilled. We checked the latter possibility in the lowest dimensional case with numerical simulations and cardiovascular data, and obtained positive results.

Armed-conflict risks enhanced by climate-related disasters in ethnically fractionalized countries

Social and political tensions keep on fueling armed conflicts around the world. Although each conflict is the result of an individual context-specific mixture of interconnected factors, ethnicity appears to play a prominent and almost ubiquitous role in many of them. This overall state of affairs is likely to be exacerbated by anthropogenic climate change and in particular climate-related natural disasters. Ethnic divides might serve as predetermined conflict lines in case of rapidly emerging societal tensions arising from disruptive events like natural disasters. Here, we hypothesize that climate-related disaster occurrence enhances armed-conflict outbreak risk in ethnically fractionalized countries. Using event coincidence analysis, we test this hypothesis based on data on armed-conflict outbreaks and climate-related natural disasters for the period 1980-2010. Globally, we find a coincidence rate of 9% regarding armed-conflict outbreak and disaster occurrence such as heat waves or droughts. Our analysis also reveals that, during the period in question, about 23% of conflict outbreaks in ethnically highly fractionalized countries robustly coincide with climatic calamities. Although we do not report evidence that climate-related disasters act as direct triggers of armed conflicts, the disruptive nature of these events seems to play out in ethnically fractionalized societies in a particularly tragic way. This observation has important implications for future security policies as several of the world's most conflict-prone regions, including North and Central Africa as well as Central Asia, are both exceptionally vulnerable to anthropogenic climate change and characterized by deep ethnic divides.

Detecting Causality by Combined Use of Multiple Methods: Climate and Brain Examples

Identifying causal relations from time series is the first step to understanding the behavior of complex systems. Although many methods have been proposed, few papers have applied multiple methods together to detect causal relations based on time series generated from coupled nonlinear systems with some unobserved parts. Here we propose the combined use of three methods and a majority vote to infer causality under such circumstances. Two of these methods are proposed here for the first time, and all of the three methods can be applied even if the underlying dynamics is nonlinear and there are hidden common causes. We test our methods with coupled logistic maps, coupled Rössler models, and coupled Lorenz models. In addition, we show from ice core data how the causal relations among the temperature, the CH4 level, and the CO2 level in the atmosphere changed in the last 800,000 years, a conclusion also supported by irregularly sampled data analysis. Moreover, these methods show how three regions of the brain interact with each other during the visually cued, two-choice arm reaching task. Especially, we demonstrate that this is due to bottom up influences at the beginning of the task, while there exist mutual influences between the posterior medial prefrontal cortex and the presupplementary motor area. Based on our results, we conclude that identifying causality with an appropriate ensemble of multiple methods ensures the validity of the obtained results more firmly.

Edit distance for marked point processes revisited: An implementation by binary integer programming

We implement the edit distance for marked point processes [Suzuki et al., Int. J. Bifurcation Chaos 20, 3699-3708 (2010)] as a binary integer program. Compared with the previous implementation using minimum cost perfect matching, the proposed implementation has two advantages: first, by using the proposed implementation, we can apply a wide variety of software and hardware, even spin glasses and coherent ising machines, to calculate the edit distance for marked point processes; second, the proposed implementation runs faster than the previous implementation when the difference between the numbers of events in two time windows for a marked point process is large.

Untangling Brain-Wide Dynamics in Consciousness by Cross-Embedding

Brain-wide interactions generating complex neural dynamics are considered crucial for emergent cognitive functions. However, the irreducible nature of nonlinear and high-dimensional dynamical interactions challenges conventional reductionist approaches. We introduce a model-free method, based on embedding theorems in nonlinear state-space reconstruction, that permits a simultaneous characterization of complexity in local dynamics, directed interactions between brain areas, and how the complexity is produced by the interactions. We demonstrate this method in large-scale electrophysiological recordings from awake and anesthetized monkeys. The cross-embedding method captures structured interaction underlying cortex-wide dynamics that may be missed by conventional correlation-based analysis, demonstrating a critical role of time-series analysis in characterizing brain state. The method reveals a consciousness-related hierarchy of cortical areas, where dynamical complexity increases along with cross-area information flow. These findings demonstrate the advantages of the cross-embedding method in deciphering large-scale and heterogeneous neuronal systems, suggesting a crucial contribution by sensory-frontoparietal interactions to the emergence of complex brain dynamics during consciousness.

Computing ensembles of transitions with molecular dynamics simulations

A molecular understanding of conformational change is important for connecting structure and function. Without the ability to sample on the meaningful large-scale conformational changes, the ability to infer biological function and to understand the effect of mutations and changes in environment is not possible. Our Dynamic Importance Sampling method (DIMS), part of the CHARMM simulation package, is a method that enables sampling over ensembles of transition intermediates. This chapter outlines the context for the method and the usage within the program.

Detecting causality from nonlinear dynamics with short-term time series

Quantifying causality between variables from observed time series data is of great importance in various disciplines but also a challenging task, especially when the observed data are short. Unlike the conventional methods, we find it possible to detect causality only with very short time series data, based on embedding theory of an attractor for nonlinear dynamics. Specifically, we first show that measuring the smoothness of a cross map between two observed variables can be used to detect a causal relation. Then, we provide a very effective algorithm to computationally evaluate the smoothness of the cross map, or "Cross Map Smoothness" (CMS), and thus to infer the causality, which can achieve high accuracy even with very short time series data. Analysis of both mathematical models from various benchmarks and real data from biological systems validates our method.

Estimating coupling directions in the cardiorespiratory system using recurrence properties

The asymmetry of coupling between complex systems can be studied by conditional probabilities of recurrence, which can be estimated by joint recurrence plots. This approach is applied for the first time on experimental data: time series of the human cardiorespiratory system in order to investigate the couplings between heart rate, mean arterial blood pressure and respiration. We find that the respiratory system couples towards the heart rate, and the heart rate towards the mean arterial blood pressure. However, our analysis could not detect a clear coupling direction between the mean arterial blood pressure and respiration.

Characterizing global evolutions of complex systems via intermediate network representations

Recent developments in measurement techniques have enabled us to observe the time series of many components simultaneously. Thus, it is important to understand not only the dynamics of individual time series but also their interactions. Although there are many methods for analysing the interaction between two or more time series, there are very few methods that describe global changes of the interactions over time. Here, we propose an approach to visualise time evolution for the global changes of the interactions in complex systems. This approach consists of two steps. In the first step, we construct a meta-time series of networks. In the second step, we analyse and visualise this meta-time series by using distance and recurrence plots. Our two-step approach involving intermediate network representations elucidates the half-a-day periodicity of foreign exchange markets and a singular functional network in the brain related to perceptual alternations.

Inferring directional interactions from transient signals with symbolic transfer entropy

We extend the concept of symbolic transfer entropy to enable the time-resolved investigation of directional relationships between coupled dynamical systems from short and transient noisy time series. For our approach, we consider an observed ensemble of a sufficiently large number of time series as multiple realizations of a process. We derive an index that quantifies the preferred direction of transient interactions and assess its significance using a surrogate-based testing scheme. Analyzing time series from noisy chaotic systems, we demonstrate numerically the applicability and limitations of our approach. Our findings obtained from an analysis of event-related brain activities underline the importance of our method to improve understanding of gross neural interactions underlying cognitive processes.