Radial basis function approach to nonlinear Granger causality of time series

Reliable detection of directional couplings using cross-vector measures

Detecting directional couplings from time series is crucial in understanding complex dynamical systems. Various approaches based on reconstructed state-spaces have been developed for this purpose, including a cross-distance vector measure, which we introduced in our recent work. Here, we devise two new cross-vector measures that utilize ranks and time series estimates instead of distances. We analyze various deterministic and stochastic dynamics to compare our cross-vector approach against some established state-space-based approaches. We demonstrate that all three cross-vector measures can identify the correct coupling direction for a broader range of couplings for all considered dynamics. Among the three cross-vector measures, the rank-based variant performs the best. Comparing this novel measure to an established rank-based measure confirms that it is more noise-robust and less affected by linear cross-correlation. To extend this comparison to real-world signals, we combine both measures with the method of surrogates and analyze a database of electroencephalographic (EEG) recordings from epilepsy patients. This database contains signals from brain areas where the patients' seizures were detected first and signals from brain areas that were not involved in the seizure onset. A better discrimination between these signal classes is obtained by the cross-rank vector measure. Additionally, this measure proves to be robust to non-stationarity, as its results remain nearly unchanged when the analysis is repeated for the subset of EEG signals that were identified as stationary in previous work. These findings suggest that the cross-vector approach can serve as a valuable tool for researchers analyzing complex time series and for clinical applications.

Robust Model-Free Identification of the Causal Networks Underlying Complex Nonlinear Systems

Inferring causal networks from noisy observations is of vital importance in various fields. Due to the complexity of system modeling, the way in which universal and feasible inference algorithms are studied is a key challenge for network reconstruction. In this study, without any assumptions, we develop a novel model-free framework to uncover only the direct relationships in networked systems from observations of their nonlinear dynamics. Our proposed methods are termed multiple-order Polynomial Conditional Granger Causality (PCGC) and sparse PCGC (SPCGC). PCGC mainly adopts polynomial functions to approximate the whole system model, which can be used to judge the interactions among nodes through subsequent nonlinear Granger causality analysis. For SPCGC, Lasso optimization is first used for dimension reduction, and then PCGC is executed to obtain the final network. Specifically, the conditional variables are fused in this general, model-free framework regardless of their formulations in the system model, which could effectively reconcile the inference of direct interactions with an indirect influence. Based on many classical dynamical systems, the performances of PCGC and SPCGC are analyzed and verified. Generally, the proposed framework could be quite promising for the provision of certain guidance for data-driven modeling with an unknown model.

Decomposing causality into its synergistic, unique, and redundant components

Causality lies at the heart of scientific inquiry, serving as the fundamental basis for understanding interactions among variables in physical systems. Despite its central role, current methods for causal inference face significant challenges due to nonlinear dependencies, stochastic interactions, self-causation, collider effects, and influences from exogenous factors, among others. While existing methods can effectively address some of these challenges, no single approach has successfully integrated all these aspects. Here, we address these challenges with SURD: Synergistic-Unique-Redundant Decomposition of causality. SURD quantifies causality as the increments of redundant, unique, and synergistic information gained about future events from past observations. The formulation is non-intrusive and applicable to both computational and experimental investigations, even when samples are scarce. We benchmark SURD in scenarios that pose significant challenges for causal inference and demonstrate that it offers a more reliable quantification of causality compared to previous methods.

Nonlinear causal network learning via Granger causality based on extreme support vector regression

For complex networked systems, based on the consideration of nonlinearity and causality, a novel general method of nonlinear causal network learning, termed extreme support vector regression Granger causality (ESVRGC), is proposed. The nonuniform time-delayed influence of the driving nodes on the target node is particularly considered. Then, the restricted model and the unrestricted model of Granger causality are, respectively, formulated based on extreme support vector regression, which uses the selected time-delayed components of system variables as the inputs of kernel functions. The nonlinear conditional Granger causality index is finally calculated to confirm the strength of a causal interaction. Generally, based on the simulation of a nonlinear vector autoregressive model and nonlinear discrete time-delayed dynamic systems, ESVRGC demonstrates better performance than other popular methods. Also, the validity and robustness of ESVRGC are also verified by the different cases of network types, sample sizes, noise intensities, and coupling strengths. Finally, the superiority of ESVRGC is successful verified by the experimental study on real benchmark datasets.

Incorporating causality in energy consumption forecasting using deep neural networks

Forecasting energy demand has been a critical process in various decision support systems regarding consumption planning, distribution strategies, and energy policies. Traditionally, forecasting energy consumption or demand methods included trend analyses, regression, and auto-regression. With advancements in machine learning methods, algorithms such as support vector machines, artificial neural networks, and random forests became prevalent. In recent times, with an unprecedented improvement in computing capabilities, deep learning algorithms are increasingly used to forecast energy consumption/demand. In this contribution, a relatively novel approach is employed to use long-term memory. Weather data was used to forecast the energy consumption from three datasets, with an additional piece of information in the deep learning architecture. This additional information carries the causal relationships between the weather indicators and energy consumption. This architecture with the causal information is termed as entangled long short term memory. The results show that the entangled long short term memory outperforms the state-of-the-art deep learning architecture (bidirectional long short term memory). The theoretical and practical implications of these results are discussed in terms of decision-making and energy management systems.

Causal Inference in Time Series in Terms of Rényi Transfer Entropy

Uncovering causal interdependencies from observational data is one of the great challenges of a nonlinear time series analysis. In this paper, we discuss this topic with the help of an information-theoretic concept known as Rényi’s information measure. In particular, we tackle the directional information flow between bivariate time series in terms of Rényi’s transfer entropy. We show that by choosing Rényi’s parameter α, we can appropriately control information that is transferred only between selected parts of the underlying distributions. This, in turn, is a particularly potent tool for quantifying causal interdependencies in time series, where the knowledge of “black swan” events, such as spikes or sudden jumps, are of key importance. In this connection, we first prove that for Gaussian variables, Granger causality and Rényi transfer entropy are entirely equivalent. Moreover, we also partially extend these results to heavy-tailed α-Gaussian variables. These results allow establishing a connection between autoregressive and Rényi entropy-based information-theoretic approaches to data-driven causal inference. To aid our intuition, we employed the Leonenko et al. entropy estimator and analyzed Rényi’s information flow between bivariate time series generated from two unidirectionally coupled Rössler systems. Notably, we find that Rényi’s transfer entropy not only allows us to detect a threshold of synchronization but it also provides non-trivial insight into the structure of a transient regime that exists between the region of chaotic correlations and synchronization threshold. In addition, from Rényi’s transfer entropy, we could reliably infer the direction of coupling and, hence, causality, only for coupling strengths smaller than the onset value of the transient regime, i.e., when two Rössler systems are coupled but have not yet entered synchronization.

Phase-based causality analysis with partial mutual information from mixed embedding

Instantaneous phases extracted from multivariate time series can retain information about the relationships between the underlying mechanisms that generate the series. Although phases have been widely used in the study of nondirectional coupling and connectivity, they have not found similar appeal in the study of causality. Herein, we present a new method for phase-based causality analysis, which combines ideas from the mixed embedding technique and the information-theoretic approach to causality in coupled oscillatory systems. We then use the introduced method to investigate causality in simulated datasets of bivariate, unidirectionally paired systems from combinations of Rössler, Lorenz, van der Pol, and Mackey-Glass equations. We observe that causality analysis using the phases can capture the true causal relation for coupling strength smaller than the analysis based on the amplitudes can capture. On the other hand, the causality estimation based on the phases tends to have larger variability, which is attributed more to the phase extraction process than the actual phase-based causality method. In addition, an application on real electroencephalographic data from an experiment on elicited human emotional states reinforces the usefulness of phases in causality identification.

Directional couplings between the respiration and parasympathetic control of the heart rate during sleep and wakefulness in healthy subjects at different ages

Cardiorespiratory interactions are important, both for understanding the fundamental processes of functioning of the human body and for development of methods for diagnostics of various pathologies. The properties of cardiorespiratory interaction are determined by the processes of autonomic control of blood circulation, which are modulated by the higher nervous activity. We study the directional couplings between the respiration and the process of parasympathetic control of the heart rate in the awake state and different stages of sleep in 96 healthy subjects from different age groups. The detection of directional couplings is carried out using the method of phase dynamics modeling applied to experimental RR-intervals and the signal of respiration. We reveal the presence of bidirectional couplings between the studied processes in all age groups. Our results show that the coupling from respiration to the process of parasympathetic control of the heart rate is stronger than the coupling in the opposite direction. The difference in the strength of bidirectional couplings between the considered processes is most pronounced in deep sleep.

A Nonlinear Effective Connectivity Measure Based on Granger Causality and Volterra Series

Estimating effective connectivity, especially in brain networks, is an important topic to find out the brain functions. Various effective connectivity measures are presented, but they have drawbacks, including bivariate structure, the problem in detecting nonlinear interactions, and high computational cost. In this paper, we have proposed a novel multivariate effective connectivity measure based on a hierarchical realization of the Volterra series model and Granger causality concept, namely hierarchical Volterra Granger causality (HVGC). HVGC is a multivariate connectivity measure that can detect linear and nonlinear causal effects. The performance of HVGC is compared with Granger causality index (GCI), conditional Granger causality index (CGCI), transfer entropy (TE), phase transfer entropy (Phase TE), and partial transfer entropy (Partial TE) in simulated and physiological datasets. In addition to accuracy, specificity, and sensitivity, the Matthews correlation coefficient (MCC) is used to evaluate the connectivity estimation in simulated datasets. Furthermore influence of different SNRs is investigated on the estimated connectivity. The obtained results show that HVGC with a minimum MCC of 0.76 performs well in the detection of both linear and nonlinear interactions in simulated data. HVGC is also applied to a physiological dataset that was cardiorespiratory interaction signals recorded during sleep from a patient suffering from sleep apnea. The results of this dataset also demonstrate the capability of the proposed method in the detection of causal interactions. Applying HVGC on the simulated fMRI dataset led to a high MCC of 0.78. Moreover, the results indicate that HVGC has slight changes in different SNRs. The results indicate that HVGC can estimate the causal effects of a linear and nonlinear system with a low computational cost and it is slightly affected by noise.

Connectivity Analysis for Multivariate Time Series: Correlation vs. Causality

The study of the interdependence relationships of the variables of an examined system is of great importance and remains a challenging task. There are two distinct cases of interdependence. In the first case, the variables evolve in synchrony, connections are undirected and the connectivity is examined based on symmetric measures, such as correlation. In the second case, a variable drives another one and they are connected with a causal relationship. Therefore, directed connections entail the determination of the interrelationships based on causality measures. The main open question that arises is the following: can symmetric correlation measures or directional causality measures be applied to infer the connectivity network of an examined system? Using simulations, we demonstrate the performance of different connectivity measures in case of contemporaneous or/and temporal dependencies. Results suggest the sensitivity of correlation measures when temporal dependencies exist in the data. On the other hand, causality measures do not spuriously indicate causal effects when data present only contemporaneous dependencies. Finally, the necessity of introducing effective instantaneous causality measures is highlighted since they are able to handle both contemporaneous and causal effects at the same time. Results based on instantaneous causality measures are promising; however, further investigation is required in order to achieve an overall satisfactory performance.

Incorporation of causality structures to complex network analysis of time-varying behaviour of multivariate time series

This paper presents a new methodology for characterising the evolving behaviour of the time-varying causality between multivariate time series, from the perspective of change in the structure of the causality pattern. We propose that such evolutionary behaviour should be tracked by means of a complex network whose nodes are causality patterns and edges are transitions between those patterns of causality. In our new methodology each edge has a weight that includes the frequency of the given transition and two metrics relating to the gross and net structural change in causality pattern, which we call [Formula: see text] and [Formula: see text]. To characterise aspects of the behaviour within this network, five approaches are presented and motivated. To act as a demonstration of this methodology an application of sample data from the international oil market is presented. This example illustrates how our new methodology is able to extract information about evolving causality behaviour. For example, it reveals non-random time-varying behaviour that favours transitions resulting in predominantly similar causality patterns, and it discovers clustering of similar causality patterns and some transitional behaviour between these clusters. The example illustrates how our new methodology supports the inference that the evolution of causality in the system is related to the addition or removal of a few causality links, primarily keeping a similar causality pattern, and that the evolution is not related to some other measure such as the overall number of causality links.

Causality indices for bivariate time series data: A comparative review of performance

Inferring nonlinear and asymmetric causal relationships between multivariate longitudinal data is a challenging task with wide-ranging application areas including clinical medicine, mathematical biology, economics, and environmental research. A number of methods for inferring causal relationships within complex dynamic and stochastic systems have been proposed, but there is not a unified consistent definition of causality in the context of time series data. We evaluate the performance of ten prominent causality indices for bivariate time series across four simulated model systems that have different coupling schemes and characteristics. Pairwise correlations between different methods, averaged across all simulations, show that there is generally strong agreement between methods, with minimum, median, and maximum Pearson correlations between any pair (excluding two similarity indices) of 0.298, 0.719, and 0.955, respectively. In further experiments, we show that these methods are not always invariant to real-world relevant transformations (data availability, standardization and scaling, rounding errors, missing data, and noisy data). We recommend transfer entropy and nonlinear Granger causality as particularly strong approaches for estimating bivariate causal relationships in real-world applications. Both successfully identify causal relationships and a lack thereof across multiple simulations, while remaining robust to rounding errors, at least 20% missing data and small variance Gaussian noise. Finally, we provide flexible open-access Python code for computation of these methods and for the model simulations.

Large-scale nonlinear Granger causality for inferring directed dependence from short multivariate time-series data

A key challenge to gaining insight into complex systems is inferring nonlinear causal directional relations from observational time-series data. Specifically, estimating causal relationships between interacting components in large systems with only short recordings over few temporal observations remains an important, yet unresolved problem. Here, we introduce large-scale nonlinear Granger causality (lsNGC) which facilitates conditional Granger causality between two multivariate time series conditioned on a large number of confounding time series with a small number of observations. By modeling interactions with nonlinear state-space transformations from limited observational data, lsNGC identifies casual relations with no explicit a priori assumptions on functional interdependence between component time series in a computationally efficient manner. Additionally, our method provides a mathematical formulation revealing statistical significance of inferred causal relations. We extensively study the ability of lsNGC in inferring directed relations from two-node to thirty-four node chaotic time-series systems. Our results suggest that lsNGC captures meaningful interactions from limited observational data, where it performs favorably when compared to traditionally used methods. Finally, we demonstrate the applicability of lsNGC to estimating causality in large, real-world systems by inferring directional nonlinear, causal relationships among a large number of relatively short time series acquired from functional Magnetic Resonance Imaging (fMRI) data of the human brain.

Nonlinear System Identification of Neural Systems from Neurophysiological Signals

The human nervous system is one of the most complicated systems in nature. Complex nonlinear behaviours have been shown from the single neuron level to the system level. For decades, linear connectivity analysis methods, such as correlation, coherence and Granger causality, have been extensively used to assess the neural connectivities and input-output interconnections in neural systems. Recent studies indicate that these linear methods can only capture a certain amount of neural activities and functional relationships, and therefore cannot describe neural behaviours in a precise or complete way. In this review, we highlight recent advances in nonlinear system identification of neural systems, corresponding time and frequency domain analysis, and novel neural connectivity measures based on nonlinear system identification techniques. We argue that nonlinear modelling and analysis are necessary to study neuronal processing and signal transfer in neural systems quantitatively. These approaches can hopefully provide new insights to advance our understanding of neurophysiological mechanisms underlying neural functions. These nonlinear approaches also have the potential to produce sensitive biomarkers to facilitate the development of precision diagnostic tools for evaluating neurological disorders and the effects of targeted intervention.

Explicit Granger causality in kernel Hilbert spaces

Granger causality (GC) is undoubtedly the most widely used method to infer cause-effect relations from observational time series. Several nonlinear alternatives to GC have been proposed based on kernel methods. We generalize kernel Granger causality by considering the variables' cross-relations explicitly in Hilbert spaces. The framework is shown to generalize the linear and kernel GC methods and comes with tighter bounds of performance based on Rademacher complexity. We successfully evaluate its performance in standard dynamical systems, as well as to identify the arrow of time in coupled Rössler systems, and it is exploited to disclose the El Niño-Southern Oscillation phenomenon footprints on soil moisture globally.

Allosteric Regulation at the Crossroads of New Technologies: Multiscale Modeling, Networks, and Machine Learning

Allosteric regulation is a common mechanism employed by complex biomolecular systems for regulation of activity and adaptability in the cellular environment, serving as an effective molecular tool for cellular communication. As an intrinsic but elusive property, allostery is a ubiquitous phenomenon where binding or disturbing of a distal site in a protein can functionally control its activity and is considered as the "second secret of life." The fundamental biological importance and complexity of these processes require a multi-faceted platform of synergistically integrated approaches for prediction and characterization of allosteric functional states, atomistic reconstruction of allosteric regulatory mechanisms and discovery of allosteric modulators. The unifying theme and overarching goal of allosteric regulation studies in recent years have been integration between emerging experiment and computational approaches and technologies to advance quantitative characterization of allosteric mechanisms in proteins. Despite significant advances, the quantitative characterization and reliable prediction of functional allosteric states, interactions, and mechanisms continue to present highly challenging problems in the field. In this review, we discuss simulation-based multiscale approaches, experiment-informed Markovian models, and network modeling of allostery and information-theoretical approaches that can describe the thermodynamics and hierarchy allosteric states and the molecular basis of allosteric mechanisms. The wealth of structural and functional information along with diversity and complexity of allosteric mechanisms in therapeutically important protein families have provided a well-suited platform for development of data-driven research strategies. Data-centric integration of chemistry, biology and computer science using artificial intelligence technologies has gained a significant momentum and at the forefront of many cross-disciplinary efforts. We discuss new developments in the machine learning field and the emergence of deep learning and deep reinforcement learning applications in modeling of molecular mechanisms and allosteric proteins. The experiment-guided integrated approaches empowered by recent advances in multiscale modeling, network science, and machine learning can lead to more reliable prediction of allosteric regulatory mechanisms and discovery of allosteric modulators for therapeutically important protein targets.

A Parsimonious Granger Causality Formulation for Capturing Arbitrarily Long Multivariate Associations

High-frequency neuroelectric signals like electroencephalography (EEG) or magnetoencephalography (MEG) provide a unique opportunity to infer causal relationships between local activity of brain areas. While causal inference is commonly performed through classical Granger causality (GC) based on multivariate autoregressive models, this method may encounter important limitations (e.g., data paucity) in the case of high dimensional data from densely connected systems like the brain. Additionally, physiological signals often present long-range dependencies which commonly require high autoregressive model orders/number of parameters. We present a generalization of autoregressive models for GC estimation based on Wiener–Volterra decompositions with Laguerre polynomials as basis functions. In this basis, the introduction of only one additional global parameter allows to capture arbitrary long dependencies without increasing model order, hence retaining model simplicity, linearity and ease of parameters estimation. We validate our method in synthetic data generated from families of complex, densely connected networks and demonstrate superior performance as compared to classical GC. Additionally, we apply our framework to studying the directed human brain connectome through MEG data from 89 subjects drawn from the Human Connectome Project (HCP) database, showing that it is able to reproduce current knowledge as well as to uncover previously unknown directed influences between cortical and limbic brain regions.

A study for multiscale information transfer measures based on conditional mutual information

As the big data science develops, efficient methods are demanded for various data analysis. Granger causality provides the prime model for quantifying causal interactions. However, this theoretic model does not meet the requirement for real-world data analysis, because real-world time series are diverse whose models are usually unknown. Therefore, model-free measures such as information transfer measures are strongly desired. Here, we propose the multi-scale extension of conditional mutual information measures using MORLET wavelet, which are named the WM and WPM. The proposed measures are computational efficient and interpret information transfer by multi-scales. We use both synthetic data and real-world examples to demonstrate the efficiency of the new methods. The results of the new methods are robust and reliable. Via the simulation studies, we found the new methods outperform the wavelet extension of transfer entropy (WTE) in both computational efficiency and accuracy. The features and properties of the proposed measures are also discussed.

Space-time nature of causality

In a causal world the direction of the time arrow dictates how past causal events in a variable X produce future effects in Y. X is said to cause an effect in Y, if the predictability (uncertainty) about the future states of Y increases (decreases) as its own past and the past of X are taken into consideration. Causality is thus intrinsic dependent on the observation of the past events of both variables involved, to the prediction (or uncertainty reduction) of future event of the other variable. We will show that this temporal notion of causality leads to another natural spatiotemporal definition for it, and that can be exploited to detect the arrow of influence from X to Y, either by considering shorter time-series of X and longer time-series of Y (an approach that explores the time nature of causality) or lower precision measured time-series in X and higher precision measured time-series in Y (an approach that explores the spatial nature of causality). Causality has thus space and time signatures, causing a break of symmetry in the topology of the probabilistic space, or causing a break of symmetry in the length of the measured time-series, a consequence of the fact that information flows from X to Y.

Estimating Brain Connectivity With Varying-Length Time Lags Using a Recurrent Neural Network

OBJECTIVE

Computer-aided estimation of brain connectivity aims to reveal information propagation in brain automatically, which has great potential in clinical applications, e.g., epilepsy foci diagnosis. Granger causality is an effective tool for directional connection analysis in multivariate time series. However, most existing methods based on Granger causality assume fixed time lags in information transmission, while the propagation delay between brain signals is usually changing constantly.

METHODS

We propose a Granger causality estimator based on the recurrent neural network, called RNN-GC, to deal with the multivariate brain connectivity detection problem. Our model takes input of time-series signals with arbitrary length of transmission time lags and learns the information flow from the data using the gated RNN model, i.e., long short-term memory (LSTM). The LSTM model can sequentially update the gates in memory cells to determine how many preceding points should be considered for prediction. Therefore, the LSTM-based RNN-GC estimator works well on varying-length time lags and shows effectiveness even on very long transmission delays.

RESULTS

Experiments are carried out in comparison with other methods using both simulation data and epileptic electroencephalography signals. The RNN-GC estimator achieves superior performance in brain connectivity estimation and shows robustness in modeling multivariate connections with varying-length time lags.

CONCLUSION

The RNN-GC method is capable of modeling nonlinear and varying-length lagged information transmission and effective in directional brain connectivity estimation.

SIGNIFICANCE

The proposed method is promising to serve as a robust brain connection analysis tool in clinical applications.

Inference of topology and the nature of synapses, and the flow of information in neuronal networks

The characterization of neuronal connectivity is one of the most important matters in neuroscience. In this work, we show that a recently proposed informational quantity, the causal mutual information, employed with an appropriate methodology, can be used not only to correctly infer the direction of the underlying physical synapses, but also to identify their excitatory or inhibitory nature, considering easy to handle and measure bivariate time series. The success of our approach relies on a surprising property found in neuronal networks by which nonadjacent neurons do "understand" each other (positive mutual information), however, this exchange of information is not capable of causing effect (zero transfer entropy). Remarkably, inhibitory connections, responsible for enhancing synchronization, transfer more information than excitatory connections, known to enhance entropy in the network. We also demonstrate that our methodology can be used to correctly infer directionality of synapses even in the presence of dynamic and observational Gaussian noise, and is also successful in providing the effective directionality of intermodular connectivity, when only mean fields can be measured.

Supervised Estimation of Granger-Based Causality between Time Series

Brain effective connectivity aims to detect causal interactions between distinct brain units and it is typically studied through the analysis of direct measurements of the neural activity, e.g., magneto/electroencephalography (M/EEG) signals. The literature on methods for causal inference is vast. It includes model-based methods in which a generative model of the data is assumed and model-free methods that directly infer causality from the probability distribution of the underlying stochastic process. Here, we firstly focus on the model-based methods developed from the Granger criterion of causality, which assumes the autoregressive model of the data. Secondly, we introduce a new perspective, that looks at the problem in a way that is typical of the machine learning literature. Then, we formulate the problem of causality detection as a supervised learning task, by proposing a classification-based approach. A classifier is trained to identify causal interactions between time series for the chosen model and by means of a proposed feature space. In this paper, we are interested in comparing this classification-based approach with the standard Geweke measure of causality in the time domain, through simulation study. Thus, we customized our approach to the case of a MAR model and designed a feature space which contains causality measures based on the idea of precedence and predictability in time. Two variations of the supervised method are proposed and compared to a standard Granger causal analysis method. The results of the simulations show that the supervised method outperforms the standard approach, in particular it is more robust to noise. As evidence of the efficacy of the proposed method, we report the details of our submission to the causality detection competition of Biomag2014, where the proposed method reached the 2nd place. Moreover, as empirical application, we applied the supervised approach on a dataset of neural recordings of rats obtaining an important reduction in the false positive rate.

Detection of time delays and directional interactions based on time series from complex dynamical systems

Data-based and model-free accurate identification of intrinsic time delays and directional interactions is an extremely challenging problem in complex dynamical systems and their networks reconstruction. A model-free method with new scores is proposed to be generally capable of detecting single, multiple, and distributed time delays. The method is applicable not only to mutually interacting dynamical variables but also to self-interacting variables in a time-delayed feedback loop. Validation of the method is carried out using physical, biological, and ecological models and real data sets. Especially, applying the method to air pollution data and hospital admission records of cardiovascular diseases in Hong Kong reveals the major air pollutants as a cause of the diseases and, more importantly, it uncovers a hidden time delay (about 30-40 days) in the causal influence that previous studies failed to detect. The proposed method is expected to be universally applicable to ascertaining and quantifying subtle interactions (e.g., causation) in complex systems arising from a broad range of disciplines.

Reconstruction of Complex Directional Networks with Group Lasso Nonlinear Conditional Granger Causality

Reconstruction of networks underlying complex systems is one of the most crucial problems in many areas of engineering and science. In this paper, rather than identifying parameters of complex systems governed by pre-defined models or taking some polynomial and rational functions as a prior information for subsequent model selection, we put forward a general framework for nonlinear causal network reconstruction from time-series with limited observations. With obtaining multi-source datasets based on the data-fusion strategy, we propose a novel method to handle nonlinearity and directionality of complex networked systems, namely group lasso nonlinear conditional granger causality. Specially, our method can exploit different sets of radial basis functions to approximate the nonlinear interactions between each pair of nodes and integrate sparsity into grouped variables selection. The performance characteristic of our approach is firstly assessed with two types of simulated datasets from nonlinear vector autoregressive model and nonlinear dynamic models, and then verified based on the benchmark datasets from DREAM3 Challenge4. Effects of data size and noise intensity are also discussed. All of the results demonstrate that the proposed method performs better in terms of higher area under precision-recall curve.

On the role of the entorhinal cortex in the effective connectivity of the hippocampal formation

Inferring effective connectivity from neurophysiological data is a challenging task. In particular, only a finite (and usually small) number of sites are simultaneously recorded, while the response of one of these sites can be influenced by other sites that are not being recorded. In the hippocampal formation, for instance, the connections between areas CA1-CA3, the dentate gyrus (DG), and the entorhinal cortex (EC) are well established. However, little is known about the relations within the EC layers, which might strongly affect the resulting effective connectivity estimations. In this work, we build excitatory/inhibitory neuronal populations representing the four areas CA1, CA3, the DG, and the EC and fix their connectivities. We model the EC by three layers (LII, LIII, and LV) and assume any possible connection between them. Our results, based on Granger Causality (GC) and Partial Transfer Entropy (PTE) measurements, reveal that the estimation of effective connectivity in the hippocampus strongly depends on the connectivities between EC layers. Moreover, we find, for certain EC configurations, very different results when comparing GC and PTE measurements. We further demonstrate that causal links can be robustly inferred regardless of the excitatory or inhibitory nature of the connection, adding complexity to their interpretation. Overall, our work highlights the importance of a careful analysis of the connectivity methods to prevent unrealistic conclusions when only partial information about the experimental system is available, as usually happens in brain networks. Our results suggest that the combination of causality measures with neuronal modeling based on precise neuroanatomical tracing may provide a powerful framework to disambiguate causal interactions in the brain.

Causality analysis detects the regulatory role of maternal effect genes in the early Drosophila embryo

In developmental studies, inferring regulatory interactions of segmentation genetic network play a vital role in unveiling the mechanism of pattern formation. As such, there exists an opportune demand for theoretical developments and new mathematical models which can result in a more accurate illustration of this genetic network. Accordingly, this paper seeks to extract the meaningful regulatory role of the maternal effect genes using a variety of causality detection techniques and to explore whether these methods can suggest a new analytical view to the gene regulatory networks. We evaluate the use of three different powerful and widely-used models representing time and frequency domain Granger causality and convergent cross mapping technique with the results being thoroughly evaluated for statistical significance. Our findings show that the regulatory role of maternal effect genes is detectable in different time classes and thereby the method is applicable to infer the possible regulatory interactions present among the other genes of this network.

Temporal Information of Directed Causal Connectivity in Multi-Trial ERP Data using Partial Granger Causality

Partial Granger causality (PGC) has been applied to analyse causal functional neural connectivity after effectively mitigating confounding influences caused by endogenous latent variables and exogenous environmental inputs. However, it is not known how this connectivity obtained from PGC evolves over time. Furthermore, PGC has yet to be tested on realistic nonlinear neural circuit models and multi-trial event-related potentials (ERPs) data. In this work, we first applied a time-domain PGC technique to evaluate simulated neural circuit models, and demonstrated that the PGC measure is more accurate and robust in detecting connectivity patterns as compared to conditional Granger causality and partial directed coherence, especially when the circuit is intrinsically nonlinear. Moreover, the connectivity in PGC settles faster into a stable and correct configuration over time. After method verification, we applied PGC to reveal the causal connections of ERP trials of a mismatch negativity auditory oddball paradigm. The PGC analysis revealed a significant bilateral but asymmetrical localised activity in the temporal lobe close to the auditory cortex, and causal influences in the frontal, parietal and cingulate cortical areas, consistent with previous studies. Interestingly, the time to reach a stable connectivity configuration (~250–300 ms) coincides with the deviation of ensemble ERPs of oddball from standard tones. Finally, using a sliding time window, we showed higher resolution dynamics of causal connectivity within an ERP trial. In summary, time-domain PGC is promising in deciphering directed functional connectivity in nonlinear and ERP trials accurately, and at a sufficiently early stage. This data-driven approach can reduce computational time, and determine the key architecture for neural circuit modeling.

Neural Network Models

The Artificial Neural Network (ANN) models gained a wide popularity owing to a number of claimed advantages such as biological plausibility, tolerance with respect to errors or noise in the input data, learning ability allowing an adaptability to environmental constraints. Notwithstanding the fact that most of these advantages are not typical only of ANNs, engineers, psychologists and neuroscientists made an extended use of ANN models in a large number of scientific investigations. In most cases, however, these models have been introduced in order to provide optimization tools more useful than the ones commonly used by traditional Optimization Theory. Unfortunately, just the successful performance of ANN models in optimization tasks produced a widespread neglect of the true – and important – objectives pursued by the first promoters of these models. These objectives can be shortly summarized by the manifesto of connectionist psychology, stating that mental processes are nothing but macroscopic phenomena, emergent from the cooperative interaction of a large number of microscopic knowledge units. This statement – wholly in line with the goal of statistical mechanics – can be readily extended to other processes, beyond the mental ones, including social, economic, and, in general, organizational ones. Therefore this chapter has been designed in order to answer a number of related questions, such as: are the ANN models able to grant for the occurrence of this sort of emergence? How can the occurrence of this emergence be empirically detected? How can the emergence produced by ANN models be controlled? In which sense the ANN emergence could offer a new paradigm for the explanation of macroscopic phenomena? Answering these questions induces to focus the chapter on less popular ANNs, such as the recurrent ones, while neglecting more popular models, such as perceptrons, and on less used units, such as spiking neurons, rather than on McCulloch-Pitts neurons. Moreover, the chapter must mention a number of strategies of emergence detection, useful for researchers performing computer simulations of ANN behaviours. Among these strategies it is possible to quote the reduction of ANN models to continuous models, such as the neural field models or the neural mass models, the recourse to the methods of Network Theory and the employment of techniques borrowed by Statistical Physics, like the one based on the Renormalization Group. Of course, owing to space (and mathematical expertise) requirements, most mathematical details of the proposed arguments are neglected, and, to gain more information, the reader is deferred to the quoted literature.

A Nonlinear Causality Estimator Based on Non-Parametric Multiplicative Regression

Causal prediction has become a popular tool for neuroscience applications, as it allows the study of relationships between different brain areas during rest, cognitive tasks or brain disorders. We propose a nonparametric approach for the estimation of nonlinear causal prediction for multivariate time series. In the proposed estimator, C NPMR , Autoregressive modeling is replaced by Nonparametric Multiplicative Regression (NPMR). NPMR quantifies interactions between a response variable (effect) and a set of predictor variables (cause); here, we modified NPMR for model prediction. We also demonstrate how a particular measure, the sensitivity Q, could be used to reveal the structure of the underlying causal relationships. We apply C NPMR on artificial data with known ground truth (5 datasets), as well as physiological data (2 datasets). C NPMR correctly identifies both linear and nonlinear causal connections that are present in the artificial data, as well as physiologically relevant connectivity in the real data, and does not seem to be affected by filtering. The Sensitivity measure also provides useful information about the latent connectivity.The proposed estimator addresses many of the limitations of linear Granger causality and other nonlinear causality estimators. C NPMR is compared with pairwise and conditional Granger causality (linear) and Kernel-Granger causality (nonlinear). The proposed estimator can be applied to pairwise or multivariate estimations without any modifications to the main method. Its nonpametric nature, its ability to capture nonlinear relationships and its robustness to filtering make it appealing for a number of applications.

Directed dynamical influence is more detectable with noise

Successful identification of directed dynamical influence in complex systems is relevant to significant problems of current interest. Traditional methods based on Granger causality and transfer entropy have issues such as difficulty with nonlinearity and large data requirement. Recently a framework based on nonlinear dynamical analysis was proposed to overcome these difficulties. We find, surprisingly, that noise can counterintuitively enhance the detectability of directed dynamical influence. In fact, intentionally injecting a proper amount of asymmetric noise into the available time series has the unexpected benefit of dramatically increasing confidence in ascertaining the directed dynamical influence in the underlying system. This result is established based on both real data and model time series from nonlinear ecosystems. We develop a physical understanding of the beneficial role of noise in enhancing detection of directed dynamical influence.

Relating Granger causality to long-term causal effects

In estimation of causal couplings between observed processes, it is important to characterize coupling roles at various time scales. The widely used Granger causality reflects short-term effects: it shows how strongly perturbations of a current state of one process affect near future states of another process, and it quantifies that via prediction improvement (PI) in autoregressive models. However, it is often more important to evaluate the effects of coupling on long-term statistics, e.g., to find out how strongly the presence of coupling changes the variance of a driven process as compared to an uncoupled case. No general relationships between Granger causality and such long-term effects are known. Here, we pose the problem of relating these two types of coupling characteristics, and we solve it for a class of stochastic systems. Namely, for overdamped linear oscillators, we rigorously derive that the above long-term effect is proportional to the short-term effects, with the proportionality coefficient depending on the prediction interval and relaxation times. We reveal that this coefficient is typically considerably greater than unity so that small normalized PI values may well correspond to quite large long-term effects of coupling. The applicability of the derived relationship to wider classes of systems, its limitations, and its value for further research are discussed. To give a real-world example, we analyze couplings between large-scale climatic processes related to sea surface temperature variations in equatorial Pacific and North Atlantic regions.

Neural networks with non-uniform embedding and explicit validation phase to assess Granger causality

A challenging problem when studying a dynamical system is to find the interdependencies among its individual components. Several algorithms have been proposed to detect directed dynamical influences between time series. Two of the most used approaches are a model-free one (transfer entropy) and a model-based one (Granger causality). Several pitfalls are related to the presence or absence of assumptions in modeling the relevant features of the data. We tried to overcome those pitfalls using a neural network approach in which a model is built without any a priori assumptions. In this sense this method can be seen as a bridge between model-free and model-based approaches. The experiments performed will show that the method presented in this work can detect the correct dynamical information flows occurring in a system of time series. Additionally we adopt a non-uniform embedding framework according to which only the past states that actually help the prediction are entered into the model, improving the prediction and avoiding the risk of overfitting. This method also leads to a further improvement with respect to traditional Granger causality approaches when redundant variables (i.e. variables sharing the same information about the future of the system) are involved. Neural networks are also able to recognize dynamics in data sets completely different from the ones used during the training phase.

A nonlinear generalization of spectral Granger causality

Spectral measures of linear Granger causality have been widely applied to study the causal connectivity between time series data in neuroscience, biology, and economics. Traditional Granger causality measures are based on linear autoregressive with exogenous (ARX) inputs models of time series data, which cannot truly reveal nonlinear effects in the data especially in the frequency domain. In this study, it is shown that the classical Geweke's spectral causality measure can be explicitly linked with the output spectra of corresponding restricted and unrestricted time-domain models. The latter representation is then generalized to nonlinear bivariate signals and for the first time nonlinear causality analysis in the frequency domain. This is achieved by using the nonlinear ARX (NARX) modeling of signals, and decomposition of the recently defined output frequency response function which is related to the NARX model.

Detecting causality from nonlinear dynamics with short-term time series

Quantifying causality between variables from observed time series data is of great importance in various disciplines but also a challenging task, especially when the observed data are short. Unlike the conventional methods, we find it possible to detect causality only with very short time series data, based on embedding theory of an attractor for nonlinear dynamics. Specifically, we first show that measuring the smoothness of a cross map between two observed variables can be used to detect a causal relation. Then, we provide a very effective algorithm to computationally evaluate the smoothness of the cross map, or "Cross Map Smoothness" (CMS), and thus to infer the causality, which can achieve high accuracy even with very short time series data. Analysis of both mathematical models from various benchmarks and real data from biological systems validates our method.

Gene network inference using continuous time Bayesian networks: a comparative study and application to Th17 cell differentiation

Background

Dynamic aspects of gene regulatory networks are typically investigated by measuring system variables at multiple time points. Current state-of-the-art computational approaches for reconstructing gene networks directly build on such data, making a strong assumption that the system evolves in a synchronous fashion at fixed points in time. However, nowadays omics data are being generated with increasing time course granularity. Thus, modellers now have the possibility to represent the system as evolving in continuous time and to improve the models’ expressiveness.

Results

Continuous time Bayesian networks are proposed as a new approach for gene network reconstruction from time course expression data. Their performance was compared to two state-of-the-art methods: dynamic Bayesian networks and Granger causality analysis. On simulated data, the methods comparison was carried out for networks of increasing size, for measurements taken at different time granularity densities and for measurements unevenly spaced over time. Continuous time Bayesian networks outperformed the other methods in terms of the accuracy of regulatory interactions learnt from data for all network sizes. Furthermore, their performance degraded smoothly as the size of the network increased. Continuous time Bayesian networks were significantly better than dynamic Bayesian networks for all time granularities tested and better than Granger causality for dense time series. Both continuous time Bayesian networks and Granger causality performed robustly for unevenly spaced time series, with no significant loss of performance compared to the evenly spaced case, while the same did not hold true for dynamic Bayesian networks. The comparison included the IRMA experimental datasets which confirmed the effectiveness of the proposed method. Continuous time Bayesian networks were then applied to elucidate the regulatory mechanisms controlling murine T helper 17 (Th17) cell differentiation and were found to be effective in discovering well-known regulatory mechanisms, as well as new plausible biological insights.

Conclusions

Continuous time Bayesian networks were effective on networks of both small and large size and were particularly feasible when the measurements were not evenly distributed over time. Reconstruction of the murine Th17 cell differentiation network using continuous time Bayesian networks revealed several autocrine loops, suggesting that Th17 cells may be auto regulating their own differentiation process.

Quantification of causal couplings via dynamical effects: a unifying perspective

Quantitative characterization of causal couplings from time series is crucial in studies of complex systems of different origin. Various statistical tools for that exist and new ones are still being developed with a tendency to creating a single, universal, model-free quantifier of coupling strength. However, a clear and generally applicable way of interpreting such universal characteristics is lacking. This work suggests a general conceptual framework for causal coupling quantification, which is based on state space models and extends the concepts of virtual interventions and dynamical causal effects. Namely, two basic kinds of interventions (state space and parametric) and effects (orbital or transient and stationary or limit) are introduced, giving four families of coupling characteristics. The framework provides a unifying view of apparently different well-established measures and allows us to introduce new characteristics, always with a definite "intervention-effect" interpretation. It is shown that diverse characteristics cannot be reduced to any single coupling strength quantifier and their interpretation is inevitably model based. The proposed set of dynamical causal effect measures quantifies different aspects of "how the coupling manifests itself in the dynamics," reformulating the very question about the "causal coupling strength."

A copula approach to assessing Granger causality

In neuroscience, as in many other fields of science and engineering, it is crucial to assess the causal interactions among multivariate time series. Granger causality has been increasingly used to identify causal influence between time series based on multivariate autoregressive models. Such an approach is based on linear regression framework with implicit Gaussian assumption of model noise residuals having constant variance. As a consequence, this measure cannot detect the cause-effect relationship in high-order moments and nonlinear causality. Here, we propose an effective model-free, copula-based Granger causality measure that can be used to reveal nonlinear and high-order moment causality. We first formulate Granger causality as the log-likelihood ratio in terms of conditional distribution, and then derive an efficient estimation procedure using conditional copula. We use resampling techniques to build a baseline null-hypothesis distribution from which statistical significance can be derived. We perform a series of simulations to investigate the performance of our copula-based Granger causality, and compare its performance against other state-of-the-art techniques. Our method is finally applied to neural field potential time series recorded from visual cortex of a monkey while performing a visual illusion task.

Correntropy-based partial directed coherence for testing multivariate Granger causality in nonlinear processes

Identification of directional influences in multivariate systems is of prime importance in several applications of engineering and sciences such as plant topology reconstruction, fault detection and diagnosis, and neurosciences. A spectrum of related directionality measures, ranging from linear measures such as partial directed coherence (PDC) to nonlinear measures such as transfer entropy, have emerged over the past two decades. The PDC-based technique is simple and effective, but being a linear directionality measure has limited applicability. On the other hand, transfer entropy, despite being a robust nonlinear measure, is computationally intensive and practically implementable only for bivariate processes. The objective of this work is to develop a nonlinear directionality measure, termed as KPDC, that possesses the simplicity of PDC but is still applicable to nonlinear processes. The technique is founded on a nonlinear measure called correntropy, a recently proposed generalized correlation measure. The proposed method is equivalent to constructing PDC in a kernel space where the PDC is estimated using a vector autoregressive model built on correntropy. A consistent estimator of the KPDC is developed and important theoretical results are established. A permutation scheme combined with the sequential Bonferroni procedure is proposed for testing hypothesis on absence of causality. It is demonstrated through several case studies that the proposed methodology effectively detects Granger causality in nonlinear processes.

Reliability of Dynamic Causal Modeling using the Statistical Parametric Mapping Toolbox

Dynamic causal modeling (DCM) is a recently developed approach for effective connectivity measurement in the brain. It has attracted considerable attention in recent years and quite widespread used to investigate brain connectivity in response to different tasks as well as auditory, visual, and somatosensory stimulation. This method uses complex algorithms, and currently the only implementation available is the Statistical Parametric Mapping (SPM8) toolbox with functionality for use on EEG and fMRI. The objective of the current work is to test the robustness of the toolbox when applied to EEG, by comparing results obtained from various versions of the software and operating systems when using identical datasets. Contrary to expectations, it was found that estimated connectivities were not consistent between different operating systems, the version of SPM8, or the version of MATLAB being used. The exact cause of this problem is not clear, but may relate to the high number of parameters in the model. Caution is thus recommended when interpreting the results of DCM estimated with the SPM8 software.