Inferring directional interactions from transient signals with symbolic transfer entropy

Time-series-analysis-based detection of critical transitions in real-world non-autonomous systems

Real-world non-autonomous systems are open, out-of-equilibrium systems that evolve in and are driven by temporally varying environments. Such systems can show multiple timescale and transient dynamics together with transitions to very different and, at times, even disastrous dynamical regimes. Since such critical transitions disrupt the systems' intended or desired functionality, it is crucial to understand the underlying mechanisms, to identify precursors of such transitions, and to reliably detect them in time series of suitable system observables to enable forecasts. This review critically assesses the various steps of investigation involved in time-series-analysis-based detection of critical transitions in real-world non-autonomous systems: from the data recording to evaluating the reliability of offline and online detections. It will highlight pros and cons to stimulate further developments, which would be necessary to advance understanding and forecasting nonlinear behavior such as critical transitions in complex systems.

Interpolation and sampling effects on recurrence quantification measures

The recurrence plot and the recurrence quantification analysis (RQA) are well-established methods for the analysis of data from complex systems. They provide important insights into the nature of the dynamics, periodicity, regime changes, and many more. These methods are used in different fields of research, such as finance, engineering, life, and earth science. To use them, the data have usually to be uniformly sampled, posing difficulties in investigations that provide non-uniformly sampled data, as typical in medical data (e.g., heart-beat based measurements), paleoclimate archives (such as sediment cores or stalagmites), or astrophysics (supernova or pulsar observations). One frequently used solution is interpolation to generate uniform time series. However, this preprocessing step can introduce bias to the RQA measures, particularly those that rely on the diagonal or vertical line structure in the recurrence plot. Using prototypical model systems, we systematically analyze differences in the RQA measure average diagonal line length for data with different sampling and interpolation. For real data, we show that the course of this measure strongly depends on the choice of the sampling rate for interpolation. Furthermore, we suggest a correction scheme, which is capable of correcting the bias introduced by the prepossessing step if the interpolation ratio is an integer.

Assessing cardiorespiratory interactions via lagged joint symbolic dynamics during spontaneous and controlled breathing

Introduction: Joint symbolic analysis (JSA) can be utilized to describe interactions between time series while accounting for time scales and nonlinear features. JSA is based on the computation of the rate of occurrence of joint patterns built after symbolization. Lagged JSA (LJSA) is obtained from the more classical JSA by introducing a delay/lead between patterns built over the two series and combined to form the joint scheme, thus monitoring coordinated patterns at different lags. Methods: In the present study, we applied LJSA for the assessment of cardiorespiratory coupling (CRC) from heart period (HP) variability and respiratory activity (R) in 19 healthy subjects (age: 27-35 years; 8 males, 11 females) during spontaneous breathing (SB) and controlled breathing (CB). The R rate of CB was selected to be indistinguishable from that of SB, namely, 15 breaths·minute-1 (CB15), or slower than SB, namely, 10 breaths·minute-1 (CB10), but in both cases, very rapid interactions between heart rate and R were known to be present. The ability of the LJSA approach to follow variations of the coupling strength was tested over a unidirectionally or bidirectionally coupled stochastic process and using surrogate data to test the null hypothesis of uncoupling. Results: We found that: i) the analysis of surrogate data proved that HP and R were significantly coupled in any experimental condition, and coupling was not more likely to occur at a specific time lag; ii) CB10 reduced CRC strength at the fastest time scales while increasing that at intermediate time scales, thus leaving the overall CRC strength unvaried; iii) despite exhibiting similar R rates and respiratory sinus arrhythmia, SB and CB15 induced different cardiorespiratory interactions; iv) no dominant temporal scheme was observed with relevant contributions of HP patterns either leading or lagging R. Discussion: LJSA is a useful methodology to explore HP-R dynamic interactions while accounting for time shifts and scales.

Ordinal methods for a characterization of evolving functional brain networks

Ordinal time series analysis is based on the idea to map time series to ordinal patterns, i.e., order relations between the values of a time series and not the values themselves, as introduced in 2002 by C. Bandt and B. Pompe. Despite a resulting loss of information, this approach captures meaningful information about the temporal structure of the underlying system dynamics as well as about properties of interactions between coupled systems. This-together with its conceptual simplicity and robustness against measurement noise-makes ordinal time series analysis well suited to improve characterization of the still poorly understood spatiotemporal dynamics of the human brain. This minireview briefly summarizes the state-of-the-art of uni- and bivariate ordinal time-series-analysis techniques together with applications in the neurosciences. It will highlight current limitations to stimulate further developments, which would be necessary to advance characterization of evolving functional brain networks.

A Fast and Efficient Ensemble Transfer Entropy and Applications in Neural Signals

The ensemble transfer entropy (TEensemble) refers to the transfer entropy estimated from an ensemble of realizations. Due to its time-resolved analysis, it is adapted to analyze the dynamic interaction between brain regions. However, in the traditional TEensemble, multiple sets of surrogate data should be used to construct the null hypothesis distribution, which dramatically increases the computational complexity. To reduce the computational cost, a fast, efficient TEensemble with a simple statistical test method is proposed here, in which just one set of surrogate data is involved. To validate the improved efficiency, the simulated neural signals are used to compare the characteristics of the novel TEensemble with those of the traditional TEensemble. The results show that the time consumption is reduced by two or three magnitudes in the novel TEensemble. Importantly, the proposed TEensemble could accurately track the dynamic interaction process and detect the strength and the direction of interaction robustly even in the presence of moderate noises. The novel TEensemble reaches its steady state with the increased samples, which is slower than the traditional method. Furthermore, the effectiveness of the novel TEensemble was verified in the actual neural signals. Accordingly, the TEensemble proposed in this work may provide a suitable way to investigate the dynamic interactions between brain regions.

Measuring the Coupling Direction between Neural Oscillations with Weighted Symbolic Transfer Entropy

Neural oscillations reflect rhythmic fluctuations in the synchronization of neuronal populations and play a significant role in neural processing. To further understand the dynamic interactions between different regions in the brain, it is necessary to estimate the coupling direction between neural oscillations. Here, we developed a novel method, termed weighted symbolic transfer entropy (WSTE), that combines symbolic transfer entropy (STE) and weighted probability distribution to measure the directionality between two neuronal populations. The traditional STE ignores the degree of difference between the amplitude values of a time series. In our proposed WSTE method, this information is picked up by utilizing a weighted probability distribution. The simulation analysis shows that the WSTE method can effectively estimate the coupling direction between two neural oscillations. In comparison with STE, the new method is more sensitive to the coupling strength and is more robust against noise. When applied to epileptic electrocorticography data, a significant coupling direction from the anterior nucleus of thalamus (ANT) to the seizure onset zone (SOZ) was detected during seizures. Considering the superiorities of the WSTE method, it is greatly advantageous to measure the coupling direction between neural oscillations and consequently characterize the information flow between different brain regions.

The Human Organism as an Integrated Interaction Network: Recent Conceptual and Methodological Challenges

The field of Network Physiology aims to advance our understanding of how physiological systems and sub-systems interact to generate a variety of behaviors and distinct physiological states, to optimize the organism's functioning, and to maintain health. Within this framework, which considers the human organism as an integrated network, vertices are associated with organs while edges represent time-varying interactions between vertices. Likewise, vertices may represent networks on smaller spatial scales leading to a complex mixture of interacting homogeneous and inhomogeneous networks of networks. Lacking adequate analytic tools and a theoretical framework to probe interactions within and among diverse physiological systems, current approaches focus on inferring properties of time-varying interactions-namely strength, direction, and functional form-from time-locked recordings of physiological observables. To this end, a variety of bivariate or, in general, multivariate time-series-analysis techniques, which are derived from diverse mathematical and physical concepts, are employed and the resulting time-dependent networks can then be further characterized with methods from network theory. Despite the many promising new developments, there are still problems that evade from a satisfactory solution. Here we address several important challenges that could aid in finding new perspectives and inspire the development of theoretic and analytical concepts to deal with these challenges and in studying the complex interactions between physiological systems.

Von Interaktionen zu Interaktionsnetzwerken: Zeitabhängige funktionelle Netzwerke am Beispiel der Epilepsie

Das menschliche Gehirn ist ein komplexes Netzwerk aus interagierenden nichtstationären Subsystemen (Netzwerk von Netzwerken), deren komplizierte räumlich-zeitliche Dynamiken bis heute nur unzureichend verstanden sind. Dabei versprechen aktuelle Entwicklungen im Bereich der Zeitreihenanalyse sowie der Theorie komplexer Netzwerke neue und verbesserte Einblicke in die Dynamiken von Hirnnetzwerken auf verschiedenen räumlich-zeitlichen Skalen. Wir geben einen Überblick über diese Entwicklungen und besprechen am Beispiel zeitabhängiger epileptischer Hirnnetzwerke Fortschritte im Verständnis von Hirndynamiken, die über multiple Skalen hinweg variieren.

Symbolic time series analysis of fNIRS signals in brain development assessment

OBJECTIVE

Assessing an infant's brain development remains a challenge for neuroscientists and pediatricians despite great technological advances. As a non-invasive neuroimaging tool, functional near-infrared spectroscopy (fNIRS) has great advantages in monitoring an infant's brain activity. To explore the dynamic features of hemodynamic changes in infants, in-pattern exponent (IPE), anti-pattern exponent (APE), as well as permutation cross-mutual information (PCMI) based on symbolic dynamics are proposed to measure the phase differences and coupling strength in oxyhemoglobin (HbO) and deoxyhemoglobin (Hb) signals from fNIRS.

APPROACH

First, simulated sinusoidal oscillation signals and four coupled nonlinear systems were employed for performance assessments. Hilbert transform based measurements of hemoglobin phase oxygenation and deoxygenation (hPod) and phase-locking index of hPod (hPodL) were calculated for comparison. Then, the IPE, APE and PCMI indices from resting state fNIRS data of preterm, term infants and adults were calculated to estimate the phase difference and coupling of HbO and Hb. All indices' performance was assessed by the degree of monotonicity (DoM). The box plots and coefficients of variation (CV) were employed to assess the measurements and robustness in the results.

MAIN RESULTS

In the simulation analysis, IPE and APE can distinguish the phase difference of two sinusoidal oscillation signals. Both hPodL and PCMI can track the strength of two coupled nonlinear systems. Compared to hPodL, the PCMI had higher DoM indices in measuring the coupling of two nonlinear systems. In the fNIRS data analysis, similar to hPod, the IPE and APE can distinguish preterm, term infants, and adults in 0.01-0.05 Hz, 0.05-0.1 Hz, and 0.01-0.1 Hz frequency bands, respectively. PCMI more effectively distinguished the term and preterm infants than hPodL in the 0.05-0.1 Hz frequency band. As symbolic time series measures, the IPE and APE were able to detect the brain developmental changes in subjects of different ages. PCMI can assess the resting-state HbO and Hb coupling changes across different developmental ages, which may reflect the metabolic and neurovascular development.

SIGNIFICANCE

The symbolic-based methodologies are promising measures for fNIRS in estimating the brain development, especially in assessing newborns' brain developmental status.

Joint dyadic action: Error correction by two persons works better than by one alone

We investigated how two people learn to coordinate their movement to achieve a joint goal. Pairs of participants oscillated a joystick with their dominant hand whilst looking at a common feedback, a Lissajous figure, where each participant controlled either the vertical or horizontal coordinate of a moving dot. In the absence of specific instructions, inter-personal coordination was highly variable, punctuated by intermittent phase locking. When participants were required to produce a circular Lissajous figure, coordination variability decreased while accuracy, transfer entropy and the incidence of stable coordinative solutions (fixed points, including bi-stability) increased as a function of practice trials. When one partner closed his/her eyes, so that the other one received the full control of error correction, the stability and accuracy of coordination decreased. A questionnaire showed that partners experienced the feeling of we-control. The results were interpreted in terms of a disturbance ∼ correction challenge: joint action is enhanced by having a flexibly adjusting co-actor rather than a more predictable, but not adjusting, partner. At transfer, partners were able to produce a new, never-practiced Lissajous pattern, evidencing the generalisability of joint learning.

The influence of filtering and downsampling on the estimation of transfer entropy

Transfer entropy (TE) provides a generalized and model-free framework to study Wiener-Granger causality between brain regions. Because of its nonparametric character, TE can infer directed information flow also from nonlinear systems. Despite its increasing number of applications in neuroscience, not much is known regarding the influence of common electrophysiological preprocessing on its estimation. We test the influence of filtering and downsampling on a recently proposed nearest neighborhood based TE estimator. Different filter settings and downsampling factors were tested in a simulation framework using a model with a linear coupling function and two nonlinear models with sigmoid and logistic coupling functions. For nonlinear coupling and progressively lower low-pass filter cut-off frequencies up to 72% false negative direct connections and up to 26% false positive connections were identified. In contrast, for the linear model, a monotonic increase was only observed for missed indirect connections (up to 86%). High-pass filtering (1 Hz, 2 Hz) had no impact on TE estimation. After low-pass filtering interaction delays were significantly underestimated. Downsampling the data by a factor greater than the assumed interaction delay erased most of the transmitted information and thus led to a very high percentage (67–100%) of false negative direct connections. Low-pass filtering increases the number of missed connections depending on the filters cut-off frequency. Downsampling should only be done if the sampling factor is smaller than the smallest assumed interaction delay of the analyzed network.

Symbolic dynamics of animal interaction

Since its introduction nearly two decades ago, transfer entropy has contributed to an improved understanding of cause-and-effect relationships in coupled dynamical systems from raw time series. In the context of animal behavior, transfer entropy might help explain the determinants of leadership in social groups and elucidate escape response to predator attacks. Despite its promise, the potential of transfer entropy in animal behavior is yet to be fully tested, and a number of technical challenges in information theory and statistics remain open. Here, we examine an alternative approach to the computation of transfer entropy based on symbolic dynamics. In this context, a symbol is associated with a specific locomotory bout across two or more consecutive time instants, such as reversing the swimming direction. Symbols encapsulate salient locomotory patterns and the associated permutation transfer entropy quantifies the ability to predict the patterns of an individual given those of another individual. We demonstrate this framework on an existing dataset on fish, for which we have knowledge of the underlying cause-and-effect relationship between the focal subject and the stimulus. Symbolic dynamics offers an intuitive and robust approach to study animal behavior, which could enable the inference of causal relationship from noisy experimental observations of limited duration.

Two step transfer entropy - An estimator of delayed directional couplings between multivariate EEG time series

Quantifying delayed directional couplings between electroencephalographic (EEG) time series requires an efficient method of causal network inference. This is especially due to the limited knowledge about the underlying dynamics of the brain activity. Recent methods based on information theoretic measures such as Transfer Entropy (TE) made significant progress on this issue by providing a model-free framework for causality detection. However, TE estimation from observed data is not a trivial task, especially when the number of variables is large which is the case in a highly complex system like human brain. Here we propose a computationally efficient procedure for TE estimation based on using sets of the Most Informative Variables that effectively contribute to resolving the uncertainty of the destination. In the first step of this method, some conditioning sets are determined through a nonlinear state space reconstruction; then in the second step, optimal estimation of TE is done based on these sets. Validation of the proposed method using synthetic data and neurophysiological signals demonstrates computational efficiency in quantifying delayed directional couplings compared with the common TE analysis.

Granger causal time-dependent source connectivity in the somatosensory network

Exploration of transient Granger causal interactions in neural sources of electrophysiological activities provides deeper insights into brain information processing mechanisms. However, the underlying neural patterns are confounded by time-dependent dynamics, non-stationarity and observational noise contamination. Here we investigate transient Granger causal interactions using source time-series of somatosensory evoked magnetoencephalographic (MEG) elicited by air puff stimulation of right index finger and recorded using 306-channel MEG from 21 healthy subjects. A new time-varying connectivity approach, combining renormalised partial directed coherence with state space modelling, is employed to estimate fast changing information flow among the sources. Source analysis confirmed that somatosensory evoked MEG was mainly generated from the contralateral primary somatosensory cortex (SI) and bilateral secondary somatosensory cortices (SII). Transient Granger causality shows a serial processing of somatosensory information, 1) from contralateral SI to contralateral SII, 2) from contralateral SI to ipsilateral SII, 3) from contralateral SII to contralateral SI, and 4) from contralateral SII to ipsilateral SII. These results are consistent with established anatomical connectivity between somatosensory regions and previous source modeling results, thereby providing empirical validation of the time-varying connectivity analysis. We argue that the suggested approach provides novel information regarding transient cortical dynamic connectivity, which previous approaches could not assess.

Assessing directionality and strength of coupling through symbolic analysis: an application to epilepsy patients

Inferring strength and direction of interactions from electroencephalographic (EEG) recordings is of crucial importance to improve our understanding of dynamical interdependencies underlying various physiological and pathophysiological conditions in the human epileptic brain. We here use approaches from symbolic analysis to investigate--in a time-resolved manner--weighted and directed, short- to long-ranged interactions between various brain regions constituting the epileptic network. Our observations point to complex spatial-temporal interdependencies underlying the epileptic process and their role in the generation of epileptic seizures, despite the massive reduction of the complex information content of multi-day, multi-channel EEG recordings through symbolization. We discuss limitations and potential future improvements of this approach.

Identifying delayed directional couplings with symbolic transfer entropy

We propose a straightforward extension of symbolic transfer entropy to enable the investigation of delayed directional relationships between coupled dynamical systems from time series. Analyzing time series from chaotic model systems, we demonstrate the applicability and limitations of our approach. Our findings obtained from applying our method to infer delayed directed interactions in the human epileptic brain underline the importance of our approach for improving the construction of functional network structures from data.

Quantifying 'causality' in complex systems: understanding transfer entropy

‘Causal’ direction is of great importance when dealing with complex systems. Often big volumes of data in the form of time series are available and it is important to develop methods that can inform about possible causal connections between the different observables. Here we investigate the ability of the Transfer Entropy measure to identify causal relations embedded in emergent coherent correlations. We do this by firstly applying Transfer Entropy to an amended Ising model. In addition we use a simple Random Transition model to test the reliability of Transfer Entropy as a measure of ‘causal’ direction in the presence of stochastic fluctuations. In particular we systematically study the effect of the finite size of data sets.

Estimating temporal causal interaction between spike trains with permutation and transfer entropy

Estimating the causal interaction between neurons is very important for better understanding the functional connectivity in neuronal networks. We propose a method called normalized permutation transfer entropy (NPTE) to evaluate the temporal causal interaction between spike trains, which quantifies the fraction of ordinal information in a neuron that has presented in another one. The performance of this method is evaluated with the spike trains generated by an Izhikevich’s neuronal model. Results show that the NPTE method can effectively estimate the causal interaction between two neurons without influence of data length. Considering both the precision of time delay estimated and the robustness of information flow estimated against neuronal firing rate, the NPTE method is superior to other information theoretic method including normalized transfer entropy, symbolic transfer entropy and permutation conditional mutual information. To test the performance of NPTE on analyzing simulated biophysically realistic synapses, an Izhikevich’s cortical network that based on the neuronal model is employed. It is found that the NPTE method is able to characterize mutual interactions and identify spurious causality in a network of three neurons exactly. We conclude that the proposed method can obtain more reliable comparison of interactions between different pairs of neurons and is a promising tool to uncover more details on the neural coding.

Disruption of frontal-parietal communication by ketamine, propofol, and sevoflurane

INTRODUCTION

Directional connectivity from anterior to posterior brain regions (or "feedback" connectivity) has been shown to be inhibited by propofol and sevoflurane. In this study the authors tested the hypothesis that ketamine would also inhibit cortical feedback connectivity in frontoparietal networks.

METHODS

Surgical patients (n = 30) were recruited for induction of anesthesia with intravenous ketamine (2 mg/kg); electroencephalography of the frontal and parietal regions was acquired. The authors used normalized symbolic transfer entropy, a computational method based on information theory, to measure directional connectivity across frontal and parietal regions. Statistical analysis of transfer entropy measures was performed with the permutation test and the time-shift test to exclude false-positive connectivity. For comparison, the authors used normalized symbolic transfer entropy to reanalyze electroencephalographic data gathered from surgical patients receiving either propofol (n = 9) or sevoflurane (n = 9) for anesthetic induction.

RESULTS

Ketamine reduced alpha power and increased gamma power, in contrast to both propofol and sevoflurane. During administration of ketamine, feedback connectivity gradually diminished and was significantly inhibited after loss of consciousness (mean ± SD of baseline and anesthesia: 0.0074 ± 0.003 and 0.0055 ± 0.0027; F(5, 179) = 7.785, P < 0.0001). By contrast, feedforward connectivity was preserved during exposure to ketamine (mean ± SD of baseline and anesthesia: 0.0041 ± 0.0015 and 0.0046 ± 0.0018; F(5, 179) = 2.07; P = 0.072). Like ketamine, propofol and sevoflurane selectively inhibited feedback connectivity after anesthetic induction.

CONCLUSIONS

Diverse anesthetics disrupt frontal-parietal communication, despite molecular and neurophysiologic differences. Analysis of directional connectivity in frontal-parietal networks could provide a common metric of general anesthesia and insight into the cognitive neuroscience of anesthetic-induced unconsciousness.

Connectivity and phase coherence in neural network models of interconnected Z(4)-bi-stable units

A phenomenological neural network model with bi-stable oscillatory units is used to model up- and down-states. These states have been observed in vivo in biological neuronal systems and feature oscillatory, limit cycle type of behavior in the up-states. A network is formed by a set of interconnected units. Two different types of network layouts are considered in this work: networks with hierarchical connections and hubs and networks with random connections. The phase coherence between the different units is analyzed and compared to the connectivity distance between nodes. In addition the connectivity degree of a node is associated to the average phase coherence with all other units. The results show that we may be able to identify the set of hubs in a network based on the phase coherence estimates between the different nodes. If the network is very dense or randomly connected, the underlying network structure, however, can not be derived uniquely from the phase coherence.

Order Patterns Networks (ORPAN)-a method to estimate time-evolving functional connectivity from multivariate time series

Complex networks provide an excellent framework for studying the function of the human brain activity. Yet estimating functional networks from measured signals is not trivial, especially if the data is non-stationary and noisy as it is often the case with physiological recordings. In this article we propose a method that uses the local rank structure of the data to define functional links in terms of identical rank structures. The method yields temporal sequences of networks which permits to trace the evolution of the functional connectivity during the time course of the observation. We demonstrate the potentials of this approach with model data as well as with experimental data from an electrophysiological study on language processing.

Connectivity measures applied to human brain electrophysiological data

Connectivity measures are (typically bivariate) statistical measures that may be used to estimate interactions between brain regions from electrophysiological data. We review both formal and informal descriptions of a range of such measures, suitable for the analysis of human brain electrophysiological data, principally electro- and magnetoencephalography. Methods are described in the space-time, space-frequency, and space-time-frequency domains. Signal processing and information theoretic measures are considered, and linear and nonlinear methods are distinguished. A novel set of cross-time-frequency measures is introduced, including a cross-time-frequency phase synchronization measure.

Computational model prospective on the observation of proictal states in epileptic neuronal systems

Epilepsy is a pathological condition of the human central nervous system in which normal brain functions are impaired by unexpected transitions to states called seizures. We developed a lumped neuronal model that has the property of switching between two states as a result of intrinsic or extrinsic perturbations, such as noisy fluctuations. In one version of the model, seizure risk is controlled by a single connectivity parameter representing excitatory couplings between two model lumps. We show that this risk can be reconstructed from calculation of the cross-covariance between the activities of the two neural populations during the nonictal phase. In a second simulation sequence, we use a system of 10 interconnected lumps with randomly generated connectivity matrices. We show again that the tendency to develop seizures can be inferred from the cross-covariances calculated during the nonictal states. Our conclusion is that the risk of epileptic transitions in biological systems can be objectively quantified. This article is part of a Supplemental Special Issue entitled The Future of Automated Seizure Detection and Prediction.

Assessing directed interactions from neurophysiological signals--an overview

The study of synchronization phenomena in coupled dynamical systems is an active field of research in many scientific disciplines including the neurosciences. Over the last decades, a number of time series analysis techniques have been proposed to capture both linear and nonlinear aspects of interactions. While most of these techniques allow one to quantify the strength of interactions, developments that resulted from advances in nonlinear dynamics and in information and synchronization theory aim at assessing directed interactions. Most of these techniques, however, assume the underlying systems to be at least approximately stationary and require a large number of data points to robustly assess directed interactions. Recent extensions allow assessing directed interactions from short and transient signals and are particularly suited for the analysis of evoked and event-related activity.