Permutation-information-theory approach to unveil delay dynamics from time-series analysis

Quantifying Deviations from Gaussianity with Application to Flight Delay Distributions

We propose a novel approach for quantifying deviations from Gaussianity by leveraging the Jensen-Shannon distance. Using stable distributions as a flexible framework, we analyze the effects of skewness and heavy tails in synthetic sequences. We employ phase-randomized surrogates as Gaussian references to systematically evaluate the statistical distance between this reference and stable distributions. Our methodology is validated using real flight delay datasets from major airports in Europe and the United States, revealing significant deviations from Gaussianity, particularly at high-traffic airports. These results highlight systematic air traffic management strategy differences between the two geographic regions.

Evaluating Methods for Detrending Time Series Using Ordinal Patterns, with an Application to Air Transport Delays

Functional networks have become a standard tool for the analysis of complex systems, allowing the unveiling of their internal connectivity structure while only requiring the observation of the system's constituent dynamics. To obtain reliable results, one (often overlooked) prerequisite involves the stationarity of an analyzed time series, without which spurious functional connections may emerge. Here, we show how ordinal patterns and metrics derived from them can be used to assess the effectiveness of detrending methods. We apply this approach to data representing the evolution of delays in major European and US airports, and to synthetic versions of the same, obtaining operational conclusions about how these propagate in the two systems.

Local Predictors of Explosive Synchronization with Ordinal Methods

We propose using the ordinal pattern transition (OPT) entropy measured at sentinel central nodes as a potential predictor of explosive transitions to synchronization in networks of various dynamical systems with increasing complexity. Our results demonstrate that the OPT entropic measure surpasses traditional early warning signal (EWS) measures and could be valuable to the tools available for predicting critical transitions. In particular, we investigate networks of diffusively coupled phase oscillators and chaotic Rössler systems. As maps, we consider a neural network of Chialvo maps coupled in star and scale-free configurations. Furthermore, we apply this measure to time series data obtained from a network of electronic circuits operating in the chaotic regime.

Rényi entropy-complexity causality space: a novel neurocomputational tool for detecting scale-free features in EEG/iEEG data

Scale-free brain activity, linked with learning, the integration of different time scales, and the formation of mental models, is correlated with a metastable cognitive basis. The spectral slope, a key aspect of scale-free dynamics, was proposed as a potential indicator to distinguish between different sleep stages. Studies suggest that brain networks maintain a consistent scale-free structure across wakefulness, anesthesia, and recovery. Although differences in anesthetic sensitivity between the sexes are recognized, these variations are not evident in clinical electroencephalographic recordings of the cortex. Recently, changes in the slope of the power law exponent of neural activity were found to correlate with changes in Rényi entropy, an extended concept of Shannon's information entropy. These findings establish quantifiers as a promising tool for the study of scale-free dynamics in the brain. Our study presents a novel visual representation called the Rényi entropy-complexity causality space, which encapsulates complexity, permutation entropy, and the Rényi parameter q. The main goal of this study is to define this space for classical dynamical systems within theoretical bounds. In addition, the study aims to investigate how well different time series mimicking scale-free activity can be discriminated. Finally, this tool is used to detect dynamic features in intracranial electroencephalography (iEEG) signals. To achieve these goals, the study implementse the Bandt and Pompe method for ordinal patterns. In this process, each signal is associated with a probability distribution, and the causal measures of Rényi entropy and complexity are computed based on the parameter q. This method is a valuable tool for analyzing simulated time series. It effectively distinguishes elements of correlated noise and provides a straightforward means of examining differences in behaviors, characteristics, and classifications. For the iEEG experimental data, the REM state showed a greater number of significant sex-based differences, while the supramarginal gyrus region showed the most variation across different modes and analyzes. Exploring scale-free brain activity with this framework could provide valuable insights into cognition and neurological disorders. The results may have implications for understanding differences in brain function between the sexes and their possible relevance to neurological disorders.

Characterizing unstructured data with the nearest neighbor permutation entropy

Permutation entropy and its associated frameworks are remarkable examples of physics-inspired techniques adept at processing complex and extensive datasets. Despite substantial progress in developing and applying these tools, their use has been predominantly limited to structured datasets such as time series or images. Here, we introduce the k-nearest neighbor permutation entropy, an innovative extension of the permutation entropy tailored for unstructured data, irrespective of their spatial or temporal configuration and dimensionality. Our approach builds upon nearest neighbor graphs to establish neighborhood relations and uses random walks to extract ordinal patterns and their distribution, thereby defining the k-nearest neighbor permutation entropy. This tool not only adeptly identifies variations in patterns of unstructured data but also does so with a precision that significantly surpasses conventional measures such as spatial autocorrelation. Additionally, it provides a natural approach for incorporating amplitude information and time gaps when analyzing time series or images, thus significantly enhancing its noise resilience and predictive capabilities compared to the usual permutation entropy. Our research substantially expands the applicability of ordinal methods to more general data types, opening promising research avenues for extending the permutation entropy toolkit for unstructured data.

Quantifying the diversity of multiple time series with an ordinal symbolic approach

The main motivation of this paper is to introduce the ordinal diversity, a symbolic tool able to quantify the degree of diversity of multiple time series. Analytical, numerical, and experimental analyses illustrate the utility of this measure to quantify how diverse, from an ordinal perspective, a set of many time series is. We have shown that ordinal diversity is able to characterize dynamical richness and dynamical transitions in stochastic processes and deterministic systems, including chaotic regimes. This ordinal tool also serves to identify optimal operating conditions in the machine learning approach of reservoir computing. These results allow us to envision potential applications for the handling and characterization of large amounts of data, paving the way for addressing some of the most pressing issues facing the current big data paradigm.

Probing Intrinsic Neural Timescales in EEG with an Information-Theory Inspired Approach: Permutation Entropy Time Delay Estimation (PE-TD)

Time delays are a signature of many physical systems, including the brain, and considerably shape their dynamics; moreover, they play a key role in consciousness, as postulated by the temporo-spatial theory of consciousness (TTC). However, they are often not known a priori and need to be estimated from time series. In this study, we propose the use of permutation entropy (PE) to estimate time delays from neural time series as a more robust alternative to the widely used autocorrelation window (ACW). In the first part, we demonstrate the validity of this approach on synthetic neural data, and we show its resistance to regimes of nonstationarity in time series. Mirroring yet another example of comparable behavior between different nonlinear systems, permutation entropy-time delay estimation (PE-TD) is also able to measure intrinsic neural timescales (INTs) (temporal windows of neural activity at rest) from hd-EEG human data; additionally, this replication extends to the abnormal prolongation of INT values in disorders of consciousness (DoCs). Surprisingly, the correlation between ACW-0 and PE-TD decreases in a state-dependent manner when consciousness is lost, hinting at potential different regimes of nonstationarity and nonlinearity in conscious/unconscious states, consistent with many current theoretical frameworks on consciousness. In summary, we demonstrate the validity of PE-TD as a tool to extract relevant time scales from neural data; furthermore, given the divergence between ACW and PE-TD specific to DoC subjects, we hint at its potential use for the characterization of conscious states.

Rao-Burbea centroids applied to the statistical characterization of time series and images through ordinal patterns

Divergences or similarity measures between probability distributions have become a very useful tool for studying different aspects of statistical objects, such as time series, networks, and images. Notably, not every divergence provides identical results when applied to the same problem. Therefore, it seems convenient to have the widest possible set of divergences to be applied to the problems under study. Besides this choice, an essential step in the analysis of every statistical object is the mapping of each one of their representing values into an alphabet of symbols conveniently chosen. In this work, we choose the family of divergences known as the Burbea-Rao centroids (BRCs). For the mapping of the original time series into a symbolic sequence, we work with the ordinal pattern scheme. We apply our proposals to analyze simulated and real time series and to real textured images. The main conclusion of our work is that the best BRC, at least in the studied cases, is the Jensen-Shannon divergence, besides the fact that it verifies some interesting formal properties.

Markov-modulated model for landing flow dynamics: An ordinal analysis validation

Air transportation is a complex system characterized by a plethora of interactions at multiple temporal and spatial scales; as a consequence, even simple dynamics like sequencing aircraft for landing can lead to the appearance of emergent behaviors, which are both difficult to control and detrimental to operational efficiency. We propose a model, based on a modulated Markov jitter, to represent ordinal pattern properties of real landing operations in European airports. The parameters of the model are tuned by minimizing the distance between the probability distributions of ordinal patterns generated by the real and synthetic sequences, as estimated by the Permutation Jensen-Shannon Distance. We show that the correlation between consecutive hours in the landing flow changes between airports and that it can be interpreted as a metric of efficiency. We further compare the dynamics pre and post COVID-19, showing how this has changed beyond what can be attributed to a simple reduction of traffic. We finally draw some operational conclusions and discuss the applicability of these findings in a real operational environment.

Complexity of Brain Dynamics as a Correlate of Consciousness in Anaesthetized Monkeys

The use of anaesthesia is a fundamental tool in the investigation of consciousness. Anesthesia procedures allow to investigate different states of consciousness from sedation to deep anesthesia within controlled scenarios. In this study we use information quantifiers to measure the complexity of electrocorticogram recordings in monkeys. We apply these metrics to compare different stages of general anesthesia for evaluating consciousness in several anesthesia protocols. We find that the complexity of brain activity can be used as a correlate of consciousness. For two of the anaesthetics used, propofol and medetomidine, we find that the anaesthetised state is accompanied by a reduction in the complexity of brain activity. On the other hand we observe that use of ketamine produces an increase in complexity measurements. We relate this observation with increase activity within certain brain regions associated with the ketamine used doses. Our measurements indicate that complexity of brain activity is a good indicator for a general evaluation of different levels of consciousness awareness, both in anesthetized and non anesthetizes states.

Spike Timing-Dependent Plasticity with Enhanced Long-Term Depression Leads to an Increase of Statistical Complexity

Synaptic plasticity is characterized by remodeling of existing synapses caused by strengthening and/or weakening of connections. This is represented by long-term potentiation (LTP) and long-term depression (LTD). The occurrence of a presynaptic spike (or action potential) followed by a temporally nearby postsynaptic spike induces LTP; conversely, if the postsynaptic spike precedes the presynaptic spike, it induces LTD. This form of synaptic plasticity induction depends on the order and timing of the pre- and postsynaptic action potential, and has been termed spike time-dependent plasticity (STDP). After an epileptic seizure, LTD plays an important role as a depressor of synapses, which may lead to their complete disappearance together with that of their neighboring connections until days after the event. Added to the fact that after an epileptic seizure the network seeks to regulate the excess activity through two key mechanisms: depressed connections and neuronal death (eliminating excitatory neurons from the network), LTD becomes of great interest in our study. To investigate this phenomenon, we develop a biologically plausible model that privileges LTD at the triplet level while maintaining the pairwise structure in the STPD and study how network dynamics are affected as neuronal damage increases. We find that the statistical complexity is significantly higher for the network where LTD presented both types of interactions. While in the case where the STPD is defined with purely pairwise interactions an increase is observed as damage becomes higher for both Shannon Entropy and Fisher information.

On the Genuine Relevance of the Data-Driven Signal Decomposition-Based Multiscale Permutation Entropy

Ordinal pattern-based approaches have great potential to capture intrinsic structures of dynamical systems, and therefore, they continue to be developed in various research fields. Among these, the permutation entropy (PE), defined as the Shannon entropy of ordinal probabilities, is an attractive time series complexity measure. Several multiscale variants (MPE) have been proposed in order to bring out hidden structures at different time scales. Multiscaling is achieved by combining linear or nonlinear preprocessing with PE calculation. However, the impact of such a preprocessing on the PE values is not fully characterized. In a previous study, we have theoretically decoupled the contribution of specific signal models to the PE values from that induced by the inner correlations of linear preprocessing filters. A variety of linear filters such as the autoregressive moving average (ARMA), Butterworth, and Chebyshev were tested. The current work is an extension to nonlinear preprocessing and especially to data-driven signal decomposition-based MPE. The empirical mode decomposition, variational mode decomposition, singular spectrum analysis-based decomposition and empirical wavelet transform are considered. We identify possible pitfalls in the interpretation of PE values induced by these nonlinear preprocessing, and hence, we contribute to improving the PE interpretation. The simulated dataset of representative processes such as white Gaussian noise, fractional Gaussian processes, ARMA models and synthetic sEMG signals as well as real-life sEMG signals are tested.

High-frequency oscillations in the ripple bands and amplitude information coding: Toward a biomarker of maximum entropy in the preictal signals

Intracranial electroencephalography (iEEG) can directly record local field potentials (LFPs) from a large set of neurons in the vicinity of the electrode. To search for possible epileptic biomarkers and to determine the epileptogenic zone that gives rise to seizures, we investigated the dynamics of basal and preictal signals. For this purpose, we explored the dynamics of the recorded time series for different frequency bands considering high-frequency oscillations (HFO) up to 240 Hz. We apply a Hilbert transform to study the amplitude and phase of the signals. The dynamics of the different frequency bands in the time causal entropy-complexity plane, H × C, is characterized by comparing the dynamical evolution of the basal and preictal time series. As the preictal states evolve closer to the time in which the epileptic seizure starts, the, H × C, dynamics changes for the higher frequency bands. The complexity evolves to very low values and the entropy becomes nearer to its maximal value. These quasi-stable states converge to equiprobable states when the entropy is maximal, and the complexity is zero. We could, therefore, speculate that in this case, it corresponds to the minimization of Gibbs free energy. In this case, the maximum entropy is equivalent to the principle of minimum consumption of resources in the system. We can interpret this as the nature of the system evolving temporally in the preictal state in such a way that the consumption of resources by the system is minimal for the amplitude in frequencies between 220-230 and 230-240 Hz.

Dynamical invariants and inverse period-doubling cascades in multi-delay systems

We investigate transitions to simple dynamics in first-order nonlinear differential equations with multiple delays. With a proper choice of parameters, a single delay can destabilize a fixed point. In contrast, multiple delays can both destabilize fixed points and promote high-dimensional chaos but also induce stabilization onto simpler dynamics. We show that the dynamics of these systems depend on the precise distribution of the delays. Narrow spacing between individual delays induces chaotic behavior, while a lower density of delays enables stable periodic or fixed point behavior. As the dynamics become simpler, the number of unstable roots of the characteristic equation around the fixed point decreases. In fact, the behavior of these roots exhibits an astonishing parallel with that of the Lyapunov exponents and the Kolmogorov-Sinai entropy for these multi-delay systems. A theoretical analysis shows how these roots move back toward stability as the number of delays increases. Our results are based on numerical determination of the Lyapunov spectrum for these multi-delay systems as well as on permutation entropy computations. Finally, we report how complexity reduction upon adding more delays can occur through an inverse period-doubling sequence.

Evaluating Temporal Correlations in Time Series Using Permutation Entropy, Ordinal Probabilities and Machine Learning

Time series analysis comprises a wide repertoire of methods for extracting information from data sets. Despite great advances in time series analysis, identifying and quantifying the strength of nonlinear temporal correlations remain a challenge. We have recently proposed a new method based on training a machine learning algorithm to predict the temporal correlation parameter, α, of flicker noise (FN) time series. The algorithm is trained using as input features the probabilities of ordinal patterns computed from FN time series, xαFN(t), generated with different values of α. Then, the ordinal probabilities computed from the time series of interest, x(t), are used as input features to the trained algorithm and that returns a value, αe, that contains meaningful information about the temporal correlations present in x(t). We have also shown that the difference, Ω, of the permutation entropy (PE) of the time series of interest, x(t), and the PE of a FN time series generated with α=αe, xαeFN(t), allows the identification of the underlying determinism in x(t). Here, we apply our methodology to different datasets and analyze how αe and Ω correlate with well-known quantifiers of chaos and complexity. We also discuss the limitations for identifying determinism in highly chaotic time series and in periodic time series contaminated by noise. The open source algorithm is available on Github.

Time-Delay Identification Using Multiscale Ordinal Quantifiers

Time-delayed interactions naturally appear in a multitude of real-world systems due to the finite propagation speed of physical quantities. Often, the time scales of the interactions are unknown to an external observer and need to be inferred from time series of observed data. We explore, in this work, the properties of several ordinal-based quantifiers for the identification of time-delays from time series. To that end, we generate artificial time series of stochastic and deterministic time-delay models. We find that the presence of a nonlinearity in the generating model has consequences for the distribution of ordinal patterns and, consequently, on the delay-identification qualities of the quantifiers. Here, we put forward a novel ordinal-based quantifier that is particularly sensitive to nonlinearities in the generating model and compare it with previously-defined quantifiers. We conclude from our analysis on artificially generated data that the proper identification of the presence of a time-delay and its precise value from time series benefits from the complementary use of ordinal-based quantifiers and the standard autocorrelation function. We further validate these tools with a practical example on real-world data originating from the North Atlantic Oscillation weather phenomenon.

ordpy: A Python package for data analysis with permutation entropy and ordinal network methods

Since Bandt and Pompe's seminal work, permutation entropy has been used in several applications and is now an essential tool for time series analysis. Beyond becoming a popular and successful technique, permutation entropy inspired a framework for mapping time series into symbolic sequences that triggered the development of many other tools, including an approach for creating networks from time series known as ordinal networks. Despite increasing popularity, the computational development of these methods is fragmented, and there were still no efforts focusing on creating a unified software package. Here, we present ordpy (http://github.com/arthurpessa/ordpy), a simple and open-source Python module that implements permutation entropy and several of the principal methods related to Bandt and Pompe's framework to analyze time series and two-dimensional data. In particular, ordpy implements permutation entropy, Tsallis and Rényi permutation entropies, complexity-entropy plane, complexity-entropy curves, missing ordinal patterns, ordinal networks, and missing ordinal transitions for one-dimensional (time series) and two-dimensional (images) data as well as their multiscale generalizations. We review some theoretical aspects of these tools and illustrate the use of ordpy by replicating several literature results.

Time delay estimation from the time series for optical chaos systems using deep learning

We propose a model-free time delay signature (TDS) extraction method for optical chaos systems. The TDS can be identified from time series without prior knowledge of the actual physical processes. In optical chaos secure communication systems, the chaos carrier is usually generated by a laser diode subject to opto-electronic/all-optical time delayed feedback. One of the most important factors to security considerations is the concealment of the TDS. So far, statistical analysis methods such as autocorrelation function (ACF) and delayed mutual information (DMI) are usually used to unveil the TDS. However, the effectiveness of these methods will be reduced when increasing the nonlinearity of chaos systems. Meanwhile, certain TDS concealment strategies have been designed against statistical analysis. In our previous work, convolutional neural network shows its effectiveness on TDS extraction of chaos systems with high loop nonlinearity. However, this method relies on the knowledge of detailed structure of the chaos systems. In this work, we formulate a blind identification method based on long short-term memory neural network (LSTM-NN) model. The method is validated against the two major types of optical chaos systems, i.e. opto-electronic oscillator (OEO) chaos system and laser chaos system based on internal nonlinearity. Moreover, some security enhanced chaotic systems are also studied. The results show that the proposed method has high tolerance to additive noise. Meanwhile, the data amount needed is less than existing methods.

Time-delay signature suppression of polarization-resolved wideband chaos in VCSELs with dual-path chaotic optical injections

The combining investigation on the time-delay signature (TDS) and chaos bandwidth have been theoretically investigated in a vertical-cavity surface-emitting laser (VCSEL) system with dual-path chaotic optical injections. In this scheme, the polarized chaos with the TDS from an external-cavity master VCSEL is routed into two different paths and then unidirectionally injected into another solitary slave VCSEL. With the aid of the autocorrelation function and the effective bandwidth calculation, the TDS and bandwidth of polarized chaos from the chaotic system are quantitatively evaluated. The results show that, in such a dual-path chaotic optical-injection system, the high-quality polarized chaos with the successful TDS suppression and chaotic bandwidth enhancement can be achieved in wider parameter regions in contrast with the case for the single-path chaotic optical injection. Further research also finds that the injected time-delay difference between two injection paths is desired to mismatch the feedback time delay, which is conducive to suppressing TDS and expanding bandwidth of polarized chaos. Besides, the better chaotic quality with low TDS and wide bandwidth can be expected by choosing the appropriate injection strengths of two injection paths.

Contrasting chaotic with stochastic dynamics via ordinal transition networks

We introduce a representation space to contrast chaotic with stochastic dynamics. Following the complex network representation of a time series through ordinal pattern transitions, we propose to assign each system a position in a two-dimensional plane defined by the permutation entropy of the network (global network quantifier) and the minimum value of the permutation entropy of the nodes (local network quantifier). The numerical analysis of representative chaotic maps and stochastic systems shows that the proposed approach is able to distinguish linear from non-linear dynamical systems by different planar locations. Additionally, we show that this characterization is robust when observational noise is considered. Experimental applications allow us to validate the numerical findings and to conclude that this approach is useful in practical contexts.

Reconstruction of parameters and unobserved variables of a semiconductor laser with optical feedback from intensity time series

We propose a method for the reconstruction of time-delayed feedback systems having unobserved variables from scalar time series. The method is based on the modified initial condition approach, which allows one to significantly reduce the number of starting guesses for an unobserved variable with a time delay. The proposed method is applied to the reconstruction of the Lang-Kobayashi equations, which describe the dynamics of a single-mode semiconductor laser with external optical feedback. We consider the case where only the time series of laser intensity is observable and the other two variables of the model are hidden. The dependence of the quality of the system reconstruction on the accuracy of assignment of starting guesses for unobserved variables and unknown laser parameters is studied. The method could be used for testing the security of information transmission in laser-based chaotic communication systems.

Unveil the time delay signature of optical chaos systems with a convolutional neural network

We propose a time delay signature extraction method for optical chaos systems based on a convolutional neural network. Through transforming the time delay signature of a one-dimensional time series into two-dimensional image features, the excellent ability of convolutional neural networks for image feature recognition is fully utilized. The effectiveness of the method is verified on chaos systems with opto-electronic feedback and all optical feedback. The recognition accuracy of the method is 100% under normal conditions. For the system with extremely strong nonlinearity, the accuracy can be 93.25%, and the amount of data required is less than traditional methods. Moreover, it is verified that the proposed method possesses a strong ability to withstand the effects of noise.

On the automatic parameter selection for permutation entropy

Permutation Entropy (PE) is a cost effective tool for summarizing the complexity of a time series. It has been used in many applications including damage detection, disease forecasting, detection of dynamical changes, and financial volatility analysis. However, to successfully use PE, an accurate selection of two parameters is needed: the permutation dimension n and embedding delay τ. These parameters are often suggested by experts based on a heuristic or by a trial and error approach. Both of these methods can be time-consuming and lead to inaccurate results. In this work, we investigate multiple schemes for automatically selecting these parameters with only the corresponding time series as the input. Specifically, we develop a frequency-domain approach based on the least median of squares and the Fourier spectrum, as well as extend two existing methods: Permutation Auto-Mutual Information Function and Multi-scale Permutation Entropy (MPE) for determining τ. We then compare our methods as well as current methods in the literature for obtaining both τ and n against expert-suggested values in published works. We show that the success of any method in automatically generating the correct PE parameters depends on the category of the studied system. Specifically, for the delay parameter τ, we show that our frequency approach provides accurate suggestions for periodic systems, nonlinear difference equations, and electrocardiogram/electroencephalogram data, while the mutual information function computed using adaptive partitions provides the most accurate results for chaotic differential equations. For the permutation dimension n, both False Nearest Neighbors and MPE provide accurate values for n for most of the systems with a value of n=5 being suitable in most cases.

Simultaneous bandwidth-enhanced and time delay signature-suppressed chaos generation in semiconductor laser subject to feedback from parallel coupling ring resonators

We propose and demonstrate an external-feedback semiconductor laser-based chaos generation scheme supporting simultaneous bandwidth enhancement and excellent time-delay-signature (TDS) suppression, by using parallel-coupling ring resonators (PCRR) as reflector. The characteristics of effective bandwidth and TDS of chaotic signals generated in three indicative PCRR configurations are thoroughly investigated. The numerical results demonstrate that with the nonlinear feedback of PCRR, the TDS of chaos can be efficiently suppressed toward an indistinguishable level, and the bandwidth of chaos in the proposed scheme can also be enhanced, with respect to the conventional optical feedback configuration. The proposed scheme shows a flexible way to generate wideband complex chaos.

Permutation Entropy and Statistical Complexity in Mild Cognitive Impairment and Alzheimer's Disease: An Analysis Based on Frequency Bands

We present one of the first applications of Permutation Entropy (PE) and Statistical Complexity (SC) (measured as the product of PE and Jensen-Shanon Divergence) on Magnetoencephalography (MEG) recordings of 46 subjects suffering from Mild Cognitive Impairment (MCI), 17 individuals diagnosed with Alzheimer's Disease (AD) and 48 healthy controls. We studied the differences in PE and SC in broadband signals and their decomposition into frequency bands ( δ , θ , α and β ), considering two modalities: (i) raw time series obtained from the magnetometers and (ii) a reconstruction into cortical sources or regions of interest (ROIs). We conducted our analyses at three levels: (i) at the group level we compared SC in each frequency band and modality between groups; (ii) at the individual level we compared how the [PE, SC] plane differs in each modality; and (iii) at the local level we explored differences in scalp and cortical space. We recovered classical results that considered only broadband signals and found a nontrivial pattern of alterations in each frequency band, showing that SC does not necessarily decrease in AD or MCI.

Complexity Changes in the US and China's Stock Markets: Differences, Causes, and Wider Social Implications

How different are the emerging and the well-developed stock markets in terms of efficiency? To gain insights into this question, we compared an important emerging market, the Chinese stock market, and the largest and the most developed market, the US stock market. Specifically, we computed the Lempel–Ziv complexity (LZ) and the permutation entropy (PE) from two composite stock indices, the Shanghai stock exchange composite index (SSE) and the Dow Jones industrial average (DJIA), for both low-frequency (daily) and high-frequency (minute-to-minute)stock index data. We found that the US market is basically fully random and consistent with efficient market hypothesis (EMH), irrespective of whether low- or high-frequency stock index data are used. The Chinese market is also largely consistent with the EMH when low-frequency data are used. However, a completely different picture emerges when the high-frequency stock index data are used, irrespective of whether the LZ or PE is computed. In particular, the PE decreases substantially in two significant time windows, each encompassing a rapid market rise and then a few gigantic stock crashes. To gain further insights into the causes of the difference in the complexity changes in the two markets, we computed the Hurst parameter H from the high-frequency stock index data of the two markets and examined their temporal variations. We found that in stark contrast with the US market, whose H is always close to 1/2, which indicates fully random behavior, for the Chinese market, H deviates from 1/2 significantly for time scales up to about 10 min within a day, and varies systemically similar to the PE for time scales from about 10 min to a day. This opens the door for large-scale collective behavior to occur in the Chinese market, including herding behavior and large-scale manipulation as a result of inside information.

Decreased electrocortical temporal complexity distinguishes sleep from wakefulness

In most mammals, the sleep-wake cycle is constituted by three behavioral states: wakefulness (W), non-REM (NREM) sleep, and REM sleep. These states are associated with drastic changes in cognitive capacities, mostly determined by the function of the thalamo-cortical system. The intra-cranial electroencephalogram or electocorticogram (ECoG), is an important tool for measuring the changes in the thalamo-cortical activity during W and sleep. In the present study we analyzed broad-band ECoG recordings of the rat by means of a time-series complexity measure that is easy to implement and robust to noise: the Permutation Entropy (PeEn). We found that PeEn is maximal during W and decreases during sleep. These results bring to light the different thalamo-cortical dynamics emerging during sleep-wake states, which are associated with the well-known spectral changes that occur when passing from W to sleep. Moreover, the PeEn analysis allows us to determine behavioral states independently of the electrodes' cortical location, which points to an underlying global pattern in the signal that differs among the cycle states that is missed by classical methods. Consequently, our data suggest that PeEn analysis of a single EEG channel could allow for cheap, easy, and efficient sleep monitoring.

Unraveling the decay of the number of unobserved ordinal patterns in noisy chaotic dynamics

In this paper, we introduce a model to describe the decay of the number of unobserved ordinal patterns as a function of the time series length in noisy chaotic dynamics. More precisely, we show that a stretched exponential model fits the decay of the number of unobserved ordinal patterns for both discrete and continuous chaotic systems contaminated with observational noise, independently of the noise level and the sampling time. Numerical simulations, obtained from the logistic map and the x coordinate of the Lorenz system, both operating in a totally chaotic dynamics were used as test beds. In addition, we contrast our results with those obtained from pure stochastic dynamics. The fitting parameters, namely, the stretching exponent and the characteristic decay rate, are used to distinguish whether the dynamical nature of the data sequence is stochastic or chaotic. Finally, the analysis of experimental records associated with the hyperchaotic pulsations of an optoelectronic oscillator allows us to illustrate the applicability of the proposed approach in a practical context.

Isochronous cluster synchronization in delay-coupled VCSEL networks subjected to variable-polarization optical injection with time delay signature suppression

The isochronous cluster synchronization with time delay (TD) signature suppression in delay-coupled vertical-cavity surface-emitting laser (VCSEL) networks subject to variable-polarization optical injection (VPOI) is theoretically and numerically studied. Based on the inherent symmetries of network topology, parameter spaces for stable cluster synchronization are presented, and zero-lag synchronization are achieved for VCSELs in same clusters. Additionally, the TD signature reduction for the dynamics of VCSELs in the stable clusters are systematically discussed. It is shown that both moderate polarizer angle and frequency detuning between different clusters have strengthen the effect of TD signature suppression. Moreover, the isochronous cluster synchronization with TD signature concealment is also verified in another VPOI-VCSEL network with different topology, indicating the generality of proposed results. Our results shed a new light on the research of chaos synchronization and chaos-based secure communications in VCSEL networks.

Symbolic diffusion entropy rate of chaotic time series as a surrogate measure for the largest Lyapunov exponent

Existing methods for estimating the largest Lyapunov exponent from a time series rely on the rate of separation of initially nearby trajectories reconstructed from the time series in phase space. According to Ueda, chaotic dynamical behavior is viewed as a manifestation of random transitions between unstable periodic orbits in a chaotic attractor, which are triggered by perturbations due to experimental observation or the roundoff error characteristic of the computing machine, and consequently consists of a sequence of piecewise deterministic processes instead of an entirely deterministic process. Chaotic trajectories might have no physical reality. Here, we propose a mathematical method for estimating a surrogate measure for the largest Lyapunov exponent on the basis of the random diffusion of the symbols generated from a time series in a chaotic attractor, without resorting to initially nearby trajectories. We apply the proposed method to numerical time series generated by chaotic flow models and verify its validity.

Reconstruction of dynamic networks with time-delayed interactions in the presence of fast-varying noises

Most complex social, biological and technological systems can be described by dynamic networks. Reconstructing network structures from measurable data is a fundamental problem in almost all interdisciplinary fields. Network nodes interact with each other and those interactions often have diversely distributed time delays. Accurate reconstruction of any targeted interaction to a node requires measured data of all its neighboring nodes together with information on the time delays of interactions from these neighbors. When networks are large, these data are often not available and time-delay factors are deeply hidden. Here we show that fast-varying noise can be of great help in solving these challenging problems. By computing suitable correlations, we can infer the intensity and time delay of any targeted interaction with the data of two related nodes (driving and driven nodes) only while all other nodes in the network are hidden. This method is analytically derived and fully justified by extensive numerical simulations.

Permutation Entropy and Information Recovery in Nonlinear Dynamic Economic Time Series

The focus of this paper is an information theoretic-symbolic logic approach to extract information from complex economic systems and unlock its dynamic content. Permutation Entropy (PE) is used to capture the permutation patterns-ordinal relations among the individual values of a given time series; to obtain a probability distribution of the accessible patterns; and to quantify the degree of complexity of an economic behavior system. Ordinal patterns are used to describe the intrinsic patterns, which are hidden in the dynamics of the economic system. Empirical applications involving the Dow Jones Industrial Average are presented to indicate the information recovery value and the applicability of the PE method. The results demonstrate the ability of the PE method to detect the extent of complexity (irregularity) and to discriminate and classify admissible and forbidden states.

Using High-Frequency Entropy to Forecast Bitcoin's Daily Value at Risk

In this paper we investigate the ability of several econometrical models to forecast value at risk for a sample of daily time series of cryptocurrency returns. Using high frequency data for Bitcoin, we estimate the entropy of intraday distribution of logreturns through the symbolic time series analysis (STSA), producing low-resolution data from high-resolution data. Our results show that entropy has a strong explanatory power for the quantiles of the distribution of the daily returns. Based on Christoffersen’s tests for Value at Risk (VaR) backtesting, we can conclude that the VaR forecast build upon the entropy of intraday returns is the best, compared to the forecasts provided by the classical GARCH models.

Causal Shannon-Fisher Characterization of Motor/Imagery Movements in EEG

The electroencephalogram (EEG) is an electrophysiological monitoring method that allows us to glimpse the electrical activity of the brain. Neural oscillations patterns are perhaps the best salient feature of EEG as they are rhythmic activities of the brain that can be generated by interactions across neurons. Large-scale oscillations can be measured by EEG as the different oscillation patterns reflected within the different frequency bands, and can provide us with new insights into brain functions. In order to understand how information about the rhythmic activity of the brain during visuomotor/imagined cognitive tasks is encoded in the brain we precisely quantify the different features of the oscillatory patterns considering the Shannon–Fisher plane H×F. This allows us to distinguish the dynamics of rhythmic activities of the brain showing that the Beta band facilitate information transmission during visuomotor/imagined tasks.

An analysis of high-frequency cryptocurrencies prices dynamics using permutation-information-theory quantifiers

This paper discusses the dynamics of intraday prices of 12 cryptocurrencies during the past months' boom and bust. The importance of this study lies in the extended coverage of the cryptoworld, accounting for more than 90% of the total daily turnover. By using the complexity-entropy causality plane, we could discriminate three different dynamics in the data set. Whereas most of the cryptocurrencies follow a similar pattern, there are two currencies (ETC and ETH) that exhibit a more persistent stochastic dynamics, and two other currencies (DASH and XEM) whose behavior is closer to a random walk. Consequently, similar financial assets, using blockchain technology, are differentiated by market participants.

Analysis of ischaemic crisis using the informational causal entropy-complexity plane

In the present work, an ischaemic process, mainly focused on the reperfusion stage, is studied using the informational causal entropy-complexity plane. Ischaemic wall behavior under this condition was analyzed through wall thickness and ventricular pressure variations, acquired during an obstructive flow maneuver performed on left coronary arteries of surgically instrumented animals. Basically, the induction of ischaemia depends on the temporary occlusion of left circumflex coronary artery (which supplies blood to the posterior left ventricular wall) that lasts for a few seconds. Normal perfusion of the wall was then reestablished while the anterior ventricular wall remained adequately perfused during the entire maneuver. The obtained results showed that system dynamics could be effectively described by entropy-complexity loops, in both abnormally and well perfused walls. These results could contribute to making an objective indicator of the recovery heart tissues after an ischaemic process, in a way to quantify the restoration of myocardial behavior after the supply of oxygen to the ventricular wall was suppressed for a brief period.

Bandt-Pompe symbolization dynamics for time series with tied values: A data-driven approach

In 2002, Bandt and Pompe [Phys. Rev. Lett. 88, 174102 (2002)] introduced a successfully symbolic encoding scheme based on the ordinal relation between the amplitude of neighboring values of a given data sequence, from which the permutation entropy can be evaluated. Equalities in the analyzed sequence, for example, repeated equal values, deserve special attention and treatment as was shown recently by Zunino and co-workers [Phys. Lett. A 381, 1883 (2017)]. A significant number of equal values can give rise to false conclusions regarding the underlying temporal structures in practical contexts. In the present contribution, we review the different existing methodologies for treating time series with tied values by classifying them according to their different strategies. In addition, a novel data-driven imputation is presented that proves to outperform the existing methodologies and avoid the false conclusions pointed by Zunino and co-workers.

Rhythmic activities of the brain: Quantifying the high complexity of beta and gamma oscillations during visuomotor tasks

Electroencephalography (EEG) signals depict the electrical activity that takes place at the surface of the brain and provide an important tool for understanding a variety of cognitive processes. The EEG is the product of synchronized activity of the brain, and variations in EEG oscillations patterns reflect the underlying changes in neuronal synchrony. Our aim is to characterize the complexity of the EEG rhythmic oscillations bands when the subjects perform a visuomotor or imagined cognitive tasks (imagined movement), providing a causal mapping of the dynamical rhythmic activities of the brain as a measure of attentional investment. We estimate the intrinsic correlational structure of the signals within the causality entropy-complexity plane H×C, where the enhanced complexity in the gamma 1, gamma 2, and beta 1 bands allows us to distinguish motor-visual memory tasks from control conditions. We identify the dynamics of the gamma 1, gamma 2, and beta 1 rhythmic oscillations within the zone of a chaotic dissipative behavior, whereas in contrast the beta 2 band shows a much higher level of entropy and a significant low level of complexity that correspond to a non-invertible cubic map. Our findings enhance the importance of the gamma band during attention in perceptual feature binding during the visuomotor/imagery tasks.

A Hybrid De-Noising Algorithm for the Gear Transmission System Based on CEEMDAN-PE-TFPF

In order to remove noise and preserve the important features of a signal, a hybrid de-noising algorithm based on Complete Ensemble Empirical Mode Decomposition with Adaptive Noise (CEEMDAN), Permutation Entropy (PE), and Time-Frequency Peak Filtering (TFPF) is proposed. In view of the limitations of the conventional TFPF method regarding the fixed window length problem, CEEMDAN and PE are applied to compensate for this, so that the signal is balanced with respect to both noise suppression and signal fidelity. First, the Intrinsic Mode Functions (IMFs) of the original spectra are obtained using the CEEMDAN algorithm, and the PE value of each IMF is calculated to classify whether the IMF requires filtering, then, for different IMFs, we select different window lengths to filter them using TFPF; finally, the signal is reconstructed as the sum of the filtered and residual IMFs. The filtering results of a simulated and an actual gearbox vibration signal verify that the de-noising results of CEEMDAN-PE-TFPF outperforms other signal de-noising methods, and the proposed method can reveal fault characteristic information effectively.

Multiscale permutation entropy analysis of laser beam wandering in isotropic turbulence

We have experimentally quantified the temporal structural diversity from the coordinate fluctuations of a laser beam propagating through isotropic optical turbulence. The main focus here is on the characterization of the long-range correlations in the wandering of a thin Gaussian laser beam over a screen after propagating through a turbulent medium. To fulfill this goal, a laboratory-controlled experiment was conducted in which coordinate fluctuations of the laser beam were recorded at a sufficiently high sampling rate for a wide range of turbulent conditions. Horizontal and vertical displacements of the laser beam centroid were subsequently analyzed by implementing the symbolic technique based on ordinal patterns to estimate the well-known permutation entropy. We show that the permutation entropy estimations at multiple time scales evidence an interplay between different dynamical behaviors. More specifically, a crossover between two different scaling regimes is observed. We confirm a transition from an integrated stochastic process contaminated with electronic noise to a fractional Brownian motion with a Hurst exponent H=5/6 as the sampling time increases. Besides, we are able to quantify, from the estimated entropy, the amount of electronic noise as a function of the turbulence strength. We have also demonstrated that these experimental observations are in very good agreement with numerical simulations of noisy fractional Brownian motions with a well-defined crossover between two different scaling regimes.

Estimation of the degree of dynamical instability from the information entropy of symbolic dynamics

A positive Lyapunov exponent is the most convincing signature of chaos. However, existing methods for estimating the Lyapunov exponent from a time series often give unreliable estimates because they trace the time evolution of the distance between a pair of initially neighboring trajectories in phase space. Here, we propose a mathematical method for estimating the degree of dynamical instability, as a surrogate for the Lyapunov exponent, without tracing initially neighboring trajectories on the basis of the information entropy from a symbolic time series. We apply the proposed method to numerical time series generated by well-known chaotic systems and experimental time series and verify its validity.

Characterizing Complex Dynamics in the Classical and Semi-Classical Duffing Oscillator Using Ordinal Patterns Analysis

The driven double-well Duffing oscillator is a well-studied system that manifests a wide variety of dynamics, from periodic behavior to chaos, and describes a diverse array of physical systems. It has been shown to be relevant in understanding chaos in the classical to quantum transition. Here we explore the complexity of its dynamics in the classical and semi-classical regimes, using the technique of ordinal pattern analysis. This is of particular relevance to potential experiments in the semi-classical regime. We unveil different dynamical regimes within the chaotic range, which cannot be detected with more traditional statistical tools. These regimes are characterized by different hierarchies and probabilities of the ordinal patterns. Correlation between the Lyapunov exponent and the permutation entropy is revealed that leads us to interpret dips in the Lyapunov exponent as transitions in the dynamics of the system.

Permutation entropy with vector embedding delays

Permutation entropy (PE) is a statistic used widely for the detection of structure within a time series. Embedding delay times at which the PE is reduced are characteristic timescales for which such structure exists. Here, a generalized scheme is investigated where embedding delays are represented by vectors rather than scalars, permitting PE to be calculated over a (D-1)-dimensional space, where D is the embedding dimension. This scheme is applied to numerically generated noise, sine wave and logistic map series, and experimental data sets taken from a vertical-cavity surface emitting laser exhibiting temporally localized pulse structures within the round-trip time of the laser cavity. Results are visualized as PE maps as a function of embedding delay, with low PE values indicating combinations of embedding delays where correlation structure is present. It is demonstrated that vector embedding delays enable identification of structure that is ambiguous or masked, when the embedding delay is constrained to scalar form.

Complexity-enhanced polarization-resolved chaos in a ring network of mutually coupled vertical-cavity surface-emitting lasers with multiple delays

The complexity properties of polarization-resolved chaotic signals generated in a ring network of vertical-cavity surface-emitting lasers (VCSELs) mutually coupled with multiple delays are investigated quantitatively by using the proposed mean permutation entropy (MPE). For direct comparison, the complexity of polarization-resolved chaos in a ring network of VCSELs coupled with single delay is also considered. The effects of injection current, coupling strength, and frequency detuning on the chaotic complexity for both a single-delay ring network (SDRN) and a multiple-delay ring network (MDRN) are evaluated quantitatively and compared by the MPE. The effects of internal parameters of VCSELs on the complexity are also discussed, and the correlation properties between different polarization-resolved modes are also analyzed. It is shown that the complexity of chaos in two polarization-resolved modes of VCSELs in MDRN is much higher than those in SDRN in a much wider parameter region. Besides, wider range of injection current, coupling strength, and frequency detuning can be tuned to achieve the enhancement of chaotic complexity in MDRN. These results provide an effective quantifier, the proposed MPE, to evaluate quantitatively the complexity of chaos generated in systems with multiple delays, and the multichannel complexity-enhanced polarization-resolved chaos generated in MDRN of mutually coupled VCSELs is extremely meaningful for the chaos-based random number generators.

Multiscale ordinal network analysis of human cardiac dynamics

In this study, we propose a new information theoretic measure to quantify the complexity of biological systems based on time-series data. We demonstrate the potential of our method using two distinct applications to human cardiac dynamics. Firstly, we show that the method clearly discriminates between segments of electrocardiogram records characterized by normal sinus rhythm, ventricular tachycardia and ventricular fibrillation. Secondly, we investigate the multiscale complexity of cardiac dynamics with respect to age in healthy individuals using interbeat interval time series and compare our findings with a previous study which established a link between age and fractal-like long-range correlations. The method we use is an extension of the symbolic mapping procedure originally proposed for permutation entropy. We build a Markov chain of the dynamics based on order patterns in the time series which we call an ordinal network, and from this model compute an intuitive entropic measure of transitional complexity. A discussion of the model parameter space in terms of traditional time delay embedding provides a theoretical basis for our multiscale approach. As an ancillary discussion, we address the practical issue of node aliasing and how this effects ordinal network models of continuous systems from discrete time sampled data, such as interbeat interval time series.This article is part of the themed issue 'Mathematical methods in medicine: neuroscience, cardiology and pathology'.

Output-only parameter identification of a colored-noise-driven Van-der-Pol oscillator: Thermoacoustic instabilities as an example

The problem of output-only parameter identification for nonlinear oscillators forced by colored noise is considered. In this context, it is often assumed that the forcing noise is white, since its actual spectral content is unknown. The impact of this white-noise forcing assumption upon parameter identification is quantitatively analyzed. First, a Van-der-Pol oscillator forced by an Ornstein-Uhlenbeck process is considered. Second, the practical case of thermoacoustic limit cycles in combustion chambers with turbulence-induced forcing is investigated. It is shown that in both cases, the system parameters are accurately identified if time signals are appropriately band-pass-filtered around the oscillator eigenfrequency.