Ordinal Patterns, Entropy, and EEG

Chaotic time series classification by means of reservoir-based convolutional neural networks

We propose a novel Reservoir Computing (RC) based classification method that distinguishes between different chaotic time series. Our method is composed of two steps: (i) we use the reservoir as a feature extracting machine that captures the salient features of time series data; (ii) the readout layer of the reservoir is subsequently fed into a Convolutional Neural Network (CNN) to facilitate classification and recognition. One of the notable advantages is that the readout layer, as obtained by randomly generated empirical hyper-parameters within the RC module, provides sufficient information for the CNN to accomplish the classification tasks effectively. The quality of extracted features by RC is independently evaluated by the root mean square error, which measures how well the training signal may be reconstructed from the input time series. Furthermore, we propose two ways to implement the RC module, namely, a single shallow RC and parallel RC configurations, to further improve the classification accuracy. The important roles of RC in feature extraction are demonstrated by comparing the results when the CNN is provided with either ordinal pattern probability features or unprocessed raw time series directly, both of which perform worse than RC-based method. In addition to CNN, we show that the readout of RC is good for other classification tools as well. The successful classification of electroencephalogram recordings of different brain states suggests that our RC-based classification tools can be used for experimental studies.

Multiscale Sample Entropy-Based Feature Extraction with Gaussian Mixture Model for Detection and Classification of Blue Whale Vocalization

A multiscale sample entropy (MSE) algorithm is presented as a time domain feature extraction method to study the vocal behavior of blue whales through continuous acoustic monitoring. Additionally, MSE is applied to the Gaussian mixture model (GMM) for blue whale call detection and classification. The performance of the proposed MSE-GMM algorithm is experimentally assessed and benchmarked against traditional methods, including principal component analysis (PCA), wavelet-based feature (WF) extraction, and dynamic mode decomposition (DMD), all combined with the GMM. This study utilizes recorded data from the Antarctic open source library. To improve the accuracy of classification models, a GMM-based feature selection method is proposed, which evaluates both positively and negatively correlated features while considering inter-feature correlations. The proposed method demonstrates enhanced performance over conventional PCA-GMM, DMD-GMM, and WF-GMM methods, achieving higher accuracy and lower error rates when classifying the non-stationary and complex vocalizations of blue whales.

Stationarity assessment of resting state condition via permutation entropy on EEG recordings

The analysis of electrophysiological recordings of the human brain in resting state is a key experimental technique in neuroscience. Resting state is the default condition to characterize brain dynamics. Its successful implementation relies both on the capacity of subjects to comply with the requirement of staying awake while not performing any cognitive task, and on the capacity of the experimenter to validate that compliance. Here we propose a novel approach, based on permutation entropy, to assess the reliability of the resting state hypothesis by evaluating its stability during a recording. We combine the calculation of permutation entropy with a method to estimate its uncertainty out of a single time series. The approach is showcased on electroencephalographic data recorded from young and elderly subjects and considering eyes-closed and eyes-opened resting state conditions. Besides highlighting the reliability of the approach, the results show higher instability in elderly subjects, hinting at qualitative differences between age groups in the distribution of unstable brain activity. The method can be applied to other kinds of electrophysiological data, like magnetoencephalographic recordings. In addition, provided that suitable hardware and software processing units are used, the implementation of the method can be translated into a real time one.

Permutation Entropy: An Ordinal Pattern-Based Resilience Indicator for Industrial Equipment

In a highly dynamic and complex environment where risks and uncertainties are inevitable, the ability of a system to quickly recover from disturbances and maintain optimal performance is crucial for ensuring operational continuity and efficiency. In this context, resilience has become an increasingly important topic in the field of engineering and the management of productive systems. However, there is no single quantitative indicator of resilience that allows for the measurement of this characteristic in a productive system. This study proposes the use of permutation entropy of ordinal patterns in time series as an indicator of resilience in industrial equipment and systems. Based on the definition of resilience, the developed method enables precise and efficient assessment of a system's ability to withstand and recover from disturbances. The methodology includes the identification of ordinal patterns and their analysis through the calculation of a permutation entropy indicator to characterize the dynamics of industrial systems. Case studies are presented and the results are compared with other resilience models existing in the literature, aiming to demonstrate the effectiveness of the proposed approach. The results are promising and highlight a highly applicable and simple indicator for resilience in industrial systems.

Revisiting the Characterization of Resting Brain Dynamics with the Permutation Jensen-Shannon Distance

Taking into account the complexity of the human brain dynamics, the appropriate characterization of any brain state is a challenge not easily met. Actually, even the discrimination of simple behavioral tasks, such as resting with eyes closed or eyes open, represents an intricate problem and many efforts have been and are being made to overcome it. In this work, the aforementioned issue is carefully addressed by performing multiscale analyses of electroencephalogram records with the permutation Jensen-Shannon distance. The influence that linear and nonlinear temporal correlations have on the discrimination is unveiled. Results obtained lead to significant conclusions that help to achieve an improved distinction between these resting brain states.

Entropy in scalp EEG can be used as a preimplantation marker for VNS efficacy

Vagus nerve stimulation (VNS) is a therapeutic option in drug-resistant epilepsy. VNS leads to ≥ 50% seizure reduction in 50 to 60% of patients, termed "responders". The remaining 40 to 50% of patients, "non-responders", exhibit seizure reduction < 50%. Our work aims to differentiate between these two patient groups in preimplantation EEG analysis by employing several Entropy methods. We identified 59 drug-resistant epilepsy patients treated with VNS. We established their response to VNS in terms of responders and non-responders. A preimplantation EEG with eyes open/closed, photic stimulation, and hyperventilation was found for each patient. The EEG was segmented into eight time intervals within four standard frequency bands. In all, 32 EEG segments were obtained. Seven Entropy methods were calculated for all segments. Subsequently, VNS responders and non-responders were compared using individual Entropy methods. VNS responders and non-responders differed significantly in all Entropy methods except Approximate Entropy. Spectral Entropy revealed the highest number of EEG segments differentiating between responders and non-responders. The most useful frequency band distinguishing responders and non-responders was the alpha frequency, and the most helpful time interval was hyperventilation and rest 4 (the end of EEG recording).

Ordinal pattern-based complexity analysis of high-dimensional chaotic time series

The ordinal pattern-based complexity-entropy plane is a popular tool in nonlinear dynamics for distinguishing stochastic signals (noise) from deterministic chaos. Its performance, however, has mainly been demonstrated for time series from low-dimensional discrete or continuous dynamical systems. In order to evaluate the usefulness and power of the complexity-entropy (CE) plane approach for data representing high-dimensional chaotic dynamics, we applied this method to time series generated by the Lorenz-96 system, the generalized Hénon map, the Mackey-Glass equation, the Kuramoto-Sivashinsky equation, and to phase-randomized surrogates of these data. We find that both the high-dimensional deterministic time series and the stochastic surrogate data may be located in the same region of the complexity-entropy plane, and their representations show very similar behavior with varying lag and pattern lengths. Therefore, the classification of these data by means of their position in the CE plane can be challenging or even misleading, while surrogate data tests based on (entropy, complexity) yield significant results in most cases.

Statistics and contrasts of order patterns in univariate time series

Order patterns apply well to many fields, because of minimal stationarity assumptions. Here, we fix the methodology of patterns of length 3 by introducing an orthogonal system of four pattern contrasts, that is, weighted differences of pattern frequencies. These contrasts are statistically independent and turn up as eigenvectors of a covariance matrix both in the independence model and the random walk model. The most important contrast is the turning rate. It can be used to evaluate sleep depth directly from EEG (electroencephalographic brain data). The paper discusses fluctuations of permutation entropy, statistical tests, and the need of new models for noises like EEG.

Ordinal methods for a characterization of evolving functional brain networks

Ordinal time series analysis is based on the idea to map time series to ordinal patterns, i.e., order relations between the values of a time series and not the values themselves, as introduced in 2002 by C. Bandt and B. Pompe. Despite a resulting loss of information, this approach captures meaningful information about the temporal structure of the underlying system dynamics as well as about properties of interactions between coupled systems. This-together with its conceptual simplicity and robustness against measurement noise-makes ordinal time series analysis well suited to improve characterization of the still poorly understood spatiotemporal dynamics of the human brain. This minireview briefly summarizes the state-of-the-art of uni- and bivariate ordinal time-series-analysis techniques together with applications in the neurosciences. It will highlight current limitations to stimulate further developments, which would be necessary to advance characterization of evolving functional brain networks.

Ordinal Pattern Analysis: A Tutorial on Assessing the Fit of Hypotheses to Individual Repeated Measures Data

PURPOSE

This article provides a tutorial introduction to ordinal pattern analysis, a statistical analysis method designed to quantify the extent to which hypotheses of relative change across experimental conditions match observed data at the level of individuals. This method may be a useful addition to familiar parametric statistical methods including repeated measures analysis of variance and generalized linear mixed-effects models, particularly when analyzing inherently individual characteristics, such as perceptual processes, and where experimental effects are usefully modeled in relative rather than absolute terms.

METHOD

Three analyses of increasing complexity are demonstrated using ordinal pattern analysis. An initial analysis of a very small data set is designed to explicate the simple mathematical calculations that make up ordinal pattern analysis, which can be performed without the aid of a computer. Analyses of slightly larger data sets are used to demonstrate familiar concepts, including comparison of competing hypotheses, handling missing data, group comparisons, and pairwise tests. All analyses can be reproduced using provided code and data.

RESULTS

Ordinal pattern analysis results are presented, along with an analogous linear mixed-effects analysis, to illustrate the similarities and differences in information provided by ordinal pattern analysis in comparison to familiar parametric methods.

CONCLUSION

Although ordinal pattern analysis does not produce familiar numerical effect sizes, it does provide highly interpretable results in terms of the proportion of individuals whose results are consistent with a hypothesis, along with individual and group-level statistics, which quantify hypothesis performance.

Slope Entropy Characterisation: The Role of the δ Parameter

Many time series entropy calculation methods have been proposed in the last few years. They are mainly used as numerical features for signal classification in any scientific field where data series are involved. We recently proposed a new method, Slope Entropy (SlpEn), based on the relative frequency of differences between consecutive samples of a time series, thresholded using two input parameters, γ and δ. In principle, δ was proposed to account for differences in the vicinity of the 0 region (namely, ties) and, therefore, was usually set at small values such as 0.001. However, there is no study that really quantifies the role of this parameter using this default or other configurations, despite the good SlpEn results so far. The present paper addresses this issue, removing δ from the SlpEn calculation to assess its real influence on classification performance, or optimising its value by means of a grid search in order to find out if other values beyond the 0.001 value provide significant time series classification accuracy gains. Although the inclusion of this parameter does improve classification accuracy according to experimental results, gains of 5% at most probably do not support the additional effort required. Therefore, SlpEn simplification could be seen as a real alternative.

Compression complexity with ordinal patterns for robust causal inference in irregularly sampled time series

Distinguishing cause from effect is a scientific challenge resisting solutions from mathematics, statistics, information theory and computer science. Compression-Complexity Causality (CCC) is a recently proposed interventional measure of causality, inspired by Wiener–Granger’s idea. It estimates causality based on change in dynamical compression-complexity (or compressibility) of the effect variable, given the cause variable. CCC works with minimal assumptions on given data and is robust to irregular-sampling, missing-data and finite-length effects. However, it only works for one-dimensional time series. We propose an ordinal pattern symbolization scheme to encode multidimensional patterns into one-dimensional symbolic sequences, and thus introduce the Permutation CCC (PCCC). We demonstrate that PCCC retains all advantages of the original CCC and can be applied to data from multidimensional systems with potentially unobserved variables which can be reconstructed using the embedding theorem. PCCC is tested on numerical simulations and applied to paleoclimate data characterized by irregular and uncertain sampling and limited numbers of samples.

Permutation Jensen-Shannon distance: A versatile and fast symbolic tool for complex time-series analysis

The main motivation of this paper is to introduce the permutation Jensen-Shannon distance, a symbolic tool able to quantify the degree of similarity between two arbitrary time series. This quantifier results from the fusion of two concepts, the Jensen-Shannon divergence and the encoding scheme based on the sequential ordering of the elements in the data series. The versatility and robustness of this ordinal symbolic distance for characterizing and discriminating different dynamics are illustrated through several numerical and experimental applications. Results obtained allow us to be optimistic about its usefulness in the field of complex time-series analysis. Moreover, thanks to its simplicity, low computational cost, wide applicability, and less susceptibility to outliers and artifacts, this ordinal measure can efficiently handle large amounts of data and help to tackle the current big data challenges.

Extracting Robust Biomarkers From Multichannel EEG Time Series Using Nonlinear Dimensionality Reduction Applied to Ordinal Pattern Statistics and Spectral Quantities

In this study, ordinal pattern analysis and classical frequency-based EEG analysis methods are used to differentiate between EEGs of different age groups as well as individuals. As characteristic features, functional connectivity as well as single-channel measures in both the time and frequency domain are considered. We compare the separation power of each feature set after nonlinear dimensionality reduction using t-distributed stochastic neighbor embedding and demonstrate that ordinal pattern-based measures yield results comparable to frequency-based measures applied to preprocessed data, and outperform them if applied to raw data. Our analysis yields no significant differences in performance between single-channel features and functional connectivity features regarding the question of age group separation.

Representation based on ordinal patterns for seizure detection in EEG signals

EEG signals carry rich information about brain activity and play an important role in the diagnosis and recognition of epilepsy. Numerous algorithms using EEG signals to detect seizures have been developed in recent decades. However, most of them require well-designed features that highly depend on domain-specific knowledge and algorithm expertise. In this study, we introduce the unigram ordinal pattern (UniOP) and bigram ordinal pattern (BiOP) representations to capture the different underlying dynamics of time series, which only assumes that time series derived from different dynamics can be characterized by repeated ordinal patterns. Specifically, we first transform each subsequence in a time series into the corresponding ordinal pattern in terms of the ranking of values and consider the distribution of ordinal patterns of all subsequences as the UniOP representation. Furthermore, we consider the distribution of the cooccurrence of ordinal patterns as the BiOP representation to characterize the contextual information for each ordinal pattern. We then combine the proposed representations with the nearest neighbor algorithm to evaluate its effectiveness on three publicly available seizure datasets. The results on the Bonn EEG dataset demonstrate that this method provides more than 90% accuracy, sensitivity, and specificity in most cases and outperforms several state-of-the-art methods, which proves its ability to capture the key information of the underlying dynamics of EEG time series at healthy, seizure-free, and seizure states. The results on the second dataset are comparable with the state-of-the-art method, showing the good generalization ability of the proposed method. All performance metrics on the third dataset are approximately 89%, which demonstrates that the proposed representations are suitable for large-scale datasets.

A Novel Recognition Strategy for Epilepsy EEG Signals Based on Conditional Entropy of Ordinal Patterns

Epilepsy is one of the most ordinary neuropathic illnesses, and electroencephalogram (EEG) is the essential method for recording various brain rhythm activities due to its high temporal resolution. The conditional entropy of ordinal patterns (CEOP) is known to be fast and easy to implement, which can effectively measure the irregularity of the physiological signals. The present work aims to apply the CEOP to analyze the complexity characteristics of the EEG signals and recognize the epilepsy EEG signals. We discuss the parameter selection and the performance analysis of the CEOP based on the neural mass model. The CEOP is applied to the real EEG database of Bonn epilepsy for identification. The results show that the CEOP is an excellent metrics for the analysis and recognition of epileptic EEG signals. The differences of the CEOP in normal and epileptic brain states suggest that the CEOP could be a judgment tool for the diagnosis of the epileptic seizure.

Using symbolic networks to analyse dynamical properties of disease outbreaks

We introduce a new methodology, which is based on the construction of epidemic networks, to analyse the evolution of epidemic time series. First, we translate the time series into ordinal patterns containing information about local fluctuations in disease prevalence. Each pattern is associated with a node of a network, whose (directed) connections arise from consecutive appearances in the series. The analysis of the network structure and the role of each pattern allows them to be classified according to the enhancement of entropy/complexity along the series, giving a different point of view about the evolution of a given disease.

Segmentation Method for Ship-Radiated Noise Using the Generalized Likelihood Ratio Test on an Ordinal Pattern Distribution

Due to the diversity of ship-radiated noise (SRN), audio segmentation is an essential procedure in the ship statuses/categories identification. However, the existing segmentation methods are not suitable for the SRN because of the lack of prior knowledge. In this paper, by a generalized likelihood ratio (GLR) test on the ordinal pattern distribution (OPD), we proposed a segmentation criterion and introduce it into single change-point detection (SCPD) and multiple change-points detection (MCPD) for SRN. The proposed method is free from the acoustic feature extraction and the corresponding probability distribution estimation. In addition, according to the sequential structure of ordinal patterns, the OPD is efficiently estimated on a series of analysis windows. By comparison with the Bayesian Information Criterion (BIC) based segmentation method, we evaluate the performance of the proposed method on both synthetic signals and real-world SRN. The segmentation results on synthetic signals show that the proposed method estimates the number and location of the change-points more accurately. The classification results on real-world SRN show that our method obtains more distinguishable segments, which verifies its effectiveness in SRN segmentation.

Detection of Hypoglycemia Using Measures of EEG Complexity in Type 1 Diabetes Patients

Previous literature has demonstrated that hypoglycemic events in patients with type 1 diabetes (T1D) are associated with measurable scalp electroencephalography (EEG) changes in power spectral density. In the present study, we used a dataset of 19-channel scalp EEG recordings in 34 patients with T1D who underwent a hyperinsulinemic-hypoglycemic clamp study. We found that hypoglycemic events are also characterized by EEG complexity changes that are quantifiable at the single-channel level through empirical conditional and permutation entropy and fractal dimension indices, i.e., the Higuchi index, residuals, and tortuosity. Moreover, we demonstrated that the EEG complexity indices computed in parallel in more than one channel can be used as the input for a neural network aimed at identifying hypoglycemia and euglycemia. The accuracy was about 90%, suggesting that nonlinear indices applied to EEG signals might be useful in revealing hypoglycemic events from EEG recordings in patients with T1D.

Decreased electrocortical temporal complexity distinguishes sleep from wakefulness

In most mammals, the sleep-wake cycle is constituted by three behavioral states: wakefulness (W), non-REM (NREM) sleep, and REM sleep. These states are associated with drastic changes in cognitive capacities, mostly determined by the function of the thalamo-cortical system. The intra-cranial electroencephalogram or electocorticogram (ECoG), is an important tool for measuring the changes in the thalamo-cortical activity during W and sleep. In the present study we analyzed broad-band ECoG recordings of the rat by means of a time-series complexity measure that is easy to implement and robust to noise: the Permutation Entropy (PeEn). We found that PeEn is maximal during W and decreases during sleep. These results bring to light the different thalamo-cortical dynamics emerging during sleep-wake states, which are associated with the well-known spectral changes that occur when passing from W to sleep. Moreover, the PeEn analysis allows us to determine behavioral states independently of the electrodes' cortical location, which points to an underlying global pattern in the signal that differs among the cycle states that is missed by classical methods. Consequently, our data suggest that PeEn analysis of a single EEG channel could allow for cheap, easy, and efficient sleep monitoring.

Slope Entropy: A New Time Series Complexity Estimator Based on Both Symbolic Patterns and Amplitude Information

The development of new measures and algorithms to quantify the entropy or related concepts of a data series is a continuous effort that has brought many innovations in this regard in recent years. The ultimate goal is usually to find new methods with a higher discriminating power, more efficient, more robust to noise and artifacts, less dependent on parameters or configurations, or any other possibly desirable feature. Among all these methods, Permutation Entropy (PE) is a complexity estimator for a time series that stands out due to its many strengths, with very few weaknesses. One of these weaknesses is the PE’s disregarding of time series amplitude information. Some PE algorithm modifications have been proposed in order to introduce such information into the calculations. We propose in this paper a new method, Slope Entropy (SlopEn), that also addresses this flaw but in a different way, keeping the symbolic representation of subsequences using a novel encoding method based on the slope generated by two consecutive data samples. By means of a thorough and extensive set of comparative experiments with PE and Sample Entropy (SampEn), we demonstrate that SlopEn is a very promising method with clearly a better time series classification performance than those previous methods.

Permutation Entropy: Enhancing Discriminating Power by Using Relative Frequencies Vector of Ordinal Patterns Instead of Their Shannon Entropy

Many measures to quantify the nonlinear dynamics of a time series are based on estimating the probability of certain features from their relative frequencies. Once a normalised histogram of events is computed, a single result is usually derived. This process can be broadly viewed as a nonlinear IRn mapping into IR, where n is the number of bins in the histogram. However, this mapping might entail a loss of information that could be critical for time series classification purposes. In this respect, the present study assessed such impact using permutation entropy (PE) and a diverse set of time series. We first devised a method of generating synthetic sequences of ordinal patterns using hidden Markov models. This way, it was possible to control the histogram distribution and quantify its influence on classification results. Next, real body temperature records are also used to illustrate the same phenomenon. The experiments results confirmed the improved classification accuracy achieved using raw histogram data instead of the PE final values. Thus, this study can provide a very valuable guidance for the improvement of the discriminating capability not only of PE, but of many similar histogram-based measures.

Sex Differences in the Complexity of Healthy Older Adults' Magnetoencephalograms

The analysis of resting-state brain activity recording in magnetoencephalograms (MEGs) with new algorithms of symbolic dynamics analysis could help obtain a deeper insight into the functioning of the brain and identify potential differences between males and females. Permutation Lempel-Ziv complexity (PLZC), a recently introduced non-linear signal processing algorithm based on symbolic dynamics, was used to evaluate the complexity of MEG signals in source space. PLZC was estimated in a broad band of frequencies (2–45 Hz), as well as in narrow bands (i.e., theta (4–8 Hz), alpha (8–12 Hz), low beta (12–20 Hz), high beta (20–30 Hz), and gamma (30–45 Hz)) in a sample of 98 healthy elderly subjects (49 males, 49 female) aged 65–80 (average age of 72.71 ± 4.22 for males and 72.67 ± 4.21 for females). PLZC was significantly higher for females than males in the high beta band at posterior brain regions including the precuneus, and the parietal and occipital cortices. Further statistical analyses showed that higher complexity values over highly overlapping regions than the ones mentioned above were associated with larger hippocampal volumes only in females. These results suggest that sex differences in healthy aging can be identified from the analysis of magnetoencephalograms with novel signal processing methods.

Embedded Dimension and Time Series Length. Practical Influence on Permutation Entropy and Its Applications

Permutation Entropy (PE) is a time series complexity measure commonly used in a variety of contexts, with medicine being the prime example. In its general form, it requires three input parameters for its calculation: time series length N, embedded dimension m, and embedded delay τ. Inappropriate choices of these parameters may potentially lead to incorrect interpretations. However, there are no specific guidelines for an optimal selection of N, m, or τ, only general recommendations such as N>>m!, τ=1, or m=3,…,7. This paper deals specifically with the study of the practical implications of N>>m!, since long time series are often not available, or non-stationary, and other preliminary results suggest that low N values do not necessarily invalidate PE usefulness. Our study analyses the PE variation as a function of the series length N and embedded dimension m in the context of a diverse experimental set, both synthetic (random, spikes, or logistic model time series) and real–world (climatology, seismic, financial, or biomedical time series), and the classification performance achieved with varying N and m. The results seem to indicate that shorter lengths than those suggested by N>>m! are sufficient for a stable PE calculation, and even very short time series can be robustly classified based on PE measurements before the stability point is reached. This may be due to the fact that there are forbidden patterns in chaotic time series, not all the patterns are equally informative, and differences among classes are already apparent at very short lengths.

Fusion of Motif- and Spectrum-Related Features for Improved EEG-Based Emotion Recognition

Emotion recognition is a burgeoning field allowing for more natural human-machine interactions and interfaces. Electroencephalography (EEG) has shown to be a useful modality with which user emotional states can be measured and monitored, particularly primitives such as valence and arousal. In this paper, we propose the use of ordinal pattern analysis, also called motifs, for improved EEG-based emotion recognition. Motifs capture recurring structures in time series and are inherently robust to noise, thus are well suited for the task at hand. Several connectivity, asymmetry, and graph-theoretic features are proposed and extracted from the motifs to be used for affective state recognition. Experiments with a widely used public database are conducted, and results show the proposed features outperforming benchmark spectrum-based features, as well as other more recent nonmotif-based graph-theoretic features and amplitude modulation-based connectivity/asymmetry measures. Feature and score-level fusion suggest complementarity between the proposed and benchmark spectrum-based measures. When combined, the fused models can provide up to 9% improvement relative to benchmark features alone and up to 16% to nonmotif-based graph-theoretic features.

Detection of time reversibility in time series by ordinal patterns analysis

Time irreversibility is a common signature of nonlinear processes and a fundamental property of non-equilibrium systems driven by non-conservative forces. A time series is said to be reversible if its statistical properties are invariant regardless of the direction of time. Here, we propose the Time Reversibility from Ordinal Patterns method (TiROP) to assess time-reversibility from an observed finite time series. TiROP captures the information of scalar observations in time forward as well as its time-reversed counterpart by means of ordinal patterns. The method compares both underlying information contents by quantifying its (dis)-similarity via the Jensen-Shannon divergence. The statistic is contrasted with a population of divergences coming from a set of surrogates to unveil the temporal nature and its involved time scales. We tested TiROP in different synthetic and real, linear, and non-linear time series, juxtaposed with results from the classical Ramsey's time reversibility test. Our results depict a novel, fast-computation, and fully data-driven methodology to assess time-reversibility with no further assumptions over data. This approach adds new insights into the current non-linear analysis techniques and also could shed light on determining new physiological biomarkers of high reliability and computational efficiency.

Multiscale Entropy Analysis for Recognition of Visually Elicited Negative Stress From EEG Recordings

Automatic identification of negative stress is an unresolved challenge that has received great attention in the last few years. Many studies have analyzed electroencephalographic (EEG) recordings to gain new insights about how the brain reacts to both short- and long-term stressful stimuli. Although most of them have only considered linear methods, the heterogeneity and complexity of the brain has recently motivated an increasing use of nonlinear metrics. Nonetheless, brain dynamics reflected in EEG recordings often exhibit a multiscale nature and no study dealing with this aspect has been developed yet. Hence, in this work two nonlinear indices for quantifying regularity and predictability of time series from several time scales are studied for the first time to discern between visually elicited emotional states of calmness and negative stress. The obtained results have revealed the maximum discriminant ability of 86.35% for the second time scale, thus suggesting that brain dynamics triggered by negative stress can be more clearly assessed after removal of some fast temporal oscillations. Moreover, both metrics have also been able to report complementary information for some brain areas.

Permutation Entropy Based on Non-Uniform Embedding

A novel visualization scheme for permutation entropy is presented in this paper. The proposed scheme is based on non-uniform attractor embedding of the investigated time series. A single digital image of permutation entropy is produced by averaging all possible plain projections of the permutation entropy measure in the multi-dimensional delay coordinate space. Computational experiments with artificially-generated and real-world time series are used to demonstrate the advantages of the proposed visualization scheme.

Identification of Auditory Object-Specific Attention from Single-Trial Electroencephalogram Signals via Entropy Measures and Machine Learning

Existing research has revealed that auditory attention can be tracked from ongoing electroencephalography (EEG) signals. The aim of this novel study was to investigate the identification of peoples’ attention to a specific auditory object from single-trial EEG signals via entropy measures and machine learning. Approximate entropy (ApEn), sample entropy (SampEn), composite multiscale entropy (CmpMSE) and fuzzy entropy (FuzzyEn) were used to extract the informative features of EEG signals under three kinds of auditory object-specific attention (Rest, Auditory Object1 Attention (AOA1) and Auditory Object2 Attention (AOA2)). The linear discriminant analysis and support vector machine (SVM), were used to construct two auditory attention classifiers. The statistical results of entropy measures indicated that there were significant differences in the values of ApEn, SampEn, CmpMSE and FuzzyEn between Rest, AOA1 and AOA2. For the SVM-based auditory attention classifier, the auditory object-specific attention of Rest, AOA1 and AOA2 could be identified from EEG signals using ApEn, SampEn, CmpMSE and FuzzyEn as features and the identification rates were significantly different from chance level. The optimal identification was achieved by the SVM-based auditory attention classifier using CmpMSE with the scale factor τ = 10. This study demonstrated a novel solution to identify the auditory object-specific attention from single-trial EEG signals without the need to access the auditory stimulus.

Transductive Feature Selection Using Clustering-Based Sample Entropy for Temperature Prediction in Weather Forecasting

Entropy measures have been a major interest of researchers to measure the information content of a dynamical system. One of the well-known methodologies is sample entropy, which is a model-free approach and can be deployed to measure the information transfer in time series. Sample entropy is based on the conditional entropy where a major concern is the number of past delays in the conditional term. In this study, we deploy a lag-specific conditional entropy to identify the informative past values. Moreover, considering the seasonality structure of data, we propose a clustering-based sample entropy to exploit the temporal information. Clustering-based sample entropy is based on the sample entropy definition while considering the clustering information of the training data and the membership of the test point to the clusters. In this study, we utilize the proposed method for transductive feature selection in black-box weather forecasting and conduct the experiments on minimum and maximum temperature prediction in Brussels for 1-6 days ahead. The results reveal that considering the local structure of the data can improve the feature selection performance. In addition, despite the large reduction in the number of features, the performance is competitive with the case of using all features.

Amplitude- and Fluctuation-Based Dispersion Entropy

Dispersion entropy (DispEn) is a recently introduced entropy metric to quantify the uncertainty of time series. It is fast and, so far, it has demonstrated very good performance in the characterisation of time series. It includes a mapping step, but the effect of different mappings has not been studied yet. Here, we investigate the effect of linear and nonlinear mapping approaches in DispEn. We also inspect the sensitivity of different parameters of DispEn to noise. Moreover, we develop fluctuation-based DispEn (FDispEn) as a measure to deal with only the fluctuations of time series. Furthermore, the original and fluctuation-based forbidden dispersion patterns are introduced to discriminate deterministic from stochastic time series. Finally, we compare the performance of DispEn, FDispEn, permutation entropy, sample entropy, and Lempel–Ziv complexity on two physiological datasets. The results show that DispEn is the most consistent technique to distinguish various dynamics of the biomedical signals. Due to their advantages over existing entropy methods, DispEn and FDispEn are expected to be broadly used for the characterization of a wide variety of real-world time series. The MATLAB codes used in this paper are freely available at http://dx.doi.org/10.7488/ds/2326.

Analysis of Electroencephalogram of patients with specific low back pain with the massage treatment

Through the investigation of the difference of approximate entropy (ApEn) and Hilbert-Huang Transform Marginal spectrum entropy (HHTMSEn) of the Electroencephalogram (EEG) signals of specific low back pain (SLBP) patients before and after the massage, we wanted to reveal the impact of the massage in the brain, and provided the basis for the massage treatment of SLBP patients. We recruited twenty-six SLBP patients and collected their spontaneous EEG signals before and after the massage. Firstly, we analyzed the ApEn and HHTMSEn of fourteen channels before and after the massage, and results showed that values of ApEn and HHTMSEn after the massage were less than the values before the massage significantly. And then, we extracted δ, θ, α and β rhythms of the EEG signal, and analyzed the ApEn and HHTMSEn of the four rhythms before and after the massage. The results showed that the ApEn values of δ and α rhythm after the massage were significantly less than the values before the massage, and the HHTMSEn values of β rhythms were significantly less than the values before the massage. The results of this study suggested the complexity of EEG signals was reduced with the relief of the pain after the massage therapy, and the change of pain of the SLBP patients was closely related to the change of the rhythms of the brain in the massage therapy, and the ApEn and the HHTMSEn features could serve as a base for quantitative assessment of SLBP condition after the massage therapy.

Index finger motor imagery EEG pattern recognition in BCI applications using dictionary cleaned sparse representation-based classification for healthy people

Electroencephalogram (EEG)-based motor imagery (MI) brain-computer interface (BCI) has shown its effectiveness for the control of rehabilitation devices designed for large body parts of the patients with neurologic impairments. In order to validate the feasibility of using EEG to decode the MI of a single index finger and constructing a BCI-enhanced finger rehabilitation system, we collected EEG data during right hand index finger MI and rest state for five healthy subjects and proposed a pattern recognition approach for classifying these two mental states. First, Fisher's linear discriminant criteria and power spectral density analysis were used to analyze the event-related desynchronization patterns. Second, both band power and approximate entropy were extracted as features. Third, aiming to eliminate the abnormal samples in the dictionary and improve the classification performance of the conventional sparse representation-based classification (SRC) method, we proposed a novel dictionary cleaned sparse representation-based classification (DCSRC) method for final classification. The experimental results show that the proposed DCSRC method gives better classification accuracies than SRC and an average classification accuracy of 81.32% is obtained for five subjects. Thus, it is demonstrated that single right hand index finger MI can be decoded from the sensorimotor rhythms, and the feature patterns of index finger MI and rest state can be well recognized for robotic exoskeleton initiation.

Multiscale ordinal network analysis of human cardiac dynamics

In this study, we propose a new information theoretic measure to quantify the complexity of biological systems based on time-series data. We demonstrate the potential of our method using two distinct applications to human cardiac dynamics. Firstly, we show that the method clearly discriminates between segments of electrocardiogram records characterized by normal sinus rhythm, ventricular tachycardia and ventricular fibrillation. Secondly, we investigate the multiscale complexity of cardiac dynamics with respect to age in healthy individuals using interbeat interval time series and compare our findings with a previous study which established a link between age and fractal-like long-range correlations. The method we use is an extension of the symbolic mapping procedure originally proposed for permutation entropy. We build a Markov chain of the dynamics based on order patterns in the time series which we call an ordinal network, and from this model compute an intuitive entropic measure of transitional complexity. A discussion of the model parameter space in terms of traditional time delay embedding provides a theoretical basis for our multiscale approach. As an ancillary discussion, we address the practical issue of node aliasing and how this effects ordinal network models of continuous systems from discrete time sampled data, such as interbeat interval time series.This article is part of the themed issue 'Mathematical methods in medicine: neuroscience, cardiology and pathology'.

Entropy measures, entropy estimators, and their performance in quantifying complex dynamics: Effects of artifacts, nonstationarity, and long-range correlations

Entropy measures are widely applied to quantify the complexity of dynamical systems in diverse fields. However, the practical application of entropy methods is challenging, due to the variety of entropy measures and estimators and the complexity of real-world time series, including nonstationarities and long-range correlations (LRC). We conduct a systematic study on the performance, bias, and limitations of three basic measures (entropy, conditional entropy, information storage) and three traditionally used estimators (linear, kernel, nearest neighbor). We investigate the dependence of entropy measures on estimator- and process-specific parameters, and we show the effects of three types of nonstationarities due to artifacts (trends, spikes, local variance change) in simulations of stochastic autoregressive processes. We also analyze the impact of LRC on the theoretical and estimated values of entropy measures. Finally, we apply entropy methods on heart rate variability data from subjects in different physiological states and clinical conditions. We find that entropy measures can only differentiate changes of specific types in cardiac dynamics and that appropriate preprocessing is vital for correct estimation and interpretation. Demonstrating the limitations of entropy methods and shedding light on how to mitigate bias and provide correct interpretations of results, this work can serve as a comprehensive reference for the application of entropy methods and the evaluation of existing studies.

Symbolic Analysis of Brain Dynamics Detects Negative Stress

The electroencephalogram (EEG) is the most common tool used to study mental disorders. In the last years, the use of this recording for recognition of negative stress has been receiving growing attention. However, precise identification of this emotional state is still an interesting unsolved challenge. Nowadays, stress presents a high prevalence in developed countries and, moreover, its chronic condition often leads to concomitant physical and mental health problems. Recently, a measure of time series irregularity, such as quadratic sample entropy (QSEn), has been suggested as a promising single index for discerning between emotions of calm and stress. Unfortunately, this index only considers repetitiveness of similar patterns and, hence, it is unable to quantify successfully dynamics associated with the data temporal structure. With the aim of extending QSEn ability for identification of stress from the EEG signal, permutation entropy (PEn) and its modification to be amplitude-aware (AAPEn) have been analyzed in the present work. These metrics assess repetitiveness of ordinal patterns, thus considering causal information within each one of them and obtaining improved estimates of predictability. Results have shown that PEn and AAPEn present a discriminant power between emotional states of calm and stress similar to QSEn, i.e., around 65%. Additionally, they have also revealed complementary dynamics to those quantified by QSEn, thus suggesting a synchronized behavior between frontal and parietal counterparts from both hemispheres of the brain. More precisely, increased stress levels have resulted in activation of the left frontal and right parietal regions and, simultaneously, in relaxing of the right frontal and left parietal areas. Taking advantage of this brain behavior, a discriminant model only based on AAPEn and QSEn computed from the EEG channels P3 and P4 has reached a diagnostic accuracy greater than 80%, which improves slightly the current state of the art. Moreover, because this classification system is notably easier than others previously proposed, it could be used for continuous monitoring of negative stress, as well as for its regulation towards more positive moods in controlled environments.

Dynamic Cross-Entropy

BACKGROUND

Complexity measures for time series have been used in many applications to quantify the regularity of one dimensional time series, however many dynamical systems are spatially distributed multidimensional systems.

NEW METHOD

We introduced Dynamic Cross-Entropy (DCE) a novel multidimensional complexity measure that quantifies the degree of regularity of EEG signals in selected frequency bands. Time series generated by discrete logistic equations with varying control parameter r are used to test DCE measures.

RESULTS

Sliding window DCE analyses are able to reveal specific period doubling bifurcations that lead to chaos. A similar behavior can be observed in seizures triggered by electroconvulsive therapy (ECT). Sample entropy data show the level of signal complexity in different phases of the ictal ECT. The transition to irregular activity is preceded by the occurrence of cyclic regular behavior. A significant increase of DCE values in successive order from high frequencies in gamma to low frequencies in delta band reveals several phase transitions into less ordered states, possible chaos in the human brain.

COMPARISON WITH EXISTING METHOD

To our knowledge there are no reliable techniques able to reveal the transition to chaos in case of multidimensional times series. In addition, DCE based on sample entropy appears to be robust to EEG artifacts compared to DCE based on Shannon entropy.

CONCLUSIONS

The applied technique may offer new approaches to better understand nonlinear brain activity.

Complexity of resting-state EEG activity in the patients with early-stage Parkinson's disease

To investigate the abnormal brain activities in the early stage of Parkinson's disease (PD), the electroencephalogram (EEG) signals were recorded with 20 channels from non-dementia PD patients (18 patients, 8 females) and age matched healthy controls (18 subjects, 8 females) during the resting state. Two methods based on the ordinal patterns of the recorded series, i.e., permutation entropy (PE) and order index (OI), were introduced to characterize the complexity of the cortical activities for two groups. It was observed that the resting-state EEG of PD patients showed lower PE and higher OI than healthy controls, which indicated that the early-stage PD caused the reduced complexity of EEG. We further applied two methods to determine the complexity of EEG rhythms in five sub-bands. The results showed that the gamma, beta and alpha rhythms of PD patients were characterized by lower PE and higher OI, i.e., reduced complexity, than healthy subjects. No significant differences were observed in theta or delta rhythms between two groups. The findings suggested that PE and OI were promising methods to detect the abnormal changes in the dynamics of EEG signals associated with early-stage PD. Further, such changes in EEG complexity may be the early markers of the cortical or subcortical dysfunction caused by PD.

Amplitude-aware permutation entropy: Illustration in spike detection and signal segmentation

BACKGROUND AND OBJECTIVE

Signal segmentation and spike detection are two important biomedical signal processing applications. Often, non-stationary signals must be segmented into piece-wise stationary epochs or spikes need to be found among a background of noise before being further analyzed. Permutation entropy (PE) has been proposed to evaluate the irregularity of a time series. PE is conceptually simple, structurally robust to artifacts, and computationally fast. It has been extensively used in many applications, but it has two key shortcomings. First, when a signal is symbolized using the Bandt-Pompe procedure, only the order of the amplitude values is considered and information regarding the amplitudes is discarded. Second, in the PE, the effect of equal amplitude values in each embedded vector is not addressed. To address these issues, we propose a new entropy measure based on PE: the amplitude-aware permutation entropy (AAPE).

METHODS

AAPE is sensitive to the changes in the amplitude, in addition to the frequency, of the signals thanks to it being more flexible than the classical PE in the quantification of the signal motifs. To demonstrate how the AAPE method can enhance the quality of the signal segmentation and spike detection, a set of synthetic and realistic synthetic neuronal signals, electroencephalograms and neuronal data are processed. We compare the performance of AAPE in these problems against state-of-the-art approaches and evaluate the significance of the differences with a repeated ANOVA with post hoc Tukey's test.

RESULTS

In signal segmentation, the accuracy of AAPE-based method is higher than conventional segmentation methods. AAPE also leads to more robust results in the presence of noise. The spike detection results show that AAPE can detect spikes well, even when presented with single-sample spikes, unlike PE. For multi-sample spikes, the changes in AAPE are larger than in PE.

CONCLUSION

We introduce a new entropy metric, AAPE, that enables us to consider amplitude information in the formulation of PE. The AAPE algorithm can be used in almost every irregularity-based application in various signal and image processing fields. We also made freely available the Matlab code of the AAPE.

Analysis and Classification of Stride Patterns Associated with Children Development Using Gait Signal Dynamics Parameters and Ensemble Learning Algorithms

Measuring stride variability and dynamics in children is useful for the quantitative study of gait maturation and neuromotor development in childhood and adolescence. In this paper, we computed the sample entropy (SampEn) and average stride interval (ASI) parameters to quantify the stride series of 50 gender-matched children participants in three age groups. We also normalized the SampEn and ASI values by leg length and body mass for each participant, respectively. Results show that the original and normalized SampEn values consistently decrease over the significance level of the Mann-Whitney U test (p < 0.01) in children of 3-14 years old, which indicates the stride irregularity has been significantly ameliorated with the body growth. The original and normalized ASI values are also significantly changing when comparing between any two groups of young (aged 3-5 years), middle (aged 6-8 years), and elder (aged 10-14 years) children. Such results suggest that healthy children may better modulate their gait cadence rhythm with the development of their musculoskeletal and neurological systems. In addition, the AdaBoost.M2 and Bagging algorithms were used to effectively distinguish the children's gait patterns. These ensemble learning algorithms both provided excellent gait classification results in terms of overall accuracy (≥90%), recall (≥0.8), and precision (≥0.8077).