Embedded Dimension and Time Series Length. Practical Influence on Permutation Entropy and Its Applications

Exploring Physiological Differences in Brain Areas Using Statistical Complexity Analysis of BOLD Signals

The brain is a fundamental organ for the human body to function properly, for which it needs to receive a continuous flow of blood, which explains the existence of control mechanisms that act to maintain this flow as constant as possible in a process known as cerebral autoregulation. One way to obtain information on how the levels of oxygen supplied to the brain vary is through of BOLD (Magnetic Resonance) images, which have the advantage of greater spatial resolution than other forms of measurement, such as transcranial Doppler. However, they do not provide good temporal resolution nor allow for continuous prolonged examination. Thus, it is of great importance to find a method to detect regional differences from short BOLD signals. One of the existing alternatives is complexity measures that can detect changes in the variability and temporal organisation of a signal that could reflect different physiological states. The so-called statistical complexity, created to overcome the shortcomings of entropy alone to explain the concept of complexity, has shown potential with haemodynamic signals. The aim of this study is to determine by using statistical complexity whether it is possible to find differences between physiologically distinct brain areas in healthy individuals. The data set includes BOLD images of 10 people obtained at the University Hospital of Leicester NHS Trust with a 1.5 Tesla magnetic resonance imaging scanner. The data were captured for 180 s at a frequency of 1 Hz. Using various combinations of statistical complexities, no differences were found between hemispheres. However, differences were detected between grey matter and white matter, indicating that these measurements are sensitive to differences in brain tissues.

Estimation of central blood pressure waveform from femoral blood pressure waveform by blind sources separation

Background

Central blood pressure (cBP) is a better indicator of cardiovascular morbidity and mortality than peripheral BP (pBP). However, direct cBP measurement requires invasive techniques and indirect cBP measurement is based on rigid and empirical transfer functions applied to pBP. Thus, development of a personalized and well-validated method for non-invasive derivation of cBP from pBP is necessary to facilitate the clinical routine. The purpose of the present study was to develop a novel blind source separation tool to separate a single recording of pBP into their pressure waveforms composing its dynamics, to identify the compounds that lead to pressure waveform distortion at the periphery, and to estimate the cBP. The approach is patient-specific and extracts the underlying blind pressure waveforms in pBP without additional brachial cuff calibration or any a priori assumption on the arterial model.

Methods

The intra-arterial femoral BPfe and intra-aortic pressure BPao were anonymized digital recordings from previous routine cardiac catheterizations of eight patients at the German Heart Centre Berlin. The underlying pressure waveforms in BPfe were extracted by the single-channel independent component analysis (SCICA). The accuracy of the SCICA model to estimate the whole cBP waveform was evaluated by the mean absolute error (MAE), the root mean square error (RMSE), the relative RMSE (RRMSE), and the intraclass correlation coefficient (ICC). The agreement between the intra-aortic and estimated parameters including systolic (SBP), diastolic (DBP), mean arterial pressure (MAP), and pulse pressure (PP) was evaluated by the regression and Bland-Altman analyses.

Results

The SCICA tool estimated the cBP waveform non-invasively from the intra-arterial BPfe with an MAE of 0.159 ± 1.629, an RMSE of 5.153 ± 0.957 mmHg, an RRMSE of 5.424 ± 1.304%, and an ICC of 0.94, as well as two waveforms contributing to morphological distortion at the femoral artery. The regression analysis showed a strong linear trend between the estimated and intra-aortic SBP, DBP, MAP, and PP with high coefficient of determination R2 of 0.98, 0.99, 0.99, and 0.97 respectively. The Bland-Altman plots demonstrated good agreement between estimated and intra-aortic parameters with a mean error and a standard deviation of difference of -0.54 ± 2.42 mmHg [95% confidence interval (CI): -5.28 to 4.20] for SBP, -1.97 ± 1.62 mmHg (95% CI: -5.14 to 1.20) for DBP, -1.49 ± 1.40 mmHg (95% CI: -4.25 to 1.26) for MAP, and 1.43 ± 2.79 mmHg (95% CI: -4.03 to 6.90) for PP.

Conclusions

The SCICA approach is a powerful tool that identifies sources contributing to morphological distortion at peripheral arteries and estimates cBP.

Information-Theoretic Approaches in EEG Correlates of Auditory Perceptual Awareness under Informational Masking

In informational masking paradigms, the successful segregation between the target and masker creates auditory perceptual awareness. The dynamics of the build-up of auditory perception is based on a set of interactions between bottom-up and top-down processes that generate neuronal modifications within the brain network activity. These neural changes are studied here using event-related potentials (ERPs), entropy, and integrated information, leading to several measures applied to electroencephalogram signals. The main findings show that the auditory perceptual awareness stimulated functional activation in the fronto-temporo-parietal brain network through (i) negative temporal and positive centro-parietal ERP components; (ii) an enhanced processing of multi-information in the temporal cortex; and (iii) an increase in informational content in the fronto-central cortex. These different results provide information-based experimental evidence about the functional activation of the fronto-temporo-parietal brain network during auditory perceptual awareness.

Slope Entropy Normalisation by Means of Analytical and Heuristic Reference Values

Slope Entropy (SlpEn) is a very recently proposed entropy calculation method. It is based on the differences between consecutive values in a time series and two new input thresholds to assign a symbol to each resulting difference interval. As the histogram normalisation value, SlpEn uses the actual number of unique patterns found instead of the theoretically expected value. This maximises the information captured by the method but, as a consequence, SlpEn results do not usually fall within the classical [0,1] interval. Although this interval is not necessary at all for time series classification purposes, it is a convenient and common reference framework when entropy analyses take place. This paper describes a method to keep SlpEn results within this interval, and improves the interpretability and comparability of this measure in a similar way as for other methods. It is based on a max-min normalisation scheme, described in two steps. First, an analytic normalisation is proposed using known but very conservative bounds. Afterwards, these bounds are refined using heuristics about the behaviour of the number of patterns found in deterministic and random time series. The results confirm the suitability of the approach proposed, using a mixture of the two methods.

EntropyHub: An open-source toolkit for entropic time series analysis

An increasing number of studies across many research fields from biomedical engineering to finance are employing measures of entropy to quantify the regularity, variability or randomness of time series and image data. Entropy, as it relates to information theory and dynamical systems theory, can be estimated in many ways, with newly developed methods being continuously introduced in the scientific literature. Despite the growing interest in entropic time series and image analysis, there is a shortage of validated, open-source software tools that enable researchers to apply these methods. To date, packages for performing entropy analysis are often run using graphical user interfaces, lack the necessary supporting documentation, or do not include functions for more advanced entropy methods, such as cross-entropy, multiscale cross-entropy or bidimensional entropy. In light of this, this paper introduces EntropyHub, an open-source toolkit for performing entropic time series analysis in MATLAB, Python and Julia. EntropyHub (version 0.1) provides an extensive range of more than forty functions for estimating cross-, multiscale, multiscale cross-, and bidimensional entropy, each including a number of keyword arguments that allows the user to specify multiple parameters in the entropy calculation. Instructions for installation, descriptions of function syntax, and examples of use are fully detailed in the supporting documentation, available on the EntropyHub website- www.EntropyHub.xyz. Compatible with Windows, Mac and Linux operating systems, EntropyHub is hosted on GitHub, as well as the native package repository for MATLAB, Python and Julia, respectively. The goal of EntropyHub is to integrate the many established entropy methods into one complete resource, providing tools that make advanced entropic time series analysis straightforward and reproducible.

TdP Incidence in Methoxamine-Sensitized Rabbit Model Is Reduced With Age but Not Influenced by Hypercholesterolemia

Metabolic syndrome is associated with hypercholesterolemia, cardiac remodeling, and increased susceptibility to ventricular arrhythmias. Effects of diet-induced hypercholesterolemia on susceptibility to torsades de pointes arrhythmias (TdP) together with potential indicators of arrhythmic risk were investigated in three experimental groups of Carlsson's rabbit model: (1) young rabbits (YC, young control, age 12-16 weeks), older rabbits (AC, adult control, age 20-24 weeks), and older age-matched cholesterol-fed rabbits (CH, cholesterol, age 20-24 weeks). TdP was induced by α-adrenergic stimulation by methoxamine and I_Kr block in 83% of YC rabbits, 18% of AC rabbits, and 21% of CH rabbits. High incidence of TdP was associated with high incidence of single (SEB) and multiple ectopic beats (MEB), but the QTc prolongation and short-term variability (STV) were similar in all three groups. In TdP-susceptible rabbits, STV was significantly higher compared with arrhythmia-free rabbits but not with rabbits with other than TdP arrhythmias (SEB, MEB). Amplitude-aware permutation entropy analysis of baseline ECG could identify arrhythmia-resistant animals with high sensitivity and specificity. The data indicate that the TdP susceptibility in methoxamine-sensitized rabbits is affected by the age of rabbits but probably not by hypercholesterolemia. Entropy analysis could potentially stratify the arrhythmic risk and identify the low-risk individuals.

ordpy: A Python package for data analysis with permutation entropy and ordinal network methods

Since Bandt and Pompe's seminal work, permutation entropy has been used in several applications and is now an essential tool for time series analysis. Beyond becoming a popular and successful technique, permutation entropy inspired a framework for mapping time series into symbolic sequences that triggered the development of many other tools, including an approach for creating networks from time series known as ordinal networks. Despite increasing popularity, the computational development of these methods is fragmented, and there were still no efforts focusing on creating a unified software package. Here, we present ordpy (http://github.com/arthurpessa/ordpy), a simple and open-source Python module that implements permutation entropy and several of the principal methods related to Bandt and Pompe's framework to analyze time series and two-dimensional data. In particular, ordpy implements permutation entropy, Tsallis and Rényi permutation entropies, complexity-entropy plane, complexity-entropy curves, missing ordinal patterns, ordinal networks, and missing ordinal transitions for one-dimensional (time series) and two-dimensional (images) data as well as their multiscale generalizations. We review some theoretical aspects of these tools and illustrate the use of ordpy by replicating several literature results.

A Review of EEG Signal Features and their Application in Driver Drowsiness Detection Systems

Detecting drowsiness in drivers, especially multi-level drowsiness, is a difficult problem that is often approached using neurophysiological signals as the basis for building a reliable system. In this context, electroencephalogram (EEG) signals are the most important source of data to achieve successful detection. In this paper, we first review EEG signal features used in the literature for a variety of tasks, then we focus on reviewing the applications of EEG features and deep learning approaches in driver drowsiness detection, and finally we discuss the open challenges and opportunities in improving driver drowsiness detection based on EEG. We show that the number of studies on driver drowsiness detection systems has increased in recent years and that future systems need to consider the wide variety of EEG signal features and deep learning approaches to increase the accuracy of detection.

A generalized permutation entropy for noisy dynamics and random processes

Permutation entropy measures the complexity of a deterministic time series via a data symbolic quantization consisting of rank vectors called ordinal patterns or simply permutations. Reasons for the increasing popularity of this entropy in time series analysis include that (i) it converges to the Kolmogorov-Sinai entropy of the underlying dynamics in the limit of ever longer permutations and (ii) its computation dispenses with generating and ad hoc partitions. However, permutation entropy diverges when the number of allowed permutations grows super-exponentially with their length, as happens when time series are output by dynamical systems with observational or dynamical noise or purely random processes. In this paper, we propose a generalized permutation entropy, belonging to the class of group entropies, that is finite in that situation, which is actually the one found in practice. The theoretical results are illustrated numerically by random processes with short- and long-term dependencies, as well as by noisy deterministic signals.

Complexity of heart rate variability during moral judgement of actions and omissions

Recent research strongly supports the idea that cardiac activity is involved in the organisation of behaviour, including social behaviour and social cognition. The aim of this work was to explore the complexity of heart rate variability, as measured by permutation entropy, while individuals were making moral judgements about harmful actions and omissions. Participants (N = 58, 50% women, age 21-52 years old) were presented with a set of moral dilemmas describing situations when sacrificing one person resulted in saving five other people. In line with previous studies, our participants consistently judged harmful actions as less permissible than equivalently harmful omissions (phenomenon known as the "omission bias"). Importantly, the response times were significantly longer and permutation entropy of the heart rate was higher when participants were evaluating harmful omissions, as compared to harmful actions. These results may be viewed as a psychophysiological manifestation of differences in causal attribution between actions and omissions. We discuss the obtained results from the positions of the system-evolutionary theory and propose that heart rate variability reflects complexity of the dynamics of neurovisceral activity within the organism-environment interactions, including their social aspects. This complexity can be described in terms of entropy and our work demonstrates the potential of permutation entropy as a tool of analyzing heart rate variability in relation to current behaviour and observed cognitive processes.

Mapping images into ordinal networks

An increasing abstraction has marked some recent investigations in network science. Examples include the development of algorithms that map time series data into networks whose vertices and edges can have different interpretations, beyond the classical idea of parts and interactions of a complex system. These approaches have proven useful for dealing with the growing complexity and volume of diverse data sets. However, the use of such algorithms is mostly limited to one-dimensional data, and there has been little effort towards extending these methods to higher-dimensional data such as images. Here we propose a generalization for the ordinal network algorithm for mapping images into networks. We investigate the emergence of connectivity constraints inherited from the symbolization process used for defining the network nodes and links, which in turn allows us to derive the exact structure of ordinal networks obtained from random images. We illustrate the use of this new algorithm in a series of applications involving randomization of periodic ornaments, images generated by two-dimensional fractional Brownian motion and the Ising model, and a data set of natural textures. These examples show that measures obtained from ordinal networks (such as average shortest path and global node entropy) extract important image properties related to roughness and symmetry, are robust against noise, and can achieve higher accuracy than traditional texture descriptors extracted from gray-level co-occurrence matrices in simple image classification tasks.

Classification of Actigraphy Records from Bipolar Disorder Patients Using Slope Entropy: A Feasibility Study

Bipolar Disorder (BD) is an illness with high prevalence and a huge social and economic impact. It is recurrent, with a long-term evolution in most cases. Early treatment and continuous monitoring have proven to be very effective in mitigating the causes and consequences of BD. However, no tools are currently available for a massive and semi-automatic BD patient monitoring and control. Taking advantage of recent technological developments in the field of wearables, this paper studies the feasibility of a BD episodes classification analysis while using entropy measures, an approach successfully applied in a myriad of other physiological frameworks. This is a very difficult task, since actigraphy records are highly non-stationary and corrupted with artifacts (no activity). The method devised uses a preprocessing stage to extract epochs of activity, and then applies a quantification measure, Slope Entropy, recently proposed, which outperforms the most common entropy measures used in biomedical time series. The results confirm the feasibility of the approach proposed, since the three states that are involved in BD, depression, mania, and remission, can be significantly distinguished.

Fever Time Series Analysis Using Slope Entropy. Application to Early Unobtrusive Differential Diagnosis

Fever is a readily measurable physiological response that has been used in medicine for centuries. However, the information provided has been greatly limited by a plain thresholding approach, overlooking the additional information provided by temporal variations and temperature values below such threshold that are also representative of the subject status. In this paper, we propose to utilize continuous body temperature time series of patients that developed a fever, in order to apply a method capable of diagnosing the specific underlying fever cause only by means of a pattern relative frequency analysis. This analysis was based on a recently proposed measure, Slope Entropy, applied to a variety of records coming from dengue and malaria patients, among other fever diseases. After an input parameter customization, a classification analysis of malaria and dengue records took place, quantified by the Matthews Correlation Coefficient. This classification yielded a high accuracy, with more than 90% of the records correctly labelled in some cases, demonstrating the feasibility of the approach proposed. This approach, after further studies, or combined with more measures such as Sample Entropy, is certainly very promising in becoming an early diagnosis tool based solely on body temperature temporal patterns, which is of great interest in the current Covid-19 pandemic scenario.

Entropy and the Brain: An Overview

Entropy is a powerful tool for quantification of the brain function and its information processing capacity. This is evident in its broad domain of applications that range from functional interactivity between the brain regions to quantification of the state of consciousness. A number of previous reviews summarized the use of entropic measures in neuroscience. However, these studies either focused on the overall use of nonlinear analytical methodologies for quantification of the brain activity or their contents pertained to a particular area of neuroscientific research. The present study aims at complementing these previous reviews in two ways. First, by covering the literature that specifically makes use of entropy for studying the brain function. Second, by highlighting the three fields of research in which the use of entropy has yielded highly promising results: the (altered) state of consciousness, the ageing brain, and the quantification of the brain networks' information processing. In so doing, the present overview identifies that the use of entropic measures for the study of consciousness and its (altered) states led the field to substantially advance the previous findings. Moreover, it realizes that the use of these measures for the study of the ageing brain resulted in significant insights on various ways that the process of ageing may affect the dynamics and information processing capacity of the brain. It further reveals that their utilization for analysis of the brain regional interactivity formed a bridge between the previous two research areas, thereby providing further evidence in support of their results. It concludes by highlighting some potential considerations that may help future research to refine the use of entropic measures for the study of brain complexity and its function. The present study helps realize that (despite their seemingly differing lines of inquiry) the study of consciousness, the ageing brain, and the brain networks' information processing are highly interrelated. Specifically, it identifies that the complexity, as quantified by entropy, is a fundamental property of conscious experience, which also plays a vital role in the brain's capacity for adaptation and therefore whose loss by ageing constitutes a basis for diseases and disorders. Interestingly, these two perspectives neatly come together through the association of entropy and the brain capacity for information processing.

Comparative Analysis of the Permutation and Multiscale Entropies for Quantification of the Brain Signal Variability in Naturalistic Scenarios

As alternative entropy estimators, multiscale entropy (MSE) and permutation entropy (PE) are utilized for quantification of the brain function and its signal variability. In this context, their applications are primarily focused on two specific domains: (1) the effect of brain pathology on its function (2) the study of altered states of consciousness. As a result, there is a paucity of research on applicability of these measures in more naturalistic scenarios. In addition, the utility of these measures for quantification of the brain function and with respect to its signal entropy is not well studied. These shortcomings limit the interpretability of the measures when used for quantification of the brain signal entropy. The present study addresses these limitations by comparing MSE and PE with entropy of human subjects' EEG recordings, who watched short movie clips with negative, neutral, and positive content. The contribution of the present study is threefold. First, it identifies a significant anti-correlation between MSE and entropy. In this regard, it also verifies that such an anti-correlation is stronger in the case of negative rather than positive or neutral affects. Second, it finds that MSE significantly differentiates between these three affective states. Third, it observes that the use of PE does not warrant such significant differences. These results highlight the level of association between brain's entropy in response to affective stimuli on the one hand and its quantification in terms of MSE and PE on the other hand. This, in turn, allows for more informed conclusions on the utility of MSE and PE for the study and analysis of the brain signal variability in naturalistic scenarios.

Using the Information Provided by Forbidden Ordinal Patterns in Permutation Entropy to Reinforce Time Series Discrimination Capabilities

Despite its widely tested and proven usefulness, there is still room for improvement in the basic permutation entropy (PE) algorithm, as several subsequent studies have demonstrated in recent years. Some of these new methods try to address the well-known PE weaknesses, such as its focus only on ordinal and not on amplitude information, and the possible detrimental impact of equal values found in subsequences. Other new methods address less specific weaknesses, such as the PE results' dependence on input parameter values, a common problem found in many entropy calculation methods. The lack of discriminating power among classes in some cases is also a generic problem when entropy measures are used for data series classification. This last problem is the one specifically addressed in the present study. Toward that purpose, the classification performance of the standard PE method was first assessed by conducting several time series classification tests over a varied and diverse set of data. Then, this performance was reassessed using a new Shannon Entropy normalisation scheme proposed in this paper: divide the relative frequencies in PE by the number of different ordinal patterns actually found in the time series, instead of by the theoretically expected number. According to the classification accuracy obtained, this last approach exhibited a higher class discriminating power. It was capable of finding significant differences in six out of seven experimental datasets-whereas the standard PE method only did in four-and it also had better classification accuracy. It can be concluded that using the additional information provided by the number of forbidden/found patterns, it is possible to achieve a higher discriminating power than using the classical PE normalisation method. The resulting algorithm is also very similar to that of PE and very easy to implement.

Complexity Changes in the US and China's Stock Markets: Differences, Causes, and Wider Social Implications

How different are the emerging and the well-developed stock markets in terms of efficiency? To gain insights into this question, we compared an important emerging market, the Chinese stock market, and the largest and the most developed market, the US stock market. Specifically, we computed the Lempel–Ziv complexity (LZ) and the permutation entropy (PE) from two composite stock indices, the Shanghai stock exchange composite index (SSE) and the Dow Jones industrial average (DJIA), for both low-frequency (daily) and high-frequency (minute-to-minute)stock index data. We found that the US market is basically fully random and consistent with efficient market hypothesis (EMH), irrespective of whether low- or high-frequency stock index data are used. The Chinese market is also largely consistent with the EMH when low-frequency data are used. However, a completely different picture emerges when the high-frequency stock index data are used, irrespective of whether the LZ or PE is computed. In particular, the PE decreases substantially in two significant time windows, each encompassing a rapid market rise and then a few gigantic stock crashes. To gain further insights into the causes of the difference in the complexity changes in the two markets, we computed the Hurst parameter H from the high-frequency stock index data of the two markets and examined their temporal variations. We found that in stark contrast with the US market, whose H is always close to 1/2, which indicates fully random behavior, for the Chinese market, H deviates from 1/2 significantly for time scales up to about 10 min within a day, and varies systemically similar to the PE for time scales from about 10 min to a day. This opens the door for large-scale collective behavior to occur in the Chinese market, including herding behavior and large-scale manipulation as a result of inside information.

Slope Entropy: A New Time Series Complexity Estimator Based on Both Symbolic Patterns and Amplitude Information

The development of new measures and algorithms to quantify the entropy or related concepts of a data series is a continuous effort that has brought many innovations in this regard in recent years. The ultimate goal is usually to find new methods with a higher discriminating power, more efficient, more robust to noise and artifacts, less dependent on parameters or configurations, or any other possibly desirable feature. Among all these methods, Permutation Entropy (PE) is a complexity estimator for a time series that stands out due to its many strengths, with very few weaknesses. One of these weaknesses is the PE’s disregarding of time series amplitude information. Some PE algorithm modifications have been proposed in order to introduce such information into the calculations. We propose in this paper a new method, Slope Entropy (SlopEn), that also addresses this flaw but in a different way, keeping the symbolic representation of subsequences using a novel encoding method based on the slope generated by two consecutive data samples. By means of a thorough and extensive set of comparative experiments with PE and Sample Entropy (SampEn), we demonstrate that SlopEn is a very promising method with clearly a better time series classification performance than those previous methods.

Permutation Entropy: Enhancing Discriminating Power by Using Relative Frequencies Vector of Ordinal Patterns Instead of Their Shannon Entropy

Many measures to quantify the nonlinear dynamics of a time series are based on estimating the probability of certain features from their relative frequencies. Once a normalised histogram of events is computed, a single result is usually derived. This process can be broadly viewed as a nonlinear IRn mapping into IR, where n is the number of bins in the histogram. However, this mapping might entail a loss of information that could be critical for time series classification purposes. In this respect, the present study assessed such impact using permutation entropy (PE) and a diverse set of time series. We first devised a method of generating synthetic sequences of ordinal patterns using hidden Markov models. This way, it was possible to control the histogram distribution and quantify its influence on classification results. Next, real body temperature records are also used to illustrate the same phenomenon. The experiments results confirmed the improved classification accuracy achieved using raw histogram data instead of the PE final values. Thus, this study can provide a very valuable guidance for the improvement of the discriminating capability not only of PE, but of many similar histogram-based measures.

Entropy Measures for Data Analysis: Theory, Algorithms and Applications

Entropies and entropy-like quantities are playing an increasing role in modern non-linear data analysis and beyond [...]

Algorithmics, Possibilities and Limits of Ordinal Pattern Based Entropies

The study of nonlinear and possibly chaotic time-dependent systems involves long-term data acquisition or high sample rates. The resulting big data is valuable in order to provide useful insights into long-term dynamics. However, efficient and robust algorithms are required that can analyze long time series without decomposing the data into smaller series. Here symbolic-based analysis techniques that regard the dependence of data points are of some special interest. Such techniques are often prone to capacity or, on the contrary, to undersampling problems if the chosen parameters are too large. In this paper we present and apply algorithms of the relatively new ordinal symbolic approach. These algorithms use overlapping information and binary number representation, whilst being fast in the sense of algorithmic complexity, and allow, to the best of our knowledge, larger parameters than comparable methods currently used. We exploit the achieved large parameter range to investigate the limits of entropy measures based on ordinal symbolics. Moreover, we discuss data simulations from this viewpoint.